James Clifton

A man and his blog.

BETA TESTERS

I made a game once. I designed it, programmed it and then tested it, fixed it, tested more, fixed, tested, fixed … e.t.c I then unleashed it into the world, waiting for the money to flow in – what I got was a rude awakening. The game had some bugs in it that were cringe worthy to say the least – and the public negative feedback destroyed my sales. The big problem with publicly releasing a game is that those who play it don’t send in bug reports – no, they let rip anonymously on public forum boards; the stupid part of this is that you need to actively search for it. The most negative post I have ever read on one of my games I found by accident – 4 hours before a major exam; I don’t care what anyone says, it does affect you and it did affect my mark on that exam. The negative post pointed out some major bugs and I fixed those bugs real quick, the problem was that the post was nasty – and 3 months old! I really wish that they had sent me something so I could have fixed them all earlier ?? It was a sports game and some of the bugs that got through all my testing were- If a player was sent off when they had the ball then they took the ball of the ground with them; the game had to wait until that player came back on! You could not take the ball out of play (naturally) but you could hit an opposition player that hard that you knocked them into the crowd (out of play) – you could then PASS the ball to a player in the crowd, they would then run to the touchdown zone and walk onto the legal area and score. If you kicked the ball along the ground to nobody the AI didn’t know where the ball was, and simply stopped where they were. At the end of the season AI teams were selling all their players for the money; and then promptly buying them back at market rates when they realised that they had no players! During player auctions if you hit the F10 key then all bidding stopped and you got your player. Star players could be bought for 10% of their price if you timed it right. There were a lot more but that should give you an idea that these bugs were not found by me because I didn’t play the game this way. If I had the ball I tried to dodge my opponent, I didn’t try to punch them and get sent off and I certainly never kicked the ball along the ground to nobody! The problems with these bugs is that I never encountered them because I played the game I had designed it to be played – but players will play the game the way they wish to play it. The point I am making is that no matter how much you test your own game you must get outside help, even a sibling (if you have one) will do. You need a fresh set of eyes who will want to try it out and see what it can do. First thing you should do with a tester is give them the game, and not say anything – don’t tell them how to play it as this is the job of your tutorial (don’t have one? MAKE ONE!). This will also help to work out what the tutorial covers and what needs improving. By all means help the player out if the tutorial doesn’t cover something – but write this down so you can upgrade the tutorial. So how do you get testers? I’ve got bad news for those in the 21st century – they are not cheap, or free, anymore. A professional games company recently spent 10,000 English pounds on advertising for testers – and they admit that it failed to get the enough testers, and the ones they did get weren’t up to the job. Supply and Demand If you don’t know the basics of the economic concept of ‘supply and demand’ then here is a real life example. Back in the 1990’s there were testers, and then there were more testers. There was no such thing as a professional game tester because there were so many of them, companies could get people to test their games for free – and lots of people! In the late 1990’s there was a games company that was flooded with Beta tester applications – 250,000! I think it was Diablo 2 but please don’t quote me on that as it was so long ago. Multiplayer games used to start with 10,000 beta testers, and then as they expanded they had no problems getting 50,000 testers in order to really stretch the server. But that was then and this is now; because of advancing technologies amateur developers can have a game up and running in 10-15 hours. My sports game was written in C++ and took me well over 1,000 hours to ‘complete’ – that includes design, coding and graphics. What this means is that there are around 100 new games released each day around the world for consoles, PC’s, mobiles e.t.c and some of these games are bigger than anything that has come before so it really needs lots of testers. But there isn’t the testers around anymore; testers are now either building games themselves, or playing multiplayer games, and don’t have the inclination to test buggy games out anymore. The demand for testers has now outstripped supply; and from personal experience I can honestly say by a large margin. These days you must pay for testers with either money or time. Out of frustration I advertised for testers on a freelance worker site; the cheapest quotes came in at US$25 to test a small game. Some people wanted over US$100 even though I put in the description that it would take no more than an hour. Your other option to get testers is time; you must test another persons game and then they will test yours. So you spend an hour on their game and they spend an hour on yours. While this is a good idea if you have the time there is one major drawback – a testers report that simply states ‘Yeah,it’s a good game’. At least if I’m paying for testers I will give them a full description of what I need tested and what feedback I need.