BETA TESTERS

I made a game once.

I designed it, programmed it and then tested it, fixed it, tested more, fixed, tested, fixed … e.t.c

I then unleashed it into the world, waiting for the money to flow in – what I got was a rude awakening.  The game had some bugs in it that were cringe worthy to say the least – and the public negative feedback destroyed my sales.

The big problem with publicly releasing a game is that those who play it don’t send in bug reports – no, they let rip anonymously on public forum boards; the stupid part of this is that you need to actively search for it.

The most negative post I have ever read on one of my games I found by accident – 4 hours before a major exam; I don’t care what anyone says, it does affect you and it did affect my mark on that exam.  The negative post pointed out some major bugs and I fixed those bugs real quick, the problem was that the post was nasty – and 3 months old!  I really wish that they had sent me something so I could have fixed them all earlier 🙁

It was a sports game and some of the bugs that got through all my testing were-

  • If a player was sent off when they had the ball then they took the ball of the ground with them; the game had to wait until that player came back on!
  • You could not take the ball out of play (naturally) but you could hit an opposition player that hard that you knocked them into the crowd (out of play) – you could then PASS the ball to a player in the crowd, they would then run to the touchdown zone and walk onto the legal area and score.
  • If you kicked the ball along the ground to nobody the AI didn’t know where the ball was, and simply stopped where they were.
  • At the end of the season AI teams were selling all their players for the money; and then promptly buying them back at market rates when they realised that they had no players!
  • During player auctions if you hit the F10 key then all bidding stopped and you got your player.  Star players could be bought for 10% of their price if you timed it right.
  •  

There were a lot more but that should give you an idea that these bugs were not found by me because I didn’t play the game this way.  If I had the ball I tried to dodge my opponent, I didn’t try to punch them and get sent off and I certainly never kicked the ball along the ground to nobody!

The problems with these bugs is that I never encountered them because I played the game I had designed it to be played – but players will play the game the way they wish to play it.

The point I am making is that no matter how much you test your own game you must get outside help, even a sibling (if you have one) will do.  You need a fresh set of eyes who will want to try it out and see what it can do.

First thing you should do with a tester is give them the game, and not say anything – don’t tell them how to play it as this is the job of your tutorial (don’t have one?  MAKE ONE!).  This will also help to work out what the tutorial covers and what needs improving.

By all means help the player out if the tutorial doesn’t cover something – but write this down so you can upgrade the tutorial.

So how do you get testers?

I’ve got bad news for those in the 21st century – they are not cheap, or free, anymore.  A professional games company recently spent 10,000 English pounds on advertising for testers – and they admit that it failed to get the enough testers, and the ones they did get weren’t up to the job.

Supply and Demand

If you don’t know the basics of the economic concept of ‘supply and demand’ then here is a real life example.

Back in the 1990’s there were testers, and then there were more testers.  There was no such thing as a professional game tester because there were so many of them, companies could get people to test their games for free – and lots of people!

In the late 1990’s there was a games company that was flooded with Beta tester applications – 250,000!  I think it was Diablo 2 but please don’t quote me on that as it was so long ago.

Multiplayer games used to start with 10,000 beta testers, and then as they expanded they had no problems getting 50,000 testers in order to really stretch the server.

But that was then and this is now; because of advancing technologies amateur developers can have a game up and running in 10-15 hours.  My sports game was written in C++ and took me well over 1,000 hours to ‘complete’ – that includes design, coding and graphics.

What this means is that there are around 100 new games released each day around the world for consoles, PC’s, mobiles e.t.c  and some of these games are bigger than anything that has come before so it really needs lots of testers.

But there isn’t the testers around anymore; testers are now either building games themselves, or playing multiplayer games, and don’t have the inclination to test buggy games out anymore.

The demand for testers has now outstripped supply; and from personal experience I can honestly say by a large margin.

These days you must pay for testers with either money or time.

Out of frustration I advertised for testers on a freelance worker site; the cheapest quotes came in at US$25 to test a small game.  Some people wanted over US$100 even though I put in the description that it would take no more than an hour.

Your other option to get testers is time; you must test another persons game and then they will test yours.  So you spend an hour on their game and they spend an hour on yours.

While this is a good idea if you have the time there is one major drawback – a testers report that simply states ‘Yeah,it’s a good game’.

At least if I’m paying for testers I will give them a full description  of what I need tested and what feedback I need.

Design Documents for games

I have two games, ‘Test Cricket Manager’ and ‘Dangerous Danika’, among several that I have worked on.

‘Test cricket manager’ is a cricket management simulation that has was started in 2009 in the language of PHP; it is multiplayer only.  This game is 90% complete (or is it?) and the release date is unknown.

‘Dangerous Danika’ is a puzzle game that was started in July 2013 and was the first game I made in Gamemaker, it was released in August 2014.

So why the difference in the amount of time?  Is a multiplayer sports simulation that much harder to do?  Does Gamemaker really make that kind of difference?

The simple answer is that the Sports game has no Design Document while Danika does.

So why can’t I do a design document the cricket game?  Simply because I am too far along to sit back down and draw everything out; every time I code the game it grows organically in my head but the simple truth is that Design Document should not be organic – you need to kill it and write it out on a piece of paper so everyone can look at it (including yourself) and understand what the designer (even if you are a one person team) is trying to achieve.

The Danika game had a design document from day one and has been going ahead according to the milestones I had set.  I can only work on the game part time so I have factored that into the design.

There are other reasons for having a design document

  • Allowing others to read it and get a full understanding of what is going on (and what they need to do if they are a part of the team).
  • Get your ideas on paper and see any conflicts (design bugs) early.
  • Ability to express your ideas and come back to them a week later to see if you would play the game.  And yes, this has happened to me where I had a great idea, wrote the design document, read it again a week later and realised that it was a stupid idea.  The advantage here is that I could make an important decision (to pull the project) before wasting time on producing a demo.

What needs to be included in the design document?

You really need the basics of the game.  You need gameplay, how it all fits together, overall story line, what will be included in the game, required artwork, music, sound

What’s a ‘Technical design document’ and do I need it?

The exact specifications are in the ‘Technical design document’ which you can probably get by without – unless you have over ten people working on the game!

Design document will be 2 to 15 pages; technical document will be 30 to 1,000 pages.

A design documents will show that what artwork needs to be done (a girl, a dingo, a crocodile e.t.c) while the technical document will explain the size of each piece as well as the format, which folder it goes into e.t.c

Milestones

Milestones need to be finished and working before the next section of a game is attempted; each milestone should be set out in the design document and explain what can and cannot be done in the game.

Example milestones for the Danika game –

Milestone 1 – Have Danika walk around the map and all environmental aspect work (she can’t walk through walls for example).

Milestone 2 – Have opponents on the map and have correct interactions with them all (Danika dies if touched by an opponent unless she has a weapon; all opponents kill, or be killed, by other opponents according to design).

Milestone 3 – Put all other parts into the game; this includes high score, lives e.t.c 

Milestone 4 – Online highscores.

Milestone 5 – Release game on the PC.

Milestone 6 – Release game on Android.

There is no specific milestone for bug finding because this should be done before  you can safely say the milestone is completed; let me repeat this because it’s important – A milestone is only complete when it is BUG FREE; realistically that should be all bugs.

I have not followed my own rules only to kick myself.

A prime example was when I didn’t fully test the code for when Danika gets to the exit (because it was so simple to program I tested it exactly once).

I proceeded to next milestone and put opponents into the game; and when testing this I noticed that sometimes when Danika got to the exit the game would crash.   The only thing I could think of was that the exit code was wrong; it was only 6 lines of code and yet it gave me nightmares for over a week!

I changed each line of code separately and tested; I took each line out one at a time and tested it all again; but after each time the bug stayed.

Finally I could see a pattern yet it took another hour to work out what was causing it – if an opponent was dying from being killed by another opponent then the game would crash; I had been testing the wrong code all along.

If I had completed milestone 1 and fully tested it I would have instantly recognised that the exit code was fine so I should have concentrated on the opponent code first, this would have saved me 30 hours of valuable time (and when you have a family every hour is valuable).

Some big studios don’t use them so why should I?

To put it simply, they are the studios that run over time and budgets.

I’ve read enough post mortems on professional game companies to realise that the design documents were never rated as important. 

In fact one of the most common items of what went wrong was that the design document was not written down so everyone could refer to it.

Why do I need one?

So you know what the heck you are trying to achieve; if you have no goal then you can’t make a path to it.

You can drive anywhere; but if you want to drive to the nearest service station for petrol/gas/diesel then you need a plan to get there.  If you simply drive randomly in the hope of finding the station then you may run out of fuel before that, and that ruins everything (trust me, been there and done that!).

Is a design document written in stone?

No.  It can be changed, altered, thrown out at any time; it is only there as a guide.

Questions to ask yourself before you design a game

I once answered a question in the game design section of the Gamemaker (GM) forums regarding how to go about starting a large game project; my answer was that the person should ask themselves several questions before they do anything else.

My reply received 3 upvotes.  The person who asked the original question then took my answer and used it to write a whole topic based on that – and their post received 5 upvotes!

I did contact them about the fact that they had basically copied and pasted my answer, with a few minor edits, and asked for a bit of credit – to which they did and I got a thanks in their post.

For this article I will take my original post and expand it .

When starting a game the first thing you should do is ask yourself several, critical, questions – WHO,WHAT, WHEN, WHERE, WHY and HOW.

Now let’s look at those questions one at a time.

WHO – Who is this game aimed at?

You simply cannot make a game that appeals to all walks of life, so don’t even try.

I personally try to make games that my five year old daughter would enjoy playing, the reason for this is that it keeps her happy and if you have a child then you will know what I mean!

So I like simple puzzle games that have easy to use controls, easy to learn and presents a simple challenge that everyone can enjoy.

Therefore my target audience is people who like simple puzzle games; my main source of revenue will be players who enjoy a challenge and parents who wish to keep their child happy playing (and there is a big market there, trust me as I’ve bought several $2 games just to keep my child happy for the day.)

One of my long term projects is a Dungeons & Dragons style game that plays like the old game of ‘Wizards Crown’ (my game is called LOBAG, short for Land of Blood and Gold).

My target audience will be completely different from my puzzle games as it targets those who like strategy, character building, RPG games. This game won’t be playable by my kids as it is too complex, but the target audience for this are willing to pay big money for expansions – just look at WoW, Ultima online or Everquest!

The next question is to ask what are the graphics going to be like? Cartoon style graphics will allow you to aim at children while realistic blood splats will be an instant turn off for that same demographic.

WHAT – What is the game about?

If you can’t describe your game in one sentence then you have either made it too complex or you yourself don’t know exactly what it’s about.

Describing it can be a simple ‘A puzzle game where Danika must rescue her father’ all the way up to ‘a strategy city building game with a first player shooter component where you will need to control your armies’.

A basic outline of the game is important for you and to others, if you describe it to a friend and they have no idea what is going on then you need to either rethink the idea or reword it.  Actually you may think your friend is an idiot and ask someone else, but remember this, if a second friend has the same problem then it’s most probably YOU!

You also start needing to know what the basics of the game will be, ‘what setting is this? Future, medieval, outer space?’, ‘what game style are we going to use? Isometric, first person perspective, overhead?’, ‘Genre?’ e.t.c

You also need to take into account your target audience, if the game is for everyone then you can’t have a realistic war game (you can have a war game with cartoon graphics though!).

WHEN – When will all this happen?

If you are working on this yourself then the simple answer might be ‘When it’s finished’. But how many hours are you going to spend on this each week? I have a large family so I try to spend an hour a day but even that’s not possible some times.

The time you really need to ask yourself this is when you are working with others. What time frame are you looking at? This allows others to see if they can do it or not, if you want the game finished, and polished, in three months then you need full time workers – so people like me, who can only spend an hour a day, are out.

One of the biggest problems I hear with teams is that someone is ‘not committed enough’; are they?  If they are like me, as in they can only spend a certain amount of time on it each week, and they are trying their hardest then you are not all on the same page.  If a person works and spends all their spare time (even if it is only an hour a week) and produces fantastic artwork – they are going to be really offended at being called ‘non committed’.

If you are not paying people then you will not get any full time workers; students can sometimes work full time for several weeks if on holidays but even that can only last several weeks – they need to go back to school sometime.

WHERE – Where will the game be released?

This is important so you can set up revenue streams from the beginning. It doesn’t matter what your target platform will be but you need to know the target audience of that platform. For example Linux users are, usually, highly knowledgeable computer users (I can use the word ‘Nerd’ here because I use Linux as well :p ) – so a simple puzzle game isn’t going to go too well.

Once you have worked out which platform you are going to release your game on you can then work out which programming language to use; while game maker would allow us to cover most platforms there other languages out there that may be more efficient.

I used to program in PHP which was a browser based language which isn’t going to work in ios or Android (and very inefficient to use as a browser game on either of those) and that is the main reason I now use GM.

WHY – Why are you making this game?

Another serious question that you need to ask is WHY you are making this game.

To gain experience?

You may be trying to get a professional job in the industry and wish to create a game to show potential employers what you can do.

Or you might wish to create the best RPG game on the market but realise that this shouldn’t be your first project, so you start with a small project to learn how it all works and what your capabilities are.

To make money?

And why not? Making money allows you to do this as a full time profession; working for a games company is the same thing except you make the money for someone else!

Please note that people don’t care whether you make money or not but they they can get upset on HOW you try to make your money – if you charge for a game and still put advertisements in it then people will get upset!

For yourself?

If you want a game that allows you to fly a Dragon around the open air then you go for it; if there is no challenge then others might not enjoy it but if that’s the way you want to play then it’s all yours.

My game Lobag is made for my entertainment because of my love for the game Wizards Crown – but I can only play that so many times and I am yet to find a replacement!  If others play the game and enjoy it then that’s good but it’s mainly for me.

For someone else?

The flying dragon game I talked about just then is actually a game I have a design document for; but the game isn’t for me, it’s for my daughter.

She has gone on and on about how there is no game where you can simply fly a dragon around the skies, so the game will be made for her.  If others enjoy it then that’s good as well but I wont lose any sleep if no one else wants to play it, just as long as my daughter plays it and gives my wife and I five minutes of peace and quiet!!

This is not a full list; you may be making the game to learn a new programming language, for a school project e.t.c

A final word on WHY – there is no wrong answer as to why you wish to make a game.

I know of people who made the game Tetris in C++, they didn’t make it for any other reason than to learn C++ and they succeeded.  From there they can use their newly acquired experience to create bigger and better (hopefully profitable) projects.

HOW – How are you making this game? And HOW are you going to cover your costs?

If you are making the game in Game maker (GM) then you have the question answered on HOW you are going to make the game and that’s a huge step in the right direction.

But what if the project cannot be handled by GM?  There are a lot of choices out there and the final decision will be dependant on what you wish to produce.

Another aspect of HOW is the covering your costs.  Unless the game is for family and friends then there will be a cost involved; costs could be hosting fees, bandwidth fees and many others (many of these you have no idea about until you have been through the process).

Try to work out how you will be getting revenue from your game as you will need to put this into your design document.  An example would be a game where the game is free but the extra characters need to be paid for; if you make this decision at the start of designing then you can work with it, but if you finish the game and THEN decide to implement this idea it could cause problems – what if you made a character that was critical to the plot into a paid character?  It could make it too hard to finish and players would be spiteful.

After the questions

After you have gone through these questions (at least gone through them in your head) you need some simple drawings of potential screenshots – I can’t draw at all yet I have no problem doing very simply diagrams that others can understand.

Don’t be afraid to draw ideas on paper, Dangerous Danika started with one piece of paper where my daughter and I drew a game she would like to play 🙂

The next step in creating a game is creating a design document.  In the next issue I will tell you WHY you need this document and WHY you need to treat it as your bible! 🙂