After writing this blog, I realized it was really long and because of that I'm breaking it up into two parts.
This is the second part of my blogs around game testing. The first blog, titled: "PLAYTESTING" IS NOT JUST PLAYING YOUR GAME, introduced how only playing your game is not sufficiently testing your game. Most forms of game testing that I've seen so far have just been Monkey Testing, and that more rigour needs to be added to the game testing process.
This blog will talk about how that rigour can be added to the process and I will aim to give a few different testing processes you can do outside of Monkey Testing. Keep in mind, this will not be a complete list and that you don't have to do all of this, but I do recommend you try some of it out.
I'm going to be using Genesis as my source of examples because that's the game I know the most about. But feel free to reach out to me and I can help you build out some examples for your game.
Where do we start?
Testing is used to verify and cover questions regarding three key components of a system: the past, the present, and the future.
The past of your system
When you add a new component to your game, are you verifying everything you've done to date interacts with it appropriately? We've all seen this in software, so tell me if this sounds familiar: Big Company pushed a new release of their Awesome Product and now features that were once working fine are broken! Pokemon Go was notorious for this (Sorry Niantic, I still love you!).
The present of your system
As you're adding new component to your game you should have a series of heuristics and equations that let you know that you're still working within the "balance" of the game. We often call these Acceptance Criteria, and this type of testing is Acceptance Testing ("Acceptance Testing is a level of software testing where a system is tested for acceptability." - softwaretestingfundamentals.com).
The future of your game
As you are adding your new component to your game, you need to be thinking about how you find what you don't know about your game. For competitive games, how do you find the "meta-strategy" of your game? For co-op games, what type of patterns do players use?
This is the part that most people spend a lot of their time on, which makes sense. I just request you don't neglect the other areas of your game too. Also, as I spoke of in my previous blog, you can add some rigour to this area to be more thorough in your testing.
As the old adage goes, "you don't know what you don't know", so it makes sense to spend most of your time here while making your game. That's also why I want to provide more ways to explore (*hint*) this area beyond the monkey testing.
Due to length, we're just going to be looking at testing the future of your game in this blog post. In the next one we will cover testing the present and the past of your game.
Testing the Future
The goal of all of these testing frameworks is to answer the question:
How do I find what I don't know?
It's pretty amazing how often I make something and someone else tells me something about it I didn't know. You might think this is crazy. But the phrase, "This game is my baby" is really true, and just like another human being, you don't know everything about them. Okay, I admit, that was a bit of a silly statement but it honestly feels really true.
Going back to that question, how do we find out what we don't know? We're in luck, scientists have given us the answer to this question: The Scientific Method! It really is the formula for discovery. There are a bunch of approaches to using the scientific method, and I'm sure you'll have some that I don't mention here. I'm just going to dig into my favourite - Exploratory Testing.
I really think I could do a full blog around this (or a vlog if that would be more interesting - let me know!).
First off, I recommend you do some reading into James Bach's work on Exploratory Testing. He is one of the best (software) testers in the world and is incredibly smart.
The whole approach is a system of Exploratory Testing is, "test design and test execution at the same time" (What is ET). Test design starts with a question or an inquiry, "what happens if I do this?" and from the test execution you record what happens and you come up with a new question, "because I did x and y occured what happens if I do z?" From this you just repeat.
What's the difference between exploratory Testing and Monkey Testing?
Intention. That's it. Monkey Testing, there's no intention. You're just doing things for the sake of doing things. In Exploratory Testing you're doing things with the intention of figuring out what happens next; and then you want you want to use that information to create a better follow up test.
Test: What if I use Idiris's Fire Ball  ability every turn?
Conclusion: I'm running out of Aura quickly.
Test: If I'm running out of Aura quickly, what if I balance that out by only running energy costing cards.
Conclusion: My opponent puts a lot of pressure from their summons.
Take Away - bring a note book
Testing is a practice skill, and you can only get better by reviewing your notes and refining your process. Keep on practicing, take notes of everything you do, review those notes to refine your process.
Other types of Future Testing
Very quickly, here are some other types of testing you can try.
User eXperience Testing
Finding end users and getting them to play your game. Try to do something different with each one of these tests, like:
- Different type of end users (designers, casual players, players who hate these types of games, players who exclusively play these type of games).
- Different scenarios (two groups using similar rule books except for one twist, a group that has the rule book and that's all they have to learn the game off of, players who are re-playing the game)
Just make sure you have intention of what you're trying to test (rulebook, combat, barrier to entry), have a few theories/assumptions that you want to validate and contradict, and take a lot of notes!
Here is a great blog if you want to read more: conversionxl.com/blog/user-experience-testing/
There are a few different ways of doing this. Just playing the game is one. Another is to give the players a really absurd goal. But yes, at the end of the day, Monkey Testing is a type of testing (as long as quality is going up, why not do this?) and it can also lead you down some interesting paths that you would never have gone down if you were too rigorous.
I can't get into all of these types of testing now, but I do recommend them.
- Heuristics Test Strategy Method (Link)
- Rapid Software Testing (Link)
- Exploratory Software Testing (Book)
I know these topics are about software testing, but the end goal is the same: In an possibly infinitely large system, how do we explore possible option and cover every edge case? I do recommend reading them and thinking about how you can apply this to game testing.
I don't think you should stop doing monkey testing, but I hope you now see there are many, many, many other types of testing you can do besides monkey testing.
In the next post we will look at how to test the Past and Present of your game using the frameworks of User Acceptance Testing, Sanity Testing, and Regression Testing.
Thank you for reading!
Please don't forget to like the post and leave a comment below. Let me know if there is anything you don't agree with or if there's anything new you learned!
Follow Genesis on Facebook (FB.com/GenesisBoC) or join our newsletter to find out when we post our next blog and all our future articles!