Level .08 - Playtesting

Testing in Game Design & Development

Testing is one of the critical processes accompanying the design and development of computer games. Game designers and developers often confuse or simplify different testing processes, leading to unsatisfactory results, e.g. for students it is most often confusion of playtesting with bugfixing (quality assurance) or playtesting with usability testing (interface). So first, we will formally clarify the testing process.

Well-known literature states that four types of testing are crucial to the design and development of computer games:

• Focus Groups: They play a role, especially in the initial stage of game design, consisting of meetings with potential players providing feedback, likes, and dislikes on the upcoming game topic. Although Focus Groups can be helpful, they are unpopular. The reason is management's fear of "killing" their ideas. So, the literature recommends switching from obtaining one-sided feedback to discussions generating new ideas. Such Focus Groups will allow iterating the existing ideas more effectively. Moreover, it doesn't have to be only the initial stages of game design. They can also be helpful later, e.g., when designing game levels or planning the marketing of an upcoming game, which is especially crucial in the final stage of game development.
• Quality Assurance (QA) Testing: This is standard bugfixing. The goal is not to play and enjoy the game but to identify all the problems and make sure the game meets all the predetermined requirements. QA engineers look for potential bugs and report them. Since games are not a common type of software, a video, image, and detailed description can help understand steps, leading to the error situation.
• Usability Testing: One of the final aspects of a tested game is accessibility. Usability Testing focuses on whether the user interface (UI) is intuitive and easy to use. Players must understand and play the game without assistance. For this reason, testers best test the game during the first-time access. Ideally, User Experience (UX) experts perform the testing.
• Playtesting: Often referred to as the most important of the four testing types. It is a process that designers and playtesters carry out throughout the game design to determine how the players experience the game. The team of designers and developers proves they are making a good game, even if it scares them. Playtesting aims to guide the design and provide feedback to the designers on whether the game meets their goals and the players' expectations. The forms are different, more or less formal.

Research

Game User Research (GUR) intends to improve player experience. GUR is a community of UX experts, game developers, and researchers, where the primary research subject is playtesting. In general, GUR supports the adaptation of standard HCI evaluation techniques in the playtesting practice, focusing on "entertainment" instead of "productivity". From the point of view of software engineering methods, GUR mainly recommends the RITE method. The research applies other sophisticated methods such as PLAY, adapting various heuristic methods, or using artificial intelligence - AI Players. The goal is to predict player behavior and gaming experience.

As already mentioned, testing is critical in the iterative process of game design and development. It helps detect problems early (it is still possible to fix them). But what exactly is an iterative process?

Rapid Iterative Testing and Evaluation (RITE) is a software development method whose main philosophy is to identify a problem as soon as possible and eliminate it. During the last two decades, the RITE method has become widespread, especially for products changing rapidly e.g. web, cloud products, and games.

It is too late to test at the end of game development. Otherwise, the changes would be too lengthy and costly. The RITE method guarantees early identification and elimination of the problem. It is mainly adopted by larger game studios and recommended by the GUR community.

There are multiple testing types in game development. Playtesting is key to the RITE method, although other types of testing also find their place in individual stages of game design and development. From a practical point of view, it is best for playtesting if playtesters can play the game as soon as possible. Rapid prototyping (including paper prototypes) is advantageous if the game is still in the design stage.

Playtesting

Although we have already talked several times about the fact that testing is an essential part of the design and creation of computer games, we have not yet talked about how, when, and who should actually perform the testing. So that's what we'll talk about in the next few minutes.

As soon as you have a prototype, you have the space to test your ideas and get adequate feedback on your design. On the one hand, it can be quite frustrating (bad feedback), but it is necessary to realize that every single opinion about your result can improve the game - a new iteration will be created. So remember that every player who plays your game can make it better.

Don't be afraid of playtesting because someone criticizes your idea. That is the most valuable feedback you can get. What seemed funny to you, may not seem funny to real players. And also - the sooner you fix the detected/discovered error/problem, the cheaper it will be for you and your team.

With sufficient testing, it can avoid happening to you, e.g., your game does not have a specified goal, which can increase its difficulty very quickly. This is a common sign that the game has only been played and tested by players who understand it (i.e., its creators). But if they let someone external test the game, they would find out very quickly.

But the test can take place and often takes place in a way where two people take part in the testing:

1. Investigator - the person who manages the test
2. Playtester - a person who participates in the test and provides feedback

Testing is often done in a special room where the gameplay is recorded along with the playtester's expression and reactions for retrospective analysis.

Investigator Rules

Talk to your players as little as possible - anything you say to them can influence their judgment, and you won't get good data (feedback) in return. Don't tell them what to do in the game or what the game is about. Let it flow naturally, and watch them discover things on their own. What they experience, and how they look. What kind of "sucks" did they get into, and where do they spend most of their time. This is all relevant data for you.

Create a questionnaire - Good practice. It is possible to answer questions that are not burdened with prejudices, and some can also be entered into graphs and looked at strictly statistically. Of course, leave room for them to make their own comments, even in the case of direct evaluation questions such as: Rate the game on a scale from 1 (weak) to 10 (strong). And follow trends in the rating development during further development and playtesting.

Identify playtester quit location - Where does the player quit? When does he stop playing? When does he stop being motivated to continue and do something else? Here you have to realize that your game is not only competing with other games on the market but also with every book, movie, series, and comic... So be prepared for that.

Don't be sad that they don't like your ideas - Don't take their comments as an insult, but write each one down and return to it. Each one is important and can improve your game's quality. Even negative feedback is feedback.

Good Playtester

This semester, we have prepared an assignment for you where you will gain experience in the role of a playtester. But what should a good playtester be?

Think Aloud - The best way for an investigator to get feedback. He sees and hears what and how you experience. If you don't have an investigator at hand, try to talk out loud, remember as much as possible, and then document it :)

Reveal Your Prejudices - Everyone has prejudices and attitudes that they carry in life. These can also be reflected in your feedback if you rate a type of game that you don't like. So be honest. While playing the game, talk about what the game reminds you of (movies, books, comics, games, experiences, ...). This will also help the investigator get to know your background.

Do your own self-analysis - Instead of just saying, "It's powerful," say the reason or identify what is powerful. For example, "It's cool to be able to burn a hole in a wall to enter a building instead of a door. It adds to the dynamic of the game."

Separate elements - Talk about individual things when you give overall feedback. So instead of writing that it's great, comment (analyze) specifically on the graphics, music, story, mechanics, etc.

Finding Playtesters

Who should test?

• team members - a specific group because they know the domain very well, and for this reason, they are the worst testers because they know the game and test it during development.
• traditional playtesters - people who make a living and can find problems very quickly,
• first-impression testers - people who don't know it, and you will be interested in how they will react,
• other designers or developers - e.g., at gaming conferences, from other teams, ... at a hackathon ;)
• non-players,
• others - family members, friends? Friends don't have to be the best because they don't have to tell you the truth. That's why small children tend to say exactly what they think ;)

The more testers you have, the better the game can be.

Video