Game Design

Written by: Akram Taghavi-Burris | © Copyright 2024

Game Design, Development & Production

The entire process of producing a game from start to finished is synonymously called, game design, game development and game production. The synonymous use of these terms can lead to some confusion especially if one is new to the game industry. For that reason, many game designers and researches have tried to develop clearer definition of each.

Game designers and educators, Collen Macklin and John Sharp defined each of these terms, in their book, Games, Design and Play. According to Macklin and Sharp, game design is "the practice of conceiving of and creating the way a game works". Game development on the other hand is the actual making of the game, from the art, to code, levels to marketing materials. They concluded by stating that game production is the "process of producing the game indicated by the game's design".

In his book "A Playful Production Process by game designer and educator Richard Lemarchand, defines game design as "the process for coming up with ideas for a game". Whereas game production is the process of ensuring - the game gets built", both game design and game production are "two sides of the same coin". Lemarchand continues by defining game development as the process of using tools, making art, writing code, etc. to create a playable game.

Degrees in Design, Development and Production

To further complicate the confusion among these terms, academic programs will often use the term design for referring to a program with an art emphasis and development for programs with a programing emphasis. Additionally the term game production is sometimes associated with degree programs in business.

Making a Game, The Game

While there is no complete consensus on what game design, development and production really is and because these terms are used so synonymously, in this text we propose thinking of these terms in relation to playing a game.

Let us call this game Game Production and in this game there anywhere from one to hundred plus players who are working collaboratively as a team. The game is played over multiple rounds each round is called a stage.

Unlike traditional games, the game of "Game Production", begins by having the team define the goal of the game. The goal is called the Game Design, which outlines the overall objective for what will be created (developed), in order to win (complete) the "Game Production".

Game Development can be thought as the procedures, steps or mechanics for the game. Basically all the things the players do each round to make it to the end of the "Game Production" game.

While this analogy might be a little unconventional, it does help to provide some distinction between each of the terms. Besides what better way to think of the process of making a game as a game.

Stages in Game Productions

In the analogy above "Game Production" is described as being a game played in rounds. These rounds are referred to as Stages in Game Production.

While the goal of the "Game Production" is the "game design" each stage (round) in game production has its' own goal. These goals are called deliverables. You can think of deliverables as side-quests that must be completed before and on the way to complete the main goal.

Stages of Game Production

In the game industry and in other texts, these stages are sometimes called Game Development Stages or Game Design Stages. However, since we have established that design and developments are elements of production, using the term Stages of Game Production makes the most sense.

The stages and deliverables of the game development process include

Stages	Deliverables
Concept	Idea Generation: exploring ideas and or features.
Pre-Production	Physical prototype: testing feasibility
	Presentation: optional step, however, if funds or buy-in is needed before moving forward. a presentation is made to stakeholders or team members
	Digital Prototype: rough development, test build
Production	Design Documentation: all documentation on all aspects of the game or features are prepared
	Development: all members of the game production team begin the actual development of the game including assets, levels, systems, etc.
Quality Assurance	Alpha and Beta Builds: near complete builds of the game for testing, used to identify any bugs or issues with the game.
Deployment/Maintenance	Game Release: production build of the game to market, ongoing maintenance (i.e., updates, patches, and extensions).

Unified Software Development Process

Software development has a lot of overlap with digital game production, and as such also breaks the production into stages called the Unified Software Development Process or Unified Process. The stages in the Unified Process are:

• Inception: exploring ideas and or features
• Elaboration: developing prototypes and presentations
• Construction: prepare design documentation and begin development
• Transition: reevaluation, testing, and refinement

Agile Game Production

Agile project management is used throughout each stage of game production. Remember that agile project management is an iterative process. So, what does agile project management look like in a game production?

Let us say for example that you are working through one of the sprints during the pre-production stage of the game production for a an action adventure game in space. Someone on the team suggests adding an ion cannon as one of the weapons available to the player in the game. The process for implementing this would look like the following:

1. Plan - propose the new feature, plan for assessing the worth of this feature on the project.
2. Research / Design - review how other games have implemented a similar features; work out some concept art and flowcharts for how the canon might work.
3. Implementation - create a working prototype for testing
4. Testing - everyone on the team tests out the new feature and assesses its usefulness, novelty, and accessibility.
5. Review - If everyone likes the new weapon it goes into further development and is included in the game, however, if the consensus is that it is not working, then it is eliminated or put on the back burner to come back to consider after other key features have been developed.

Player-Centric Design

Game production process is highly collaborative and uses a Player-centric design approach.

Player-centric design builds on the idea of user-centered design in which all your design decisions are made to meet the needs of the user or player. However, the player-centric design model goes further by also ensuring a certain type of player experience in the design and development of the game.

When using the player-centric design approach the key question being asked throughout development, is "What is the player experiencing?" Thinking about how the player will experience a game level, an interaction a storyline, and whatever else in the game can dynamically shift how the game is developed.

Advocate for the Player

In a game production to title of Game Designer is given to the individual(s) who ensures that the team is working towards their overall goal, which is the Game Design itself. According to game designer and author Tracy Fullerton, the role of the Game Designer in a game production is to be an "advocate for the player".

What exactly is an "advocate for the player"? The game designer needs to ensure that everything implemented in the game meets the needs and expectations of its intended players. This advocacy for the player ensures that a player-centric design approach is applied in to the game production.

The problem that game designers sometimes face, is that they are not part of the target market for which the game they are designing is intended. For example, maybe you have come up with a game aimed at 2nd graders to help teach them math. Chances are there are not many 2nd graders on your game production team needing help with math, though that math one may be debatable.

The point here is that game designers need to ensure that the game will be fun to play for their target market and that no unnecessary elements are added to the game that will alienate, confuse, frustrate, or bore their players.

Think Like a Game Designer

Everyone working on a game production team should think like a game designer when it comes to being an advocate for the player.

Collaborative Process

Game Production can be a most intense, collaboration process experience. As such any successful team know that it is important that all members of the team contribute to the design.

Every member of the team should feel like they have a say in the direction of the project and that their thoughts, ideas, and suggestions will be heard.

Some best practices for collaboration include:

• Know everyone's name: from the game designer to the individual artists and programmers everyone should know everyone on the game design team.
• Brainstorming sessions: during the design phase all for anyone on the team to attend and participate in the initial brainstorming.
• Suggestion lists: create an open list of ideas that may or may not be implemented through the production process.
• Weekly lead meetings: allow team leads to head up the meeting giving a diverse perspective on the project.
• One-on-one creative talks: allocate time talking to each member of the team about the creative process to give them a sense of value
• Ask for help: team members should feel open to ask their colleagues for help or advice on any one of their assigned tasks.
• Share authorship: use the term "we" not "I" when talking about the project; remember that everyone is a valued part of the project

Idea Generation

The first stage of game production is the concept phase, in which the initial idea of the game is developed. But how do game designers come up with a game idea?

Play: Taboo

📝Critical Thinking | 🕒20 minutes | 📂Required Resources: Taboo cards

This activity is designed to help students practice creative thinking and generate ideas under constraints by encouraging them to find alternative ways to communicate concepts.

Discussion

After playing the game, have students discuss the following questions in their groups, using the back of the paper for notes:

• How did you come up with alternate ways to describe the word without using the taboo words?
• What strategies did you use when you felt stuck or couldn't think of a different way to communicate an idea?
• How does limiting certain words in Taboo encourage more creative solutions?
• What role did collaboration play in generating ideas during the game?
• How did working under constraints change the way you approached the challenge compared to if you had no restrictions?
• How might the process of finding new ways to express a concept help you think outside the box when coming up with game ideas?

Where do ideas come from

Ideas can come from anywhere and at any time, say 4:30 am in the morning. This is why you should always have a notebook by your bedside table to jot down those ideas!

Seriously though ideas can come from many different places, often from personal experiences, analyzing and critiquing others' ideas, and brainstorming.

How do we come up with good ideas? This is the question that former NASA Engineer and Youtuber Mark Rober aim to highlight in the following TED Talk from 2015.

How to come up with good ideas

Author and game designer Tracy Fullerton states that "Game designers think of the world in terms of challenges, structures and play. We can break this down by further defining these elements as

• Challenges: what is it that needs to be done, what is the goal
• Structures: what are the constraints or rules to get the challenge done
• Play: what is fun or playful about the challenge

Innovative Ideas

We've already established that Good ideas can come upon us at any time, but how do we know if the idea is innovative? Designing for innovation pushes game designers to look beyond the current types of games and ask themselves what is next?

In the following TED talk, bestselling author Steven Johnson aims to provide a complete, exciting, and encouraging story of how we generate the ideas that push our careers, our lives, society, and culture forward.

Where do good ideas come from

Generating Ideas are Creative Process

You might have noticed in the Steven Johnson's talk he states that good ideas often need time to "incubate". Incubation is one of the 5 stages of creativity. Therefore one can make the analogy that generating good ideas is a creative process.

Designing for Innovation

After generating a game idea, the game design team should consider how innovative is the idea. One method of determining this is to ask the following questions:

• What unique or new play mechanics will be implemented?
• What new or niche player type will this game appeal to?
• What new or emerging platforms will this game take advantage of?
• How can this game be integrated into our daily lives?
• What new business models will be used? Is your game free-to-play or subscription-based?
• What current game design problems do you hope to fix? Better story integration, and emotionally rich gameplay?
• What is the impact will the game have on individuals, society, or culture?

Methods of Idea Generation

While it might take years for good ideas to come to fruition, game designers often need to generate many ideas on the fly. There are several methods that can help with idea generation, the following are just a few.

Get Inspiration

Getting inspiration from games that have come before is a great way to generate new ideas. There are countless game designers who have made a huge impact on the industry and it is important as a game designer to be familiar with these individuals and their contribution to the industry.

Brainstorming

One of the best ways to generate ideas is to brainstorm. Brainstorming is the technique used to quickly generate ideas. Best practices for brainstorming include:

• Define the challenge: what are you generating ideas for? Be specific, do not just say to make the best game ever, instead how about making the best multiplayer first-person puzzle game ever?
• No Criticism: all ideas are welcomed during a brainstorming session regardless of how off the wall or non-related they may seem. One crazy idea might spark a new idea in someone else.
• Collaborative environment: make sure that everyone is in the brainstorming session participates, even at this early stage of development everyone on the team needs to take ownership of the project.
• Record ideas: generating ideas is not helpful if you forget them the next day. Make sure that all ideas are recorded in one format or another so that they can be referenced as the ideas are refined.
• Generate lots of ideas: the more ideas the better, again one idea might spark another
• Don’t go long: being creative takes effort, and going for too long can wear out your creative juices, keep brainstorming sessions under 60 minutes.

Research

Research means to immerse yourself in a subject to further develop your idea. For example, say your team has an idea for a gangster role-playing game (RPG) set in 1920s Chicago. The team should begin by researching 1920s Chicago, what events were taking place during that time period and region, and what challenges were people facing that could be incorporated into the game. This research will provide an added sense of realism and perhaps make the game somewhat educational.

Benefits of Research

Game Designer and author Richard Lemarchand, states that "every game can benefit from some research to give its foundations in the real world"

Research doesn't end with narrative design. Maybe your team has decided to develop a virtual reality (VR) game. In this scenario, the team should begin by researching the technology and answering questions like what games have already been developed using this technology, what are the best practices for designing in VR, what are some technical limitations, what are the best tools for development, and so forth.

Narrowing Down Ideas

After generating ideas the game design team needs to take time to reflect on those ideas. After a few days, the team will come back together and narrow down the ideas by asking questions such as:

• Is it technically feasible? Will we be able to make it?
• Is there a market? Who would buy this game? Will it be profitable?
• Can it be produced within budget? Will further funds need to be secured?
• Are you in love with it? How passionate are you about making this game?

Game Prototyping

Once a game idea has been settled on the next phase of the concept stage is to develop a prototype. There are two types of prototypes, physical and digital.

A physical prototype is a working model of the game to test the feasibility and fun factor. However, if the final output of the game is to be digital a digital prototype will be required to fully test the mechanics of the game idea.

Building Prototypes

Physical prototypes are sometimes called paper prototypes because these earliest prototypes are made not as digital games, but with pen and paper.

You might be thinking how in the world can one create a physical prototype for a complex video game idea? The goal of a physical prototype is to provide only a rough idea of how the game or parts of the game will function and serves as a blueprint for game development.

info

Testing game ideas as physical prototypes help to identify if the game is interesting, fun, and feasible, without spending too much time and money in development, just to find out the game idea does not work.

Game Mechanics, Gameplay, and Core Mechanics

When building physical prototypes, it will not be possible to test the fluidity of running, aiming, and shooting in a 3d environment. However, physical prototypes can help game designers identify tactical and strategic issues as well as issues with the game balance.

Creating physical prototypes also provides game designers to start laying the foundation of the game mechanics (rules and procedures), gameplay (choices, challenges, and consequences), and core mechanics (repeating player actions).

Play: Tank Turn Tactics

📝Critical Thinking | 🕒20 minutes | 📂Required Resources: Taboo cards

This activity shows how early prototyping can identify and address problems before full development.

Discussion

After playing the game, have students discuss the following questions in their groups, using the back of the paper for notes:

• What elements of "Tank Turn Tactics" do you feel need improvement to enhance the gameplay experience?
• Does the game feel balanced in terms of mechanics and player interaction? Why or why not?
• Would you say the game is enjoyable? What factors contribute to or detract from its fun?
• How would you assess the complexity level of the game? Is it appropriate for the target audience?
• What changes would you recommend to address any issues you’ve identified in the prototype?

Prototyping Steps

Physical prototype development should not be ignored and should be implemented at the earliest stages of the concept stage of game production.

The process of developing a physical prototype can be broken down into steps using a similar iterative process to the scientific method and project management.

Step 1: Lay the Foundation

• Build a representation of the key gameplay (challenges, choices, and consequences)
• Identify the basic game objects, such as setting, units, resources, etc.
• Establish the game mechanics (rules and procedures)
• Develop the core mechanics (repeating player actions) of the game

info

Units refer to the players' characters. In physical prototypes, these units can be represented by any number of items, from paper cutouts to peg people. Regardless of what item is used, units in physical prototypes should fit within a single grid cell of the playing field, have an obvious forward direction and be distinguishable from other units currently in play.

Step 2: Build the framework

• Prioritize what is the most important in the game
• Decide which rules are essential
• Keep a distinction between features and rules

Features vs. Rules

Features are attributes that make a game richer, while rules are a modification to the game mechanics that change how the game functions

Step 3: Add Finishings

• Define all necessary rules for the game
• Test validity of each rule
• Add and remove rules as needed

tip

Not every rule is critical to a game. Avoid adding rules that are non-essential or overly complex.

Step 4: Refinement

• Play through the prototype. Is your game compelling? What makes it so or not so?
• Assess and refine the prototype as needed
• Evaluate other features and ideas for the game, and implement if deemed relevant

Productive Prototyping

Game prototypes both physical and digital provide game designers with beneficial information about their game ideas. However, the information gained from a prototype depends on how useful the prototype is.

In his book, The Art of Game Design: A Book of Lenses, author Jesse Schell provides several tips for product prototyping. Regardless of whether you are creating a physical prototype or a digital prototype the following tips should be applied.

1. Answer a Question: The prototype should aim to answer a specific question of the game idea, such as: Is it fun? Is it technologically possible? Does the premise make sense?
2. Build Physical Prototype: Physical prototypes provide the earliest and quickest way for game designers to begin testing their game ideas.
3. Forget Quality: Prototypes are just quick renditions of a game, testing only questions about the game at a time. Do not spend too much time fretting about the aesthetics or refining specific code. Focus on the basic functionality enough to test only a rough idea of the project.
4. Don't Get Attached: Seeing a game idea first come to life can sometimes cause game designers to get too attached to the prototype. Remember that this is not the final project, it is simply a starting point that will evolve through research and feedback throughout the development.
5. Prioritize Your Prototypes: Prototypes are usually only small portions of a game, focusing on specific features. With that said, game designers need to prioritize what features are the most in need of testing. Say for example that your game idea is an FPS with a puzzle element. The mechanics of an FPS is pretty straightforward, and the puzzle element, being a unique feature of the game, would be the priority when it comes to prototyping.
6. Parallelize Prototypes Productively: While it is important to prioritize what aspect of the game should be prototyped, it can also be helpful to have your game production team, work on multiple prototypes for the game at one time. One group for example can be testing for example puzzle feature while another perhaps the weapons balance.
7. Build a Toy First: Games are interactive challenges; in contrast, a toy can be interactive but provides no challenge. Despite the differences, most games are built around a toy. A ball for example is a toy but a basketball is a game. Game designers should start the prototyping process by building the toy of the game. As the prototype is tested, the challenges will begin to evolve.

Schell also lists three productive prototyping tips specifically applied to digital prototyping, these include:

8. Pick a "Fast Loop" Game Engine: Prototyping is a fast process to quickly test and revise ideas before starting the development process. Choosing the right tools for digital prototypes can make all the difference in the turnaround time for a prototype. Certain types of programming languages and game engines provide libraries and resources that can make this process go much faster and provide more opportunities for feedback loops.

9. It Doesn't Have to Be Interactive: Perhaps your game has the option for the player to mount different weapons to their ship. This feature is something that may require custom scripts and models for it to be implemented. During the prototype stage, this feature can easily be visualized through linear animatics or storyboards, allowing game designers to focus the prototype on the prioritized features, while still providing reference to other game features for evaluation.

10. Seize Opportunities for More Loops: The prototyping phase of game production has very short turnaround times. However, game designers should never waste an opportunity to iterate upon a prototype if the opportunity should arise.

Rapid Prototypes

The process of making physical prototypes is usually a quick one, that focuses on what is called "rapid prototyping". The idea is that you are quickly trying to get your ideas together and test out the feasibility of the game without spending too much effort, before moving on to the next stage of development.

The following video provides supplemental information in regard to the benefits of cheap, rapid paper-prototyping early in the creative process.

Paper Prototyping

Pre-production Stage

Far too often game designers do not see the value in developing physical prototypes and skip this vital step in the pre-production stage of the game production process.

Physical prototypes however provide game designers with insightful information about the game idea and what direction to move forward with it. It also provides the earliest opportunity for game designers to playtest their game ideas with others.

One good example of how prototyping can be determining if a game should go into development, is a prototype developed by Halfbrick Studios, developers of the games Fruit Ninja and Jetpack Joyride. The following video by People Who Make Games, chronicles how after coming up with a relatively simple game idea, the game design team developed a simple paper prototype that went horribly wrong.

The Game Prototype that had to be Banned

Playtesting

Testing is an essential part of any agile development, especially game productions. Testing takes place during each sprint of agile development and of all the types of testing the playtest is by far the most important in player-centric design.

Player Experience Goals

A Playtest assesses if the game is meeting the player's experience goals. The player experience goals relate to the player's motivation in the game and what the player will do and feel throughout the game.

Playtests should help game designers answer certain questions about the game, such as:

• Is it playable? Does it function as intended or as the player perceives it should?
• Is it enjoyable? Would the players play it again? Does the player consider it fun?
• Is it challenging? Does it motivate players to complete? Is there the right amount of risk and reward?
• Is it engaging? Does it evoke a certain emotion and behavior in the player?
• Is it balanced? Are the outcomes perceived to be fair and consistent? Are there no unintended outcomes?
• Is it Accessible? Are all game elements understandable and complete?

caution

Remember the game designer is not always part of the target market for the game but regardless they must be an advocate for the player.

Target Players

The player-centric design is all about the player, with that said to properly playtest a game, game designers need to identify target players (market) for the game.

Market Segmentation

While it would be amazing to say that your game is intended for everyone, it is impossible to design for everyone. Instead, game designers need to identify a narrow representation of their ideal player defining each of the 4 types of market segmentation:

• Demographics: age, gender, income, location, education, ethnicity
• Psychographics: personality traits, values, attitudes, interests, lifestyles, motivations
• Behavioral: purchasing habits, spending habits, brand/platform interactions
• Geographic: country, climate, urban or rural

Each of the segmentations provides helpful information about your player and why they would or would not play your game. For example, if your game requires many in-game purchases to be able to level up, those from lower-income families may be unable to continue playing the game, long-term. If your game portrays certain morals or values in contrast to the player, they may find it offensive. If the game is developed as a PC-only game, console players may not even consider trying it. Certain countries ban certain depictions in media. For example: In Germany, all depictions and imagery of the swastika are illegal. Thus, a World War II game, for instance, may have to be modified or not sold in Germany.

Player Persona

One method for game designers to help identify the ideal player is to create a player persona. A persona in marketing is a fictional representation of an actual user (player).

During the early stages of the game production, a persona is prepared as a one-page description that identifies all of the market segmentation of the individual based on data gathered from actual people representing your target player.

Quantic Foundry a marketing research company focused on gamer motivations, developed the Gamer Motivation Model based on data from over 400,00 video gamers. This model represents 6 key pairs of motivations by game players and can be used as a starting point for creating a player persona.

Fig 1. Gamer Motivation Model from Quantic Foundry

Once completed the player persona will be used throughout the development to help the entire game production team make informed decisions when designing, building, and testing the game.

Activity: Player Persona

📝Critical Thinking | 🕒15 minutes | 📂Required Resources: Persona Sheets

This activity is designed to help students identify the ideal player for a game, by way of creating a player persona.

Discussion

As a group share your player persona. Do you feel that the player persona accurately fits the game? Why or why not?

Fig 2. Sample Player Persona for Telltale Game - The Walking Dead

Playtesters

Just as playtesting should take place during each sprint of the game production, there are different groups of playtesters; who should be playtesting.

1. Self-Testers: the game production team should playtest frequently and often. These playtest help identify any major red flags before they become a larger issue. However, these playtests may have biased results since those who playtesting are the same individuals on the game production team.
2. Friends & Colleagues: Once the self-testing has been deemed sufficient, friends and colleagues should be asked to playtest. This will provide feedback from an unbiased (or mostly unbiased) opinion. These individuals are not always your target market but can still provide vital feedback on issues before you share the game with your target market.
3. Target Market: To fully assess the potential success of your game playtest with your target market is essential. These strangers should provide you with 100% unbiased feedback.

Conducting Playtests

Playtesting should take place as early as possible in the game production, starting with the physical prototype. Keep in mind different groups of playtesters should be invited to playtest during different steps of physical prototype development.

Depending on the stage of the game production and the type of feedback data the game designer is seeking, different types of playtesting methods might be employed.

The methods of playtests can vary, therefore it is important to first identify the types of feedback data that game designers are seeking answers for.

Methods of Playtesting

There are many ways to conduct playtests each having its pros and cons. The most common methods of playtesting are through:

• One-on-one: observe and ask questions of each playtester as they play the game. This can provide insight into how the player plays the game but can also be skewed if the players are nervous or distracted by the observations and questions.
• Group Testing: observe and ask questions of a group of playtesters as they play through the game all at the same time and place. This can provide helpful observation data in a greater sum, but again players may be uncomfortable or distracted during the observation and questioning process.
• Feedback Forms: Have the playtesters play the game independently and submit feedback through a survey. Quantitative data on feedback forms relay on asking the right questions, while qualitative data depends on the articulation and willingness of the playtester.
• Interviews: Interview playtesters individually after they have played the game. Much like feedback forms, quantitative data depends on how the interview questions are asked and qualitative data depends on how verbal the playtesters are.
• Open Discussions: Have an open discussion with a group of playtesters after they have played the game. Open discussions can stimulate conversation among playtesters, in contrast to the interviews. However, there may be playtesters unwilling to share their opinions amongst a large group.
• Metrics: Collect in-game data about the player's gameplay behaviors. Metrics can collect a great amount of data on how the player unconsciously plays the game. However, this method of playtesting can only be implemented once a digital prototype is created.

Playtest Tools

A play matrix is a tool used to categorize and analyze games based on their core mechanics and player experiences.

• Skill vs. Chance: This axis measures whether a game's outcome relies more on player skill and strategy or on random elements and luck.
• Mental Calculation vs. Physical Dexterity: This axis assesses whether the game's demands are more about cognitive skills and strategic planning or about physical actions and coordination.

Fig 3. Sample Play Matrix

Playtest Sessions

Regardless of which method of playtesting is being utilized the actual playtesting sessions should consist of the following:

1. Introduction: Provide a brief overview of the playtesting process
2. Warm-Up: ask the playtesters questions about their current gaming background and preferences.
3. Play Session: have the playtesters play the game. The length of the play session depends on the game, but usually between 15-20 minutes and no more than an hour at a time.
4. Game Experience: obtain the data about how the playtesters perceived the game experience.
5. Wrap-up: always thank playtesters and show that you value their input.

There are many ways to conduct playtests each having its pros and cons. The following video from Brackeys provides another perspective of how and when to playtest.

How to Playtest

Playtest Data

As mentioned above the methods of playtests can vary, therefore it is important to first identify the types of feedback data that game designers are seeking answers for.

Here we identify the playtest criteria types, which aim to answer specific questions.

1. Concept Testing: Is it playable?
2. Scattershot Testing: Is it enjoyable?
3. Experience Testing: Is it engaging? Is it challenging?
4. Stress Testing: Is it balanced?
5. Accessibility Testing: Is it accessible, does it make sense and complete?

These different types of playtests are conducted during different stages of the game production and can be administered separately or as a whole playtest, depending on the needs, timeline, and budget of the game production.

Quantitative vs Qualitative Data

Quantitative data provide numerical data that can easily be measured. Qualitative data on the other hand represents opinions or feelings, which can not easily be represented numerically.

Both quantitative and qualitative data are equally important in playtesting. Quantitative data can help identify overall issues with a game and qualitative data can provide insight as to why players had certain issues with the game.

Collecting Playtest Data

Playtest data is collected a different times during the playtest session, which includes:

• In-Game: this data is recorded while the testers are playing the game. This can include observations notes taken of the playtesters or metrics recorded about their gameplay.
• Post-Game: this data is recorded immediately following the play session. These questions are designed to help elicit opinions about the key aspects of a game system.
• Reflection: this data is recorded after the playtester, and formulated by the game designer to quickly reflect on what revisions should be made to the game.

Taking Notes

Note-taking during playtests is very important. Game designers should always take notes during the playtest session, these notes can help provide some context to the playtest data.

Formulating Playtest Questions

To collect playtest data, game designers need to be asking the right questions. However, it is not as simple as just asking players if the game is playable? This type of question is open-ended. To what extent is the game playable? Was the player able to complete or achieve the goal. And for that matter are we even sure the player understood the goal?

Game designers should break each of their player experience goals, into smaller more direct, and defined questions. Breaking these questions into sections based on game mechanics, gameplay, and core mechanics. Questions should also vary depending on the playtest criteria types.

Analyzing the Data

Once a playtest is conducted it is important to review and analyze the data. Every game production team will have their own method for evaluating the playtest data. The most prominent elements which the playtest data should identify are:

• Which elements of the game are most praised?
• Which elements of the game are most disliked or confusing?
• What can you do to highlight or improve these elements?

Each time a playtest is conducted more data will help to illustrate a better picture of the direction the game is taking and whether or not the game is getting closer to or achieving the intended player experience goals.

Evil Data

When analyzing the data it is important to be aware of evil data. According to indie developer Adriaan de Jongh defines evil data as playtest results that are distracting, unclear or misleading. During his 2017 Game Developer Conference talk, Jongh explains how to avoid evil data during a playtest session.

Playtesting: Avoiding Evil Data

Survivorship Bias

During World War II, the American military sought to reduce the losses of their bombers in battle. They initiated collaboration with the Statistical Research Group (SRG), providing extensive data on the locations of damage on the aircraft.

The initially expected solution was to reinforce the areas with the most damage. However, SRG researcher Abraham Wald reached a different conclusion. He found that the areas requiring reinforcement were not the ones with the most damage, but rather the ones with the least damage. Wald's analysis focused on aircraft for which data was available—those that safely returned to base. These surviving aircraft indicated that damage in certain areas was survivable, while those that didn't make it back likely sustained damage in critical areas. Therefore, he recommended reinforcing the areas that had been least hit, such as the cockpit and around the engines.

This logical error, concentrating on what survives a specific selection process while overlooking what doesn't, is known as survivorship bias. Survivorship bias can also be seen as a type of evil data.

Refinement

Once the playtest data has been reviewed it is time to evaluate and refine the game based on the feedback. During the refinement stage, the game production team should identify which elements, features, and functions must be refined, eliminated, or let be.

The rule of thumb is to work based on the majority of the data. For example, if the data indicates that all or most playtesters voiced a complaint about the jumping feature in the game, then that is a feature that must be examined and revised. However, if there was a minor few who shared a complaint about the organization of the inventory screen, that might be an element to let be for now and perhaps re-evaluate later down the road.

Regardless of what direction the game production team takes with the playtest data, this data should be valued and help the game production team as they move closer towards their goal of developing a game that meets their player experience goals.