Beta testing, the practice of testing products with a small group of “real users” before a public release, has always been popular amongst video game developers. Arguably, at least for a time, beta testing an upcoming game was as much about building the hype than it was about testing.
With the meteoric rise of startups in recent years, the term “beta testing” has become more mainstream, and many tech companies now leverage a “soft launch” period as some type of beta test before an “official” release.
Beta testing generally implies testing a product with members of the public, and not “professional testers”. For this reason, the objective of the tests is mainly about gathering feedback more than to find, investigate and report bugs.
While beta versions are typically sent to volunteers who experience these games or products in the comfort of their own homes, it isn’t rare for playtests to be conducted in a controlled environment. Many developers film users as they experience a game or application for the first time. Testers are encouraged to speak up about their questions and feelings as they explore the product.
But playtests can also be conducted with a group of participants working from home. Distributing games and applications has become much easier with tools like TestFlight and Hockeyapp, and the key to leveraging beta or playtests is to carefully select the participants and to properly guide them to obtain the desired output.
Given our background in the video games industry, this article will be primarily oriented towards games. However a lot of it applies to apps and sites, so don’t hesitate to give us a call if you’re interested in running a beta testing project for your company.
Are playtests beta tests?
Playtests are beta tests in a sense, because they consist of testing upcoming game releases with a small group of users, presumably resembling the targeted end user in terms of personal/social characteristics.
The main difference is that beta testing is generally unscripted and therefore yields a greater variety of results that need to be analyzed and interpreted.
To really make the most of beta tests, a game development team needs to identify specific questions to which they want answers and prepare hypotheses as to what the results will be.
They must then compare their hypotheses with the data produced by beta testers, and tweak their game/levels accordingly.
For a lot of companies, playtests consist of watching users as they experiment a game. While it can be insightful to some extent, we have found that the sole act of looking at users play a game generally does not yield meaningful, actionable insights.
It’s the data that does, and the comparison with the desired output.
Playtests are more targeted and guided than beta tests. We gather users of a particular profile (in terms of gender, location and age group) and we assign them specific tasks to accomplish. Their opinion is obviously welcome and considered, but what we look for is specific data sets that reveal insights; failure rate, for example.
If a game is too hard or too easy, its users can churn pretty quickly. For this reason, proper balancing is key, and play tests are extremely useful to ensure that end users appreciate the experience that your game offers.
The benefits of playtests
Playtests can help improve the end users’ appreciation of the game right from then moment it is released.
Playtests can help your company release more balanced levels, therefore increasing conversion.
Playtests offer an external, unbiased point of view from users who could be your real players.
Using a managed playtesting service can free up your design team’s time, allowing them to focus on what they do best: designing, and analyzing historical results. Having your design team test is an inefficient use of their time and provides only a biased perspective. Be careful!
Managed beta testing / playtests services
Whether you are interested in beta tests with curated professional testers or targeted playtests with members of your target audience, we have been recruiting and evaluating the contributions of thousands of beta testers worldwide.
We love video game testing and we are able to think like business analysts and ask the right questions that will yield actionable insights for you team.
Our free test plan management tool Lean Testing can also be used to manage your beta testing efforts. You can recruit passionate users and provide them a platform to report bugs in an clear and organized fashion.
So whether you already have millions of active monthly users or you are about to release the next Angry Birds, contact us and we can talk about how using beta testing and playtests can help you retain and convert your users.
If you are a video game enthusiast and would like to take part in beta tests for upcoming games, sign up as a tester on Crowdsourced Testing and we’ll contact you when an opportunity fits your profile.