I was linked to an article with the following tagline: “Doubts raised on the value of public betas.” It’s essentially a commentary on open betas not being useful to the development process. It’s interesting, because I thought we’d figured this whole thing out back in about 2004. I’m not a panel, but I can say fairly definitively that open betas are very useful, in roughly these statistically-irrelevant percentages:
- 70% Marketing: It builds the hype and acts as a game demo, driving players to buy (or not buy) the game based on how positive their experience is during open beta.
- 20% Stress Test: While closed beta provides a good stress test, open beta is more variable and has a larger sampling of different machines and connections.
- 10% Sanity Check: Any hot-button issues that did not come to light during closed beta will become apparent in open beta.
Closed beta, I still believe, is extremely useful as long as you are handpicking the majority of your users. For example, if you target specific guilds or enthusiastic fans, you’re going to get useful feedback and bug reports.