Too Much Beta Spoiled the Game

December 8, 2007 at 10:48 pm (Musings) (, , )

Gamasutra has an interesting article about some comments made by Richard Garriott at the recent Independant Game Conference in Austin. He gave a speech about making MMOs and afterwards was asked what, in his opinion, went wrong with his latest game Tabula Rasa. I think it’s fair to say that TR has undersold its potential by a long chalk. I haven’t played it myself so I’m not going to comment on whether it’s any good or not, however for a title like this with a name like Lord British behind it, most people (including NCSoft) would have expected it to make more of a splash than it did.

Rather bizarrely Richard Garriott believes that at least part of the reason for the underwhelming reception is due to beta burnout.

“I think the formal marketing did fine,” he replied. “They let people know the game existed, and was coming out. I actually think the biggest mistake was made not by the marketing department, but by the development team. We invited too many people into the beta when the game was still too broken.”

NowI’m going to have to be careful of what I say here and beat around the bush a little. As many of you probably know, I’m involved in a project which is still in a beta state and under a strict NDA. I can’t therefore draw parallels between his game and mine, nor am I interested in either sounding like a corporate shill or giving people who want to throw stones at WAR/Goa/EA Mythic any ammunition.

All of that out of the way, I can agree that beta burnout is a very real issue however (and it’s a big however) why are your core audience all in the beta to begin with? That says to me that either your potential fanbase was very small or that your beta was way overweight. That counts double if they’re all in the beta at an early stage where basic systems like ‘fun gameplay’ haven’t been implemented yet.

I’m not unsympathetic. I know that a lot of beta testers have unrealistic expectations of what a game in beta will be like. They are put off by and complain about things like pink placeholder textures all over the place, unfinished UIs, game systems that don’t yet work or that only work to a point. These people however are not your early fan base – these people are tourists. The true fans are those who acknowledge the half built walls around them and can still see the shape of the edifice that you’re building. If you have a game at all these people will hold the faith up to and after release, they will be the ones spreading the word for you and bringing in new members to your fledgeling congregation. If you don’t have these people then it might be time to admit that you don’t actually have a game.

Advertisements

7 Comments

  1. Jeff Freeman said,

    I’m not unsympathetic. I know that a lot of beta testers have unrealistic expectations of what a game in beta will be like.

    Yeh, but I think the actual problem is that a lot of beta testers have realistic expectations.

    If they judge a game is too broken in beta not to still be broken at launch, odds are for most of them their judgment is probably pretty good.

    I don’t mean that they can’t be pleasantly surprised to discover that they were wrong about it, from time to time… just that it doesn’t really matter (since the end result is the same) in the accuracy of their assessment.

    Interesting point about the true fans vs. tourists.

    I have about a half-formed thought regarding the division of loyalty that might arise when a player has become entrenched in a guild or clan, versus their first exposure to an MMO (where it doesn’t have to compete with their multi-game guild for their affection).

    But it’s only a half-formed thought at this point, so no more really than me wondering if that’s at all relevant.

  2. Count Nerfedalot said,

    The true fans are those who acknowledge the half built walls around them and can still see the shape of the edifice that you’re building. If you have a game at all these people will hold the faith up to and after release…

    Only true if you actually release the game completed. Pushing it out the door prematurely to make some arbitrary marketing/finance/business deadline with pretty much anything incomplete or non-functional will turn those faithful into bitter and betrayed-feeling anti-fans.

    Closed Beta is for finding new bugs that in-house testing didn’t catch. That works best if the old obvious bugs are fixed first, so you don’t get a zillion new reports of the old bugs. Open Beta is for load-testing systems that are otherwise feature complete and presumed ready for release (until proven otherwise by the Beta test).

    Using Open Beta as a marketing tool to give everyone a taste of the game when the game isn’t ready to be tasted is just insanity.

  3. antipwn said,

    Only true if you actually release the game completed. Pushing it out the door prematurely to make some arbitrary marketing/finance/business deadline with pretty much anything incomplete or non-functional will turn those faithful into bitter and betrayed-feeling anti-fans.

    Well yes to a point. The real true believers will still evangelise about your game even if it doesn’t actually install on any system not owned by NASA, In general though we’re talking about a game that launches with the standard amount of finishedness (yeah, I know..).

    During beta though – at least that part of beta where you have peopel who don’t work for you on your servers – the core will close their eyes to even quite gaping flaws as long as they believe that it will get fixed.

    Even if the timescale for that is post-release.

    My wife has been playing a lot of Hellgate: London and the forums for that game demonstrate this truth quite nicely.

  4. Ima said,

    I’m only one random game player, but the reason I’m not playing TR is because I did *not* beta test it, none of my friends were in a beta test, and only one of them is playing it. I know a beta test is not a demo, but the only way I’m going to spend $50 on a game is if you give me a chance to play it first, or *several* friends are playing it and have positive things to say (and I better think those friends like games similar to me). The last game I purchased that I didn’t demo or beta test first was … Black & White. Maybe I’m still reeling from that burn, or maybe it’s just a part of getting old, but I’m finding that unless I am reasonably certain I’m going to enjoy your game, I’m not going to drop $50 on it. I’m not the 13-20 demographic whose pocket gaming companies covet, but I do have a pocket that is available if the game developers give me a chance to fall for their product.

  5. Count Nerfedalot said,

    During beta though – at least that part of beta where you have peopel who don’t work for you on your servers – the core will close their eyes to even quite gaping flaws as long as they believe that it will get fixed.

    Even if the timescale for that is post-release.

    At that point you’ve sliced and diced the core population down to a few noisy knob-polishing fanbois who are generally either in it for the attention or in hopes of getting a gig in the industry. Not exactly the sort of reliable tester you are after, I would hope. The vast majority of testers will be screaming bloody murder at the first signs of the game girding for release while those gaping flaws are still unresolved.

    All of which brings us full circle to the question, is beta for testing or for marketing? The answer is ‘yes’, of course, but then you need to distinguish which role you are discussing.

    If beta is for testing then you only want people who will actually test and report their results, and only as many of them as you can realistically communicate with and process their results. Comments about the real true believers in this context are just silly.

    If beta is for marketing then opening up the floodgates is downright stupid if the game isn’t ready to be viewed and judged yet.

    And your Hellgate: London example is indeed appropriate. Barring some radical Christmas Miracle, that game is yet another example of one with a lot of promise missing the ring when it released too soon, in spite of massive beta tester opposition. (“real true believers” excepted)

    I think you may be placing too much value on the evangelistic effectiveness of rabid fanbois. I don’t think they have any access to the general consumer, and they certainly have no credibility with the knowlegeable discriminating consumer, so exactly how will they help you grow your market share?

    (apologies for rambling – I just woke up)

  6. skarbonefist said,

    The quote you took from the Garriot is exactley on the button. What concerns me is that an experienced developer could make such a simple mistake. I am sure that this mistake will not be repeated 😉

    The more you ramp up numbers the closer to finish the product must be. Everything that is poorly concieved or you look at it and know its not right, then the greater the burn will be. Especially if these are cores game mechanics.

    Alot of people don’t understand that betaing is not a free game for x months.

  7. Too much beta testing? - FreddysHouse said,

    […] stumbled on this yesterday, but I found a certain community managers ramblings on beta testing (see Too Much Beta Spoiled the Game), which mostly talks about the Gamasutra article on Tabula Rasa’s beta testing. He can’t comment on […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: