The word “beta” has become synonymous with “bug hunting” in the context of unfinished software, but vendors are embracing the idea that beta tests, both public and private, can improve products in the eyes of users before they’re even released.
Adobe Systems has warmed to this concept, said Dave Story, vice-president of engineering at the San Jose, Calif.-based software firm, and has moved its newest product, Lightroom, into a public beta program far sooner than it might have in previous few years. In fact, it’s the first time Adobe has offered a beta without asking users to register and qualify for the program.
“We’ve turned all our development processes upside down,” he said. The traditional paradigm of rushing a product to meet a release deadline, then putting a beta out to iron out the kinks is not always what users want to see from software providers, he said.
Lightroom is a workflow tool for digital photographers, both amateur and professional. The idea is to cut down on the amount of time they have to spend tweaking pictures on a PC, allowing them to concentrate on the photography itself. Adobe is looking for as much feedback as it can garner, said Story. The feature set will be finalized after the beta testers (and there are more than a quarter of a million of them, according to Adobe) have had a chance to weigh in.
“Software that’s too complicated is being seen as a real problem by users. Witness Vista. It’s just too difficult to get out the door. There’s nothing wrong with Microsoft. They’re really smart guys. It’s just that Vista is really, really, really complicated,” said Story.
Microsoft’s Vista operating system has experienced delays, but Mark Relph, vice-president of the developer and platform group at Microsoft Canada, said the company is very responsive to user feedback.
Some of Microsoft’s beta programs are massive, like Vista, for instance, while others are more low-key, but they all follow the same basic pattern. Groups of Microsoft users, partners and customers are invited to participate in software testing, either through downloads or in those cases through receiving the software directly from Microsoft reps. “Customers can put the product through its paces, see how it affects their environment. (We) help them through the hurdles that they have, but most importantly take the feedback. They might discover they like something one way rather than another,” said Relph.
That might not add up to major changes, but could result in some tweaking. For instance, Microsoft received a lot of feedback on the format of the menu bars for an upcoming release of Microsoft’s Office suite. “The final product does not look tremendously different but is a fine-tuned version of the first beta. I think it’s a great example of the feedback coming in,” said Relph.
Most software vendors may not have much choice but to expand their beta programs, said Strategic Counsel of Canada analyst Warren Shiau. The potential for security flaws is so great that it would be irresponsible for vendors to release a product before beta testers have taken a shot at it.
On top of that, user feedback has taken on a primacy now that minor application features and add-ons are what distinguish one software package from the next.
Basic functionality was all that was needed in the early days of desktop software. It was enough, for example, to develop a word processor and release it to the market. “But now everyone has something like that, so you have to make the differences in the actual usability,” said Shiau.
Stephen Ibaraki, national vice-president at the Canadian Information Processing Society, has been beta testing software for almost 30 years. He said he’s seen a marked improvement in the way software vendors accept and act on feedback from the user community.
“I would say that programs have improved 200 per cent.
I find that companies do listen. I find today they’re very targeted in terms of who I am, what my skill set is, the market segment I represent,” he said. “I find that companies, when they’re setting up their beta programs, are soliciting people from very specific segments so they can get a very detailed feedback.”
The interest level of beta software in users is reflected in the number of wikis and blogs that are dedicated to the topic, said Ibaraki.
Vendors are backing this movement and providing outlets of their own, inviting users to participate openly in forums, then trawling them for feedback and ideas. According to Story, 25,000 users have posted comments on labs.adobe.com site about the beta edition of Adobe’s Lightroom.
In the same vein, Microsoft offers a connect.microsoft.com forum which allows users and Microsoft developers to mingle and share information, hopefully leading to eventual product improvements, said Relph. “Some of it is often quite pointed feedback, but that’s good. You don’t always want to hear, ‘Hey, we really like it.’ We do want to hear, ‘Hey, we think this could be better.’ That helps us evolve,” he said.
