Google Inc. for the past three years has been allowing its employees to place bets on various questions like if a new Google office will open on time, or if a new product will ship on time.
The company, which had logged 80,000 bets through March, is aiming to use this method of gathering opinions from employees – a method often called crowdsourcing through the use of prediction markets technology – to better gauge their employees’ real opinions.
And Google has found that its employee bets are usually right!
Although Google is one of the most successful Internet companies and can afford this type of Web 2.0 experiment, a recent report from Forrester Research Inc. suggests other companies can benefit can from this technique.
There’s a lot of a value to be gained by putting in place prediction markets to more effectively tap employee opinions on topics ranging from if a store will open on time to picking specific features for a new product.
“One of the biggest struggles most companies have – not surprisingly – is predicting the future,” said Oliver Young, Forrester analyst and author of “Prediction Markets: Wisdom of the Crowd Comes to the Enterprise,” released last week.
“Doing that today is a very politically charged system.
Simple things like project updates are full of politics, full of meetings. One of the biggest values of prediction markets is you get a lot more people looking at these major questions … and hopefully get through a lot of that political pressure to really get truthful answers.”
The report found that in addition to Google, companies such as Best Buy, Corning, Hewlett-Packard and Qualcomm have begun using prediction markets technology to aid with business decision-making.
Prediction markets are speculative markets where employees operate as traders to earn points — and potential prizes and other recognition — for correctly predicting future events.
Qualcomm, the report found, uses prediction markets to gauge the performance of product ideas and features, while Corning used prediction markets to forecast the retail sales of LCD TVs, which are a major consumer of its glass products, the report noted.
Here are some common types of questions used in prediction markets and hypothetical examples of each from the report:
- Sales projections: How many Nintendo Wii systems will be sold in January 2008?
- Feature or product evaluation: Which type of minivan seat will be most popular among buyers?
- Project delivery and management: Will Boeing introduce the 787 Dreamliner airplane on time?
- Competitive events: Will Blockbuster exit the brick-and-mortar movie rental business?
- Market conditions: What will be the average cost for a metric ton of steel over the next three months?
Young noted that companies should choose questions with answers that can be easily measured and questions with short time frames. For example, asking employees to forecast 10 years out is not a good question, he noted.
Rather, consider asking employees to forecast sales for the current week or quarter, he suggested.
Companies should also provide incentives as part of prediction markets to encourage employees to participate, he noted. However, monetary incentives are often the least successful for this type of project, the research revealed.
Recognition by management for employee prediction success was found to be the best incentive. For example, a person who has been good at predicting how new products will be received can be invited into meetings where new product designs are discussed, Young said.
“That sort of behavior sends a real sign to employees that this is something that is important,” he noted.
The report noted that there are several vendors, including Consensus Point, Inkling, Gexid, NewsFutures and Spigit, that provide prediction market software to manage the trading and consulting services to help train employees. There are also several open-source options like Serotonin and Zocalo available, the report added.
The range of issues predictions can cover can be very wide.
In Google’s case, questions the company has asked employees over the years range from: Will Google open its Russian office on schedule? Will Gmail users use search more? How many Gmail users will the company accumulate by the end of 2007? Will Harriet Miers be confirmed as a Supreme Court Justice?
In that time, more than 1,500 employees have bet on the outcome of 280 such questions as part of the corporate predictions program.
The purpose of the effort, which lets employees make bets using fake “Google money,” is to determine employee opinions on questions like how much demand there will be for a future Google product or how the company will perform during a specific period of time, said Google project manager Bo Cowgill.
Employees with the most successful betting results are awarded T-shirts or checks, Cowgill said during the O’Reilly ETech 2008 conference here today.
Betting records are not shared among employees, though management has access to the bets of individual workers.
“The reason we did it was to try to get better information about what our employees thought,” Cowgill said.
“If you let people bet on things anonymously, they will tell you what they really believe. This is a conversation that is happening without politics. Nobody has any incentive to try to kiss up or fudge the numbers.”
Because Cowgill and the others running the market know who is betting and how – Google has mapped out the GPS coordinates of every employee’s desk worldwide – they are able to analyze the predictions of individual employees to find out what factors make a bettor successful.
Cowgill said that he and his team have found that the biggest indicator of how a person will bet and the success or failure of those bets is where the individual sits in an office.
Even people who sit together but don’t work together bet the same, he said. “We have evidence of persistence of relationships,” Cowgill said.
“There’s a persistence of relationship that is going on. I will continue to trade a little bit like [a former office mate] even though we don’t sit together anymore. Space matters – that is kind of the bottom line.”
Usually within one and a half months of moving into a new office with someone, an employee will start betting like his officemate, he added. But within even a week of moving in to a new office, a person “starts to show glimmers” of betting like the new officemate, the analysis found.
Cowgill noted the irony of this, considering that Google’s bread and butter is products that allow geographically dispersed people to communicate better.
The analysis of the results also found that Google employees are more likely to make the right bet than the wrong one, he added. In addition, the rank of a bettor within the company had no affect on whether a bet was won or lost, he said.
Cowgill did note that the study of betting patterns did turn up some biases among employees. For example, employees – especially new hires – were more likely to overprice bets on matters that would be good for Google, Cowgill said.
In addition, analysis showed that bets made following a rise in Google’s stock price tended to find bettors predicting good things for Google.
“All of a sudden they start betting on things that are good for Google in a way that is probably not entirely rational,” Cowgill noted. “It doesn’t exactly make sense to say, ‘Google stock has done well, therefore we are likely to get a bunch of users in the future or open an office on time.’ “
The company is now planning to start analyzing how a person’s knowledge of or work on a particular product affects his betting, Cowgill said.
For example, he added, the company is looking to find out whether employees who work on a certain product are more likely to make an objective — and correct – bet or are more likely to “drink the Kool-Aid” and have a biased view of any possible shortcomings.
“The idea is that we can extract information from [employees] to improve business decisions,” he added. “The whole point of the exercise is for people to be able to express their true beliefs.”