The little USB stick on your keychain and the memory in your iPod is fueling a revolution in the enterprise storage world.
It seems everyone is talking about flash memory, a type of solid-state storage that offers faster and more energy-efficient performance than rotating disk drives. The downside is that it’s about 20 times more expensive than high-performance Fibre Channel drives, but that’s where the popularity of USB sticks and the iPod comes in. Consumers are demanding flash memory and getting it — in digital cameras, the iPhone, the iPod Touch and even the MacBook Air laptop.
The consumer demand for flash and another major event — EMC’s entry into the enterprise flash market this year — are combining to drive prices down, making it feasible for enterprise use, experts say. (Compare storage products.)
Big businesses are already starting to use flash storage for I/O-intensive applications, such as Oracle databases, credit card processing systems and stock trading applications. Many observers expect solid-state flash drives to be commonplace in enterprises within a year or two.
Solid state is electrical, unlike rotating drives, which are mechanical and have moving parts that make them inherently slower. Mark Peters, an analyst with the Enterprise Strategy Group, compares a spinning hard drive to your hand hovering over a checkerboard. Just as the drive head needs to move over the right piece of data, your hand has to be in the right spot to grab a checker. With flash, however, there are no physical movements and “everything is always immediately available,” Peters says.
Direct- or network-attached?
The experience of early adopter Neovest, a financial services firm in Utah, illustrates the benefits of flash memory and one potentially vexing question customers and vendors must wrestle with.
A couple months ago, Neovest purchased a 160-gigabyte flash device for $4,800 from the start-up Fusion-io, which makes a PCIe flash storage card that’s inserted directly into servers.
“As someone who is responsible for processing data and disseminating it out to our clients, we’re always looking for ways to be able to handle that data with extremely low latency,” says Brandon Farmer, senior network engineer at Neovest.
EMC, however, contends that putting flash directly in the servers is unnecessarily restrictive.
“We put it in the network and make it accessible to multiple hosts,” says Bob Wambach, senior director of Symmetrix marketing for EMC. “Anything you put into a server is basically locked to that physical server. It becomes less flexible, less dynamic and adaptable.”
Farmer acknowledges that the lack of network accessibility is limiting, saying “we could use some shared storage for databases.”
But in order to get maximum speeds, the flash storage must be near the server, says Michael Workman, president and CEO of storage vendor Pillar Data Systems.
“The best use of solid-state disk is direct-attached, not in a shared network array,” Workman says. “The reason for that is the latencies for solid-state disk are so low that putting it on a network to get at it actually makes the latency of the solid-state disk much worse than it could be.”
Pillar doesn’t sell flash today, but designed its Axiom storage system so that flash can be easily embedded when it becomes more cost-effective. Sun Microsystems has gone the direct-attached route, saying it plans to embed flash storage in nearly all of its servers by year-end.
Turning point for flash
Solid-state disks and flash drives are nothing new. Texas Memory Systems, for example, has been selling solid-state disks for three decades, notes Woody Hutsell, the vendor’s executive vice president. Texas Memory specialized in RAM-based solid state drives, which are used for the most latency-sensitive applications because it’s even faster than flash, he says.
The downside is RAM requires backup batteries, because data can be lost if the power goes out. That’s not a problem with flash, but Texas Memory Systems resisted the urge to sell a flash product until January of this year.
“Every other year, we’d look at flash and say ‘is it time?'” Hutsell recalls. “For the longest time, the answer was no.”
Only recently have manufacturers begun making flash chips with higher densities and at a lower price point than RAM, he says. Texas Memory made its RamSan product available with flash in January.
Hutsell credits the consumer electronics market and EMC’s decision to sell flash as the turning points.
“What’s happened over the last two years is that due to rapid adoption of flash in the consumer electronics market, the price of flash chips dropped dramatically,” Hutsell says. “The real effect of EMC entering the solid-state market is adding credibility to the message we’ve been delivering for 30 years, which is that there are applications that are slowed down by hard disk drives.”
EMC in January unveiled its plans to sell 73- and 146-gigabyte solid-state drives using flash memory, and made a bold prediction last month: Flash storage technology will be nearly as inexpensive as high-end disk drives within two years.
EMC, which uses flash chips built by manufacturer STEC, is counting on its own bulk buying power to drive prices down. Currently, flash chips are about 20 times more expensive than high-performance Fibre Channel drives on a per-gigabyte basis, EMC’s Wambach says. By 2010, flash will only be double or triple the cost of those high-end drives, making it cost-effective enough to hit the mainstream, he says.
Less waste
It’s no problem that flash costs more per-gigabyte, Fowler says, because customers typically use a high percentage of flash’s storage space, whereas they often waste typical hard drive storage.
An enterprise building a large database application needs lots of speed in the form of Input/Output operations Per Second (IOPS).
“What you end up doing is buying a lot of disks because every one gives you another 300 IOPS,” Fowler says. “But you’re not necessarily filling the disks. You’re taking a bunch of really fast 73-gigabyte disks but you may only put 20 gigabytes on each one. What you’re doing is spreading out the IO rate across all the disks in order to get to thousands of IOPS. With flash you can get to thousands of IOPS on one device.”
Moreover, flash “consumes one-fifth the power and is a hundred times faster [than rotating disk drives],” Fowler says. “The fact that it’s not the same dollars per gigabyte is perfectly OK.”
Neovest measured speed increases of tenfold and more over Serial Attached SCSI, while saving money on memory costs. “Most of the savings come from the reduction of system memory,” Farmer says. “Normally these servers we’re testing them on require 64 gigabytes of system memory. We can pretty much cut that in half, so we don’t have to buy as large of a platform.”
Solid-state technology, including flash memory, is certainly on the upswing, with IDC predicting 76% annual shipment growth through 2012 in a market that generated nearly $400 million in revenue in 2007. Fowler predicts that the majority of organizations building I/O-intensive applications will use some form of flash within a year.
But EMC notes that rotating disks aren’t going to fall by the wayside any time soon. Currently, Fibre Channel storage is used primarily for high-performance applications, and Serial Advanced Technology Attachment (SATA) drives — a less expensive alternative to Serial Attached SCSI — are used when high capacity is the main need, Wambach says. Most data is accessed only rarely, so high-cost flash drives are usually unnecessary, he says.
“There is a relatively small percentage of capacity that can be justified on flash drives today,” Wambach says. “What you’ll see in the future is that Fibre Channel capacity is going to rapidly diminish as a percentage of overall capacity. You’ll see SATA drives increase and you’ll see Flash drives increase.”
EMC has seen demand cut across a wide swath of use cases, including credit card transactions, financial markets and law enforcement agencies that need to do extremely fast searches. “What we found is that in almost any industry you look at, there are applications that are response time limited,” Wambach says.
But some customers may be a bit too ambitious, and should carefully consider which applications really need the high speeds of flash, Fowler says. “Some people are asking us to do Oracle databases 100% on flash,” he says. “I’m a little skeptical myself. Most peoples’ read and write activities don’t need to be full bore on every piece of data.”
