When CORE Feature Animation set out to make an animated feature, it needed a computer with the power to move large amounts of data. A traditional 1U rack-mounted server configuration would have required 500 rack units of space to meet its needs.
So the motion picture company looked into setting up a blade server farm instead, proving that supercomputers are no longer the sole domain of governments, universities and research institutes.
The small Toronto-based company, which is a subsidiary of CORE Digital Pictures, started out by examining what its requirements would be, says Tom Burns, CORE’s director of special projects. CORE looked into 1U rack-mountable servers, but decided they would take up too much space.
To meet its needs, the company would have needed 500 rack units of space, or more than 10 racks, and quite a bit of cooling, he said.
Instead, CORE considered a high-density solution and looked into blade servers. Burns looked into Sun Microsystems Inc.’s blade offering, but rejected it because he wasn’t sure about the kind of support he could hope to get from Sun around Linux.
“We looked at the Sun offering briefly – but Sun has been kind of schizophrenic in its attitude towards Linux in the past little while, and we didn’t feel like we were going to get the Linux support we needed,” Burns says.
Sun’s pricing was attractive, but its solution was one of the low-density 1U configurations.
“We were trying to stay in a high-density configuration,” he says.
CORE instead chose to go with an IBM blade server farm because they were well engineered, Burns says.
“The cooling was good, the management of them was good. They weren’t the cheapest, but they weren’t the most expensive either.”
CORE’s supercomputer has 84 servers on six racks, making it easier to manage than a low-density solution would have been.
The server farm was just one part of the solution. CORE built its supercomputer using an IP networking solution from Xtreme Networks Ltd. and storage solutions from a plethora of vendors, though the workhorse of the storage end is the Network File System from BlueArc Corp., Burns says.
Putting the system together was no easy task, he says.
“It’s a big job. Think about 500 computers spread in a room and how many times you have to hop in from one computer to another. There was a lot of work in terms of putting the right disk in every single blade, getting the right Linux OS installed in every blade, and doing the networking.”
CORE got help from IBM Global Services for the job, which understood the company’s need to rely on solutions from other vendors, Burns says.
“IBM was very accommodating in terms of saying, ‘We understand you want to work with these components’ and worked with us to tune all three” aspects of the supercomputer, he says.
So far the system hasn’t presented any problems.
“I would say it’s performed very, very reliably, and we haven’t had anything, even the usual, like every now and then a hard drive might go or a piece of memory might go,” he says. “And that’s the advantage of working in a blade configuration – it’s easier when a blade goes down to just pull the old blade and put one in from your spares pool. It takes you 12 minutes to update the new one with everything you need, and bam, it’s out crunching.”
Designing a backup system for the farm was challenging, as hundreds of terabytes of data had to be backed up, Burns says. He had to take a new approach to it, he says.
“We dealt with it by throwing out everything we ever learned about small systems and building a system designed to be scaled and to be extended and to be as flexible as we can while still keeping the performance metric very, very high.”
Though supercomputers have traditionally been used by academia, the research community and governments, small- and medium-sized businesses such as CORE are also starting to use them, says Dominic Lam, IBM Canada’s Toronto-based national eServer manager for deep computing.
Supercomputers are becoming more affordable on both the hardware and software side, Lam says.
They are now more of a commodity, he says, adding there are also more applications being written in parallel mode.
“Blade servers are really beneficial for businesses when you have a large number of servers,” says Alan Freedman, a Toronto-based research manager for infrastructure hardware at IDC Canada. Blade servers still represent a very small market, he says. In Q2 of 2004, only 1,800 units were sold in Canada. IBM led the market at 37 per cent, HP followed at 31 per cent, RLX Technologies Inc. at 11.5 per cent and Sun at six per cent.
Comment: [email protected]
