Ensuring your servers stamp as small a carbon footprint as possible on the earth and in your data center can encompass everything from making sure they are shipped in recyclable packaging to hiring an analyst who can predict the total life-cycle environmental impact.
For this test, we examined power consumption as a way to judge whether Windows Server 2008 or Linux is, in fact, the ‘greener’ operating system. As the price of power hits record heights, power reduction mechanisms shipping within an operating system should play a key role in you energy conservation plan.
Our tests point to Linux as the winner of the green flag by margins that topped out at 12%. But we must note that our results are full of stipulations imposed by our test bed, and as the more truthful car advertisements might warn — your wattage may vary.
We ran multiple power consumption tests using Windows Server 2008 Enterprise Edition, Red Hat’s Enterprise Linux (RHEL) 5.1 and SUSE Enterprise Linux 10 SP1 on four, popular 1U server machines, one each from Dell and IBM and two from HP. The results showed that while Windows Server 2008 drew slightly less power in a few test cases when it had its maximum power saving settings turned on, it was RHEL that did the best job of keeping the power draw in check across the board.
The variable settings allowed by both Windows and Linux — which let you toggle between having a high energy efficient server vs. a high performing one — can certainly have an impact on overall server consumption. But again, your mileage will also vary given the workloads you place on your servers and whether or not you’re using popular virtual machine hypervisors to support multiple operating system instances on the same physical server (see related story).
The edge in either test category will likely not last as operating systems become more finely tuned to work in lockstep with advanced server chipsets, and as additional coding techniques that more closely tie operating systems and applications to power considerations take hold across the industry.
Part of the current “green” operating system difficulty lies in the disconnect between how an operating system and its applications can be optimized to let the underlying system quiet itself down to a lower power-consuming state while at the same time not sacrificing the ability to react to servicing application (and therefore system and user) needs.
In our testing, we found that the CPU ‘throttle-back’ mechanism — the main technique for how an operating system can aid in reducing a server’s energy draw — requires new firmware and updated drivers that specifically support that feature. Only the IBM x3550 and the HP DL-360 G5 arrived ready for optimal power conservation.
The HP DL-160 and Dell 1950 servers required several updates throughout our six-week test period to accommodate the CPU throttling features of Windows 2008 and Linux.
We truly know from the trenches that it really isn’t easy getting your servers to be green.
CPU conservation
No matter the operating system, Windows or Linux, the leading form of power conservation comes from throttling back the CPU to let the server rest during quiet activity times. Spinning down hard disks to a quieter state is the other major power-saving setting available to Windows servers.
Even though Linux desktop distributions can use the Advanced Configuration and Power Interface (ACPI) specification, which is designed for laptops, rather than servers, that feature was not implemented by Red Hat or SUSE for the servers we tested.
Some chipsets are designed for throttle-back, while others (especially older ones, predating 2007) always work at full pace and power full time. Only in the past three years have processors shipped from Intel, Advanced Micro Devices, Via Technology, and others in the x86 family have been specifically designed to cycle between fast (and higher power consuming) and slow (power savings) states.
The systems used in our testing ship with Intel Xeon multi-core CPUs, which can support throttle-back (manifested by slowing down the CPU clock so that the power strobes through the CPU more slowly against the slowed clock); but as we stated earlier the server’s BIOS and firmware must be sufficiently upgraded to correctly supported this.
The IBM x3550 and HP DL-160 house a single quad-core CPU (of different models, see How we did it), while the Dell 1950 and HP DL360G5 each housed two quad-core CPUs for a total of eight cores.
Once throttled back, the millions of transistors in CPUs can turn back on almost spontaneously at virtually the speed of the CPU (we could detect no latency issues in our testing), and can throttle-back down at nearly the same speed. The throttle back condition can save quite a bit of power in the four systems we tested, but most other electronics within the system remain on and therefore continue to consume power.
Because the system must be ready to service application requests, it must have at minimum, some electronics running to monitoring application, user, network and other peripheral service requests. This minimal amount of power drawn is what you see measured in our quiescent state (sometimes referred to as minimal ready-state) results.
Operating systems must allow a CPU to throttle back to this minimal ready state, to be considered green from the power consumption perspective, and both Linux and Windows allow for this. However, there is a ‘tickless’ version of Linux on the horizon that may prove to have power savings characteristics.
System interrupt ticks are ‘time slices’ that the operating system uses to queue activities, and they’ve been traditionally set in the past half dozen plus years to a 1,000 ticks per second, each of which serves as an interruption to the CPU. A tickless version of the Linux kernel now reportedly exists that interrupts the CPU less frequently, but was not part of the Linux distribution kernels we tested — although that addition is planned in future editions of Red Hat and SUSE.
Choosing which level is green enough for your servers
In advance of setting up and running our testing, we talked with Novell/SUSE, Red Hat and Microsoft regarding their respective green initiatives.
We also asked IBM, HP, and Dell to supply the server samples they believed promoted their fullest green potential, although measuring the hardware elements was not our primary point here. There is much contention in the server marketplace over power supply efficiency, and other hardware energy conservation initiatives that often use vendor-specific hardware management APIs that have nothing to do with the deployed operating system.
We weren’t interested in a server hardware ‘bake-off’ that measured power consumption of the hardware elements, as this is the operating systems view of consumption.
Windows 2008 Server and Windows Vista power saving modes are essentially identical. They allow a system to fall back to increased resting states (principally in CPU power consumption and disk hibernation). These modes fit the Advanced Configuration and Power Interface V3 support that is more generally associated with personal computer, rather than server application use.
There are three states to these Windows power plans — Power Savings, Balanced and High Performance — which are selected through the Windows Control Panel Power Settings options. These options can also become enforced through the Active Directory through group policies. A program, powercfg.exe is also available to help establish very highly detailed performance policy settings, but the nearly endless permutations available with that executable were clearly beyond the scope of this test.
We chose to test the Windows Power Savings and High Performance power plans because they offer the highest comparability to power consumption parameters available in Linux — a full throttle performance mode and a rational, low-latency power savings mode.
The energy saving choices available with the Linux 2.6 kernels shipped with RHEL 5.1 and SUSE Enterprise Linux 10 center on the ability to throttle back CPU clock speed through a kernel module called cpufreq. These modes are called “governors”, and are officially referred to as performance, ondemand, powersave, and conservative. There is a fifth, user-space governor, but it homes in on only specific, policy-defined root objects, and we’re testing the operating system rather than discrete processes governed by the operating system.
We were able to initially test all of the servers in all of the savings modes that the cpufreq module supports to determine which would be most applicable to our test. We chose performance for our full throttle tests and ondemand for our power saving mode.
We rejected conservative mode because it introduces unnecessary latency to a server in random-accessible, 24/7 operations service. And we rejected the powersave mode because it slows down the processor, and everything takes longer — not a representative application for this test as there aren’t many situations where ‘stepping on the garden hose’ makes sense, except in periodic batch processing applications and other places where you’re simply willing to wait much longer for a required result.
Both Linux’s cpufreq kernel module and Windows’ power settings can be changed dynamically if needed, although we didn’t change them during the duration of our tests.
We chose two tests to measure power consumption. The first was a quiescent server test in which each operating system and hardware server pair sat idle for four hours in both performance (high power use) and the power savings mode for each operating system. The second active test sought to gauge consumption under load in which we sent a continuous stream of e-mails to each server and operating system pair for the duration of the four-hour test in both performance and power savings profile.
The active test used an e-mail test script to continuously send e-mails to the server and operating system pair under test. In the case of the two Linux distributions, we used sendmail/procmail as our SMTP server with 1,000 users.
We installed Microsoft Exchange Server 2007 under Windows 2008 Enterprise Server Edition, imported the same 1,000 users, and assaulted it in the same way, by using scripts we generated. We let the operating system and application pick the number of cores to use as a default.
Summary results
With the IBM x3550, HP DL360G5, and Dell 1950 servers we performed our quiescent and active state tests twice: once with High Performance settings in place; and, the other with the Power Saver mode applied.
For the HP DL-160G5 server, we could only complete the tests without the maximum power saving setting applied as the server crashed each time we attempted to toggle between the power saving settings. HP says a fix for this issue should be available by the time this test publishes.
In most cases when sitting idle, Windows Server 2008 drew slightly more power than either Linux did on the same server. The exception was when Windows Server 2008 was running in power savings mode on the Dell server, where it drew on average 3% less power.
While we’ve already noted that RHEL conserved more power than Windows Server 2008 in most cases, we’d be remiss if we did not also mention that RHEL drew less power than its Linux brethren in all quiescent tests. The spread for that difference was a low of .5 watts less on the IBM server when the systems were tuned for top performance to a high of over 5 watts less on the HP DL-160G5 server when the systems were in power savings mode.
During the active tests, Windows Server 2008 running in power-savings mode on the Dell box used over 7% more power than the Linux average on the same hardware. But on the IBM and HP DL-360G5 server, Windows Server 2008 pretty much ran on par with the lowest Linux consumer in those tests.
When running in the high performance mode in the active test, Windows Server 2008 used as much as 11 percent more wattage than the average Linux power draw on the same hardware. That said, Windows Server 2008 had the best power consumption rating in this test run on the HP DL-160G5 server, spending on average about 6.5 watts less than Linux.
The server hardware impact
Server makers responded to our requests to be part of the test bed with several kinds of CPU and disk configurations all housed within 1U hardware casings. Overall, the savings wasn’t startling between the highest and lowest number for a server in terms of watts consumed in any test.
There’s little doubt that advertised power savings numbers can be achieved, but servers will need to be tuned for both the operating system, as well as the power-savings applications’ ability to control which cores are either used or put to rest in a load-balancing situation. Rather than push configurations for core optimizations (we couldn’t find settings to do this for the applications tested), we let the operating systems take care of the details.
The details, it turns out, was that all of the cores in all of the tests saw at least some activity both in quiescence but also in application use. SMP kernels were used for all tests.
IBM’s x3550 was the leanest and greenest, both in terms of CPU ‘horsepower’ but also power consumed. In quiescent tests, there was less than a 2-watt difference among any of the three operating systems tested in either performance or savings mode.
In the active tests, the power draws stayed within that 2-watt range with the exception of when Windows Server 2008 drew 87.8 watts, compared with SUSE’s 79.6 watts and RHEL’s 78.3 watts in our active test when the systems were tuned from performance.
The dual-quad core Dell 1950 sucked more power overall, but with more cores, it also delivers more computing power. In the quiescent tests, the test where settings were tuned for performance, Red Hat defied logic and used slightly more power than it did in the power savings mode, but otherwise, the results followed as logic and settings expected.
The HP DL-160 didn’t show dramatic behavior changes in settings in the quiescent tests, and made Windows 2008 Server a winner in the active, performance-modes test where it seemed to give its best performance.
When we tested the DL-360G5 late in the series after having difficulties with the HP DL-160G5’s Windows testing, we found it behaved similarly to the Dell 1950 (which has the same number of CPUs as this bigger HP box), consuming the highest number of watts, but also doing so with the largest number of drives.
Summary
Microsoft, Red Hat and Novell/SUSE each have power savings and green initiatives that are widely publicized. Nonetheless, we were astounded by the effort we needed to undertake in order to chase down of firmware, BIOS and other updates was necessary to get real savings in the tests we conducted.
Tuning servers for optimized power savings could yield better results, but would create a new painstakingly tedious server management discipline required to constantly control the deep complexities of the configuration variables involved.
We recommend that every potentially green server deployment be thoroughly checked, as each server model may or may not have the necessary BIOS settings and operating system chipset recognition features that will result in a power savings.
While all of this leg work might ring true to that famous line in the Kermit the Frog song, entitled “It Ain’t Easy Bein’ Green”, if you consider the power savings over a five year service life, the boost to the bottom line will likely be worth the effort.
Henderson is principal researcher and Dvorak is a researcher for ExtremeLabs in Indianapolis. They can be reached at [email protected].
