Product Review: Intel S2600COE

Introduction

The Intel S2600COE is a part of the recently launched Romley platform and is intended exclusively for use in server systems. We have been very happy with the Asus Z9PE-D8 WS, but there are times when the workstation features of that type of motherboard are not wanted in a server system. If a feature is not necessary in a server environment, it is often preferred for it to simply not exist on the motherboard at all so that there are fewer components to potentially cause problems. The other advantage of this motherboard is that it allows for 16 sticks of DDR3 ECC RAM to be installed and used, versus the 8 sticks available on the Asus Z9PE-D8 WS.

This motherboard is designed for use with two CPUs, but can run with only a single CPU installed if needed. Keep in mind that since the PCIe and RAM slots are controlled by the CPU, half of the RAM and PCIe slots are non-functional without a second CPU installed. According to the block diagram from Intel (and confirmed in our testing), slots 3, 4 and 6 are controlled by the second CPU. For the RAM, the two banks directly above and below the CPU socket are those that are controlled by that CPU. The other two banks of RAM, three PCIe slots and everything on the chipset (USB, SATA, NIC, etc.) are still fully functional even with only one CPU installed.

BIOS

As is the norm on server motherboards, the BIOS options are fairly limited. The RAM configuration is very basic, allowing you to set the frequency, but not timings or voltage. Of the onboard devices, you can enable/disable the SATA controller, onboard video, LAN ports, serial ports, and the USB controller.

Initial BIOS screen	Advanced CPU settings (Single CPU)
Onboard Video settings	Fan throttling settings

The BIOS has a few options for controlling the fan headers, but they are altitude-based and do very little in the way of actually throttling the CPU fans. On this motherboard, you should not expect the fans to ramp to more than 10-15% of their maximum RPMs. As is true on most server motherboards, this board is designed to keep the CPUs running as cool as possible with only a small consideration for noise levels.

This motherboard does not support profiles and has no GUI-based method of updating the BIOS. Updating the BIOS is done through the EFI shell and even without a GUI is straight-forward, if a bit confusing for those not familiar with EFI shells. There are five steps to update the BIOS:

1. Load the new BIOS (downloaded from Intel's support page) onto a USB key
2. Boot into the EFI shell (either through the Boot Manager page in the BIOS or by hitting F6 during POST)
3. Mount the USB key with the command "map -r"
4. Navigate to the USB key with the command "fs0:"
5. Run the file "startup.sh" that was included with the BIOS files you downloaded in step 1

Connectivity

Connectivity covers the external ports on the I/O panel, the internal headers, the PCI-E slots on the motherboard, as well as the networking options. Starting with the external and internal ports/headers, let's take a look at what is available on this motherboard:

*Ports 0-3 functional by default, ports 4-7 require an Intel RAID C600 Upgrade Key

There are two SATA 6Gb/s ports, and eight SATA 3Gb/s ports. All ten of these ports support hot-swap as long as the Intel RSTe software is installed, but one very important thing to note is that only four of the SATA 3Gb/s ports are active unless you purchase and install an Intel RAID C600 Upgrade Key. These Upgrade Keys unlock additional RAID levels and/or allow the ports to function as SAS ports. Depending on the RAID levels needed, these keys range from being fairly cheap to very expensive. We tested using a RKSATA8 Upgrade Key and beyond the physical installation of the key itself, there was no additional setup needed to activate the additional 4 SATA ports. 

For networking, there are an impressive four gigabit Intel LAN ports on the rear I/O panel. Internally, there are two IEEE1394a firewire headers, although the motherboard does not come with a Firewire Bracket to allow external use of these ports. Finally, this motherboard includes a Matrox G200e onboard graphics adapter that can be used either through the Intel AXXRMM4LITE remote management module (purchased separately) or with the external VGA port. Note that if the onboard video is enabled, it becomes the primary display even if a more powerful discrete GPU is installed.

Moving on to the PCIe slots, here is a list and picture of the slots available on this motherboard:

PCI Express 3.0 x16 PCI Express 3.0 x16 PCI Express 3.0 x8 PCI Express 3.0 x16 PCI Express 3.0 x16 PCI Express 2.0 x4 in an x8 size

Since there are multiple CPUs, the PCIe lanes are divided between both CPUs. Slots 1, 3 and 4 are controlled by CPU2, while slots 2 and 5 are controlled by CPU1. The bottom slot is controlled by the chipset and as such does not support PCIe 3.0.

Motherboard Layout

After reviewing the layout of this motherboard, there are four issues we found that range from minor inconveniences to major issues. In the order of least to most problematic, they are:

1. The headers for the front LED/switches (BN on the schematic) are on the bottom-left corner of the motherboard. In some chassis, this will cause issues with the front panel cables not being long enough to reach the headers. All of the EATX chassis currently offered by Puget Systems do not have this issue, which is why this is the least of our concerns.
    
2. The CMOS battery (F on the schematic) is located just under the primary PCIe x16 slot. What this means is that if the CMOS ever needs to be cleared by removing the battery, the video card (if it exists) must first be removed. This is not a very big deal, but is still something to keep in mind.
    
3. Large heatsinks such as the Gelid Tranquillo Rev2 overhang the RAM slots on either side of the CPU socket. RAM can still be installed into those slots, but the CPU cooler must be removed in order to install/remove them making it a huge hassle. Coolers such as the Puget Hydro CL3 Liquid Cooling System or the Dynatron R17 will not have this issue. This is simply a result of so much hardware being crammed onto this motherboard. Realistically, we could not think of a better motherboard layout that would not have this issue.
    
4. Lastly, this motherboard is not a true EATX motherboard as the mounting holes are in slightly non-standard locations. On this motherboard the top-left (near the rear I/O panel) and the top-middle (between the top RAM banks) do not correspond to standard mounting locations. So in order to have this motherboard securely mounted into a standard chassis, minor chassis modification must be done to add two additional motherboard standoff locations.

Stability and Compatibility Testing

Our extended testing procedure for motherboards (see below) may at first glance seem short, but in actuality is very extensive. What you need to keep in mind is that the very first item – running the test system through our standard build process – is in itself a 98-point checklist.

The majority of the other checkpoints are designed to verify that the motherboard will function properly with a wide range of hardware. For that reason, we test using onboard video, Nvidia Geforce, AMD Radeon HD and Nvidia Quadro based video cards. We also test the motherboard with the largest quantity of RAM we can (16 x 8GB sticks), the fastest supported RAM currently offered by Puget Systems (DDR3-1333MHz) and since this is a server motherboard with 16 sticks of both ECC and non-ECC RAM.

Test
Run test system through the Puget Systems build process
Review Device Manager to ensure all drivers loaded correctly
Loop test system through >50 reboot loops
Loop test system through >50 standby loops
Verify standby functionality using onboard video, Nvidia Geforce, AMD Radeon HD, and Nvidia Quadro GPUs
Run 3D graphics testing using onboard video, Nvidia Geforce (single and SLI), AMD Radeon HD (single and Crossfire), and Nvidia Quadro GPUs
Test all internal SATA controllers
Verify stability with the largest quantity/size of RAM available
Verify stability with the fastest RAM offered by Puget Systems
Review Event Log for any unexpected warnings/errors
Verify basic funtionality with latest version of Ubuntu (11.10)

In our testing, there are six issues that we found. Note that only the first four issues are actually caused by the motherboard. The other two are actually operating system issues, so they cannot be held against this motherboard. Since they did come up in our testing, however, we still felt the need to point them out.

1. This motherboard only supports S1 sleep. S3 is not simply non-functional, but not present at all on this motherboard. This is not unusual for server-class motherboards since servers are designed for maximum uptime, but it worth knowing if you ever want to use this motherboard in a more traditional workstation system.
    
2. While this motherboard does have onboard video, the onboard video is a simple Matrox G200e which does not do well running 3D graphics. This is completely to be expected, but since it technically fails our 3D graphics testing step we wanted to point it out. With both Nvidia and AMD based video cards, this motherboard passed our 3D graphics testing with no problems.
    
3. The SATA 3Gb/s ports (both the original four and the four activated with an RAID Upgrade Key) do not play very well with CD/DVD/Blu-ray drives. Optical drives will work fine in an OS as long as the Intel RSTe software is installed, but they are flat out not detected by the BIOS. This means that to install an OS or run any bootable media from the CD/DVD drive, it must be plugged into one of the two SATA 6Gb/s ports. We have discussed this issue with Intel and they told us that 3Gb/s ports do not support optical drives and that you must use the white 6Gb/s ports for CD/DVD drives.
    
4. Both of the onboard RAID options (ERST2 or RSTe) do not have a configuration utility that can be accessed during POST and the arrays must be setup from within an operating system. This limits the onboard RAID to being used solely as secondary storage since you cannot configure a primary array prior to installing an OS. We have also talked to Intel about this and they have told us that there should be option ROMs (configuration utilities that run during POST) to configure a RAID array so this appears to be a bug that should be fixed in a future BIOS update. In the mean time, not being able to configure a primary RAID is a very big issue that really limits the usage of RAID on this motherboard.
    
5. During our test OS installation of Windows 7 Ultimate, we found that with the higher end Xeon E5 CPUs (specifically those with 20MB of cache) were unable to complete the Windows 7 Experience Index assessment. During the assessment, an error of "Could not measure system memory performance" occured which halted the test. After discussing this with Intel, it was determined that WinSAT (which Experience Index uses) detects the CPU cache size (20MB) and doubles the cache size value as the input parameter of the buffer size (40MB). WinSAT is limited to support a buffer size of only 32MB, so a buffer larger than this size causes WEI to error. This problem has only been found to affect WEI and should not impact system performance. This is really an OS/CPU combination issue, however, so we really can't hold it against this motherboard. Server editions of Windows and the upcoming Windows 8 OS should not have this issue.
    
6. Our last problem is also an OS issue, but has to do with Linux. The current release of Ubuntu (11.10) does not support the SATA controller and as such cannot install or run as a live environment from an optical drive. We did confirm that Ubuntu 12.04 Beta loads and runs without any issues, so this problem will be resolved once Ubuntu 12.04 is released in late April 2012.

Our testing not only found problems, but also confirmed a few very positive points of this motherboard. Like the Asus Z9PE D8-WS, the chipset on this board runs much, much cooler than on the previous generation of chipsets. In fact, even during our most intense testing, the chipset only hit 41° C. We also had absolutely no driver or BIOS issues at any point in our testing. Running 16 sticks of DDR3-1333 8GB RAM worked without any issues giving us a huge 128GB of usable onboard RAM.

Conclusion

We are used to working with server motherboards like the Intel S5520SC which, while technically a server board, still retains many desktop features such as onboard audio and S3 sleep. The Intel S2600COE completely departs from this and is as true a server motherboard as can be. What this means is that for a workstation-type computer, the Asus Z9PE D8-WS is likely a much better option than this motherboard.

The fact that many desktop features are not present actually lends well to a server system, however. One thing we have found to be true over the years is that the more features present on a motherboard, the more likely it is that one or more problems will arise. It is simply a matter that the more features present, the more individual things that can go wrong at any point.

There are still a few things that we don't like about this motherboard (the SATA 3Gb/s issues and the lack of a way to configure a RAID form the BIOS for example) but overall this motherboard is a great choice for a high-powered server system. Our list of problems are longer than we would normally allow on any product we carry, but many of the problems will not be an issue for a server system.

Pros:

1. Supports up to 16 sticks of DDR3 ECC RAM
2. 10 onboard SATA ports (8 x 3Gb/s, 2 x 6Gb/s)
3. Includes two internal IEE1394a headers
4. Four integrated Intel Gigabit LAN ports
5. Integrate onboard video controller
6. Chipset runs very cool 
7. Can run either one or two CPUs (some features non-functional with only one CPU)

Cons:

1. 4 SATA 3Gb/s ports are not active unless an Intel RAID C600 Upgrade Key is installed
2. SATA 3Gb/s ports do not detect optical drives in the BIOS. Drives are detect and work fine from within an OS as long as the Intel RSTe software is installed 
3. CPU fans do not ramp any appreciable amount
4. Large heatsinks overhang RAM slots on either side of the CPU sockets
5. Mounting holes are non-standard and requires minor chassis modification to mount the motherboard
6. S3 sleep not supported
7. Limited USB ports/headers
8. No USB 3.0 support
9. No configuration utility for onboard RAID from the BIOS (should be resolved in a future BIOS update)
10. Having the onboard video enabled makes it the primary graphics adapter even with a discrete GPU installed (BIOS and POST screens only show through the primary adapter)