What Makes An ARM-Based Server Different?

SoftIron’s Director, Platform Software – Alan Ott, gives us the run down.

With ARM’s release of the ARMv8-A architecture introducing a 64-bit execution environment, its natural to begin to consider the use of ARM processors in new domains such as servers, networking appliances, and infrastructure. Unlike 32-bit processors which are limited to only addressing 4GB of RAM, 64-bit processors can address a seemingly unlimited amount of memory, enabling use in data centers currently dominated by x86-based servers. But what makes an ARM-based server different? Certainly ARM-based CPUs have an established track record of operating at lower power than their x86-based counterparts, but what are the other advantages? In addition, how can existing knowledge and infrastructure be used with new servers in a complementing way?

ARMv8

ARM-VS-ARM

It’s important to note that ARM Holdings does not actually manufacture processors. ARM sells the designs for CPU cores and silicon manufacturers build processors and other peripherals using these cores. While all processors which share the same core can run the same programs (as they share an instruction set), the processors themselves will differ in varying ways. RAM controllers, on-board peripherals (USB, Ethernet, PCIe, SATA), hardware-acceleration modules (Cryptography processors, random-number generators, Digital Signal Processors), DMA controllers and interconnects can vary greatly between ARM processors from different manufacturers.
As we’ve seen with 32-bit ARM CPUs used in cell phones and industrial applications, the features in ARM-based server processors will vary by manufacturer. While it might seem like this could cause fragmentation in the market and instability with respect to existing software, the reality is that almost all of this differentiation will be handled by the operating system kernel. In the short term this kernel is Linux, but in the near future we can expect other server operating systems to be ported to ARMv8-A as well. This is ultimately no different than the differentiation in x86 server processors which we see today, where operating systems have achieved a high-level of compatibility between competing hardware products.

HARDWARE

While each ARM-based server motherboard will be different, they will generally share many design aspects with existing x86-based servers. The AMD Opteron A1100 CPU contains a two-channel DDR4 controller, with standard slots just as would be on an x86 server. A1100 motherboards will be able to take advantage of commodity RAM pricing in the same way as their x86 counterparts. The A1100 also contains a PCI Express controller. Because of the driver infrastructure modern operating systems provide internally, this means most if not all PCI devices can be used on an ARMv8-A server without any issue (and without any driver modification), opening up a variety of commodity hardware for use in ARM-based servers. The A1100 also contains on-board hot-pluggable SATA controllers leveraging commodity storage.

An important way in which ARMv8-A servers differ from their 32-bit ARMv7 predecessors is the boot process. On the 32-bit processors present in cellphones and industrial devices, the boot process is extremely processor-specific. Processors from different manufacturers have different requirements for system start-up, CPU and peripheral initialization, and image loading. Bootloaders such as Das U-Boot take care of the complexity, containing different and complex start-up code for each supported processor. This has always differed from the x86 space where systems almost universally conform to the loose set of standards and practices we know as the PC, including BIOS, physical memory-mapped peripherals such as video, keyboards, mice, and hard drives. The result has been that x86 PC’s (including those with 32 and 64 bit processors) have had the ability to boot standard operating system images since the early days of the PC, as much of the boot-up process is taken care of in ROM. This situation is not without its complexity, however. While many modern concepts have been implemented in what we call the modern PC, backwards compatibility has been painstakingly maintained, at the cost of significant architectural overhead. Throughout the history of the PC, we have reached and been required to work around architectural limits many times, each time leaving complex backwards compatibility scars.

Starting with ARMv8-A, ARM has put together a standard called the ARM Server Base System Architecture (SBSA). The SBSA mandates a set of standards for ARM-based servers focused on the low-level non-core peripherals and boot procedures . SBSA requires ARM-based servers to conform to a basic level of commonality in the areas required for a system to boot, including memory initialization, ACPI and UEFI. The result is that ARMv8-A servers can boot from standard installation media the same way x86 servers can. Installation of a standard Linux distribution is now no more difficult on an ARMv8-based server than it would be on its x86-based counterpart. This is a radical departure from 32-bit ARM processors designed for embedded systems, and makes ARMv8-A a viable contender in the data center.

KEY DIFFERENCES

ARM-based servers as they currently exist do have some important differences from their x86-based counterparts. While x86-based servers typically embrace a more componentized model of a CPU and a separate associated chipset which implements peripherals and buses, ARM-based CPUs have taken a different approach of putting the CPU and peripheral controllers on the same physical die. This combined CPU and peripheral controller is called a System-on-Chip (or SoC). The advantages of this have traditionally been ease in board design (with one fewer high-speed interconnect), power consumption (fewer dies total), layout complexity (fewer passive components), and performance (peripherals not competing for the same chipset interconnect). The A1100 follows this same model, with SATA and 2 10Gb Ethernet MACs located on-die. With the success of the SoC model on embedded systems, ARM-based server SoC’s will bring the same advantages to the data center.

ARM-based servers will typically not have on-board graphics. While VGA graphics are a part of PC architecture, there is no such equivalent in the SBSA. That’s not to say that one couldn’t add a PCIe graphics card and run X11 the way one might expect, but EFI, bootloader, and Linux boot-up messages will not be visible on the video output.

While each ARM-based server could potentially be different in this respect, the Opteron A1100’s pre-boot configuration and boot-time messages use the serial port. This should be a familiar to users of other server platforms where bootup can be optionally be done the same way.

Since they’re designed for the data center, ARM-based server motherboards may be missing other common peripherals, such as USB. With available PCIe however, USB and other peripherals can easily be added to a server if desired.

SUMMARY

ARMv8-A is an instruction set with available CPU core designs. ARM’s Server Base System Architecture (SBSA) standard ensures boot-time interoperability between ARM-based servers, ensuring easy installations and boot of operating systems from standard media, being CDROM, rotational media, or SSD. ARMv8-A implementations like the Opteron A1100 from AMD provide standard memory peripheral buses designed to be used with off-the-shelf hardware. With good support from the major Linux vendors, deploying servers on ARMv8-A hardware will be just as easy as we’ve been used to on x86.