IBM Roadrunner | 2008 ~ 2009
Roadrunner was a supercomputer built by IBM for the Los Alamos National Laboratory in New Mexico, USA. The $100-million Roadrunner was designed for a peak performance of 1.7 petaflops and achieved 1.026 petaflops on May 25, 2008, becoming the world’s first system to reach a sustained 1.0 petaflop in the TOP500 LINPACK. In November 2008, it reached a top performance of 1.456 petaflops, maintaining its top spot on the TOP500 list. It was also ranked as the fourth-most energy-efficient supercomputer in the world on the Supermicro Green500 list, with a performance of 444.94 megaflops per watt of power used. The hybrid design of Roadrunner was later reused in several other energy-efficient supercomputers. Roadrunner was decommissioned by Los Alamos on March 31, 2013, and was replaced by a supercomputer named Cielo, installed in 2010.
IBM created this supercomputer for the U.S. Department of Energy’s National Nuclear Security Administration (NNSA). Roadrunner featured a hybrid design using 12,960 IBM PowerXCell 8i processors and 6,480 AMD Opteron dual-core processors in specially designed blade servers connected via InfiniBand. The system ran Red Hat Enterprise Linux and Fedora as its operating systems, and was managed using the xCAT distributed computing software. Open MPI Message Passing Interface was also used.
Roadrunner occupied approximately 296 server racks covering 560 square meters (6,000 square feet) and became operational in 2008. It was used by the U.S. Department of Energy to simulate the aging of nuclear materials to predict whether the nation’s aging nuclear weapons were safe and reliable. The system also found applications in the science, finance, automotive, and aerospace industries.
Roadrunner differed from other supercomputers of its time due to its continued use of a hybrid design first introduced by Seymour Cray in 1964 with the Control Data Corporation CDC 6600 and further advanced in 1969 with the CDC 7600. Unlike previous supercomputers, which typically used a single processor architecture, Roadrunner required all software to be specifically written to take full advantage of its hybrid architecture. This design involved dual-core AMD Opteron processors paired with IBM-designed PowerXCell 8i processors. Roadrunner was considered an Opteron cluster with Cell accelerators, as each node consisted of an Opteron core connected to a PowerXCell processor, with Opterons connected to one another.
Development of Roadrunner began in 2002 and went online in 2006. Due to its complex and novel design, it was built in three phases and became fully operational in 2008. Its predecessor was the Dark Horse machine, developed at Los Alamos, which was one of the first hybrid architecture systems based initially on ARM processors before transitioning to the Cell processor.
The first phase involved building a standard Opteron-based cluster while evaluating the feasibility of constructing and programming the hybrid version. The first phase of Roadrunner, which achieved 71 teraflops, was fully operational at Los Alamos in 2006. The second phase, known as AAIS (Advanced Architecture Initial System), involved creating a small hybrid version of the system using an older version of the Cell processor and was used to prototype applications for the hybrid architecture. This phase went online in January 2007. Phase 3 aimed for sustained performance above 1 petaflop, with additional Opteron nodes and more powerful PowerXCell processors. This phase, built at IBM’s Poughkeepsie, New York facility, successfully broke the 1 petaflop barrier during its fourth attempt on May 25, 2008. The complete system was then moved to New Mexico in the summer of 2008.
Roadrunner used two types of processors. The first type was the AMD Opteron 2210, operating at 1.8 GHz. Opteron processors were used for both computational nodes and system operations and communication nodes. Roadrunner had a total of 6,912 Opteron processors, with 6,480 used for computation and 432 for operations. Each Opteron had two cores, totaling 13,824 cores. The second type was the IBM PowerXCell 8i, operating at 3.2 GHz. These processors had one general-purpose core (PPE) and eight special performance cores (SPE) for floating-point operations. Roadrunner had a total of 12,960 PowerXCell processors, providing 116,640 cores (12,960 PPE cores and 103,680 SPE cores).
The TriBlade module was a logical structure consisting of two dual-core Opterons, 16 GB of RAM, and four PowerXCell 8i processors with 16 GB of Cell RAM. Physically, a TriBlade included one LS21 Opteron blade, one expansion blade, and two QS22 Cell blades, connected via InfiniBand and HyperTransport.
The entire system consisted of 18 Connected Units (CUs), each containing 180 TriBlades and connected by 288-port Infiniband switches. The final system had 6,480 Opteron processors, 12,960 Cell processors, 216 I/O nodes, 26 Infiniband switches, and a total of 296 racks.
Roadrunner was decommissioned on March 31, 2013. While it was one of the fastest supercomputers in the world, its energy efficiency was relatively low. Roadrunner delivered 444 megaflops per watt, compared to 886 megaflops per watt for a similar supercomputer. After its decommissioning, parts of the supercomputer were shredded, with some components kept for historical purposes, as some of its calculations were classified.
