Inside IBM’s Decision to Purchase FPGA Chips

In the fast-paced world of technology, it’s easy to overlook the underdogs. Two days ago, there was headline news about AMD securing some major deals, including Oracle and IBM. Oracle plans to use AMD’s Instinct MI300X AI chip in its cloud services, along with HPC (High-Performance Computing) GPUs. IBM, on the other hand, is expected to adopt AMD’s Xilinx FPGA solution for artificial intelligence workloads.

01

A New Challenger Emerges

The high demand for NVIDIA GPUs has led to an overflow of demand, with Oracle becoming one of the first companies to deploy the MI300X. It’s somewhat like going to a popular restaurant with long lines – you have to wait for your turn, but you still need to eat.

The MI300X is still in its infancy and is set to be released in the fourth quarter, currently in the sampling stage. AMD’s software ecosystem is not as mature as NVIDIA’s. Training and running large AI models depend not only on GPU performance but also on system design.

02

IBM’s Unique Approach

IBM, on the other hand, has taken a different approach. IBM’s AI inference platform uses NeuReality’s NR1 chip, with AMD (Xilinx) FPGAs playing a key role. NeuReality, a startup established in Israel in 2019, introduced the NR1-P in February 2021, a platform centered around AI. In November 2021, NeuReality announced a partnership with IBM, which includes licensing IBM’s low-precision AI cores to build NR1.

NR1 belongs to the NeuReality NAPU series and is an FPGA-based chip, featuring an embedded AI inference accelerator and network and virtualization capabilities in a SoC. According to NeuReality, NR1’s performance per dollar is expected to improve 15 times compared to other deep learning chip vendors’ GPU and ASIC solutions.

In my understanding, NeuReality can be considered a solution provider in the AI domain for Xilinx FPGA, offering an FPGA-based AI inference acceleration platform.

03

IBM’s Enduring Legacy

In the world of chips and artificial intelligence, discussions often revolve around prominent companies like TSMC, Intel, NVIDIA, and AMD, and IBM’s presence seems to have faded into the background.

However, IBM remains a heavyweight in the chip and AI domain and cannot be overlooked. On October 25th, IBM released its third-quarter financial report, with quarterly revenue reaching $14.75 billion and an operating profit margin expanding from 11.4% to 14.8%.

IBM has a rich history of contributions in the chip field. In 1960, they developed flip-chip packaging technology to enhance component reliability. In 1966, IBM proposed the idea of single-crystal transistor DRAM. In 1974, IBM’s research institute designed a prototype for a Reduced Instruction Set Computing (RISC) architecture computer, which is still in use today.

IBM’s contributions in the chip field also encompass CMP, SiGe stress, ArF lithography, computerized lithography techniques, chemical amplification lithography, Silicon-on-Insulator (SOI) technology, Power processors, AI chips, quantum chips, and more.

In 2020, IBM developed a non-von Neumann architecture chip technology based on phase-change memory (PCM), enabling low-power execution of complex and accurate deep neural network inference tasks.

In October 2022, IBM introduced the first Artificial Intelligence Unit (AIU) chip, designed and optimized for accelerating deep learning models’ matrix and vector calculations, outperforming CPUs significantly.

In the field of quantum computing, IBM released the 65-qubit Quantum Hummingbird in 2020, followed by the 433-qubit Osprey chip in November 2022, and plans to launch the 1123-qubit IBM Quantum Condor in 2023, with a system exceeding 4000 qubits in 2025.

IBM has been ahead of traditional chip manufacturers in chip fabrication research and has introduced prototype chips for new processes. For example, the 10nm chip was developed by IBM in 2014 and went into production in 2017, while the 5nm chip was proposed in 2015 and entered production in 2018.

In 2021, IBM was the first to release a 2nm chip globally, using nanosheet transistor stacking, also known as Gate-All-Around (GAA) transistors.

IBM’s strength in fundamental research is a cornerstone of the global IT technology landscape. IBM and Xilinx’s strategic alliance has been ongoing for many years.

When Xilinx introduced the V5 series of FPGAs, they already integrated IBM’s PowerPC hard cores into their chips. In 2015, when Microsoft successfully incorporated Altera FPGAs to accelerate its Bing search engine data centers, IBM immediately started a partnership with Xilinx to jointly develop FPGA acceleration platforms.

In 2017, IBM introduced a new server architecture that separated FPGAs and server CPUs, directly connecting FPGAs to data center networks. This approach allows FPGAs to serve as standalone computing units, forming clusters of multiple FPGA units for use in emerging large-scale data centers. This solution greatly improves the performance per dollar of servers by sharing resources such as power supply, PCB backplanes, and network connections. More details can be found in IBM’s paper, “An FPGA Platform for Hyperscalers,” published at the IEEE Hot Interconnects Conference in August 2017.

04

FPGAs: The Future of AI Computing

Compared to GPUs, FPGAs have advantages in lower power consumption and latency. GPUs often struggle to efficiently utilize on-chip memory and frequently need to access off-chip DRAM, resulting in high power consumption. FPGAs, on the other hand, can flexibly use on-chip storage, leading to significantly lower power consumption. Additionally, the architecture of FPGAs gives them a strong advantage in terms of latency for AI inference compared to GPUs.

In 2018, FPGA accelerator cards had a market size of only $1 billion, but a research report from Semicon predicts it will exceed $5 billion this year.

In the data center AI computing market, NVIDIA GPUs currently dominate with a market share of 91.9%. NPU, ASIC, and FPGA markets have market shares of 6.3%, 1.5%, and 0.3%, respectively.

The main reason for NVIDIA’s success, in my opinion, lies in the strength of the CUDA ecosystem. It has broad programmer coverage and offers a wealth of open-source resources and mature solutions. Many large-scale models in China are adaptations of overseas open-source projects, so most of them opt for NVIDIA solutions.

However, when looking at the rapid advancement of AI computing technology, it is clear that various players from around the world will continue to compete vigorously. Whether it’s Google’s TPU, IBM’s Power architecture, or Intel/AMD’s heterogeneous acceleration chips, all will be vying for a share in this significant market. In such a large and rapidly evolving field, no major player is likely to give up.

05

Conclusion

The battle for AI computing power is just beginning. As AMD secures key partnerships and IBM continues to innovate in the chip and AI domain, the landscape of AI computing is undergoing a transformation. The race to provide faster, more efficient, and powerful AI solutions is intensifying, benefiting not just these companies but also the entire technology ecosystem.

In this dynamic environment, it’s the consumers and businesses that stand to gain the most. With the advent of powerful AI chips and accelerators, the possibilities for applications in fields like healthcare, finance, autonomous vehicles, and more are boundless. Smarter, faster AI can lead to breakthroughs in medical diagnostics, improved financial models, and safer transportation systems.

As we move forward, it’s essential to keep an eye on these developments and the role that companies like AMD and IBM play in shaping the future of AI computing. Their contributions not only advance the technology but also inspire others to push the boundaries of what’s possible.

06

Frequently Asked Questions