Recently, there has been a continuous stream of news related to GPUs, such as OpenAI exploring in-house AI chips, Microsoft set to launch its in-house AI chip in November this year, Samsung introducing the Exynos 2400 processor, upgrading its in-house GPU, and more. These are all manifestations of the market and manufacturers seeking more diversity.
In the overall downturn of the global semiconductor industry in 2023, only AI server systems and related chips stand out, with most of these systems using GPUs. Unfortunately, for most GPU manufacturers, in this exceptional market, most are left with meager gains, and some can’t even manage that. The only one feasting on this market is NVIDIA. According to visualcapitalist statistics, NVIDIA already holds more than 80% of the $150 billion global AI market, with AMD and Intel lagging far behind. More importantly, this market is growing every day, and neither GPU chip manufacturers nor system and internet giants want to see NVIDIA continue to expand its market share.
Apart from AI applications, GPUs continue to play a critical role in traditional graphics processing applications, whether it’s in PCs or smartphones. The market space for this is still vast. In the smartphone application market, major IP companies like Arm and Imagination Technologies dominate, and although Qualcomm is not an IP company, its dominant position in the smartphone SoC market forces many smartphone manufacturers to not only purchase Arm or Imagination’s GPU cores but also be dependent on Qualcomm. In this situation, many are exploring alternative GPU options.
In conclusion, an increasing number of major GPU application companies (smartphone and internet manufacturers), as well as new chip design companies, are beginning to develop their own GPUs or introduce new types of AI processors that can rival NVIDIA’s GPUs, with the hope of gaining more commercial benefits in the large-scale application market.
01
How AI Server Demand Could Transform the Industry Chain
The widespread adoption of AI, represented by ChatGPT, has led to a shortage of NVIDIA GPUs. At the same time, similar products from other chip suppliers cannot quickly meet the needs of AI server system manufacturers in terms of software and hardware ecosystems. This has caused the prices of NVIDIA processors to skyrocket. However, as the saying goes, too much of anything is not good. Major system and internet companies cannot accept this situation in the long term and are embarking on the journey of developing their own AI chips.
Recently, even the creator of ChatGPT, OpenAI, has joined the ranks of self-developed chips. Insider information reveals that OpenAI is exploring in-house AI chip development and has started evaluating a potential acquisition target. OpenAI’s CEO, Sam Altman, mentioned that acquiring a chip company can expedite the process of OpenAI developing its chips. An individual familiar with OpenAI’s plans revealed that the company has already conducted due diligence on potential acquisition targets, but it is currently unknown which company they are considering.
It’s been reported that since at least 2022, OpenAI has been discussing various solutions to reduce their heavy reliance on NVIDIA GPUs. Running the ChatGPT system is very costly. Analyst Stacy Rasgon from Bernstein estimates that each query costs about 4 cents. If ChatGPT’s search volume were to grow to one-tenth of Google’s search volume, it would require approximately $48.1 billion worth of GPUs and $16 billion worth of chips to maintain the system’s operation annually.
NVIDIA is expected to earn a profit of 56.51 cents for every dollar of revenue this year, making it one of the most profitable tech companies globally. As Jeff Bezos, the founder of Amazon, has previously stated, cloud providers don’t have to hand over these profits to chip manufacturers. They can control an increasingly vital part of their cost structure by investing time and money. In-house chip development is a key avenue for this.
Meanwhile, OpenAI’s major shareholder, Microsoft, is also set to launch its in-house AI chip, Athena, in November to reduce its dependence on NVIDIA GPUs. Recently, there have been reports that Microsoft is reducing orders for NVIDIA’s H100 chips and slowing down shipments. Microsoft’s in-house chip is similar to NVIDIA GPUs and is designed for data center servers for training and running large language models.
Microsoft initiated its in-house AI chip project, codenamed Athena, back in 2019. However, Microsoft isn’t looking to completely replace NVIDIA chips but rather to lower costs and incorporate AI capabilities into various services.
Before Microsoft, both Amazon and Google had already introduced in-house AI chips. Among NVIDIA’s major customers, six have already begun developing their chips. Everyone wants to avoid over-reliance on a single supplier, as it can make them too vulnerable.
02
Mobile Giants Venturing into In-House GPU Development
Mobile giants are also looking to reduce their reliance on GPU chips and IP suppliers, with notable examples being Samsung and Apple.
Recently, Samsung introduced the Exynos 2400 mobile processor, featuring the Xclipse 940 GPU, which utilizes AMD’s latest RDNA 3 architecture. This GPU stands out with enhanced hardware ray tracing capabilities, a technology becoming increasingly important in today’s mobile gaming, making it a standard feature in high-performance products.
To develop their in-house GPU, Samsung began collaborating with AMD in 2019. At that time, they signed their initial agreement, permitting Samsung to use AMD’s custom GPU IP based on the RDNA architecture for smartphones and other mobile devices. Since then, Samsung has incorporated AMD Radeon graphics solutions in their Exynos series processors.
Samsung’s move towards developing its own mobile GPU aims to reduce its dependency on Arm Mali GPU IP. For many years leading up to the release of the Exynos 2100, Samsung relied on Mali GPU IP. In early 2022, Samsung announced the new mobile processor, the Exynos 2200, featuring the Xclipse graphics processing unit based on AMD’s RDNA 2 architecture. However, this processor did not perform well upon release, as it was only included in certain Galaxy S22 series phones and underperformed in performance tests compared to Qualcomm’s Snapdragon 8 Gen1. The main reasons for this were design flaws and suboptimal optimization in the Xclipse GPU.
Following the Exynos 2200, Samsung introduced the Exynos 2300, but it did not enter mass production due to persistent design flaws.
In April of this year, Samsung and AMD signed a long-term agreement to strengthen their strategic partnership. Despite the setbacks with the previous two processors, Samsung remains committed to developing its mobile GPU. The recent launch of the Exynos 2400 mobile processor is a testament to this commitment.
However, developing an in-house mobile GPU is not an easy task. While Samsung is undeterred by past failures, the road ahead may be even more challenging, especially when partnering with AMD. This is because AMD’s RDNA architecture is primarily designed for PCs, and adapting it for mobile use presents significant challenges, especially in terms of power consumption. This requires deep collaboration between the two companies to optimize design and architecture, which is a complex task with no proven precedent. Intel faced similar issues when they attempted to produce mobile chips.
To expedite the optimization process, Samsung Electronics has been increasing its investments. In April 2022, Samsung Electronics Vice President Kim Tae-Hyun took charge of the GPU development team at the Samsung Austin Research Center’s Advanced Computing Lab in the United States. At the end of 2022, the Samsung Electronics System LSI division and MX division jointly formed an AP Solution Development Team to optimize AP and conduct research on the next-generation AP. In addition, Samsung has recruited key personnel from AMD and Arm and established an in-house AP optimization team while continuously expanding its R&D workforce. Since the beginning of 2023, Samsung’s U.S. subsidiary has been actively recruiting GPU development personnel, aiming to work closely with AMD to overcome the challenges of developing a mobile GPU.
Compared to Samsung, Apple entered the realm of in-house mobile GPU development earlier, but it wasn’t a straightforward journey.
In the earlier years, Apple consistently purchased Imagination’s PowerVR GPU IP. However, Apple decided to discontinue its use of the company’s IP to develop its in-house GPU. This move aimed to have more control over the core technology of its products, reduce costs, ensure high profits in the smartphone market, and prepare for future innovations, particularly in the areas of VR and AR applications.
To pursue in-house GPU development, Apple hired many engineers from Imagination. However, after approximately two years of research and development efforts, the project was declared a failure. Apple had to resume collaboration with Imagination by purchasing its IP to design its own GPU module for the A-series mobile processors.
Discussing in-house GPU development has two levels of meaning: firstly, the most comprehensive form where a company starts development from the ground up, including the fundamental IP layer, which is the most thorough but also the most challenging approach. Secondly, the approach currently employed by both Apple and Samsung involves purchasing GPU IP from partners and integrating it into their in-house mobile SoCs. This approach is relatively easier and more efficient but is less comprehensive compared to companies like Qualcomm.
For instance, Apple’s latest mobile processor, the A17 Pro, features an enhanced GPU as a major highlight. It increased the core count from 5 in the A16 to 6 in the A17 Pro, introduced a new architecture design, improved peak performance by 20%, introduced mesh shading, and added real-time hardware-level ray tracing support for the first time. This aligns Apple with the likes of Qualcomm and MediaTek.
Comparing the GPU performance of the A17 Pro and Qualcomm’s Snapdragon 8 Gen 2 using 3DMark Wildlife, the A17 Pro scored 11,860, while the Snapdragon 8 Gen 2 scored 13,500. This reflects a roughly 15% performance gap in favor of the Snapdragon 8 Gen 2. The reason for this difference is that Qualcomm’s in-house Adreno GPU, developed over several years with extensive technological refinement, is already highly sophisticated and mature. Samsung and Apple would need more time and accumulation to compete with Qualcomm in the GPU department.
In summary, mobile manufacturers developing in-house GPUs are not yet fully independent of traditional supply chains. Achieving complete self-sufficiency will require more time and effort.
03
Challenges Faced by Startup Companies in the GPU Space
and mobile GPU sectors, which have significant market influence and resources to explore new GPU development paths, providing them with a safety net even if they encounter setbacks. However, the situation is less optimistic for startup companies that have been developing new processors in recent years to compete in the AI server GPU market. Many of these companies have faced challenges and some have struggled to continue, primarily due to their processors failing to meet customer ecosystem requirements, making them difficult to sell.
Recently, the UK-based IC design company Graphcore, a major competitor to NVIDIA, submitted a statement to the UK Companies House, indicating that the company is in negotiations with potential investors but has not yet reached an agreement. The statement also revealed that the company’s pre-tax losses increased by 11% compared to the previous year, reaching $204.6 million. The company also mentioned that it had closed its offices in Norway, Japan, and South Korea in 2022, reducing its workforce from 631 employees in 2021 to 494 employees last year. Additionally, the company’s revenue decreased by 46% in 2022.
Graphcore was founded in 2016 and was once one of the most promising startups in the UK. The company’s IPU (Intelligence Processing Unit) products, which can be compared to NVIDIA’s GPUs, have been used on the Microsoft Azure platform.
04
China’s GPU Industry: Seizing Greater Opportunities
In recent years, driven by concerns about domestic industry chain security and commercial interests, Chinese GPU manufacturers have been making significant efforts, and some distinctive companies have emerged.
International GPU leaders like NVIDIA have already established a strong presence in the market. Chinese domestic manufacturers face various difficulties and challenges during product development, including patent issues and ecosystem compatibility. However, just as Loongson (龙芯) invested years in researching and fully understanding the MIPS architecture to develop their own instruction set and chips, with the determination and perseverance to work for a decade, the development of a self-contained GPU is a promising endeavor.
Not long ago, Loongson Zhongke announced that their in-house GPU IP core had been verified in mass-produced bridge chips and new chips. The first SoC (System on Chip) with Loongson’s in-house GPGPU (General-Purpose Graphics Processing Unit) is expected to tape out in the first half of 2024, supporting graphics acceleration, scientific computing, and AI calculations. By the second half of 2024, they plan to complete the tape-out of a dedicated chip that combines graphics and computational acceleration.
Jingjia Micro (景嘉微) is a well-established Chinese GPU company, pioneering the development of in-house GPUs that have been applied in large-scale engineering projects. Their JM5, JM7, and JM9 series of GPUs are examples, and they are expanding from commercial to civilian markets. In addition to Jingjia Micro, Haiguang’s DCU series, based on the GPGPU architecture, is compatible with the ROCm and CUDA ecosystems and can be applied in areas like big data processing and AI.
Emerging companies such as Biren Technology, Tiensu ZhiXin, Xindong Technology, Moxi Integrated Circuits, and Muxi Integrated Circuits have been gaining more attention.
In the domain of cloud GPGPU, Tiensu ZhiXin has risen as a local company in recent years. In 2021, the company introduced the 7nm fully in-house cloud training GPGPU “Tianyi 100” and, in 2022, released the 7nm cloud inference GPGPU “Zhikai 100.” These GPUs provide high computational power and energy efficiency for cloud AI training and HPC (High-Performance Computing) general-purpose computing.
Other companies have also introduced GPUs for data centers and AI in the past two to three years, such as Moxi Integrated Circuits’ MXC series, Biren Technology’s BR100, and Moxi Threads’ MTT S2000.
Compared to international AI server chip manufacturers, Chinese domestic GPU manufacturers have better growth and development prospects. Firstly, the Chinese market offers vast growth opportunities. Secondly, much of the competition from international giants like NVIDIA and AMD is somewhat blocked due to U.S. government policies. Lastly, Chinese domestic systems and internet giants do not have the same urgent need for in-house GPU development as companies like Microsoft, Amazon, and Google. After the capital rush subsides and market competition takes its toll, companies that deliver valuable products and maintain healthy cash flow will have a favorable development space.
Related:
- OpenAI Account Ban: ByteDance’s Statement
- Groq: The AI Phenom Igniting the SRAM Market Boom!
- Compare NVIDIA, AMD, Intel GPUs: Key Benefits & Summary
- Save Money: 4 Tips to Upgrade Old GPU Performance
- Win10 22H2 vs Win11 23H2: Gaming Performance Comparison
- NVIDIA RTX 3050A with AD106 GPU: What You Need to Know
Disclaimer: This article is created by the original author. The content of the article represents their personal opinions. Our reposting is for sharing and discussion purposes only and does not imply our endorsement or agreement. If you have any objections, please contact us through the provided channels.
