Who will innovate the next wave of AI semiconductors?

Investment in Artificial Intelligence (AI) within the semiconductor industry is rapidly increasing, mainly driven by GPU-based AI chips. As custom AI semiconductors emerge, the question arises: Who will take the lead?

The answer is complex, as workloads shift from training to inference – that is, using AI models to get work done. With nearly half (49%) of technology leaders confirming AI is "fully integrated" into their core business strategies (PwC Pulse Survey, October 2024), the focus shifts to tailored semiconductor technologies. These chips, designed for efficiency, scalability, and specific customer needs, are driving the next wave of AI applications and redefining industry standards.

Unlike general-purpose chips, AI chips are designed to handle the unique demands of artificial intelligence workloads being optimized for parallel processing. By leveraging low-precision arithmetic and faster memory access, they execute the repetitive computations required by AI algorithms rapidly and effectively. This makes them indispensable for real-time applications, from large-scale training in data centers to edge-based inference in robots, Internet of Things (IoT) systems, and smart devices.

AI Chips can be categorized into training and inference types based on their main purpose. Training chips are used to train AI models by processing large datasets to optimize model parameters while balancing speed and accuracy. These chips prioritize raw computational power and scalability, enabling them to handle increasingly large and complex workloads required for developing AI models. Once a model is trained, an inference chip can be used to make predictions or decisions on new, unseen data in real time or in batch mode. Inference chips are designed for efficiency, focusing on low latency, low power consumption, and cost-effectiveness. Their compact design makes them ideal for deployment in edge and cloud environments.

The demand for such specialized chips is growing rapidly as AI services become increasingly complex and diverse. 

This transformation is fundamentally impacting AI chip development. The future lies in the ability to develop state of the art AI chips for specific applications - be it by reducing costs, enabling new application possibilities or efficiently leveraging supply chains. This development not only opens a competitive advantage but will become a decisive factor for companies that want to be successful in the dynamic world of AI technology in the long term.

A key challenge lies in the rising adoption costs of AI chips. Supply chain disruptions and surging demand have driven up production costs for GPUs, which have been the backbone of AI infrastructure. Escalating costs affect not only procurement but also exacerbate the scalability challenges of AI services.

A comparison highlights the difference: While conventional GPU based AI chips remain indispensable for AI training due to their flexibility, they are often overpowered and energy-intensive for inference phases. Customized AI chips, designed for specific workloads, provide a cost-efficient alternative from 40~60%.

Compounding the issue are the exponentially increasing operational costs associated with the growing use of AI services. Energy consumption is a particularly critical factor here: while high-performance GPU based AI Chip are renowned for computational capacity, they often exhibit higher energy consumption compared to dedicated inference-customized chips. AI inference chips, specifically optimized for certain AI tasks, achieve up to 50% greater power efficiency, offering the potential to save 10~20% of the operational costs.

To manage costs in the long term, companies are increasingly turning to specialized inference solutions that significantly reduce both scaling costs and energy requirements. This trend is reflected in the growing demand for AI chips, which achieve a balance between performance and cost-effectiveness.

The supply chain for semiconductor production has fundamentally changed and enables the rapid development of customized AI chips for specific applications. Previously, integrated device manufacturers (IDMs) such as Intel or Samsung dominated the entire production chain - from design to manufacturing. However, this vertically integrated model required very high levels of capital investment, resulting in high barriers to entry and relatively few options for customers looking to buy chips customized to their specific needs.

This model has been replaced by a fragmented and specialized supply chain. Today, specialized players such as design houses, foundries (e.g. TSMC) and OSAT (Outsourced Semiconductor Assembly and Test) service providers each take over a part of the value chain.

Design houses play a central role in this new ecosystem, bridging the gap between design and production. They work closely with foundries and use advanced design tools to develop optimized chips for specific applications, such as AI inference at the edge. This close collaboration speeds up prototyping, reduces development costs and enables chips to be customized to end-user requirements. The specialized supply chain opens new opportunities for innovation, reduces costs and provides companies of all sizes with access to semiconductor development.

The rapid growth of artificial intelligence and the Internet of Things has given rise to two main applications: Cloud AI, which processes data centrally in large data centers, and Edge AI, which analyzes data directly at its source—on devices such as IoT systems, smartphones, or edge servers. Looking to differentiate in a competitive market, Edge AI offers a strategic advantage by enabling localized, efficient processing that complements centralized cloud solutions.

Applications such as autonomous vehicles that process data from sensors and cameras are dependent on immediate reactions. Edge computing enables the ultra-fast processing needed for safety and efficiency in such applications.

Smart glasses, wearable devices that integrate technology into eyeglasses to provide augmented reality or other digital features, leverage edge computing to process audio, video, and command inputs directly on the device. This enables real-time functionalities such as hands-free photography, navigation, and augmented reality. Another groundbreaking application are brain-computer interfaces (BCIs), systems that enable direct communication between the brain and external devices. Many companies are currently developing devices that interpret neural signals, enabling people with paralysis to control digital devices through their thoughts alone.

Edge computing complements the growing demands of AI and IoT by providing localized, efficient data processing that central systems cannot always achieve. Its ability to reduce latency and support real-time applications makes it a practical solution for scenarios requiring immediate responses and minimal power consumption. With continued advancements in customized AI chips, edge computing allows businesses to stay ahead in a rapidly evolving market by providing scalable, high-performance solutions for emerging applications.

AI is transforming the semiconductor industry and opening up opportunities for companies to develop customized AI chips to meet their specific needs. To succeed in this dynamic environment and secure competitive advantage, companies should set the following strategic priorities:

1. Customization Potential
   Understanding how much of an AI chip can be customized is essential for companies aiming to optimize performance and align hardware with specific application needs. Evaluating the degree of adaptability within current AI chips allows organizations to identify opportunities for tailored functionality, whether through advanced design modifications or the integration of workload-specific features.
2. Capability Internalization
   Building internal capabilities for AI chip development enables companies to reduce reliance on external resources and enhance strategic flexibility. This involves assessing existing expertise in areas like chip design and integration, identifying gaps, and establishing frameworks to internalize critical development processes that align with long-term innovation goals.. 
3. Strategic Partnership 
   Collaborating with design houses, foundries, and specialized providers such as OSAT services enables companies to access advanced design tools, state-of-the-art manufacturing technologies, and third-party assembly, testing, and packaging expertise. These partnerships accelerate the development of custom AI chips while fostering innovation through shared expertise. Integrating best-in-class components into chiplet designs and leveraging a flexible semiconductor network enhances supply chain efficiency and allows internal resources to focus on high-value innovation, resulting in highly optimized and scalable solutions.
4. Emerging Technologies
   Staying ahead in the competitive AI semiconductor market requires a continuous focus on investigating and adopting new technologies. From advanced materials and novel architectures to breakthrough fabrication techniques, identifying and assessing emerging innovations ensures that companies remain at the forefront of performance. By dedicating resources to exploring these technologies, organizations can position themselves to leverage new opportunities and maintain a competitive edge in the evolving market.

AI evolution opens massive opportunities for those who optimize AI semiconductors specifically to customer requirements, combining efficiency, scalability and innovation. The next big player will not only deliver performance, but also close the gap between customized technology and cost efficiency. Whoever master these challenges will dominate the market for the next generation of AI chips.