LOGO

In this issue, we explore NVIDIA, an American semiconductor manufacturer that has been in the news since its market capitalization briefly reached the $1 trillion mark at the end of May this year.

About GPUs manufactured by NVIDIA

NVIDIA manufactures GPUs (image processing semiconductors), which are chips designed for display functions such as video, image, and animation display, and have been used in gaming PCs to display images smoothly. In recent years, GPUs have come into the limelight as the bearers of advanced arithmetic processing in automated driving technology and cryptographic asset mining operations.

Now, demand for these GPUs is growing. The catalysts for this are “data centers” and “generative AI” represented by ChatGPT.

GPU, generative AI and data center

Until now, it was common for data centers to be equipped with only CPUs (Central Processing Units), but with the spread of AI, data centers are increasingly being equipped with GPUs in addition to CPUs. However, only about 10-20% of data centers are equipped with GPUs.

However, the situation will change with the spread of generative AI.

Generative AI, such as image generation and natural language generation, requires a process called “inference,” in which an AI model created through training is run to reach a conclusion. The inference process requires more computation than the learning process. Therefore, it is necessary to have a GPU that is also suitable for a large amount of computation.

It is clear that generative AI will be the primary information-generating task in most of the world’s data centers in the future, and that within another decade, most of the world’s data centers will be equipped with GPUs.

In NVIDIA’s most recent quarterly results (May-July), sales in the data center division more than doubled in just three months, even though shipments are not keeping up with demand due to a severe supply shortage. Analysts expect the division’s revenues to exceed $60 billion in the next fiscal year (ending January 31, 2025), more than four times last fiscal year’s (ending January 31, 2023).

Why does Nvidia have such a strong lead?

Background to NVIDIA’s near monopoly on the GPU market

NVIDIA was positioned to advance AI very early on. In 2006, NVIDIA announced CUDA, a programming language for developers to write applications for GPUs. CUDA became an important component for subsequent AI projects.

CUDA eventually grew to include 250 software libraries used by AI developers, and this breadth effectively made NVIDIA the go-to platform for AI developers.

CUDA protects NVIDIA as a competitive “dig” that rivals can never overcome. In a July conference call hosted by Bernstein Research, former NVIDIA Vice President Michael Douglas noted that software is the key to NVIDIA’s ability to pull away from its competitors. He predicted that much of the performance improvement of Envidia’s systems over the next few years “will be software-driven, not hardware-driven.”

The key to Nvidia’s monopoly was software development.

For the time being, NVIDIA remains strong in the market.

For the time being, the market for GPUs for data centers is expected to be almost exclusively dominated by NVIDIA.

Nevertheless, competition is likely to intensify. In addition to competition with semiconductor manufacturers such as Intel and AMD (Advanced Micro Devices) that already handle GPUs, giant IT companies such as Google, Amazon, and Meta are also beginning to develop their own AI semiconductors.

Along with the further evolution of generative AI and NVIDIA’s developments, we will also be keeping a close eye on other companies dealing with GPUs.

Demand for AI is growing rapidly due to advances in digital technology and the widespread use of smartphones, and this in turn is driving demand for data centers to store information servers for businesses and other organizations.

Why does the growing demand for AI require data centers in the first place?

Expanding use of AI and data centers

The digitalization of society, known as “DX” (Digital Transformation), is progressing. Businesses are increasingly using AI, with “ChatGPT,” in which AI answers questions in natural sentences, in the limelight, and an ever-increasing number of services requiring quick information processing, such as smartphone shopping sites, social networking sites (SNS), and game applications.

As the scale of AI use expands and demands become more diverse every day, companies need stable and secure data and system operations.

Data centers are specialized facilities where IT equipment, network equipment and servers are stored securely.Even when IT equipment should be managed in-house, there may be insufficient space and security measures. To solve these various problems, more and more companies are turning to data centers. Data centers are excellent for operating and managing IT equipment and can meet the requirements of various companies.

Massive adoption of AI has led to increased investment in data centers

Since November 2022, the mass adoption of AI, symbolized by the huge success of ChatGPT, has generated California gold rush-like interest. The investment market is once again taking advantage of the Silicon Valley investment zeitgeist to reward pioneering companies like NVidia, Google, and Microsoft that have earned first-mover advantage through extensive R&D efforts. Investors are naturally looking for the next group of beneficiaries, avoiding high-risk or loss-making companies.

NVidia’s Q1 2024 earnings call was a turning point for the AI supply chain, as it highlighted the recent phenomenal demand growth for Nvidia’s hardware, leading analysts to revise their full-year revenue estimates upward by about 40%. (Source: Refinitiv)

The company mentioned “data centers” more than 56 times when explaining its better-than-expected revenue outlook during its May 24, 2023 investor earnings call. It was clear that the company’s advanced GPUs are entirely dependent on a high-performance, secure, and stable data center environment.

Issues with AI-enabled data centers

AI-enabled data centers also have their problems. Machine learning and AI applications require a lot of power for HPC and GPU servers. These servers consume so much power that they cannot fit in a single rack. The high power consumption of HPC and GPU servers is uncontrollable, so the servers accumulate a lot of heat and need to be cooled properly. Therefore, AI-enabled data centers are further required to improve power-saving and air conditioning technologies.

Summary

In this issue, we have discussed the growing use of AI and the demand for data centers.

Data centers will continue to play an important role in providing consumers and businesses with new AI tools as a critical infrastructure for the digital economy. They will also be required to provide services that meet the technological trends and business needs of companies.

We hope that data center operators will ride the wave of DX by carefully assessing and discussing the issues involved.

The global market for data center liquid cooling is projected to reach US$2.6 billion to US$7.8 billion during the forecast period (2023-2028), at a compound annual growth rate (CAGR) of 24.84%.

According to the latest study, among the various regions of the world, Asia Pacific is expected to be the fastest growing region in the data center liquid cooling market during the forecast period. There is a growing awareness of the value of sustainable practices and green data centers in Asia Pacific.

What is a liquid cooling system?

In order to reduce greenhouse gas emissions from data centers, green data centers are being built using renewable energy and the development of energy efficient solutions is accelerating.
According to current estimates, data centers consume about 3% of the world’s total electricity, and nearly all of the energy used in data centers is related to cooling.

Liquid cooling systems use water or other liquids to directly cool servers and other equipment in data centers. These systems are more efficient than air-based cooling systems, but require specialized equipment and maintenance. Cooling is essential for data centers to ensure that equipment operates at optimal temperatures and avoids overheating that can lead to system failure and data loss.

Background to the expansion of the liquid cooling market

Advances in technology have made liquid cooling easier to maintain, easier to expand, and more affordable. As a result, liquid usage has been reduced by more than 15% in built-in hot and humid climate data centers and 80% in cooler regions. The energy dedicated to liquid cooling can be recycled to heat the building and water, and the application of advanced engineered coolants can effectively reduce air conditioning’s carbon dioxide emissions effectively.

The edge data center market is expected to expand significantly at an estimated compound annual growth rate (CAGR) of 22% from 2023 to 2031.

According to Research And Markets, this surge is attributed to the growing need for low-latency data processing and storage solutions. Edge data centers are in close proximity to end users, which speeds data processing and reduces network congestion.

The main drivers for this are the growing population, rising trends in smartphone usage, and the push for smart cities in Asia Pacific, especially in China, Japan, and India, which are fueling demand for edge data centers. In addition, the growing adoption of cloud services and the expansion of e-commerce are also contributing to the rapid market growth in Asia Pacific.

In India, Indian digital technology company Varanium Cloud announced in February that it will launch its second edge data center under its Hydra brand in Kudal, Sindhudurg, Maharashtra, India.

According to the company, the edge data center will be housed in a shipping container to enhance accessibility and enable effective data sharing and communication in even the most remote areas of the country. They are also small and portable, allowing them to be easily transported to any location in the country.

What is the Edge Data Center?

Edge data centers are data centers that are located in close proximity to each other to prevent network latency and enable low-latency processing.

The advantage of an edge data center is that it does not require the development of a wide-area communications network and provides strong security.

When connecting to an edge data center via the Internet or other means, the use of edge servers is effective. Placing the contents as close to the site as possible stabilizes the communication path and prevents delays by using a roundabout communication path.

Status of Edge Data Center Utilization

In general, edge data centers are favored by telecommunications companies because of their latency capabilities with end users, but they have also become essential in many other areas.

In the healthcare industry, electronic medical records have been increasingly adopted and vast amounts of patient data have been accumulated. Edge data centers are being utilized to store this vast amount of data and manage the information at a high level of security.

By utilizing edge data centers, appropriate medical care can be provided without communication delays.

In the financial industry, edge data centers enable smooth transactions without delays. More profit can be gained by processing large amounts of data and ensuring secure communications.

IoT devices, which are hardware programmed for specific applications and transmitting data, are also active subscribers, especially in time-critical scenarios. Even in manufacturing, where predictive maintenance exists, edge equipment can help improve inventory management efficiency.

Contributes to decentralized edge data center communications

The number of people teleworking has increased due to COVID19 and online communication, such as web conferencing, has become more common.

The ability to work remotely has led to an increase in communication traffic around the world. The increase in the number of companies offering subscriber services has also contributed to the increase in the scale of personal communication.

Against this backdrop, the expansion of data centers is evolving as communications are concentrated in large data centers. However, unipolarization of communication destinations can have a significant impact on social infrastructure in the event of a disaster.

Edge data centers are attracting attention, perhaps partly for the purpose of contributing to the decentralization of communications.

Toward the Expansion of the Edge Data Center Market

Edge computing is necessary for building a high-speed, latency-free communication environment. It is important to design a network that successfully combines cloud computing and edge computing.

Companies and service providers working on IoT will need to consider building a new infrastructure environment based on a good understanding of the characteristics of edge data centers.

The Digital Agency has announced a policy to ease the selection requirements for providers of government clouds (government clouds) shared by the government and local governments.

Government cloud refers to a common infrastructure of cloud services used by the government and local governments. Called the government cloud, the government has set a goal to make systems related to 20 mission-critical tasks handled by municipalities, such as taxes and national pensions, available on the government cloud by the end of fiscal 2025.

The current rules, which require a single company to meet approximately 330 requirements, will be revised to allow for participation by a coalition of companies. This may make it easier for domestic companies to enter government cloud services that rely on foreign capital.

Until now, there have never been many businesses that can satisfy the wide variety of selection requirements on their own. In the 2022 public offering, cloud services from US companies that meet technical requirements such as security and business continuity have demonstrated their presence. In addition to Amazon Web Services (AWS), there are only four companies: Google, Microsoft Japan, and Oracle Japan. selected.

Domestic companies are unable to meet the requirements due to the scale of their business and the nature of their services. Particular hurdles include building a system to provide operational support from system development, using multiple data centers, and providing a development environment where artificial intelligence (AI) can perform machine learning.

Only giant IT (information technology) companies, known as “hyperscalers” such as AWS and Google, are able to achieve this on their own.

The agency is expected to announce new selection requirements for government cloud providers and start public offerings as early as August. The selected businesses will be allowed to provide services jointly with other companies as long as they are responsible for core technologies such as data management and authentication while maintaining most of the current items in the new requirements.

The selection of a government cloud provider is expected to be decided in late October.

Background of Easing Selection Requirements

The background behind this easing of selection requirements is

“We should review the selection criteria for cloud service providers that store and provide government cloud services.

and “the selection criteria for cloud service providers that store and provide government cloud services should be reviewed. It is extremely likely that companies like Sakura Internet and Internet Initiative Japan will seize this new opportunity to enter the domestic cloud market.

Despite these amendments, however, the Digital Agency stated that the right of municipalities to select providers will be maintained. In other words, the changes that the amendments will bring to the actual selection process may only be limited.

Many cloud service providers are requesting joint participation in the government cloud by multiple companies. The results of the selection process after the requirements are eased are likely to attract much anticipation and attention.

In this issue, we will first look at the current status and future intentions of the data center business of Japanese companies.

NTT

The company plans to spend more than 1.5 trillion yen over the next five years to expand its data centers. India will be the place where NTT will increase the most, and they plan to increase the number from the current 12 to about 24 by fiscal 2025, where potential demand is expected due to the expansion of major overseas IT companies and population growth. NTT also wants to increase the number of locations in North America from 14 to 23.

Softbank

SoftBank Corp., in collaboration with U.S. semiconductor giant NVIDIA, aims to build a platform for generative artificial intelligence and 5G/6G applications and introduce it to new AI data centers in Japan.

This application is based on NVIDIA chip technology. And to reduce costs and improve energy efficiency, SoftBank plans to build data centers that can host generative AI and wireless applications on a multi-tenant common server platform.

Kansai Electric Power

Kansai Electric Power Company (KEPCO), in partnership with U.S. data center operator CyrusOne, has begun work on developing a data center in Japan with the ambitious goal of achieving a 900 MW operation. the CyrusOne KEP joint venture is a hyperscale platform company Focusing on the development of new data centers specifically tailored to meet demand, the joint venture aims to enhance resiliency, efficiency, and smart development in the industry by linking data center infrastructure with the broader power grid.

In this way, the data center industry of Japanese domestic companies seems to be invigorating and achieving positive growth.

So what is behind this?

Background of the activation of the data center industry

Behind this is the progress of digitization, such as generative AI (Artificial Intelligence). If we become a data-driven society in which decisions are made based on data, data will accumulate at an accelerated pace.

Akira Shimada, president of NTT, which is focusing on the data center business, said, “We want to develop semiconductors that use light (instead of electrons) after 30 years. We will invest 100 billion yen per year in research and development. As a start, we plan to begin manufacturing related components that use light after 2013. In addition to incorporating them into telecommunications equipment and servers, we also aim to apply them to more general electronic devices.” and suggests that the data center business is closely linked to semiconductor development.

Semiconductors that use light will consume far less electricity, which is in line with the times in terms of sustainability.

Junichi Miyagawa, President and CEO of Softbank, said, “We are entering an era of coexistence with AI, data processing, and rapidly increasing demand for electricity. We aim to provide next-generation social infrastructure to support a digitalized society in Japan.

Today, the development of generation AI (Artificial Intelligence) is remarkable. We may be at a turning point, upgrading the services we deploy regarding our growth strategy.