NVIDIA's two 2021 GTCs: three chips, one architecture, an annual refresh. Data center revenue grew 54.5%, and hyperscale is down to 30% of it from 50%.
We have followed this wave of the technology revolution since 2016, and three companies have been our main gauges of it: NVIDIA, TSMC, and Microsoft. They are the arms dealers of the revolution, the ones selling the tools rather than the end product, and each is far ahead in its own lane. Watching what they do, what they say about the future, and what their operating numbers show is usually enough to place where the industry sits in its build-out.
Key takeaways
- Three chips, one architecture, refreshed every year: the Grace ARM based CPU in early 2023 on TSMC 5nm, the BlueField-3 data processing unit (DPU) with 22 billion transistors and 16 ARM cores, and the next GPU after Ampere in 2022.
- Data center growth recovered to 54.5%, and the increase in absolute dollars was the largest the segment has posted outside the Mellanox acquisition. Quarterly data center revenue is now close to $3 billion.
- Hyperscale cloud customers are down to 30% of data center revenue, from 50% in the same quarter of 2020. The buyers are now spread across vertical industries.
- Professional visualization revenue rose to 1.5 times the year-earlier level, which is what it looks like when the workloads start moving from the cloud out to the endpoint.
Whether you call the current revolution AI, digital transformation, or the metaverse, the same thing sits under all three: data is driving every industry to change, and with it business models and business logic. NVIDIA holds a commanding position in that shift, which is why NVIDIA GTC is the one event we never skip. NVIDIA held two GTCs in 2021, in April and in November, and this post draws on both. We opened this Data Driven Tech Revolution series in April 2021 largely because of the run of products NVIDIA announced at the first of them. This installment is about the most important arms dealer under the enterprise digital transformation theme.
NVIDIA shows its firepower and locks in the full computing platform
At the April 2021 GTC, NVIDIA ran out an integrated set of hardware and software launches that made its position as the technology enabler for enterprises hard to miss. On hardware it moved from GPUs to what NVIDIA calls XPUs, meaning the whole set of accelerators (GPU, CPU and DPU), with three chips, one architecture, and an annual refresh cadence. On software it picked two end markets, the enterprise and autonomous vehicles, and shipped a full set of modular professional services aimed at the pain points each one has today.
Jensen Huang kept coming back to the same framing: NVIDIA is a full computing platform. The company uses the word platform more than any other chip company does.
The software platform: getting enterprises to actually use AI and the metaverse
NVIDIA sells enterprises virtual factories, speech systems, and recommender systems, all of it aimed at making operations more efficient.
- Already in production: BMW uses Omniverse to simulate every element of 31 factories. It can model real conditions quickly, adjust production lines, and run logistics with NVIDIA robots. The result was a 30% improvement in planning efficiency, and lower cost.
- NVIDIA Jarvis, the conversational AI framework: trained on millions of hours of speech and 1 billion web pages, with recognition accuracy up to 90%. At the November 2021 GTC the company demonstrated it handling realistic accents. Customers with specialized vocabulary can retrain it through NVIDIA TAO.
- NVIDIA Merlin, the recommender system: recommenders are the engine behind advertising, online shopping, movies, and user generated content. To speed them up, Merlin is available through NGC, NVIDIA's catalog of deep learning framework containers.
Natural language training has pushed model parameters 10 to 100 times higher than what came before. The point of shipping software services is to get applications into production faster. The more data gets used, and the more value users pull out of it, the more the complementary goods effect works in NVIDIA's favor: demand for its data center chips grows exponentially.
The enterprise metaverse, meaning Omniverse, is the same idea. A company can use the same software and the same models it uses in the real world to simulate a factory where robots and people work side by side, then put that design into production. The more enterprises adopt it, the more demand there is for high performance computing (HPC), because HPC is the complement. None of this appeared out of nowhere. NVIDIA launched Isaac, its autonomous machine platform for robotics, and NVIDIA DRIVE, its autonomous vehicle simulation environment, back in 2018, and those services were already being used to simulate factory builds. In 2021 the company simply put its existing virtual platform services under one name.
The hardware: products built around data, not around compute
A software platform that size needs serious compute underneath it, and the April GTC made NVIDIA's hardware plan clearer.
- Grace, the ARM based CPU: aimed at complex supercomputing work. NVIDIA says a Grace system paired with its own GPUs will deliver 10 times the performance of today's top x86 CPU based NVIDIA DGX systems. It is scheduled for early 2023, built on TSMC 5nm with the ARM v9 architecture. The company frames it as freeing the GPU to run at its full performance.
- BlueField-3, the DPU: accelerating data handling is the other half of raising usable compute. The new part is 10 times faster than BlueField-2, built from 22 billion transistors, with 16 ARM CPU cores. NVIDIA also released DOCA 1.0, the software development kit (SDK) for its data center architecture, so security and management applications can run on the DPU.
- Ampere GPU: our May 2020 post on NVIDIA's GTC keynote said the Ampere upgrade was the important one, the first major redesign in two years, with a large step up in performance. GPUs were not the focus this time.
- DGX supercomputers: natural language processing generates enormous training datasets, and that is one of the main jobs DGX is built for. NVIDIA also introduced the Megatron framework to help train transformers.
Huang put a number on the shift at GTC: of the 30 million data center servers shipped each year, one third go to software defined data centers, and the workloads on them are growing much faster than Moore's Law. A new era needs new chips, new system architectures, new networking, new software, and new tools. Without a way to accelerate, he argued, the compute available for real work keeps shrinking.
Those three product lines, GPU, DPU, and CPU, are the answer: three chips, one architecture, refreshed every year. That is NVIDIA's complete chip lineup for the new era.
The stated schedule is the next GPU after Ampere and the BlueField-3 DPU in 2022, the Grace CPU in 2023, the following GPU generation and BlueField-4 in 2024, and the next ARM based CPU in 2025.
Industry specific solutions: the two pain points in autonomous vehicles and the enterprise
Alongside the software and hardware updates, NVIDIA put out packaged solutions for its two priority applications.
- Drive AV for autonomous vehicles: an end to end AV platform covering chips, sensor architecture, map data, and data processing, delivered to automakers as a modular full system. The current partner is Mercedes-Benz. NVIDIA also introduced Orin, a dedicated AV chip that handles both road data and in car infotainment, scheduled for production in 2022. AV systems can be trained inside the NVIDIA Omniverse virtual environment. The next generation AV chip, Atlan, with 1,000 TOPS (trillions of operations per second) of compute, is planned for 2024.
- NVIDIA EGX for Enterprise: an enterprise grade cloud AI service, with NVIDIA AI accessible through VMware. The company also announced a computing platform designed for the edge, which shows how many enterprise scenarios NVIDIA has already mapped out.
NVIDIA's data center business is booming, and growth is clearly accelerating
All these applications push data volumes up, and the workloads are moving from the cloud out to the endpoint. NVIDIA's latest results, reported in November 2021, show the same thing.
The first highlight of the quarter was data center revenue, which jumped. Growth recovered to 54.5%, and the increase in absolute dollars was the largest the segment has posted outside the Mellanox acquisition.
NVIDIA broke data center out as its own revenue line in 2016, the same year NVLink first shipped in the Pascal architecture, and data center revenue started growing sharply from there. The customer base then was mostly hyperscale cloud buyers. Quarterly data center revenue is now close to $3 billion, and hyperscale customers are down to 30% of it, from 50% in the same quarter of 2020. Demand for data center products has broadened well beyond hyperscale deployments into a much wider set of vertical industries.
The second highlight was professional visualization, NVIDIA's workstation graphics segment, which kept climbing, to 1.5 times the year-earlier level. As more people collaborate in virtual spaces, workstation demand for professional graphics cards rises with it. On the earnings call, Huang made the same point: as applications multiply, demand spreads from the cloud out to the endpoint.
With data growing exponentially, compute is the horsepower of the new era
A data driven market means chips end up in a wide range of places, and at GTC NVIDIA keeps stressing that it works with customers on an open platform to grow the pie together: cloud game streaming with Amazon on AWS, a partnership with MediaTek to bring GPUs to ARM based PCs, AMD server CPUs inside NVIDIA's own DGX high performance computing systems, and a continuing relationship with Intel on endpoint PCs.
At the end of March 2021, ARM released the v9 architecture, its first new architecture in a decade, which should make the chip ecosystem more varied. The new architecture, described as the biggest technical change in ten years, is aimed at the fast growing server market, with stronger security and better AI compute.
ARM CEO Simon Segars said the future will be defined by AI, and before that arrives the industry has to get the compute foundation right for the challenges coming. Machine learning will be everywhere, internet of things (IoT) devices will be as common as phones, and more of that computing will run on ARM processors.
Every chip vendor is sharpening its tools for a digital era that needs far more compute. Across its two GTCs in 2021, from virtual software platforms to a year by year XPU hardware plan, NVIDIA showed how wide its moat in data centric computing has become.
Our view: NVIDIA stopped being a traditional hardware vendor some time ago. What it sells now is an integrated hardware and software computing platform. The role of the semiconductor vendor is changing, and that change is a strong push behind the progress of data driven computing. NVIDIA's coverage of the whole stack will keep driving this industrial revolution forward.