Design cost hits $650 million at 3nm. Chiplet packaging cuts processor cost by more than half, and $82.57 billion of 2020 cloud capex pays for it.
In After Moore's Law, TSMC's 3D Packaging Builds an Absolute Lead, Part 1: The 3DFabric Platform we argued that 3D packaging is where semiconductor progress goes next, and that the industry has to stop describing technology by nanometer node. The follow-on is vertical integration, with TSMC providing one stop service from front end design to back end packaging and pulling far ahead of its competitors on technology. The standard pushback is that advanced nodes and advanced packaging cost so much to develop that not many customers will buy. This post takes the cost side and the demand side and explains why we are not worried about it at all.
Key takeaways
- Development cost per node has run away: IBS (International Business Strategies) estimates put a 16nm design at $100 million and a 3nm design at $650 million.
- Advanced packaging is the answer to that cost, not another line item on top of it. AMD said at ISSCC, a chip industry technical conference, in early 2020 that chiplet system in package can cut processor manufacturing cost by more than half in some cases, and half of TSMC's customers at 7nm and below already use TSMC advanced packaging.
- The buyer has moved from the consumer to the enterprise. Capex at the four largest US data center spenders we track grows an estimated 19.6% to $82.57 billion in 2020 and another 6.6% to $88 billion in 2021, and data center is now the best line at the chip vendors: about 30% growth year over year, more than 30% of revenue, and gross margin roughly 10 points above their other products.
- Even if Intel's 7nm reaches volume production between late 2022 and early 2023 and catches up on scaling, performance and power no longer come from scaling alone, and TSMC's valuation today does not reflect the lead it holds.
Advanced packaging is expensive. Who pays for it?
IBS estimates reported in the press put the development cost of a design at $100 million at 16nm, $174 million at 10nm, $300 million at 7nm, $436 million at 5nm and $650 million at 3nm.
Fewer and fewer companies can fund numbers like that, which is why the market keeps worrying about demand.
But the expense of the leading edge is exactly the reason advanced packaging lowers the customer's cost rather than raising it.
AMD's own framing at ISSCC in early 2020: in some cases, chiplet system in package technology can cut processor manufacturing cost by more than half.
A chiplet approach splits a large multicore design into small individual dies at the design stage, then stacks them and assembles them into a single chip in packaging. Each die can sit on whichever node suits it, so not everything has to run on the most advanced process, and smaller dies yield better.
So using advanced packaging alongside an advanced node brings total cost down. Half of TSMC's customers at 7nm and below already use TSMC advanced packaging.
Leading edge silicon does keep getting more expensive with every node, so the question that decides everything is where the end demand sits.
Data center build outs are the biggest driver: what consumers cannot afford, businesses buy
The industry has changed a lot over the past few years, and the rise of outsourcing is the biggest reason. It has changed daily habits and it has changed how companies compete.
- Streaming, which Netflix started: the DVD a consumer used to rent turned into a vendor's data center holding the content and serving it.
- Public cloud, which AWS started: the computer a consumer used to buy turned into a subscription, with the vendor carrying the silicon and adding services on top of it.
- Business to business (B2B) software as a service (SaaS): information technology, human resources, finance, manufacturing and procurement all get outsourced, so standing up a company or a new product line takes less capital than it used to.
- TSMC is the same move inside the semiconductor supply chain. Foundry lets designers develop their own silicon without building a fab, while the expensive advanced node investment and packaging development stay with TSMC and reach customers through an open platform.
Widespread outsourcing has created a new set of leaders: the internet era technology giants, the cloud service and SaaS companies. And those new leaders moved demand for advanced node silicon from the consumer to the enterprise, from business to consumer (B2C) to B2B.
Our data center capex tracking shows exactly that shift. Capital spending at the four largest US data center spenders (Microsoft, Amazon, Google, Facebook) is still rising steadily: we estimate 2020 growth of 19.6% to $82.57 billion, and another 6.6% in 2021 to $88 billion.
On top of that, data demand created by AI algorithms is growing faster than hardware capability is improving. That makes high performance computing (HPC) chips the most likely place to adopt advanced packaging integration in order to get more out of the hardware.
NVIDIA's recent earnings calls have all stressed a change in the character of the data center business:
- From mostly public cloud to a roughly even split between public cloud and industry customers.
- From mostly training to a rising share of inference at the endpoint.
Data center demand keeps broadening. From the traditional public cloud out to enterprise and telecom, compute keeps moving toward the endpoint.
So the operator's problem is simple: hold down the cost of the build, and hold down the electricity bill, the largest single item in operating expense.
The HPC problem: raise performance and cut power
Delta Electronics, the Taiwanese power and cooling supplier, estimates that over a data center life of at least 15 years, 75% of operating expense goes to electricity, three to five times the original investment.
So saving power is the core question in every data center plan.
A large data center needs a full plan before it is built: power supply, power distribution, cooling and floor space, tied into an energy management system that tracks operations at every stage. If the power draw of the HPC chips themselves comes down sharply, that saves the operator money too.
As Part 1 argued, the point of pushing both process and packaging forward is to raise compute while cutting energy use, which is another way of saying raise efficiency.
That is why HPC chips built for the data center mostly use TSMC advanced nodes and 3D packaging:
- The second generation EPYC server CPU TSMC makes for AMD on 7nm delivers more than twice the compute of the prior 14nm generation, with power down 50%.
- The 7nm A100 TSMC makes for NVIDIA, the data center supercomputer part, lifts compute so much that, by NVIDIA's own launch math, server build cost is one tenth of the prior generation and power draw one twentieth, in far less rack space. CEO Jensen Huang kept repeating the same line at launch: the more you buy, the more you save.
So advanced node silicon keeps getting more expensive, but as long as compute performance and power efficiency keep improving, it is the better value for a data center operator. Same logic as paying up for an efficient air conditioner and getting it back on the power bill.
At this year's semiconductor forum, TSMC chairman Mark Liu looked back at how 7nm improved chip performance and power for MediaTek, AMD, NVIDIA and Xilinx, and said that lowering power consumption now matters as much as raising compute performance.
You can see it in the revenue lines too. At the major chip vendors we follow, data center is the main growth product line today, growing about 30% a year, more than 30% of revenue, and carrying gross margin roughly 10 points better than their other lines.
So pushing more energy efficient products into the data center to take share is work every HPC vendor has to do. And products like that can only be built by chasing advanced nodes and 3D packaging.
We know the pandemic sped up technology penetration, and more companies are moving to the cloud. Capex at the large cloud companies was revised higher in the most recent quarter, against the macro, which says demand for data center build outs held up strongly through the pandemic.
In that setting AMD and NVIDIA, which are getting new products out on schedule, keep taking share. If Intel's product schedule slips further, it clearly speeds up its competitors' share gains again.
That competition could get considerably hotter as soon as the fourth quarter of 2020.
Bottom line: from now to 2023 is when TSMC's position gets settled
On the demand side we agree with what TSMC chairman Mark Liu said: the market's appetite for better compute performance for AI and 5G is never satisfied, and technical innovation will keep pushing energy efficiency forward. Energy efficiency should keep doubling every two years, with no sign of stopping.
On Intel's own 7nm plan, if volume production arrives between the end of 2022 and the start of 2023, Intel could catch back up with TSMC on scaling. But performance and power no longer come from scaling alone. Only a vendor holding integrated 3D packaging technology can put out the better product.
So TSMC's future is not only about advanced nodes. Once advanced packaging takes on a bigger role, what decides the valuation and the standing is whether TSMC can lead vertical integration up and down the semiconductor chain.
Our view: the market spends the next two years arguing about the competitive shift, then re-rates the winner.
And TSMC's valuation today still does not fully reflect the absolute lead it holds.
We will still go through one clear inventory correction before 2023. But our work on semiconductor inventory says that at least through the second quarter of 2021 we are still in the bull leg of the semiconductor cycle, so the inventory correction is not a near term worry.