Meta unveils its latest customized AI chip because it races to catch up

Meta, hell-bent on catching as much as rivals within the generative AI area, is spending billions by itself AI efforts. A portion of these billions goes towards recruiting AI researchers. However a fair bigger chunk is being spent creating {hardware}, particularly chips to run and prepare Meta’s AI fashions.

Meta unveiled the latest fruit of its chip dev efforts right now, conspicuously a day after Intel announced its newest AI accelerator {hardware}. Referred to as the “next-gen” Meta Coaching and Inference Accelerator (MTIA), the successor to last year’s MTIA v1, the chip runs fashions together with for rating and recommending show adverts on Meta’s properties (e.g. Fb).

In comparison with MTIA v1, which was constructed on a 7nm course of, the next-gen MTIA is 5nm. (In chip manufacturing, “process” refers back to the measurement of the smallest part that may be constructed on the chip.) The following-gen MTIA is a bodily bigger design, full of extra processing cores than its predecessor. And whereas it consumes extra energy — 90W versus 25W — it additionally boasts extra inside reminiscence (128MB versus 64MB) and runs at a better common clock pace (1.35GHz up from 800MHz).

Meta says the next-gen MTIA is at present stay in 16 of its knowledge heart areas and delivering as much as 3x general higher efficiency in comparison with MTIA v1. If that “3x” declare sounds a bit imprecise, you’re not mistaken — we thought so too. However Meta would solely volunteer that the determine got here from testing the efficiency of “four key models” throughout each chips.

“Because we control the whole stack, we can achieve greater efficiency compared to commercially available GPUs,” Meta writes in a weblog put up shared with TechCrunch.

Meta’s {hardware} showcase — which comes a mere 24 hours after a press briefing on the company’s various ongoing generative AI initiatives — is uncommon for a number of causes.

One, Meta reveals within the blog post that it’s not utilizing the next-gen MTIA for generative AI coaching workloads in the intervening time, though the corporate claims it has “several programs underway” exploring this. Two, Meta admits that the next-gen MTIA gained’t change GPUs for working or coaching fashions — however as an alternative will complement them.

Studying between the strains, Meta is transferring slowly — maybe extra slowly than it’d like.

Meta’s AI groups are virtually definitely below stress to chop prices. The corporate’s set to spend an estimated $18 billion by the top of 2024 on GPUs for coaching and working generative AI fashions, and — with coaching prices for cutting-edge generative fashions ranging within the tens of hundreds of thousands of {dollars} — in-house {hardware} presents a pretty various.

And whereas Meta’s {hardware} drags, rivals are pulling forward, a lot to the consternation of Meta’s management, I’d suspect.

Google this week made its fifth-generation customized chip for coaching AI fashions, TPU v5p, typically accessible to Google Cloud clients, and revealed its first devoted chip for working fashions, Axion. Amazon has several custom AI chip households below its belt. And Microsoft final yr jumped into the fray with the Azure Maia AI Accelerator and the Azure Cobalt 100 CPU.

Within the blog post, Meta says it took fewer than 9 months to “go from first silicon to production models” of the next-gen MTIA, which to be honest is shorter than the standard window between Google TPUs. However Meta has lots of catching as much as do if it hopes to realize a measure of independence from third-party GPUs — and match its stiff competitors.