Home > AI

Meta to build open-source AGI with world’s biggest AI data centre

Meta is spending billions on AI hardware
Last Updated on January 19, 2024
Meta AI building world's largest artificial intelligence data center with 350,000 NVIDIA H100 GPUs.
PC Guide is reader-supported. When you buy through links on our site, we may earn an affiliate commission. Read More
You can trust PC Guide: Our team of experts use a combination of independent consumer research, in-depth testing where appropriate - which will be flagged as such, and market analysis when recommending products, software and services. Find out how we test here.

Meta has placed what may be the largest order yet for NVIDIA H100 GPUs — the most in-demand AI processors on Earth. A record-breaking 350,000 GPUs are allocated for Meta AI data centers, at a total cost in the billions of dollars. It could take up to a year to fulfill, but when 2025 rolls around Meta will be in a front-running position for the AI race. CEO Mark Zuckerberg’s goal of open-source AGI is now within reach.

Is Meta building the world’s biggest AI data center?

Meta has 350,000 NVIDIA H100 GPUs inbound, probably forming the largest AI data center in the world. To put this into perspective, each unit can cost around $35,000 – $40,000 if bought individually on the secondary market. The total cost to Meta is estimated to be roughly $10.5 billion. This is a ‘reasonable’ wholesale cost, roughly $3 billion less than if bought on the secondary consumer market.

For the low price of someone’s annual salary, you’ll get an 80GB GPU for PCIe5.0 capable of 26 teraFLOPS (FP64). Pulling the most impressive number, you’ll see a wildly different figure of 3,026 teraFLOPs. This figure is “with sparsity”, and specifically refers to FP8 tensor core performance of the standard PCIe model. The higher bandwidth H100 SXM model increases this figure to an almost unheard-of 3,958 teraFLOPS.

Some services will allow you to ‘rent out’ a H100 or H200, at an optimistic cost of $2/hour+. Given the extreme corporate demand for the processor, this is the only way that smaller businesses will be able to get their hands on one. Enterprise data centers like those of Meta AI will benefit from the NVIDIA AI Enterprise initiative. The social media giant is an NVIDIA HGX H100 Partner with NVIDIA-certified systems.

Essential AI Tools

Editor’s pick
Only $0.00019 per word!

Content Guardian – AI Content Checker – One-click, Eight Checks

8 Market leading AI Content Checkers in ONE click. The only 8-in-1 AI content detector platform in the world. We integrate with leading AI content detectors to give unparalleled confidence that your content appear to be written by a human.
EXCLUSIVE DEAL 10,000 free bonus credits

Jasper AI

On-brand AI content wherever you create. 100,000+ customers creating real content with Jasper. One AI tool, all the best models.
TRY FOR FREE

WordAI

10x Your Content Output With AI. Key features – No duplicate content, full control, in built AI content checker. Free trial available.
TRY FOR FREE

Copy.ai

Experience the full power of an AI content generator that delivers premium results in seconds. 8 million users enjoy writing blogs 10x faster, effortlessly creating higher converting social media posts or writing more engaging emails. Sign up for a free trial.
TRY FOR FREE

Writesonic

Create SEO-optimized and plagiarism-free content for your blogs, ads, emails, and website 10X faster. Start for free. No credit card required.

Meta is creating open-source AGI ‘for all’

Speaking via the Threads platform, CEO Mark Zuckerberg states that “it’s become clearer that the next generation of services requires building full general intelligence”. Here, the Facebook founder is referring to artificial general intelligence (AGI). This is a level of computer intelligence equal to human intelligence. Of course, computers are already much faster at processing data than we humans and can retrieve a far greater variety of information than any person could possibly remember.

This processing speed and information retrieval are not what defines AGI. ‘Full general intelligence’ is something that we have not yet achieved. It is the simulation of intelligence, thinking in ‘the same way’ as a human, with all its inferences and biases. An AGI will be proactive in telling us information that we don’t even know that we don’t know — something that traditional search engines cannot do.

It goes without saying that most businesses cannot afford to compete with a $10.5 billion hardware investment. This may even put them ahead of their leading competition — OpenAI and Microsoft — at least in terms of H100 acceleration.

When will LLaMA 3 be released?

The open-source AI most likely to emerge first from this world-leading hardware advantage could be LLaMA 3. The next generation of Meta AI models is expected to become available at some point this year, but there is no official release date for LLaMA 3 just yet. One thing’s for sure — 350,000 AI accelerators will bring this forward substantially.

Steve is an AI Content Writer for PC Guide, writing about all things artificial intelligence. He currently leads the AI reviews on the website.