Home > AI

NVIDIA HGX H200 Tensor Core GPU – World’s most powerful AI superchip

The world's most powerful AI processor.

Reviewed By: Kevin Pocock

Last Updated on November 15, 2023
Is the NVIDIA HGX H200 the world's most powerful GPU?
You can trust PC Guide: Our team of experts use a combination of independent consumer research, in-depth testing where appropriate - which will be flagged as such, and market analysis when recommending products, software and services. Find out how we test here.

NVIDIA is powering the generative AI race with its high-performance Grace Hopper superchips. Todays multi-billion-parameter LLM’s (large language models) use vast amounts of data, processed with high bandwidth memory such as the NVIDIA GH200 Grace Hopper superchip. Now, the firm has announced a new NVIDIA HGX H200 Tensor Core GPU

What is the NVIDIA HGX H200 Tensor Core GPU?

The new AI superchip is being touted as the new “The World’s Most Powerful GPU“. That’s right, not just the most powerful for AI, but the most powerful full-stop!

NVIDIA’s HGX will accelerate inference times for the worlds most powerful LLM’s, such as OpenAI’s ChatGPT model GPT-4 Turbo, Google’s PaLM 2, and Meta’s open-source LLaMA 2. Today’s AI applications, particularly large language models, are trained on massive amounts of data. After all, it’s in the name!

The new NVIDIA H200 Tensor Core GPU “supercharges generative AI and high-performance computing (HPC) workloads with game-changing performance and memory capabilities.” All of this cutting-edge tech requires an equally cutting-edge bandwidth platform to handle never-before seen HPC applications (High-Performance Computing). NVIDIA CEO Jensen Huang’s latest hardware aims to solve this problem. “As the first GPU with HBM3e, the H200’s larger and faster memory fuels the acceleration of generative AI and large language models (LLMs) while advancing scientific computing for HPC workloads.”

Nvidia HGX H200 is a highly powerful server platform designed by NVIDIA.
NVIDIA HGX H200 performance results for modern LLM’s.

These performance leaps are come down to some very impressive advanced memory hardware. In fact, “the NVIDIA H200 is the first GPU to offer 141 gigabytes (GB) of HBM3e memory at 4.8 terabytes per second (TB/s) —that’s nearly double the capacity of the NVIDIA H100 Tensor Core GPU with 1.4X more memory bandwidth. As a result, “the H200’s larger and faster memory accelerates generative AI and LLMs, while advancing scientific computing for HPC workloads with better energy efficiency and lower total cost of ownership.”

Based on the same NVIDIA Hopper architecture we saw earlier this year, in the announcement of NVIDIA Grace CPUs. Its predecessor, the high-speed, high-bandwidth NVIDIA GH200 Grace Hopper superchip, was already the front-runner – making the firm its own biggest competitor. At this point, can anyone (or anyfirm) catch up?

Essential AI Tools

Editor’s pick
Only $0.00019 per word!

Content Guardian – AI Content Checker – One-click, Eight Checks

8 Market leading AI Content Checkers in ONE click. The only 8-in-1 AI content detector platform in the world. We integrate with leading AI content detectors to give unparalleled confidence that your content appear to be written by a human.
EXCLUSIVE DEAL 10,000 free bonus credits

Jasper AI

On-brand AI content wherever you create. 100,000+ customers creating real content with Jasper. One AI tool, all the best models.
TRY FOR FREE

WordAI

10x Your Content Output With AI. Key features – No duplicate content, full control, in built AI content checker. Free trial available.
TRY FOR FREE

Copy.ai

Experience the full power of an AI content generator that delivers premium results in seconds. 8 million users enjoy writing blogs 10x faster, effortlessly creating higher converting social media posts or writing more engaging emails. Sign up for a free trial.
TRY FOR FREE

Writesonic

Create SEO-optimized and plagiarism-free content for your blogs, ads, emails, and website 10X faster. Start for free. No credit card required.

The AI servers industry – Can anyone compete with NVIDIA’s deep learning data center?

Ian Buck, NVIDIA’s Vice President of Hyperscale and HPC is confident in the firms position.

The H200 boosts inference speed by up to 2X compared to H100 GPUs when handling LLMs like Llama2.

NVIDIA

With all the worlds giant-scale HPC providers Amazon Web Services (AWS), Google Cloud, Microsoft Azure, and Oracle Cloud Infrastructure set to deploy the new H200, competing with NVIDIA (now a trillion-dollar company, due to artificial intelligence) seems unlikely.

✓ Steve says

NVIDIA in 2nd place . . . to NVIDIA

I recently called NVIDIA the ‘key hardware manufacturer of the AI industry’. As the world’s newest trillion-dollar company, thanks to the doubling down of efforts into artificial intelligence, that claim grows ever more true. When all the worlds tech giants are defaulting to the same hardware manufacturer, as Elon Musk did with the purchase of 10,000 GPUs to launch xAI, it’s clear you’re doing something right. While it’s not widely reported which GPU model they were, I’d safely assume them to be a sweet deal on GH200 chips direct from CEO Jen-Hsun “Jensen” Huang.

This all represents a steadfast hold on the industry, with NVIDIA’s server manufacturers network set to deploy the new hardware over the course of 2024. A network that includes ASUS, Dell Technologies, GIGABYTE, Hewlett Packard Enterprise, Lenovo, Supermicro and Wistron.

What’s the best GPU for AI?

The NVIDIA GHX H200 GPU is the new best hardware for AI and HPC applications. However, as most of us can’t afford the 5-figure price tag, any modern 4000 series NVIDIA GPU will perform well enough for hobbyist or semi-professional LLM training purposes.

Steve is an AI Content Writer for PC Guide, writing about all things artificial intelligence. He currently leads the AI reviews on the website.

'),b.insertAdjacentHTML(d,e));return document.getElementById(a)};c.prototype._repeatDynamicAds=function(b){var a= [],d=this._removeRecipeCapExperiment.result&&"Recipe"===b.location?99:this.locationMaxLazySequence.get(b.location),e=b.lazy?u(d,0):0;d=b.max;var f=b.lazyMax;e=Math.max(d,0===e&&b.lazy?d+f:Math.min(Math.max(e-b.sequence+1,0),d+f));for(f=0;f=d;a.push(y(y({},b),{sequence:g,lazy:h}))}return a};c.prototype._locationEnabled=function(b){b=this._clsOptions.enabledLocations.includes(b.location);var a=this._clsOptions.disableAds&&this._clsOptions.disableAds.all|| document.body.classList.contains("adthrive-disable-all"),d=!document.body.classList.contains("adthrive-disable-content")&&!this._clsOptions.disableAds.reasons.has("content_plugin");return b&&!a&&d};return c}(),Za=function(c){function b(a,d){var e=c.call(this,a,"ClsVideoInsertion")||this;e._videoConfig=a;e._clsOptions=d;e._IN_POST_SELECTOR=".adthrive-video-player";e._WRAPPER_BAR_HEIGHT=36;e._playersAddedFromPlugin=[];d.removeVideoTitleWrapper&&(e._WRAPPER_BAR_HEIGHT=0);return e}v(b,c);b.prototype.init= function(){this._initializePlayers()};b.prototype._wrapJWPlayerWithCLS=function(a,d,e){void 0===e&&(e=0);return a.parentNode?(d=this._createGenericCLSWrapper(.5625*a.offsetWidth,d,e),a.parentNode.insertBefore(d,a),d.appendChild(a),d):null};b.prototype._createGenericCLSWrapper=function(a,d,e){var f=document.createElement("div");f.id="cls-video-container-".concat(d);f.className="adthrive";f.style.minHeight="".concat(a+e,"px");return f};b.prototype._getTitleHeight=function(a){a.innerText="Title";a.style.visibility= "hidden";document.body.appendChild(a);var d=window.getComputedStyle(a),e=parseInt(d.height,10),f=parseInt(d.marginTop,10);d=parseInt(d.marginBottom,10);document.body.removeChild(a);return Math.min(e+d+f,50)};b.prototype._initializePlayers=function(){var a=document.querySelectorAll(this._IN_POST_SELECTOR);a.length&&this._initializeRelatedPlayers(a);this._shouldRunAutoplayPlayers()&&this._determineAutoplayPlayers()};b.prototype._createStationaryRelatedPlayer=function(a,d,e){var f="mobile"===this._device? [400,225]:[640,360],g=G.Video_In_Post_ClicktoPlay_SoundOn;d&&a.mediaOrPlaylistId&&(e="".concat(a.mediaOrPlaylistId,"_").concat(e),d=this._wrapJWPlayerWithCLS(d,e),this._playersAddedFromPlugin.push(a.mediaOrPlaylistId),d&&this._clsOptions.setInjectedVideoSlots({playerId:a.playerId,playerName:g,playerSize:f,element:d,type:"stationaryRelated"}))};b.prototype._createStickyRelatedPlayer=function(a,d){var e="mobile"===this._device?[400,225]:[640,360],f=G.Video_Individual_Autoplay_SOff;this._stickyRelatedOnPage= !0;this._videoConfig.mobileStickyPlayerOnPage="mobile"===this._device;if(d&&a.position&&a.mediaId){var g=document.createElement("div");d.insertAdjacentElement(a.position,g);d=document.createElement("h3");d.style.margin="10px 0";d=this._getTitleHeight(d);d=this._wrapJWPlayerWithCLS(g,a.mediaId,this._WRAPPER_BAR_HEIGHT+d);this._playersAddedFromPlugin.push(a.mediaId);d&&this._clsOptions.setInjectedVideoSlots({playlistId:a.playlistId,playerId:a.playerId,playerSize:e,playerName:f,element:g,type:"stickyRelated"})}}; b.prototype._createPlaylistPlayer=function(a,d){var e=a.playlistId,f="mobile"===this._device?G.Video_Coll_SOff_Smartphone:G.Video_Collapse_Autoplay_SoundOff,g="mobile"===this._device?[400,225]:[640,360];this._videoConfig.mobileStickyPlayerOnPage=!0;var h=document.createElement("div");d.insertAdjacentElement(a.position,h);d=this._wrapJWPlayerWithCLS(h,e,this._WRAPPER_BAR_HEIGHT);this._playersAddedFromPlugin.push("playlist-".concat(e));d&&this._clsOptions.setInjectedVideoSlots({playlistId:a.playlistId, playerId:a.playerId,playerSize:g,playerName:f,element:h,type:"stickyPlaylist"})};b.prototype._isVideoAllowedOnPage=function(){var a=this._clsOptions.disableAds;if(a&&a.video){var d="";a.reasons.has("video_tag")?d="video tag":a.reasons.has("video_plugin")?d="video plugin":a.reasons.has("video_page")&&(d="command queue");q.error(d?"ClsVideoInsertionMigrated":"ClsVideoInsertion","isVideoAllowedOnPage",Error("DBP: Disabled by publisher via ".concat(d||"other")));return!1}return this._clsOptions.videoDisabledFromPlugin? !1:!0};return b}(function(c){function b(a,d){var e=c.call(this)||this;e._videoConfig=a;e._component=d;e._stickyRelatedOnPage=!1;e._relatedMediaIds=[];a=void 0;void 0===a&&(a=navigator.userAgent);a=/Windows NT|Macintosh/i.test(a);e._device=a?"desktop":"mobile";e._potentialPlayerMap=e.setPotentialPlayersMap();return e}v(b,c);b.prototype.setPotentialPlayersMap=function(){var a=this._videoConfig.players||[],d=this._filterPlayerMap();a=a.filter(function(e){return"stationaryRelated"===e.type&&e.enabled}); d.stationaryRelated=a;return this._potentialPlayerMap=d};b.prototype._filterPlayerMap=function(){var a=this,d=this._videoConfig.players,e={stickyRelated:[],stickyPlaylist:[],stationaryRelated:[]};return d&&d.length?d.filter(function(f){var g;return null===(g=f.devices)||void 0===g?void 0:g.includes(a._device)}).reduce(function(f,g){f[g.type]||(q.event(a._component,"constructor","Unknown Video Player Type detected",g.type),f[g.type]=[]);g.enabled&&f[g.type].push(g);return f},e):e};b.prototype._checkPlayerSelectorOnPage= function(a){var d=this;a=this._potentialPlayerMap[a].map(function(e){return{player:e,playerElement:d._getPlacementElement(e)}});return a.length?a[0]:{player:null,playerElement:null}};b.prototype._getOverrideElement=function(a,d,e){a&&d?(e=document.createElement("div"),d.insertAdjacentElement(a.position,e)):(d=this._checkPlayerSelectorOnPage("stickyPlaylist"),a=d.player,d=d.playerElement,a&&d&&(e=document.createElement("div"),d.insertAdjacentElement(a.position,e)));return e};b.prototype._shouldOverrideElement= function(a){a=a.getAttribute("override-embed");return"true"===a||"false"===a?"true"===a:this._videoConfig.relatedSettings?this._videoConfig.relatedSettings.overrideEmbedLocation:!1};b.prototype._checkPageSelector=function(a,d,e){void 0===e&&(e=[]);return a&&d&&0===e.length?("/"!==window.location.pathname&&q.event("VideoUtils","getPlacementElement",Error("PSNF: ".concat(a," does not exist on the page"))),!1):!0};b.prototype._getElementSelector=function(a,d,e){if(d&&d.length>e)return d[e];q.event("VideoUtils", "getPlacementElement",Error("ESNF: ".concat(a," does not exist on the page")));return null};b.prototype._getPlacementElement=function(a){var d=a.pageSelector,e=a.elementSelector;a=a.skip;var f=da(d),g=f.valid,h=f.elements;f=S(f,["valid","elements"]);var k=ca(e),l=k.valid,p=k.elements;k=S(k,["valid","elements"]);return""===d||g?l?this._checkPageSelector(d,g,h)?this._getElementSelector(e,p,a)||null:null:(q.error("VideoUtils","getPlacementElement",Error("".concat(e," is not a valid selector")),k),null): (q.error("VideoUtils","getPlacementElement",Error("".concat(d," is not a valid selector")),f),null)};b.prototype._getEmbeddedPlayerType=function(a){(a=a.getAttribute("data-player-type"))&&"default"!==a||(a=this._videoConfig.relatedSettings?this._videoConfig.relatedSettings.defaultPlayerType:"static");this._stickyRelatedOnPage&&(a="static");return a};b.prototype._getMediaId=function(a){return(a=a.getAttribute("data-video-id"))?(this._relatedMediaIds.push(a),a):!1};b.prototype._createRelatedPlayer= function(a,d,e,f){"collapse"===d?this._createCollapsePlayer(a,e):"static"===d&&this._createStaticPlayer(a,e,f)};b.prototype._createCollapsePlayer=function(a,d){var e=this._checkPlayerSelectorOnPage("stickyRelated"),f=e.player;e=e.playerElement;var g=f?f:this._potentialPlayerMap.stationaryRelated[0];g&&g.playerId?(this._shouldOverrideElement(d)&&(d=this._getOverrideElement(f,e,d)),d=document.querySelector("#cls-video-container-".concat(a," > div"))||d,this._createStickyRelatedPlayer(y(y({},g),{mediaId:a}), d)):q.error(this._component,"_createCollapsePlayer","No video player found")};b.prototype._createStaticPlayer=function(a,d,e){this._potentialPlayerMap.stationaryRelated.length&&this._potentialPlayerMap.stationaryRelated[0].playerId?this._createStationaryRelatedPlayer(y(y({},this._potentialPlayerMap.stationaryRelated[0]),{mediaOrPlaylistId:a}),d,e):q.error(this._component,"_createStaticPlayer","No video player found")};b.prototype._shouldRunAutoplayPlayers=function(){return this._isVideoAllowedOnPage()&& (this._potentialPlayerMap.stickyRelated.length||this._potentialPlayerMap.stickyPlaylist.length)?!0:!1};b.prototype._determineAutoplayPlayers=function(){var a=this._component,d="VideoManagerComponent"===a,e=this._config;if(this._stickyRelatedOnPage)q.event(a,"stickyRelatedOnPage",d&&{device:e&&e.context.device,isDesktop:this._device}||{});else{var f=this._checkPlayerSelectorOnPage("stickyPlaylist"),g=f.player;f=f.playerElement;g&&g.playerId&&g.playlistId&&f?this._createPlaylistPlayer(g,f):q.event(a, "noStickyPlaylist",d&&{vendor:"none",device:e&&e.context.device,isDesktop:this._device}||{})}};b.prototype._initializeRelatedPlayers=function(a){for(var d=new Map,e=0;e