AI Agents, NIMs Blueprints, Reasoning Models, and DGX Spark Supercomputers

Hey, welcome to the next wave podcast. I'm Matt Wolfe and in today's episode, we're talking about AI agents. I had the opportunity to go spend a few days with Nvidia out at the Nvidia GTC conference and in this episode, I'm going to deep dive with Amanda Saunders from Nvidia. She's been at the heart of the AI agent revolution that's happening right now while working with Nvidia, the company that's enabling the AI revolution. In this episode, we'll unpack exactly what agentic AI is and how it's reshaping industries from healthcare and telecom to sports coaching, as well as why that's just scratching the surface of what's possible. You'll also discover Nvidia's secret blueprint for easily building powerful AI agents, as well as get into some of the fears that people have around AI agents, like is an AI agent going to take your job and replace you? Yeah, we're going to go there. And after the interview, I'm going to share some cool clips from Nvidia's GTC where I got Bob Pettey, also from Nvidia, to break down exactly what Nvidia's DGX Spark is and how they're going to be putting AI supercomputers in normal people's homes. You can have an AI supercomputer in your home by this time next year. We're going to get into that in today's episode. So, without further do, here's my discussion with Amanda Saunders, followed by my tour of the DGX Spark with Bob Pettey. Enjoy. Let's talk about agentic AI cuz that's the hot topic, it seems like uh in 2025, everybody's saying this is the year of agentic AI. How would you define agentic AI? So, agents are digital employees that help us augment our work. And what's really important about agents and and agentic AI is that it can perceive. So, it sees the world around it, it sees the data that it has access to, it takes that information. It reasons about that information, it thinks about it, and then it can actually make actions based on that data. So, it can perceive, it can reason, and it can act. And that's what really sort of sets it apart from generative AI that we had in the past is being able to take those actions, whether that action is alerting a human or actually actively using a tool and and making you know, something happen. What do you think makes agentic AI like powerful right now? Why why not a year ago? Why not 2 years ago? Why is now the time everybody's talking about it? Well, it started with having to have the accelerated computing. You had to have the computing power to do that and that's a problem that Nvidia's been working on for 30 years. And finally, we've got to the place where we had the the computing systems, we also then needed the software systems. And so, it started with the open models that became available. I think Llama was a was a huge advancement and step forward. And more and more models have come out recently. Reasoning models, in fact, are one of the the big pieces that have sort of driven that and sort of added to that. Yeah, and then just the the full ecosystem of tools and things that are required. AI is one of those really interesting fields where the more people use it, the more they want to use more of it and they want to do more. And so, it's really spurring this incredibly fast growth that we're seeing, which is, you know, breakneck speed, but that's what's really driving us from, you know, the early onsets of generative AI where you're just spitting out answers into these worlds now where they're actually thinking things through and making really smart actions. Yeah, or in my case, I got into AI and now I just make content about it because I'm obsessed with it. It's all I ever want to talk about. Because, yeah, once you get into it, really that's all you want to do is is you you figure out what it can do, you learn where it's, you know, boundaries are, and then you want to do more and more and more. And as it continues to, you know, get better, you can do more and more things with it. can we walk through like a agentic workflow? Like give some examples of like this is the types of things an agent can do? Yeah, absolutely. I think, you know, some of the most basic things that we see from agents are really about being able to talk to data. I think that's probably the first thing that we see most people try to do is they have their own personal data, they connect an LLM to that data, and then they're able to ask questions. And I think, you know, more recently, they're able to ask more and deeper questions. Deep research has been a big topic. Um it's a big topic here at GTC. And the more information you can get out of that data and the more quickly you can query that, that's a pretty standard agent that we would see out there. Now, then they start to get a lot more complex. I think there's some really cool stuff going on in the the telecom space where they're actually using agents to improve the network. So, they can actually predict when there might be network outages coming. And they can make recommendations to the human employees who, you know, maybe you want to make these changes because maybe there's a large show in town, like GTC, and there's going to be a lot of a lot of traffic on that network. Um here are the recommended changes so that you can actually, you know, still provide the the service that you kind of thing happening at GTC right now? Are they are they using that kind of stuff? I I wish they would. Um I think I think the the agents are just starting to be developed, so maybe next year we'll start to actually see that, but it's definitely something um I think we're we can we're going to see more of. Um but yeah, so those I think are, you know, some, you know, varying levels of examples and then there's obviously hundreds more in the healthcare space, you know, helping nurses and doctors become more efficient because we know they need the help. Um to, you know, everyday people like you and I. I mean, I I use it for almost everything I do, whether it's my personal life or my work life. Are there any like agents that that might surprise people? Anything that's like really sort of like interesting that I don't know, I wouldn't have thought people would be using agents in that area. The most interesting one for me on the agent side is really as you started to get into video and other types of data sources. Like text, I think people have gotten a handle on. Um but adding sort of video understanding and things coming through it. Actually, one of the really cool use cases that we recently did was Jensen got to throw out the first pitch at a baseball game. I saw the clip of that. And and we used a video agent to be able to critique his I remember that. Yes. Which I just think is really cool. So, you can imagine for, you know, athletes, whether they're amateurs or professionals, being able to use that is really helpful. Yeah, yeah, I can see that. Like golfers and stuff like that. Analyze my swing, tell me what I'm doing wrong, stuff like that. probably too much for the AI on my swing, but you know, it still would be helpful to know. Yeah, yeah. Well, let's let's talk about the Nvidia blueprints cuz when I was at CES, that was a big topic during CES was the Nvidia blueprints. And to me, that's sort of like the the beginnings of agents, right? But they're sort of this pre-built agent. You could probably explain it better than I could, so Well, essentially, we tried to name them blueprints so that people would get the idea that these are reference architectures. And what they do is they take the different building blocks that Nvidia's offering to make these agents and it helps you with a recipe on how to put them together. So, it starts out with our NIMs microservices. And these are packaged, containerized, optimized models. So, all the leading open models that are out there in the market, we take them, we containerize them, we add a standard API call to them so that people, you know, developers out there can start building. And we we package those up. So, now you've got the models. Now, you need to connect that model to things, right? A model on its own only does so much. So, that's So, like a NIM would be like you've got a a video NIM that this one NIM can produce AI-generated video. This NIM could produce text-to-speech. This NIM could do speech-to-text. Exactly. Exactly. So, we actually have 100 NIMs that are now available that you can download and use. And they do they're 40 different uh sort of domains or modalities that these different NIMs can do. So, yeah, so you can take a bunch of them, piece them together. And so, when we built those, everyone was like, "This is great. We need the models, we need them to run really well on on, you know, our Nvidia GPUs." But then they said, "Well, but how do we build them into the next thing?" And that's where blueprints came So, they are the recipe that allows you to take the NIM, start with this recipe, put piece this together, and you'll get to an agent. And then what's even better is you can then customize them. So, you can, you know, add your own data sources, you can add your own pieces in. You can combine blueprints together. So, maybe you started with a chatbot and you wanted to add a digital human on the front end. We have a blueprint for both of those, you piece them together, and all of a sudden, you have a digital avatar who can, you know, talk to you with a full, you know, face and expressions and and natural language. voice if you wanted to, yeah. Absolutely. And you can you can start with pieces from Nvidia or you can start with pieces from the ecosystem. We try to make it really easy to piece it all together. Let's talk let's talk a little bit about like security. I know when it comes to AI agents, they could be used for both good and bad. What kind of things can we do to to sort of protect and secure and make sure that people aren't going and using these agents for, you know, bad actor type stuff? You know, what's really cool about AI is where it raises concerns or security challenges, it can actually also help answer them. So, there's actually some really cool um there's a there's a blueprint out there for container security that does a lot of those pieces. Uh but we also have other applications that can actually track activities and and make alerts based on detecting some anomaly. Right? So, that's tends to be how you recognize that something's doing something it maybe wasn't supposed to. Um and AIs are just really good at that because they can, you know, consume a lot of information. So, I think where, you know, AI potentially can raise some concerns, it's also in some ways the answer to addressing some of those concerns, which I think is really helpful. Right. So, like the the sort of like sci-fi movie scenario of like the AI rising up against us, I mean, is this something that people should be concerned about or is this something that that you feel there's pretty good guardrails in place for? I I I think you know the the builders of these apps need to make sure those guard rails are in place, but I think yes in general I think there the tools exist and then it's just about us as the you know application builders being smart about how we deploy them. You know in general I think agents are really powerful. They're incredible tools that help humans. But on their own they're not they're not about to to sort of go off the rails. It really takes the human to to take it there. So I think again the tools it is a tool. It is something that can you know work alongside you. But I don't see the rise of the machines quite yet. So along similar lines when it comes to like you know the I think one of the fears a lot of people have is like is an agent going to take my job. So you know where where what are your thoughts in that sort of realm? I think yeah I mean the best thing I've ever heard is an agent's not going to take your job but somebody using an agent might. And that's that's always the thing that I think it's right. No I think I think exactly that like we we as humans have the capacity to do so much more. There's just often so many hours in the day. And so we don't you know accomplish you know with with agents the work that we were trying to do and then just stop. Right. No it allows us to go and do more and try more. And so I think you know for me it's not about replacing your job it's about making you more effective at your job and being able to do more and and you know be more powerful. I think you know health care for me is one of those industries that this is you know driven out there. There's actually a great partner of ours in the mental health space and they're using agents to free up their time from all the administrative work that they have to do. We know their jobs are hard enough as it is. If we can free that time up it allows them to spend more time with their patients actually deeply understanding and working with them as opposed to worrying about calendars notes and scheduling. So I think those are great examples where it's no it's something there that's going to help your job it's not going to take your job. Yeah no I was actually talking I bumped into somebody on the street while I was walking yesterday and they had a very similar concept for their company. They they work with therapists and they create chatbots specifically for the therapist so that their customers can go and have the initial conversation with the chatbot but then the conversations get passed along to the therapist so the therapist can kind of, you know, detect when some bigger issue that they need to address comes up but it's not replacing the therapist it's just hey I can now handle more patients you know. Yeah and give them more of my dedicated time. And I think there's something actually interesting as you start to see chatbots and particularly those with digital avatars is humans will open up in some ways to one of these you know chatbots or these avatars in ways that they may not necessarily feel comfortable opening up. So I think it's it's just giving us new avenues to do things that we were already doing anyway and I think that's pretty cool. Yeah do do you think this is this is sort of switching gears a little bit but do you think there's like a compute bottleneck for agents? Do you think we're going to are we going to be able to scale agents and get to agents as fast as people would want to cuz there's there was a narrative I feel like the narrative sort of faded a little bit but there's a narrative maybe 6 months ago that there's a wall and AI is hitting a wall. Do you think there's a wall? Do you think we're going to hit some compute bottlenecks? I think you know this is this is our life's work at Nvidia is to make sure that we have the compute to you know drive the world's agents. And so I think you know a lot of the announcements we talked about today both on the hardware and on the software side are really focused on making these as efficient as possible. And I think what's really cool is as you're looking at this AI space when the first comes out it's usually big it's boxy and maybe not the most efficient and then over time that efficiency comes in and allows us to do the next big leap. And again that starts out you know it big and and heavy and again gets more and more efficient over time. So I think I'm not seeing a bottleneck but I do know it's an area we need to continue to drive because I do think the compute requirements are going to continue to grow. Cool well so let's extrapolate out a little bit. So if you're looking like 5 10 years down the road what do you what do you sort of envision with AI agents? Where where do you think this is all headed? And I know that's hard because it's it's sort of exponential technology and it's really hard for humans to grasp exponentials. incredibly hard. I mean I can barely keep 6 months out of you know and new things keep popping up. I mean I think certainly the the biggest things that we're going to see are teams of agents. And that's going to be you know that we're already seeing it today it's starting now and it's going to continue to grow. And so you know the way that I see it is you know to when we first got agents and we first got you know chatbots and things you could ask a question and it would respond to a question and then you could ask it to do a function and it could now do that function and then you can ask it to do a bigger job and you can start to these these agents are just getting more and more sophisticated. So 5 to 10 years out you know we may be able to give something you know as simple as you know design me my you know retirement home and it will come back with all of the pieces involved in that without a human ever having to do another prompt follow it up. And I think that that could be pretty cool. So that might even be just two or three years out of you know but yes I certainly see that I think that's where we're going. Yeah so so you mentioned teams of agents that's really fascinating to me. Like do you have any examples of like what sort of agents would team up to work together and what sort of tasks will accomplish? Yeah absolutely. So I I work in marketing right? And so so a lot of my job is about doing messaging writing blogs getting imagery building demo videos things like that. And today you know you start with one you know app that can help you write and then you go to another app that helps you build images and then you go to another app that that helps you write the you know the code for the website and then you know right? And they're they're all separate and and each one of them returns to me and I you know move to the next step. I think in the future what we're going to see is those agents will all be connected through function calls and we actually at GTC announced a blueprint that's going to help us do this. It's called IQ. And so by connecting all these together again as a human I'll be able to start put in my my overall request I need to build a new website for a new announcement that's coming out and it will be able to do all those functions together. And I think what's really cool about this is it's going to allow us to design agents that solve specific tasks but by combining them together in that sort of that composable way they'll be able to do bigger and bigger job functions. Very cool. So do you see like what sort of like bigger world clock problems do you see like AI and AI agents solving for us? I mean I think networking is a really interesting one because anything that has that much data that humans can't solve I think is is amazing. You know digital twins and simulation of those types of environments whether it's you know the climate and weather whether it's you know the businesses and and things that we run I think all of these are areas where agents are just going to add to the ability to work with these. Then I think businesses of course it's going to be about you know providing those tools so that their employees are just that much more effective. Yeah. And that's that's not a big world problem but it's a very common serious world problem. Everyone everywhere is talking about AI agents right now. But here's the thing most companies are going about it all wrong. This guide cuts through the hype and shows you what's actually working right now. HubSpot has gathered insights from top industry leaders who are implementing AI agents the right way. You'll discover which agent setups actually deliver ROI and how businesses are automating their marketing sales and operations without replacing their teams. Get it right now by scanning the code or clicking the link in the description. Now let's get back to the show. One of the things that I think was really fascinating and I've heard I've I make jokes about how I I feel like I'm on like the Jensen tour cuz I've actually seen his last like five keynotes. But one of the things that I've I've seen him talk about that's really fascinating to me is the Earth 2 where it's got the they basically can map out the weather patterns and figure out weather events a lot earlier. So I'm really excited to see this sort of overlap of like the Earth 2 concept and agents and solving some of the more like bigger climate type issues as well. 100% I mean I think one of the things when when we introduced Earth 2 is this idea that we we think a lot about these problems. We we have conversations about them but it's really hard as humans to sort of visualize what you know some of the changes that we make in government and things like that are going to do in the next 5 to 10 years. We're very you know immediate creatures right? So that's where we're focused. But I think you know imagine being able to go to Earth 2 and through an agent say hey can you you know if we made these changes what would happen or you know if we you know took these steps how could that affect the world and and what more can we do? Maybe there are things that we're not even recognizing that the agent could recommend. Because I think that's what's really cool about these agents is they're not humans. They don't think the way we do and so by giving them all that data and giving them an Earth simulation they might be able to uncover things that we've never thought of. Yeah I think that's amazing. What are some of the things in the AI world you know agents or otherwise that have you personally really excited? Like what what sort of stuff do you use? What do you play with? Like what what's your AI stack that you use? I use as many as I can get my hands on. We have a lot of them in Nvidia. I mean one of my favorites and and and I know Jensen talked about it a little bit on stage is Perplexity. Yeah oh I love Perplexity. I love Perplexity because I think it takes it it changes the dynamic of of humans and how we interact. Um, rather than searching for information and having to spend the thought process on that, it's really about who can ask the best questions. And I think that's a really powerful change and and, you know, power dynamic that that Perplexity has given to the user. It's now if you can ask great questions, you can find great information. So, I love that tool. Um, we have AI agents in our company that help us with everything from, you know, our benefits and and understanding how to make the right decisions, um, you know, for each employee, which I think is pretty cool, into, you know, how we do our jobs. Whether that's video creation, image creation, um, certainly content, uh, which which you have to write a lot of. Uh, so yeah, so all of those I think are great. And then, you know, I take them into my personal life in terms of organizing and planning. Yes. Um, I'm very organized at work and I don't always save a lot of that organization for my personal life. And so, I can hand it off to, you know, any source of chatbot. I think, uh, ChatGPT is excellent for this. Um, it's just pretty cool. Um, and and lately deep research. Um, so I've been using a lot of that and and again, we just introduced a blueprint that's going to allow, um, deep research on on your own personal things. And so, I can imagine that's going to be pretty powerful. Cool. So, for people that wanted sort of stay in the loop on AI that are curious, maybe they're worried about it. Like, what sort of advice would you give them to to sort of stay on top of things? My best advice on AI is use it. It is it it sounds really obvious, but I think, one, I think it actually alleviates a lot of concerns when you understand how the technology works. And by using it, you can see what works today, where the limitations are, and and how it kind of functions. And I think it lets people see it as that tool versus the you know, something that might be scary. Um, so that's really the first piece of of advice. I think that's really important. And then I think it's about, you know, trying to identify where many of those, um, you know, concerns might be coming from, whether that's, uh, data or security or things like that. And understand how AI can also help um, with those. And so, that tends to be, you know, one of one of my best pieces of advice. Cool. Let's talk about, um, was it Llama 3 and Nemotron? Yeah, so Llama Nemotron. Llama Nemotron. Let's talk about Llama Nemotron. Absolutely. So, in video, we work with all of the the leading model builders that are out there, including Meta who released Llama. And what's really cool about Llama is there are, I think it's 85,000 derivatives of this model. And so, of course, in video, we've got a lot of really smart people who know how to optimize and and make models more efficient. And when reasoning came out and when and DeepSeek introduced this this reasoning wave into the open source, we sort of said, well, how can we bring this and and make it really efficient for, you know, people who wanted to play this on in video. And so, by starting with the the Llama model, which is an incredible model, we brought in our, um, expertise to train it so that a model that previously couldn't do reasoning now could actually think through problems. And so, we used we found, you know, the data set to go be able to train the model on how to do this this new task. And we trained it and then we made the data source open. Which I think is really cool. So, if others want to do their own training or if they want to train a different model or anything like that, that data source is available. Um, but it's just teaching a model a new skill is, I think, a really powerful thing to see because it shows how quickly the the space is evolving. So, is there anything actually happening like underneath Llama or is it just you've got the Llama model, but now there's a new layer on top of it that knows how to think? Is there did any like new training happen to the model? Yes. So, yeah, the model was absolutely So, we post-trained the model. So, this is what a lot of a lot of companies out there are doing today is they take a base model and they actually train it with new data. And in this case, sometimes you're training it so it has more information on a particular topic. Um, this is particularly popular when you've got domains and industries that have specific languages that they speak, things like maybe the finance industry. Um, but in this case, we actually were training it on a skill. And that skill is to think through problems. And I think this is what's really interesting about reasoning is the way a reasoning model, uh, works is it starts by thinking through the question that you're asking it. And it breaks down that question into multiple steps and multiple parts. And then it actually goes through and and comes up with answers for each of those parts. And then it checks to say, okay, now that I've got those answers, does this actually, you know, come back with the right question? And it continues to do that until it gets to this highly accurate response. Right. And so, not only are reasoning models really good for improving accuracy, which we all know that's when an when an AI model is useful, it's when it's accurate. Right. um, but it also allows us to solve problems we can never solve. Right. So, a great example for me on this is I I love puzzles. I love all sorts of puzzles, but particularly Sudoku cuz I hear it's good for your brain. Um, Sudoku is a problem that humans can solve, Mhm. but actually traditional LLMs couldn't. Mhm. There are actually almost 80 different decisions that go into solving a Sudoku puzzle. And each one of those affects the other decision cuz obviously based on the rules that you have to follow. Reasoning models, and Llama Nemotron is a great example of this, can solve Sudoku. Ooh, interesting. Where a Llama model on its own couldn't. Wow. And so again, that's the type of thing. It doesn't sound like, you know, solving Sudoku puzzles going to change the world, but when you look at a Sudoku puzzle, there's actually a a lot of, um, things that go on in the world that are related to that. Yeah. I have a supply chain. I've got to ship, um, you know, uh, items and and and get them to different stores around the country. I happen to know there's a snowstorm coming in. And I need to make sure that my trucks are taking the most efficient routes. All of those are steps that impact the other decisions that are being made. And so, it's actually quite a complex problem, um, that relates in some ways to Sudoku. Yeah. Well, it's so interesting, too, because you take a normal model and it can't tell you how many Rs are in strawberry. Give it a thinking model and it will actually count and then double-check, "Did I do that right?" And then And then and then answer. get the right answers. It's so fascinating cuz it seems like such a simple problem to a human brain. But I then that's I think the thing is we we look at models and we think of them like we we, you know, personify them as humans and they're not. And so yeah, no, reasoning models are really cool for that and we've seen a lot of those, um, you know, great examples that come out of what these reasoning models can do. And it and it is. It's just I think it's really interesting to watch Well, it's so cool to Yeah, it's so cool to actually see the thought process because you'll actually see the models think through something and then go, "Wait, that's probably not right. Let me think that through again." And you actually see that text come out of it thinking through. And that to me is fascinating. It's fascinating and it's a great example of the compute, uh, story we were talking about, which is, you know, we do need more compute so it can think. The more it thinks, the more compute it requires. So, it's this really, um, you know, interesting cycle that we we're watching these things go through. One of my favorite things to do with reasoning models is to ask them to describe things to me like a five-year-old. Yes. And it will come up with the description. It will check if a five-year-old would understand that description. It will then make changes. And it's really interesting to see what models think five-year-olds understand. So, some point I'll have to go test it with a real five. Yeah, read this. Does this make sense to you? actually make sense? Exactly. Exactly. Very cool. Well, along the same lines, let's talk about real quick. Let's talk about hallucinations. Do you see like a path to zero hallucinations? Do we want to get rid of hallucinations completely? Like, what are your thoughts on that? I think, you know, there's certainly things we can do, um, to reduce hallucinations. And some of those are, you know, as simple as putting a guardrail in place, um, that says, "If you're not 100% confident in the answer or 99% confident in the answer, don't answer the question." So, that's a that's that's sort of it stops the the model from doing things that it shouldn't. Um, and Nemo guardrails, which is, uh, one of the the products that in video offers, helps with, uh, building those. But to your point about do we want to stop hallucinations altogether? Of course, we want it to give the right answers. But in being creative, we're asking it to come up with new things. So, it's you have to be able to distinguish between a hallucination and a creative generative response. So, I think that's where the this balance plays. So, there absolutely steps that can be taken. And depending on how targeted and focused you want the model to be, you can put more and more of those sort of guardrails or those policies in place that will keep it from hallucinating. Right. Yeah, cuz I think in a lot of scenarios, hallucinations are a feature, not a bug, right? If If you want it to write a short story for you, you want it to hallucinate that short story for you. Exactly. So, I think that's where we have to understand what's the hallucination versus what's the model doing what it's supposed to do. Right. And again, and that's where it becomes what's the use case. What are you trying to have it do? And really think those through and then find the right tools, to be able to actually, you know, go and do that. Yeah, so if somebody wanted to get started with agentic AI, they want to start playing with agents and testing the waters. What what do they do? What are our our steps? Well, so from in video, we have something called buildonvideo.com. So, we made the URL the URL super easy. If you're trying to build something, we have a one-stop shop for you. It's got, um, all the models on there so that you can test them out whether it's the new llama nemotron model with reasoning. You can actually turn reasoning on and off. It also has all the blueprints so you can actually test them and experiment from them. And then from there there are also steps to go deploy them and test them out and build them yourself. So I think that's a great starting point. Very cool. And for the more like technical people that maybe are trying to develop something like is there a place they can go play with the APIs like what do we do there? Build.nvidia.com it's literally whether you're you know an enthusiast whether you're a developer whether you're actually trying to you know build something to put in production. This is the one-stop shop because everything's on there. You can take it as far as you want. You can play around with the UI. You can play around with the APIs. You can actually download and deploy these models on any you know Nvidia hardware. It's it's it's your one-stop shop for everything. Yeah. Actually that brings me to another question. Does this stuff work on older Nvidia hardware? If you have a you know a 3080 or a 4070 can I use this stuff on those as well? Absolutely. The only restriction is does it fit within the memory of the GPU. But if it fits it ships. So yes this will run on you know GPUs that are out there in the market today. Very cool. Awesome. Well thank you very much. This is this has been amazing fascinating. I love talking AI and nerd nails especially agents. Agents is the hottest topic so really appreciate you taking the time with me. Absolutely. I could also do this all day. So yeah have a good time. Thank you. One of the things that I've been really excited about that Nvidia is getting ready to release is their project digits now renamed the DGX Spark. Well while I was at Nvidia GTC I got to have a chat with Bob Pettey one of the guys who's leading up that project and he gave me a tour of the sort of personal units that you're going to be able to have in your own home or in your business to run your own AI supercomputers. Hi this is Bob Pettey vice president and general manager of enterprise platforms at Nvidia. A lot of the infrastructure that people are buying today in the cloud or through some of our server partners is Grace Blackwell. It was when it was Hopper Grace Hopper now Grace Blackwell. Right. Grace being the the arm CPU right? Right. And that's kind of the same technology that all the big big like open AI the those types of companies are using. Exactly. And the beauty of that is the arm CPU uses up so much less power. So if the majority of the workload was in the GPU and the CPU is kind of a traffic manager you'd you'd want you want to be able to do what you needed to do for as little as power as possible so you can put more GPUs in there. So hence Grace Blackwell. Right. Well you can certainly develop AI on Windows workstations with RTX Pro or GeForce. It's all great. But if you need an arm port of your software or getting familiar that's that's what that's where Spark fits the gap. And this is this is the same thing as the project digits that was announced at CES right? digits yeah. We we finally chose the name Spark. We had a people send in a lot of comments. But it's uh as Jensen mentioned in the keynote you know what was a box that was like this several years ago 20 core CPU one petaflop is now in this little 5 by 5 by you know less than two two inch box. It's got the C2C memory between the the Grace processor and the Blackwell GPU is like 256 gigabytes a second. You won't have that on a traditional workstation because you're going to go over the PCI bus right? So high high memory bandwidth between the CPU and the GPU 128 gigs of memory available from us online but that's really for the enthusiast who who want the gorgeous bezel and and So what kind of things can I do with this now that I I can't do with my 5090 at home? Good question. So from a frame buffer size there are so many more models that you can run on this. You can do fine tuning on 70B models. You can you put two of these together with this cable. This is a connect X ethernet. Put two of these together you can run a 400B 400B in parameter model. Can't do that on the 5090. You can't do the 70B on a 5090 right? And so the size of the model is very dependent. The other thing with a 5090 your your memory bandwidth between your CPU and the GPU is throttled by the PCI bus. Okay great. And this has you know cache coherent high speed memory bandwidth. So it really enables you to to test what your code might look like running on one of the data center providers OEMs out there because same same C2C memory cache coherency speed you can do more than just say it works. You can you can get it to the point where you can remove a lot of the bottlenecks whatever whether you're doing a vision language model or you know multimodal model. That's that's the the biggest benefit. One is getting your code ready for what's the predominant AI infrastructure out there but the other is testing in a way that simulates how it's going to run you know when you run it on a node there. And then the idea is you're not wasting data center time or cloud time just debugging right or eliminating bottlenecks. When it when you here you deploy and and you scale from one GPU to N GPUs. So Got it. the sole purpose and not the sole purpose the main purpose of this was really to to help help uh spread the the Grace Blackwell ecosystem. Right. They're going as fast as we can make them but they're not necessarily accessible to enthusiasts who want to use FP4 features of Blackwell which you can do on 5090 but want to use it with some of the more popular models that have higher higher parameter sizes. So easy box the one terabyte version of storage 128 gigs is 2999. The four terabyte version is 3999. You can reserve on nvidia.com. Our initial go-to-market partners are Dell HP Asus. They've got their own branded branded boxes without the gold foil and then we'll expand that. Yeah I remember Jensen said something to the effect of like imagine it's a cloud computer just sitting on your desk. It's not going to the cloud. You don't you don't have to worry about internet connections anything like that. It's just a cloud computer sitting on your desk that can do all of the inference right there on the bigger models. It's an AI supercomputer on your desk and there's a bigger one that we'll walk to in a second. But the other thing about this is we're not suggesting that everybody just replace their existing you know laptop or workstation. You got a GeForce laptop or RTX Pro laptop you you plug this guy into it. So you might do all everything you need to on a 5090 running your games and everything. You do an AI development or want to write AI inferencing that helps you on your your 5090. Yeah. Plug that into it. It's that's why Jensen showed the you know the MacBook and the and the Spark cuz it's it's really meant to be both a a plugin to Uber assist Right. maybe less less capable machines whether they have a GPU in them or not. And it's not using all the processing on your computer so you can be running AI models on this while playing Cyberpunk on your computer. Exactly. Yeah. Exactly. So and you know just form factor wise cost wise we think it'll be easy for people to to do it as an add-on but certainly there's you know there's a great GPU in here. We've we're running games on this. I you know I wouldn't want it as my GeForce laptop but probably want to connect it to my GeForce if I was you know doing AI AI tuning for game development or things like that right? So that's DGX Spark. Let's walk this way. We've got this is the RTX Pro line. Our 6000 line is the one that's you know somewhat akin to the 5090 on the GeForce side. The reason we have this Pro line you know the the manufacturing of it is a very precise bomb. It's not built by many different ICs. There are uh computing benefits on here that we perceive the gaming community doesn't need so some of the some of the high end compute performance here is going to be much better. The AI inferencing performance is about the same. Big difference is frame buffer. Right. And doesn't this one have like 96 gigs of VRAM? 96 yeah 96 gigs of RAM and we we've got the if you want to get full power we've got the 600 watt version. We've got a 300 watt version that most desktops can take today. And then you get the same technology in the server version that would go in a in a rack here. So we used to call these in the past we've had A40s or L40s based on Ada Lovelace. The the Ampere Lovelace. We're going to call that B40 based on Blackwell. But kind of aligned around RTX Pro workstation Max-Q Max-Q is that optimal power point and in the server edition. So again same infrastructure you can code and develop and then deploy on your RTX server and in a rack in the data center meant to save time. Heading down this way. So um you'll see the the brand here, but these are these are our DGX stations. Um initial initial partners um uh Dell, HP, Uh-huh. um ASUS, Lambda, and Supermicro. uh we show these two here because these are their boxes. This is what it will look like. Um if you get look in here, this is our a GB300 board. Okay. So, much more powerful um Grace processor, and then a extremely powerful B300. B300 is the same GPU in the latest DGX. Okay. So, uh 784 GB of memory again. So, the benefit is you're doing ARM development, um a lot of memory and and very very high-speed memory bandwidth like like Spark. So, you're you're not just seeing that code works, you're seeing how well it works before you chew up time on on your data center rack. Now, does this need a like a separate CPU, like a Intel AMD kind of thing? Nope, it's uh the on both the Spark um and uh and the station, we're providing the Grace CPU. Okay. So, Grace CPU and the Blackwell GPU there. Um uh graphics out, we we don't put a big graphics card in here. So, this one has a 4000. That's uh small form factor. The reason being we want this to plug into a standard 15 amp wall outlet. Right. Might need to be a dedicated one. Uh cuz you know, it's at six 15 amp B1650 watts, I guess. Um and we're going to come pretty close to that. Uh which is why the manufacturers are liquid cooling this. Um and uh they started doing that in the gaming side. So, some of the Alienware chassis you see the liquid cool. Right. So, they're well adept at that. Um they will liquid cool this so we can stay, you know, thermally be good um and not need to take up uh a lot more power with a a lot of fans. So, um again, if you're if you're an enthusiast, just getting started, uh even an enterprise where you're you know, you know the size of your models and what you want to get done, Mhm. uh easily connect that. If your full-time job is like prepping AI, developing AI to deploy to the data center, Right. you have your choice of logging in to the data center, Mhm. maybe you get a virtual workstation delivered back to you and you do your job, or putting this at your desk. Okay. And this is it literally is is like one of those B300 notes. It is a server node in a desktop um with graphics out right there. From a data privacy standpoint, from a IP protection standpoint, I'm not sending anything anywhere. It's right there. and that's us not that's more important than just privacy and and IP protection, it's just the time and the cost of of transport of data. Right. You you might run a, you know, a 600 billion parameter model, um but the data that you're running it on, whether let's say it's Cosmos with BLM, all the videos that you're going to be processing and the amount of data, you'd want to just sit here versus upload all that data or even if your own dedicated data center and run it there and then download that. So, you got to ingress and egress cost of data transport all happening right here. um that guy will be available in uh summertime frame. Okay. um reservable today. This guy will be late summer. Okay. um we have a founders edition for that because it's it's cool and enthusiasts are going to want Yeah, absolutely. desk, right? Uh this one is all only available for the OEMs. The way for us to scale our enterprise businesses through the OEMs. And so, um if you were to go to the Dell booth today or the HPI booth or ASUS, you'll see their versions of these, which here, you'll see their their versions of the Spark with the Dell blue and the Nvidia green LED and the HPE blue. Um and that's that's the way to expand the ecosystem for Grace ARM Uh-huh. development Grace Blackwell development, expand the access to technology that normally is only available if you've got the capital expense to put one of these racks in. And now it's at your desktop. Very cool. Yeah. Amazing. Well, thanks, Bob. This has been really uh informative and I appreciate it. Yeah, no problem. Thank you. Once again, thank you so much to Amanda and Bob for having those conversations with me. Before I wrap this one up, I want to ask for a favor. The Next Wave podcast was actually nominated for a Webby Award in the business category, and we need your help to try to win that award. It's voted on by listeners. So, if you enjoy these podcast episodes, we'll make sure there's a link in the description where you can go and let your voice be heard, and you can vote for the Next Wave podcast to win a Webby Award. Thank you once again for tuning in. We really appreciate you. Hopefully, we'll see you in the next episode.