As artificial intelligence keeps gaining momentum, enterprises are increasingly focused on integrating large language models and private data capabilities. This growing demand makes high-performance AI chips more essential than ever.
AI hardware leader SambaNova Systems Inc. offers cutting-edge solutions, such as Reconfigurable Dataflow Architecture, that accelerate AI tasks through the dynamic maximization of computing resources, according to the company’s co-founder and chief executive officer, Rodrigo Liang (pictured).
SambaNova’s Rodrigo Liang talks to theCUBE about the importance of high-performance AI chips.
“10X performance of one-10th the power; that’s what we do,” Liang said. “People want to go to production, you want to run a Llama 400B model, which we just announced. We’re able to do that 115 tokens per second, which is 10 times faster than anybody else out there, and we run it at less than 19 kilowatt. We build everything from the architecture all the way down to the substrates and we use all of those technologies — it’s a Dataflow chip.”
Liang spoke with theCUBE Research’s John Furrier and Dave Vellante at the AI Infrastructure Silicon Valley – Executive Series event, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed the need for high-performance AI chips in the current enterprise era and how SambaNova makes this a reality.
How Dataflow fits into the high-performance AI chips’ picture
To meet the massive computational demands of AI applications, Dataflow utilizes a system of reconfigurable functional units, which plays a crucial role in the development of high-performance AI chips, according to Liang.
“Dataflow is ultimately the new way of computing these things,” he said. “We don’t have an ISA architecture, and so we are able to take this hardware, compile these models from Hugging Face directly on the hardware, and then just use these x86 hosts just to transfer the data over to the accelerator. It eliminates the need to have an on-chip ISA.”
Thanks to its prowess of building complex systems, SambaNova is able to offer high-performance AI chips since the company is natively Dataflow. As a result, it is able to incorporate Llama 400B and GPT-4-like models into enterprises’ environments, Liang pointed out.
“On one end, we build chips to compete with Nvidia,” he said. “We build our own substrates that run the largest models and the most complex models, and we run them faster than anybody else. On the other bookend, we build a full stack that allows you to actually deploy these models without having to learn CUDA, without having to do all the low-level manual work. You can just download these models, and then run them really fast.”
Here’s the complete video interview, part of SiliconANGLE’s and theCUBE Research’s coverage of the AI Infrastructure Silicon Valley – Executive Series event: