As artificial intelligence drives faster insights and real-time decision-making across the enterprise, the Cloudera AI Inference service, designed to operationalize machine learning at scale, is gaining traction.
To boost large language model performance and the private deployment of models, the Cloudera AI Inference service uses Nvidia NIM microservices and accelerated computing, according to Priyank Patel (pictured), vice president of artificial intelligence and machine learning at Cloudera Inc.
Cloudera’s Priyank Patel talks to theCUBE about the transformative power of the Cloudera AI Inference service.
“What we are integrating is the software stack that the Nvidia team has built out, something called NIM — NIM microservices,” Patel stated. “It’s an integrated hardware-software layer that sits above their [graphics processing units]. We learned more of what goes into the NIM, and that really formed the basis of the Cloudera AI Inference service. It’s the model serving offering from Cloudera that works anywhere on public clouds as well as on-premises and fundamentally enables our customers and enterprises to have private endpoints for AI to be able to build and run AI privately.”
Patel spoke with theCUBE Research’s Bob Laliberte and co-host Rebecca Knight at the Cloudera Evolve24 event during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed how the Cloudera AI Inference service enables fast deployment of models. (* Disclosure below.)
Delving deeper into the Cloudera AI Inference service
Given that data is growing exponentially, broadening AI solutions with products such as the Cloudera AI Inference service is important. The solution helps enhance user experience, scalability and operational efficiency, Patel pointed out.
“AI with Cloudera is about us building the best platform for our customers to build their AI applications with,” he noted. “AI in Cloudera is about us infusing AI within our platform without our customers ever needing to know about it, and that means there are dozens of teams internally within our organization who are building the copilots, the assistants and the capabilities that would ease the regular day-to-day user of the Cloudera platform. Cloudera manages a significant amount of data estate both on-premise and the cloud.”
Making developers’ lives easier is top of mind for enterprises. As a result, AI fits into the picture since it transforms developers’ work through enhanced collaboration, improved productivity and automated code generation, according to Patel.
“When we started out two years ago, the core competence of actually building these AI systems was with the data science teams, the AI teams [and] the machine-learning teams because that’s the technology evolution of these deep learning networks,” he said. “As it has progressed to now, we see and … internally use the term gen AI builders, intentionally not calling them developers [or] scientists because we think that there is a simplification of the skill set and up-leveling of skill set that has gone through in the industry.”
Here’s the complete video interview, part of SiliconANGLE’s and theCUBE Research’s coverage of the Cloudera Evolve24 event:
(* Disclosure: Cloudera Inc. sponsored this segment of theCUBE. Neither Cloudera nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)