Artificial intelligence is already part of catching a ride, as Uber Technologies Inc. continues to upgrade its platform with scalable GPU clusters.
Building upon its history of containerized workloads, Uber has adopted Anyscale Ray to run clusters with greater speed and scale, according to Zhitao Li (pictured), director of engineering, AI and model infrastructure, at Uber.
Uber’s Zhitao Li talks about upgrading Uber’s AI platform.
“Before Ray, we were operating distributed Spark clusters that trie[d] to run machine learning model training on the inside, but it’s very difficult to retrofitted those things to run on the GPU native cluster, managing the life cycle of the workload,” said Zhitao Li, who highlighted Uber’s early adoption of containers. “It’s almost a decade of history of the containers. So, these containers provide the best isolation of the workload and allowed us to utilize the GPUs and run heterogeneous computation cluster in the most maintainable and scalable way. With Ray, we are able to run many clusters. I believe we are doing more than 20,000 model training jobs every month on our clusters and in our data centers, and this would not be possible without Ray.“
Li spoke with theCUBE Research’s John Furrier at the Anyscale Ray Summit 2024 event, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed how Uber is adapting to the AI era and its focus on open source. (* Disclosure below.)
Managing heterogeneous data with scalable GPU clusters
As AI technology evolves, Uber has extended its eight-year-old Michelangelo AI platform, adding incremental innovations to keep up with the times. The company now enables users to rapidly expand their data workloads, based on the large number of clusters it can run on Ray.
“When we get to tabular data, the way we handle is we create abstractions for ourselves,” Li said. “So, we convert some of tabular data into unstructured Parquet files and make sure we process these things under as intermediate representations so that we hide all this complexity from our users. We operate one shared ML platform … and all the rising use cases of computer vision, video, all those new multimodal cases, allowed us to gracefully enter the generative era, because gen AI is all about techs.”
Uber is also training AI by elastically sharing resources through its Kubernetes clusters, which saves time and money. Going forward, Li emphasizes that open source is crucial for setting up an innovative AI ecosystem.
“Uber is entering hybrid cloud setup,” Li said. “Evolving the data lake, especially ML aspect of the data lake in the multicloud, multi-region, multi- provider environment, is going to be an interesting challenge … we are always thinking about open source. We want to open source the great innovations, those elastic resource sharing stuff — some of the tools we develop for responsible AI like explainability on top of Ray, some of workflow stuff. We want to always look around about great innovations in open source and absorb them in our platform.”
Here’s the complete video interview, part of SiliconANGLE’s and theCUBE Research’s coverage of the Anyscale Ray Summit 2024 event:
(* Disclosure: Anyscale Inc. sponsored this segment of theCUBE. Neither Anyscale nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)