Scaling GPU Deployments - Iguazio Data Science PaaS

Faster and Scalable ML Applications

GPUs are useful resources for compute-intensive and highly parallelizable operations tasks such as streaming and parallel computing. However, they aren’t well adept when working with large data sets and inefficient resource management generates heavy expenses.

Data Science Platform with GPUs

Large scale processing on GPUs
Extreme performance with serverless
Shared GPU resources
Fully managed data science PaaS

GPU Challenges

Limited data access: GPUS access only 10s of gigabytes at a time, while typical data intensive applications require 10s or 100s of terabytes. Lack of appropriate integration block data scientists from enjoying the benefits of GPUs.
Inefficient resource sharing: GPUs are a relatively expensive resource that are not used all the time and in the same capacity. Developers need the ability to dynamically adjust the usage of GPUs instead of owning GPUs that sit idle.

GPUs aren’t the right choice for all tasks: GPUs are great for certain tasks, but not always cost effective for other tasks. In order to make specific tasks GPU or CPU optimized, applications must be fragmented into smaller microservices. But that’s not enough, micorservices which don’t communicate over a high speed data layer cause a negative performance effect.

Data Science PaaS with GPU Scaling

Iguazio provides a fully managed data science PaaS over Kubernetes, with integrated leading machine learning tools such as Jupyter notebook, Scikit Learn, TensorFlow and Pytorch. It enables automatic scaling to multiple GPU servers and rapid processing of hundreds of terabytes of data.

Integration with NVIDIA RAPIDS for Scalable GPUs

Iguazio is integrated with NVIDIA RAPIDS’s open-source machine learning libraries, for faster and scalable data processing. It enables:

Direct writes/reads into/from the GPU’s memory using RAPIDS data frames
Predicate push down – offloading queries and analytics to Iguazio
Streaming data in chunks directly into GPU
Full parallelism – multiple nodes can read data, each only one shard

Maximum Utilization of GPUs and CPUs

Iguazio fragments applications to microservices for better CPU and GPU optimization as opposed to monolithic architectures, and runs them over a high speed data fabric. It also enables users to scale GPUs in and out with serverless for maximum resource efficiency. Instead of wasting costs on every data scientist having his own GPU, 10 data scientists share 3-4 GPUs and use them as needed.

The use of Iguazio’s serverless functions (Nuclio) improves GPU utilization and sharing, resulting in almost four times faster application performance when compared to the use of GPUs within monolithic architectures. Nuclio is fifty times faster than serverless solutions that do not offer GPU support, such as Amazon’s Lambda. Serverless and Kubernetes target key challenges in data science: they simplify operationalization, eliminate manual devops processes and cut time to market.