What are Deep Learning Pipelines?

In any AI project, most of the complexity arises from the data: ingesting, exploring, processing, storing, monitoring it, and more. That’s why the concept of a machine learning pipeline is so important, regardless of the use case of particular technologies involved. Bringing deep learning use cases to production is a particularly complex AI project, given the scale of data required and therefore the critical data-related tasks involved. As complexity increases, the success or failure of an AI project can easily hinge on how effective the pipeline is designed.

The challenges of bringing deep learning use cases to production

Typically, deep learning operations comprise the following stages: 

• Data ingestion: Ingestion often occurs at the edge—for example, capturing data from cars, point-of-sale devices, or cameras. Depending on the use case, data can be streamed, or cached at the edge and forwarded in batch periodically. 
• Data preparation: Preprocessing in some form is almost always necessary to prepare data for inference through the model. 
• Training: The architecture for deep learning models may be very large, with many parameters and weights to train. The volume of the training data means that efficiency is a critical factor, with the use of GPUs and distributed training strategies. 
• Validation: During training there is typically a validation set so that the model hasn’t been over-fit to the training data, but generalized over unseen data as well. The evaluation process is used to check the model on a big unseen data to see if the model performs well. Evaluation is (or at least should be) done again and again to make sure the model is still performing as we expect, or if it needs to be retrained. Collecting data for evaluation is another challenge, as it needs to be tagged.
• Deployment: The model is moved to software development, to develop the application, and then to production. The model must be monitored, periodically evaluated and if needed, retrained.

When approaching deep learning pipelines, the prevalent mindset most enterprise ML teams have today is to begin by developing in the research environment. Data scientists will take the data (which can be images, video, text, etc.) from an object, data lake or warehouse, prepare it, train the model and validate it. However, when data scientists throw the model “over the wall” to engineers, this process quickly becomes a convoluted mess.

Like any other component of the product, the model needs to be a reliable asset, with many dependencies and requirements like performance monitoring, logging, scalability, and so on. These requirements mean that “beginning with the end in mind” is a critical factor in accelerating the path to production. 

Deep Learning Use Cases

• Recommender systems for Retail
• NLP text analysis
• Dynamic retail pricing
• Image classification
• Financial investment strategies and risk management
• Supply chain optimization
• Autonomous driving systems

Watch: How IHS Markit is scaling NLP deep learning pipelines by processing and analyzing documents with composite data.

Deep Learning Frameworks

There are various tools that ML teams can use to develop deep learning models. Popular deep learning frameworks include:

• TensorFlow
• Keras 
• PyTorch
• MXNet

Deep Learning Pipeline Orchestration with Iguazio

MLRun is an open source framework, built and maintained by Iguazio, that automates and  orchestrates the machine learning pipeline end to end. 

MLRun is seamlessly integrated with the most popular ML/DL frameworks, like SciKit-Learn, Tensorflow, PyTorch, XGBoost and more. With a few lines of code, MLRun takes training code and runs it in Kubernetes, providing automatic logging, profiling, versioning and distributed training.

Deep learning pipeline management is important, especially when it comes to resource management, because deep learning models involve heavy computational workloads. ML teams working with deep learning use cases need a way to control and manage resources, especially when using GPUs. MLRun can help lower costs by orchestrating different types of instances and spot instances throughout the pipeline flow. This includes GPU sharing and scale to zero options, to make the most efficient use of GPU investments.

Demo: Deep Learning with MLRun

Check out the demo here to see an example of how to use MLRun with a deep learning application. The demo shows how MLRun can be added to existing training and serving code (in this case, Tensorflow or PyTorch) with minimal effort – only one line is added to be more specific.