Documentation Home

Introduction

• Introducing the Platform

• In-Depth Platform Overview

• The Platform Dashboard (UI)

• APIs Overview

• High Availability (HA)

• Iguazio Trial Quick-Start

Data Science and MLOps

Data Layer

• Data Containers

  • The Predefined Containers

  • Container Names and IDs

  • Working with Data Containers

• Data Objects

  • Objects Attributes

  • Object Names and Primary Keys

    • Best Practices for Defining Primary Keys and Distributing Data Workloads

  • Ingesting and Consuming Files

• The Data-Layer APIs

  • APIs Overview

  • API Data Paths

• Ingesting and Preparing Data

• NoSQL (Key-Value) Databases

• Streams

• Time-Series Databases (TSDB)

  • Working with Time-Series Databases (TSDBs)

  • The TSDB CLI (tsdbctl)

• Spark Data-Ingestion Quick-Start

• Trino

  • Overview

  • The Trino CLI

• References

  • Frames API (Python)

    • Overview

    • Client Constructor

    • NoSQL Backend

      • Overview

      • write Method

      • read Method

      • delete Method

      • execute Method

    • TSDB Backend

      • Overview

      • create Method

      • write Method

      • read Method

      • delete Method

    • Attribute Data Types

  • Spark APIs

    • Overview

    • Spark Datasets API (Pytyon and Scala)

      • Overview

      • The NoSQL DataFrame

      • Spark DataFrame Data Types

    • Spark-Streaming Integration API (Scala)

      • Overview

      • PayloadWithMetadata Class

      • RecordAndMetadata Class

      • V3IOUtils Object

      • Encoding and Decoding Types

        • Decoder Trait

        • DefaultDecoder Class

        • StringDecoder Class

        • Encoding Trait

  • Web APIs (REST)

    • Overview

    • Securing Your Web-API Requests

    • Data-Service Web-API General Structure

    • Simple-Object Web API

      • Overview

      • Container Operations

      • Data-Object Operations

    • NoSQL Web API

      • Overview

      • GetItem

      • GetItems

      • PutItem

      • UpdateItem

      • Common Objects

        • Attribute

        • Attribute Value

    • Streaming Web API

      • Overview

      • CreateStream

      • DescribeStream

      • GetRecords

      • PutRecords

      • Seek

      • UpdateStream

  • Common Data Types

    • Attribute Data Types

  • System Attributes

  • Expressions

    • Overview

    • Condition Expression

    • Update Expression

    • Functions

      • Array Functions

      • If Exists/Not-Exists Functions

      • Maximal-/Minimal-Value Functions

      • String Functions

    • Operators

      • Arithmetic Operators

      • Array Operator ([ ])

      • Assignment Operator (=)

      • Bitwise Operators

      • Comparison Operators

      • IN Operator

      • Logical Operators

  • Reserved Names

  • NoSQL Table Schema

Platform Services

• Working with Services

• The Platform's Application Services

  • Overview

  • Jupyter

  • Python ML and Scientific Computation

  • Running Applications over GPUs

  • MLOps: MLRun and Kubeflow Pipelines

  • Time-Series Database (TSDB)

  • Dask

  • V3IO Frames

  • Trino and Hive

  • Spark

  • Horovod / MPI Operator

  • Data Monitoring and Visualization

  • Logging and Monitoring

  • Web Shell

  • Nuclio Serverless Functions

  • Hadoop

  • Tensorboard

• Monitoring Platform Services

• Adding Grafana Dashboards

Cluster Management and Deployment

• Logging, Monitoring, and Debugging

• Backup Using Gibby

• Setting Labels on App Nodes (for jobs/functions assignments)

• Managing a Kubernetes Cluster remotely with the Kubeflow Configuration File

• Deployment and Specifications

  • Support and Certification Matrix

  • Software Specifications and Restrictions

  • Hardware-Configuration and Deployment Options

  • Cloud Deployment

    • Amazon Web Services (AWS)

      • Deployment Specifications

      • Installation Guide

      • Deployment How-Tos

        • Calculating Required Infrastructure Resources

        • Creating an IAM User

        • Creating an IAM Role and Instance Profile

        • Configuring VPC Subnet Allocation of Public IP Addresses

        • Deploying an Amazon EKS Application Cluster

        • Network Security Groups Configuration

        • Post-Installation Steps

    • Microsoft Azure

      • Deployment Specifications

      • Installation Guides

        • Installation Guide

        • Installation Guide AKS

      • Deployment How-Tos

        • Pre-Installation Steps Using the Azure CLI

        • Calculating Required Infrastructure Resources

        • Network Security Groups Configuration

        • Post-Installation Steps

        • Troubleshooting Installations

    • Google Cloud Platform (GCP)

      • Deployment Specifications

      • Installation Guide

      • Deployment How-Tos

        • Calculating Required Infrastructure Resources

        • Creating a GCP Service Account

        • Post-Installation Steps

  • On-Premises Deployment

    • Deployment Specifications

    • Virtual-Machine (VM) Deployment

      • VMware vSphere

        • Installation Guide

        • Deployment How-Tos

          • Configuring Virtual Networking

          • Deploying the Platform Nodes

          • Attaching Data Disks to the Data Nodes

          • Configuring IP Addresses

          • Offline Installation

          • Post-Installation Steps

      • Proxmox VE

        • Installation Guide

        • Deployment How-Tos

          • Configuring Virtual Networking

          • Deploying the Platform Nodes

          • Attaching Data Disks to the Data Nodes

          • Configuring IP Addresses

          • Offline Installation

          • Post-Installation Steps

  • Post-Deployment How-Tos

    • Configuring the DNS Server

    • Configuring the SMTP Server

    • Configuring the Python Repositories

    • Configuring Environmental Variables

    • Air Gapped System Configuration

• References

  • Management APIs (REST)

    • Overview

    • General Management-API Structure

    • Sessions API

      • Overview

      • Create Session

    • Containers API

      • Overview

      • Create Container

      • Delete Container

      • List Containers

    • Cluster-Information API

      • Overview

      • Get Cluster Information

Platform Users and Security

• Platform Users

• Security

Release Notes

• Known Issues

• Version 3.5 Release Notes

  • V3.5.0 Release Notes

• Version 3.4 Release Notes

  • V3.4.2 Release Notes

  • V3.4.0 Release Notes

• Version 3.2 Release Notes

  • V3.2.3 Release Notes

  • V3.2.2 Release Notes

  • V3.2.1 Release Notes

  • V3.2.0 Release Notes

• Version 3.0 Release Notes

  • V3.0.3 Release Notes

  • V3.0.2 Release Notes

  • V3.0.0 Release Notes

• Version 2.10 Release Notes

  • V2.10.0 Release Notes

• Version 2.8 Release Notes

  • V2.8.0 Release Notes

• Version 2.5 Release Notes

  • V2.5.0 Release Notes

• Version 2.3 Release Notes

  • V2.3.1 Release Notes

  • V2.3.0 Release Notes

• Version 2.2 Release Notes

  • V2.2.0 Release Notes

• Version 2.1 Release Notes

  • V2.1.0 Release Notes

• Version 2.0 Release Notes

  • V2.0.0 Release Notes

• Version 1.9 Release Notes

  • V1.9.5 Release Notes

  • V1.9.4 Release Notes

• Version 1.7 Release Notes

  • V1.7.2 Release Notes

  • V1.7.1 Release Notes

• Version 1.5 Release Notes

  • V1.5.4 Release Notes

  • V1.5.3 Release Notes

  • V1.5.2 Release Notes

  • V1.5.1 Release Notes

  • V1.5.0 Release Notes

• Version 1.0 Release Notes

  • V1.0.0 Release Notes

Frequently Asked Questions

• General

• Platform Deployment

• Data

  • Data Collection

  • Feature Store

• Model Development

• Model Deployment and Monitoring

• Monitoring and Troubleshooting