Documentation Home

Introduction

• Introducing the Platform

• In-Depth Platform Overview

• The Platform Dashboard (UI)

• APIs Overview

• High Availability (HA)

• Iguazio Trial Quick-Start

Data Science and MLOps

Platform Services

• Working with Services

• Working from Remote

• The Platform's Application Services

  • Overview

  • Jupyter

  • Python ML and Scientific Computation

  • Running Applications over GPUs

  • MLOps: MLRun and Kubeflow Pipelines

  • Time-Series Database (TSDB)

  • Dask

  • V3IO Frames

  • Trino and Hive

  • Spark

  • Horovod / MPI Operator

  • Data Monitoring and Visualization

  • Logging and Monitoring

  • Web API

  • Web Shell

  • Nuclio Serverless Functions

  • Hadoop

  • Tensorboard

  • Docker Registry

• Monitoring Platform Services

• Adding Grafana Dashboards

• Data Layer

  • Data Containers

    • The Predefined Containers

    • Container Names and IDs

    • Working with Data Containers

  • Data Objects

    • Objects Attributes

    • Object Names and Primary Keys

      • Best Practices for Defining Primary Keys and Distributing Data Workloads

    • Ingesting and Consuming Files

  • The Data-Layer APIs

    • APIs Overview

    • API Data Paths

  • Ingesting and Preparing Data

  • NoSQL (Key-Value) Databases

  • Streams

  • Time-Series Databases (TSDB)

    • Working with Time-Series Databases (TSDBs)

    • The TSDB CLI (tsdbctl)

  • Trino

    • Overview

    • The Trino CLI

  • References

    • Frames API (Python)

    • Spark APIs

      • Overview

      • Spark Data-Ingestion Quick-Start

      • Spark Datasets API (Pytyon and Scala)

        • Overview

        • The NoSQL DataFrame

        • Spark DataFrame Data Types

      • Spark-Streaming Integration API (Scala)

        • Overview

        • PayloadWithMetadata Class

        • RecordAndMetadata Class

        • V3IOUtils Object

        • Encoding and Decoding Types

          • Decoder Trait

          • DefaultDecoder Class

          • StringDecoder Class

          • Encoding Trait

    • Web APIs (REST)

      • Overview

      • Securing Your Web-API Requests

      • Data-Service Web-API General Structure

      • Simple-Object Web API

        • Overview

        • Container Operations

        • Data-Object Operations

      • NoSQL Web API

        • Overview

        • GetItem

        • GetItems

        • PutItem

        • UpdateItem

        • Common Objects

          • Attribute

          • Attribute Value

      • Streaming Web API

        • Overview

        • CreateStream

        • DescribeStream

        • GetRecords

        • PutRecords

        • Seek

        • UpdateStream

    • Common Data Types

      • Attribute Data Types

    • System Attributes

    • Expressions

      • Overview

      • Condition Expression

      • Update Expression

      • Functions

        • Array Functions

        • If Exists/Not-Exists Functions

        • Maximal-/Minimal-Value Functions

        • String Functions

      • Operators

        • Arithmetic Operators

        • Array Operator ([ ])

        • Assignment Operator (=)

        • Bitwise Operators

        • Comparison Operators

        • IN Operator

        • Logical Operators

    • Reserved Names

    • NoSQL Table Schema

Cluster Management and Deployment

• Best Practices

• Logging, Monitoring, and Debugging

• Backing Up the Platform

  • Backup Using Gibby

• Setting Labels on App Nodes (for jobs/functions assignments)

• Managing a Kubernetes Cluster remotely with the Kubeflow Configuration File

• Deployment and Specifications

  • Support and Certification Matrix

  • Software Specifications and Restrictions

  • Hardware-Configuration and Deployment Options

  • Cloud Deployment

    • Amazon Web Services (AWS)

      • Deployment Specifications

      • Installation Guide

      • Deployment How-Tos

        • Calculating Required Infrastructure Resources

        • Creating an IAM User

        • Creating an IAM Role and Instance Profile

        • Configuring VPC Subnet Allocation of Public IP Addresses

        • Network Security Groups Configuration

        • Post-Installation Steps

    • Microsoft Azure

      • Deployment Specifications

      • Installation Guides

        • Installation Guide

      • Deployment How-Tos

        • Pre-Installation Steps Using the Azure CLI

        • Calculating Required Infrastructure Resources

        • Network Security Groups Configuration

        • Post-Installation Steps

        • Troubleshooting Installations

    • Google Cloud Platform (GCP)

      • Deployment Specifications

      • Installation Guide

      • Deployment How-Tos

        • Calculating Required Infrastructure Resources

        • Creating a GCP Service Account

        • Post-Installation Steps

  • On-Premises Deployment

    • Deployment Specifications

    • Virtual-Machine (VM) Deployment

      • VMware vSphere

        • Installation Guide

        • Deployment How-Tos

          • Configuring Virtual Networking

          • Deploying the Platform Nodes

          • Attaching Data Disks to the Data Nodes

          • Configuring IP Addresses

          • Offline Installation

          • Post-Installation Steps

      • Proxmox VE

        • Installation Guide

        • Deployment How-Tos

          • Configuring Virtual Networking

          • Deploying the Platform Nodes

          • Attaching Data Disks to the Data Nodes

          • Configuring IP Addresses

          • Offline Installation

          • Post-Installation Steps

  • Post-Deployment How-Tos

    • Configuring the DNS Server

    • Configuring the SMTP Server

    • Configuring the Python Repositories

    • Configuring Environmental Variables

    • Air Gapped System Configuration

• References

  • Management APIs (REST)

    • Overview

    • General Management-API Structure

    • Sessions API

      • Overview

      • Create Session

    • Containers API

      • Overview

      • Create Container

      • Delete Container

      • List Containers

    • Cluster-Information API

      • Overview

      • Get Cluster Information

Platform Users and Security

• Platform Users

• Security

Release Notes

• Known Issues

• Version 3.5 Release Notes

  • V3.5.3 Release Notes

  • V3.5.2 Release Notes

  • V3.5.1 Release Notes

  • V3.5.0 Release Notes

• Version 3.4 Release Notes

  • V3.4.2 Release Notes

  • V3.4.0 Release Notes

• Version 3.2 Release Notes

  • V3.2.3 Release Notes

  • V3.2.2 Release Notes

  • V3.2.1 Release Notes

  • V3.2.0 Release Notes

• Version 3.0 Release Notes

  • V3.0.3 Release Notes

  • V3.0.2 Release Notes

  • V3.0.0 Release Notes

• Version 2.10 Release Notes

  • V2.10.0 Release Notes

• Version 2.8 Release Notes

  • V2.8.0 Release Notes

• Version 2.5 Release Notes

  • V2.5.0 Release Notes

• Version 2.3 Release Notes

  • V2.3.1 Release Notes

  • V2.3.0 Release Notes

• Version 2.2 Release Notes

  • V2.2.0 Release Notes

• Version 2.1 Release Notes

  • V2.1.0 Release Notes

• Version 2.0 Release Notes

  • V2.0.0 Release Notes

• Version 1.9 Release Notes

  • V1.9.5 Release Notes

  • V1.9.4 Release Notes

• Version 1.7 Release Notes

  • V1.7.2 Release Notes

  • V1.7.1 Release Notes

• Version 1.5 Release Notes

  • V1.5.4 Release Notes

  • V1.5.3 Release Notes

  • V1.5.2 Release Notes

  • V1.5.1 Release Notes

  • V1.5.0 Release Notes

• Version 1.0 Release Notes

  • V1.0.0 Release Notes

Frequently Asked Questions

• General

• Platform Deployment

• Data

  • Data Collection

  • Feature Store

• Model Development

• Model Deployment and Monitoring

• Monitoring and Troubleshooting