Backing Up the Platform

On This Page

Overview

Recommendation
Iguazio Platform clients are strongly encouraged to backup their data and configuration periodically.

Backup can be performed on external servers or the platform nodes. A full backup collects all V3IO data (using Gibby), as well as the platform configuration (users, projects, services), MLRun DB and Kubernetes entities.

For more information and to set up the backup solution, contact Iguazio Support.

Prerequisites

  • A mounted storage drive. A cloud based storage service (such as EFS) is highly recommended.
  • Network access to Iguazio system (if running from external server)

Basic Commands

Generating the Manifest

./bkp backup generate-manifest

See a typical manifest.

Running the Backup

./bkp backup run-backup

Backup Tool Options

Options specified via CLI override the manifest configuration
To view all available CLI options, run

./bkp backup run-backup --help

Common options

--help

Show this message and exit

-v, --log-level [error|warning|info|debug]

default: info

--standalone

Run Gibby in standalone mode (instead of k8s job)

-m, --mode [dry-run|normal]

Run mode. dry-run runs the configured flow without applying the execution phase

--preset [default|platform|external-server|gibby|migration]

Preset backup tasks according to common use-cases
Run `./bkp backup presets` to view components of each preset

-z, --archive

Archive backup instance folder

-ip, --data-node-ip

IP address of the data node

Manifest Example

Note
Do not use the manifest example for your platform backup. To generate your own manifest on your server, run the command
./bkp backup generate-manifest
apiVersion: 0.4.alpha
kind: backupManifest
metadata:
 name: "template-backup-and-restore-manifest"
spec:
 description: "iguazio system backup manifest file template"
 apiCredentials:
   system:
     username: "USERNAME"
     password: "PASSWORD"
 backupSpec:
   mode: "dry-run"  # Change to make run actionable by default, or use --mode OPTION to override
#    dataNodeIp: "127.0.0.1"  # if running from an application node, specify the data node IP to register
#    preset: "external-server"  # Flow preset for which components to backup
#    components:  # Or specify the components and execution order yourself
#      - "rethinkdb"
#      - "nuclio"
#      - "etc"
   backupTargetFolder: # No default. Please set "/full/path/to/backup/folder" and validate r+w permission
   cacheUpperBound: "100GiB"  # estimated size of system cache
   checkAppServices: True  # If True, will check for overall app services state == ready
#    staticServeHelmChartsPath:  "/full/path/to/helm/charts"
#    kubernetesKind: "KIND" # manual override for app cluster kubernetes kind
#    kubernetesJobNode: "dedicated bkp node?" # manual override for app cluster kubernetes Gibby job node
   retentionPeriod: "3 days"  # On change, please validate retentionPeriod is valid
   rotateOnBackup: False  # If True, rotation will run on each backup. Please be aware of retentionPeriod above
   archive: False  # If True, backup will be archived to a tar.gz file
   gibbyTimeout: "23 hours"  # will kill the data backup process on timeout. Set to 0 to disable
   standalone: False  # If True, gibby will be run in standalone mode, rather than as a kubernetes job
#    gibbyExecPath: "/full/path/to/executable"  # override bundle with absolute path to run gibby as standalone option
#    diskRequiredMultiplier: 1.5  # multiplier for backup disk requirements, to allow for compression overhead
#    gibbyImage: "gcr.io/iguazio/gibby:0.8.30"  # image to run gibby as a kubernetes job
   gibbyJobMountPath: "/mnt"
#    namespace: "default-tenant"
   reclaimPolicy: "Delete"  # persistent volume reclaim policy for kubernetes Gibby job
   storageClass: "gibby-backup"
#    storageProvisioner: "kubernetes.io/aws-ebs" # manual override for app cluster kubernetes storage provisioner
#    gibbyJobPvcDesiredState: "Pending"  # manual override for app cluster kubernetes Gibby job PVC desired state
   gibbyCommands:
     - "create"
     - "snapshot"
#    gibbyOptions:
#      --backup-name: "gibby-backup"
#      --data-plane-url: "https://webapi.default-tenant.app.your-system.iguazeng.com"
#      --control-plane-url: "https://dashboard.default-tenant.app.your-system.iguazeng.com"
#      --data-access-key: "DATA_SESSION_ID"
#      --control-access-key: "CONTROL_SESSION_ID"
#      --path: "FROM_BACKUP_FOLDER"
#      --logger-file-path: "FROM_BACKUP_FOLDER/gibby.log"
#      --log-level: "FROM_CLI_FLAG"
#      --backup-config-spec: "INLINE-JSON-CONFIG"
#      --logger-no-color: ""  # binary flags pass as dict keys with empty values