Overview

On This Page

Introduction

Iguazio V3IO Frames (“Frames”) is a multi-model open-source data-access library that provides a unified high-performance DataFrame API for working with NoSQL and time-series (TSDB) data in the data store of the Iguazio Data Science Platform (“the platform”). This reference describes the library’s DataFrame API for Python 3.6 and 3.7. See also the Frames restrictions in the Software Specifications and Restrictions documentation.

In addition to the examples provided in this reference, you can find many examples of using the Frames API in the platform’s tutorial Jupyter notebooks. It’s recommended that you begin your development with the frames.ipynb notebook, which has getting-started examples and related documentation.

Creating a Frames Service

To use Frames, you first need to create a Frames service:

  1. On the Services dashboard page, select New Service from the top action toolbar.
  2. In the Basic Settings tab, select V3IO Frames from the Service type drop-down menu. You can optionally edit the service name and description.
    Note
    You can only create a single shared tenant-wide instance of the Frames service. If the V3IO Frames option is disabled, this means that the parent tenant already has a Frames service (as indicated in the tooltip for this service type). In this case, cancel the service creation, locate the Frames service in the services table, and verify that it’s enabled. Otherwise, proceed to the next step.
  3. Proceed to the Common Parameters tab and optionally change the default resource configuration; this isn’t advisable for new users.
  4. Proceed to the Custom Parameters tab and select Save Service to save your changes.
  5. Select Apply Changes from the top action toolbar of the Services page to deploy your changes. When the deployment completes, you should be able to see the Frames service in the services table, as demonstrated in the following image:

    Frames service

Initialization

To use the Frames API, you need to import the (v3io_frames) Python library. For example:

import v3io_frames as v3f

Then, you need to create and initialize an instance of the Client class; see Frames Client Constructor. After you have a client object, you can use the Client methods to perform different data operations for the supported backend types.

Backend Types

All Frames client methods receive a backend parameter for setting the Frames backend type. Frames supports the following backend types:

  • nosql or kv — a NoSQL backend for working with platform NoSQL (key/value) tables. See Frames NoSQL Backend.
  • tsdb — a time-series database (TSDB) backend for working with TSDB tables. See Frames TSDB Backend.
  • csv — a CSV backend for working with comma-separated-value (CSV) files. This backend type is used only for testing purposes.

Client Methods

The Client class features the following methods for supporting basic data operations on a data collection, such as a NoSQL or TSDB table:

  • create — creates a new collection.
    Note
    The create method isn’t applicable to the NoSQL backend (nosql | kv), because NoSQL tables in the platform don’t need to be created prior to ingestion; when ingesting data into a table that doesn’t exist, the table is automatically created. See NoSQL Databases.
  • delete — deletes a collection or specific collection items.
  • read — reads data from a collection into pandas DataFrames.
  • write — writes data from pandas DataFrames to a collection.
  • execute — executes a backend-specific command on a collection. Each backend may support multiple commands.

While most methods and commands are supported by all backends, there are differences in the supported parameters and their significance.

User Authentication

When creating a Frames client, you must provide valid platform credentials for accessing the backend data, which Frames will use to identify the identity of the user. This can be done by using any of the following alternative methods (documented in order of precedence):

  • Provide the authentication credentials in the Client constructor parameters by using either of the following methods:

    • Set the token constructor parameter to a valid platform access key with the required data-access permissions. You can get the access key from the Access Keys window that’s available from the dashboard user-profile menu, or by copying the value of the V3IO_ACCESS_KEY environment variable in a web-shell or Jupyter Notebook service.
    • Set the user and password constructor parameters to the username and password of a platform user with the required data-access permissions.
    Note

    You cannot use both methods concurrently: setting both the token and user and password parameters in the same constructor call will produce an error.

  • Set the authentication credentials in environment variables, by using either of the following methods:

    • Set the V3IO_ACCESS_KEY environment variable to a valid platform access key with the required data-access permissions.
      Note
      The platform’s Jupyter Notebook service automatically defines the V3IO_ACCESS_KEY environment variable and initializes it to a valid access key for the running user of the service.
    • Set the V3IO_USERNAME and V3IO_PASSWORD environment variables to the username and password of a platform user with the required data-access permissions.
    Note
    • When the client constructor is called with authentication parameters (option #1), the authentication-credentials environment variables (if defined) are ignored.
    • When V3IO_ACCESS_KEY is defined, V3IO_USERNAME and V3IO_PASSWORD are ignored.

See Also