Data-Service Web-API General Structure

On This Page

The data-service web APIs enable complex manipulation of specific data types. These APIs include the Streaming Web API Reference and the NoSQL Web API Reference.

Request Syntax

The data-service API operations are HTTP requests that you send to the web-APIs (web-gateway) service of a platform tenant using the PUT or POST HTTP method. The operation to perform is specified within an X-v3io-function HTTP header. Data parameters (where required) are passed within the request's HTTP body, in JSON format. The requests conform to the following general format; (the order of the headers isn't important):

<method> /<container>/<resource> HTTP/1.1
Host: <web-APIs URL>
Content-Type: application/json
X-v3io-function: <operation>
<authentication header>: <value>

{<data parameters>}

import requests

url = "<web-APIs URL>/<container>/<resource>"
headers = {
            "Content-Type": "application/json",
            "X-v3io-function": "<operation>",
            "<authentication header>": "<value>"
          }
payload = {<data parameters>}

response = requests.<method>(url, json=payload, headers=headers)
print(response.text)

Following is an explanation of the <...> placeholders used in the request syntax:

<method>

The HTTP method for the request — POST or PUT (post or put in Python).
POST and PUT behave the same in the platform.

<container>

An identifier of the data container of the operation's resource. The container can be identified by its name or by its ID. For example, projects.

Note

It is strongly recommended that you identify a container by its name and not by its ID. The option to specify a container ID is deprecated and will eventually be removed. For more information, see Container Names and IDs.

<resource>

URL resource parameters, signifying the full path to the operation's resource within the container, or the first part of this path. The path can point, for example, to a collection (such as stream or a table), a collection component (such as a shard), or an object in the collection (such as a table item). It is also possible to provide the last part of the resource path via data parameters in the request's HTTP body (in JSON format) — see <data parameters> and Setting the Resource Path.

<web-APIs URL>

The URL of the web-APIs (web gateway) service of a platform tenant.

Get this URL by copying the API URL of the web-APIs service (webapi) from the Services dashboard page. You can select between two types of URLs:

HTTPS Direct (recommended) — a URL of the format https://<tenant IP>:<web-APIs port>; for example, https://default-tenant.app.mycluster.iguazio.com:8443. Requests of this format are assigned to web-APIs servers by the DNS server of the web-APIs service: the DNS cache contains a random list of web-APIs servers, and the server attempts to assign each request to the first server on the list; if the first server becomes unavailable, the request is assigned to the next server in the list, and so forth. This is the recommended method in most cases, as it's typically more efficient; a possible exception is a single web-APIs client.
HTTPS — a URL of the format https://webapi.<tenant IP>; for example, https://webapi.default-tenant.app.mycluster.iguazio.com. Requests of this format are redirected to web-APIs servers by the Kubernetes ingress of the web-APIs service, which selects a server per request.

Note

To run the examples in this reference, you must replace the sample web-APIs URL in the examples with a tenant web-APIs URL for your platform environment.

<operation>

The requested operation to be performed. For example, GetRecords.

<authentication header>

You must authenticate the identity of the sender of the request by using either an X-v3io-session-key or Authorization header, as outlined in the Securing Your Web-API Requests documentation.

<data parameters>

Data parameters for the request, where relevant, in JSON format.

Note

If the request URL doesn't include the full path to the target resource (see <resource>), the remaining path (such as a table or stream name, an item's primary key, or a shard ID) must be assigned to relevant JSON parameters in the request's HTTP body.
Note that the values of the request data parameters cannot contain path slashes (/). See Setting the Resource Path.

Setting the Resource Path

The resource path for the operation can be provided using any of the following alternative methods. The examples are for a GetItem operation that retrieves an item (resource) from a "MyDirectory/Students" table (collection) in a "mycontainer" container. The item's name and primary-key ("StudentId") value is "0358123":

The full path is provided in the request URL in the header (see <resource>).

For example, the GetItem request URL has the full path to the student item —
```
POST /mycontainer/MyDirectory/Students/0358123 HTTP/1.1
```
The resource path is split across the request URL in the header and the request body: the start of the path is set in the URL, after the container identifier, and the rest is set in the JSON body.

Note
When using this option, the path in the URL must end in a forward slash (/).

For example, the GetItem request URL sets the path to a "MyDirectory/" directory:
```
POST /mycontainer/MyDirectory/ HTTP/1.1
```
And the request's JSON body sets the TableName parameter to the name of the "Students" table (located in "MyDirectory/"), and the Key parameter to a StudentID attribute with the primary-key value "0358123" (which is also the item's name — see Item Name and Primary Key):
```
{
    "TableName": "Students",
    "Key":       {"StudentID": {"N": "0358123"}}
}
```
Another option is for the request URL to set the full path to the "Students" table —
POST /mycontainer/MyDirectory/Students/ HTTP/1.1
url = "<web-APIs URL>/mycontainer/MyDirectory/Students/"
— and for the request's JSON body to set the Key parameter to the item's primary-key attribute ("StudentID" with the value "0358123"):
{"Key": {"StudentID": {"N": "0358123"}}}
payload = {"Key": {"StudentID": {"N": "0358124"}}}

Response Syntax

The HTTP response to a web-API request includes headers that contain the HTTPS status code of the operation and additional information. Some responses also return data in the response's HTTP body, in JSON format. The responses conform to the following general format; (the order of the headers isn't important):

HTTP/1.1 <HTTP status code> <HTTP reason phrase>
Content-Type: <content type>
<last modification-time header>
...

{<data elements>}

Following is an explanation of the <...> placeholders used in the response syntax:

<content type>

The HTTP media type (content type) of the response data. For example, application/json or application/octet-stream.

<last modification-time header>

Some web-API responses return one or both of the following headers, which contain the last modification time of the operation object (such as a NoSQL table item, a stream shard, or a binary object) in different formats:

X-v3io-transaction-verifier — returns the values of the __mtime_secs and __mtime_nsecs system attributes, which together provide the object's last modification time as a Unix timestamp with nano-seconds resolution:
```
X-v3io-transaction-verifier: __mtime_secs==<Unix timesamp in seconds> and __mtime_nsecs==<nanoseconds>
```
For example:
```
X-v3io-transaction-verifier: __mtime_secs==1542838673 and __mtime_nsecs==250965583
```
Last-Modified — returns the object's last modification time as a string of the format ddd, dd MMM yyyy HH:mm:ss K:
```
Last-Modified: ddd, dd MMM yyyy HH:mm:ss K
```
For example:
```
Last-Modified: Wed, 24 Nov 2018 22:17:53 GMT
```

<data elements>

Some successful operations return, within the response HTTP body, a JSON object with additional information relating to the operation that was performed. The structure of this data object is operation-specific.
The response object might also contain additional error information for failed operations, as explained in the Error Information section.

Error Information

In the event of an error, in addition to the HTTP status code returned in the response header, the operation's response HTTP body includes an error-information JSON object. This object has an ErrorCode element that contains a unique numeric error code for the failed operation (for example, -1 or -201326592), and an ErrorMessage element that contains an error-message string (for example, "Operation not permitted" or "InvalidArgumentException"):

HTTP/1.1 <HTTP status code> <HTTP reason phrase>
Content-Type: application/json
...

{
    "ErrorCode":    <numeric error code>,
    "ErrorMessage": "<error message>"
}

For multi-object operations, error information might be returned separately for each object that produced an error (see, for example, the PutRecords response).

Examples

The following example is of an HTTP request for a GetRecords operation that retrieves records from shard 199 in a "MyDirectory/MyStream" stream within a "mycontainer" container. The path to the shard resource is set entirely in the request URL, and the request's JSON body sets the values of the Location and Limit parameters.

Request

POST /mycontainer/MyDirectory/MyStream/199 HTTP/1.1
Host: https://default-tenant.app.mycluster.iguazio.com:8443
Content-Type: application/json
X-v3io-function: GetRecords
X-v3io-session-key: e8bd4ca2-537b-4175-bf01-8c74963e90bf

{
    "Location": "AQAAAAAAAAAAAAAAAAAAAA==",
    "Limit":    2
}

import requests
url = "https://default-tenant.app.mycluster.iguazio.com:8443/mycontainer/MyDirectory/MyStream/199"
headers = {
"Content-Type": "application/json",
"X-v3io-function": "GetRecords",
"X-v3io-session-key": "e8bd4ca2-537b-4175-bf01-8c74963e90bf"
}
payload = {"Location": "AQAAAAAAAAAAAAAAAAAAAA==", "Limit": 2}
response = requests.post(url, json=payload, headers=headers)
print(response.text)

Response

The following response example is for a successful GetRecords operation that returned one matching record:

HTTP/1.1 200 OK
Content-Type: application/json
...

{
    "NextLocation": "AQAAAAsAAAAFAEC2/+sAAA==",
    "MSecBehindLatest": 0,
    "RecordsBehindLatest": 0,
    "Records": [
        {
            "ArrivalTimeSec": 1485685671,
            "ArrivalTimeNSec": 160186781,
            "SequenceNumber": 15756,
            "PartitionKey": "MyKey",
            "Data": "ZGF0YQ0K"
        }
    ]
}

The following response example is for a failed GetRecords operation that returned HTTP status code 400 (Bad Request) and web-API error code -201326594 and error message ShardIDOutOfRangeException, indicating that the shard ID specified in the request (199) doesn't exist in the specified stream ("MyStream"):

HTTP/1.1 400 Bad Request
Content-Type: application/json
...

{
    "ErrorCode":    -201326594,
    "ErrorMessage": "ShardIDOutOfRangeException"
}

Data-Service Web-API General Structure

Request Syntax

Response Syntax

Error Information

Examples

See Also