create Method

On This Page

Description

Creates a new TSDB table.

Syntax

create(backend, table, if_exists=FAIL, **kw)

The following syntax statement replaces the kw parameter with the additional keyword arguments that can be passed for the NoSQL backend via this parameter:

create(backend, table, if_exists=FAIL, rate[, aggregates, aggregation_granularity])

Note

The method has additional parameters that aren't currently supported for the TSDB backend. Therefore, when calling the method, be sure to explicitly specify the names of all parameters after table.

Parameters

aggregates (kw argument) | aggregation_granularity (kw argument) | backend | kw | if_exists | rate (kw argument) | table

backend

The backend type — "tsdb" for the TSDB backend. See Backend Types.

  • Type: str
  • Requirement: Required
table

The relative path to the backend data — a directory in the target data container (as configured for the client object) that represents a TSDB table. For example, "mytable" or "examples/tsdb/my_metrics".

  • Type: str
  • Requirement: Required
if_exists

Determines whether to raise an error when the specified TSDB table (table) already exists.

  • Type:

    pb.ErrorOptions enumeration. To use the enumeration, import the frames_pb2 module; for example:

    from v3io_frames import frames_pb2 as fpb
  • Requirement: Optional
  • Valid Values: FAIL to raise an error when the specified table already exists; IGNORE to ignore this
  • Default Value: FAIL
kw

This parameter is used for passing a variable-length list of additional keyword (named) arguments. See the following kw Arguments section for a list of additional arguments that are supported for the TSDB backend via the kw parameter.

  • Type: ** — variable-length keyword arguments list
  • Requirement: Required

kw Arguments

The TSDB backend supports the following create arguments via the kw parameter for passing a variable-length list of additional keyword arguments:

rate

The ingestion rate of the TSDB metric samples. It's recommended that you set the rate to the average expected ingestion rate for a unique label set (for example, for a single server in a data center), and that the ingestion rates for a given TSDB table don't vary significantly; when there's a big difference in the ingestion rates (for example, x10), consider using separate TSDB tables.

  • Type: str
  • Requirement: Required
  • Valid Values: A string of the format "[0-9]+/[smh]" — where 's' = seconds, 'm' = minutes, and 'h' = hours. For example, "1/s" (one sample per minute), "20/m" (20 samples per minute), or "50/h" (50 samples per hour).
aggregates

A list of aggregation functions ("aggregators") for performing over-time aggregation in real time during the metric-samples ingestion ("pre-aggregation"). Over-time aggregation returns an aggregation time series for each unique metric label set. The aggregations results are stored in the TSDB table as array attributes ("pre-aggregates") and used to handle relevant aggregation queries. Use the aggregation_granularity argument to configure the table's pre-aggregation granularity — the time interval for applying these aggregations.

Note

You can also perform aggregation queries for TSDB tables without pre-aggregates, but when configured correctly, pre-aggregation queries are more efficient. To ensure that pre-aggregation is used to process aggregation queries and improve performance —

  • When creating the TSDB table, set its aggregation granularity (aggregation_granularity) to an interval that's significantly larger than the table's metric-samples ingestion rate (rate).
  • When querying the table using the read method, set the aggregation window (which is currently the aggregation step — see the step parameter) to a sufficient multiplier of the table's aggregation granularity.

For example, if the table's ingestion rate is 1 sample per second ("1/s") and you want to use hourly queries (i.e., use a query aggregation window of "1h"), you might set the table's pre-aggregation granularity to 20 minutes ("20m").

For more information about aggregation queries, see the read method.

  • Type: str
  • Requirement: Optional

  • Valid Values: A string containing a comma-separated list of supported aggregation functions; for example, "count,avg,min,max". The following aggregation functions are supported:

    • avg — the average of the sample values.
    • count — the number of ingested samples.
    • last — the value of the last sample (i.e., the sample with the latest time).
    • max — the maximal sample value.
    • min — the minimal sample value.
    • rate — the change rate of the sample values, which is calculated as <last sample value of the previous interval> - <last sample value of the current interval>) / <aggregation granularity>.
    • stddev — the standard deviance of the sample values.
    • stdvar — the standard variance of the sample values.
    • sum — the sum of the sample values.
aggregation_granularity

The TSDB table's pre-aggregation granularity — i.e., the time interval for executing the aggregation functions that are configured in the aggregates argument for real-time aggregation during metric-samples ingestion.

Note

To get the expected pre-aggregation performance impact, the size of the aggregation-granularity interval (as set in the aggregation_granularity argument) should be significantly larger than the size of the table's metric-samples ingestion rate (as set in the rate argument). See the aggregates argument notes.

  • Type: str
  • Requirement: Optional
  • Valid Values: A string of the format "[0-9]+[mhd]" — where 'm' = minutes, 'h' = hours, and 'd' = days. For example, "30m" (30 minutes), "2h" (2 hours), or "1d" (1 day).
  • Default Value: "1h" (1 hour)

Errors

In case of an error, the method raises a CreateError error.

Examples

Following are some usage examples for the create method of the Frames TSDB backend:

  1. Create a TSDB table named "mytsdb" in the root directory of the client's data container (table) with an ingestion rate of 1 sample per second (rate):

    tsdb_table = "mytsdb"
client.create(backend="tsdb", table=tsdb_table, rate="1/s")

  2. Create a TSDB table named "my_metrics" in a tsdb directory in the client's data container (table) with an ingestion rate of 12 samples per hour (rate). The table is created with the count, avg, min, and max aggregation functions (aggregates) and an aggregation granularity of 1 hour (aggregation_granularity):

     tsdb_table = "/tsdb/my_metrics"
 client.create("tsdb", table=tsdb_table, rate="12/h",
                      aggregates="count,avg,min,max",
                      aggregation_granularity= "1h")

See Also

Bring Your Data Science to Life

© Iguazio Ltd. - All Rights Reserved

© Iguazio Ltd. - All Rights Reserved

Privacy policy | Terms of use