The Data Fabric
The platform has a built-in multi-model data layer (a.k.a. “data store” or “database”) for storing and analyzing various types of data structures — such as NoSQL (“key/value”) tables, time-series databases, data streams, binary objects, and files. The data can be accessed through multiple industry-standard and industry-compatible programming interfaces; users can ingest data through one interface and consume it through another interface, depending on their preferences and needs. This unique unified data model eliminates the need for multiple data stores, constant synchronization, complex pipelines, and painful extract-transform-load (ETL) processes. The following table shows the provided programming interfaces for working with different types of data in the platform’s data store:
|NoSQL (Wide-Column Key/Value) Data||
The platform’s NoSQL data store was built to take advantage of a distributed cluster of physical and virtual machines that use flash memory to deliver in-memory performance while keeping flash economy and density. You can access NoSQL data through these interfaces:
You can work with SQL data in the platform through these interfaces:
You can create and manage time-series databases in the platform’s data store through these interfaces:
You can stream data directly into the platform and consume data from platform streams through the following interfaces:
|File / Simple Data Object||
You can work with data files and simple data objects — such CSV, Parquet, or Avro files, or binary image or video files — through these interfaces:
- See APIs Overview for a summary of the platform APIs; the Platform Fundamentals tutorial for explanations on how to set the data paths for each API; and References for comprehensive references.
- See the platform’s tutorial Jupyter notebooks for code examples and full use-case applications that demonstrate how to use the different APIs.
- The platform’s web APIs (for working with NoSQL, streaming, and simple-object data) are exposed as an application service.
The API endpoint URL of this service is available from the dashboard
Servicespage. For an introduction to working with the web APIs, see Sending Web-API Requests in the Platform Fundamentals tutorial.
- See also the Application Services and Tools introduction for information on related application services — and specifically Spark, Presto, Time-Series Database (TSDB) Services, pandas and Dask, and V3IO Frames.