Adding a Custom Grafana Dashboard
The platform includes a pre-deployed version of the open-source Grafana analytics and monitoring platform.
For detailed information about working with Grafana, see the Grafana documentation.
Grafana allows the creation of custom web dashboards that visualize real-time data queries.
This tutorial outlines the steps for creating your own Grafana dashboard for querying data in the your cluster’s data containers by using the custom
iguazio Grafana data source, or by using a Prometheus data source for running Prometheus queries on platform TSDB tables.
You can repeat this procedure to create as many dashboards as you wish.
To use Grafana to visualize data in the platform, you first need to create a new Grafana service from the dashboard
Servicespage. For general information about creating new services, see creating a new service.Note
To add the
iguaziodata source to your Grafana service, you must set the service’s
Platform data-access userparameter to a platform user whose credentials will be used for accessing the data in the platform.
To use a Prometheus data source, you first need to create a TSDB table and a Prometheus service that’s configured to query this table.
When you create a new Grafana service, it automatically includes data sources for all existing platform Prometheus services that are accessible to the running user of the Grafana service. However, to add a Grafana data source for a new Prometheus service that’s create after the creation of the Grafana service, you need to do one of the following:
Restart the Grafana service. This will update the service’s data sources to include all relevant existing Prometheus services. However, note that any Grafana UI changes that were previously made for this service will be lost.
Manually add the data source for the Prometheus service:
- Open the dashboard
Servicespage and copy the HTTPS URL of the target Prometheus service from the UIcolumn.
- Open your Grafana service by selecting the service’s HTTPS link from the
- From the Grafana side navigation menu, select the configuration gear-wheel icon
()to display the Configuration > Data Sourcetab.
Add Data Source, and then select Prometheusfrom the data-sources list.
- Optionally edit the default data-source name in the
Namefield (for example, to use the name of the selected Prometheus service).
- In the
URLfield, paste the Prometheus service URL that you copied in the first step.
- Optionally set additional configuration parameters, and then select
Save & Test.
- Open the dashboard
To successfully submit queries and visualize data with Grafana, you need to have data in you cluster’s data containers that matches your queries. The instructions in this tutorial demonstrate how to create a dashboard that queries the following tables; to query other tables, you’ll need to adjust your dashboard configuration, and especially the query parameters, to match the path and attributes of an existing table in your cluster.
bank_nosqlNoSQL table in the “bigdata” container. You can create a compatible table by following the instructions in the Converting a CSV File to a NoSQL Table tutorial.
sample-tsdbTSDB table in the “bigdata” container. You can create such a table by using the TSDB CLI ( tsdbctl). For example, you can run the following command from a web-based shell or a Jupyter terminal to create the table used in the tutorial examples:
tsdbctl create -c bigdata -t sample-tsdb -r 1/s
After creating the TSDB table, ingest some sample metrics into the table so that you have data to visualize with Grafana. You can do this by running the V3IO TSDB CLI or by using a Nuclio TSDB ingestion function. For detailed TSDB CLI usage instructions, see the Working with the TSDB CLI tutorial.
Using the iguazio Data Source
In the new-panel
Addtab, select the desired visualization type. In this step, select Table. (A graph example is provided in Step 11.)
In the first metrics-query row (
A), select the query-type drop-down arrow and select the query type. You can select between timeseriesfor querying a time-series database (TSDB), and tablefor querying a standard NoSQL table. In this step, select table. (A timeseriesexample is provided in Step 12.)
Select the drop-down arrow for the query-parameters box (next to the query type) and enter your query parameters. The parameters are entered using the syntax
<parameter name>=<parameter value>; [<parameter name>=<parameter value>; ...].NoteTo see the query parameters in the UI, select the query fly-out menu icon
()and then select the Toggle Edit Modeoption from the menu. To run the defined queries, select the refresh button ()from the top menu toolbar.
tablequeries, you must set this parameter to “
timeseriesqueries, you must set this parameter to “
tablequeries, this parameter should be set to a comma-separated list of item-attribute names — for example, “
timeseriesqueries, this parameter should be set to a comma-separated list of sample metric names — for example, “
temperature” or “
You must set either the
fieldsor filterparameter; you can also select to set both parameters.
filter— a filter expression that restricts the query to specific items.
tablequeries, the filter is applied to the names of item attributes (table columns). For example,
timeseriesqueries, the filter is applied to the names of labels and/or metrics. To reference a metric name in the filter, use the
__name__attribute. For example, “
starts(__name__,'metric') AND device==1”.
As indicated, you must set the
Enter the following query parameters to query selected attributes in the “bank_nosql” table in the root directory of the “bigdata” container, or edit the parameters to match the data in another NoSQL table in your cluster:
backend=kv; container=bigdata; table=zeppelin_getting_started_example/%!s(<nil>); fields=age,job,marital,education,balance,housing,loan,contact,day,month,duration;
You can also add additional queries, if you wish.
From the top-menu toolbar, select the save icon
(). Enter a new name for your dashboard, optionally change the dashboard folder, and select Save. For example, set the name to “Iguazio Dashboard” and save the dashboard in the default “General” folder.TipInstead of selecting the save icon from the toolbar, you can select the settings icon ()to display the Settingspage. In addition to setting the dashboard name and folder, this page lets you make other configurations; for example, you can define variables to use in your query parameters.
You now have a “Bank” dashboard panel with a table that provides real-time visualization of the data in the “bank_nosql” table in the “bigdata” container.
In the following steps, you’ll add a TSDB graph to your dashboard.
From the top-menu toolbar, select the add-panel icon
() to display the new-panel. This time, select the Addtab that you saw in Step 3 Graphvisualization type.
Panel Title | Editmenu option (as you did in Step 4). In the Metricstab, select iguazioas the data source (as you did in Step 5), but this time select the timeseriesquery type (unlike the tableselection in Step 6).
Select the drop-down arrow for the query-parameters box (next to the query type) and enter your query parameters using the syntax
<parameter name>=<parameter value>; [<parameter name>=<parameter value>; ...]— similar to what you did in Step 7, except that you need to enter time-series query parameters for a TSDB table; see the supported query parameters in Step 7.NoteAs explained for the table query, you can toggle the query edit mode from the query menu
()to see the parameters in the UI, and use the refresh button ()in the top menu toolbar to run the query.
Enter the following query parameters to query “metric0” samples in the “sample-tsdb” table in the root directory of the “bigdata” container, or edit the parameters to match the data in another TSDB table in your cluster:
backend=tsdb; container=bigdata; table=sample-tsdb; fields=metric0;
You can also add additional queries, if you wish.
Generaltab and edit the panel title, as you did in Step 8, but this time enter another title — for example, “Sample TSDB”.
You now have a “Sample TSDB” dashboard panel with a graph that provides real-time visualization of the data in the
sample-tsdbTSDB table in the “bigdata” container.
Go to the Grafana home page and select your “Iguazio Dashboard” dashboard from the list. You’ll see your two panels on the dashboard. Note that you can change the time range for the visualization from the top-menu toolbar. You can also customize the dashboard by resizing panels and dragging them to change their arrangement. The following image demonstrates a dashboard that displays the panels that you defined in this tutorial:
Using a Prometheus Data Source
Log into your Grafana service and select your Grafana dashboard. To edit an existing dashboard, open the dashboard and then, from the top-menu toolbar, select the add-panel icon
()to display the new-panel Addtab. To create a new dashboard, follow Steps 2–4 of the
iguaziodata-source instructions; select the visualization type that suites your need (such as table or graph).
Generaltab and edit the panel title, as you did in Step 8, but use another title — for example, the name of your Prometheus data source.
Go to the Grafana home page and select your dashboard from the list. You should be able to see your new Prometheus panel on the dashboard.