ARISE PoC Guide

Overview

The ARISE Proof of Concept (PoC) is a FIWARE-based Minimum Viable Platform (MVP) composed of various Docker containers, which encapsulate the application code along with all its dependencies to ensure it runs consistently and reliably across different computing environments. This containerized approach simplifies deployment, eliminates dependency conflicts, and provides a scalable platform for testing and validation.

This tutorial is a step-by-step guide that demonstrates how to set up the necessary components and workflows, in order to enable the ARISE PoC to connect seamlessly with various information sources. It also provides the means to test a wide range of use cases (UCs) defined and implemented in the Test and Experimentation Facilities (TEFs).

ARISE PoC Schema

Actors

The actors involved in the scenario are:

OPC UA Server

It represents the data source utilized by the TEF.

IoT Agent OPC UA

The connector through which to join the industrial environment to this FIWARE-based platform can be configured as described in Documentation and in GitHub. In order to start working with the above mentioned OPC UA server, configuration files have been already edited and made available in the conf folder.

ROS 2 and Fast DDS

This connector implements both an Application Programming Interface (API) and a communication protocol that deploys a Data-Centric Publisher-Subscriber (DCPS) model, with the purpose of establishing efficient and reliable information distribution among Real-Time Systems. More information can be found in Documentation and in GitHub

Orion LD Context Broker

It is the entry point of each FIWARE platform that adopts the NGSI LD data model. It can be external, however, in order to ensure a self-contained and self-consistent testing environment, it is included in this PoC as part of the Docker Compose setup. More information can be found in Documentation and in GitHub

MongoDB

The database that saves the current state of the data passed through the context broker. It is used by the Orion LD to hold context data information such as data entities, subscriptions and registrations. More information can be found in Documentation and in GitHub

Mintaka

It is a Generic Enabler (GE) of the FIWARE ecosystem designed for time series data storage and management. It serves as a lightweight, scalable API that allows the ingestion, querying, and analysis of historical context information from FIWARE Context Brokers. More information can be found in Documentation and in GitHub

TimescaleDB

It is a time series database built on PostgreSQL, designed to handle large-scale, time-stamped data efficiently. It offers powerful SQL support for managing time series data, with features like automatic data partitioning, real-time aggregation, and advanced query capabilities. More information can be found in Documentation and in GitHub

Grafana

It is an open-source analytics and visualization platform that allows users to visualize data stored in TimeScaleDB through customizable dashboards. It supports data representation using charts, graphs, tables, and alerts. Besides TimeScaleDB, Grafana integrates with a wide range of data sources, including Prometheus, InfluxDB, MySQL, PostgreSQL, Elasticsearch, and more. It is released under the Apache License 2.0. More information can be found in Documentation and in GitHub.

Step-by-step Guide

This section outlines the process for rapidly deploying a fully functional testbed that includes all relevant actors.

Requirements

  • Docker (Version 26.1.5)

  • Docker-compose (Version 1.29+)

  • RAM 8GB minimum

Install docker and docker-compose by following the instructions available on the official web site:

Verify Installation: Run the following commands to check Docker and Docker Compose:

docker --version
docker compose --version

Step 1 - Clone the ARISE PoC

Open a terminal and move into a folder in which to create the new folder containing the ARISE PoC components.

Then run:

git clone "https://github.com/Engineering-Research-and-Development/arise-poc.git"

Step 2 - Configure the ARISE PoC

Before launching the ARISE PoC, it is essential to edit the docker-compose.yaml file and configure IoT Agent OPC UA service to align with your specific OPC UA Server’s specifications.

cd arise-poc/
nano docker-compose.yaml

Here are some environment variables that must be configured:

  • IOTA_CB_SERVICE allows to define where the OPCUA IoT Agent sends NGSI-LD context data;

  • IOTA_SERVICE defines the devices are registered;

  • IOTA_OPCUA_ENDPOINT is used in FIWARE IoT Agent for OPC UA to specify the OPC UA server endpoint.

For a more complete description on how to configure the IoT Agent, go to link.

Step 3 - Build & Run the ARISE PoC

To launch the whole PoC:

docker compose up --build -d

After that you can run:

docker ps

to check if all the required components are running.

Container view

Before starting the ROS 2 demo, as X11 session owners, other users must be allowed to use the X Window System (to show the turtles on screen). For that, the following command is executed:

xhost local:root

When the dockers have started, connect to bash in the container of ROS 2:

docker exec -ti ros2 bash

Now the turtlesim node and the keyboard controller can be started:

source /ros2-ws/install/setup.bash

# Show the turtles on the screen
ros2 run docs_turtlesim turtlesim_node_keys &

# Keyboard controller to move the turtles.
ros2 run docs_turtlesim turtlesim_multi_control
turtlesim view

Step 4 - Access the Grafana Dashboard

In the context of the ARISE PoC, Grafana is used to visualize and analyze the data collected from OPC UA and ROS 2 devices, offering an intuitive interface to monitor the performance, and trends of connected systems. This tool plays a critical role in helping users test various use cases by providing a clear view of the data flow and operational metrics. Grafana is distributed under the Apache License 2.0, ensuring that it is free to use, modify, and distribute. Its active open-source community and extensive documentation make it accessible to users of all expertise levels. For more details, including installation guides, plugins, and advanced configuration options, visit the official Grafana website at https://grafana.com/.

For this PoC, Grafana is accessible at the link https://localhost/login using the default credentials admin/admin

Configuring a Data Source

In Grafana the Timescale data source can be configured using the datasources.yaml file, which is part of Grafana’s provisioning system. This allows for automated setup and consistent configuration of the data source when Grafana starts. TimescaleDB, a powerful time-series database built on PostgreSQL, integrates seamlessly with Grafana, enabling efficient visualization of time-series data.

To configure the TimescaleDB data source, follow these steps:

cd arise-poc\conf\grafana\datasources
nano datasources.yaml

apiVersion: 1

datasources:
  - name: orion
    type: grafana-postgresql-datasource
    url: timescale:5432
    user: orion
    secureJsonData:
      password: ''
    jsonData:
      database: orion
      sslmode: 'disable'

  - name: orion_cartif
    type: grafana-postgresql-datasource
    url: timescale:5432
    user: orion
    secureJsonData:
      password: ''
    jsonData:
      database: orion_cartif
      sslmode: 'disable'

  - id: 3
    uid: "aee7znssxxvr4c"
    orgId: 1
    name: "yesoreyeram-infinity-datasource"
    type: "yesoreyeram-infinity-datasource"
    url: ""
    basicAuth: false
    basicAuthUser: ""
    isDefault: false
    jsonData:
      global_queries: []
    readOnly: false
    apiVersion: ""

When the ARISE PoC is executed, this datasource.yaml file is mounted into the Grafana’s provisioning folder to ensure proper configuration and access to the necessary data sources.

Creating a new Dashboard

Creating a new dashboard in Grafana allows you to visualize and analyze your data effectively. Here’s a step-by-step guide to help you set up a new dashboard:

  • Log in Grafana instance and click on the “+” icon in the top right menu, selecting “New Dashboard”;

    Select New Dashboard

  • Click on “Add a new panel” and in the Query section, choose your data source.

    Add Visualization Select DataSource

  • Construct your query to fetch the desired data, selecting the appropriate visualization type (e.g., graph, table, gauge) from the options. More information on how to structure queries useful for creating dashboards can be found in the Example Query Dashboard section of the present tutorial

    Insert Query Selecting the appropriate visualization type

  • To configure Panel Settings, you adjust the visualization settings such axes, legends, and thresholds

  • Save the Panel and the Dashboard

Example Dashboard in ARISE PoC

For the ARISE Proof of Concept (PoC), example dashboards have been developed to graphically represent data from both OPC UA devices and ROS 2 devices.

Dashboard for data from an OPCUA device

As shown in the figure, this example dashboard is based on some of the data produced by an OPCUA device provided by TEF of CARTIF for the ARISE research project. Specifically, the following parameters have been selected and represented from the entity implemented on the device: ALARM, Alarm1, Alarm2, Area, Target Position, and Step Data Out.

Dashboard TEF1-CARTIF

Dashboard for data from an ROS 2 device

This example dashboard represents the data of the entity defined in a ROS 2 device. Specifically, for the ARISE research project, three turtlesim instances have been implemented as ROS 2 devices, and for each of them, the following data is represented:

  • Turtle Pose: This includes the turtle’s linear velocity, its position coordinates (X and Y), and the orientation angle (Theta). These parameters provide crucial information about the turtle’s current location and direction of movement in the coordinate system.

  • Turtle Linear Velocity: This metric indicates the speed at which the turtle is moving in a straight line. It is essential for understanding the turtle’s movement dynamics and can be visualized as a gauge or graph, showing real-time changes in speed.

  • Turtle Angular Velocity: This metric represents the speed of rotation around the Z-axis. Angular velocity is critical for analyzing how quickly the turtle can change its direction, which is particularly important in dynamic environments or when navigating complex paths.

By visualizing these parameters on the dashboard, users can monitor the performance and behavior of the turtlesim devices in real-time. The dashboard facilitates a comprehensive overview of the turtles’ movements, allowing for effective analysis and decision-making during experiments.

Dashboard turtlesim

Dashboard for historical data using Mintaka

To building dashboards with historical data, the Grafana instance in ARISE PoC integrates the “Grafana Infinity Datasource” plugin, which allows external APIs to be added as data sources, supporting various formats such as JSON, XML, and more.

More information about this plugin are available at the following link

The APIs to access historical data are provided by Mintaka, a FIWARE Generic Enabler (GE), which implements an APIs to access the retrieval of temporal data in NGSI-LD format. Also it allows transparent to the user, the database used for historicization.

More information about Mintaka and the implemented APIs are available at the following link

The Mintaka API that was chosen has the following syntax:

http://<hostname_mintaka_instance>:<port>/temporal/entities/<entityId>?timeproperty=modifiedAt&timerel=between&timeAt=<"start date in the ISO 8601 format YYYY-MM-DDT:HH:MM:SS">&endTimeAt=<"End date in the  ISO 8601 format YYYY-MM-DDT:HH:MM:SS">

Creating a dashboard for the graphical representation of historical data follows the same steps described in the “Creating a New Dashboard” section. However, there are a few minor variations to consider, such as:

  • Select the yesoreyeram-infinity-datasource installed by default in Grafana instance in ARISE PoC

    Historical Dashboard - Grafana Configuration - Choose Datasource

  • Define the Endpoint from which the data will be fetched. Specifically, add the URI of the selected Mintaka API, properly configured as shown in the example.

    http://mintaka_opcua:8081/temporal/entities/urn:ngsi-ld:Device:servidor_1DBRVC?timeproperty=modifiedAt&timerel=between&timeAt=2024-01-01T00:00:00&endTimeAt=2026-01-01T00:00:00

  • Configure HTTP Headers: Add the necessary HTTP headers, such as Accept and Content-Type specifications, to ensure proper communication with the data source.

    Historical Dashboard - Grafana Configuration - Configure URI and HTTP Headers

Based on the JSON obtained from the Mintaka API, it is possible to identify the key/column of interest which is selected to extract the desired information. As shown in the following figure, the key/column “Alarm” has been selected. It determines the main data structure from which the information will be extracted.

Historical Dashboard - Grafana Configuration - Parsing Rules

The Historical Dashboard obtained is the one shown in the following figure

Historical Dashboard - Grafana Configuration - Configure URI and HTTP Headers

Based on the JSON obtained from the Mintaka API, some transformations may be needed using the Grafana plugin as shown in the following figures for the historical data used in ROS 2 dashboards.

Historical Dashboard - Grafana Configuration - Configure URI and HTTP Headers Historical Dashboard - Grafana Configuration - Configure URI and HTTP Headers

Grafana Alerting System in ARISE PoC

For the ARISE PoC, the Grafana alert system is activated to monitor specific events or circumstances. This system allows monitoring defined metrics by creating queries and expressions from multiple data sources, defining thresholds and alarm messages.

In this context, the Contact Point, i.e., the notification distribution channel, must therefore be defined. Among these channels, we have Telegram, Microsoft Teams, Slack, Discord, etc.

As an example, Telegram was chosen, and the BOT API Token and Chat ID were inserted.

More details can be found at the link.

Step 1 - Define Contact Point

Select the Alerting from the Grafana menu and then Contact Points. Fill in the fields shown in the figure with the Bot API Token and Chat ID obtained from Telegram link

Select Alerting/Contact Points menu Define Contact Points

Step 2 - Define Alert Rule

Alert Rules have a dashboard that summarizes the status of the Alerts on the monitored data, the show data history, the time interval for which the rule is applied and other parameters.

Dashboard Alert Rule

Click on “New Alert Rule” to define a new Rule, enter the name, define the query and alert condition,

Query and Alert Condition

Define a new threshold, specifying both the parameter to be evaluated (Input) and the condition for exceeding the threshold. Specify a folder where the alert rules will be saved, set an evaluation behavior to define how the alert rule will be evaluated. Finally, select the previously defined contact point.

Threshold and Contact Point

It is possible to configure and customize the notification message that will be forwarded via the selected contact point

Configure Notification Message

Step 3 - ARISE Notification in Telegram

When the thresholds defined on Grafana are exceeded, alert messages are sent to the ARISE Alert Project Notification channel

ARISE Alert Notification