MQTT Integration

Aug 22, 2023

Introduction

Integration

This document focuses on one of Pani’s data connectivity options, MQTT integration. This integration technique allows your site’s IT department to select and control what data is sent to Pani and when.

The three, high-level steps involved in this process are outline in the diagram below. The steps are:

Consolidation
- A member of your team writes a script to query your local data stores.
Integration
- The consolidated data is pushed to Pani via MQTT integration.
Service Usage
- Pani’s system processes your data and presents an interface for analyzing your ingested data alongside derived process intelligence.

Fig. 1 MQTT Integration

MQTT

The following sections provide information about how the MQTT functions and is for anyone that has basic experience with scripting and networking. These are the specifics for setting up:

Getting Started

The structure of a valid HTTP request to our API for data integration consist of the following:

Broker
Client
Credentials
- CA Certificate
- Client Certificate
- Private Key
Payload

To use MQTT, additional setup is required with Pani’s CS team to ensure configuration is complete prior to data being submitted to our data servers. Contact [email protected] for details on access.

MQTT must conform to the following specification.

Broker

The MQTT broker is the endpoint on the Internet where messages are sent. The Pani broker is hosted on AWS and uses TLS for security and client identification.

a1l4r9ehdvr5cz-ats.iot.us-east-1.ama

Client

There are many different MQTT clients available. This document will demonstrate 2, Eclipse Paho and MQTT Explorer.

Credentials

There are 3 credential files required to connect to the broker.

CA Certificate

The CA certificate is provided by AWS. It will be named “root-CA.crt”.

Client Certificate
The client certificate is provided by Pani. It will be named “<plant_name>.cert.pem”.

Private Key
The private key is provided by Pani, should be kept secret and only installed on the client device. It will be named “<plant_name>.private.key”.

Payload

The request payload must conform to the following:

Must be in JSON format.
Must include a “deviceId” field with a JSON string value.
Must include a “timestamp” field whose value is an ISO 8601-formatted JSON string.
All fields other than "deviceId" and "timestamp" represent sensor readings or tags, and their values must be numeric and in the JSON format. Unlike deviceId and timestamp, sensor/tag values are not enclosed in quotes.

{
       "deviceId": "string",
       "timestamp": "string (iso8601 format)",
       "tag1": val1 (number),
       "tag2": val2 (number),
       "tag3": val2 (number)
}

Null Values
There may be situations where you may want to use a null value, such as when a system is offline with no data versus a value of zero, which is a valid value. The Tag name must be in quotes, and the tag value must be a numeric value or the word null, none of them in quotes, and conforming to the payload formatting.

NOTE: Multiple tags can be passed in a payload, and multiple timestamps can be passed as well. They can be sent in an array as shown in Figures 4 and 5 below.

IMPORTANT! The "timestamp" and "deviceId" pair create a unique key. Each tag can only be sent once with this unique key. Multiple payloads may use the unique key but not duplicate tags.

The payload shape that is expected can be seen in the following:

NOTE: If no timezone info is included e.g. (missing the "Z" or "+00" ISO8601 qualifiers) it will be assumed to be UTC.

Examples:

[

{

"deviceId": "1234-546-98",

"timestamp": "2020-11-23T23:43:36Z",

"tag1": 01,

"tag2": 02,

"tag3": 03

}

]

Fig. 2 Payload example 1

[

{

"lmdc051wor_val0": 29772,

"lmdc052wor_val0": 29590,

"lmdc053wor_val0": 760,

"timestamp": "2021-11-01-01T12:10:15+00",

"deviceId": "Plant1"

}

]

Fig. 3 Payload example 2

[

{

"tmtag": "799733.01",

"pivalue": 5.771,

"deviceId": "1",

"timestamp": "2022-10-01T12:09:00+00:00"

{

"tmtag": "437963891.01",

"pivalue": 8260,

"deviceId": "1",

"timestamp": "2022-10-01T12:08:00+00:00"

}

]

Fig. 4 Payload example 3

[

{

"80bfb2-dcwf936af481": 27.42,

"deviceId": "82323a3-4fd2-4fff40164fd26b81",

"timestamp": "2021-09-01T15:57:59Z"

{

"71ef64-xyz870dd3b2e0": 87.73,

"deviceId": "207823a-b52e-40164f40164fd4f",

"timestamp": "2021-09-01T15:57:59Z"

}

]

Fig. 5 Payload example4

{

"D208": 77318.49,

"H212": -0.927,

"deviceId": "2391-6008-57",

"timestamp": "2021-08-026 15:46:05"

}

Fig. 6 Payload example5

Eclipse Phao (Sample Code)
The following code presents an example script written in Python that could be used to push data via MQTT integration.

MQTT Explorer Test Client

There are many MQTT client programs that can be used to test the connection. One example of such a program is called MQTT Explorer. It can be downloaded for free from the Internet.

Before it can be used it must be configured. Create a new connection and configure it as follows:

Host: a1l4r9ehdvr5cz-ats.iot.us-east-1.amazonaws.com
Encryption (tls): True
Port: 8883

Fig. 7 Connection Settings

Remove the default topic subscriptions; “#” and “$SYS/#”. The broker will disconnect the client if these topics are subscribed to.

Subscribing to topics is optional but can be helpful in troubleshooting publishing.

Client Id can be anything but must be unique on the broker. Using the Plant Id plus a unique identifier should ensure a unique name.

Subscription: iot/<plant_name>
MQTT Client ID: <plant_name>_<unique_id>

Fig. 8 Advanced Settings

Use the credentials provided by Pani.

Fig. 9 Certificate Settings

Publishing to the topic can be tested from the client. Use the correct topic and properly formed json payloads to test.

If configured correctly, messages published to the broker will appear on the left side.

Topic: iot/<plant_name>

Fig. 10 Connected

Ingress Dashboard

Using the Ingress Dashboard on the Pani Platform (accessible from the Admin Configure page), you can verify the current connection and payload status.

Fig. 11 Ingress Dashboard