Thursday, June 7, 2018

LabPlot getting support for MQTT

Hello everyone! I'm participating in Google Summer of Code, I'm working on KDE's LabPlot. At first, let me introduce you my project:
"Currently, LabPlot has some support for the plotting of live data, which can be read from file/named pipe, TCP socket, UDP socket, local socket and serial port. The goal of this project is to provide support for the plotting of data received via MQTT protocol from a certain broker. LabPlot currently doesn’t provide any support for MQTT, even though it would be quite important for LabPlot to provide this kind of support, since it is a scientific data plotting software and MQTT protocol is widely used to transmit certain sensor data.
This project would increase LabPlot’s usability regarding analyses of scientific data, monitoring data collected by sensors, that is transmitted by MQTT protocol, or even make LabPlot usable in the field of Internet of Things or Smart Home Appliances."
If you are more interested in the project, you can check out my proposal, to find out in detail what the project really is about.
It is also important to know that this project is based on my mentor's work, who implemented the live data source functionality. To find out more check out: http://krajszgsoc.blogspot.com/2017

I can happily tell you that I managed to adapt quite a few of the already existing features of live data sources to MQTT protocol.
But let's start with the beginning. The very first task was to update the Import File Widget. Using the widget the user can set:
  • the MQTT client's host and port to the host and port of the broker, which the user wants to connect to,
  • the username and password (if authentication is required by the broker) 
  • the client ID. 
Setting the MQTT client options

After successfully connecting to the broker. The MQTT client subscribes to the "#" wildcard. This means that the client gets every message published on the broker, so we can add every active topic to a combo box. The user can choose from these topics the ones the MQTT client will subscribe to. Since lots of topics are added to the combo box, by starting to type the topic's name we can narrow down the list the user has to choose from. Subscribing and unsubscribing to a topic is also implemented.


The next logical step is to offer a preview of the data, that is received via MQTT messages, so the user is able to see what kind of data is transmitted in every topic that the user subscribed to. This is useful, because the user can decide whether the expected data was received or not. If not the user can unsubscribe from the topic, as shown in the previous video.





The next step was adapting already existing options and functionality of live data sources to MQTT protocol. In the already existing implementation every data was read from one QIODevice, data for every column was present in every line read from the QIODevice. However this method can't be used for MQTT, since data for every column is received from individual topics in multiple messages. These messages may or may not be sent synchronously. After discussing the matter with my mentor we decided that we will enforce synchrony. What does this mean? It means that we only process a set of data, if the client received a message for every subscribed topic. The next problem is that these topics are completely INDIVIDUAL, so they may send totally different amount of data (this amount of data may differ from message to message as well given one topic). After talking this problem through with my mentor, we decided on implementing the features assuming that every set of messages (the currently processed messages, one from each topic) contains the same amount of data, the amount may differ from one set to another (for starters at least).

Here's a list of the options I adapted to MQTT:
  • Reading type
    • Continuously fixed
    • From end
    • Till the end
  •  Update type
    • Periodically
    • On new data
  • Keep last N values
  • Update now
  • Pause/Continue
  • Update interval
  • Sample rate
I will demonstrate these features with some videos, using brownians motion data, provided by another Qt application I wrote:

This Video Presents how Read Type and Sample rate work
This video presents how the update types and update interval work



This video presents the keep last N values option


This video presents the Pause/Continue and Update now options

The next steps most probably will be investigating for further errors the already adapted options, setting will messages for the client using some statistical data, run time subscribe/unsubscribe, probably thinking through and finding a solution for sets of messages with not equal amount of data, and of course any new idea that may come up regarding the project.

This is it for now. I will continue to work on the project with my mentor Kristóf, who really helps me a lot, and if I'm not sure about something he guides me. When anything new will be finished and running I'll let you know.

See you soon!