LabPlot getting prettier and also support for online datasets

Introduction

Hello everyone! I'm participating in Google Summer of Code for the second time. I'm working on KDE's LabPlot, just like last year. I'm very happy that I can work again with my former and current mentor Kristóf Fábián, and with Alexander Semke, an invaluable member of the LabPlot team, who is like a second mentor to me. At first, let me introduce you my current project: 

"There are many internet pages providing data sets for educational and academic purposes concerning various fields of science, and not only (astrophysics, statistics, medicine, etc.). Some tools used in the scientific field provide some "wrappers" for such online sources and allow the user to easily investigate these data sets and work with them in all kinds of applications, whilst the technical details and methodology like the fetching of data from the server and parsing are done completely transparent for the user. The user doesn’t even know what happens in the “background”.
The goal of this project is to add similar functionality to LabPlot. This would make LabPlot more fit for educational purposes, students and teachers could use LabPlot for visualizing and analyzing data connected to the currently studied field. And also could bring LabPlot into the life of the average student."

If the synopsis caught your attention and you are interested in the project itself, you can check out my proposal, to find out in detail what the project really is about.

Bonding period

Let's start with the bonding period. I used this time to investigate/analyze already existing solutions for uploading/downloading with KNS3 also its API documentation, checking out various welcome screens of other applications to get some inspiration, checking out some simpler caching implementations.

I communicated with my mentor and others from the LabPlot team to properly  design the project and the course of implementation. I also tried to get involved in the KDE community.

First month

At the end of the first month I can state that fortunately I was able to make a quite good progress. Everything is successfully implemented from what was proposed for the first month, and I could also proceed to other tasks. Now let's see what's been done.

Dealing with datasets

The very first step was to implement a new widget, called ImportDatasetWidget which could provide the functionality to:

• list the available categories and subcategories of datasets
• list the available datasets for a certain subcategory
• refresh the list of datasets and delete the downloaded metadata files

ImportDatasetWidget

The user can select from the categories and subcategories of the available datasets, as visible in the picture above. In order to visualize these I used a QTreeWidget. When the user clicks on a subcategory then every dataset, belonging to it, is listed in a QListView. The user is also provided with the possibility to search a particular category/subcategory, since we estimate that there will be a considerable amount of datasets by the end of the project. This is also the case with the dataset list.

We had to create metadata files in order to record additional information about datasets, and also to divide them into categories and subcategories. We use a metadata data file which contains every category and subcategory and a list of datasets for every subcategory. Additionally there is a metadata file for every dataset containing various data about the dataset itself.

In the "Datasets" section we highlight every dataset the metadata of which is locally available (in the labplot directory located in the user's home directory). When the user clicks on the "Clear cache" button every file is deleted from the above mentioned directory. The "Refresh" button provides the possibility to refresh the locally available metadata file, which contains the categories and subcategories.

In order to make possible the import of datasets into LabPlot, and saving them into Spreadsheets I had to implement a helper class: DatasetHandler. This class processes a dataset's metadata file, configures the Spreadsheet into which the data will be loaded, downloads the dataset, processes it (based on the preferences present in the metadata file) then loads its content into the spreadsheet.

 

ImportDatasetWidget basic functionality

There also is a "Add new Dataset" button. This makes it possible for the user to add own datasets to LabPlot's list. When the button is clicked a new dialog is shown to the user: DatasetMetadataManagerDialog

Adding new dataset to the collection

This dialog provides an interface for the user, so the user can easily set the options necessary for the dataset's metadata file. The user doesn't have to create the metadata file himself/herself, the dialog does this instead based on the data provided by the user. The dialog also adds the new dataset's name to the categories' metadata file. Therefore the user can easily add new datasets, and later load them easily, using the ImportDatasetWidget. 

 Adding new dataset

While implementing these functionalities we were faced with some problems, which still need to be solved. We wanted to make possible the download and upload to store.kde.org, using the KNS3 library.  So users could really add new datasets to the basic list, and could download only those metadata files which are needed. This was our intention, however KNS3 gave us a hard time. It provides its functionality only by two dialogs, which we wouldn't prefer to use. We want to incorporate the dialogs' functionality into LabPlot somehow, but we didn't figure this out yet. but, we are thinking about it. Another problem with KNS3 is that according to KDE's mailing list, uploading with KNS3 is disabled for an indefinite amount of time due to errors caused by the library. Therefore the question arises: Do we need downloading if uploading is not possible? 

Initial Welcome Screen

Due to successfully implementing quite fast the dataset part and also to the difficulties caused by KNS3, I started to design and to create a prototype for our Welcome Screen. The first step was to decide what technology we want to adapt in order to implement this welcome screen. We considered creating a widget based GUI, or using QML for this purpose. We chose QML, because despite  it being more complex and cumbersome to work with, it offers a greater freedom to implement ideas. So this is the main reason this technology was chosen. Then we had to think through, what functionalities we want to be provided by the welcome screen. We came up with:

• Recently opened projects
• Help section: Documentation, FAQ, etc.
• Exploring datasets
• Example projects
• Latest release information 
• News section

The current state of the welcome screen

Every part, except the Examples section, is fully functional. In the recent projects section the user can choose from the lastly opened projects and load them only by a click. The help section navigates the user to the Documentation, FAQ, Features and Support part of LabPlot's web page. In the release section the user can read about the last release. The news section is connected to the RSS feed of LabPlot's webpage, so the user can see the new posts of the web page. A central piece is the "Start exploring data" section where the user can browse the available datasets, display information about them, and open them with only one click. The functionality of the welcome screen is presented in the following video:

Finally I'd like to say some words about the next steps. The first will be implementing the "Examples" section of the welcome screen. In order to do so we'll have to design a metadata file for the example  projects and nonetheless we'll need to create some example projects to have something to work with. When functionally the welcome screen works, we can proceed to refining the design itself, making it more pleasant and adapting it to the user's theme's color. We'll still have to figure out what to do with the KNS3 library, should we use it or not, and if yes then how, in what manner. Nonetheless, we have to collect more datasets in order to provide the users of LabPlot with a considerable dataset collection.

This is it for now. I will continue to work on the project alongside Kristóf and Alexander. I think we form a quite good team. I'm thankful to them, for their guidance. When anything new will be finished and running I'll let you know. 

See you soon!

Bye!