Listener service on a raspbian server that writes to a MySQL DB

Listener Service that writes inbound data to MySQL--

Summary - To build a secure listening service intended to run on edge devices running raspbian. This is a listener that will listen for traffic that will come in the form of XML formatted messages via TCP or UDP. This listener should be able to be updated

*Note- This is a component of a larger system that is collecting Point of Sale or POS data to integrate or marry it with CCTV Video data as well as to report on it primarily looking for fraud.

Notes about the XML feed- The XML feed consists of 102 elements defined in extensive documentation provided by our POS partner. This is covered by NDA so we will not be able to share this until we execute an agreement and NDA covering all of our work including this documentation. I have a spreadsheet with all of the elements we need to capture and some notes on elements that are of special interest to us.

General functionalities-

Runs as a service under a user, doesn't need root. User that runs service should not be able to log in

Protocol and service will need to be configurable but default will be https on port 443.

Transport will be up to our GCP endpoint (configurable, we will have many).

The application may need to be updated, upgraded, changed from time to time. We will need to understand how that will be done cleanly while properly preserving data.

There are a few areas where configuration input will be required. My expectation is to use a text gui screens like the interface used during the installation of the MySQL package similar to seen here.

Our work should be done in UTC

It is likely we will run into issues where data is coming from devices are suffering from a time skew. I don’t think this should be a problem but I thought I would bring it up

Listener- The listener component needs to be configurable to a port. Data can come in via-

TCP-needs to be able to function without acknowledgement. In other words, the application sending the data to OUR listener doesn’t care if it gets back anything. It just drops it on the floor.

UDP- data can come in via UDP

NOT VERSION 1.0 THIS IS A ROADMAP ITEM RS232- We have cases where data comes in via RS232 or serial port

Format-

Needs to be completely modular and agnostic of input or stdin if you will

XML- I have an XML document for the first integration

ASCii in the future

Data insertion-

Data is inserted encrypted into a MySQL DB running on the local device with a time hack and a POS terminal hack.

POS data streams are associated with POS terminals

POS terminals are associated with one or more cameras

In some cases one camera will cover two POS terminals - we will need to accommodate for that.

We will need a textual gui tool to configure

Data is then pointed to an endpoint at a MySQL GCP endpoint and pushed up as quickly as reliably possible.

Data cleans up after itself, the local MySQL DB should be holding only a few days of data. Up to 7 configurable by quantity of hours.

If the DB dies or something bad happens we need to know, system reports up to the cloud MySQL that everything is Ok every minute or so

I would like to track message byte size

Demonstrate that no

Particular elements are interesting to me so when they go in the DB maybe they can be flagged for easy retrieval

I want to be able to run a script over the data to collect information

The edge system will collect the data

Debian MySQL Administración de redes Administración de sistemas

Nº del proyecto: #19183599

Sobre el proyecto