Overview¶

All meta data scraping or data profiling takes place as close as possible to the data transport mechanism (e.g. Apache Kafka). Therefore Agoora SaaS uses an Agent model, where the agent resides locally close to the transport and communicates towards the transport mechanisms but also towards Agoora SaaS.

Agent components & connections¶

Currently, Agoora SaaS provides agents for the following technologies:

Kafka
MQTT
Postgresql
Open API

All those agents use the "agoora-profiler-service" which basically receives some samples (JSON format) and try to find a schema and generate a profiling of the data.

All of our agents are opensource and can be found on github: https://github.com/spoud/agoora-agents

Feel free to contribute by submitting issues or pull requests. Don't forget to look at the CONTRIBUTION.md

Architecture¶

The following diagram shows how the components communicate between each other and towards the SaaS.

"SaaS communication design"

How to start¶

After creating a transport and an agent on Agoora you will be asked if you want to download the docker-compose file so you can easily bootstrap an agent.

See Quickstart as a base example for running the agent locally by docker-compose.