For an example of the former, if a sensor fails to PING for a while, it can be
assumed to be down or unreachable. This can be detected with traditional
network management applications, this scenario is shown with MIMIC
NetFlow Simulator generating flows to ELK.
generating too much data, as the highlighted green node in the Kibana
For reference, check these white papers
Database techniques can be used to populate your data repository for
priming an anomaly detection algorithm, but only real-time generation
of precisely tailored data verifies that end-to-end processing works
MIMIC MQTT Simulator can simulate large numbers of heterogeneous
sensors generating desirable data patterns in real-time over MQTT. For
example, you can have miriads of sensors generating MQTT payloads
containing a "normal" pattern, and instruct a small subset of them to
"misbehave" predictably, then observe how long it take to detect this
By deterministically varying the anomaly patterns in the simulator you are
able to tune and regression test iterations in your detection algorithm.
You are able even to explore boundary conditions of the infrastructure
requirements, such as message rates, failure conditions, etc.