Fire-and-forget vs. complete control of MQTT message generation

Free, open-source MQTT performance tools follow the "fire-and-forget"
paradigm: you configure message generation in advance, then fire the
tool to perform the load testing.

This is fine for static scenarios, but when are real-world scenarios
static? Message characteristics change all the time: sensors connect and
disconnect, you get Denial-of-Service (DOS) attacks or mal-functioning
sensors (the "crying baby" scenario). How will you test that your
application load balancer (ALB) throttles misbehaving connections in
real-time?

MIMIC MQTT Simulator lets you change any message characteristic
(frequency, topic, payload, QOS, etc) at any point in real-time, not just for
one sensor, but also for many thousands if not millions. This allows you to
create anomaly scenarios that simpler tools don't allow you to.

In this 4-minute Youtube video we show our open-source "MQTT Topic
Statistics" mqtt-stats  subscriber client connected to a IBM MessageSight
broker instance. MIMIC MQTT Simulator is used to generate different
message loads dynamically during the video, verified by mqtt-stats.

In the simulator we first start 10 simulated Sparkplug sensors. Each
sensor publishes to 3 topics (NBIRTH, DBIRTH and DDATA) in order to generate
telemetry. By default each is sending 1 DDATA message per second.

In the steady-state (when all sensors are registered) mqtt-stats shows the 30
topics and 10 active topics (see in video). We show the 10 NBIRTH, DBIRTH,
and DDATA topics as defined in the Sparkplug specification. At this point we
can clear out the topics with File -> New to only show the active DDATA
topics. All sensors are showing a rate of 1 message / second.

Now we can change message rates dynamically, we pick 2 sensors to send 5, then
10, then 20 messages / second. Then we pick 3 different sensors and change their
frequency to 50 / second. And yet a different one with 100 / second. Then we reset
all 10 back to 1 message / second.  What we did interactively you can script
programmatically with the MIMIC API in 6 languages.

The rest of the video show a total of 1000 sensors being monitored by the
mqtt-stats client.

Node-RED and 1000 simulated Sparkplug B sensors

In a previous post we discussed a technique to reduce the number
of tracked telemetry in the Node-RED IoT application, which only graphed
interesting temperature values from 20 simulated Bosch sensors.

In this experiment, MIMIC MQTT Simulator is simulating 1000 Sparkplug
sensors sending telemetry to Node-RED, graphing 10 of them in real-time.
We notice that each sensor has unique, expected telemetry.

This open-source Github repository has the flow source code to generate
these graphs.

The MQTT input node sends messages to a decode-protobuf node, which
decodes the Sparkplug messages coming from the simulated sensors and
passes them on as JSON message objects. The next node switches between
NBIRTH / DBIRTH / DDATA messages. The first 2 just count the arrivals. The
DDATA messages are graphed only for the first 10 sensors, else there would
be too many lines in the graph. Unknown messages are counted as well.
We implemented a simple message performance instrumentation by
resetting the message count every second.

Graphing arbitrary Sparkplug telemetry in Ignition

We followed this Ignition tutorial to create a real-time graph of simulated
telemetry exported from MIMIC MQTT Simulator.

This short Youtube video shows the 3 independently controlled temperature
values being graphed in the Easy Chart. We had to apply this workaround
to get the tags to be graphed.

With MIMIC, you can export arbitrary, precise, customizable telemetry in
real-time to any MQTT-based IoT platform for thousands to millions of
simulated sensors over the internet or in your network edge.

(This is a followup to our previous post.)

Update 10/3/2018: This Youtube video shows 2000 EON nodes with 20,000
devices publishing 200,000 tags, as monitored by Ignition and
our open source demo app.