How to test the accuracy of your IoT applications

While implementing your MQTT-based application's back-end processing,
such as real-time archiving, analytics, edge-processing, graphing, how
do you  verify that it will store and process all received telemetry messages,
with no messages missing, no extra messages, no bytes altered, in the
correct order? Furthermore, it is really hard to make sure that it works at
required scale and speeds, eg. at millisecond granularity.

You need to come up with a performance test that goes beyond simple
load testing and handles each of the test requirements (message integrity,
sequencing and frequency).

To test this case, you can apply the following features found in MIMIC MQTT Simulator:

1. copy the existing Bosch XDK simulation as detailed in an earlier post .
This ships with MIMIC, and generates a realistic JSON payload similar to

{
 "sn":"20:19:AB:F4:00:01",
 "data":{
  "acc":{"x":26,"y":32,"z":1012,"unit":"mG"},
  "gyro":{"x":1220,"y":-6835,"z":-2319,"unit":"mdeg/s"},
  "mag":{"x":40,"y":1,"z":-4,"unit":"uT"},
  "light":{"value":999,"unit":"mLux"},
  "temp":{"value":"100","unit":"mCelsius"},
  "pressure":{"value":98897,"unit":"Pascal"},
  "humidity":{"value":39,"unit":"%rh"}
 }
}


By copying it to a new simulation, you can now customize it without
disturbing the original simulation.

2. change the frequency of the generated messages to 1 msec;

3. change the number of messages to exactly 1000, not one more, not one
less;

4.  change the JSON payload function to return light values in sequence;

5. change the QOS to 2.

This is a total of about 5 lines of change to the existing simulation
configuration.

Once you configure a MIMIC sensor with this simulation, it generates the
1000 messages, and you are able to verify that they are all there in the
correct order. This entire exercise should only take about 15 minutes,
as it did when we tried it


Follow-up tests can be to scale up to run multiple sensors (as many as
required), as detailed in this previous post, and for longer periods of time
(see also this post), with payloads that resemble those in your application
(see this previous post).