Rehearse a Data Center Crisis Against 10,000 Simulated Devices

 

Data centers are getting more public scrutiny than they have in years. Whatever
your view of that debate, one thing is clear: a data center that does get built should

work on day one. It should not fail its first crisis, and it should not be over-built to 

cover for what nobody tested.

Most of what goes wrong in a new facility is not the building. It is the network and 

the tools watching it: the detection rule that never fires, the monitoring system that 

drowns in its first alarm storm, the configuration push that stalls at device four 

thousand. None of those can be tested on the production network, and none of 

them show up at the scale of a lab.

Rehearse it, don't hope for it

 

MIMIC Simulator assists in hardening a data center by giving you a data center's 

worth of network devices that cost nothing to break -- thousands of simulated 

switches, routers and servers, each on its own IP address, answering SNMP, 

exporting NetFlow, IPFIX and sFlow, and accepting CLI logins over Telnet and 

SSH, all running on a server rather than in racks.

Point your security, monitoring and configuration tools at it, and you can run the 

attack, the outage and the bad change before they happen for real. The resilience 

comes from the rehearsal.

If you read our August post, you have seen how to lay out a ten-thousand-device 

topology in minutes. This is what to do with it.

1. Rehearse the attack

A threat-detection platform is only as good as the rules you have tuned in it,
and the honest way to tune a rule is to show it the thing it is meant to catch.
On a production network you cannot schedule a DDoS or a port scan. On a
simulated one, you can, many times.

The MIMIC NetFlow Simulator exports flow records from every simulated device, 

and you control every field in them: source and destination addresses, protocols, 

ports, packet and byte counts, export intervals. That lets you produce, on demand 

and as often as you like:

  - a DDoS over TCP or UDP against one of your own addresses;
  - a port scan or reconnaissance sweep across a subnet;
  - brute-force login attempts;
  - large outbound flows to an unknown destination at an odd hour -- the
    shape of data exfiltration;
  - and the quiet versions of each, which are the ones worth testing: the
    low-volume exfiltration, the scan hidden in normal traffic.

Then watch what your security application does with it. Does the alert fire?
Does it fire on the stealthy version, or only the loud one? Does the ordinary
application traffic running alongside it (simulated by Cisco AVC application
flows) trigger false positives? Adjust the threshold, replay the same traffic,
and compare. Because the traffic is simulated, the second run is identical to
the first, which is what makes the comparison mean something.

 

Video: "MIMIC NetFlow Simulator and ElastiFlow NetObserv"

The video above shows this against ElastiFlow NetObserv: MIMIC generates 

DDoS, port-scan, reconnaissance and brute-force traffic, and ElastiFlow picks it 

up. The full walkthrough is in our ElastiFlow post. Seceon used the same 

approach to develop its data-center security platform, with simulated switches on 

every rack exporting NetFlow into its detection engine.

2. Rehearse the outage

The first real crisis in a new data center is also the first time the monitoring 

system sees one. That is a poor time to learn that its dashboards slow to a crawl 

at ten thousand devices, or that an alarm storm buries the one alarm that 

mattered.

With the MIMIC SNMP Simulator, every simulated device carries a live MIB you 

can change while it runs. So you can stage the outage itself:

  - take interfaces down and bring them back, one or a thousand at a time;
  - send the trap storm that a failing core switch produces;
  - raise error counters and utilization on a link until it saturates;
  - stop devices outright, and see how long it takes your system to notice.

Two things come out of that. The first is a measured answer to "does our 

monitoring cope?", taken at the scale you will actually run. The second is a 

trained team. Operators can work through a staged crisis on a network that looks 

exactly like the real one, without anything real being at risk. Pepco Holdings did 

exactly this -- see Identifying and preparing for Crisis Situations --  running 

10,000 simulated devices to prepare its operations center for worst-case events.

3. Rehearse the change

Configuration management tools are tested on a handful of devices and then
trusted with thousands. The MIMIC IOS Simulator gives your chosen tool
thousands of devices to log in to over Telnet or SSH, each answering the CLI.
Run your change against the whole fleet and find out how long it takes, what
happens when devices stop answering halfway through the run, and whether your
tool reports those failures or quietly skips them. Vistara used MIMIC's SNMP 

and Telnet/SSH simulation to test its platform at enterprise scale.

Build it right

Whether a data center should be built is not a question a simulator can answer. 

How well it works once it is built is, and the answer depends on how much of the 

first crisis you have already seen.

Every rehearsal above runs on your schedule, repeats identically, and costs 

nothing when it goes wrong -- which is the point. If there is a scenario your
team has been meaning to test and never could, tell us what it looks like.

Customers have used MIMIC for:
  - data-center security:  Seceon
  - crisis preparation:  Pepco Holdings
  - configuration management:  Vistara
  - data-center infrastructure management:  Device42, Graphical Networks

From Empty Canvas to 10,000 Simulated Devices in Minutes

 

A network simulation worth testing against is more than just fifty devices in 

a row.  It should reflect your production network, usually consisting of a core, a

distribution layer, an access layer and the end  systems hanging off it -- each device

with the right interface count, each link connected to something real, and not one 

duplicated IP address anywhere in it. Built by hand that is days of data entry, and 

by the time the topology is right the thing you wanted to test has moved on.

If you have used the Topology Wizard in MIMIC Simulator, you already have most 

of this solved. Configuration->Auto builds a large topology from a description, 

Configuration->Manual connects ports one at a time, and Replicate stamps out

copies. The MIMIC Topology Designer is its successor, and the difference is what 

happens after generation.
 

The Wizard is a sequence of dialogs: you answer them, it produces a topology, and

to change anything you go back through the dialogs or edit connections one port at

a time. The Designer puts the whole topology on an interactive canvas.  Generation

is one thing you can do there, not the only thing.

 

 

 

Two ways to build: generate it, or draw it

Most large topologies start generated and end up edited, so the Topology Designer 

does both on the same canvas.

 

GENERATE. Describe the shape you want -- how many devices, how many levels, 

which device types at each level, and the address space to draw from -- and the 

Auto Topology Wizard builds it: agents created, interfaces wired, addresses 

assigned, laid out and ready to inspect. The device types come from the Device 

Library, so every generated agent arrives with the right simulation, scenario and 

interface set already on it.

DRAW. The canvas is where the rest of the work happens -- adjusting what was 

generated, or building what the generator has no shape for. Drag a device in to 

place it. Select one device and click another to link them. Change any agent's 

addressing, simulation or management interface afterwards and every connection

that depends on it follows.

 

Additionally, interactive connection of imported topologies allows Lego-style 

build-up of your simulated network.

Sanity-checking what you asked for

Three of the wizard's inputs are not independent of each other: the total device 

count, the number of levels, and the port density of the device types you assigned 

to each level. Together they decide whether the network you described can exist 

at all -- a shallow tree of four-port devices cannot hold ten thousand nodes no 

matter how firmly you type the number.

If the shape you specified cannot hold the count you asked for, generation does 

not start. You get the exact maximum that shape will hold, and four ways to 

change the answer: fewer devices, another level, higher-port devices at the inner 

levels, or more end systems. Adding a higher-port device type to a level is usually 

the cheapest fix, because the generator picks the best fan-out available at each 

level.

Dealing with scalability

Generating ten thousand agents is one problem; visualizing them afterwards is 

the one that makes large test networks unusable.

Submaps collapse a set of agents -- or an agent and its whole subtree -- to a single

icon. Past roughly a hundred agents they stop being a convenience and start 

being the reason the canvas is still readable. The Submap Manager handles 

nesting, dissolving and editing.

 

 

Search finds any device in a ten-thousand-agent topology in seconds. Duplicate-IP

protection refuses a collision as you make it, rather than reporting it to you later.

Seven layouts arrange the canvas for you: grid, hierarchical top-down or left-to-

right, circular, star -- which suits generated topologies -- and force-directed or 

spring when you want to see how the graph really connects.

Replicate Topology is still the fastest route to scale: get one branch office exactly

right, then stamp out twenty with fresh IP subnets per copy.

Past a point, drawing the canvas stops being useful, and there the Designer offers

to generate straight to a configuration file instead. The limit on what you can 

build is not the limit on what your screen can show you.

From configuration to a running network

What the Designer saves is a standard MIMIC lab configuration -- the same .cfg 

the rest of MIMIC already consumes. The topology you just designed loads and

starts exactly like any other MIMIC configuration, in the same workflow you use 

today.

Load it, start the agents, and the network is live: every device answering SNMP 

on its own IP address, with the interface tables, the system descriptions, and the 

connectivity you laid out on the canvas. Point your management application at it 

and it discovers a ten-thousand-device network that does not exist.

 

 

That is the whole point of the exercise. The topology is not the deliverable -- the 

test bed is. What you can now do to your NMS, on demand and without a lab full 

of hardware:

  - run discovery against ten thousand devices and find out how long it
    actually takes, and what it does to your database;
  - see whether the topology map your product draws matches the topology you
    designed -- you have the ground truth, which you never do with real
    equipment;
  - break links, take devices down, change interface state, and watch how the
    application reacts;

The network you could not justify building

Every test network is a compromise between the one you need and the one you 

can afford to build. Fifty devices because that is what the lab holds. One vendor 

because that is what you have. A flat topology because wiring a realistic one by 

hand was going to take a week you did not have.

Those compromises are what the MIMIC Topology Designer removes. The network 

is described rather than assembled, so its size is a number you type; it is a file, 

so it costs nothing to keep, version, hand to a colleague, or rebuild identically in 

six months; and it is simulated, so the ten-thousandth device is as cheap as the 

first.

The question stops being "what can we build?" and becomes "what should we test

against?" -- which is the question you wanted to be asking.

If you have a test network you have been putting off building, tell us what it looks like.

Operate MIMIC Simulator in Plain English with Claude Code Skills

 

 

 

 

 

If you run MIMIC Simulator in your lab, you already know its interfaces: the MIMICview GUI, the WebUI, mimicsh, 

and the programming APIs. Now there is one more -- and it speaks your language. With the new MIMIC Skills for 

Claude Code, you state your intent in natural language (English, or any other):  for example "start agents 1-100 

with a warm-up delay", "why is throughput down since yesterday?", "zeige mir die aktivsten Agenten", "make 

agent 5 export IPFIX to my collector", ... Claude translates it into the right MIMIC API calls, runs them, and 

shows you exactly what it did.

 

1. What the Skills Are

 

Claude Code Skills are structured instruction packages that Claude reads at invocation time. The MIMIC skill

package turns Claude into a capable MIMIC operator by pairing an orchestrating skill with a bundled knowledge 

base of the MIMIC API surface and object model. The package currently contains three skills:

 

1. mimic -- the core operator skill: configuration, agent lifecycle, runtime simulation changes, troubleshooting --  

    any operation the installed MIMIC APIs expose;

 

2. mimic-diagnostics -- an add-on performance analyst: captures mimicd's built-in instrumentation over time and answers 

    "why is it slow, and what is the bottleneck?";

 

3. mimic-netflow -- an add-on flow-export operator: configures and drives NetFlow (v5/v9/v10-IPFIX) and sFlow sources on 

    your simulated agents, pointed at your collector.

The design goal is generality, not a fixed menu of workflows. Because the skill carries the API knowledge base 

rather than a list of canned recipes, it allows you to quickly become productive with MIMIC and handles any operations 

you may want to do.

 

2. Safe by Design

 

The skills are entirely optional and non-invasive. They require no changesto your MIMIC configuration, running processes, 

or workflows -- your installation behaves identically whether or not the skills are installed.

Key guarantees:

* Claude operates MIMIC only through its APIs -- it never edits files
   under the install area, never kills processes, even if asked;
 

* MIMIC remains multi-user: a colleague can drive the GUI, WebUI, or a
   CLI on the same instance at the same time, and the skill interoperates
   with state it did not create;
 

* inverse-action undo lets you back out of a change;
 

* an opt-in audit log keeps a durable record of everything the skill did.

 

3. Learn MIMIC by Watching

 

A secondary goal shaped the whole design: transparency. Every operation is echoed as the underlying API call before 

it runs, with a brief explanation, so you see not just the result but HOW it was done. Engineers new to a corner of 

MIMIC -- say, the flow-export configuration keys -- pick it up simply by watching the skill work. The skill is also aware 

of the MIMIC documentation, both your local install's help and the cloud docs, and presents the relevant pages on request.

 

4. Diagnose Performance in a Conversation

 

The mimic-diagnostics add-on shows what this looks like for a harder task. For example, instead of manually 

collecting profiling dumps and eyeballing counters, you say "profile the daemon for the next ten minutes and tell

me where the CPU is going." The skill:

 

* captures a time series of mimicd's built-in instrumentation (no daemon
   changes needed -- capturing is effectively read-only);

* reads each dump as rates and per-thread CPU against a model of mimicd's
   internal pipeline;
* reports the bottleneck -- throughput, latency, lock contention,
   per-request cost -- and compares against a known-good baseline if you
   have one.

 

5. Protocol Module expertise

Likewise, protocol specific skills like mimic-netflow turns a multi-step flow-export setup -- load the protocol 

module, attach a flow-source configuration, set the collector, enable, verify emission -- into one request: 

for example "have agent 12 export NetFlow v9 to 10.0.0.5:2055 and confirm the collector is receiving." 

The skill checks that the NetFlow module is loaded on your instance, walks the configuration through the
daemon's protocol commands, and verifies the session through statistics
and trace output.

 

Other protocol module expertise will be loadable on demand via the MIMIC Update Wizard.

6. Wrapping Up

The MIMIC Skills package adds a natural-language interface alongside the interfaces you already use -- 

one that executes your intent through the same APIs, shows its work, and leaves everything else untouched. For labs
already scripting MIMIC scenarios against Zabbix, Dynatrace, ElastiFlow, or LiveNX, this is the fastest 

path yet from "what I want to test" to a running simulation.

Contact us to try the MIMIC Skills for Claude Code in your environment.

MIMIC Simulator: Exercise threat detection in ElastiFlow

As we have seen before, MIMIC allows you to simulate network telemetry/flow exporter 

(e.g., NetFlow, IPFIX, sFlow) and SNMP agent of many virtual devices. You can fully customise 

the flow records and device behavior: e.g., set source/destination IPs, protocols, ports, 

packet/byte counts, export intervals, etc. You can manipulate the instrumentation 

of agent MIBs in the simulation in real-time: change MIB object values, add/remove MIB 

table entries, simulate traps, etc.

 

 

NetObserv is a network-flow and telemetry analytics platform: it ingests flows 

(NetFlow/IPFIX/sFlow) and SNMP/telemetry, normalises/enriches them, then provides 

dashboards, alerts, and security/operational analytics.  On the security side, NetObserv 

can detect things like port scans, unusual protocol usage, data exfiltration attempts, 

link saturation, routing anomalies, DDOS, among others.

When you combine the two tools, you get a controlled lab environment in which you can 

simulate threat-scenarios via MIMIC, and then ensure that your NetObserv setup detects 

them. Here’s how you can customise threat detection:

1. Simulate specific malicious/abnormal flows

Your flow source(s) will export flows for many ports, unusual source / destination combinations, 

unexpected protocol usage, high volume flows from internal to external, etc.

 

Your SNMP agents can simulate devices or network segments going into abnormal 

states via SNMP that might reflect threat behaviour (e.g., interface up/down, weird 

routing, high error rates).

 

Because you control every field in the flow record and instrumentation, you can test 

how NetObserv will behave if certain vendor-specific fields are present, or if flows are 

mal-formatted, or spoofed sources are used.

2. Configure NetObserv rules/analytics to catch your scenarios

Once the simulated flows appear in NetObserv, you can inspect how the detection logic 

(alerts, machine-learning models, anomaly detectors) handles them.

You can then fine-tune thresholds, detection logic, filters, enrichment settings so that your 

crafted malicious/abnormal flows trigger the appropriate alerts (or don’t trigger false positives).

For example: if you simulate “data exfiltration” flows (large outbound flows at odd hours 

to unknown destinations), you can validate that NetObserv flags those; if not, you adjust 

the detection rule.

Because you have full control of simulation, you can test edge cases: low-volume stealth 

exfiltration, internal lateral movement, scanning disguised as normal traffic, etc.

 

The following Youtube video shows this in 3 minutes:

 

 

 

Customize Zabbix with MIMIC Simulator

If you have a lab to test Zabbix prior to deployment, you can use MIMIC Simulator 

not just for monitoring, but also any operational customizations you’ve made (like triggers, 

escalations, actions, scripts, dashboards, etc.) without touching the production network. 

 

 Here’s how you can set it up and test systematically:

1. Define What You’re Testing

Operational customizations in Zabbix usually include:

• Triggers: thresholds, dependencies, recovery expressions
• Actions: notifications, escalations, scripts, integrations
• User roles: who gets what alerts, permissions
• Dashboards / Widgets: visualizations of problem states
• Custom items / discovery rules: SNMP, IPMI, JMX, or scripts

MIMIC gives you the data feed (SNMP, NetFlow, Syslog, MQTT, etc.) to exercise those.

2. Connect Zabbix to MIMIC

Configure MIMIC to simulate the network devices or servers Zabbix expects:

• SNMP agents (routers, switches, firewalls, servers) with custom MIBs
• Interfaces / traffic patterns for NetFlow/sFlow/IPFIX
• Syslog events for log-based monitoring
• Ping / ICMP / TCP services for availability checks

Point Zabbix to those MIMIC devices as if they were real.

3. Drive Scenarios in MIMIC

To test Zabbix customizations, you can script scenarios in MIMIC:

• Threshold violation

        Example: Raise interface utilization above 80% to trigger a Zabbix alert.

• Flapping conditions

        Oscillate values around the threshold to test hysteresis and trigger dependencies.

• Multiple-failure cascades

        Simulate a router outage that makes downstream devices unreachable, then

        see if your trigger dependencies suppress noise.

• Custom MIB objects

        Simulatr enterprise MIBs and vary them to trigger your Zabbix custom items.

• Logs/events

        Send specific syslog entries (e.g., authentication failure, hardware error) to test actions.

• High-volume scenarios

        Generate events from hundreds of devices to test scalability and load on Zabbix plus 

        your custom dashboards.

4. Verify Zabbix Customizations

As you run scenarios:

• Check whether triggers fire correctly (no false positives/negatives).
• Validate actions: did the right people get notified? Did escalation scripts run?
• Watch dashboards update in real time.
• Confirm permissions/roles: does each user see only what they should?
• Measure response time: does Zabbix handle bursts of simulated alerts as expected?

5. Automate Regression Testing

Because MIMIC is scriptable (via APIs and scenarios), you can build a test suite to run on-demand:

• Run a set of MIMIC-driven failures.
• Capture Zabbix responses (via API, audit logs, or UI checks).
• Compare against expected results.

This gives you a repeatable regression test bed for Zabbix customizations before 

deploying changes.

How a Simulator Like MIMIC Simulator Helps nGenius Customers

Netscout nGenius is a service assurance and performance management platform. 

It ingests NetFlow/IPFIX  and packets, metadata, and application-level information. 

Customers use it to monitor end-to-end service delivery, VoIP/UC quality, and application 

performance.

Common problems that customers can run into are:

1. High Data Rates – Full packet capture plus flows can overwhelm storage and analysis systems.

2. Service/Application Visibility Gaps – Correlating flows, packets, and user experience is complex.

3. Scalability and Cost – Packet-based monitoring requires very powerful hardware and lots of storage.

4. Multi-Vendor Complexity – Different devices export different flows/metadata.

5. Training & Troubleshooting – Staff need to learn how to interpret flow + packet data for root cause analysis.

6. Integration Challenges – Feeding nGenius data into ITSM/SIEM/SOC tools isn’t always straightforward.

    
   

 

 

 

MIMIC Simulator Suite virtualizes large network environments to help tackle some of these problems:

1. Validate Scale – Generate realistic traffic (flows + emulated devices) to see how nGenius handles high loads before production.

2. Application/Service Testing – Simulate voice, video, or application flows so teams can practice monitoring service quality.

3. Multi-Vendor Assurance – Emulate Cisco, Juniper, Palo Alto, etc. devices to test interoperability.

4. Training Lab – Give engineers real scenarios (DDoS, poor QoS, packet loss) without touching live users.

5. Safer Testing – Use simulated instrumentation data (SNMP, NetFlow, sFlow) instead of actual sensitive user data, avoiding compliance risks.

6. Integration Validation – Feed nGenius with reproducible test data to confirm workflows with SIEM, NMS, or service desks.
   

 

 

 

MIMIC Simulator and LiveNX

LiveNX customers can benefit from MIMIC Simulator to complement their in-house lab at a fraction of the cost of real equipment: 

 

Customize LiveNX

• Enables rapid development of custom features by recreating the exact scenario in MIMIC with repeatable test data.

• This makes development, troubleshooting and support faster and more precise.

   

Large-Scale Test Environments

• LiveNX is designed to monitor enterprise and service provider networks with thousands of devices and interfaces.

• With MIMIC, LiveNX customers can simulate tens of thousands of routers, switches, firewalls, and endpoints before deploying in production.

• This allows validation of LiveNX scalability without needing physical gear. Resources can be allocated well in advance to ensure smooth operations.
  

Figure 1 - Topology view with connections between sites

Training and Demos Without Real Hardware

• MIMIC can generate realistic SNMP responses from a variety of vendor devices and pathological scenarios, as well as NetFlow, sFlow, and syslog traffic.
  

• LiveNX teams can train staff and demonstrate LiveNX features using fully controlled, reproducible simulated environments — no need to access the live production network.

• Disaster preparation can be done safely in the simulated lab.

 

Figure 2 - Traffic breakdown for one of the connections

 

Network Change / Upgrade Validation

• Before rolling out firmware upgrades, new device models, or topology changes, LiveNX users can simulate the new environment in MIMIC.

• This ensures LiveNX’s discovery, topology visualization, and performance analytics work correctly ahead of time.

Proof-of-Concept (PoC) Acceleration

• Customers evaluating LiveNX can set up large testbeds overnight with MIMIC instead of waiting for lab hardware.

• This reduces time-to-value and makes the PoC process smoother.

MIMIC Simulator and Dynatrace

In our quest to support all possible network management platforms, we have

interoperated with Dynatrace by discovering large networks:

 

 

 

 

 

 

 

 

 

 

and drilling into devices:

MIMIC MQTT Lab: Test MQTT 5 support on AWS IoT Core

 AWS recently announced MQTT 5 support for AWS IoT.

We tested it in less than 5 minutes with MIMIC MQTT Lab AWS . You can do the same to make sure your
AWS IoT application uses the latest MQTT 5 features such as properties in PUBLISH messages, etc. 

Check the 2-minute Youtube video that shows the MQTT 5 CONNACK with new reason code:

CONNACK rc=0x00 Session Expiry Interval 0,Receive Maximum 100,Maximum QoS 1,Retain Available 1,Maximum Packet Size 149504,Topic Alias Maximum 8,Wildcard Subscription Available 1,Subscription Identifiers Available 0,Shared Subscription Available 1,Server Keep Alive 50

When we connect with the disallowed QOS 2, we get a new self-explanatory error code:

CONNACK rc=0x9b Reason String CONNACK:QOS 2 is not supported:861b3462-65d8-ba70-5472-63869294a5a1

and when we send a malformed PUBLISH (empty topic and topic alias):

INFO  12/02.10:53:07 - MQTT[AGT=3916] - sent CONNECT (51 bytes)
INFO  12/02.10:53:07 - MQTT[AGT=3916] - rcvd CONNACK rc=0x00 Session Expiry Interval 0,Receive Maximum 100,Maximum QoS 1,Retain Available 1,Maximum Packet Size 149504,Topic Alias Maximum 8,Wildcard Subscription Available 1,Subscription Identifiers Available 0,Shared Subscription Available 1,Server Keep Alive 50
INFO  12/02.10:53:08 - MQTT[AGT=3916] - sent PUBLISH (126 bytes)
INFO  12/02.10:53:08 - MQTT[AGT=3916] - rcvd DISCONNECT reason 0x82 (Reason String DISCONNECT:Data in packet does not conform to MQTT specification:19ec6dc1-0b50-888c-6c3e-3be26faee968)

MQTT performance testing - Best Practices

The MIMIC Simulator performance testing methodology attempts to overcome
common problems with published performance benchmarks, specially in the 

IoT arena. In this article we examine one recently published report and discuss 

how to make it better.

 

The main problem with any performance test is that the results apply only to the 

specific test scenario. If the test scenario is carefully selected, the results will be 

relevant for a wide variety of situations. If the test report is good, then the exact 

methodology is documented, so you can evaluate it, and determine whether the 

results can be useful for you. For example this report

https://www.researchgate.net/publication/354610718_Stress-Testing_MQTT_Brokers_A_Comparative_Analysis_of_Performance_Measurements

performed one test scenario for an uncommon situation of a small set (3) of high-

frequency publishers, and 15 mosquitto_sub subscribers. Plain text MQTT is only 

used in trivial situations, and there is no indication that TLS transport is measured.

Latency measurements suffer from the time synchronization problem on different 

systems.

 

Specifically, it says right at the beginning in the abstract

 

"The evaluation of the brokers is performed by a realistic test scenario"

 but then, in section 4.1.1. Evaluation Conditions:

"

Number of topics:                   3
                                    (via 3 publisher threads)
Number of publishers:               3
Number of subscribers:              15 (subscribing to all 3 topics)
Payload:                            64 bytes
Topic names used to publish large
number of messages:                 ‘topic/0’, ‘topic/1’, ‘topic/2’
Topic used to calculate latency:    ‘topic/latency

"

 

so rather than testing a large-scale environment, a small set (3) of high-frequency 

publishers, and 15 mosquitto_sub subscribers was used. In our experience, no 

recent broker has any problem with less than 1000 publishers.

Second, in section 4. the subscriber back-end is detailed:

"The subscriber machine used the “mosquitto_sub” command line
subscribers, which is an MQTT client for sub- scribing to topics and
printing the received messages. During this empirical evaluation, the
“mosquitto_sub” output was redirected to the null device (/dev/null) "

using a the simple mosquitto_sub client which is single threaded. In addition,

the subscribers subscribe to all topics, probably the wildcard topic #. So, out of 

many code paths in the broker, the least commonly used is tested. If your 

application uses a topic hierarchy, with different subscribers subscribing to 

different topic trees, then topic matching performance needs to be exercised.

Third, while QOS 0, 1 and 2 seem to be tested, only a single payload size 

was used, and there is no indication that TLS transport is measured.

Fourth, they attempt to measure latency correctly, ie. section 4.1.2

"Latency is defined as the time taken by a system to transmit a message
from a publisher to a subscriber"

but their methodology is flawed, since it is almost impossible to synchronize the 

clocks on 2 separate systems to millisecond accuracy and in table 6 the latencies 

are in the 1ms range. So, the measurements rely on unknown synchronization. 

For an example of the MIMIC latency testing methodology see this blog post.

Migrating from shuttered IBM Watson IoT platform

In a previous article simulation was recognized as helping prevent IoT project failures
so prevalent in the industry.

 

With the recent announcements of the shuttering of the Google IoT Core and 

IBM Watson IoT platforms, we can suggest that MIMIC IoT Simulator can be used 

to help migrate from the obsolete IoT platforms to a new offering by:

1) running a facsimile of your environment in MIMIC

2) staging migration to the new platform

3) testing requirements at various scales to make it future-proof

before you impact your production network.

How to scale your MQTT lab to 1000 sensors in minutes

TL;DR Money saved: $40,000. Time saved: immeasurable.

We needed to create a MQTT lab with 1000 sensors to test a subscriber client with

realistic telemetry. The open-source client tracks any key value, and alerts if any

arbitrarily pre-selected value exceeds a threshold.

We bought 1 real Shelly Plus H&T sensor for $40.

After you have configured it, it sends MQTT messages to the broker, but only 

every time the temperature and humidity changes. So, to test our application, we 

would have had to run to the refridgerator quite often to make it change the 

temperature.

As you can see from the screenshot

it sends JSON payloads, but very infrequently. In our case, after 6 minutes

So, every time we wanted a message, we needed to change the temperature.

To accelerate development, we used MIMIC.

First we just captured the messages with wireshark, recorded into MIMIC MQTT Simulator, 

and generated messages whenever and however we wanted. Rather than waiting for minutes, we can

send any message with any value in seconds, speeding up development time. Then we multiplied

the sensor 1000-fold, quickly reaching the required scalability at no additional cost.

This video shows the process in 2 minutes:

Money saved: $40,000. Time saved: immeasurable.

IoT platform demo requirements

IoT platform software is complicated leading to project delays and frequent failures. To 

select the right platform for their needs, the customer has to learn and evaluate features such 

as data acquisition, analytics, graphing, database integration, alarming, etc.

Product evaluation is usually done in a couple of steps, starting with a survey leading to practical 

hands-on trial, each further filtering from the wide assortment of products. The initial evaluation needs 

to whittle down to a couple of candidates for deeper investigation before a proof-of-concept effort. 

Free brochures, videos, and other static marketing collateral can only go so far to get a customer

interested in the IoT platform software. What is needed is a more engaging, interactive format.

Thus, in order to best advertise platform features, an initial product demo has to be

1. free - to get the foot in the door, you should not be charged. Further, the customer
   should not need to give a credit card number, or any kind of personal information
2. immediate - the customer should get to the demo in a couple of clicks, and the instructions
   should not consist of more than a couple of lines. In total, the demo should be done in a
   couple of minutes.
3. always on - in a global economy, the demos should be accessible 24 hours a day, 7 days
   a week.
4. interactive - the demos should go beyond replay of videos, and consist of live interactions
   for each customer.
5. dynamic - the customer should be able to view the features you want to show off in the
   most compelling way and be most realistic.
6. predictable - the demo should behave the same way each time they visit, so that you can
   control it, describe it, and it matches the customer's expectations.
7. diverse - every customer scenario is unique, requiring different platform features and
   types of sensor hardware. The demos should showcase the various features that the customer wants.
8. scalable - scaling up an IoT application is the hardest part. The demo should prove
   how the platform enables large scale applications.
   

For examples of demos with MIMIC IoT Simulator that we have provided to our partners 

• Cumulocity
• Losant
• Ubidots 
• TagoIO
   

see also this blog post .

MIMIC SNMP Simulator and neteXpose DNA

MIMIC SNMP Simulator can simulate details of each device to the finest level. In 

this integration, netExpose DNA discovered the chassis details of a simulated 

Alcatel switch, allowing them to test their management application with all sorts

of device scenarios.

Migrating to Google IoT Core alternatives

If you are contemplating migrating from Google IoT before it retires in a year, you 

might want to use MIMIC MQTT Simulator and its SaaS offering for Google IoT 

https://mqttlab.iotsim.io/gcp to design / develop / test your migration strategy.

 

MIMIC MQTT Simulator allows you to simulate a large number of your assets that 

connect to Google IoT Core, then migrate them to the alternative of your choice.

You can duplicate your current IoT environment in MIMIC to make sure the new 

IoT platform handles your specific requirements.

MIMIC MQTT Lab for Azure IoT Hub - get started in 5 minutes

Available for immediate use is MIMIC MQTT Lab Azure to get started with Azure IoT Hub

in minutes, as shown in this 3-minute Youtube video

Start with #nocode, #loweffort, #lowcost, then graduate to more advanced IoT simulation

to develop, test, prototype, demo and train your Azure IoT application.

Generate predictable, customized telemetry graphed in Azure Time Series Insight in under

2 minutes

Live, dynamic, immediate, 24/7, interactive Ubidots demo

 FYI, we have published a demo with MIMIC MQTT Lab for the Ubidots IoT platform that is

1. live - real-time telemetry, no pre-canned videos;
2. dynamic - things are changing all the time, GPS coordinates, telemetry;
3. immediate - no need to sign up, etc. just open 2 web browser windows;
4. predictable - telemetry looks deliberately artificial to show complete control;
5. interactive - can change the telemetry in real-time;
6. 24/7 - any time of day or night.

The demo shows asset tracking features for a fleet of vehicles: real-time telemetry updates the location

on the map. If the vehicle is going too fast or is stopped, the colors of the gauge widgets turn to yellow and red respectively.

To try it now, open 2 web browser windows side by side with the URLs to

• MIMIC lab
• Ubidots dashboard

eg. as in this Youtube video

 You can change the speed of vehicle 1 with the Demo menu. Allow a couple of seconds to see the impact in the dashboard.

UPDATE: You can now try the same demo for several more platforms:

• Losant
  
• TagoIO
  

Self-service MQTT latency testing

We have created a SaaS lab at https://mqttlab.iotsim.io/mqttlatency to test real-time round-trip 

latency to/from your internet-accessible MQTT broker.

 

This 4-minute Youtube video shows how easy it is by measuring latency for 3 publically accessible 

MQTT brokers. To add your own would be a minute more.

 

 

Self-service online MQTT Lab up to 10,000 sensors available on AWS Marketplace

We have recently added a 10k size for our online, self-service MQTT
labs on the AWS Marketplace, designed for development, testing,
proof-of-concept, training of large IoT applications. The affordable
prices are

Lab                                                   Hourly       Annual

MIMIC MQTT Lab - 10 sensors         $0.10        $ 300.00

MIMIC MQTT Lab - 100 sensors       $0.30        $ 600.00

MIMIC MQTT Lab - 1000 sensors     $0.90        $2000.00

MIMIC MQTT Lab - 10000 sensors   $1.80        $4000.00

(all + AWS usage fees)

For details check

https://aws.amazon.com/marketplace/seller-profile?id=b654d165-74ca-4af3-b597-07b3cf484b91

and

https://mqttlab.iotsim.io/

For custom pricing (size, duration) contact us.

Here is a 2-minute video of 1000 sensors publishing to the EMQX broker
running in a separate EC2 instance:

https://www.youtube.com/watch?v=JUu6nvW6pcE

insight.tech: Avoid IoT Project Failure with Better Simulators

MIMIC IoT Simulator is featured in this Intel insight.tech article

"Proper network simulation is essential to a successful scale-up.
...
Rather than attempting to simulate physical hardware, Gambit simulates
the traffic that IoT devices generate when they communicate across
the network.
...
When end users can model their network and sensors, they can determine
resource requirements before deploying a single device.  "

Check out MIMIC IoT Simulator on the Intel Marketplace to
get your IoT Simulation on the right track.