Friday, December 16, 2016

IoT Sensors Need to be Managed

In order to be adopted, the components of the Internet of Things must be
manageable and managed, according to this Gartner report .
Specially in light of the revelations of vulnerabilities in commercial
networking equipment, such as the recent CERT advisory regarding Netgear
routers, CIOs are going to be hesitant of adding to the exposure to hackers
with this new class of networked gadgets.

Whether the sensors themselves have management interfaces, or the
network infrastructure (edge gateways, wireless access points, ...), the IoT
environment consists not just of the data plane, but must consider the
management plane.

MIMIC IoT Simulator provides a comprehensive, integrated framework for
simulating large IoT environments by providing common network
management APIs to simulated devices, such as SNMP, command line
interfaces (CLI), Web services, NetFlow, etc.  By combining the required
interfaces into your simulation you can exercise your management
plane for intrusion detection, fault and performance monitoring while
generating desired payloads to your IoT platform for large data analytics
and anomaly detection.

From original post

Monday, December 12, 2016

MIMIC MQTT Simulator for testing IoT Anomaly Detection

Data generated by your IoT sensors are a special case data source for
Anomaly Detection. This case is even more interesting because a fault
in the IoT infrastructure can be an anomaly itself.

For reference, check these white papers

https://www.bosch-si.com/internet-of-things/iot-downloads/iot-analytics-white-paper/anomaly-detection.html

https://aws.amazon.com/blogs/iot/anomaly-detection-using-aws-iot-and-aws-lambda/

https://www.oreilly.com/ideas/the-elements-of-anomaly-detection-in-the-internet-of-things

https://software.intel.com/en-us/articles/change-and-anomaly-detection-framework-for-internet-of-things-data-streams

Database techniques can be used to populate your data repository for
priming an anomaly detection algorithm, but only real-time generation
of precisely tailored data verifies that end-to-end processing works
as intended.

MIMIC MQTT Simulator can simulate large numbers of heterogeneous
sensors generating desirable data patterns in real-time over MQTT. For
example, you can have miriads of sensors generating MQTT payloads
containing a "normal" pattern, and instruct a small subset of them to
"misbehave" predictably, then observe how long it take to detect this
anomaly.

By deterministically varying the anomaly patterns in the simulator you are
able to tune and regression test iterations in your detection algorithm.
You are able even to explore boundary conditions of the infrastructure
requirements, such as message rates, failure conditions, etc.

Friday, December 2, 2016

Simulate thousands of Bosch sensors with MQTT Simulator


You can use the Bosch XDK Cross Domain Development Kit to connect your
Bosch sensor implementation to your IoT platform of choice. But how do you
load test with thousands or hundreds of thousands of sensors?

Here is a Youtube video that shows this in real-time.

We simulated the sample sensor in MIMIC MQTT Simulator with the sample
JSON in under one minute by just doing a copy/paste of the message from
the web page into MIMIC.

The "Subscriber" in the screenshot shows the unmodified message received
by the Mosquitto subscriber as the first message.

Then we modified certain fields to return different values. In the screenshot
they are the underlined "sn" and "value" fields.

Starting 1000 sensors to generate those values to the broker took another
minute. The "MIMICview" shows 5000 sensors configured, and 1000 sensors
started. The "Broker" terminal shows the IP addresses of the connecting
clients.



Friday, November 18, 2016

Dell/EMC performs complete testing using MIMIC Simulator

EMC has multiple copies of MIMIC Simulator. There is a team of 60 engineers
using MIMIC for scalability, performance, feature and regression testing.

Using MIMIC, they are able to simulate a lab with thousands of switches,
routers, firewalls, load balancers, and hosts from Cisco, Juniper, F5, Riverbed
and many other vendors at a fraction of the cost of hardware. They can also
recreate their customer provided  scenarios to support them better, and
certify newer devices with Smarts.

The engineers use MIMIC’s user-friendly GUI in addition to the scripting
interface. They are able to write scripts in multiple languages (Tcl, Java, Perl,
Python, C++, PHP); they can easily change simulations dynamically and
create interesting scenarios like "interface down" or "core router down", and
test disastrous conditions. They are able to query simulated devices
using SNMPv1, SNMPv2c or SNMPv3, and verify the same information using
command line interfaces (CLI) via Telnet/SSH. They can get the system
configuration using Cisco IOS commands, and then get the same information
using SNMP. They can also generate NetFlow (v5, v9, IPFIX, NBAR) flows.
Since all of these are integrated in MIMIC, they see very consistent data, just
like real devices

For details see our case study.


Wednesday, November 16, 2016

MIMIC MQTT Simulator and Elk Stack

Thinking of feeding MQTT messages of your sensors into an
Elk Stack via a MQTT input plugin?

You could deploy hardware sensors or test clients for your testbed.
Alternatively, MIMIC MQTT Simulator provides a large virtual sensor
network of up to 100,000 sensors per host, so you can rapidly prototype
different types of sensor scenarios and scale up cost-effectively.



Wednesday, October 5, 2016

MQTT performance methodology using MIMIC MQTT Simulator

Performance of small-scale environments never predicts behavior of
large-scale deployments. But, it is too expensive to setup large numbers
of your MQTT sensors to load test your IoT back-office platform, including
MQTT broker and client applications.

With MIMIC MQTT Simulator, it is simple to create large sensor simulations
(you can run up to 100,000 simulated sensors on a single server)
to verify performance. The methodology is to use MIMIC to simulate a
large environment with synthetic background throughput, then verify
your performance requirements (eg. maximum round-trip delay) either
with a small number of your real-world sensor, or with another MIMIC
setup measuring end-to-end latency. That way you are sure the synthetic
load is not impacting your measurement setup except through the broker.






In the screenshot above we are running 10 sensors with an end-to-end
measuring instrumentation, and the end-to-end delay is graphed in the
bottom graph. It shows minimum, average and maximum delay for messages
from those sensors to a subscriber running in the same MIMIC.

From another MIMIC instance, we keep adding a synthetic load onto the
MQTT broker under test, from 0 to 1000 in steps of 100. The upper graph
shows the size of the background load over the 15 minutes of the test.
Each background load sensor publishes at 1 message per second, so the
throughput is the same as the number of sensors. This is trivial to
change in MIMIC to conform to your real-world expectations.

As you can see, the delay is only slightly increasing over time, except
for 2 notable bumps at 600 and 1000 sensors. It is trivial to repeat the
scenario, and verify that indeed there is a reproduceable problem. You
would never know if you did not do the tests.

Friday, September 16, 2016

How MIMIC Simulator creates large flow matrices

Traffic matrices have been around for a long time as detailed in this
article. A cell in the matrix denotes a quantity of traffic flowing from a
source to a destination. Partitioning each of the source and destination
dimensions into interesting subsets allows to draw conclusions about
traffic flowing into and out of those subsets. For example, traffic flowing
into and out of a company network, or along geographic or organizational
boundaries.

With the advent of NetFlow and Cisco's NBAR the 2-dimensional source
IP/destination IP (network traffic) matrices have been extended to add
the application dimension: not only do we know how much traffic flows
between network nodes, but also which kind. For example, we can
quantify how much Youtube traffic flows in and out of the company network
vs. plain web-surfing or shopping.

MIMIC NetFlow Simulator can simulate any flow matrix for network
performance and security monitoring. In a vast sea of normal traffic
flows slight deviations can be added deterministically to test and
demonstrate performance monitoring features or security policies.

For example, MIMIC is being used to shape application traffic for
StealthWatch, Scrutinizer, Intermapper, and even in the on-demand
demo cloud for Cisco's dCloud.