MIMIC 9.00 Performance Report: Windows vs. Linux on dual-core Pentium
MIMIC is SNMP simulation software that supports up to 20,000 agents on one workstation. The main concern is the performance for a fully loaded workstation. You want at least hundreds of PDUs per second to make a simulation viable. Each hardware platform / operating system combination has different performance characteristics.
For MIMIC, performance is primarily governed by the amount of physical memory (RAM). The memory requirements depend on the simulations you are going to run. Obviously, a high-end router simulation with hundreds of interfaces, RMON tables, etc. is going to take more memory than the simulation of an end system.
As a ball-park estimate, we like to see at least 1MB of dedicated physical RAM per simulated agent, e.g., a 100 agent scenario should run fine on a 128MB system (depending on how much memory is used by the OS and other processes). For better performance (less swapping), 2MB per agent is recommended. When the agents are running the same simulation, MIMIC optimizes memory usage to contain only one copy of the simulation data for all agents of the same type.
You can more accurately measure this by running a simulation configuration, and checking on memory usage before and after starting the desired agent simulations. Notice that MIMIC uses memory on demand, so you should measure the memory after doing a walk of the desired tables (or a complete MIB walk). Eg. on Windows NT use the Windows Task Manager to check "Memory Usage", and on Unix use the "top" utility. The memory usage by MIMIC is approximately the same for all platforms.
The CPU is of secondary importance. Most modern processors (e.g., Intel Pentium 2GHz or faster, and Ultra Sparc) are adequate. MIMIC works with multi-processor systems, since it is a multi-threaded, distributed application. Agent thread processing will be distributed across multiple CPUs.
The final bottleneck would be the network pipe to your agents. 10Mb Ethernet is adequate for low-volume traffic, 100Mb is better for more demanding applications. MIMIC works with multiple network adapters on your system, so you can talk to the simulations over separate network pipes. MIMIC works with the OS-native protocol stacks, so that all network interface cards that your OS supports can be used. You can even run MIMIC over PPP.
This performance test is designed to measure peak performance of MIMIC on various supported platforms under common access scenarios. The variables in the test are the number of agents running simultaneously, and the number of agents being accessed simultaneously. It is assumed that the most common SNMP request is the GETNEXT, as is done in sequential table traversals.
This report contrasts Windows Server 2003 and XP to various flavors
of Linux on a dual-core 1.8 GHz Pentium E2160.
It shows:
1. The best performer at the low end (10 and 100 agents) is Ubuntu, by
10 to 20% better than Windows Server 2003 or XP.
2. In the mid range (1,000 agents) openSUSE is best by 10%.
3. At 2,000 agents, all OSs are comparable (within 10%), except for
Fedora 9, which lags drastically.
4. At the high-end (5,000 to 20,000 agents), the Linux contenders
(Fedora 9 and openSuSE) are slightly (upto 10%+) better than Windows
2003, which performs better when accessing the first agents, but worse
when accessing higher agents.
Thursday, August 28, 2008
Wednesday, August 13, 2008
new FAQ: Why is my throughput intermittently slow on a dual NIC Linux system?
New FAQ entry for MIMIC SNMP Agent Simulator:
Q. Why is my throughput intermittently slow on a dual NIC Linux system?
A. If your Linux MIMIC system has multiple active network interfaces
connected to the same LAN, and you are experiencing drops in PDU throughput
while polling MIMIC agents in the SNMP simulation, it may be due to one network
interface running slower than the other. Linux will route traffic through
the NIC according to the arp tables regardless of the interface an agent
IP address is assigned to. The command "ethtool" run as root can be used
to discover the current speed and duplex settings of each NIC.
Any protocol analyzer can be used to capture packets between the
walkhost and the agent to determine the MAC address of the NIC handling
the traffic.
Q. Why is my throughput intermittently slow on a dual NIC Linux system?
A. If your Linux MIMIC system has multiple active network interfaces
connected to the same LAN, and you are experiencing drops in PDU throughput
while polling MIMIC agents in the SNMP simulation, it may be due to one network
interface running slower than the other. Linux will route traffic through
the NIC according to the arp tables regardless of the interface an agent
IP address is assigned to. The command "ethtool" run as root can be used
to discover the current speed and duplex settings of each NIC.
# ethtool eth0
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Advertised auto-negotiation: Yes
Speed: 100Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: umbg
Wake-on: g
Current message level: 0x00000001 (1)
Link detected: yes
Any protocol analyzer can be used to capture packets between the
walkhost and the agent to determine the MAC address of the NIC handling
the traffic.
Thursday, August 7, 2008
Network World: Cisco simulator can help thwart exam cheating
Gambit Communications says its MIMIC Virtual Lab software can help resolve the cheating on Cisco certification tests
By Jim Duffy , Network World , 07/30/2008
A Nashua, N.H., maker of Cisco network simulators says its software can help enterprises make sure they are hiring legitimate Cisco-certified engineers to run their networks.
Gambit Communications says its MIMIC Virtual Lab software, which has been on the market for about four years, can help resolve the recent spate of cheating on Cisco certification tests by enabling enterprises to run network operations candidates through sample scenarios before hiring them. This allows enterprises to screen candidates to ensure they are not hiring fraudulent network operators at handsome salaries.
Cisco recently moved to thwart cheating on certification tests by employing photo identification requirements and a data forensics program. According to Cisco, pilot programs using the new detection methods have already uncovered 1,400 suspected cheaters who hired proxies to take the exams for them.
But Gambit claims the photo and forensics programs only go so far: what about the many unqualified candidates already hired by enterprises prior to the new Cisco enforcement programs?
Sit 'em down and run them through a simulated lab environment, Gambit says.
Gambit's CCNA Virtual LAB software starts at $99 and can be downloaded to a laptop or PC. It creates a simulated environment with seven Cisco devices – Catalyst 2950, 3550 and 6500 switches and 2620, 3640 and 7206 series routers -- and users can type in IOS and SNMP commands to configure devices and protocols.
Test conductors and "students" can replace and establish LAN, WAN, ISDN and serial links, change IP addresses and create virtual LANs with the program, but cannot change the devices themselves. Also, the program is not certified by Cisco but is resold by a Cisco – certified training partner, Tech 2000. Cisco also uses the CCNA Virtual Lab's predecessor, the MIMIC Simulator Suite for IOS, Gambit says.
Gambit says it has 1,000 customers for CCNA Virtual Lab since it was introduced in 2002, including AT&T, IBM, the U.S. Army and several financial firms.
By Jim Duffy , Network World , 07/30/2008
A Nashua, N.H., maker of Cisco network simulators says its software can help enterprises make sure they are hiring legitimate Cisco-certified engineers to run their networks.
Gambit Communications says its MIMIC Virtual Lab software, which has been on the market for about four years, can help resolve the recent spate of cheating on Cisco certification tests by enabling enterprises to run network operations candidates through sample scenarios before hiring them. This allows enterprises to screen candidates to ensure they are not hiring fraudulent network operators at handsome salaries.
Cisco recently moved to thwart cheating on certification tests by employing photo identification requirements and a data forensics program. According to Cisco, pilot programs using the new detection methods have already uncovered 1,400 suspected cheaters who hired proxies to take the exams for them.
But Gambit claims the photo and forensics programs only go so far: what about the many unqualified candidates already hired by enterprises prior to the new Cisco enforcement programs?
Sit 'em down and run them through a simulated lab environment, Gambit says.
Gambit's CCNA Virtual LAB software starts at $99 and can be downloaded to a laptop or PC. It creates a simulated environment with seven Cisco devices – Catalyst 2950, 3550 and 6500 switches and 2620, 3640 and 7206 series routers -- and users can type in IOS and SNMP commands to configure devices and protocols.
Test conductors and "students" can replace and establish LAN, WAN, ISDN and serial links, change IP addresses and create virtual LANs with the program, but cannot change the devices themselves. Also, the program is not certified by Cisco but is resold by a Cisco – certified training partner, Tech 2000. Cisco also uses the CCNA Virtual Lab's predecessor, the MIMIC Simulator Suite for IOS, Gambit says.
Gambit says it has 1,000 customers for CCNA Virtual Lab since it was introduced in 2002, including AT&T, IBM, the U.S. Army and several financial firms.
Thursday, April 17, 2008
Gambit's MIMIC® Enables Xirrus for Superior Testing
Scaleable and highly functional Wi-Fi management applications are vital in today’s growing mobile network environments. They provide essential functions to ensure that the wireless network’s health, performance and availability meet the rigorous demands of business.
Xirrus sells the enterprise-grade Wi-Fi Array that embeds multiple radios, a Wi-Fi controller, antennas, and threat sensors into a single device. The Xirrus Management System (XMS) centrally manages networks of Wi-Fi Arrays. The company chose Gambit Communications’ MIMIC Simulator to ensure that the XMS software not only scales and manages hundreds of devices with real-time performance, but also correctly tracks faulty conditions of the devices.
Xirrus
Xirrus is the leader in High Performance Wi-Fi™. It designs and manufactures the patented Wi-Fi Array. The Wi-Fi Array integrates 4, 8, or 16 radios and high-gain directional antennas into a single device along with an onboard Gigabit Switch, Wi-Fi Controller, Firewall, and dedicated Wi-Fi Threat Sensor, providing the performance and security to replace traditional workgroup switches with Wi-Fi technology.
Xirrus’ XMS network management system manages up to 500 Wi-Fi Arrays from a central location. It centralizes the monitoring, reporting, and configuration management for up to 500 Arrays from anywhere in the network. It is available on a stand-alone dedicated network appliance (XM-3300), or as a software-only application (XA-3300) that can be loaded on an existing server.
The XMS provides a flexible, scalable solution that can be leveraged as the organization’s Wi-Fi network grows. It carries out automatic discovery and configuration of Arrays, policy-based management and aggregates alerts and alarms for monitoring. It uses SNMPv1/v2c with support for multiple public and private MIBs.
The Challenge
The Xirrus Management System manages large networks with up to 500 Wi-Fi Arrays with thousands of Wi-Fi clients. Xirrus has 40 physical Arrays in their XMS lab which are used for testing. It was budget prohibitive to grow that to 500 Wi-Fi Arrays along with the required networking gear. It was a challenge to test the XMS software with that size of a network.
The QA Assurance team continuously needs to verify the XMS’ performance and scalability in large networks. The tests need to cover the product features and all possible operational conditions. An affordable solution was sought to make the test lab more scalable and flexible.
The Solution – MIMIC Simulator
Xirrus purchased the MIMIC Simulator Suite, Campus edition, which supports simulation of up to 500 agents simultaneously. Each of the agents can represent any type of SNMP-manageable device.
Xirrus recorded the management information from one of their Arrays with the MIMIC Recorder, creating a simulation which was then cloned to 400 unique simulated devices with a total of 6000 clients connected to those arrays. Along with the simulated Array devices, they simulated traffic load between devices and clients. They could run their XMS against the simulated lab and test different features, along with performance measurements and negative/positive conditions.
The Benefits
With the use of MIMIC, Xirrus reduced the lab budget requirement to one tenth of what was initially needed. Furthermore, MIMIC made it a lot easier to test their XMS software. They were able to perform many tests effortlessly with MIMIC, which are very hard to setup with the real devices. They could now do more complete testing, and confidently release their new software to their customers.
Additionally, with MIMIC they can now save those negative/positive scenarios, add more tests, and reuse them for regression tests during every test cycle.
The Technology – MIMIC Simulator
MIMIC Simulator can simulate 20,000 SNMP, Cisco IOS, IPMI, Telnet, TFTP and DHCP-based devices on one workstation.
MIMIC appears to management applications as if they are connected to a real network. It allows simulation of any type of SNMP-based devices (Routers, Switches, Arrays, Hubs), and combine them in the creation of many different scenarios.
MIMIC ships with 1800+ MIBs and many out-of-the-box network, device and scenario script libraries, which make it very easy to simulate a variety of networks from different manufacturers. It includes a MIB compiler, recorder, network discovery wizard and topology editor to further extend the simulation capability.
The simulator allows real-time changes of device parameters to add interfaces, change traffic patterns and error conditions.
Testimonial
Ali Fatollahi, Sr. QA Engineer, said, “We needed to test our XMS software and verify its performance and scalability in large networks. Selecting MIMIC made it easy and cost effective for us to do that. With MIMIC, it is as close as you can get virtually to have a physical local network with such a scale.
MIMIC technical support is competent and prompt. They were always there and helped so effectively that now we use MIMIC daily for our XMS development and testing.
MIMIC is a great product (ease of use and phenomenal tech support) and has saved us time and money. We strongly recommend it.”
--------------
© 2008 Gambit Communications, Inc. All Rights Reserved.
MIMIC is a registered trademark of Gambit Communications, Inc.
All other trademarks or service marks are the property of their respective owners.
Xirrus sells the enterprise-grade Wi-Fi Array that embeds multiple radios, a Wi-Fi controller, antennas, and threat sensors into a single device. The Xirrus Management System (XMS) centrally manages networks of Wi-Fi Arrays. The company chose Gambit Communications’ MIMIC Simulator to ensure that the XMS software not only scales and manages hundreds of devices with real-time performance, but also correctly tracks faulty conditions of the devices.
Xirrus
Xirrus is the leader in High Performance Wi-Fi™. It designs and manufactures the patented Wi-Fi Array. The Wi-Fi Array integrates 4, 8, or 16 radios and high-gain directional antennas into a single device along with an onboard Gigabit Switch, Wi-Fi Controller, Firewall, and dedicated Wi-Fi Threat Sensor, providing the performance and security to replace traditional workgroup switches with Wi-Fi technology.
Xirrus’ XMS network management system manages up to 500 Wi-Fi Arrays from a central location. It centralizes the monitoring, reporting, and configuration management for up to 500 Arrays from anywhere in the network. It is available on a stand-alone dedicated network appliance (XM-3300), or as a software-only application (XA-3300) that can be loaded on an existing server.
The XMS provides a flexible, scalable solution that can be leveraged as the organization’s Wi-Fi network grows. It carries out automatic discovery and configuration of Arrays, policy-based management and aggregates alerts and alarms for monitoring. It uses SNMPv1/v2c with support for multiple public and private MIBs.
The Challenge
The Xirrus Management System manages large networks with up to 500 Wi-Fi Arrays with thousands of Wi-Fi clients. Xirrus has 40 physical Arrays in their XMS lab which are used for testing. It was budget prohibitive to grow that to 500 Wi-Fi Arrays along with the required networking gear. It was a challenge to test the XMS software with that size of a network.
The QA Assurance team continuously needs to verify the XMS’ performance and scalability in large networks. The tests need to cover the product features and all possible operational conditions. An affordable solution was sought to make the test lab more scalable and flexible.
The Solution – MIMIC Simulator
Xirrus purchased the MIMIC Simulator Suite, Campus edition, which supports simulation of up to 500 agents simultaneously. Each of the agents can represent any type of SNMP-manageable device.
Xirrus recorded the management information from one of their Arrays with the MIMIC Recorder, creating a simulation which was then cloned to 400 unique simulated devices with a total of 6000 clients connected to those arrays. Along with the simulated Array devices, they simulated traffic load between devices and clients. They could run their XMS against the simulated lab and test different features, along with performance measurements and negative/positive conditions.
The Benefits
With the use of MIMIC, Xirrus reduced the lab budget requirement to one tenth of what was initially needed. Furthermore, MIMIC made it a lot easier to test their XMS software. They were able to perform many tests effortlessly with MIMIC, which are very hard to setup with the real devices. They could now do more complete testing, and confidently release their new software to their customers.
Additionally, with MIMIC they can now save those negative/positive scenarios, add more tests, and reuse them for regression tests during every test cycle.
The Technology – MIMIC Simulator
MIMIC Simulator can simulate 20,000 SNMP, Cisco IOS, IPMI, Telnet, TFTP and DHCP-based devices on one workstation.
MIMIC appears to management applications as if they are connected to a real network. It allows simulation of any type of SNMP-based devices (Routers, Switches, Arrays, Hubs), and combine them in the creation of many different scenarios.
MIMIC ships with 1800+ MIBs and many out-of-the-box network, device and scenario script libraries, which make it very easy to simulate a variety of networks from different manufacturers. It includes a MIB compiler, recorder, network discovery wizard and topology editor to further extend the simulation capability.
The simulator allows real-time changes of device parameters to add interfaces, change traffic patterns and error conditions.
Testimonial
Ali Fatollahi, Sr. QA Engineer, said, “We needed to test our XMS software and verify its performance and scalability in large networks. Selecting MIMIC made it easy and cost effective for us to do that. With MIMIC, it is as close as you can get virtually to have a physical local network with such a scale.
MIMIC technical support is competent and prompt. They were always there and helped so effectively that now we use MIMIC daily for our XMS development and testing.
MIMIC is a great product (ease of use and phenomenal tech support) and has saved us time and money. We strongly recommend it.”
--------------
© 2008 Gambit Communications, Inc. All Rights Reserved.
MIMIC is a registered trademark of Gambit Communications, Inc.
All other trademarks or service marks are the property of their respective owners.
Thursday, February 28, 2008
MIMIC FAQ: What is the fastest SNMP simulation I can run? Why is it not the default?
Q. What is the fastest SNMP simulation I can run? Why is it not the default?
A. By default, MIMIC runs the most useful basic simulation clause for a wide variety of applications: a close snapshot of a device SNMP agent, with static objects taking on the values that were observed, and Counter objects taking on a rate that was interpolated from observed values, both entirely configurable at run-time. For details, see the QuickStart tutorial.
This default basic simulation is more expensive than faster, simpler simulations. The simplest simulation is to just return a constant value for all object instances. If the management application does not care about values retrieved, then you can use this simulation.
The sample simulation clauses that were compared in a performance experiment were (for more details, see the SIMULATE clause section in the Compiler Guide):
* INDEX
* SIMULATE{ 1 }
* SIMULATE{ constant(1) }
* SIMULATE{ uniform(1) }
* SIMULATE{ constant_per_tu(1, 60) }
* SIMULATE{ uniform_per_tu(1, 60) }
* SIMULATE{ constant(lookup("r")) }
* SIMULATE{ uniform(lookup("r")) }
* SIMULATE{ constant_per_tu(lookup("r"), lookup("tu")) }
* SIMULATE{ uniform_per_tu(lookup("r"), lookup("tu")) }
* SIMULATE{ uniform_per_tu ( (lookup ("r") == 0 ? 1 : lookup ("r")), lookup ("tu")) }
Here is a chart comparing the performance of different simulation clauses for objects. When requests are performed with only one variable per PDU, the difference in performance between the fastest and default simulations is approximately 50%. But, the more variables that are packed into a PDU, the larger the computation part becomes (compared to network overhead, etc), and for 100 variables per PDU, the difference between the fastest and default simulations is a factor of more than 5.
A. By default, MIMIC runs the most useful basic simulation clause for a wide variety of applications: a close snapshot of a device SNMP agent, with static objects taking on the values that were observed, and Counter objects taking on a rate that was interpolated from observed values, both entirely configurable at run-time. For details, see the QuickStart tutorial.
This default basic simulation is more expensive than faster, simpler simulations. The simplest simulation is to just return a constant value for all object instances. If the management application does not care about values retrieved, then you can use this simulation.
The sample simulation clauses that were compared in a performance experiment were (for more details, see the SIMULATE clause section in the Compiler Guide):
* INDEX
* SIMULATE{ 1 }
* SIMULATE{ constant(1) }
* SIMULATE{ uniform(1) }
* SIMULATE{ constant_per_tu(1, 60) }
* SIMULATE{ uniform_per_tu(1, 60) }
* SIMULATE{ constant(lookup("r")) }
* SIMULATE{ uniform(lookup("r")) }
* SIMULATE{ constant_per_tu(lookup("r"), lookup("tu")) }
* SIMULATE{ uniform_per_tu(lookup("r"), lookup("tu")) }
* SIMULATE{ uniform_per_tu ( (lookup ("r") == 0 ? 1 : lookup ("r")), lookup ("tu")) }
Here is a chart comparing the performance of different simulation clauses for objects. When requests are performed with only one variable per PDU, the difference in performance between the fastest and default simulations is approximately 50%. But, the more variables that are packed into a PDU, the larger the computation part becomes (compared to network overhead, etc), and for 100 variables per PDU, the difference between the fastest and default simulations is a factor of more than 5.
Thursday, February 14, 2008
Network Computing Magazine: "Gambit to Scale India Operations "
Gambit Communications, Inc., leader in SNMP simulation
software, will scale operations in India, according to this article.
Excerpt:
Gambit Communications, one of the providers in network and SNMP simulation tools that enhance the productivity of management software developers and enterprise users is scaling up India presence.
software, will scale operations in India, according to this article.
Excerpt:
Gambit Communications, one of the providers in network and SNMP simulation tools that enhance the productivity of management software developers and enterprise users is scaling up India presence.
Silicon India: How to MIMIC your complex Network
Gambit Communications, Inc., leader in SNMP simulation,
was named company of the month in the December issue of Silicon India
magazine.
Excerpt:
MIMIC Simulator, an innovation by Gambit Communications offers a virtual lab environment for enterprises and cuts down the time and money for testing.
was named company of the month in the December issue of Silicon India
magazine.
Excerpt:
MIMIC Simulator, an innovation by Gambit Communications offers a virtual lab environment for enterprises and cuts down the time and money for testing.
Subscribe to:
Posts (Atom)
