Application Note How to use Emulex OneConnect® OCe12000-D Adapters with FastStack™ Sniffer10G™ for packet injection or replay This application note explains how to install FastStack Sniffer10G and how to use the toolkit provided with this product to demonstrate its packet injection and replay capabilities. Introduction Emulex has introduced FastStack Sniffer10G (Sniffer10G), a very low cost option for building a 10Gb Ethernet (10GbE) wire-rate packet generator or packet replay tool. A key feature of Sniffer10G is its ability to support wire-rate packet injection. However, unlike packet capture, you cannot simply plug Sniffer10G into an existing application using a generic libpcap interface. While you could leverage an application programming interface (API), Emulex provides a number of useful tools with Sniffer10G1 that can be used out of the box to quickly build a powerful packet generator or packet replay device. Installing Sniffer10G 1. Download and install Sniffer10G. To obtain the download, contact help@myricom.com and request a user id and password. While Sniffer10G is available for both Microsoft Windows and Linux, Linux is used in this application note. After downloading the product, issue the following command to install Sniffer10G in /opt/snf: # rpm -i myri_snf-2.0.6.50271-2831.x86_64.rpm 1 Along with the source code Application Note 2. Validate that your OCe12000-D adapter has a valid Sniffer10G license and is ready to use. Issue the following command, as shown in Figure 1: # /opt/snf/sbin/myri_license Figure 1: Verifying that you have a valid Sniffer10G license In this example, NICs 0 and 1 have been found on system Rosebowl1 – one NIC for each port on an OCe12000-D adapter. There are license keys for Sniffer10G (SNF), as well as FastStack DBL (DBL) and VideoPump (VPUMP). If you do not see a license key for Sniffer10G, you can obtain a temporary license for testing purposes by sending an email to sales@emulex.com with the serial number of your adapter.2 Simply copy the license record you receive from Emulex and paste into a new file using a text editor. Save the file and place in a convenient location. Open a Linux terminal window as user root. Run the following command: # myri_license –f <name of license file> This procedure programs the license key into network adapters installed on a particular system and should be repeated for other systems containing network adapters. 2 2 Note that the serial number in this example is 423691. Application Note | How to use Emulex OCe12000-D Adapters with FastStack Sniffer10G Application Note 3. Use myri_start_stop command to confirm that the Sniffer10G kernel module (driver) is running. You can use start, stop or restart options to stop, start or restart the module. If you do not know the current state of the module, it may be best to use the following command, as shown in Figure 2: # myri_start_stop restart Figure 2: Verifying that the Sniffer10G code is running 4. Check the kernel’s message buffer (dmesg) to confirm that the hardware has been installed properly. Use the following command, as shown in Figure 3: # dmesg | grep myri_snf | tail -5 Figure 3: Two links are up, confirming that the hardware has been installed properly You are now ready to use Sniffer10G. Demonstrating packet generation capabilities To demonstrate the Sniffer10G packet generation and replay capabilities, you must create a physical path between ports 0 and 1 on your OCe12000-D adapter, either by wiring them together or using a switch. One of the best ways to confirm that traffic can pass easily between the ports is to run a simple receive program on one port and the packet generator on the other. Open two console windows on your server – in this example, Window-A and Window-B. 3 Application Note | How to use Emulex OCe12000-D Adapters with FastStack Sniffer10G Application Note In Window-A, run the following command (as shown in Figure 4), which sets up a port to receive packets indefinitely from a second port: # /opt/snf/bin/tests/snf_simple_recv -p0 -t 1 Figure 4: Setting up the first port to receive traffic In Window-B, run the following command, which generates fifty million 60-byte packets on port 0: # /opt/snf/bin/tests/snf_pktgen -p0 -s 60 -n 50000000 After approximately four seconds, the snf_pktgen command in Window-B finishes, with a result such as that shown in Figure 5. Figure 5: Window-B, indicating the maximum packet rate was 14.875Mpps The result in Window-A should be similar to that shown in Figure 6. Remember that Window-A has been set to receive traffic indefinitely; thus, when snf_pkt_gen finishes in Window-B, you should press Ctrl-C in Window-A. Figure 6: Window-A, indicating that Sniffer10G has received the packets on port 0; packet rate is reported every second 4 Application Note | How to use Emulex OCe12000-D Adapters with FastStack Sniffer10G Application Note By reporting data rate (Gbps) and packet rate (pps), Window-A demonstrates the efficiency of the packet generation using snf_pktgen . As calculated from the results shown in Figure 6, the actual packet rate was 14.8Mpps, the line rate for 60-byte packets. To add complexity, you could vary packet size from, say, 60 bytes to 9,000 bytes. Alternatively, it would be simple to create a script that mixes a collection of snf_pktgen commands to vary packet size and quantity over a given test cycle. Note: When creating such a script, note that the -n option of snf_pktgen defines the number of packets you wish to send. As an aside, you can see that, with such small packets, the actual bandwidth is smaller than might have been expected. This apparent anomaly is due to the gaps between packets adding up to create a significant overhead, resulting in an overall efficiency of approximately 71 percent. Larger packets carry more data, with less relative overhead. Demonstrating replay capabilities The replay tool (snf_replay) that comes with Sniffer10G allows you to play libpcap format capture files or inject such files back on to the Ethernet. You can vary playback from wire-rate down to a specified value using the -R option, which specifies the replay rate in Mpps. In this example, -R 1.0 signifies a replay rate of 1.0Mpps; in practice, however, the actual replay rate will be approximately 1.5 times the value specified by -R. As shown in Figure 7, the measured rate was 1.49Mpps. Decimal values for -R are acceptable; for example, -R 0.15 would place approximately 225,000 packets per second (0.2Mpps) on the wire. If -R is omitted, packets are injected at wire-rate. In this example, the following command (as shown in Figure 7) iterates through input file ex3.pcap 5,000 times: snf_replay -v -p0 -R 1.0 -i 5000 ex3.pcap Figure 7: Using the replay tool The -v option is used display what is happening on port 0. 5 Application Note | How to use Emulex OCe12000-D Adapters with FastStack Sniffer10G Application Note In this example, packets were paced at approximately 1.5Mpps and inserted on to port 0. As shown in Figure 8, the replay tool includes a -t option that can be used to increase the number of threads concurrently sending the file to the specified port. Figure 8: Increasing the number of threads used to send the file Since snf_replay is a command-line program, you can craft a simple script that utilizes several different commands, each with different libpcap files running at potentially different rates to build a very useful packet generation testing tool. All snf_ programs referred to above are shipped as source code with Sniffer10G so you can use them as building blocks for your own projects. Summary FastStack Sniffer10G is more than just a lossless packet capture and wire-rate injection driver for OCe12000-D adapters; it also provides tools that can be used to demonstrate the power of this platform. These tools can be leveraged out of the box to quickly build simple network testing tools for generating and replaying packets at wirerate over 10GbE. 6 Application Note | How to use Emulex OCe12000-D Adapters with FastStack Sniffer10G Application Note More information The Implementer’s Lab website www.implementerslab.com To help us improve our documents, please provide feedback at implementerslab@emulex.com. © Copyright 2012 Emulex Corporation. The information contained herein is subject to change without notice. The only warranties for Emulex products and services are set forth in the express warranty statements accompanying such products and services. Emulex shall not be liable for technical or editorial errors or omissions contained herein. 7 Application Note | How to use Emulex OCe12000-D Adapters with FastStack Sniffer10G World Headquarters 3333 Susan Street, Costa Mesa, California 92626 +1 714 662 5600 Bangalore, India +91 80 40156789 | Beijing, China +86 10 68499547 Dublin, Ireland+35 3 (0)1 652 1700 | Munich, Germany +49 (0) 89 97007 177 Paris, France +33 (0) 158 580 022 | Tokyo, Japan +81 3 5322 1348 Wokingham, United Kingdom +44 (0) 118 977 2929