Linux Network Performance

Linux Network Performance

RAW ethernet vs. UDP

Andreas Schaufler

Abstract

This document describes how to do RAW ethernet programming on Linux and gives a comparison of Linux' networking performance on UDP and ethernet layer.

A visitor of my website has pointed out to me that there is a mistake in the performance measuring code. This means the figures presented are very likely wrong. Currently I do not have the time to rerun the tests. The value of this article is therefore reduced to that it is a tutorial for RAW Ethernet programming on Linux but nothing more.


List of Tables

3.1. Results

List of Examples

Chapter 1. RAW ethernet programming

The first chapter gives a short overview of how to establish a RAW ethernet communication between two Linux hosts.

Networking layers

IP networking is using four layers:

Normaly applications are placed above the TCP/UDP layer. Thus, communication between two programs is going two times through all the layers. On the sender side from TCP/ UDP down to the physical layer, on the receiver side from the physical layer up to the TCP/ UDP layer. The actual data transmission is carried out by the physical layer.

However, it is also possible for two programs to communicate on the ethernet layer. In this case the IP and the TCP/UDP layer are not used and as a result an increased data transmission rate is to be expected. It is one purpose of this document to show exact performance figures.

UDP

First of all let us take a look at UDP. It is necessary to create a socket. This socket can later be used to send or receive a UDP message.

Example 1.1. Create a UDP socket

#include<string.h>
#include<sys/socket.h>
#include<netinet/in.h>
						
int create_udp_socket(int port) {
	/*socketdescriptor*/
	int    s;				
	/*struct used for binding the socket to a local address*/
	struct sockaddr_in host_address;	
	
	/*create the socket*/
	s=socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP);
	if (s < 0) { /*errorhandling ....*/}
	
	/*init the host_address the socket is beeing bound to*/
	memset((void*)&host_address, 0, sizeof(host_address));
	/*set address family*/
	host_address.sin_family=PF_INET;
	/*accept any incoming messages:*/
	host_address.sin_addr.s_addr=INADDR_ANY;
	/*the port the socket i to be bound to:*/
	host_address.sin_port=htons(port);
	
	/*bind it...*/
	if (
	   bind(s, (struct sockaddr*)&host_address, sizeof(host_address)) < 0
	   ) {
		/*errorhandling...*/
	}
	return s;
}

Example 1.2. Use a UDP socket to send a message

#define BUF_SIZE 1000
#define LOCAL_PORT 5000
#define REMOTE_PORT 5000

...

int s; /*the socket descriptor*/
char buffer[BUF_SIZE]; 			/*the message to send*/
struct sockaddr_in target_host_address;	/*the receiver's address*/
unsigned char* target_address_holder;	/*a pointer to the ip address*/

...

/*create the socket*/
s = create_udp_socket(LOCAL_PORT);
if (s == -1) {errorhandling.....}

/*init target address structure*/
target_host_address.sin_family=PF_INET;
target_host_address.sin_port=htons(REMOTE_PORT);
target_address_holder=(unsigned char*)&target_host_address.sin_addr.s_addr;
target_address_holder[0]=10;
target_address_holder[1]=0;
target_address_holder[2]=0;
target_address_holder[3]=1;

/*fill message with random data....*/
for (j = 0; j < BUF_SIZE; j++) {
	buffer[j] = (unsigned char)((int) (255.0*rand()/(RAND_MAX+1.0)));
}

/*send it*/
sendto(s, buffer, BUF_SIZE, 0, 
	(struct sockaddr*)&target_host_address, sizeof(struct sockaddr));

Example 1.3. Use a UDP socket to receive a message

#define BUF_SIZE 1000
#define LOCAL_PORT 5000

...

char buffer[BUF_SIZE];

/*address of the sender will be stored here*/
struct sockaddr_in host_address;
int hst_addr_size = sizeof(host_address);

/*length of the incoming packet*/
int length = 0;					
/*socketdescriptor*/
int s;						

...

/*create the socket*/
s = create_udp_socket(LOCAL_PORT); 
if (s == -1) {errorhandling.....}

/*wait for incoming message*/
length = recvfrom(s, buffer, BUF_SIZE, 0, 
	(struct sockaddr*)&host_address, &hst_addr_size);

RAW ethernet

Now let us look at RAW ethernet communication. From the programmers' point of view it is quite similar to UDP. The differences are:

  • the parameters for the function used to create a socket

  • instead of IP addresses MAC addresses are used

  • an ethernet frame needs to be created manually

ethernet Frame Spec (IEEE 802.3)

Example Code

Example 1.4. Create a RAW ethernet socket

#include <sys/socket.h>
#include <linux/if_packet.h>
#include <linux/if_ether.h>
#include <linux/if_arp.h>

...

int s; /*socketdescriptor*/

s = socket(AF_PACKET, SOCK_RAW, htons(ETH_P_ALL));
if (s == -1) { errorhandling ... }

Example 1.5. Send a RAW ethernet frame

#define ETH_FRAME_LEN 1518

...

/*target address*/
struct sockaddr_ll socket_address;

/*buffer for ethernet frame*/
void* buffer = (void*)malloc(ETH_FRAME_LEN);
 
/*pointer to ethenet header*/
unsigned char* etherhead = buffer;
	
/*userdata in ethernet frame*/
unsigned char* data = buffer + 14;
	
/*another pointer to ethernet header*/
struct ethhdr *eh = (struct ethhdr *)etherhead;
 
int send_result = 0;

/*our MAC address*/
unsigned char src_mac[6] = {0x00, 0x01, 0x02, 0xFA, 0x70, 0xAA};

/*other host MAC address*/
unsigned char dest_mac[6] = {0x00, 0x04, 0x75, 0xC8, 0x28, 0xE5};

/*prepare sockaddr_ll*/

/*RAW communication*/
socket_address.sll_family   = PF_PACKET;	
/*we don't use a protocoll above ethernet layer
  ->just use anything here*/
socket_address.sll_protocol = htons(ETH_P_IP);	

/*index of the network device
see full code later how to retrieve it*/
socket_address.sll_ifindex  = 2;

/*ARP hardware identifier is ethernet*/
socket_address.sll_hatype   = ARPHRD_ETHER;
	
/*target is another host*/
socket_address.sll_pkttype  = PACKET_OTHERHOST;

/*address length*/
socket_address.sll_halen    = ETH_ALEN;		
/*MAC - begin*/
socket_address.sll_addr[0]  = 0x00;		
socket_address.sll_addr[1]  = 0x04;		
socket_address.sll_addr[2]  = 0x75;
socket_address.sll_addr[3]  = 0xC8;
socket_address.sll_addr[4]  = 0x28;
socket_address.sll_addr[5]  = 0xE5;
/*MAC - end*/
socket_address.sll_addr[6]  = 0x00;/*not used*/
socket_address.sll_addr[7]  = 0x00;/*not used*/


/*set the frame header*/
memcpy((void*)buffer, (void*)dest_mac, ETH_ALEN);
memcpy((void*)(buffer+ETH_ALEN), (void*)src_mac, ETH_ALEN);
eh->h_proto = 0x00;
/*fill the frame with some data*/
for (j = 0; j < 1500; j++) {
	data[j] = (unsigned char)((int) (255.0*rand()/(RAND_MAX+1.0)));
}

/*send the packet*/
send_result = sendto(s, buffer, ETH_FRAME_LEN, 0, 
	      (struct sockaddr*)&socket_address, sizeof(socket_address));
if (send_result == -1) { errorhandling... }

Example 1.6. Receive a RAW ethernet frame

void* buffer = (void*)malloc(ETH_FRAME_LEN); /*Buffer for ethernet frame*/
int length = 0; /*length of the received frame*/ 
...
length = recvfrom(s, buffer, ETH_FRAME_LEN, 0, NULL, NULL);
if (length == -1) { errorhandling .... }

Chapter 2. Testing Environment

Table of Contents

Equipment
What was measured ?

In this chapter the testing environment is described.

Equipment

The following hardware equipment was used:

  • Netgear Fast ethernet Switch FS516 16-Port

  • Host 1:

    • Intel Pentium IV 2.0 GHz

    • Intel Corp. 82801BD PRO/100 VE (LOM)

    • Linux 2.4.18

  • Host 2:

    • Intel Celeron 533 MHz

    • 3Com Corporation 3c905C-TX [Fast Etherlink]

    • Linux 2.4.20

  • Host 3:

    • Intel Pentium III M 500 MHz

    • SMC8035TX CardBus 10M/100M (PCMCIA)

    • Linux 2.4.20

  • Host 4:

    • AMD Athlon 1200 MHz

    • 3Com Corporation 3c905C-TX/TX-M [Tornado]

    • Linux 2.4.18

What was measured ?

  • two hosts were communicating with each other

  • amount of user data in the packets was varying from 50 octets up to 1500 octets. (step: 50 octets)

  • each measurement was carried out for UDP and RAW ethernet

  • the value taken as result is the average of 100.000 measurements for one roundtrip (back and forth both hosts, as shown below)

way of a packet

Chapter 3. Testing Results

This chapter shows the testing results.

Results in numbers

Table 3.1. Results

* Host3 / Host 4 (SMC/ 3Com) Host1 / Host 4 (Intel/ 3Com) Host2 / Host 4 (3Com/ 3Com)
#octets RAW (us) UDP (us) RAW (us) UDP (us) RAW (us) UDP (us)
             
50 29 227 116 100 14 208
100 19 290 104 119 13 227
150 13 296 124 137 14 232
200 21 360 142 154 26 285
250 22 361 159 173 27 300
300 22 418 177 192 27 306
350 20 442 196 209 28 355
400 18 478 214 226 30 366
450 23 511 232 244 30 371
500 17 523 251 263 31 384
550 19 590 116 263 32 426
600 18 601 124 330 33 447
650 18 651 50 317 34 453
700 17 682 10 335 35 467
750 17 708 10 394 35 505
800 17 751 10 372 36 520
850 17 765 10 388 36 524
900 17 816 10 406 37 557
950 17 823 10 424 38 593
1000 17 874 10 443 38 595
1050 17 906 10 461 39 611
1100 18 901 10 478 40 641
1150 21 915 11 496 41 664
1200 21 952 11 558 41 671
1250 21 965 11 532 42 697
1300 19 973 11 550 43 742
1350 17 976 11 568 44 739
1400 17 1000 11 586 44 745
1450 22 1142 12 605 45 781
1500 19 1155 12 633 46 767

.

Comparison of UDP results

The following graph is showing the results of the UDP measurements for each of the different hardware configurations.

comparison of UDP performance

These results are not surprising. The more user data a packet contains the more time is taken for one roundtrip (linear growing). It is also possible to see that the performance is different for different hardware configurations.

Comparison of RAW results

The following graph is showing the results of the RAW ethernet measurements for each of the different hardware configurations.

comparison of RAW ethernet performance

Looking at SMC/ 3com and 3com/ 3com performance the results are as expected. The time required for one roundtrip is increasing almost linearly, when the amount of user data is increasing. However, the results which were produced by the Intel NIC were unexpected. Although the Intel NIC was very slow when it came to sending packets containing 50 up tp 600 octets of user data, it was outperforming the other two, when the amount of data in a packet is increasing. A possible explanation might be that the driver or the hardware is somehow optimized for larger packages. However this behaviour is not reflected in the UDP comparison graph above.

Comparison of UDP and RAW ethernet results

Finally both graphs are merged into one diagram, in order to compare the performance of UDP and RAW ethernet.

The results show that RAW ethernet in general is much faster than UDP. Looking at the features that are offered by UDP and RAW ethernet it becomes visible that those two are quite similiar. Both are not connection oriented, both are packet based and both do not give any guarantee for success of transmission. Thus, in the view of real world systems which are currently using UDP running over ethernet or systems which require very high network performance, RAW ethernet could offer a very good solution.

posted on 2006-10-22 01:58 darkstax 阅读(647) 评论(0)  编辑 收藏 引用 所属分类: linux技术

只有注册用户登录后才能发表评论。
<2024年4月>
31123456
78910111213
14151617181920
21222324252627
2829301234
567891011

导航

统计

常用链接

随笔分类(17)

随笔档案(19)

文章分类(27)

文章档案(28)

新闻分类

新闻档案(15)

相册

收藏夹

c

版面相关

积分与排名