Gigabit ethernet speed?
|
| Author | Content |
|---|---|
| Sander_Marechal Nov 14, 2007 3:28 PM EDT |
Can anyone help me debug my network? I've recently installed a Gigabit switch (8 Gb switching capacity) and replaced my cat5 with cat6. All machines are equipped with Gb NICs. When I tested the network speed using netcat, my calculations came down to about 380 Mbit/s instead of 900 or so which I expected. What can I do to find out what's causing the slowdown? Here's how I did the test: Listening on machine A: nc -v -v -l -p 2222 >/dev/null Sending from machine B: time yes|nc -v -v -n 192.168.1.2 2222 >/dev/null waiting a bit, then hit Ctrl+C I got: sent 630054912, rcvd 0 real 0m12.713s user 0m12.053s sys 0m2.660s That's in bytes. So multiplying by 8 and dividing the time and by 1048576 (1024*1024) I get: 378.1 Mbit/s |
| herzeleid Nov 14, 2007 3:36 PM EDT |
What does the load average look like during the test? Are you using NAPI? |
| tuxchick Nov 14, 2007 4:02 PM EDT |
Are you familiar with iperf? That's a seriously slick little utility made for this kind of testing:
http://www.enterprisenetworkingplanet.com/netos/article.php/... Any idea about your PCI bus speeds? That's often the biggest bottleneck. This article might help: http://www.enterpriseitplanet.com/networking/features/articl... |
| Sander_Marechal Nov 14, 2007 4:41 PM EDT |
Quoting:Any idea about your PCI bus speeds? That's often the biggest bottleneck. Both machines have the Gb NIC integrated on the motherboard. But even then, machine A has a recent Asus 8An-SLI board which come with 66 Mhz PCI slots. The other one is a server with 100 Mhz 64-bit PCI-X slots. The article you linked has a few nice tips. I'll try them out tomorrow. @herzeleid: I'm using stock Debian Etch. I don't think that does NAPI by default. Load is pretty low during the tests. Machine A (which has GNOME running) averages at 0.13. Machine B (the sender, no X.org) slowly climbs from 0.2 to 0.5 during the test. |
| NoDough Nov 15, 2007 5:38 AM EDT |
Not sure how netcat handles end of line, but there's a possibility that you are sending single character packets. Even if the packets are not single character, the command you used does not account for network overhead, as your time command is counting only the data, not the packet overhead. Also, I noticed that you have two -v arguments on the nc command line for machine B. I hope you meant to type -u for one of those. If not, then you'll definitely want to try that. It will force the network conversation to use UDP mode which has far less overhead. Just some thoughts. Hope it helps. Edit: Just realized that only the time stamps come from the time command, and the data count comes from netcat. Still don't know if it's counting data+overhead, though. |
| Sander_Marechal Nov 15, 2007 6:11 AM EDT |
netcat is counting the bytes it sends. So that will include the newlines. Anyway, the nc tests does match my other tests. Copying large files using scp is roughly 3,5 times faster than it was with 100 Mb/s lines and switches. |
| NoDough Nov 15, 2007 6:48 AM EDT |
> netcat is counting the bytes it sends. So that will include the newlines. Sorry, I was a little ambiguous. It's the packet overhead that I'm wondering about. I sincerely doubt that netcat is counting the overhead. My question concerning the newlines was whether that affects the size of the packets. Is nc getting 'y'+\n and sending a packet, or is it waiting until it has 1k of data, or waiting until it has 4k of data? This can make a huge difference because for every data packet there are around 20 bytes of overhead. http://en.wikipedia.org/wiki/IPv4#Packet_structure Also, with a TCP connection the packets are CRC'd and verified back to the sending node, consuming even more bandwidth. That doesn't happen with a UDP connection. So, if your packets are two bytes long traveling via TCP then your overhead is _at least_ 40 bytes to send two. If you are using UDP then your overhead is at least 20 bytes/packet. OTOH, if you are sending 4k packets the impact from the overhead is minimal. |
| Sander_Marechal Nov 15, 2007 12:54 PM EDT |
Ah. Now I understand. I don't really know how netcat does it, but other sites that explained how to use netcat to test network speed say that you should be able to get to 90% of your network speed (i.e. 900 Mbit/s in this case). So, I assume it's doing the correct think (i.e. gathering in 4K packets). Makes sense too since netcat just gets a continuous stream of yn without EOFs in between. |
| Sander_Marechal Feb 11, 2008 2:42 PM EDT |
Wow. I finally got around to re-testing my network. Turns out I had a bad connector (again!) on one of my cables. I sorta fixed it and now iperf reports 942 Mbits/sec :-) But this is I think the third or fourth time I ran into a crappy connector. I have to do a lot of crimping myself because the jacks don't fit through the holes in the wall. So I have to cut off the factory crimped connectors, run the cable and then re-crimp the ends. Can someone recommend me a decent crimping tool? Right now I'm using a simple one, but about half the connectors I crimp with it are duds because it doesn't crimp all the pins. Often it will only crimp 6 of the 8 pins. Thanks in advance. |
| NoDough Feb 13, 2008 4:20 AM EDT |
The one I use isn't all that fancy, but it seems to work reliably. http://www.newegg.com/Product/Product.aspx?Item=N82E16899888... |
| Sander_Marechal Feb 13, 2008 5:03 AM EDT |
A steel one :-) Thanks. Now to find someplace that sells them in Europe. Newegg doesn't ship here unfortunately. |
| hkwint Feb 13, 2008 2:57 PM EDT |
I suppose if the new world has Newegg, it is called Oldegg over here probably? |
| rijelkentaurus Feb 13, 2008 3:01 PM EDT |
Quoting: I suppose if the new world has Newegg, it is called Oldegg over here probably? That little joke has the smell of Rottenegg, anyway.... ;^) |
| gus3 Feb 13, 2008 9:40 PM EDT |
Is the malodor wafting from Denmark? |
You cannot post until you login.