Hi Bogdan,

I must admit that I've never seen anyone have much success with
USB-to-ethernet adapters. In fact, you're the one to report the highest
PC-to-PC bandwidth I've heard of, so I'd be interested to know how many
packet errors and dropped packets you saw with iperf. Also, what is your
CPU load whilst doing that rate? Do these adapters offload checksumming,
or is that an additional burden to your CPU?

Depending on your nc/iperf usage, you might have made advantageous usage
of higher-level flow control in TCP; since the USRPs talk UDP only,
there's no possibility to resend packets.

I've actually often seen these adapters simply drop or reorder packets
under load. Whilst a benchmarking tool might be tolerant to reordering,
UHD is not; maybe that's what you're seeing (though you'd usually see
'S'equence errors instead of 'D's).

You might want to experiment with smaller and bigger packet sizes, see
[1]; the thing is that USB adapters have to cut up the gigabit ethernet
frames (which can be rather long, depending on what your network
hardware supports) into small USB3 packets (which can only be up to 1KB
long), and the CPU then has to reassemble the ethernet packets. That
means a lot of workload, so maybe it's better to sacrifice a bit of
network to overload, but make the UHD sample packets small enough to fit
into one USB packet.

Greetings,
Marcus

[1]
http://files.ettus.com/manual/page_transport.html#transport_udp_params ;
you'd screw around with recv_frame_size, setting it to something like
900 for a start.||

On 03/10/2015 03:33 PM, Bogdan Diaconescu via USRP-users wrote:

You're asking the right question :)
So at this point, you're right, analyzing what goes wrong during
communication sounds wise. I think the tool of choice here is wireshark,
which has an UHD display filter; I haven't used that myself, though, but
from the source code I'd assume it would display sequence numbers.

I was wrong on "S" being the indication for sequence errors, by the way
[1], so what we'd try to find out would be whether there's dropped
packets or reordered packets or something really goes wrong on the UHD
side.

Greetings,
Marcus

[1] lib/transport/super_recv_packet_handler.hpp:571
https://github.com/EttusResearch/uhd/blob/master/host/lib/transport/super_recv_packet_handler.hpp#L571

On 03/10/2015 11:09 PM, Bogdan Diaconescu wrote:

Hi Bogdan,

Um, good question. My version (1.12.3 as installed in Fedora 21) has the
dissector, but you might need to explicitely do "decode as..." on the
packets; I'm sorry, I haven't tested that with actual traffic myself.

Actually, you've got a point, but on the other hand, printing
documentation is kind of undesirable for a library (too much chattering
annoys users).

Well, that's what iperf/nc can already do. Simulating a USRP down to the
finer aspects of network behaviour is inherently complex (part of the
network handling being spread across different firmwares and FPGA
implementations), so I'd argue that this is not really something that
should be done in the UHD framework.

However, maybe we should canonize our test set ups. What I'm currently
doing is on my laptop, simulating a single threaded, single buffer
application

netstat -s|grep -i udp: -A 6 > before
iperf -u -s -P 1 -p 7000
netstat -s|grep -i udp: -A 6 > after

and on the server here that I randomly have access to (simulating the
data rate of 25MS/s complex 16 bit)

iperf -P4 -c $laptop -u -p7000 -b $(( 25* 1000000 216 ))

which yields:

Client connecting to $laptop, UDP port 7000
Sending 1470 byte datagrams
UDP buffer size:  224 KByte (default)

[  3] local $server port 56983 connected with $laptop port 7000
[ ID] Interval      Transfer    Bandwidth
[  3]  0.0-10.0 sec  1001 MBytes    840 Mbits/sec
[  3] Sent 714285 datagrams
[  3] Server Report:
[  3]  0.0-10.0 sec    964 MBytes    808 Mbits/sec  0.022 ms
26851/714284 (3.8%)
[  3]  0.0-10.0 sec  1 datagrams received out-of-order

I've varied some parameters, but at these rates, I'm constantly getting
a packet loss of 3.7-3.8% for my network hardware. Comparing the
statistics before and after:

diff before after

yields exactly the number of lost packets iperf reports as buffer and as
packet recv errors, so these are things that have reached my card's
driver, but my laptop was unable to process. I blame the aged CPU of my
X201.

Can you try the same?

Greetings,
Marcus

On 03/11/2015 01:10 PM, Bogdan Diaconescu wrote:

Yeah; I get the same feeling, something's fishy with my NIC; actually,
my Laptop has an i5 and an intel controller, too, so I don't really know
what's going wrong at my end. Anyway, that doesn't really contribute to
solving your issue, so let's leave the poor thing alone.

So the idea is to capture UDP traffic coming from / going to the IP
address of your USRP:
when using wireshark, open the "capture options" dialog, then use "host
XXX.XXX.XXX.XXX && udp" in the "capture filter field" (of course,
replacing XXX with your USRP's IP address). The overall dialog
should/could somewhat like the attached picture.

Then, run a small test (rx_samples_to_file --nsamps 10000 or something,
the minimum amount necessary to get a D), and have a look at the
incoming packets. Maybe you could save the capture, and share it
(compressed?) somehow. I think it would be nice to find out whether
there's reordering or dropping taking place; or if there's simply some
packets that fail checksum checks or so.

Greetings,
Marcus

On 03/11/2015 04:25 PM, Bogdan Diaconescu wrote:

S is most definitely a sequence error….but on TX….note a sequence error can indicate 2 things packet re-ordering or lost packet.
D is the equivalent RX direction error.

Thanks so much!

On 03/11/2015 09:27 PM, Bogdan Diaconescu wrote:

Hi Bogdan,

my initial analysis shows no dropped packets (yay), but strangely
reordered ones (wut?). The UHD dissector doesn't seem to be that useful
anymore.
Thus: You can find my own analysis script at[1].

I've first filtered the capture to only contain sample packets coming
from the device by UDP source port.

Then, I've saved that filtered output in PCAP (as opposed to PCAPng)
format, since dpkt (which is the only packet capture analysis python
module I'm somewhat familiar with) only knows how to deal with that. See
the capture file[2]
https://gist.github.com/marcusmueller/b0ccc0c9b07e6704905a/raw/58e9f4be9636ac194478886c6d0648ac08369f51/n210_capture_samples_only.pcap

Then, I've run the script on that:

|./analyze.py n210_capture_samples_only.pcap
|

giving me

|Packet    45788: Seq. Nr 7 does not follow 5. Time 1426104210.261552, length 1472B
Packet    45789: Seq. Nr 6 does not follow 7. Time 1426104210.261187, length 1472B
Packet    45790: Seq. Nr 8 does not follow 6. Time 1426104210.261913, length 1472B
Packet    53602: Seq. Nr D does not follow B. Time 1426104213.097992, length 1472B
Packet    53606: Seq. Nr C does not follow 0. Time 1426104213.097629, length 1472B
Packet    53607: Seq. Nr 1 does not follow C. Time 1426104213.099440, length 1472B
|

Packets 45788 and 45789 came in reordered, and 53606 should have come
after 53601.

• 45788/45789: Ok, hm, switched places on consecutive packets.
• 53602/53606: a packet delayed by four packets? I'm a bit confused.

Now, I must admit I have no idea whether that reordering takes place
in the device (that would require at least 5*(1516 B/ethernet packet)
buffer) or in the kernel. In the latter case, there might be something
that one can do about it. I'd have to investigate that myself, though,
since I have no idea if one could enforce non-reordering behaviour (I
don't even know which advantage reordering packets would have).

Greetings,
Marcus

[1] https://gist.github.com/marcusmueller/b0ccc0c9b07e6704905a
[2]
https://gist.github.com/marcusmueller/b0ccc0c9b07e6704905a/raw/58e9f4be9636ac194478886c6d0648ac08369f51/n210_capture_samples_only.pcap

On 03/12/2015 10:01 AM, Bogdan Diaconescu wrote:

Hello,

so the question is: When you run iperf in UDP mode at 32MBit/s (equiv.
1MS/s @16bit), do you see any message about reordered packets?

Greetings,
Marcus

On 03/12/2015 01:14 PM, Marcus Müller wrote:

Ah yet another question (having skimmed through the AX88179_178a.c
kernel driver): There's no reordering happening in the device driver, as
far as I can tell. You don't happen to have a switch or router between
your USB/ethernet adapter and the USRP?

On 03/12/2015 01:31 PM, Marcus Müller wrote:

Hi Marcus,
so the results are that there are some reordered packets. The number of reordered packets seem to be about the number of Ds I was seeing.
To answer to your questions: 
"so the question is: When you run iperf in UDP mode at 32MBit/s (equiv. 1MS/s @16bit), do you see any message about reordered packets?"

"Ah yet another question (having skimmed through the AX88179_178a.c kernel driver): There's no reordering happening in the device driver, as far as I can tell. You don't happen to have a switch or router between your USB/ethernet adapter and the USRP?"

Bogdan

 On Thursday, March 12, 2015 3:01 PM, Marcus Müller <marcus.mueller@ettus.com> wrote:

Ah yet another question (having skimmed through the AX88179_178a.c kernel driver): There's no reordering happening in the device driver, as far as I can tell. You don't happen to have a switch or router between your USB/ethernet adapter and the USRP?

On 03/12/2015 01:31 PM, Marcus Müller wrote:

Hello,

so the question is: When you run iperf in UDP mode at 32MBit/s (equiv. 1MS/s @16bit), do you see any message about reordered packets?

Greetings,
Marcus

On 03/12/2015 01:14 PM, Marcus Müller wrote:

Hi Bogdan,

my initial analysis shows no dropped packets (yay), but strangely reordered ones (wut?). The UHD dissector doesn't seem to be that useful anymore.
Thus: You can find my own analysis script at[1].
I've first filtered the capture to only contain sample packets coming from the device by UDP source port. Then, I've saved that filtered output in PCAP (as opposed to PCAPng) format, since dpkt (which is the only packet capture analysis python module I'm somewhat familiar with) only knows how to deal with that. See the capture file[2] Then, I've run the script on that:  ./analyze.py n210_capture_samples_only.pcap
giving me Packet    45788: Seq. Nr 7 does not follow 5. Time 1426104210.261552, length 1472B
Packet    45789: Seq. Nr 6 does not follow 7. Time 1426104210.261187, length 1472B
Packet    45790: Seq. Nr 8 does not follow 6. Time 1426104210.261913, length 1472B
Packet    53602: Seq. Nr D does not follow B. Time 1426104213.097992, length 1472B
Packet    53606: Seq. Nr C does not follow 0. Time 1426104213.097629, length 1472B
Packet    53607: Seq. Nr 1 does not follow C. Time 1426104213.099440, length 1472B
Packets 45788 and 45789 came in reordered, and 53606 should have come after 53601.

• 45788/45789: Ok, hm, switched places on consecutive packets.
• 53602/53606: a packet delayed by four packets? I'm a bit confused.
  Now, I must admit I have no idea whether that reordering takes place in the device (that would require at least 5*(1516 B/ethernet packet) buffer) or in the kernel. In the latter case, there might be something that one can do about it. I'd have to investigate that myself, though, since I have no idea if one could enforce non-reordering behaviour (I don't even know which advantage reordering packets would have).

Greetings,
Marcus

[1] https://gist.github.com/marcusmueller/b0ccc0c9b07e6704905a
[2] https://gist.github.com/marcusmueller/b0ccc0c9b07e6704905a/raw/58e9f4be9636ac194478886c6d0648ac08369f51/n210_capture_samples_only.pcap

On 03/12/2015 10:01 AM, Bogdan Diaconescu wrote:

Hi Marcus,
here is the info: http://iot.mobil4.org/uhd/
Let me know your thoughts and findings.

Thanks, Bogdan

   On Thursday, March 12, 2015 8:54 AM, Marcus Müller <marcus.mueller@ettus.com> wrote:


Thanks so much!

On 03/11/2015 09:27 PM, Bogdan Diaconescu wrote:

Thanks Ian, good to know that D is for both reordering and missing packets.

Now, I did a wireshark capture for a spectrum capture with 1Ghz frequency and 1Msps  until I saw severals Ds on command line.
I attach a screenshot of the packet decoding which looks ugly and perhaps tells that my  wireshark v1.10.x, the default in Ubuntu 14.04, is old. I'm I right here?
I wonder if the 1516 frame size that I see is good for UHD?
I will put the pcap and spectrum record on a server in the wild tomorrow mornig.
Thanks,
Bogdan

   On Wednesday, March 11, 2015 8:02 PM, Ian Buckley <ianb@ionconcepts.com> wrote:


S is most definitely a sequence error….but on TX….note a sequence error  can indicate 2 things packet re-ordering or lost packet. D is the equivalent RX direction error. 

I was wrong on "S" being the indication for  sequence errors, by the way [1], so what we'd try to find out would be  whether there's dropped packets or reordered  packets or something really goes wrong on the UHD side.

Greetings,
Marcus

[1] lib/transport/super_recv_packet_handler.hpp:571
https://github.com/EttusResearch/uhd/blob/master/host/lib/transport/super_recv_packet_handler.hpp#L571

On 03/10/2015 11:09 PM, Bogdan Diaconescu  wrote:

Ok, so I did several tests with various buffers passed in the uhd_fft command line:
I tried almost everything in the reference given by you Marcus going wild from min to  max and mostly notable lowering sent_frame_size down to 100 does reduce a bit the  rate of Ds. However, it seems that the moment when Ds start to appear is delayed because of that  but once they start appearing they continue to increase.

And esentially the Ds are showing up in any situation I tried.

It will be nice to have a test built in FPGA so that once triggered from the host, it  will send known data to the PC allowing testing the connection.
So, what's next?

   On Tuesday, March 10, 2015 7:24 PM, Bogdan Diaconescu <b_diaconescu@yahoo.com> wrote:


 Hi Marcus, 

those dongles are real good ones and they are the winners after almost 1 years of trial  and error with different similar devices. I so far used them  in a non USRP project though

To answer your questions: in a test with iperf (TCP) sending 10Gbit of data  between 2 PCs (consistent at 935Mbit/s) there was no packet error  and retransmission. With wireshark I've seen no TCP retransmission  at all after several tests.
We did also tests with UDP using the same setup and the speed is about 812Mbit/s  which is somehow consistent with the fact that in the  linux kernel TCP is optimized for bulk transfer but UDP is optimized for  low latency. We actually got the same results when using direct  desktop PC-2-PC connections that have onboard 1Gb/s NICs. With the same OS and kernel versions.
So, all above is w/o USRP, just PC to PC.
Now with USRP: I'll try playing with buffers according to the link you gave but for  1Msps uhd_fft should work never the less. I haven't look  at VITA and its details but for let's say 64bit/sample over the wire  you only get 64Mbit/sample which is very low and should work like a  charm.
You are right with the difference between packet size over USB vs. Ethernet. It may be  something there.
Played now with UDP Fragmentation Ofload and checksum offload with no difference,  the same problems.

I would take the processing power out of question as the processor is quite new  (broadwell i7 in 14nm).
Thanks, Bogdan

    On Tuesday, March 10, 2015 4:55 PM, Marcus Müller via USRP-users <usrp-users@lists.ettus.com> wrote:


Hi Bogdan, 

I must admit that I've never seen anyone have much success  with USB-to-ethernet adapters. In fact, you're the one to report the highest PC-to-PC bandwidth I've heard of, so I'd be  interested to know how many packet errors and dropped packets you  saw with iperf. Also, what is your CPU load whilst doing that rate? Do  these adapters offload checksumming, or is that an additional  burden to your CPU?

Depending on your nc/iperf usage, you might have made  advantageous usage of higher-level flow control in TCP; since the USRPs talk UDP only, there's no possibility to resend packets.

I've actually often seen these adapters simply drop or reorder  packets under load. Whilst a benchmarking tool might be tolerant to  reordering, UHD is not; maybe that's what you're seeing (though you'd usually  see 'S'equence errors instead of 'D's).

You might want to experiment with smaller and bigger packet sizes,  see [1]; the thing is that USB adapters have to cut up the gigabit  ethernet frames (which can be rather long, depending on what your  network hardware supports) into small USB3 packets (which  can only be up to 1KB long), and the CPU then has to reassemble the ethernet packets. That means a lot of workload, so maybe it's better to  sacrifice a bit of network to overload, but make the UHD sample  packets small enough to fit into one USB packet.

Greetings,
Marcus

[1] http://files.ettus.com/manual/page_transport.html#transport_udp_params ; you'd screw around with recv_frame_size, setting it to something like  900 for a start.

On 03/10/2015 03:33 PM, Bogdan Diaconescu via USRP-users  wrote:

Hi list,
I'm a long time user of USRP and now I'm trying to use my setup with a new laptop  that does not have an integrated ethernet card. I am using for  now two types of Ethernet 2 USB adapters that have 1Gbps/USB3.0 characteristics. 
Apparently, for PC-to-PC connection using the dongles I can get around  935Msps speed using iperf or nc which seems just fine. However, when  I do just USRP to PC connection I get many Ds on the command line just  using the simple uhd_fft. Those Ds are shown even when  using a 1Msps rate. They increase when using higher sample rates.
I wonder if someone tried this and had some success.

My set-up is:
Asus ZenBook UX31A (core i7). Ethernet 1Gbps/USB 3.0 dongles: D-Link DAB-1312 and Edimax EU-4306. Ubuntu 14.04
I should mention that I passed the phase in which the dongles connect to laptop as  USB2.0 because on the other side there was not ethernet cable  connected. Yes, this happens with both dongles.

Thanks for your support, Bogdan/YO3IIU

USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

Ah, right, I forgot you've already attached the iperf results. Thanks!
I'm currently trying to understand some linux low-level networking; I
really want to know where that reordering happens.

On 03/12/2015 03:06 PM, Bogdan Diaconescu wrote:

Yeah, in this case there is an added subsystem that is USB that the packets go through in the first place.

One question revolves in my mind, though: are we sure that USRP sends correctly the packets?
Bogdan

 On Thursday, March 12, 2015 4:19 PM, Marcus Müller <marcus.mueller@ettus.com> wrote:

Ah, right, I forgot you've already attached the iperf results. Thanks!
I'm currently trying to understand some linux low-level networking; I really want to know where that reordering happens.

On 03/12/2015 03:06 PM, Bogdan Diaconescu wrote:

Hi Marcus,
so the results are that there are some reordered packets. The number of reordered packets seem to be about the number of Ds I was seeing.
To answer to your questions: 
"so the question is: When you run iperf in UDP mode at 32MBit/s (equiv. 1MS/s @16bit), do you see any message about reordered packets?"

   "Ah yet another question (having skimmed through the AX88179_178a.c kernel driver): There's no reordering happening in the device driver, as far as I can tell. You don't happen to have a switch or router between your USB/ethernet adapter and the USRP?"

Bogdan

   On Thursday, March 12, 2015 3:01 PM, Marcus Müller <marcus.mueller@ettus.com> wrote:


Ah yet another question (having skimmed through the AX88179_178a.c kernel driver): There's no reordering happening in the device driver, as far as I can tell. You don't happen to have a switch or router between your USB/ethernet adapter and the USRP?

On 03/12/2015 01:31 PM, Marcus Müller wrote:

 Hello,

so the question is: When you run iperf in UDP mode at 32MBit/s (equiv. 1MS/s @16bit), do you see any message about reordered packets?

Greetings,
Marcus

On 03/12/2015 01:14 PM, Marcus Müller wrote:

Hi Bogdan,

my initial analysis shows no dropped packets (yay), but strangely reordered ones (wut?). The UHD dissector doesn't seem to be that useful anymore.
Thus: You can find my own analysis script at[1].
I've first filtered the capture to only contain sample packets coming from the device by UDP source port. Then, I've saved that filtered output in PCAP (as opposed to PCAPng) format, since dpkt (which is the only packet capture  analysis python module I'm somewhat familiar with) only knows how to deal with that. See the capture file[2] Then, I've run the script on that:  ./analyze.py n210_capture_samples_only.pcap
giving me Packet    45788: Seq. Nr 7 does not follow 5. Time 1426104210.261552, length 1472B
Packet    45789: Seq. Nr 6 does not follow 7. Time 1426104210.261187, length 1472B
Packet    45790: Seq. Nr 8 does not follow 6. Time 1426104210.261913, length 1472B
Packet    53602: Seq. Nr D does not follow B. Time 1426104213.097992, length 1472B
Packet    53606: Seq. Nr C does not follow 0. Time 1426104213.097629, length 1472B
Packet    53607: Seq. Nr 1 does not follow C. Time 1426104213.099440, length 1472B
Packets 45788 and 45789 came in reordered, and 53606 should have come after 53601.

• 45788/45789: Ok, hm, switched places on consecutive packets.
• 53602/53606: a packet delayed by four packets? I'm a bit confused.
  Now, I must admit I have no idea whether that reordering takes place in the device (that would require at least 5*(1516 B/ethernet packet) buffer) or in the kernel. In the latter case, there might be something that one can do about it.  I'd have to investigate that myself, though, since I have no idea if one could enforce non-reordering behaviour (I don't even know which advantage reordering packets would have).

Greetings,
Marcus

[1] https://gist.github.com/marcusmueller/b0ccc0c9b07e6704905a
[2] https://gist.github.com/marcusmueller/b0ccc0c9b07e6704905a/raw/58e9f4be9636ac194478886c6d0648ac08369f51/n210_capture_samples_only.pcap

On 03/12/2015 10:01 AM, Bogdan Diaconescu wrote:

Hi Marcus,
here is the info: http://iot.mobil4.org/uhd/
Let me know your thoughts and findings.

Thanks, Bogdan

   On Thursday, March 12, 2015 8:54 AM, Marcus Müller <marcus.mueller@ettus.com> wrote:


Thanks so much!

On 03/11/2015 09:27 PM, Bogdan Diaconescu wrote:

Thanks Ian, good to know that D is for both reordering and  missing packets.

Now, I did a wireshark capture for a spectrum capture with  1Ghz frequency and 1Msps until I saw severals Ds on command line.
I attach a screenshot of the packet decoding which looks ugly and  perhaps tells that my wireshark v1.10.x, the default in Ubuntu 14.04, is old. I'm I right here?
I wonder if the 1516 frame size that I see is good for UHD?
I will put the pcap and spectrum record on a server in the wild  tomorrow mornig.
Thanks,
Bogdan

   On Wednesday, March 11, 2015 8:02 PM,  Ian Buckley <ianb@ionconcepts.com> wrote:


S is most definitely a sequence  error….but on TX….note a sequence error can indicate 2 things packet  re-ordering or lost packet. D is the equivalent RX direction  error. 

I was wrong on "S" being the  indication for sequence errors, by the  way [1], so what we'd try  to find out would be whether  there's dropped packets or reordered packets or  something really goes wrong on the  UHD side.

Greetings,
Marcus

[1] lib/transport/super_recv_packet_handler.hpp:571
https://github.com/EttusResearch/uhd/blob/master/host/lib/transport/super_recv_packet_handler.hpp#L571

On 03/10/2015 11:09 PM,  Bogdan Diaconescu wrote:

Ok, so I did several tests with various buffers passed in the uhd_fft  command line:
I tried almost everything in the reference given by you Marcus going wild from min  to max and mostly notable lowering sent_frame_size down to 100  does reduce a bit the rate of Ds. However, it seems that the moment when Ds start to appear is delayed because of that but once  they start appearing they continue to increase.

And esentially the Ds are showing up in any situation I tried.

It will be nice to have a test built in FPGA so that once triggered from the host, it  will send known data to the PC allowing testing the  connection.
So, what's next?

   On Tuesday, March 10, 2015 7:24 PM, Bogdan Diaconescu <b_diaconescu@yahoo.com> wrote:


 Hi Marcus, 

those dongles are real good ones and they are the winners after almost 1 years of trial  and error with different similar devices. I so far used them  in a non USRP project though

To answer your questions: in a test with iperf (TCP) sending 10Gbit of data  between 2 PCs (consistent at 935Mbit/s) there was no packet error  and retransmission. With wireshark I've seen no TCP retransmission  at all after several tests.
We did also tests with UDP using the same setup and the speed is about 812Mbit/s  which is somehow consistent with the fact that in the  linux kernel TCP is optimized for bulk transfer but UDP is optimized for  low latency. We actually got the same results when using direct  desktop PC-2-PC connections that have onboard 1Gb/s NICs. With the same OS and kernel versions.
So, all above is w/o USRP, just PC to PC.
Now with USRP: I'll try playing with buffers according to the link you gave but for  1Msps uhd_fft should work never the less. I haven't look  at VITA and its details but for let's say 64bit/sample over the wire  you only get 64Mbit/sample which is very low and should work like a  charm.
You are right with the difference between packet size over USB vs. Ethernet. It may be  something there.
Played now with UDP Fragmentation Ofload and checksum offload with no difference,  the same problems.

I would take the processing power out of question as the processor is quite new  (broadwell i7 in 14nm).
Thanks, Bogdan

    On Tuesday, March 10, 2015 4:55 PM, Marcus Müller via USRP-users <usrp-users@lists.ettus.com> wrote:


Hi Bogdan, 

I must admit that I've never seen anyone have much success  with USB-to-ethernet adapters. In fact, you're the one to report the highest PC-to-PC bandwidth I've heard of, so I'd be  interested to know how many packet errors and dropped packets you  saw with iperf. Also, what is your CPU load whilst doing that rate? Do  these adapters offload checksumming, or is that an additional  burden to your CPU?

Depending on your nc/iperf usage, you might have made  advantageous usage of higher-level flow control in TCP; since the USRPs talk UDP only, there's no possibility to resend packets.

I've actually often seen these adapters simply drop or reorder  packets under load. Whilst a benchmarking tool might be tolerant to  reordering, UHD is not; maybe that's what you're seeing (though you'd usually  see 'S'equence errors instead of 'D's).

You might want to experiment with smaller and bigger packet sizes,  see [1]; the thing is that USB adapters have to cut up the gigabit  ethernet frames (which can be rather long, depending on what your  network hardware supports) into small USB3 packets (which  can only be up to 1KB long), and the CPU then has to reassemble the ethernet packets. That means a lot of workload, so maybe it's better to  sacrifice a bit of network to overload, but make the UHD sample  packets small enough to fit into one USB packet.

Greetings,
Marcus

[1] http://files.ettus.com/manual/page_transport.html#transport_udp_params ; you'd screw around with recv_frame_size, setting it to something like  900 for a start.

On 03/10/2015 03:33 PM, Bogdan Diaconescu via USRP-users  wrote:

Hi list,
I'm a long time user of USRP and now I'm trying to use my setup with a new laptop  that does not have an integrated ethernet card. I am using for  now two types of Ethernet 2 USB adapters that have 1Gbps/USB3.0 characteristics. 
Apparently, for PC-to-PC connection using the dongles I can get around  935Msps speed using iperf or nc which seems just fine. However, when  I do just USRP to PC connection I get many Ds on the command line just  using the simple uhd_fft. Those Ds are shown even when  using a 1Msps rate. They increase when using higher sample rates.
I wonder if someone tried this and had some success.

My set-up is:
Asus ZenBook UX31A (core i7). Ethernet 1Gbps/USB 3.0 dongles: D-Link DAB-1312 and Edimax EU-4306. Ubuntu 14.04
I should mention that I passed the phase in which the dongles connect to laptop as  USB2.0 because on the other side there was not ethernet cable  connected. Yes, this happens with both dongles.

Thanks for your support, Bogdan/YO3IIU

USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

Pretty sure. If I'm not completely mistaken, and Marcus Leech thinks so
too, there's not even enough state machine to hold back a packet for
five packets on the N2x0.

On 03/12/2015 04:16 PM, Bogdan Diaconescu wrote:

It is fundamentally not possible for the N210 to reorder packets.  Certain
ethernet chipsets with poor drivers show reordered or lost packets.  When
used for TCP/IP connections, these problems are masked because TCP recovers
and reorders.  Most UDP applications do as well, but UHD is too
time-sensitive for that to work.  Because of this, you need a decent
quality ethernet card is required.  When we first started with gigabit
ethernet in 2008, the available cards were of varying quality, but today,
nearly everything modern works well except for some Ethernet-over-USB
systems.

Matt

On Thu, Mar 12, 2015 at 8:35 AM, Marcus Müller usrp-users@lists.ettus.com
wrote:

Hi Matt,
I can confirm that my bunch of N210s worked pretty good for last years using various computing hardware, both desktops and laptops with integrated 1Gb links.

On my view, the new usecase I'm trying will become more of just one request in the future as the new ultrabook laptops will become the norm and the integrated ethernet will be replaced with USB3.0 to ethernet adapters. This things happens already. I'm not talking about silicon photonics users, though, just the average ones.

It may not be hardware in N210 that reorders the packets but the software in FPGA or on the path to the wire, not sure, will look at the code.
However, the important question that remains is: why replacing the USRP N210 with a PC that sends almost the same stream of data, using the same setup with the dongle, will not produce reordering of the packets.
Thanks,Bogdan

 On Friday, March 13, 2015 9:39 AM, Matt Ettus <matt@ettus.com> wrote:

It is fundamentally not possible for the N210 to reorder packets.  Certain ethernet chipsets with poor drivers show reordered or lost packets.  When used for TCP/IP connections, these problems are masked because TCP recovers and reorders.  Most UDP applications do as well, but UHD is too time-sensitive for that to work.  Because of this, you need a decent quality ethernet card is required.  When we first started with gigabit ethernet in 2008, the available cards were of varying quality, but today, nearly everything modern works well except for some Ethernet-over-USB systems.
Matt
On Thu, Mar 12, 2015 at 8:35 AM, Marcus Müller usrp-users@lists.ettus.com wrote:

Pretty sure. If I'm not completely mistaken, and Marcus Leech thinks so too, there's not even enough state machine to hold back a packet for five packets on the N2x0.

On 03/12/2015 04:16 PM, Bogdan Diaconescu wrote:

Yeah, in this case there is an added subsystem that is USB that the packets go through in the first place.

One question revolves in my mind, though: are we sure that USRP sends correctly the packets?
Bogdan

   On Thursday, March 12, 2015 4:19 PM, Marcus Müller <marcus.mueller@ettus.com> wrote:


Ah, right, I forgot you've already attached the iperf results. Thanks! 

I'm currently trying to understand some linux low-level networking; I really want to know where that reordering happens.

On 03/12/2015 03:06 PM, Bogdan Diaconescu wrote:

Hi Marcus,
so the results are that there are some reordered packets. The number of reordered packets seem to be about the number of Ds I was seeing.
To answer to your questions: 
"so the question is: When you run iperf in UDP mode at 32MBit/s (equiv. 1MS/s @16bit), do you see any message about reordered packets?"

   "Ah yet another question (having skimmed through the AX88179_178a.c kernel driver):  There's no reordering happening in the device driver, as far as I can tell. You don't happen to have a switch or router between your USB/ethernet adapter and the USRP?"

Bogdan

   On Thursday, March 12, 2015 3:01 PM, Marcus Müller <marcus.mueller@ettus.com> wrote:


Ah yet another question (having skimmed through the AX88179_178a.c kernel driver):  There's no reordering happening in the device driver, as far as I can tell. You don't happen to have a switch or router between your USB/ethernet adapter and the USRP?

On 03/12/2015 01:31 PM, Marcus Müller wrote:

 Hello,

so the question is: When you run iperf in UDP mode at 32MBit/s  (equiv. 1MS/s @16bit), do you see any message about reordered packets?

Greetings,
Marcus

On 03/12/2015 01:14 PM, Marcus Müller wrote:

Hi Bogdan,

my initial analysis shows no dropped packets (yay), but strangely reordered ones (wut?). The UHD dissector doesn't seem to be that useful anymore.
Thus: You can find my own analysis script at[1].
I've first filtered the capture to only contain sample packets coming from  the device by UDP source port. Then, I've saved that filtered output in PCAP (as opposed to PCAPng)  format, since dpkt (which is the only packet capture analysis python module I'm somewhat familiar with) only knows how to deal  with that. See the capture file[2] Then, I've run the script on that:  ./analyze.py n210_capture_samples_only.pcap
giving me Packet    45788: Seq. Nr 7 does not follow 5. Time 1426104210.261552, length 1472B
Packet    45789: Seq. Nr 6 does not follow 7. Time 1426104210.261187, length 1472B
Packet    45790: Seq. Nr 8 does not follow 6. Time 1426104210.261913, length 1472B
Packet    53602: Seq. Nr D does not follow B. Time 1426104213.097992, length 1472B
Packet    53606: Seq. Nr C does not follow 0. Time 1426104213.097629, length 1472B
Packet    53607: Seq. Nr 1 does not follow C. Time 1426104213.099440, length 1472B
Packets 45788 and 45789 came in reordered, and 53606 should have come  after 53601.

• 45788/45789: Ok, hm, switched places on consecutive packets.
• 53602/53606: a packet delayed by four packets? I'm a bit confused.
  Now, I must admit I have no idea whether that reordering takes place in the device (that would require at least 5*(1516 B/ethernet packet) buffer) or in  the kernel. In the latter case, there might be something that one can do about it. I'd have to investigate that myself, though, since I have no idea if one  could enforce non-reordering behaviour (I don't even know which advantage reordering packets would have).

Greetings,
Marcus

[1] https://gist.github.com/marcusmueller/b0ccc0c9b07e6704905a
[2] https://gist.github.com/marcusmueller/b0ccc0c9b07e6704905a/raw/58e9f4be9636ac194478886c6d0648ac08369f51/n210_capture_samples_only.pcap

On 03/12/2015 10:01 AM, Bogdan Diaconescu wrote:

Hi Marcus,
here is the info: http://iot.mobil4.org/uhd/
Let me know your thoughts and findings.

Thanks, Bogdan

   On Thursday, March 12, 2015 8:54 AM,  Marcus Müller <marcus.mueller@ettus.com> wrote:


Thanks so much!

On 03/11/2015 09:27 PM,  Bogdan Diaconescu wrote:

Thanks Ian, good to know that D is  for both reordering and missing  packets.

Now, I did a wireshark  capture for a spectrum capture with  1Ghz frequency and 1Msps  until I saw severals Ds on command line.
I attach a screenshot of  the packet decoding which looks ugly and  perhaps tells that my  wireshark v1.10.x, the default in  Ubuntu 14.04, is old. I'm I  right here?
I wonder if the 1516 frame  size that I see is good for UHD?
I will put the pcap and  spectrum record on a server in the  wild tomorrow mornig.
Thanks,
Bogdan

   On Wednesday, March 11, 2015  8:02 PM, Ian Buckley <ianb@ionconcepts.com> wrote:


S is most definitely a sequence  error….but on TX….note a sequence error can indicate 2 things packet  re-ordering or lost packet. D is the equivalent RX direction  error. 

I was wrong on "S" being the  indication for sequence errors, by the  way [1], so what we'd try  to find out would be whether  there's dropped packets or reordered packets or  something really goes wrong on the  UHD side.

Greetings,
Marcus

[1] lib/transport/super_recv_packet_handler.hpp:571
https://github.com/EttusResearch/uhd/blob/master/host/lib/transport/super_recv_packet_handler.hpp#L571

On 03/10/2015 11:09 PM,  Bogdan Diaconescu wrote:

Ok, so I did several tests  with various buffers passed in the uhd_fft command line:
I tried almost everything in the reference  given by you Marcus going wild from min to max and mostly notable  lowering sent_frame_size down to 100 does reduce a bit the rate  of Ds. However, it seems that the  moment when Ds start to  appear is delayed because of  that but once they start appearing they continue to increase.

And esentially the Ds are showing up in any  situation I tried.

It will be nice to have a test built in FPGA  so that once triggered from  the host, it will send known data to  the PC allowing testing the  connection.
So, what's next?

   On Tuesday, March 10, 2015 7:24  PM, Bogdan Diaconescu <b_diaconescu@yahoo.com> wrote:


 Hi Marcus, 

those dongles are real good ones and they are  the winners after almost 1  years of trial and error with different  similar devices. I so far used them in a non USRP project though

To answer your questions: in a test with  iperf (TCP) sending 10Gbit of data between 2 PCs (consistent at  935Mbit/s) there was no packet error and retransmission.  With wireshark I've seen no TCP  retransmission at all after  several tests.
We did also tests with UDP using the same setup  and the speed is about  812Mbit/s which is somehow  consistent with the fact that in the  linux kernel TCP is  optimized for bulk transfer but UDP is  optimized for low latency.  We actually got the same results when  using direct desktop PC-2-PC connections that have  onboard 1Gb/s NICs. With the same OS and  kernel versions.
So, all above is w/o USRP, just PC to PC.
Now with USRP: I'll try playing with buffers  according to the link you  gave but for 1Msps uhd_fft should work  never the less. I haven't look at VITA and its details  but for let's say 64bit/sample  over the wire you only get  64Mbit/sample which is very low and should  work like a charm.
You are right with the difference between packet  size over USB vs. Ethernet.  It may be something there.
Played now with UDP Fragmentation Ofload and  checksum offload with no difference, the same problems.

I would take the processing power out of  question as the processor is quite new (broadwell i7 in 14nm).
Thanks, Bogdan

    On Tuesday, March 10, 2015 4:55  PM, Marcus Müller via USRP-users <usrp-users@lists.ettus.com> wrote:


Hi Bogdan, 

I must admit that I've  never seen anyone have much success  with USB-to-ethernet  adapters. In fact, you're the one to  report the highest PC-to-PC bandwidth I've heard of, so  I'd be interested to know how many  packet errors and dropped  packets you saw with iperf. Also,  what is your CPU load whilst doing that rate? Do these adapters  offload checksumming, or is that an additional burden to your  CPU?

Depending on your nc/iperf  usage, you might have made  advantageous usage of  higher-level flow control in TCP; since  the USRPs talk UDP only, there's no possibility to resend  packets.

I've actually often seen  these adapters simply drop or reorder  packets under load. Whilst a  benchmarking tool might be tolerant to  reordering, UHD is not;  maybe that's what you're seeing (though  you'd usually see 'S'equence  errors instead of 'D's).

You might want to experiment  with smaller and bigger packet sizes,  see [1]; the thing is that  USB adapters have to cut up the gigabit  ethernet frames (which  can be rather long, depending on  what your network hardware supports) into small USB3  packets (which can only be up to 1KB long), and the CPU then has to  reassemble the ethernet packets. That  means a lot of workload, so  maybe it's better to sacrifice a  bit of network to overload,  but make the UHD sample packets small  enough to fit into one USB  packet.

Greetings,
Marcus

[1] http://files.ettus.com/manual/page_transport.html#transport_udp_params ; you'd screw around with  recv_frame_size, setting it to something like 900 for a start.

On 03/10/2015 03:33 PM,  Bogdan Diaconescu via USRP-users  wrote:

Hi list,
I'm a long time user of USRP and now I'm trying  to use my setup with a  new laptop that does not have an  integrated ethernet card. I am using for now two types of Ethernet 2  USB adapters that have 1Gbps/USB3.0  characteristics. 
Apparently, for PC-to-PC connection  using the dongles I can get around 935Msps speed using iperf or  nc which seems just fine. However, when I do just USRP to PC  connection I get many Ds on the command  line just using the  simple uhd_fft. Those Ds are shown  even when using a 1Msps rate. They increase when using higher  sample rates.
I wonder if someone tried  this and had some success.

My set-up is:
Asus ZenBook UX31A (core i7). Ethernet 1Gbps/USB 3.0 dongles: D-Link  DAB-1312 and Edimax EU-4306. Ubuntu 14.04
I should mention that I passed the phase in which  the dongles connect to  laptop as USB2.0 because on the other  side there was not ethernet cable connected. Yes, this  happens with both dongles.

Thanks for your support, Bogdan/YO3IIU

USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

Hi Bogdan,

that's what Matt referred to: There's no way the N210 as a whole system
would reorder. It's not technically possible.

That's an interesting question, but one would have to raise it with the
manufacturer of the dongle. The USRP definitely sends these packets
in-order, so this is not something that we can fix :( Actually, when
looking into the linux kernel module driver for your dongle, I couldn't,
just by looking find a place where packets might undergo reordering. So
my final guess here is that the dongle firmware reorders ethernet
packets. That's an absolute black box to me.

Greetings,
Marcus

On 03/13/2015 12:16 PM, Bogdan Diaconescu wrote:

Not technically possible? Oh, not again ... I don't think Matt is saying that.
Look, what I am going to tell you below is something based on my experience and you should not see it as a flame.  I am offering to be the first to buy a beer when we'll meet face to face as this is what I consider a friendly conversation :)

I have seen in 25 yers people signing contracts based on the "technical impossibility of failure", people guaranteeing with their finacial security based on "technical imposibility of failure", people guaranteeing with their jobs based on "technical impossibility of failure"  and I have seen the same amount of failures. Just one example, I have seen from inside the P4 floating point problem where milion of $ needed to be spent because people said it is "technically impossible to fail".
So allow me to have my own doubts when someone tells me that "it is technically impossible to fail".
I understand the customer support has ended. I still need to solve my problem. What should I say to D-Link when reporting the problem ? That I have an UDP reordering problem with one device but all the other 7 devices I am using are working just fine. Do you think D-Link customer support will answer to that?
Thanks,Bogdan

 On Friday, March 13, 2015 5:47 PM, Marcus Müller <marcus.mueller@ettus.com> wrote:

Hi Bogdan,

It may not be hardware in N210 that reorders the packets but the software in FPGA or on the path to the wire, not sure, will look at the code.,
that's what Matt referred to: There's no way the N210 as a whole system would reorder. It's not technically possible.

However, the important question that remains is: why replacing the USRP N210 with a PC that sends almost the same stream of data, using the same setup with the dongle, will not produce reordering of the packets.
That's an interesting question, but one would have to raise it with the manufacturer of the dongle. The USRP definitely sends these packets in-order, so this is not something that we can fix :( Actually, when looking into the linux kernel module driver for your dongle, I couldn't, just by looking find a place where packets might undergo reordering. So my final guess here is that the dongle firmware reorders ethernet packets. That's an absolute black box to me.

Greetings,
Marcus

On 03/13/2015 12:16 PM, Bogdan Diaconescu wrote:

Hi Matt,
I can confirm that my bunch of N210s worked pretty good for last years using various computing hardware, both desktops and laptops with integrated 1Gb links.

On my view, the new usecase I'm trying will become more of just one request in the future as the new ultrabook laptops will become the norm and the integrated ethernet will be replaced with USB3.0 to ethernet adapters. This things happens already. I'm not talking about silicon photonics users, though, just the average ones.

It may not be hardware in N210 that reorders the packets but the software in FPGA or on the path to the wire, not sure, will look at the code.
However, the important question that remains is: why replacing the USRP N210 with a PC that sends almost the same stream of data, using the same setup with the dongle, will not produce reordering of the packets.
Thanks, Bogdan

   On Friday, March 13, 2015 9:39 AM, Matt Ettus <matt@ettus.com> wrote:

It is fundamentally not possible for the N210 to reorder packets.  Certain ethernet chipsets with poor drivers show reordered or lost packets.  When used for TCP/IP connections, these problems are masked because TCP recovers and reorders.  Most UDP applications do as well, but UHD is too time-sensitive for that to work.  Because of this, you need a decent quality ethernet card is required.  When we first started with gigabit ethernet in 2008, the available cards were of varying quality, but today, nearly everything modern works well except for some Ethernet-over-USB systems.
Matt
On Thu, Mar 12, 2015 at 8:35 AM, Marcus Müller usrp-users@lists.ettus.com wrote:

Pretty sure. If I'm not completely mistaken, and Marcus Leech thinks so too, there's not even enough state machine to  hold back a packet for five packets on the N2x0.

On 03/12/2015 04:16 PM, Bogdan Diaconescu wrote:

Yeah, in this case there is an added subsystem that is USB that the packets go through in the first  place.

One question revolves in my mind, though: are we sure that USRP sends correctly the packets?
Bogdan

   On Thursday, March 12, 2015 4:19 PM, Marcus Müller <marcus.mueller@ettus.com> wrote:


Ah, right, I forgot you've already attached the iperf results. Thanks!  

I'm currently trying to understand some linux  low-level networking; I really want to know where that reordering happens.

On 03/12/2015 03:06 PM, Bogdan Diaconescu wrote:

Hi Marcus,
so the results are that there are  some reordered packets. The number of  reordered packets seem to be about the number of Ds I was seeing.
To answer to your questions: 
"so the question is: When you run iperf  in UDP mode at 32MBit/s (equiv. 1MS/s @16bit), do you see any message about  reordered packets?"

   "Ah yet another question  (having skimmed through the  AX88179_178a.c kernel driver): There's no reordering  happening in the device driver, as far as I can tell. You don't  happen to have a switch or router between  your USB/ethernet  adapter and the USRP?"

Bogdan

   On Thursday, March 12, 2015  3:01 PM, Marcus Müller <marcus.mueller@ettus.com> wrote:


Ah yet another question  (having skimmed through the  AX88179_178a.c kernel driver): There's no reordering  happening in the device driver, as far  as I can tell. You don't  happen to have a switch or router between  your USB/ethernet  adapter and the USRP?

On 03/12/2015 01:31 PM,  Marcus Müller wrote:

 Hello,

so the question is:  When you run iperf in UDP mode at  32MBit/s (equiv. 1MS/s  @16bit), do you see any message about  reordered packets?

Greetings,
Marcus

On 03/12/2015 01:14 PM,  Marcus Müller wrote:

Hi Bogdan,

my initial analysis shows  no dropped packets (yay), but strangely  reordered ones (wut?). The  UHD dissector doesn't seem to be that  useful anymore.
Thus: You can find my own  analysis script at[1].
I've first filtered the capture to  only contain sample packets coming from the device by UDP source  port. Then, I've saved that filtered  output in PCAP (as opposed to PCAPng)  format, since dpkt (which is  the only packet capture analysis  python module I'm somewhat  familiar with) only knows how to deal with  that. See the capture file[2] Then, I've run the script on  that:  ./analyze.py n210_capture_samples_only.pcap
giving me Packet    45788: Seq. Nr 7 does not follow 5. Time 1426104210.261552, length 1472B
Packet    45789: Seq. Nr 6 does not follow 7. Time 1426104210.261187, length 1472B
Packet    45790: Seq. Nr 8 does not follow 6. Time 1426104210.261913, length 1472B
Packet    53602: Seq. Nr D does not follow B. Time 1426104213.097992, length 1472B
Packet    53606: Seq. Nr C does not follow 0. Time 1426104213.097629, length 1472B
Packet    53607: Seq. Nr 1 does not follow C. Time 1426104213.099440, length 1472B
Packets 45788 and 45789 came in  reordered, and 53606 should have come after 53601.

• 45788/45789: Ok, hm, switched  places on consecutive packets.
• 53602/53606: a packet delayed by  four packets? I'm a bit confused.
  Now, I must admit I have  no idea whether that reordering  takes place in the device  (that would require at least 5*(1516  B/ethernet packet) buffer) or in the kernel. In the latter  case, there might be something that one can do about it. I'd  have to investigate that myself,  though, since I have no idea  if one could enforce non-reordering  behaviour (I don't even  know which advantage reordering  packets would have).

Greetings,
Marcus

[1] https://gist.github.com/marcusmueller/b0ccc0c9b07e6704905a
[2] https://gist.github.com/marcusmueller/b0ccc0c9b07e6704905a/raw/58e9f4be9636ac194478886c6d0648ac08369f51/n210_capture_samples_only.pcap

On 03/12/2015 10:01 AM,  Bogdan Diaconescu wrote:

Hi Marcus,
here is the info: http://iot.mobil4.org/uhd/
Let me know your thoughts and  findings.

Thanks, Bogdan

   On Thursday, March 12, 2015  8:54 AM, Marcus Müller <marcus.mueller@ettus.com> wrote:


Thanks so much!

On 03/11/2015 09:27 PM,  Bogdan Diaconescu wrote:

Thanks Ian, good to know that D is  for both reordering and missing packets.

Now, I did a wireshark  capture for a spectrum capture with  1Ghz frequency and 1Msps  until I saw severals Ds on command line.
I attach a screenshot of  the packet decoding which looks ugly and  perhaps tells that my  wireshark v1.10.x, the default in  Ubuntu 14.04, is old. I'm I  right here?
I wonder if the 1516 frame  size that I see is good for UHD?
I will put the pcap and  spectrum record on a server in the  wild tomorrow mornig.
Thanks,
Bogdan

   On Wednesday, March 11, 2015  8:02 PM, Ian Buckley <ianb@ionconcepts.com> wrote:


S is most definitely a sequence  error….but on TX….note a sequence error can indicate 2 things packet  re-ordering or lost packet. D is the equivalent RX direction  error. 

I was wrong on "S" being the  indication for sequence errors, by the  way [1], so what we'd try  to find out would be whether  there's dropped packets or reordered packets or  something really goes wrong on the  UHD side.

Greetings,
Marcus

[1] lib/transport/super_recv_packet_handler.hpp:571
https://github.com/EttusResearch/uhd/blob/master/host/lib/transport/super_recv_packet_handler.hpp#L571

On 03/10/2015 11:09 PM,  Bogdan Diaconescu wrote:

Ok, so I did several tests  with various buffers passed in the uhd_fft command line:
I tried almost everything in  the reference given by you Marcus going  wild from min to max and  mostly notable lowering sent_frame_size  down to 100 does reduce a  bit the rate of Ds. However, it  seems that the moment when Ds  start to appear is delayed  because of that but once they start  appearing they continue to  increase.

And esentially the Ds are showing  up in any situation I tried.

It will be nice to have a test  built in FPGA so that once triggered from the host, it will send  known data to the PC allowing  testing the connection.
So, what's next?

   On Tuesday, March 10, 2015  7:24 PM, Bogdan Diaconescu <b_diaconescu@yahoo.com> wrote:


 Hi Marcus, 

those dongles are real good ones  and they are the winners after almost 1 years of trial and error with  different similar devices. I so  far used them in a non USRP  project though

To answer your questions: in  a test with iperf (TCP) sending 10Gbit  of data between 2 PCs  (consistent at 935Mbit/s) there was no  packet error and retransmission. With wireshark I've seen no  TCP retransmission at all after several tests.
We did also tests with UDP using  the same setup and the speed is about 812Mbit/s which is  somehow consistent with the fact  that in the linux kernel  TCP is optimized for bulk transfer  but UDP is optimized for low latency. We actually got the same  results when using direct desktop  PC-2-PC connections  that have onboard 1Gb/s NICs. With the  same OS and kernel versions.
So, all above is w/o USRP, just  PC to PC.
Now with USRP: I'll try  playing with buffers according to  the link you gave but for  1Msps uhd_fft should work never the  less. I haven't look at VITA and its details but for let's  say 64bit/sample over the wire  you only get 64Mbit/sample  which is very low and should work like a  charm.
You are right with the difference  between packet size over USB vs. Ethernet. It may be something  there.
Played now with UDP Fragmentation  Ofload and checksum offload with  no difference, the same  problems.

I would take the processing  power out of question as the processor  is quite new (broadwell i7  in 14nm).
Thanks, Bogdan

    On Tuesday, March 10, 2015  4:55 PM, Marcus Müller via USRP-users  <usrp-users@lists.ettus.com> wrote:


Hi Bogdan, 

I must admit that I've  never seen anyone have much success  with USB-to-ethernet  adapters. In fact, you're the one to  report the highest PC-to-PC bandwidth I've heard of, so  I'd be interested to know how many  packet errors and dropped  packets you saw with iperf. Also,  what is your CPU load whilst doing that rate? Do these adapters  offload checksumming, or is that an additional burden to your  CPU?

Depending on your nc/iperf  usage, you might have made  advantageous usage of  higher-level flow control in TCP; since  the USRPs talk UDP only, there's no possibility to resend  packets.

I've actually often seen  these adapters simply drop or reorder  packets under load. Whilst a  benchmarking tool might be tolerant to  reordering, UHD is not;  maybe that's what you're seeing (though  you'd usually see 'S'equence  errors instead of 'D's).

You might want to experiment  with smaller and bigger packet sizes,  see [1]; the thing is that  USB adapters have to cut up the gigabit  ethernet frames (which  can be rather long, depending on  what your network hardware supports) into small USB3  packets (which can only be up to 1KB long), and the CPU then has to  reassemble the ethernet packets. That  means a lot of workload, so  maybe it's better to sacrifice a  bit of network to overload,  but make the UHD sample packets small  enough to fit into one USB  packet.

Greetings,
Marcus

[1] http://files.ettus.com/manual/page_transport.html#transport_udp_params ; you'd screw around with  recv_frame_size, setting it to something like 900 for a start.

On 03/10/2015 03:33 PM,  Bogdan Diaconescu via USRP-users  wrote:

Hi list,
I'm a long time user of USRP  and now I'm trying to use my setup with  a new laptop that does not  have an integrated ethernet card.  I am using for now two types  of Ethernet 2 USB adapters that have  1Gbps/USB3.0 characteristics. 
Apparently, for PC-to-PC connection  using the dongles I can get around 935Msps speed using iperf or  nc which seems just fine. However, when I do just USRP to PC  connection I get many Ds on the command  line just using the  simple uhd_fft. Those Ds are shown  even when using a 1Msps rate. They increase when using higher  sample rates.
I wonder if someone tried  this and had some success.

My set-up is:
Asus ZenBook UX31A (core i7). Ethernet 1Gbps/USB 3.0 dongles:  D-Link DAB-1312 and Edimax  EU-4306. Ubuntu 14.04
I should mention that I passed  the phase in which the dongles  connect to laptop as  USB2.0 because on the other side there was  not ethernet cable connected. Yes, this happens with  both dongles.

Thanks for your support, Bogdan/YO3IIU

USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

Hi Bogdan!

Looking forward to buying the one after that, if occasion allows :)

So, the point is, we're sure that the FPGA, which is directly generating
these packets, doesn't have the capability to reorder. The verilog code
is out there [1], and so is the firmware that we let do the "real" IP
handling (which means firmware flashing, network set up, that kind of
stuff) [2]. Ask the D-Link guys if you can have their firmware's source
code :)

I really don't think we ever stopped supporting the N210! In fact:

I think the two of us, Bogdan, have done quite a good job determining
that the packets which UHD later sees are reordered, even before they
enter the IP stack of your OS.

Now, I can understand your doubts, but I don't personally share them:
Since I can exclude the possibility that the N210 reorders, and I see
that your ethernet-level captures contain reordered packets, only the
NIC or the NIC driver might be to blame. I even went ahead and read a
bit of the latter, hoping I could spot something obvious. I didn't, so
my maximum likelihood estimation for root of this problem is the NIC itself.

Cheers!
Marcus

[1] https://github.com/EttusResearch/fpga/tree/master/usrp2/simple_gemac
: In essence, the FPGA has samples, pushes them through the gigabit
ethernet interface hardware lines (GMII_TX*). There's no buffer for packets.
[2] which is just a LWIP port;
https://github.com/EttusResearch/uhd/tree/master/firmware/usrp2