usrp-users@lists.ettus.com
Discussion and technical support related to USRP, UHD, RFNoC
View all threadsUHD SPP problem with E320 and XG link
Hi,
I have an E320 device connected to the host with XG fiber connection and XG FPGA loaded on the E320. UHD version is 4.3.0. I have set the mtu to 9000 on both sides. Below you can find the configurations. I am trying to set spp to 1919 but I always get 492. I read in the documents that it depends on the mtu and by default it is set to the maximum value possible based on mtu. Below there is the result of benchmark rate as well.
host:$ sudo ethtool eno12399np0
Settings for eno12399np0:
Supported ports: [ FIBRE ]
Supported link modes: 1000baseT/Full
10000baseT/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 1000baseT/Full
10000baseT/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: Not reported
Link partner advertised pause frame use: Symmetric
Link partner advertised auto-negotiation: No
Link partner advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Port: Direct Attach Copper
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: g
Wake-on: d
Current message level: 0x00000000 (0)
Link detected: yes
host:$ ip -f inet addr show eno12399np0
263: eno12399np0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
inet 10.40.2.1/16 brd 10.40.255.255 scope global eno12399np0
valid_lft forever preferred_lft forever
ni-e320-3238B97:~# ip -f inet addr show sfp0
15: sfp0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc pfifo_fast qlen 1000
inet 10.40.3.1/16 brd 10.40.255.255 scope global sfp0
valid_lft forever preferred_lft forever
host:$ ./benchmark_rate --args "addr=10.40.3.1,master_clock_rate=61.44e6" --duration 60 --channels "0" --rx_rate 61.44e6 --rx_subdev "A:0" --tx_rate 61.44e6 --tx_subdev "A:0"
[INFO] [UHD] linux; GNU C++ version 9.4.0; Boost_107100; UHD_4.3.0.HEAD-0-g1f8fd345
[00:00:00.000070] Creating the usrp device with: addr=10.40.3.1,master_clock_rate=61.44e6...
[INFO] [MPMD] Initializing 1 device(s) in parallel with args: mgmt_addr=10.40.3.1,type=e3xx,product=e320,serial=3238B97,name=ni-e320-3238B97,fpga=XG,claimed=False,addr=10.40.3.1,master_clock_rate=61.44e6
[INFO] [MPM.PeriphManager] init() called with device args `fpga=XG,master_clock_rate=61.44e6,mgmt_addr=10.40.3.1,name=ni-e320-3238B97,product=e320'.
[INFO] [0/Radio#0] Performing CODEC loopback test on channel 0 ...
[INFO] [0/Radio#0] CODEC loopback test passed
[INFO] [0/Radio#0] Performing CODEC loopback test on channel 1 ...
[INFO] [0/Radio#0] CODEC loopback test passed
[INFO] [0/DmaFIFO#0] BIST passed (Estimated Minimum Throughput: 1361 MB/s)
[INFO] [0/DmaFIFO#0] BIST passed (Estimated Minimum Throughput: 1361 MB/s)
Using Device: Single USRP:
Device: E300-Series Device
Mboard 0: e320
RX Channel: 0
RX DSP: 0
RX Dboard: A
RX Subdev: E3xx
TX Channel: 0
TX DSP: 0
TX Dboard: A
TX Subdev: E3xx
[00:00:04.923395067] Setting device timestamp to 0...
Setting TX spp to 492
[00:00:04.935123785] Testing receive rate 61.440000 Msps on 1 channels
[00:00:04.977439269] Testing transmit rate 61.440000 Msps on 1 channels
UUUU[00:01:04.979477903] Benchmark complete.
Benchmark rate summary:
Num received samples: 3686373960
Num dropped samples: 0
Num overruns detected: 0
Num transmitted samples: 3686410860
Num sequence errors (Tx): 0
Num sequence errors (Rx): 0
Num underruns detected: 4
Num late commands: 0
Num timeouts (Tx): 0
Num timeouts (Rx): 0
Done
As you can see TX spp is set to 492. But I want 1919 with 9000 mtu. Is there anything I am missing here?
Best,
Samie
Hi,
I could narrow down the problem to tx_spp. Tried the following command instead for benchmark and it seems that RX spp could be set to 1919 but TX spp max that is reported by the radio is 492. Does anyone know how I can increase TX spp?
sudo ./benchmark_rate --args "mgmt_addr=10.10.3.1,addr=10.40.3.1,master_clock_rate=61.44e6" --duration 60 --channels "0" --rx_rate 61.44e6 --rx_subdev "A:0" --tx_rate 61.44e6 --tx_subdev "A:0" --rx_spp 1919 --tx_spp 1919 --tx_stream_args "spp=1919" --rx_stream_args "spp=1919"
[INFO] [UHD] linux; GNU C++ version 9.4.0; Boost_107100; UHD_4.3.0.HEAD-0-g1f8fd345
[00:00:00.000069] Creating the usrp device with: mgmt_addr=10.10.3.1,addr=10.40.3.1,master_clock_rate=61.44e6...
[INFO] [MPMD] Initializing 1 device(s) in parallel with args: mgmt_addr=10.10.3.1,type=e3xx,product=e320,serial=3238B97,name=ni-e320-3238B97,fpga=XG,claimed=False,addr=10.40.3.1,master_clock_rate=61.44e6
[INFO] [MPM.PeriphManager] init() called with device args `fpga=XG,master_clock_rate=61.44e6,mgmt_addr=10.10.3.1,name=ni-e320-3238B97,product=e320'.
[INFO] [0/Radio#0] Performing CODEC loopback test on channel 0 ...
[INFO] [0/Radio#0] CODEC loopback test passed
[INFO] [0/Radio#0] Performing CODEC loopback test on channel 1 ...
[INFO] [0/Radio#0] CODEC loopback test passed
[INFO] [0/DmaFIFO#0] BIST passed (Estimated Minimum Throughput: 1361 MB/s)
[INFO] [0/DmaFIFO#0] BIST passed (Estimated Minimum Throughput: 1361 MB/s)
Using Device: Single USRP:
Device: E300-Series Device
Mboard 0: e320
RX Channel: 0
RX DSP: 0
RX Dboard: A
RX Subdev: E3xx
TX Channel: 0
TX DSP: 0
TX Dboard: A
TX Subdev: E3xx
[00:00:04.448876213] Setting device timestamp to 0...
Setting RX spp to 1919
Setting TX spp to 492
[00:00:04.457322205] Testing receive rate 61.440000 Msps on 1 channels
[00:00:04.502995425] Testing transmit rate 61.440000 Msps on 1 channels
Best,
Samie
From: Samie Mostafavi ssmos@kth.se
Sent: Friday, December 16, 2022 11:09:56 AM
To: usrp-users@lists.ettus.com
Subject: [USRP-users] UHD SPP problem with E320 and XG link
Hi,
I have an E320 device connected to the host with XG fiber connection and XG FPGA loaded on the E320. UHD version is 4.3.0. I have set the mtu to 9000 on both sides. Below you can find the configurations. I am trying to set spp to 1919 but I always get 492. I read in the documents that it depends on the mtu and by default it is set to the maximum value possible based on mtu. Below there is the result of benchmark rate as well.
host:$ sudo ethtool eno12399np0
Settings for eno12399np0:
Supported ports: [ FIBRE ]
Supported link modes: 1000baseT/Full
10000baseT/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 1000baseT/Full
10000baseT/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: Not reported
Link partner advertised pause frame use: Symmetric
Link partner advertised auto-negotiation: No
Link partner advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Port: Direct Attach Copper
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: g
Wake-on: d
Current message level: 0x00000000 (0)
Link detected: yes
host:$ ip -f inet addr show eno12399np0
263: eno12399np0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
inet 10.40.2.1/16 brd 10.40.255.255 scope global eno12399np0
valid_lft forever preferred_lft forever
ni-e320-3238B97:~# ip -f inet addr show sfp0
15: sfp0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc pfifo_fast qlen 1000
inet 10.40.3.1/16 brd 10.40.255.255 scope global sfp0
valid_lft forever preferred_lft forever
host:$ ./benchmark_rate --args "addr=10.40.3.1,master_clock_rate=61.44e6" --duration 60 --channels "0" --rx_rate 61.44e6 --rx_subdev "A:0" --tx_rate 61.44e6 --tx_subdev "A:0"
[INFO] [UHD] linux; GNU C++ version 9.4.0; Boost_107100; UHD_4.3.0.HEAD-0-g1f8fd345
[00:00:00.000070] Creating the usrp device with: addr=10.40.3.1,master_clock_rate=61.44e6...
[INFO] [MPMD] Initializing 1 device(s) in parallel with args: mgmt_addr=10.40.3.1,type=e3xx,product=e320,serial=3238B97,name=ni-e320-3238B97,fpga=XG,claimed=False,addr=10.40.3.1,master_clock_rate=61.44e6
[INFO] [MPM.PeriphManager] init() called with device args `fpga=XG,master_clock_rate=61.44e6,mgmt_addr=10.40.3.1,name=ni-e320-3238B97,product=e320'.
[INFO] [0/Radio#0] Performing CODEC loopback test on channel 0 ...
[INFO] [0/Radio#0] CODEC loopback test passed
[INFO] [0/Radio#0] Performing CODEC loopback test on channel 1 ...
[INFO] [0/Radio#0] CODEC loopback test passed
[INFO] [0/DmaFIFO#0] BIST passed (Estimated Minimum Throughput: 1361 MB/s)
[INFO] [0/DmaFIFO#0] BIST passed (Estimated Minimum Throughput: 1361 MB/s)
Using Device: Single USRP:
Device: E300-Series Device
Mboard 0: e320
RX Channel: 0
RX DSP: 0
RX Dboard: A
RX Subdev: E3xx
TX Channel: 0
TX DSP: 0
TX Dboard: A
TX Subdev: E3xx
[00:00:04.923395067] Setting device timestamp to 0...
Setting TX spp to 492
[00:00:04.935123785] Testing receive rate 61.440000 Msps on 1 channels
[00:00:04.977439269] Testing transmit rate 61.440000 Msps on 1 channels
UUUU[00:01:04.979477903] Benchmark complete.
Benchmark rate summary:
Num received samples: 3686373960
Num dropped samples: 0
Num overruns detected: 0
Num transmitted samples: 3686410860
Num sequence errors (Tx): 0
Num sequence errors (Rx): 0
Num underruns detected: 4
Num late commands: 0
Num timeouts (Tx): 0
Num timeouts (Rx): 0
Done
As you can see TX spp is set to 492. But I want 1919 with 9000 mtu. Is there anything I am missing here?
Best,
Samie
Hi Samie,
In your example, you requested to use "addr=10.40.3.1". To use the 10 GbE
SFP port, be sure to specify the address of the XG port, which is typically
192.168.10.2. I don't think jumbo frames are supported over the 1 GbE RJ45
port.
Thanks,
Wade
On Fri, Dec 16, 2022 at 4:47 AM Samie Mostafavi ssmos@kth.se wrote:
Hi,
I could narrow down the problem to tx_spp. Tried the following command
instead for benchmark and it seems that RX spp could be set to 1919 but TX
spp max that is reported by the radio is 492. Does anyone know how I can
increase TX spp?
sudo ./benchmark_rate --args
"mgmt_addr=10.10.3.1,addr=10.40.3.1,master_clock_rate=61.44e6" --duration
60 --channels "0" --rx_rate 61.44e6 --rx_subdev "A:0" --tx_rate 61.44e6
--tx_subdev "A:0" --rx_spp 1919 --tx_spp 1919 --tx_stream_args
"spp=1919" --rx_stream_args "spp=1919"
[INFO] [UHD] linux; GNU C++ version 9.4.0; Boost_107100;
UHD_4.3.0.HEAD-0-g1f8fd345
[00:00:00.000069] Creating the usrp device with:
mgmt_addr=10.10.3.1,addr=10.40.3.1,master_clock_rate=61.44e6...
[INFO] [MPMD] Initializing 1 device(s) in parallel with args:
mgmt_addr=10.10.3.1,type=e3xx,product=e320,serial=3238B97,name=ni-e320-3238B97,fpga=XG,claimed=False,addr=10.40.3.1,master_clock_rate=61.44e6
[INFO] [MPM.PeriphManager] init() called with device args
`fpga=XG,master_clock_rate=61.44e6,mgmt_addr=10.10.3.1,name=ni-e320-3238B97,product=e320'.
[INFO] [0/Radio#0] Performing CODEC loopback test on channel 0 ...
[INFO] [0/Radio#0] CODEC loopback test passed
[INFO] [0/Radio#0] Performing CODEC loopback test on channel 1 ...
[INFO] [0/Radio#0] CODEC loopback test passed
[INFO] [0/DmaFIFO#0] BIST passed (Estimated Minimum Throughput: 1361 MB/s)
[INFO] [0/DmaFIFO#0] BIST passed (Estimated Minimum Throughput: 1361 MB/s)
Using Device: Single USRP:
Device: E300-Series Device
Mboard 0: e320
RX Channel: 0
RX DSP: 0
RX Dboard: A
RX Subdev: E3xx
TX Channel: 0
TX DSP: 0
TX Dboard: A
TX Subdev: E3xx
[00:00:04.448876213] Setting device timestamp to 0...
Setting RX spp to 1919
Setting TX spp to 492
[00:00:04.457322205] Testing receive rate 61.440000 Msps on 1 channels
[00:00:04.502995425] Testing transmit rate 61.440000 Msps on 1 channels
Best,
Samie
From: Samie Mostafavi ssmos@kth.se
Sent: Friday, December 16, 2022 11:09:56 AM
To: usrp-users@lists.ettus.com
Subject: [USRP-users] UHD SPP problem with E320 and XG link
Hi,
I have an E320 device connected to the host with XG fiber connection and
XG FPGA loaded on the E320. UHD version is 4.3.0. I have set the mtu to
9000 on both sides. Below you can find the configurations. I am trying to
set spp to 1919 but I always get 492. I read in the documents that it
depends on the mtu and by default it is set to the maximum value possible
based on mtu. Below there is the result of benchmark rate as well.
host:$ sudo ethtool eno12399np0
Settings for eno12399np0:
Supported ports: [ FIBRE ]
Supported link modes: 1000baseT/Full
10000baseT/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 1000baseT/Full
10000baseT/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: Not reported
Link partner advertised pause frame use: Symmetric
Link partner advertised auto-negotiation: No
Link partner advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Port: Direct Attach Copper
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: g
Wake-on: d
Current message level: 0x00000000 (0)
Link detected: yes
host:$ ip -f inet addr show eno12399np0
263: eno12399np0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq
state UP group default qlen 1000
inet 10.40.2.1/16 brd 10.40.255.255 scope global eno12399np0
valid_lft forever preferred_lft forever
ni-e320-3238B97:~# ip -f inet addr show sfp0
15: sfp0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc pfifo_fast qlen
1000
inet 10.40.3.1/16 brd 10.40.255.255 scope global sfp0
valid_lft forever preferred_lft forever
host:$ ./benchmark_rate --args "addr=10.40.3.1,master_clock_rate=61.44e6"
--duration 60 --channels "0" --rx_rate 61.44e6 --rx_subdev "A:0" --tx_rate
61.44e6 --tx_subdev "A:0"
[INFO] [UHD] linux; GNU C++ version 9.4.0; Boost_107100;
UHD_4.3.0.HEAD-0-g1f8fd345
[00:00:00.000070] Creating the usrp device with:
addr=10.40.3.1,master_clock_rate=61.44e6...
[INFO] [MPMD] Initializing 1 device(s) in parallel with args:
mgmt_addr=10.40.3.1,type=e3xx,product=e320,serial=3238B97,name=ni-e320-3238B97,fpga=XG,claimed=False,addr=10.40.3.1,master_clock_rate=61.44e6
[INFO] [MPM.PeriphManager] init() called with device args
`fpga=XG,master_clock_rate=61.44e6,mgmt_addr=10.40.3.1,name=ni-e320-3238B97,product=e320'.
[INFO] [0/Radio#0] Performing CODEC loopback test on channel 0 ...
[INFO] [0/Radio#0] CODEC loopback test passed
[INFO] [0/Radio#0] Performing CODEC loopback test on channel 1 ...
[INFO] [0/Radio#0] CODEC loopback test passed
[INFO] [0/DmaFIFO#0] BIST passed (Estimated Minimum Throughput: 1361 MB/s)
[INFO] [0/DmaFIFO#0] BIST passed (Estimated Minimum Throughput: 1361 MB/s)
Using Device: Single USRP:
Device: E300-Series Device
Mboard 0: e320
RX Channel: 0
RX DSP: 0
RX Dboard: A
RX Subdev: E3xx
TX Channel: 0
TX DSP: 0
TX Dboard: A
TX Subdev: E3xx
[00:00:04.923395067] Setting device timestamp to 0...
Setting TX spp to 492
[00:00:04.935123785] Testing receive rate 61.440000 Msps on 1 channels
[00:00:04.977439269] Testing transmit rate 61.440000 Msps on 1 channels
UUUU[00:01:04.979477903] Benchmark complete.
Benchmark rate summary:
Num received samples: 3686373960
Num dropped samples: 0
Num overruns detected: 0
Num transmitted samples: 3686410860
Num sequence errors (Tx): 0
Num sequence errors (Rx): 0
Num underruns detected: 4
Num late commands: 0
Num timeouts (Tx): 0
Num timeouts (Rx): 0
Done
As you can see TX spp is set to 492. But I want 1919 with 9000 mtu. Is
there anything I am missing here?
Best,
Samie
USRP-users mailing list -- usrp-users@lists.ettus.com
To unsubscribe send an email to usrp-users-leave@lists.ettus.com
Hi Wade,
Thanks for your answer. I figured out the problem. It was the switch in between that I had to reboot. I had jumbo frame option enabled on the switch but a reboot was needed. Now that I try it works:
host$ ping -M do -s 5850 10.40.3.1
PING 10.40.3.1 (10.40.3.1) 5850(5878) bytes of data.
5858 bytes from 10.40.3.1: icmp_seq=1 ttl=64 time=2.24 ms
5858 bytes from 10.40.3.1: icmp_seq=2 ttl=64 time=1.25 ms
5858 bytes from 10.40.3.1: icmp_seq=3 ttl=64 time=1.19 ms
5858 bytes from 10.40.3.1: icmp_seq=4 ttl=64 time=1.19 ms
host$ ./benchmark_rate --args "mgmt_addr=10.10.3.1,addr=10.40.3.1,master_clock_rate=61.44e6" --duration 60 --channels "0" --rx_rate 61.44e6 --rx_subdev "A:0" --tx_rate 61.44e6 --tx_subdev "A:0" --rx_spp 1919 --tx_spp 1919 --tx_stream_args "spp=1919" --rx_stream_args "spp=1919"
[INFO] [UHD] linux; GNU C++ version 9.4.0; Boost_107100; UHD_4.3.0.HEAD-0-g1f8fd345
[00:00:00.000254] Creating the usrp device with: mgmt_addr=10.10.3.1,addr=10.40.3.1,master_clock_rate=61.44e6...
[INFO] [MPMD] Initializing 1 device(s) in parallel with args: mgmt_addr=10.10.3.1,type=e3xx,product=e320,serial=3238B97,name=ni-e320-3238B97,fpga=XG,claimed=False,addr=10.40.3.1,master_clock_rate=61.44e6
[WARNING] [MPM.RPCServer] A timeout event occured!
[INFO] [MPM.PeriphManager] init() called with device args `fpga=XG,master_clock_rate=61.44e6,mgmt_addr=10.10.3.1,name=ni-e320-3238B97,product=e320'.
[WARNING] [RFNOC::GRAPH] One or more blocks timed out during flush!
[INFO] [0/Radio#0] Performing CODEC loopback test on channel 0 ...
[INFO] [0/Radio#0] CODEC loopback test passed
[INFO] [0/Radio#0] Performing CODEC loopback test on channel 1 ...
[INFO] [0/Radio#0] CODEC loopback test passed
[INFO] [0/DmaFIFO#0] BIST passed (Estimated Minimum Throughput: 1361 MB/s)
[INFO] [0/DmaFIFO#0] BIST passed (Estimated Minimum Throughput: 1361 MB/s)
Using Device: Single USRP:
Device: E300-Series Device
Mboard 0: e320
RX Channel: 0
RX DSP: 0
RX Dboard: A
RX Subdev: E3xx
TX Channel: 0
TX DSP: 0
TX Dboard: A
TX Subdev: E3xx
[00:00:05.502443989] Setting device timestamp to 0...
Setting RX spp to 1919
Setting TX spp to 1916
[00:00:05.512753951] Testing receive rate 61.440000 Msps on 1 channels
[00:00:05.554696549] Testing transmit rate 61.440000 Msps on 1 channels
[00:00:05.555824707] Detected Rx sequence error.
DU[00:01:05.558922952] Benchmark complete.
Benchmark rate summary:
Num received samples: 3686368018
Num dropped samples: 3196728
Num overruns detected: 0
Num transmitted samples: 3686403160
Num sequence errors (Tx): 0
Num sequence errors (Rx): 1
Num underruns detected: 1
Num late commands: 0
Num timeouts (Tx): 0
Num timeouts (Rx): 0
Done!
Best,
Samie
From: Wade Fife wade.fife@ettus.com
Sent: Friday, December 16, 2022 2:09:21 PM
To: Samie Mostafavi
Cc: usrp-users@lists.ettus.com
Subject: Re: [USRP-users] Re: UHD SPP problem with E320 and XG link
Hi Samie,
In your example, you requested to use "addr=10.40.3.1". To use the 10 GbE SFP port, be sure to specify the address of the XG port, which is typically 192.168.10.2. I don't think jumbo frames are supported over the 1 GbE RJ45 port.
Thanks,
Wade
On Fri, Dec 16, 2022 at 4:47 AM Samie Mostafavi <ssmos@kth.semailto:ssmos@kth.se> wrote:
Hi,
I could narrow down the problem to tx_spp. Tried the following command instead for benchmark and it seems that RX spp could be set to 1919 but TX spp max that is reported by the radio is 492. Does anyone know how I can increase TX spp?
sudo ./benchmark_rate --args "mgmt_addr=10.10.3.1,addr=10.40.3.1,master_clock_rate=61.44e6" --duration 60 --channels "0" --rx_rate 61.44e6 --rx_subdev "A:0" --tx_rate 61.44e6 --tx_subdev "A:0" --rx_spp 1919 --tx_spp 1919 --tx_stream_args "spp=1919" --rx_stream_args "spp=1919"
[INFO] [UHD] linux; GNU C++ version 9.4.0; Boost_107100; UHD_4.3.0.HEAD-0-g1f8fd345
[00:00:00.000069] Creating the usrp device with: mgmt_addr=10.10.3.1,addr=10.40.3.1,master_clock_rate=61.44e6...
[INFO] [MPMD] Initializing 1 device(s) in parallel with args: mgmt_addr=10.10.3.1,type=e3xx,product=e320,serial=3238B97,name=ni-e320-3238B97,fpga=XG,claimed=False,addr=10.40.3.1,master_clock_rate=61.44e6
[INFO] [MPM.PeriphManager] init() called with device args `fpga=XG,master_clock_rate=61.44e6,mgmt_addr=10.10.3.1,name=ni-e320-3238B97,product=e320'.
[INFO] [0/Radio#0] Performing CODEC loopback test on channel 0 ...
[INFO] [0/Radio#0] CODEC loopback test passed
[INFO] [0/Radio#0] Performing CODEC loopback test on channel 1 ...
[INFO] [0/Radio#0] CODEC loopback test passed
[INFO] [0/DmaFIFO#0] BIST passed (Estimated Minimum Throughput: 1361 MB/s)
[INFO] [0/DmaFIFO#0] BIST passed (Estimated Minimum Throughput: 1361 MB/s)
Using Device: Single USRP:
Device: E300-Series Device
Mboard 0: e320
RX Channel: 0
RX DSP: 0
RX Dboard: A
RX Subdev: E3xx
TX Channel: 0
TX DSP: 0
TX Dboard: A
TX Subdev: E3xx
[00:00:04.448876213] Setting device timestamp to 0...
Setting RX spp to 1919
Setting TX spp to 492
[00:00:04.457322205] Testing receive rate 61.440000 Msps on 1 channels
[00:00:04.502995425] Testing transmit rate 61.440000 Msps on 1 channels
Best,
Samie
From: Samie Mostafavi <ssmos@kth.semailto:ssmos@kth.se>
Sent: Friday, December 16, 2022 11:09:56 AM
To: usrp-users@lists.ettus.commailto:usrp-users@lists.ettus.com
Subject: [USRP-users] UHD SPP problem with E320 and XG link
Hi,
I have an E320 device connected to the host with XG fiber connection and XG FPGA loaded on the E320. UHD version is 4.3.0. I have set the mtu to 9000 on both sides. Below you can find the configurations. I am trying to set spp to 1919 but I always get 492. I read in the documents that it depends on the mtu and by default it is set to the maximum value possible based on mtu. Below there is the result of benchmark rate as well.
host:$ sudo ethtool eno12399np0
Settings for eno12399np0:
Supported ports: [ FIBRE ]
Supported link modes: 1000baseT/Full
10000baseT/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 1000baseT/Full
10000baseT/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: Not reported
Link partner advertised pause frame use: Symmetric
Link partner advertised auto-negotiation: No
Link partner advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Port: Direct Attach Copper
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: g
Wake-on: d
Current message level: 0x00000000 (0)
Link detected: yes
host:$ ip -f inet addr show eno12399np0
263: eno12399np0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
inet 10.40.2.1/16<http://10.40.2.1/16> brd 10.40.255.255 scope global eno12399np0
valid_lft forever preferred_lft forever
ni-e320-3238B97:~# ip -f inet addr show sfp0
15: sfp0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc pfifo_fast qlen 1000
inet 10.40.3.1/16<http://10.40.3.1/16> brd 10.40.255.255 scope global sfp0
valid_lft forever preferred_lft forever
host:$ ./benchmark_rate --args "addr=10.40.3.1,master_clock_rate=61.44e6" --duration 60 --channels "0" --rx_rate 61.44e6 --rx_subdev "A:0" --tx_rate 61.44e6 --tx_subdev "A:0"
[INFO] [UHD] linux; GNU C++ version 9.4.0; Boost_107100; UHD_4.3.0.HEAD-0-g1f8fd345
[00:00:00.000070] Creating the usrp device with: addr=10.40.3.1,master_clock_rate=61.44e6...
[INFO] [MPMD] Initializing 1 device(s) in parallel with args: mgmt_addr=10.40.3.1,type=e3xx,product=e320,serial=3238B97,name=ni-e320-3238B97,fpga=XG,claimed=False,addr=10.40.3.1,master_clock_rate=61.44e6
[INFO] [MPM.PeriphManager] init() called with device args `fpga=XG,master_clock_rate=61.44e6,mgmt_addr=10.40.3.1,name=ni-e320-3238B97,product=e320'.
[INFO] [0/Radio#0] Performing CODEC loopback test on channel 0 ...
[INFO] [0/Radio#0] CODEC loopback test passed
[INFO] [0/Radio#0] Performing CODEC loopback test on channel 1 ...
[INFO] [0/Radio#0] CODEC loopback test passed
[INFO] [0/DmaFIFO#0] BIST passed (Estimated Minimum Throughput: 1361 MB/s)
[INFO] [0/DmaFIFO#0] BIST passed (Estimated Minimum Throughput: 1361 MB/s)
Using Device: Single USRP:
Device: E300-Series Device
Mboard 0: e320
RX Channel: 0
RX DSP: 0
RX Dboard: A
RX Subdev: E3xx
TX Channel: 0
TX DSP: 0
TX Dboard: A
TX Subdev: E3xx
[00:00:04.923395067] Setting device timestamp to 0...
Setting TX spp to 492
[00:00:04.935123785] Testing receive rate 61.440000 Msps on 1 channels
[00:00:04.977439269] Testing transmit rate 61.440000 Msps on 1 channels
UUUU[00:01:04.979477903] Benchmark complete.
Benchmark rate summary:
Num received samples: 3686373960
Num dropped samples: 0
Num overruns detected: 0
Num transmitted samples: 3686410860
Num sequence errors (Tx): 0
Num sequence errors (Rx): 0
Num underruns detected: 4
Num late commands: 0
Num timeouts (Tx): 0
Num timeouts (Rx): 0
Done
As you can see TX spp is set to 492. But I want 1919 with 9000 mtu. Is there anything I am missing here?
Best,
Samie
USRP-users mailing list -- usrp-users@lists.ettus.commailto:usrp-users@lists.ettus.com
To unsubscribe send an email to usrp-users-leave@lists.ettus.commailto:usrp-users-leave@lists.ettus.com
On 16/12/2022 08:25, Samie Mostafavi wrote:
Hi Wade,
Thanks for your answer. I figured out the problem. It was the switch
in between that I had to reboot. I had jumbo frame option enabled on
the switch but a reboot was needed. Now that I try it works:
host$ ping -M do -s 5850 10.40.3.1
PING 10.40.3.1 (10.40.3.1) 5850(5878) bytes of data.
5858 bytes from 10.40.3.1: icmp_seq=1 ttl=64 time=2.24 ms
5858 bytes from 10.40.3.1: icmp_seq=2 ttl=64 time=1.25 ms
5858 bytes from 10.40.3.1: icmp_seq=3 ttl=64 time=1.19 ms
5858 bytes from 10.40.3.1: icmp_seq=4 ttl=64 time=1.19 ms
host$ ./benchmark_rate --args
"mgmt_addr=10.10.3.1,addr=10.40.3.1,master_clock_rate=61.44e6"
--duration 60 --channels "0" --rx_rate 61.44e6 --rx_subdev "A:0"
--tx_rate 61.44e6 --tx_subdev "A:0" --rx_spp 1919 --tx_spp
1919--tx_stream_args "spp=1919" --rx_stream_args "spp=1919"
[INFO] [UHD] linux; GNU C++ version 9.4.0; Boost_107100;
UHD_4.3.0.HEAD-0-g1f8fd345
[00:00:00.000254] Creating the usrp device with:
mgmt_addr=10.10.3.1,addr=10.40.3.1,master_clock_rate=61.44e6...
[INFO] [MPMD] Initializing 1 device(s) in parallel with args:
mgmt_addr=10.10.3.1,type=e3xx,product=e320,serial=3238B97,name=ni-e320-3238B97,fpga=XG,claimed=False,addr=10.40.3.1,master_clock_rate=61.44e6
*[WARNING] [MPM.RPCServer] *A timeout event occured!
[INFO] [MPM.PeriphManager] init() called with device args
`fpga=XG,master_clock_rate=61.44e6,mgmt_addr=10.10.3.1,name=ni-e320-3238B97,product=e320'.
*[WARNING] [RFNOC::GRAPH] *One or more blocks timed out during flush!
[INFO] [0/Radio#0] Performing CODEC loopback test on channel 0 ...
[INFO] [0/Radio#0] CODEC loopback test passed
[INFO] [0/Radio#0] Performing CODEC loopback test on channel 1 ...
[INFO] [0/Radio#0] CODEC loopback test passed
[INFO] [0/DmaFIFO#0] BIST passed (Estimated Minimum Throughput: 1361 MB/s)
[INFO] [0/DmaFIFO#0] BIST passed (Estimated Minimum Throughput: 1361 MB/s)
Using Device: Single USRP:
Device: E300-Series Device
Mboard 0: e320
RX Channel: 0
RX DSP: 0
RX Dboard: A
RX Subdev: E3xx
TX Channel: 0
TX DSP: 0
TX Dboard: A
TX Subdev: E3xx
[00:00:05.502443989] Setting device timestamp to 0...
Setting RX spp to 1919
Setting TX spp to 1916
[00:00:05.512753951] Testing receive rate 61.440000 Msps on 1 channels
[00:00:05.554696549] Testing transmit rate 61.440000 Msps on 1 channels
[00:00:05.555824707] Detected Rx sequence error.
DU[00:01:05.558922952] Benchmark complete.
Benchmark rate summary:
Num received samples: 3686368018
Num dropped samples:3196728
Num overruns detected:0
Num transmitted samples:3686403160
Num sequence errors (Tx): 0
Num sequence errors (Rx): 1
Num underruns detected: 1
Num late commands:0
Num timeouts (Tx):0
Num timeouts (Rx):0
Done!
Best,
Samie
You might also try some of the things noted here:
https://kb.ettus.com/USRP_Host_Performance_Tuning_Tips_and_Tricks#Adjust_Network_Buffers