usrp-users@lists.ettus.com
Discussion and technical support related to USRP, UHD, RFNoC
View all threadsRe: NIC suggestion
On Tue, Apr 29, 2025 at 1:31 AM Marcus D. Leech patchvonbraun@gmail.com
wrote:
On 28/04/2025 18:04, Nikos Balkanas wrote:
Hi Marcus,
spb(=samples/buffer) is not what I hoped for. It is just another way to
set the
stream_cmd.num_samples in the streamer. I already use there MAXSMPS.
Besides benchmark_rate reports maxsmps 1996 like me:(
That part is only controlled by the MTU. Seems it is the same
for everyone, and that we are using the generic chdr_sc16_to_xx
instead the guts (sse2) conversion:( Unless they use the multi_usrp
class and set spp on their own.
I think that MAXSMPS used to be 1996, before someone changed it to
19960.
BR
Nikos
BR
Nikos
Well, regardless of all of this, there's no way to pack more than a
certain number of samples into an MTU frame.
If your high-level needs are to process "things" in larger chunks, you
need to layer something on top of what UHD
does, since it is, at the end of the day, a hardware driver library.
It's not an application layer programming framework.
Not really. I just need for FFT 1024 samples at the moment. My concern is
more about the sse2 code not used:(
What about packet fragmentation? NICs can handle those...
On Mon, Apr 28, 2025 at 5:22 PM Nikos Balkanas nbalkanas@gmail.com
wrote:
Thx Marcus for the clarifications,
On Mon, Apr 28, 2025 at 4:37 PM Marcus D. Leech patchvonbraun@gmail.com
wrote:
On 28/04/2025 05:33, Nikos Balkanas wrote:
Compiled uhd 4.6.0 in debug mode.
From the output I get:
[DEBUG] [0/Radio#0] spp(= samples per package) value 2032 exceeds MTU of
8000! Coercing to 1996
Mon Apr 28 09:57:02 2025 [00] [*] scanner.l:1443:main Incorrect
maxsamples (1996). Expected 19960.
Mon Apr 28 09:57:02 2025 [00] [+] Max samples/buffer[0]: 1996
- Line mtu is 9000 not 8000
- 2032 is not larger than 8000 <= Bug?
- seems that spp is setting my maxsmps
That's number of SAMPLES. Samples are 4 bytes total.
Aaaah. I'll have to check the SPB option. Otherwise an 80K MTU is
unreasonable:)
This also means that 1 sample = 1 real + 1 imag = 32 bits with sc16
encoding
.
A lot of network hardware, particularly 1Gbit hardware doesn't ACTUALLY
support an MTU of more than 8000, and I think
UHD uses PMTU discovery. I found that with RealTek NICs, even when
you set the MTU to 9000, it actually only supports
8000.
Same case with my Mellanox NIC. But 8000 is close enough:)
Of the examples I tried the rx_samples_c. It is the same case like mine:
single usrp. We use the same commands
and we are getting the same output:( 1996 maxsmpls.
The error text and code are from host/lib/rfnoc/radio_control_impl.cpp:
199
I would rather not touch it. I don't know the uhd architecture and
especially the rfnoc/uhd interface.
Besides I am a c programmer, not c++:(
multi_usrp class has a set_rx_spp function, but it is not for me:(
You can look at the benchmark_rate example to see how to set a
samples-per-buffer other than the default, which is
based on the MTU. It uses an "SPB" command-line parameter.
Thx, I will check it out, when I get back to ubuntu.. Now I am in
windows:(
benchmark_rate uses the multi-usrp class.
HTH
Nikos
On Mon, Apr 28, 2025 at 6:02 AM Nikos Balkanas nbalkanas@gmail.com
wrote:
Point taken:) I'm proposing smt different:
pchar +ICMP are just to test line and connectors. First step. Not to
bench USRP.
benchmark_rate is to bench/stress usrp.
These 2 are independent, and complementary.
Pchar is telling me nothing more than my fiber cable and connectors are
good.
It saved me a trip tomorrow to my local PC store, where fiber cable and
connectors are ~7 E each.
benchmark_rate on the other hand is quite interesting.
It points to software,and particularly my uhd_init()
Just found and downloaded the sources to uhd 4.6.0 from Ubuntu
Launchpad.
Now I can go through the source of the example you told me:)
Ettus used to keep an archive of old uhd sources around. Seems it's
gone:(
Open source means, among others, free to choose the source version that
you need...
Having the latest source in Github is only partly open source.
During development we need to freeze updates. When in 5 years we go
into production we can't find the old sources anymore:(
If a customer discovers a bug, not in our code, but in one of the
libraries that
we use, what are we gonna do?
BR
Nikos
On Mon, Apr 28, 2025 at 5:01 AM Marcus D. Leech <
patchvonbraun@gmail.com> wrote:
On 27/04/2025 21:58, Nikos Balkanas wrote:
My bad:
throughput of 5.619 Kb/s requesting ICMP replies, +> throughput of
5,619 Kb/s requesting ICMP replies
Local thousand separator is ".", whereas in the US is ",":(
It is STILL the case that the ICMP machinery in these radios is
ABSOLUTELY NOT on the fast-path inside
the hardware. The only way to get a good feel for how much sample
bandwidth they can process is
with examples like "benchmark_rate".
On Mon, Apr 28, 2025 at 12:37 AM Nikos Balkanas nbalkanas@gmail.com
wrote:
Hi Marcus,
You were right. No need to change NIC:)
This is not a software issue. uhd_rx_streamer_max_num_samps runs
right after uhd initialization before
any other code had the chance to run.
Link capacity doesn't seem to be the issue either...
Running pchar on the link, descendant of pathchar, reports a
throughput of 5.619 Kb/s requesting ICMP replies,
to varying packet sizes (32->9000 (MTU), incr by 32).
sudo pchar -m 9000 -p ipv4icmp usrp
https://www.kitchenlab.org/www/bmah/Software/pchar/
It corresponds to 351.218.019 16-bit samples or 175,609.044 32-bit
samples, if each sample is 32-bit(real + imag)
Seems that uhd is not running at link capacity but is doing smt else.
I will have to download and check with the sources...
The package version for Ubuntu 24.04 is uhd 4.6.0.
Where can I download the sources for uhd 4.6.0?
BR
Nikos
On Sat, Apr 26, 2025 at 7:02 AM Nikos Balkanas nbalkanas@gmail.com
wrote:
Thanks for your time.
I will check out the example.
This is not a buffer problem. I just need 1024 Samples
(real+imaginary) for FFT...
I should be able to get them in a single pass.
You saw my code, not a smoking gun there.
This is probably is a physical problem.
Cable is an SFP fiber dedicated line. Cannot go bad.
Maybe the connections are not sitting right :(...
BR
Nikos
On Sat, Apr 26, 2025 at 6:45 AM Marcus D. Leech <
patchvonbraun@gmail.com> wrote:
On 25/04/2025 23:33, Nikos Balkanas wrote:
Actually MTU is 9000. This is one of the recommendations...
I tried it with MTU 1500. It was worse:(
maxsamples dropped to 364...
Right, 9000, rather than 8000.
Upgrading to 10Gbit wont' give you larger MTU.
What you're trying to do, I think, is to solve a buffer-management
problem in your application at entirely the wrong
level in the stack.
It is EXCEEDINGLY COMMON for hardware drivers to only be able to
deliver in chunks that may not be perfectly adapted to
the requirements of your application. So, a common programming
pattern is to deal with this in your application.
You should probably look at example code like rx_samples_to_file
[INFO] [UHD] linux; GNU C++ version 13.2.0; Boost_108300;
UHD_4.6.0.0+ds1-5.1ubuntu0.24.04.1
[INFO] [X300] X300 initialization sequence...
[INFO] [X300] Maximum frame size: 1472 bytes.
[WARNING] [X300] For the 192.168.40.2 connection, UHD recommends a
send frame size of at least 8000 for best
performance, but your configuration will only allow 1472.This may
negatively impact your maximum achievable sample rate.
Check the MTU on the interface and/or the send_frame_size argument.
[WARNING] [X300] For the 192.168.40.2 connection, UHD recommends a
receive frame size of at least 8000 for best
performance, but your configuration will only allow 1472.This may
negatively impact your maximum achievable sample rate.
Check the MTU on the interface and/or the recv_frame_size argument.
[INFO] [GPS] No GPSDO found
[INFO] [X300] Radio 1x clock: 200 MHz
[WARNING] [UDP] The send buffer could not be resized sufficiently.
Target sock buff size: 24912805 bytes.
Actual sock buff size: 1048576 bytes.
See the transport application notes on buffer resizing.
Please run: sudo sysctl -w net.core.wmem_max=24912805
Sat Apr 26 06:30:34 2025 [00] [+] Created USRP with args
Sat Apr 26 06:30:34 2025 [00] [+] Master clock is at 200 Mhz
Sat Apr 26 06:30:34 2025 [00] [+] Tuner[0] gain set to 30 (30) dB
[WARNING] [UDP] The send buffer could not be resized sufficiently.
Target sock buff size: 24912805 bytes.
Actual sock buff size: 1048576 bytes.
See the transport application notes on buffer resizing.
Please run: sudo sysctl -w net.core.wmem_max=24912805
Sat Apr 26 06:30:34 2025 [00] [*] scanner.l:1446:main Incorrect
maxsamples (364). Expected 19960.
Sat Apr 26 06:30:34 2025 [00] [+] Max samples/buffer[0]: 364
[WARNING] [0/Radio#0] Ignoring stream command for finite
acquisition of zero sam
Nikos
On Sat, Apr 26, 2025 at 5:46 AM Marcus D. Leech <
patchvonbraun@gmail.com> wrote:
On 25/04/2025 22:26, Nikos Balkanas wrote:
Thanks Marcus,
for your fast reply.
On Sat, Apr 26, 2025 at 4:08 AM Marcus D. Leech <
patchvonbraun@gmail.com> wrote:
On 25/04/2025 20:50, Nikos Balkanas wrote:
Hello,
I need to buy a new NIC. What would you suggest?
The one I use is an old Mellanox 10 Gbs, before the Connect-4
series.
It can only do 1996 S/s, need 19960 (10x more) to work with
latest uhd.
Using Ubuntu 24.04 and uhd 4.6.0
So, 1.996ksps vs 19.960ksps? You hardly need a 10Gbit link to
support that. So, perhaps something
is being lost here in your requirements?
True. Can't explain it in terms of bandwidth. 16 * 1996 = 31.936
Kbps, 16 * 19960 = 319.360 Kbps well short of a 10 Gbps line:(
Does a complex pair count as 1 sample, or 2?
I have followed all the instructions in
https://kb.ettus.com/USRP_Host_Performance_Tuning_Tips_and_Tricks,
Even installed the DPDK drivers. My Mellanox is too old to use
their OFED drivers:(
On a related question. it seems that the streamer doesn't crash
anymore
when receiving less than MAXSPS samples, instead it reads 0:(
This was due to the sse2 code not aligned in convert.
I change my stream args to cpu_format=sc16, otw=sc16, so no
conversion required.
Streamer still doesn't read anything. Is there a reason for it?
You'd need to share more of your code. This should just work as
far as I can tell.
Thanks. these are just the usrp code:
int main()
{
uhd_stream_args_t stream_args =
{
.cpu_format = "sc16",
.otw_format = "sc16",
.args = "",
.n_channels = 1,
.channel_list = &channel
};
..uhd_stream_cmd_t stream_cmd =
{
.stream_mode = UHD_STREAM_MODE_NUM_SAMPS_AND_DONE,
.stream_now = true
};
if (uhd_init(0, 0, gain)) do_exit(20);
if ((err = uhd_usrp_get_rx_stream(dev[0], &stream_args,
rx_streamer[0])))
{
uhd_get_last_error(errmsg, 127);
error(log, "Failed to get streamer[0] (%d). %s.\n", 0,
FL, LN, FN, err, errmsg);
uhd_rx_streamer_free(&rx_streamer[0]);
rx_streamer[0] = NULL;
uhd_rx_metadata_free(&md[0]);
md[0] = NULL;
do_exit(30);
}
if ((err = uhd_rx_streamer_max_num_samps(rx_streamer[0],
&maxsamps)))
{
uhd_get_last_error(errmsg, 127);
error(log, "Failed to get max samples/buffer[0] (%d).
%s.\n", 0, FL, LN, FN, err,
..errmsg);
do_exit(35);
}
if (maxsamps != MAXSMPS)
warn(log, "Incorrect maxsamples (%ld). Expected %d.\n",
0, FL, LN, FN, maxsamps,
MAXSMPS);
info(log, "Max samples/buffer[0]: %ld\n", 0, maxsamps);
if ((err = uhd_rx_streamer_issue_stream_cmd(rx_streamer[0],
&stream_cmd)))
{
uhd_get_last_error(errmsg, 127);
error(log, "Failed to start streamer[0] (%d). %s.\n", 0,
FL, LN, FN, err, errmsg);
do_exit(40);
}
[...]
do_exit(0)
}
bool uhd_init(size_t channel, double srate, double gain)
{
double tmp;
uhd_rx_metadata_error_code_t err;
if ((err =
uhd_set_thread_priority(uhd_default_thread_priority, true)))
warn(log, "Unable to set main thread priority (%d).
%s.\n", 0, FL, LN, FN,
err, uhdError(err));
/* Create USRP */
f ((err = uhd_usrp_make(&dev[channel], "type=x300")))
{
error(log, "Failed to create USRP (%d). %s.\n", 0, FL,
LN, FN, err,
uhdError(err));
dev[channel] = NULL;
return(FAIL);
}
info(stderr, "Created USRP with args\n", 0);
/* Create RX streamer */
if ((err = uhd_rx_streamer_make(&rx_streamer[channel])))
{
error(log, "Failed to create rx_streamer[%d] (%d).
%s.\n", 0, FL, LN, FN,
channel, err, uhdError(err));
return(FAIL);
}
/* Create RX metadata */
if ((err = uhd_rx_metadata_make(&md[channel])))
{
error(log, "Failed to create md[%d] (%d). %s.\n", 0, FL,
LN, FN, channel,
err, uhdError(err));
return(FAIL);
}
/* Get master clock rate */
if ((err = uhd_usrp_get_master_clock_rate(dev[channel], 0,
&tmp)))
{
error(log, "Failed to set master clock to %.0lf Mhz
(%d). %s.\n", 0, FL,
LN, FN, tmp/1000000, err, uhdError(err));
return(FAIL);
}
info(stderr, "Master clock is at %.0lf Mhz\n", 0,
tmp/1000000);
/* Set the sample rate /
if (srate && !uhd_set_rx_rate_check(channel, srate))
return(FAIL);
/ Set the tuner gain SBX-120 is 0-31.5 in .5 db steps */
if ((err = uhd_usrp_set_rx_gain(dev[channel], gain,
channel, "")))
{
error(log, "Failed to set tuner[%d] gain to %.0lf db
(%d). %s.\n", 0, FL,
LN, FN, channel, gain, err, uhdError(err));
return(FAIL);
}
if (!(err = uhd_usrp_get_rx_gain(dev[channel], channel,
"", &tmp)))
info(log, "Tuner[%d] gain set to %.0lf (%.0lf)
dB\n", 0, channel, tmp, gain);
./* Set channel bw to conserve tuner resources. Not
needed, set by srate /
uhd_usrp_set_rx_bandwidth(dev[channel], srate,
channel);
./ Disable subtracting constant averaged background.
Signal looks cleaner */
if ((err =
uhd_usrp_set_rx_dc_offset_enabled(dev[channel], false, channel)))
{
warn(log, "Failed to disable FPGA DC offset on
channel %d(%d). %s.\n", 0,
FL, LN, FN, channel, err, uhdError(err));
}
info(stderr, "Disabled FPGA DC offset on channel
%d\n", 0, channel);
return(SUCCESS);
}
This is the generated output:
[INFO] [UHD] linux; GNU C++ version 13.2.0; Boost_108300;
UHD_4.6.0.0+ds1-5.1ubuntu0.24.04.1
[INFO] [X300] X300 initialization sequence...
[INFO] [X300] Maximum frame size: 8000 bytes.
[INFO] [X300] Radio 1x clock: 200 MHz
Sat Apr 26 03:33:48 2025 [00] [+] Created USRP with args
Sat Apr 26 03:33:48 2025 [00] [+] Master clock is at 200 Mhz
Sat Apr 26 03:33:48 2025 [00] [+] Tuner[0] gain set to 30 (30) dB
Sat Apr 26 03:33:48 2025 [00] [*] scanner.l:1446:main Incorrect
maxsamples (1996). Expected 19960.
Sat Apr 26 03:33:48 2025 [00] [+] Max samples/buffer[0]: 1996
[WARNING] [0/Radio#0] Ignoring stream command for finite
acquisition of zero samples
I hope this reads OK. Maybe next time I should attach the code:)
TIA
Nikos
USRP-users mailing list -- usrp-users@lists.ettus.com
To unsubscribe send an email to usrp-users-leave@lists.ettus.com
I believe that max number of samples-per-buffer is limited by MTU
size. Which is typically around 8000 or so for "jumbo frames".
On 28/04/2025 19:08, Nikos Balkanas wrote:
On Tue, Apr 29, 2025 at 1:31 AM Marcus D. Leech
patchvonbraun@gmail.com wrote:
On 28/04/2025 18:04, Nikos Balkanas wrote:
Hi Marcus,
spb(=samples/buffer) is not what I hoped for. It is just another
way to set the
stream_cmd.num_samples in the streamer. I already use there
MAXSMPS.
Besides benchmark_rate reports maxsmps 1996 like me:(
That part is only controlled by the MTU. Seems it is the same
for everyone, and that we are using the generic chdr_sc16_to_xx
instead the guts (sse2) conversion:( Unless they use the multi_usrp
class and set spp on their own.
I think that MAXSMPS used to be 1996, before someone changed it to
19960.
BR
Nikos
BR
Nikos
Well, regardless of all of this, there's no way to pack more than
a certain number of samples into an MTU frame.
If your high-level needs are to process "things" in larger chunks,
you need to layer something on top of what UHD
does, since it is, at the end of the day, a hardware driver
library. It's not an application layer programming framework.
Not really. I just need for FFT 1024 samples at the moment. My concern
is more about the sse2 code not used:(
What about packet fragmentation? NICs can handle those...
I think that by default UDP on most systems these days sets the "DF" bit
-- so that higher-layer protocols like UDP
restrict themselves to non-fragmented packets at that level. That's
because in the larger internet, you can't rely
on in-order fragment handling, and re-ordering is a HUGE performance
loss.
Anyway, for a 1024-sample FFT, you have what you need.
On Mon, Apr 28, 2025 at 5:22 PM Nikos Balkanas
<nbalkanas@gmail.com> wrote:
Thx Marcus for the clarifications,
On Mon, Apr 28, 2025 at 4:37 PM Marcus D. Leech
<patchvonbraun@gmail.com> wrote:
On 28/04/2025 05:33, Nikos Balkanas wrote:
Compiled uhd 4.6.0 in debug mode.
From the output I get:
[DEBUG] [0/Radio#0] spp(= samples per package) value
2032 exceeds MTU of 8000! Coercing to 1996
Mon Apr 28 09:57:02 2025 [00] [*] scanner.l:1443:main
Incorrect maxsamples (1996). Expected 19960.
Mon Apr 28 09:57:02 2025 [00] [+] Max samples/buffer[0]:
1996
1) Line mtu is 9000 not 8000
2) 2032 is not larger than 8000 <= Bug?
3) seems that spp is setting my maxsmps
That's number of *SAMPLES*. Samples are 4 bytes total.
Aaaah. I'll have to check the SPB option. Otherwise an 80K
MTU is unreasonable:)
This also means that 1 sample = 1 real + 1 imag = 32 bits
with sc16 encoding
.
A lot of network hardware, particularly 1Gbit hardware
doesn't *ACTUALLY* support an MTU of more than 8000, and
I think
UHD uses PMTU discovery. I found that with RealTek
NICs, even when you set the MTU to 9000, it actually only
supports
8000.
Same case with my Mellanox NIC. But 8000 is close enough:)
Of the examples I tried the rx_samples_c. It is the same
case like mine: single usrp. We use the same commands
and we are getting the same output:( 1996 maxsmpls.
The error text and code are
from host/lib/rfnoc/radio_control_impl.cpp: 199
I would rather not touch it. I don't know the uhd
architecture and especially the rfnoc/uhd interface.
Besides I am a c programmer, not c++:(
multi_usrp class has a set_rx_spp function, but it is
not for me:(
You can look at the benchmark_rate example to see how to
set a samples-per-buffer other than the default, which is
based on the MTU. It uses an "SPB" command-line
parameter.
Thx, I will check it out, when I get back to ubuntu.. Now I
am in windows:(
benchmark_rate uses the multi-usrp class.
HTH
Nikos
On Mon, Apr 28, 2025 at 6:02 AM Nikos Balkanas
<nbalkanas@gmail.com> wrote:
Point taken:) I'm proposing smt different:
pchar +ICMP are just to test line and connectors.
First step. Not to bench USRP.
benchmark_rate is to bench/stress usrp.
These 2 are independent, and complementary.
Pchar is telling me nothing more than my fiber cable
and connectors are good.
It saved me a trip tomorrow to my local PC store,
where fiber cable and connectors are ~7 E each.
benchmark_rate on the other hand is quite interesting.
It points to software,and particularly my uhd_init()
Just found and downloaded the sources to uhd 4.6.0
from Ubuntu Launchpad.
Now I can go through the source of the example you
told me:)
Ettus used to keep an archive of old uhd sources
around. Seems it's gone:(
Open source means, among others, free to choose the
source version that you need...
Having the latest source in Github is only partly
open source.
During development we need to freeze updates. When
in 5 years we go
into production we can't find the old sources anymore:(
If a customer discovers a bug, not in our code, but
in one of the libraries that
we use, what are we gonna do?
BR
Nikos
On Mon, Apr 28, 2025 at 5:01 AM Marcus D. Leech
<patchvonbraun@gmail.com> wrote:
On 27/04/2025 21:58, Nikos Balkanas wrote:
My bad:
throughput of 5.619 Kb/s requesting ICMP
replies, +> throughput of 5,619 Kb/s requesting
ICMP replies
Local thousand separator is ".", whereas in the
US is ",":(
It is STILL the case that the ICMP machinery in
these radios is ABSOLUTELY NOT on the fast-path
inside
the hardware. The only way to get a good feel
for how much sample bandwidth they can process is
with examples like "benchmark_rate".
On Mon, Apr 28, 2025 at 12:37 AM Nikos Balkanas
<nbalkanas@gmail.com> wrote:
Hi Marcus,
You were right. No need to change NIC:)
This is not a software
issue. uhd_rx_streamer_max_num_samps runs
right after uhd initialization before
any other code had the chance to run.
Link capacity doesn't seem to be the issue
either...
Running pchar on the link, descendant of
pathchar, reports a throughput of 5.619
Kb/s requesting ICMP replies,
to varying packet sizes (32->9000 (MTU),
incr by 32).
sudo pchar -m 9000 -p ipv4icmp usrp
https://www.kitchenlab.org/www/bmah/Software/pchar/
It corresponds to 351.218.019 16-bit
samples or 175,609.044 32-bit samples, if
each sample is 32-bit(real + imag)
Seems that uhd is not running at link
capacity but is doing smt else.
I will have to download and check with the
sources...
The package version for Ubuntu 24.04 is uhd
4.6.0.
Where can I download the sources for uhd 4.6.0?
BR
Nikos
On Sat, Apr 26, 2025 at 7:02 AM Nikos
Balkanas <nbalkanas@gmail.com> wrote:
Thanks for your time.
I will check out the example.
This is not a buffer problem. I just
need 1024 Samples (real+imaginary) for
FFT...
I should be able to get them in a
single pass.
You saw my code, not a smoking gun there.
This is probably is a physical problem.
Cable is an SFP fiber dedicated line.
Cannot go bad.
Maybe the connections are not sitting
right :(...
BR
Nikos
On Sat, Apr 26, 2025 at 6:45 AM Marcus
D. Leech <patchvonbraun@gmail.com> wrote:
On 25/04/2025 23:33, Nikos Balkanas
wrote:
Actually MTU is 9000. This is one
of the recommendations...
I tried it with MTU 1500. It was
worse:(
maxsamples dropped to 364...
Right, 9000, rather than 8000.
Upgrading to 10Gbit wont' give you
larger MTU.
What you're trying to do, I think,
is to solve a buffer-management
problem in your *application* at
entirely the wrong
level in the stack.
It is EXCEEDINGLY COMMON for
hardware drivers to only be able to
deliver in chunks that may not be
perfectly adapted to
the requirements of your
application. So, a common
programming pattern is to deal with
this in your application.
You should probably look at example
code like rx_samples_to_file
[INFO] [UHD] linux; GNU C++
version 13.2.0; Boost_108300;
UHD_4.6.0.0+ds1-5.1ubuntu0.24.04.1
[INFO] [X300] X300 initialization
sequence...
[INFO] [X300] Maximum frame size:
1472 bytes.
[WARNING] [X300] For the
192.168.40.2 connection, UHD
recommends a send frame size of at
least 8000 for best
performance, but your
configuration will only allow
1472.This may negatively impact
your maximum achievable sample rate.
Check the MTU on the interface
and/or the send_frame_size argument.
[WARNING] [X300] For the
192.168.40.2 connection, UHD
recommends a receive frame size of
at least 8000 for best
performance, but your
configuration will only allow
1472.This may negatively impact
your maximum achievable sample rate.
Check the MTU on the interface
and/or the recv_frame_size argument.
[INFO] [GPS] No GPSDO found
[INFO] [X300] Radio 1x clock: 200 MHz
[WARNING] [UDP] The send buffer
could not be resized sufficiently.
Target sock buff size: 24912805 bytes.
Actual sock buff size: 1048576 bytes.
See the transport application
notes on buffer resizing.
Please run: sudo sysctl -w
net.core.wmem_max=24912805
Sat Apr 26 06:30:34 2025 [00] [+]
Created USRP with args
Sat Apr 26 06:30:34 2025 [00] [+]
Master clock is at 200 Mhz
Sat Apr 26 06:30:34 2025 [00] [+]
Tuner[0] gain set to 30 (30) dB
[WARNING] [UDP] The send buffer
could not be resized sufficiently.
Target sock buff size: 24912805 bytes.
Actual sock buff size: 1048576 bytes.
See the transport application
notes on buffer resizing.
Please run: sudo sysctl -w
net.core.wmem_max=24912805
Sat Apr 26 06:30:34 2025 [00] [*]
scanner.l:1446:main Incorrect
maxsamples (364). Expected 19960.
Sat Apr 26 06:30:34 2025 [00] [+]
Max samples/buffer[0]: 364
[WARNING] [0/Radio#0] Ignoring
stream command for finite
acquisition of zero sam
Nikos
On Sat, Apr 26, 2025 at 5:46 AM
Marcus D. Leech
<patchvonbraun@gmail.com> wrote:
On 25/04/2025 22:26, Nikos
Balkanas wrote:
Thanks Marcus,
for your fast reply.
On Sat, Apr 26, 2025 at
4:08 AM Marcus D. Leech
<patchvonbraun@gmail.com> wrote:
On 25/04/2025 20:50,
Nikos Balkanas wrote:
Hello,
I need to buy a new NIC.
What would you suggest?
The one I use is an old
Mellanox 10 Gbs, before
the Connect-4 series.
It can only do 1996 S/s,
need 19960 (10x more) to
work with latest uhd.
Using Ubuntu 24.04 and
uhd 4.6.0
So, 1.996ksps vs
19.960ksps? You hardly
need a 10Gbit link to
support that. So, perhaps
something
is being lost here in
your requirements?
True. Can't explain it in
terms of bandwidth. 16 * 1996
= 31.936 Kbps, 16 * 19960 =
319.360 Kbps well short of a
10 Gbps line:(
Does a complex pair count as
1 sample, or 2?
I have followed all the
instructions in
https://kb.ettus.com/USRP_Host_Performance_Tuning_Tips_and_Tricks,
Even installed the DPDK
drivers. My Mellanox is too
old to use their OFED drivers:(
On a related question.
it seems that the
streamer doesn't crash
anymore
when receiving less than
MAXSPS samples, instead
it reads 0:(
This was due to the sse2
code not aligned in convert.
I change my stream args
to cpu_format=sc16,
otw=sc16, so no
conversion required.
Streamer still doesn't
read anything. Is there
a reason for it?
You'd need to share more
of your code. This should
just work as far as I can
tell.
Thanks. these are just the
usrp code:
int main()
{
uhd_stream_args_t
stream_args =
{
.cpu_format = "sc16",
.otw_format = "sc16",
.args = "",
.n_channels = 1,
.channel_list = &channel
};
..uhd_stream_cmd_t stream_cmd =
{
.stream_mode =
UHD_STREAM_MODE_NUM_SAMPS_AND_DONE,
.stream_now = true
};
if (uhd_init(0, 0, gain))
do_exit(20);
if ((err =
uhd_usrp_get_rx_stream(dev[0],
&stream_args,
rx_streamer[0])))
{
uhd_get_last_error(errmsg,
127);
error(log, "Failed to get
streamer[0] (%d). %s.\n",
0, FL, LN, FN, err, errmsg);
uhd_rx_streamer_free(&rx_streamer[0]);
rx_streamer[0] = NULL;
uhd_rx_metadata_free(&md[0]);
md[0] = NULL;
do_exit(30);
}
if ((err =
uhd_rx_streamer_max_num_samps(rx_streamer[0],
&maxsamps)))
{
uhd_get_last_error(errmsg,
127);
error(log, "Failed to get
max samples/buffer[0]
(%d). %s.\n", 0, FL, LN,
FN, err,
..errmsg);
do_exit(35);
}
if (maxsamps != MAXSMPS)
warn(log, "Incorrect
maxsamples (%ld).
Expected %d.\n", 0, FL,
LN, FN, maxsamps,
MAXSMPS);
info(log, "Max
samples/buffer[0]:
%ld\n", 0, maxsamps);
if ((err =
uhd_rx_streamer_issue_stream_cmd(rx_streamer[0],
&stream_cmd)))
{
uhd_get_last_error(errmsg,
127);
error(log, "Failed to
start streamer[0] (%d).
%s.\n", 0, FL, LN, FN,
err, errmsg);
do_exit(40);
}
[...]
do_exit(0)
}
bool uhd_init(size_t
channel, double srate,
double gain)
{
double tmp;
uhd_rx_metadata_error_code_t
err;
if ((err =
uhd_set_thread_priority(uhd_default_thread_priority,
true)))
warn(log, "Unable to set
main thread priority
(%d). %s.\n", 0, FL, LN, FN,
err, uhdError(err));
/* Create USRP */
f ((err =
uhd_usrp_make(&dev[channel],
"type=x300")))
{
error(log, "Failed to
create USRP (%d). %s.\n",
0, FL, LN, FN, err,
uhdError(err));
dev[channel] = NULL;
return(FAIL);
}
info(stderr, "Created
USRP with args\n", 0);
/* Create RX streamer */
if ((err =
uhd_rx_streamer_make(&rx_streamer[channel])))
{
error(log, "Failed to
create rx_streamer[%d]
(%d). %s.\n", 0, FL, LN, FN,
channel, err, uhdError(err));
return(FAIL);
}
/* Create RX metadata */
if ((err =
uhd_rx_metadata_make(&md[channel])))
{
error(log, "Failed to
create md[%d] (%d).
%s.\n", 0, FL, LN, FN,
channel,
err, uhdError(err));
return(FAIL);
}
/* Get master clock rate */
if ((err =
uhd_usrp_get_master_clock_rate(dev[channel],
0, &tmp)))
{
error(log, "Failed to set
master clock to %.0lf Mhz
(%d). %s.\n", 0, FL,
LN, FN, tmp/1000000, err,
uhdError(err));
return(FAIL);
}
info(stderr, "Master
clock is at %.0lf Mhz\n",
0, tmp/1000000);
/* Set the sample rate */
if (srate &&
!uhd_set_rx_rate_check(channel,
srate)) return(FAIL);
/* Set the tuner gain
SBX-120 is 0-31.5 in .5
db steps */
if ((err =
uhd_usrp_set_rx_gain(dev[channel],
gain, channel, "")))
{
error(log, "Failed to set
tuner[%d] gain to %.0lf
db (%d). %s.\n", 0, FL,
LN, FN, channel, gain,
err, uhdError(err));
return(FAIL);
}
if (!(err =
uhd_usrp_get_rx_gain(dev[channel],
channel, "", &tmp)))
info(log, "Tuner[%d] gain
set to %.0lf (%.0lf)
dB\n", 0, channel, tmp,gain);
./* Set channel bw to
conserve tuner resources. Not
needed, set by srate */
uhd_usrp_set_rx_bandwidth(dev[channel],
srate, channel);
./* Disable subtracting
constant averaged background.
Signal looks cleaner */
if ((err =
uhd_usrp_set_rx_dc_offset_enabled(dev[channel],
false, channel)))
{
warn(log, "Failed to disable
FPGA DC offset on channel
%d(%d). %s.\n", 0,
FL, LN, FN, channel, err,
uhdError(err));
}
info(stderr, "Disabled FPGA
DC offset on channel %d\n",
0, channel);
return(SUCCESS);
}
This is the generated output:
[INFO] [UHD] linux; GNU C++
version 13.2.0; Boost_108300;
UHD_4.6.0.0+ds1-5.1ubuntu0.24.04.1
[INFO] [X300] X300
initialization sequence...
[INFO] [X300] Maximum frame
size: 8000 bytes.
[INFO] [X300] Radio 1x clock:
200 MHz
Sat Apr 26 03:33:48 2025 [00]
[+] Created USRP with args
Sat Apr 26 03:33:48 2025 [00]
[+] Master clock is at 200 Mhz
Sat Apr 26 03:33:48 2025 [00]
[+] Tuner[0] gain set to 30
(30) dB
Sat Apr 26 03:33:48 2025 [00]
[*] scanner.l:1446:main
Incorrect maxsamples (1996).
Expected 19960.
Sat Apr 26 03:33:48 2025 [00]
[+] Max samples/buffer[0]: 1996
[WARNING] [0/Radio#0]
Ignoring stream command for
finite acquisition of zero
samples
I hope this reads OK. Maybe
next time I should attach the
code:)
TIA
Nikos
_______________________________________________
USRP-users mailing list --usrp-users@lists.ettus.com
To unsubscribe send an email tousrp-users-leave@lists.ettus.com
I believe that max number of
samples-per-buffer is limited
by MTU size. Which is
typically around 8000 or so
for "jumbo frames".
True and thanks for all your help:)
On Tue, Apr 29, 2025 at 2:55 AM Marcus D. Leech patchvonbraun@gmail.com
wrote:
On 28/04/2025 19:08, Nikos Balkanas wrote:
On Tue, Apr 29, 2025 at 1:31 AM Marcus D. Leech patchvonbraun@gmail.com
wrote:
On 28/04/2025 18:04, Nikos Balkanas wrote:
Hi Marcus,
spb(=samples/buffer) is not what I hoped for. It is just another way to
set the
stream_cmd.num_samples in the streamer. I already use there MAXSMPS.
Besides benchmark_rate reports maxsmps 1996 like me:(
That part is only controlled by the MTU. Seems it is the same
for everyone, and that we are using the generic chdr_sc16_to_xx
instead the guts (sse2) conversion:( Unless they use the multi_usrp
class and set spp on their own.
I think that MAXSMPS used to be 1996, before someone changed it to
19960.
BR
Nikos
BR
Nikos
Well, regardless of all of this, there's no way to pack more than a
certain number of samples into an MTU frame.
If your high-level needs are to process "things" in larger chunks, you
need to layer something on top of what UHD
does, since it is, at the end of the day, a hardware driver library.
It's not an application layer programming framework.
Not really. I just need for FFT 1024 samples at the moment. My concern is
more about the sse2 code not used:(
What about packet fragmentation? NICs can handle those...
I think that by default UDP on most systems these days sets the "DF" bit
-- so that higher-layer protocols like UDP
restrict themselves to non-fragmented packets at that level. That's
because in the larger internet, you can't rely
on in-order fragment handling, and re-ordering is a HUGE performance
loss.
Anyway, for a 1024-sample FFT, you have what you need.
On Mon, Apr 28, 2025 at 5:22 PM Nikos Balkanas nbalkanas@gmail.com
wrote:
Thx Marcus for the clarifications,
On Mon, Apr 28, 2025 at 4:37 PM Marcus D. Leech patchvonbraun@gmail.com
wrote:
On 28/04/2025 05:33, Nikos Balkanas wrote:
Compiled uhd 4.6.0 in debug mode.
From the output I get:
[DEBUG] [0/Radio#0] spp(= samples per package) value 2032 exceeds MTU
of 8000! Coercing to 1996
Mon Apr 28 09:57:02 2025 [00] [*] scanner.l:1443:main Incorrect
maxsamples (1996). Expected 19960.
Mon Apr 28 09:57:02 2025 [00] [+] Max samples/buffer[0]: 1996
- Line mtu is 9000 not 8000
- 2032 is not larger than 8000 <= Bug?
- seems that spp is setting my maxsmps
That's number of SAMPLES. Samples are 4 bytes total.
Aaaah. I'll have to check the SPB option. Otherwise an 80K MTU is
unreasonable:)
This also means that 1 sample = 1 real + 1 imag = 32 bits with sc16
encoding
.
A lot of network hardware, particularly 1Gbit hardware doesn't
ACTUALLY support an MTU of more than 8000, and I think
UHD uses PMTU discovery. I found that with RealTek NICs, even when
you set the MTU to 9000, it actually only supports
8000.
Same case with my Mellanox NIC. But 8000 is close enough:)
Of the examples I tried the rx_samples_c. It is the same case like
mine: single usrp. We use the same commands
and we are getting the same output:( 1996 maxsmpls.
The error text and code are from host/lib/rfnoc/radio_control_impl.cpp:
199
I would rather not touch it. I don't know the uhd architecture and
especially the rfnoc/uhd interface.
Besides I am a c programmer, not c++:(
multi_usrp class has a set_rx_spp function, but it is not for me:(
You can look at the benchmark_rate example to see how to set a
samples-per-buffer other than the default, which is
based on the MTU. It uses an "SPB" command-line parameter.
Thx, I will check it out, when I get back to ubuntu.. Now I am in
windows:(
benchmark_rate uses the multi-usrp class.
HTH
Nikos
On Mon, Apr 28, 2025 at 6:02 AM Nikos Balkanas nbalkanas@gmail.com
wrote:
Point taken:) I'm proposing smt different:
pchar +ICMP are just to test line and connectors. First step. Not to
bench USRP.
benchmark_rate is to bench/stress usrp.
These 2 are independent, and complementary.
Pchar is telling me nothing more than my fiber cable and connectors
are good.
It saved me a trip tomorrow to my local PC store, where fiber cable
and connectors are ~7 E each.
benchmark_rate on the other hand is quite interesting.
It points to software,and particularly my uhd_init()
Just found and downloaded the sources to uhd 4.6.0 from Ubuntu
Launchpad.
Now I can go through the source of the example you told me:)
Ettus used to keep an archive of old uhd sources around. Seems it's
gone:(
Open source means, among others, free to choose the source version
that you need...
Having the latest source in Github is only partly open source.
During development we need to freeze updates. When in 5 years we go
into production we can't find the old sources anymore:(
If a customer discovers a bug, not in our code, but in one of the
libraries that
we use, what are we gonna do?
BR
Nikos
On Mon, Apr 28, 2025 at 5:01 AM Marcus D. Leech <
patchvonbraun@gmail.com> wrote:
On 27/04/2025 21:58, Nikos Balkanas wrote:
My bad:
throughput of 5.619 Kb/s requesting ICMP replies, +> throughput of
5,619 Kb/s requesting ICMP replies
Local thousand separator is ".", whereas in the US is ",":(
It is STILL the case that the ICMP machinery in these radios is
ABSOLUTELY NOT on the fast-path inside
the hardware. The only way to get a good feel for how much sample
bandwidth they can process is
with examples like "benchmark_rate".
On Mon, Apr 28, 2025 at 12:37 AM Nikos Balkanas nbalkanas@gmail.com
wrote:
Hi Marcus,
You were right. No need to change NIC:)
This is not a software issue. uhd_rx_streamer_max_num_samps runs
right after uhd initialization before
any other code had the chance to run.
Link capacity doesn't seem to be the issue either...
Running pchar on the link, descendant of pathchar, reports a
throughput of 5.619 Kb/s requesting ICMP replies,
to varying packet sizes (32->9000 (MTU), incr by 32).
sudo pchar -m 9000 -p ipv4icmp usrp
https://www.kitchenlab.org/www/bmah/Software/pchar/
It corresponds to 351.218.019 16-bit samples or 175,609.044 32-bit
samples, if each sample is 32-bit(real + imag)
Seems that uhd is not running at link capacity but is doing smt else.
I will have to download and check with the sources...
The package version for Ubuntu 24.04 is uhd 4.6.0.
Where can I download the sources for uhd 4.6.0?
BR
Nikos
On Sat, Apr 26, 2025 at 7:02 AM Nikos Balkanas nbalkanas@gmail.com
wrote:
Thanks for your time.
I will check out the example.
This is not a buffer problem. I just need 1024 Samples
(real+imaginary) for FFT...
I should be able to get them in a single pass.
You saw my code, not a smoking gun there.
This is probably is a physical problem.
Cable is an SFP fiber dedicated line. Cannot go bad.
Maybe the connections are not sitting right :(...
BR
Nikos
On Sat, Apr 26, 2025 at 6:45 AM Marcus D. Leech <
patchvonbraun@gmail.com> wrote:
On 25/04/2025 23:33, Nikos Balkanas wrote:
Actually MTU is 9000. This is one of the recommendations...
I tried it with MTU 1500. It was worse:(
maxsamples dropped to 364...
Right, 9000, rather than 8000.
Upgrading to 10Gbit wont' give you larger MTU.
What you're trying to do, I think, is to solve a buffer-management
problem in your application at entirely the wrong
level in the stack.
It is EXCEEDINGLY COMMON for hardware drivers to only be able to
deliver in chunks that may not be perfectly adapted to
the requirements of your application. So, a common programming
pattern is to deal with this in your application.
You should probably look at example code like rx_samples_to_file
[INFO] [UHD] linux; GNU C++ version 13.2.0; Boost_108300;
UHD_4.6.0.0+ds1-5.1ubuntu0.24.04.1
[INFO] [X300] X300 initialization sequence...
[INFO] [X300] Maximum frame size: 1472 bytes.
[WARNING] [X300] For the 192.168.40.2 connection, UHD recommends a
send frame size of at least 8000 for best
performance, but your configuration will only allow 1472.This may
negatively impact your maximum achievable sample rate.
Check the MTU on the interface and/or the send_frame_size argument.
[WARNING] [X300] For the 192.168.40.2 connection, UHD recommends a
receive frame size of at least 8000 for best
performance, but your configuration will only allow 1472.This may
negatively impact your maximum achievable sample rate.
Check the MTU on the interface and/or the recv_frame_size argument.
[INFO] [GPS] No GPSDO found
[INFO] [X300] Radio 1x clock: 200 MHz
[WARNING] [UDP] The send buffer could not be resized sufficiently.
Target sock buff size: 24912805 bytes.
Actual sock buff size: 1048576 bytes.
See the transport application notes on buffer resizing.
Please run: sudo sysctl -w net.core.wmem_max=24912805
Sat Apr 26 06:30:34 2025 [00] [+] Created USRP with args
Sat Apr 26 06:30:34 2025 [00] [+] Master clock is at 200 Mhz
Sat Apr 26 06:30:34 2025 [00] [+] Tuner[0] gain set to 30 (30) dB
[WARNING] [UDP] The send buffer could not be resized sufficiently.
Target sock buff size: 24912805 bytes.
Actual sock buff size: 1048576 bytes.
See the transport application notes on buffer resizing.
Please run: sudo sysctl -w net.core.wmem_max=24912805
Sat Apr 26 06:30:34 2025 [00] [*] scanner.l:1446:main Incorrect
maxsamples (364). Expected 19960.
Sat Apr 26 06:30:34 2025 [00] [+] Max samples/buffer[0]: 364
[WARNING] [0/Radio#0] Ignoring stream command for finite
acquisition of zero sam
Nikos
On Sat, Apr 26, 2025 at 5:46 AM Marcus D. Leech <
patchvonbraun@gmail.com> wrote:
On 25/04/2025 22:26, Nikos Balkanas wrote:
Thanks Marcus,
for your fast reply.
On Sat, Apr 26, 2025 at 4:08 AM Marcus D. Leech <
patchvonbraun@gmail.com> wrote:
On 25/04/2025 20:50, Nikos Balkanas wrote:
Hello,
I need to buy a new NIC. What would you suggest?
The one I use is an old Mellanox 10 Gbs, before the Connect-4
series.
It can only do 1996 S/s, need 19960 (10x more) to work with
latest uhd.
Using Ubuntu 24.04 and uhd 4.6.0
So, 1.996ksps vs 19.960ksps? You hardly need a 10Gbit link to
support that. So, perhaps something
is being lost here in your requirements?
True. Can't explain it in terms of bandwidth. 16 * 1996 = 31.936
Kbps, 16 * 19960 = 319.360 Kbps well short of a 10 Gbps line:(
Does a complex pair count as 1 sample, or 2?
I have followed all the instructions in
https://kb.ettus.com/USRP_Host_Performance_Tuning_Tips_and_Tricks
,
Even installed the DPDK drivers. My Mellanox is too old to use
their OFED drivers:(
On a related question. it seems that the streamer doesn't crash
anymore
when receiving less than MAXSPS samples, instead it reads 0:(
This was due to the sse2 code not aligned in convert.
I change my stream args to cpu_format=sc16, otw=sc16, so no
conversion required.
Streamer still doesn't read anything. Is there a reason for it?
You'd need to share more of your code. This should just work as
far as I can tell.
Thanks. these are just the usrp code:
int main()
{
uhd_stream_args_t stream_args =
{
.cpu_format = "sc16",
.otw_format = "sc16",
.args = "",
.n_channels = 1,
.channel_list = &channel
};
..uhd_stream_cmd_t stream_cmd =
{
.stream_mode = UHD_STREAM_MODE_NUM_SAMPS_AND_DONE,
.stream_now = true
};
if (uhd_init(0, 0, gain)) do_exit(20);
if ((err = uhd_usrp_get_rx_stream(dev[0], &stream_args,
rx_streamer[0])))
{
uhd_get_last_error(errmsg, 127);
error(log, "Failed to get streamer[0] (%d). %s.\n", 0,
FL, LN, FN, err, errmsg);
uhd_rx_streamer_free(&rx_streamer[0]);
rx_streamer[0] = NULL;
uhd_rx_metadata_free(&md[0]);
md[0] = NULL;
do_exit(30);
}
if ((err = uhd_rx_streamer_max_num_samps(rx_streamer[0],
&maxsamps)))
{
uhd_get_last_error(errmsg, 127);
error(log, "Failed to get max samples/buffer[0] (%d).
%s.\n", 0, FL, LN, FN, err,
..errmsg);
do_exit(35);
}
if (maxsamps != MAXSMPS)
warn(log, "Incorrect maxsamples (%ld). Expected %d.\n",
0, FL, LN, FN, maxsamps,
MAXSMPS);
info(log, "Max samples/buffer[0]: %ld\n", 0, maxsamps);
if ((err = uhd_rx_streamer_issue_stream_cmd(rx_streamer[0],
&stream_cmd)))
{
uhd_get_last_error(errmsg, 127);
error(log, "Failed to start streamer[0] (%d). %s.\n",
0, FL, LN, FN, err, errmsg);
do_exit(40);
}
[...]
do_exit(0)
}
bool uhd_init(size_t channel, double srate, double gain)
{
double tmp;
uhd_rx_metadata_error_code_t err;
if ((err =
uhd_set_thread_priority(uhd_default_thread_priority, true)))
warn(log, "Unable to set main thread priority
(%d). %s.\n", 0, FL, LN, FN,
err, uhdError(err));
/* Create USRP */
f ((err = uhd_usrp_make(&dev[channel], "type=x300")))
{
error(log, "Failed to create USRP (%d). %s.\n", 0,
FL, LN, FN, err,
uhdError(err));
dev[channel] = NULL;
return(FAIL);
}
info(stderr, "Created USRP with args\n", 0);
/* Create RX streamer */
if ((err = uhd_rx_streamer_make(&rx_streamer[channel])))
{
error(log, "Failed to create rx_streamer[%d] (%d).
%s.\n", 0, FL, LN, FN,
channel, err, uhdError(err));
return(FAIL);
}
/* Create RX metadata */
if ((err = uhd_rx_metadata_make(&md[channel])))
{
error(log, "Failed to create md[%d] (%d). %s.\n", 0, FL,
LN, FN, channel,
err, uhdError(err));
return(FAIL);
}
/* Get master clock rate */
if ((err = uhd_usrp_get_master_clock_rate(dev[channel], 0,
&tmp)))
{
error(log, "Failed to set master clock to %.0lf Mhz
(%d). %s.\n", 0, FL,
LN, FN, tmp/1000000, err, uhdError(err));
return(FAIL);
}
info(stderr, "Master clock is at %.0lf Mhz\n", 0,
tmp/1000000);
/* Set the sample rate /
if (srate && !uhd_set_rx_rate_check(channel, srate))
return(FAIL);
/ Set the tuner gain SBX-120 is 0-31.5 in .5 db steps */
if ((err = uhd_usrp_set_rx_gain(dev[channel], gain,
channel, "")))
{
error(log, "Failed to set tuner[%d] gain to %.0lf
db (%d). %s.\n", 0, FL,
LN, FN, channel, gain, err, uhdError(err));
return(FAIL);
}
if (!(err = uhd_usrp_get_rx_gain(dev[channel],
channel, "", &tmp)))
info(log, "Tuner[%d] gain set to %.0lf (%.0lf)
dB\n", 0, channel, tmp, gain);
./* Set channel bw to conserve tuner resources. Not
needed, set by srate /
uhd_usrp_set_rx_bandwidth(dev[channel], srate,
channel);
./ Disable subtracting constant averaged background.
Signal looks cleaner */
if ((err =
uhd_usrp_set_rx_dc_offset_enabled(dev[channel], false, channel)))
{
warn(log, "Failed to disable FPGA DC offset on
channel %d(%d). %s.\n", 0,
FL, LN, FN, channel, err, uhdError(err));
}
info(stderr, "Disabled FPGA DC offset on channel
%d\n", 0, channel);
return(SUCCESS);
}
This is the generated output:
[INFO] [UHD] linux; GNU C++ version 13.2.0; Boost_108300;
UHD_4.6.0.0+ds1-5.1ubuntu0.24.04.1
[INFO] [X300] X300 initialization sequence...
[INFO] [X300] Maximum frame size: 8000 bytes.
[INFO] [X300] Radio 1x clock: 200 MHz
Sat Apr 26 03:33:48 2025 [00] [+] Created USRP with args
Sat Apr 26 03:33:48 2025 [00] [+] Master clock is at 200 Mhz
Sat Apr 26 03:33:48 2025 [00] [+] Tuner[0] gain set to 30 (30) dB
Sat Apr 26 03:33:48 2025 [00] [*] scanner.l:1446:main Incorrect
maxsamples (1996). Expected 19960.
Sat Apr 26 03:33:48 2025 [00] [+] Max samples/buffer[0]: 1996
[WARNING] [0/Radio#0] Ignoring stream command for finite
acquisition of zero samples
I hope this reads OK. Maybe next time I should attach the code:)
TIA
Nikos
USRP-users mailing list -- usrp-users@lists.ettus.com
To unsubscribe send an email to usrp-users-leave@lists.ettus.com
I believe that max number of samples-per-buffer is limited by
MTU size. Which is typically around 8000 or so for "jumbo frames".
Dear all,
I stand corrected for the benchmark_rate results:
[DEBUG] [RFNOC::MGMT] Finished RX stream setup for EPID=2
[DEBUG] [0/Radio#0] spp value 2032 exceeds MTU of 8000! Coercing to 1996
[00:00:02.843054239] Testing receive rate 2.000000 Msps on 1 channels
[00:00:12.845077647] Benchmark complete.
Debug messages show that maxsampls = 1996 with --rx_spb = 19960.
Further testing, printing just before the SSE2 code,
either --rx_spp, or --rx_spb = 19960 will change the streamer's
cmd.num_sampls
to 19960 and cause buffer to get converted by the SSE2 code. Single
USRPs can't execute the SSE2 code. No matter what they use
in cmd.num_sampls or in the streamer, maxsps = 1969 and that's
what reaches the convert function:(
As noted MAXSPS packages will cause excessive network fragmentation, and
cause
more delays in the NIC. As it stands right now it'a a wash in my system.
In either case I get exactly the same benchmark results with SSE2 code
or without: 2e6 S/s. In your systems it might make a difference. If there
are
further improvements in the SSE2 code (ie embedded asm, instead of macro)
It could make more of a difference.
Sorry about the confusion:(
On Tue, Apr 29, 2025 at 1:04 AM Nikos Balkanas nbalkanas@gmail.com wrote:
Hi Marcus,
spb(=samples/buffer) is not what I hoped for. It is just another way to
set the
stream_cmd.num_samples in the streamer. I already use there MAXSMPS.
Besides benchmark_rate reports maxsmps 1996 like me:(
That part is only controlled by the MTU. Seems it is the same
for everyone, and that we are using the generic chdr_sc16_to_xx
instead the guts (sse2) conversion:( Unless they use the multi_usrp
class and set spp on their own.
I think that MAXSMPS used to be 1996, before someone changed it to
19960.
BR
Nikos
BR
Nikos
On Mon, Apr 28, 2025 at 5:22 PM Nikos Balkanas nbalkanas@gmail.com
wrote:
Thx Marcus for the clarifications,
On Mon, Apr 28, 2025 at 4:37 PM Marcus D. Leech patchvonbraun@gmail.com
wrote:
On 28/04/2025 05:33, Nikos Balkanas wrote:
Compiled uhd 4.6.0 in debug mode.
From the output I get:
[DEBUG] [0/Radio#0] spp(= samples per package) value 2032 exceeds MTU of
8000! Coercing to 1996
Mon Apr 28 09:57:02 2025 [00] [*] scanner.l:1443:main Incorrect
maxsamples (1996). Expected 19960.
Mon Apr 28 09:57:02 2025 [00] [+] Max samples/buffer[0]: 1996
- Line mtu is 9000 not 8000
- 2032 is not larger than 8000 <= Bug?
- seems that spp is setting my maxsmps
That's number of SAMPLES. Samples are 4 bytes total.
Aaaah. I'll have to check the SPB option. Otherwise an 80K MTU is
unreasonable:)
This also means that 1 sample = 1 real + 1 imag = 32 bits with sc16
encoding
.
A lot of network hardware, particularly 1Gbit hardware doesn't ACTUALLY
support an MTU of more than 8000, and I think
UHD uses PMTU discovery. I found that with RealTek NICs, even when
you set the MTU to 9000, it actually only supports
8000.
Same case with my Mellanox NIC. But 8000 is close enough:)
Of the examples I tried the rx_samples_c. It is the same case like mine:
single usrp. We use the same commands
and we are getting the same output:( 1996 maxsmpls.
The error text and code are from host/lib/rfnoc/radio_control_impl.cpp:
199
I would rather not touch it. I don't know the uhd architecture and
especially the rfnoc/uhd interface.
Besides I am a c programmer, not c++:(
multi_usrp class has a set_rx_spp function, but it is not for me:(
You can look at the benchmark_rate example to see how to set a
samples-per-buffer other than the default, which is
based on the MTU. It uses an "SPB" command-line parameter.
Thx, I will check it out, when I get back to ubuntu.. Now I am in
windows:(
benchmark_rate uses the multi-usrp class.
HTH
Nikos
On Mon, Apr 28, 2025 at 6:02 AM Nikos Balkanas nbalkanas@gmail.com
wrote:
Point taken:) I'm proposing smt different:
pchar +ICMP are just to test line and connectors. First step. Not to
bench USRP.
benchmark_rate is to bench/stress usrp.
These 2 are independent, and complementary.
Pchar is telling me nothing more than my fiber cable and connectors are
good.
It saved me a trip tomorrow to my local PC store, where fiber cable and
connectors are ~7 E each.
benchmark_rate on the other hand is quite interesting.
It points to software,and particularly my uhd_init()
Just found and downloaded the sources to uhd 4.6.0 from Ubuntu
Launchpad.
Now I can go through the source of the example you told me:)
Ettus used to keep an archive of old uhd sources around. Seems it's
gone:(
Open source means, among others, free to choose the source version that
you need...
Having the latest source in Github is only partly open source.
During development we need to freeze updates. When in 5 years we go
into production we can't find the old sources anymore:(
If a customer discovers a bug, not in our code, but in one of the
libraries that
we use, what are we gonna do?
BR
Nikos
On Mon, Apr 28, 2025 at 5:01 AM Marcus D. Leech <
patchvonbraun@gmail.com> wrote:
On 27/04/2025 21:58, Nikos Balkanas wrote:
My bad:
throughput of 5.619 Kb/s requesting ICMP replies, +> throughput of
5,619 Kb/s requesting ICMP replies
Local thousand separator is ".", whereas in the US is ",":(
It is STILL the case that the ICMP machinery in these radios is
ABSOLUTELY NOT on the fast-path inside
the hardware. The only way to get a good feel for how much sample
bandwidth they can process is
with examples like "benchmark_rate".
On Mon, Apr 28, 2025 at 12:37 AM Nikos Balkanas nbalkanas@gmail.com
wrote:
Hi Marcus,
You were right. No need to change NIC:)
This is not a software issue. uhd_rx_streamer_max_num_samps runs
right after uhd initialization before
any other code had the chance to run.
Link capacity doesn't seem to be the issue either...
Running pchar on the link, descendant of pathchar, reports a
throughput of 5.619 Kb/s requesting ICMP replies,
to varying packet sizes (32->9000 (MTU), incr by 32).
sudo pchar -m 9000 -p ipv4icmp usrp
https://www.kitchenlab.org/www/bmah/Software/pchar/
It corresponds to 351.218.019 16-bit samples or 175,609.044 32-bit
samples, if each sample is 32-bit(real + imag)
Seems that uhd is not running at link capacity but is doing smt else.
I will have to download and check with the sources...
The package version for Ubuntu 24.04 is uhd 4.6.0.
Where can I download the sources for uhd 4.6.0?
BR
Nikos
On Sat, Apr 26, 2025 at 7:02 AM Nikos Balkanas nbalkanas@gmail.com
wrote:
Thanks for your time.
I will check out the example.
This is not a buffer problem. I just need 1024 Samples
(real+imaginary) for FFT...
I should be able to get them in a single pass.
You saw my code, not a smoking gun there.
This is probably is a physical problem.
Cable is an SFP fiber dedicated line. Cannot go bad.
Maybe the connections are not sitting right :(...
BR
Nikos
On Sat, Apr 26, 2025 at 6:45 AM Marcus D. Leech <
patchvonbraun@gmail.com> wrote:
On 25/04/2025 23:33, Nikos Balkanas wrote:
Actually MTU is 9000. This is one of the recommendations...
I tried it with MTU 1500. It was worse:(
maxsamples dropped to 364...
Right, 9000, rather than 8000.
Upgrading to 10Gbit wont' give you larger MTU.
What you're trying to do, I think, is to solve a buffer-management
problem in your application at entirely the wrong
level in the stack.
It is EXCEEDINGLY COMMON for hardware drivers to only be able to
deliver in chunks that may not be perfectly adapted to
the requirements of your application. So, a common programming
pattern is to deal with this in your application.
You should probably look at example code like rx_samples_to_file
[INFO] [UHD] linux; GNU C++ version 13.2.0; Boost_108300;
UHD_4.6.0.0+ds1-5.1ubuntu0.24.04.1
[INFO] [X300] X300 initialization sequence...
[INFO] [X300] Maximum frame size: 1472 bytes.
[WARNING] [X300] For the 192.168.40.2 connection, UHD recommends a
send frame size of at least 8000 for best
performance, but your configuration will only allow 1472.This may
negatively impact your maximum achievable sample rate.
Check the MTU on the interface and/or the send_frame_size argument.
[WARNING] [X300] For the 192.168.40.2 connection, UHD recommends a
receive frame size of at least 8000 for best
performance, but your configuration will only allow 1472.This may
negatively impact your maximum achievable sample rate.
Check the MTU on the interface and/or the recv_frame_size argument.
[INFO] [GPS] No GPSDO found
[INFO] [X300] Radio 1x clock: 200 MHz
[WARNING] [UDP] The send buffer could not be resized sufficiently.
Target sock buff size: 24912805 bytes.
Actual sock buff size: 1048576 bytes.
See the transport application notes on buffer resizing.
Please run: sudo sysctl -w net.core.wmem_max=24912805
Sat Apr 26 06:30:34 2025 [00] [+] Created USRP with args
Sat Apr 26 06:30:34 2025 [00] [+] Master clock is at 200 Mhz
Sat Apr 26 06:30:34 2025 [00] [+] Tuner[0] gain set to 30 (30) dB
[WARNING] [UDP] The send buffer could not be resized sufficiently.
Target sock buff size: 24912805 bytes.
Actual sock buff size: 1048576 bytes.
See the transport application notes on buffer resizing.
Please run: sudo sysctl -w net.core.wmem_max=24912805
Sat Apr 26 06:30:34 2025 [00] [*] scanner.l:1446:main Incorrect
maxsamples (364). Expected 19960.
Sat Apr 26 06:30:34 2025 [00] [+] Max samples/buffer[0]: 364
[WARNING] [0/Radio#0] Ignoring stream command for finite
acquisition of zero sam
Nikos
On Sat, Apr 26, 2025 at 5:46 AM Marcus D. Leech <
patchvonbraun@gmail.com> wrote:
On 25/04/2025 22:26, Nikos Balkanas wrote:
Thanks Marcus,
for your fast reply.
On Sat, Apr 26, 2025 at 4:08 AM Marcus D. Leech <
patchvonbraun@gmail.com> wrote:
On 25/04/2025 20:50, Nikos Balkanas wrote:
Hello,
I need to buy a new NIC. What would you suggest?
The one I use is an old Mellanox 10 Gbs, before the Connect-4
series.
It can only do 1996 S/s, need 19960 (10x more) to work with
latest uhd.
Using Ubuntu 24.04 and uhd 4.6.0
So, 1.996ksps vs 19.960ksps? You hardly need a 10Gbit link to
support that. So, perhaps something
is being lost here in your requirements?
True. Can't explain it in terms of bandwidth. 16 * 1996 = 31.936
Kbps, 16 * 19960 = 319.360 Kbps well short of a 10 Gbps line:(
Does a complex pair count as 1 sample, or 2?
I have followed all the instructions in
https://kb.ettus.com/USRP_Host_Performance_Tuning_Tips_and_Tricks,
Even installed the DPDK drivers. My Mellanox is too old to use
their OFED drivers:(
On a related question. it seems that the streamer doesn't crash
anymore
when receiving less than MAXSPS samples, instead it reads 0:(
This was due to the sse2 code not aligned in convert.
I change my stream args to cpu_format=sc16, otw=sc16, so no
conversion required.
Streamer still doesn't read anything. Is there a reason for it?
You'd need to share more of your code. This should just work as
far as I can tell.
Thanks. these are just the usrp code:
int main()
{
uhd_stream_args_t stream_args =
{
.cpu_format = "sc16",
.otw_format = "sc16",
.args = "",
.n_channels = 1,
.channel_list = &channel
};
..uhd_stream_cmd_t stream_cmd =
{
.stream_mode = UHD_STREAM_MODE_NUM_SAMPS_AND_DONE,
.stream_now = true
};
if (uhd_init(0, 0, gain)) do_exit(20);
if ((err = uhd_usrp_get_rx_stream(dev[0], &stream_args,
rx_streamer[0])))
{
uhd_get_last_error(errmsg, 127);
error(log, "Failed to get streamer[0] (%d). %s.\n", 0,
FL, LN, FN, err, errmsg);
uhd_rx_streamer_free(&rx_streamer[0]);
rx_streamer[0] = NULL;
uhd_rx_metadata_free(&md[0]);
md[0] = NULL;
do_exit(30);
}
if ((err = uhd_rx_streamer_max_num_samps(rx_streamer[0],
&maxsamps)))
{
uhd_get_last_error(errmsg, 127);
error(log, "Failed to get max samples/buffer[0] (%d).
%s.\n", 0, FL, LN, FN, err,
..errmsg);
do_exit(35);
}
if (maxsamps != MAXSMPS)
warn(log, "Incorrect maxsamples (%ld). Expected %d.\n",
0, FL, LN, FN, maxsamps,
MAXSMPS);
info(log, "Max samples/buffer[0]: %ld\n", 0, maxsamps);
if ((err = uhd_rx_streamer_issue_stream_cmd(rx_streamer[0],
&stream_cmd)))
{
uhd_get_last_error(errmsg, 127);
error(log, "Failed to start streamer[0] (%d). %s.\n", 0,
FL, LN, FN, err, errmsg);
do_exit(40);
}
[...]
do_exit(0)
}
bool uhd_init(size_t channel, double srate, double gain)
{
double tmp;
uhd_rx_metadata_error_code_t err;
if ((err =
uhd_set_thread_priority(uhd_default_thread_priority, true)))
warn(log, "Unable to set main thread priority (%d).
%s.\n", 0, FL, LN, FN,
err, uhdError(err));
/* Create USRP */
f ((err = uhd_usrp_make(&dev[channel], "type=x300")))
{
error(log, "Failed to create USRP (%d). %s.\n", 0, FL,
LN, FN, err,
uhdError(err));
dev[channel] = NULL;
return(FAIL);
}
info(stderr, "Created USRP with args\n", 0);
/* Create RX streamer */
if ((err = uhd_rx_streamer_make(&rx_streamer[channel])))
{
error(log, "Failed to create rx_streamer[%d] (%d).
%s.\n", 0, FL, LN, FN,
channel, err, uhdError(err));
return(FAIL);
}
/* Create RX metadata */
if ((err = uhd_rx_metadata_make(&md[channel])))
{
error(log, "Failed to create md[%d] (%d). %s.\n", 0, FL,
LN, FN, channel,
err, uhdError(err));
return(FAIL);
}
/* Get master clock rate */
if ((err = uhd_usrp_get_master_clock_rate(dev[channel], 0,
&tmp)))
{
error(log, "Failed to set master clock to %.0lf Mhz
(%d). %s.\n", 0, FL,
LN, FN, tmp/1000000, err, uhdError(err));
return(FAIL);
}
info(stderr, "Master clock is at %.0lf Mhz\n", 0,
tmp/1000000);
/* Set the sample rate /
if (srate && !uhd_set_rx_rate_check(channel, srate))
return(FAIL);
/ Set the tuner gain SBX-120 is 0-31.5 in .5 db steps */
if ((err = uhd_usrp_set_rx_gain(dev[channel], gain,
channel, "")))
{
error(log, "Failed to set tuner[%d] gain to %.0lf db
(%d). %s.\n", 0, FL,
LN, FN, channel, gain, err, uhdError(err));
return(FAIL);
}
if (!(err = uhd_usrp_get_rx_gain(dev[channel], channel,
"", &tmp)))
info(log, "Tuner[%d] gain set to %.0lf (%.0lf)
dB\n", 0, channel, tmp, gain);
./* Set channel bw to conserve tuner resources. Not
needed, set by srate /
uhd_usrp_set_rx_bandwidth(dev[channel], srate,
channel);
./ Disable subtracting constant averaged background.
Signal looks cleaner */
if ((err =
uhd_usrp_set_rx_dc_offset_enabled(dev[channel], false, channel)))
{
warn(log, "Failed to disable FPGA DC offset on
channel %d(%d). %s.\n", 0,
FL, LN, FN, channel, err, uhdError(err));
}
info(stderr, "Disabled FPGA DC offset on channel
%d\n", 0, channel);
return(SUCCESS);
}
This is the generated output:
[INFO] [UHD] linux; GNU C++ version 13.2.0; Boost_108300;
UHD_4.6.0.0+ds1-5.1ubuntu0.24.04.1
[INFO] [X300] X300 initialization sequence...
[INFO] [X300] Maximum frame size: 8000 bytes.
[INFO] [X300] Radio 1x clock: 200 MHz
Sat Apr 26 03:33:48 2025 [00] [+] Created USRP with args
Sat Apr 26 03:33:48 2025 [00] [+] Master clock is at 200 Mhz
Sat Apr 26 03:33:48 2025 [00] [+] Tuner[0] gain set to 30 (30) dB
Sat Apr 26 03:33:48 2025 [00] [*] scanner.l:1446:main Incorrect
maxsamples (1996). Expected 19960.
Sat Apr 26 03:33:48 2025 [00] [+] Max samples/buffer[0]: 1996
[WARNING] [0/Radio#0] Ignoring stream command for finite
acquisition of zero samples
I hope this reads OK. Maybe next time I should attach the code:)
TIA
Nikos
USRP-users mailing list -- usrp-users@lists.ettus.com
To unsubscribe send an email to usrp-users-leave@lists.ettus.com
I believe that max number of samples-per-buffer is limited by MTU
size. Which is typically around 8000 or so for "jumbo frames".