usrp-users@lists.ettus.com

Discussion and technical support related to USRP, UHD, RFNoC

View all threads

polyphase clock sync eats 100% cpu once it gets samples

VR
Vladimir Rytikov
Tue, Oct 3, 2017 10:54 AM

Hi,

I am trying to run an example from GNU Radio -
examples/digital/packet/uhd_packet_rx and uhd_packet_tx with real USPR
radios connected via an attenuator and a coax cable.

When I enable receiver by clicking on 'On' check box - the whole RX flow
graph freezes.
I think I manually adjusted transmit power and receive gain to be within
reasonable ranges.

By disabling different blocks one at a time I found is that polyphase clock
sync inside packet_rx blocks kills the flow graph. It seems like the signal
is gaited by Correlation Estimator and once it gets the correct sync word -
the signal goes to Polyphase Clock Sync and CPU dies - it is 100% loaded
all the time.

If I run only loop back simulation the whole flow graph seems working fine.
I wonder if noise or not very good signal destroys Polyphase Clock Sync.

I installed GNU Radio via PyBOMS - 3.7.12git* version. Please help to find
better idea on debugging it.

with a different simple flow graphs Polyphase Clock Sync works fine even
when I feed noise to it and signal later.

Thanks,
Vladimir

Hi, I am trying to run an example from GNU Radio - examples/digital/packet/uhd_packet_rx and uhd_packet_tx with real USPR radios connected via an attenuator and a coax cable. When I enable receiver by clicking on 'On' check box - the whole RX flow graph freezes. I think I manually adjusted transmit power and receive gain to be within reasonable ranges. By disabling different blocks one at a time I found is that polyphase clock sync inside packet_rx blocks kills the flow graph. It seems like the signal is gaited by Correlation Estimator and once it gets the correct sync word - the signal goes to Polyphase Clock Sync and CPU dies - it is 100% loaded all the time. If I run only loop back simulation the whole flow graph seems working fine. I wonder if noise or not very good signal destroys Polyphase Clock Sync. I installed GNU Radio via PyBOMS - 3.7.12git* version. Please help to find better idea on debugging it. with a different simple flow graphs Polyphase Clock Sync works fine even when I feed noise to it and signal later. Thanks, Vladimir
MM
Marcus Müller
Tue, Oct 3, 2017 11:57 AM

Hi Vladimir,

synchronization is usually among the most CPU-intense things a receiver
does (only, if at all, contested by channel decoding for complex codes).
So, the 100% CPU utilization don't sound totally unreasonable, depending
on your system.

That being said, I don't want to rule out bugs, but for the time being,
I'd declare this issue as "unclear, probably insufficient compute power".

Can you tell us a bit about your computer, in terms of CPU model,
motherboard chipset, RAM configuration, OS? If you install and run
"htop"¹, you'll see which block does how much without much complication,
and maybe also significant non-GNU Radio CPU usage (for example, my mail
client and my browser idling use serious amounts of CPU).

Another thing worth trying is to close all software that might be using
CPU (check with htop!) and then run "volk_profile"; this should test a
lot of hand-written implementations for certain math operations, which
might significantly speed up the polyphase clock sync.

Best regards,

Marcus

¹: I'm assuming you're using windows; after starting htop, press F2 for
setup, go into the "Display Options", enable "Show custom thread names",
press Esc

On 03.10.2017 12:54, Vladimir Rytikov via USRP-users wrote:

Hi,

I am trying to run an example from GNU Radio -
examples/digital/packet/uhd_packet_rx and uhd_packet_tx with real USPR
radios connected via an attenuator and a coax cable.

When I enable receiver by clicking on 'On' check box - the whole RX
flow graph freezes.
I think I manually adjusted transmit power and receive gain to be
within reasonable ranges.

By disabling different blocks one at a time I found is that polyphase
clock sync inside packet_rx blocks kills the flow graph. It seems like
the signal is gaited by Correlation Estimator and once it gets the
correct sync word - the signal goes to Polyphase Clock Sync and CPU
dies - it is 100% loaded all the time.

If I run only loop back simulation the whole flow graph seems working
fine. I wonder if noise or not very good signal destroys Polyphase
Clock Sync.

I installed GNU Radio via PyBOMS - 3.7.12git* version. Please help to
find better idea on debugging it.

with a different simple flow graphs Polyphase Clock Sync works fine
even when I feed noise to it and signal later.

Thanks,
Vladimir


USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

Hi Vladimir, synchronization is usually among the most CPU-intense things a receiver does (only, if at all, contested by channel decoding for complex codes). So, the 100% CPU utilization don't sound totally unreasonable, depending on your system. That being said, I don't want to rule out bugs, but for the time being, I'd declare this issue as "unclear, probably insufficient compute power". Can you tell us a bit about your computer, in terms of CPU model, motherboard chipset, RAM configuration, OS? If you install and run "htop"¹, you'll see which block does how much without much complication, and maybe also significant non-GNU Radio CPU usage (for example, my mail client and my browser idling use *serious* amounts of CPU). Another thing worth trying is to close all software that might be using CPU (check with htop!) and then run "volk_profile"; this should test a lot of hand-written implementations for certain math operations, which might significantly speed up the polyphase clock sync. Best regards, Marcus ¹: I'm assuming you're using windows; after starting htop, press F2 for setup, go into the "Display Options", enable "Show custom thread names", press Esc On 03.10.2017 12:54, Vladimir Rytikov via USRP-users wrote: > Hi, > > I am trying to run an example from GNU Radio - > examples/digital/packet/uhd_packet_rx and uhd_packet_tx with real USPR > radios connected via an attenuator and a coax cable. > > When I enable receiver by clicking on 'On' check box - the whole RX > flow graph freezes. > I think I manually adjusted transmit power and receive gain to be > within reasonable ranges. > > By disabling different blocks one at a time I found is that polyphase > clock sync inside packet_rx blocks kills the flow graph. It seems like > the signal is gaited by Correlation Estimator and once it gets the > correct sync word - the signal goes to Polyphase Clock Sync and CPU > dies - it is 100% loaded all the time. > > If I run only loop back simulation the whole flow graph seems working > fine. I wonder if noise or not very good signal destroys Polyphase > Clock Sync. > > I installed GNU Radio via PyBOMS - 3.7.12git* version. Please help to > find better idea on debugging it. > > with a different simple flow graphs Polyphase Clock Sync works fine > even when I feed noise to it and signal later. > > Thanks, > Vladimir > > > _______________________________________________ > USRP-users mailing list > USRP-users@lists.ettus.com > http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
VR
Vladimir Rytikov
Tue, Oct 3, 2017 12:41 PM

Marcus,

cpu: Intel Core i7-7820HQ (Quad Core 2.90GHz, 3.90GHz Turbo, 8MB)
chipset: Intel Mobile CM238
RAM: 32GB (2x16GB) 2400MHz DDR4
OS: ubuntu 16.04 TLS
almost no other software running - only GNU Radio.

htop shows digital/packet/uhd_packet_rx.py takes ~192 % CPU.
header_payload2 - ~100%
pfb_clock_sync2 - ~90%

and the flow graph visually is frozen.
I run volk_profile - it saved some files in .volk directory.
I restarted the test again after it. the same result. htop shows the same
numbers.

the RF signal has correct shape at the freq doman - I am splitting the
signal to a spectrum analyzer- no overdriving the transmitter. the signal
inside the coax cable. 433 MHz freq as in the example. visually looking at
the Time tab - input signal within  -1 to 1. The recv dies at the moment
when I press 'On' check box - it ungates receiver chain and polyphase clock
sync block. transmitter sends bursts every 2 seconds.
the only modifications comparing to the vanila example - Clock/Time sources

  • I changed to Default instead of O/B GPSDO.

-- Vladimir

On Tue, Oct 3, 2017 at 4:57 AM, Marcus Müller via USRP-users <
usrp-users@lists.ettus.com> wrote:

Hi Vladimir,

synchronization is usually among the most CPU-intense things a receiver
does (only, if at all, contested by channel decoding for complex codes).
So, the 100% CPU utilization don't sound totally unreasonable, depending on
your system.

That being said, I don't want to rule out bugs, but for the time being,
I'd declare this issue as "unclear, probably insufficient compute power".

Can you tell us a bit about your computer, in terms of CPU model,
motherboard chipset, RAM configuration, OS? If you install and run "htop"¹,
you'll see which block does how much without much complication, and maybe
also significant non-GNU Radio CPU usage (for example, my mail client and
my browser idling use serious amounts of CPU).

Another thing worth trying is to close all software that might be using
CPU (check with htop!) and then run "volk_profile"; this should test a lot
of hand-written implementations for certain math operations, which might
significantly speed up the polyphase clock sync.

Best regards,

Marcus

¹: I'm assuming you're using windows; after starting htop, press F2 for
setup, go into the "Display Options", enable "Show custom thread names",
press Esc

On 03.10.2017 12:54, Vladimir Rytikov via USRP-users wrote:

Hi,

I am trying to run an example from GNU Radio - examples/digital/packet/uhd_packet_rx
and uhd_packet_tx with real USPR radios connected via an attenuator and a
coax cable.

When I enable receiver by clicking on 'On' check box - the whole RX flow
graph freezes.
I think I manually adjusted transmit power and receive gain to be within
reasonable ranges.

By disabling different blocks one at a time I found is that polyphase
clock sync inside packet_rx blocks kills the flow graph. It seems like the
signal is gaited by Correlation Estimator and once it gets the correct sync
word - the signal goes to Polyphase Clock Sync and CPU dies - it is 100%
loaded all the time.

If I run only loop back simulation the whole flow graph seems working
fine. I wonder if noise or not very good signal destroys Polyphase Clock
Sync.

I installed GNU Radio via PyBOMS - 3.7.12git* version. Please help to find
better idea on debugging it.

with a different simple flow graphs Polyphase Clock Sync works fine even
when I feed noise to it and signal later.

Thanks,
Vladimir


USRP-users mailing listUSRP-users@lists.ettus.comhttp://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com


USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

Marcus, cpu: Intel Core i7-7820HQ (Quad Core 2.90GHz, 3.90GHz Turbo, 8MB) chipset: Intel Mobile CM238 RAM: 32GB (2x16GB) 2400MHz DDR4 OS: ubuntu 16.04 TLS almost no other software running - only GNU Radio. htop shows digital/packet/uhd_packet_rx.py takes ~192 % CPU. header_payload2 - ~100% pfb_clock_sync2 - ~90% and the flow graph visually is frozen. I run volk_profile - it saved some files in .volk directory. I restarted the test again after it. the same result. htop shows the same numbers. the RF signal has correct shape at the freq doman - I am splitting the signal to a spectrum analyzer- no overdriving the transmitter. the signal inside the coax cable. 433 MHz freq as in the example. visually looking at the Time tab - input signal within -1 to 1. The recv dies at the moment when I press 'On' check box - it ungates receiver chain and polyphase clock sync block. transmitter sends bursts every 2 seconds. the only modifications comparing to the vanila example - Clock/Time sources - I changed to Default instead of O/B GPSDO. -- Vladimir On Tue, Oct 3, 2017 at 4:57 AM, Marcus Müller via USRP-users < usrp-users@lists.ettus.com> wrote: > Hi Vladimir, > > synchronization is usually among the most CPU-intense things a receiver > does (only, if at all, contested by channel decoding for complex codes). > So, the 100% CPU utilization don't sound totally unreasonable, depending on > your system. > > That being said, I don't want to rule out bugs, but for the time being, > I'd declare this issue as "unclear, probably insufficient compute power". > > Can you tell us a bit about your computer, in terms of CPU model, > motherboard chipset, RAM configuration, OS? If you install and run "htop"¹, > you'll see which block does how much without much complication, and maybe > also significant non-GNU Radio CPU usage (for example, my mail client and > my browser idling use *serious* amounts of CPU). > > Another thing worth trying is to close all software that might be using > CPU (check with htop!) and then run "volk_profile"; this should test a lot > of hand-written implementations for certain math operations, which might > significantly speed up the polyphase clock sync. > > Best regards, > > Marcus > > > ¹: I'm assuming you're using windows; after starting htop, press F2 for > setup, go into the "Display Options", enable "Show custom thread names", > press Esc > > On 03.10.2017 12:54, Vladimir Rytikov via USRP-users wrote: > > Hi, > > I am trying to run an example from GNU Radio - examples/digital/packet/uhd_packet_rx > and uhd_packet_tx with real USPR radios connected via an attenuator and a > coax cable. > > When I enable receiver by clicking on 'On' check box - the whole RX flow > graph freezes. > I think I manually adjusted transmit power and receive gain to be within > reasonable ranges. > > By disabling different blocks one at a time I found is that polyphase > clock sync inside packet_rx blocks kills the flow graph. It seems like the > signal is gaited by Correlation Estimator and once it gets the correct sync > word - the signal goes to Polyphase Clock Sync and CPU dies - it is 100% > loaded all the time. > > If I run only loop back simulation the whole flow graph seems working > fine. I wonder if noise or not very good signal destroys Polyphase Clock > Sync. > > I installed GNU Radio via PyBOMS - 3.7.12git* version. Please help to find > better idea on debugging it. > > with a different simple flow graphs Polyphase Clock Sync works fine even > when I feed noise to it and signal later. > > Thanks, > Vladimir > > > _______________________________________________ > USRP-users mailing listUSRP-users@lists.ettus.comhttp://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com > > > > _______________________________________________ > USRP-users mailing list > USRP-users@lists.ettus.com > http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com > >
MM
Marcus Müller
Tue, Oct 3, 2017 1:23 PM

That's very interesting, indeed! If I had to infer (sorry, not right now
on a computer where I can test) from the thread name "header_payload2",
I'd say that for some reason that I don't know, the header/payload
demuxer in packet_rx "spins" on something.

If you want to, throw some more runtime analysis at this:

sudo apt-get install linux-tools
sudo sysctl -w kernel.perf_event_paranoid=-1
perf record -ag python /path/to/uhd_packet_rx.py

[let it run for a while, e.g. 30s, end it]

perf report

That should give you a quick insight in which function/code line your
processors where stuck the most time. Maybe that brings us one step forward.

Best regards,
Marcus

On 03.10.2017 14:41, Vladimir Rytikov wrote:

Marcus,

cpu: Intel Core i7-7820HQ (Quad Core 2.90GHz, 3.90GHz Turbo, 8MB)
chipset: Intel Mobile CM238
RAM: 32GB (2x16GB) 2400MHz DDR4
OS: ubuntu 16.04 TLS
almost no other software running - only GNU Radio.

htop shows digital/packet/uhd_packet_rx.py takes ~192 % CPU.
header_payload2 - ~100%
pfb_clock_sync2 - ~90%

and the flow graph visually is frozen.
I run volk_profile - it saved some files in .volk directory.
I restarted the test again after it. the same result. htop shows the
same numbers.

the RF signal has correct shape at the freq doman - I am splitting the
signal to a spectrum analyzer- no overdriving the transmitter. the
signal inside the coax cable. 433 MHz freq as in the example. visually
looking at the Time tab - input signal within  -1 to 1. The recv dies
at the moment when I press 'On' check box - it ungates receiver chain
and polyphase clock sync block. transmitter sends bursts every 2 seconds.
the only modifications comparing to the vanila example - Clock/Time
sources - I changed to Default instead of O/B GPSDO.

-- Vladimir

On Tue, Oct 3, 2017 at 4:57 AM, Marcus Müller via USRP-users
<usrp-users@lists.ettus.com mailto:usrp-users@lists.ettus.com> wrote:

 Hi Vladimir,

 synchronization is usually among the most CPU-intense things a
 receiver does (only, if at all, contested by channel decoding for
 complex codes). So, the 100% CPU utilization don't sound totally
 unreasonable, depending on your system.

 That being said, I don't want to rule out bugs, but for the time
 being, I'd declare this issue as "unclear, probably insufficient
 compute power".

 Can you tell us a bit about your computer, in terms of CPU model,
 motherboard chipset, RAM configuration, OS? If you install and run
 "htop"¹, you'll see which block does how much without much
 complication, and maybe also significant non-GNU Radio CPU usage
 (for example, my mail client and my browser idling use *serious*
 amounts of CPU).

 Another thing worth trying is to close all software that might be
 using CPU (check with htop!) and then run "volk_profile"; this
 should test a lot of hand-written implementations for certain math
 operations, which might significantly speed up the polyphase clock
 sync.

 Best regards,

 Marcus


 ¹: I'm assuming you're using windows; after starting htop, press
 F2 for setup, go into the "Display Options", enable "Show custom
 thread names", press Esc


 On 03.10.2017 12:54, Vladimir Rytikov via USRP-users wrote:
 Hi,

 I am trying to run an example from GNU Radio -
 examples/digital/packet/uhd_packet_rx and uhd_packet_tx with real
 USPR radios connected via an attenuator and a coax cable.

 When I enable receiver by clicking on 'On' check box - the whole
 RX flow graph freezes.
 I think I manually adjusted transmit power and receive gain to be
 within reasonable ranges.

 By disabling different blocks one at a time I found is that
 polyphase clock sync inside packet_rx blocks kills the flow
 graph. It seems like the signal is gaited by Correlation
 Estimator and once it gets the correct sync word - the signal
 goes to Polyphase Clock Sync and CPU dies - it is 100% loaded all
 the time.

 If I run only loop back simulation the whole flow graph seems
 working fine. I wonder if noise or not very good signal destroys
 Polyphase Clock Sync.

 I installed GNU Radio via PyBOMS - 3.7.12git* version. Please
 help to find better idea on debugging it.

 with a different simple flow graphs Polyphase Clock Sync works
 fine even when I feed noise to it and signal later.

 Thanks,
 Vladimir 


 _______________________________________________
 USRP-users mailing list
 USRP-users@lists.ettus.com <mailto:USRP-users@lists.ettus.com>
 http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
 <http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com>
 _______________________________________________
 USRP-users mailing list
 USRP-users@lists.ettus.com <mailto:USRP-users@lists.ettus.com>
 http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
 <http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com>
That's very interesting, indeed! If I had to infer (sorry, not right now on a computer where I can test) from the thread name "header_payload2", I'd say that for some reason that I don't know, the header/payload demuxer in packet_rx "spins" on something. If you want to, throw some more runtime analysis at this: sudo apt-get install linux-tools sudo sysctl -w kernel.perf_event_paranoid=-1 perf record -ag python /path/to/uhd_packet_rx.py [let it run for a while, e.g. 30s, end it] perf report That should give you a quick insight in which function/code line your processors where stuck the most time. Maybe that brings us one step forward. Best regards, Marcus On 03.10.2017 14:41, Vladimir Rytikov wrote: > Marcus, > > cpu: Intel Core i7-7820HQ (Quad Core 2.90GHz, 3.90GHz Turbo, 8MB) > chipset: Intel Mobile CM238 > RAM: 32GB (2x16GB) 2400MHz DDR4 > OS: ubuntu 16.04 TLS > almost no other software running - only GNU Radio. > > htop shows digital/packet/uhd_packet_rx.py takes ~192 % CPU. > header_payload2 - ~100% > pfb_clock_sync2 - ~90% > > and the flow graph visually is frozen. > I run volk_profile - it saved some files in .volk directory. > I restarted the test again after it. the same result. htop shows the > same numbers. > > the RF signal has correct shape at the freq doman - I am splitting the > signal to a spectrum analyzer- no overdriving the transmitter. the > signal inside the coax cable. 433 MHz freq as in the example. visually > looking at the Time tab - input signal within -1 to 1. The recv dies > at the moment when I press 'On' check box - it ungates receiver chain > and polyphase clock sync block. transmitter sends bursts every 2 seconds. > the only modifications comparing to the vanila example - Clock/Time > sources - I changed to Default instead of O/B GPSDO. > > -- Vladimir > > On Tue, Oct 3, 2017 at 4:57 AM, Marcus Müller via USRP-users > <usrp-users@lists.ettus.com <mailto:usrp-users@lists.ettus.com>> wrote: > > Hi Vladimir, > > synchronization is usually among the most CPU-intense things a > receiver does (only, if at all, contested by channel decoding for > complex codes). So, the 100% CPU utilization don't sound totally > unreasonable, depending on your system. > > That being said, I don't want to rule out bugs, but for the time > being, I'd declare this issue as "unclear, probably insufficient > compute power". > > Can you tell us a bit about your computer, in terms of CPU model, > motherboard chipset, RAM configuration, OS? If you install and run > "htop"¹, you'll see which block does how much without much > complication, and maybe also significant non-GNU Radio CPU usage > (for example, my mail client and my browser idling use *serious* > amounts of CPU). > > Another thing worth trying is to close all software that might be > using CPU (check with htop!) and then run "volk_profile"; this > should test a lot of hand-written implementations for certain math > operations, which might significantly speed up the polyphase clock > sync. > > Best regards, > > Marcus > > > ¹: I'm assuming you're using windows; after starting htop, press > F2 for setup, go into the "Display Options", enable "Show custom > thread names", press Esc > > > On 03.10.2017 12:54, Vladimir Rytikov via USRP-users wrote: >> Hi, >> >> I am trying to run an example from GNU Radio - >> examples/digital/packet/uhd_packet_rx and uhd_packet_tx with real >> USPR radios connected via an attenuator and a coax cable. >> >> When I enable receiver by clicking on 'On' check box - the whole >> RX flow graph freezes. >> I think I manually adjusted transmit power and receive gain to be >> within reasonable ranges. >> >> By disabling different blocks one at a time I found is that >> polyphase clock sync inside packet_rx blocks kills the flow >> graph. It seems like the signal is gaited by Correlation >> Estimator and once it gets the correct sync word - the signal >> goes to Polyphase Clock Sync and CPU dies - it is 100% loaded all >> the time. >> >> If I run only loop back simulation the whole flow graph seems >> working fine. I wonder if noise or not very good signal destroys >> Polyphase Clock Sync. >> >> I installed GNU Radio via PyBOMS - 3.7.12git* version. Please >> help to find better idea on debugging it. >> >> with a different simple flow graphs Polyphase Clock Sync works >> fine even when I feed noise to it and signal later. >> >> Thanks, >> Vladimir >> >> >> _______________________________________________ >> USRP-users mailing list >> USRP-users@lists.ettus.com <mailto:USRP-users@lists.ettus.com> >> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >> <http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com> > > > _______________________________________________ > USRP-users mailing list > USRP-users@lists.ettus.com <mailto:USRP-users@lists.ettus.com> > http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com > <http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com> > >
MW
Michael Wentz
Wed, Oct 4, 2017 2:20 PM

Hi,

I ran into a similar problem several months ago - what I found was that the
correlation estimator produced a huge number of false positives (and a
tag for each of them) which caused the clock recovery block to be super
overwhelmed. If I recall correctly, there were also some cases that these
tags would send the clock recovery block into an infinite loop. We rolled
back the correlation estimator to the previous version, which uses absolute
thresholds, as a quick fix.

-Michael

On Tue, Oct 3, 2017 at 9:23 AM, Marcus Müller via USRP-users <
usrp-users@lists.ettus.com> wrote:

That's very interesting, indeed! If I had to infer (sorry, not right now
on a computer where I can test) from the thread name "header_payload2", I'd
say that for some reason that I don't know, the header/payload demuxer in
packet_rx "spins" on something.

If you want to, throw some more runtime analysis at this:

sudo apt-get install linux-tools
sudo sysctl -w kernel.perf_event_paranoid=-1
perf record -ag python /path/to/uhd_packet_rx.py

[let it run for a while, e.g. 30s, end it]

perf report

That should give you a quick insight in which function/code line your
processors where stuck the most time. Maybe that brings us one step forward.

Best regards,
Marcus

On 03.10.2017 14:41, Vladimir Rytikov wrote:

Marcus,

cpu: Intel Core i7-7820HQ (Quad Core 2.90GHz, 3.90GHz Turbo, 8MB)
chipset: Intel Mobile CM238
RAM: 32GB (2x16GB) 2400MHz DDR4
OS: ubuntu 16.04 TLS
almost no other software running - only GNU Radio.

htop shows digital/packet/uhd_packet_rx.py takes ~192 % CPU.
header_payload2 - ~100%
pfb_clock_sync2 - ~90%

and the flow graph visually is frozen.
I run volk_profile - it saved some files in .volk directory.
I restarted the test again after it. the same result. htop shows the same
numbers.

the RF signal has correct shape at the freq doman - I am splitting the
signal to a spectrum analyzer- no overdriving the transmitter. the signal
inside the coax cable. 433 MHz freq as in the example. visually looking at
the Time tab - input signal within  -1 to 1. The recv dies at the moment
when I press 'On' check box - it ungates receiver chain and polyphase clock
sync block. transmitter sends bursts every 2 seconds.
the only modifications comparing to the vanila example - Clock/Time
sources - I changed to Default instead of O/B GPSDO.

-- Vladimir

On Tue, Oct 3, 2017 at 4:57 AM, Marcus Müller via USRP-users <
usrp-users@lists.ettus.com> wrote:

Hi Vladimir,

synchronization is usually among the most CPU-intense things a receiver
does (only, if at all, contested by channel decoding for complex codes).
So, the 100% CPU utilization don't sound totally unreasonable, depending on
your system.

That being said, I don't want to rule out bugs, but for the time being,
I'd declare this issue as "unclear, probably insufficient compute power".

Can you tell us a bit about your computer, in terms of CPU model,
motherboard chipset, RAM configuration, OS? If you install and run "htop"¹,
you'll see which block does how much without much complication, and maybe
also significant non-GNU Radio CPU usage (for example, my mail client and
my browser idling use serious amounts of CPU).

Another thing worth trying is to close all software that might be using
CPU (check with htop!) and then run "volk_profile"; this should test a lot
of hand-written implementations for certain math operations, which might
significantly speed up the polyphase clock sync.

Best regards,

Marcus

¹: I'm assuming you're using windows; after starting htop, press F2 for
setup, go into the "Display Options", enable "Show custom thread names",
press Esc

On 03.10.2017 12:54, Vladimir Rytikov via USRP-users wrote:

Hi,

I am trying to run an example from GNU Radio -
examples/digital/packet/uhd_packet_rx and uhd_packet_tx with real USPR
radios connected via an attenuator and a coax cable.

When I enable receiver by clicking on 'On' check box - the whole RX flow
graph freezes.
I think I manually adjusted transmit power and receive gain to be within
reasonable ranges.

By disabling different blocks one at a time I found is that polyphase
clock sync inside packet_rx blocks kills the flow graph. It seems like the
signal is gaited by Correlation Estimator and once it gets the correct sync
word - the signal goes to Polyphase Clock Sync and CPU dies - it is 100%
loaded all the time.

If I run only loop back simulation the whole flow graph seems working
fine. I wonder if noise or not very good signal destroys Polyphase Clock
Sync.

I installed GNU Radio via PyBOMS - 3.7.12git* version. Please help to
find better idea on debugging it.

with a different simple flow graphs Polyphase Clock Sync works fine even
when I feed noise to it and signal later.

Thanks,
Vladimir


USRP-users mailing listUSRP-users@lists.ettus.comhttp://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com


USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

Hi, I ran into a similar problem several months ago - what I found was that the correlation estimator produced a *huge* number of false positives (and a tag for each of them) which caused the clock recovery block to be super overwhelmed. If I recall correctly, there were also some cases that these tags would send the clock recovery block into an infinite loop. We rolled back the correlation estimator to the previous version, which uses absolute thresholds, as a quick fix. -Michael On Tue, Oct 3, 2017 at 9:23 AM, Marcus Müller via USRP-users < usrp-users@lists.ettus.com> wrote: > That's very interesting, indeed! If I had to infer (sorry, not right now > on a computer where I can test) from the thread name "header_payload2", I'd > say that for some reason that I don't know, the header/payload demuxer in > packet_rx "spins" on something. > > If you want to, throw some more runtime analysis at this: > > sudo apt-get install linux-tools > sudo sysctl -w kernel.perf_event_paranoid=-1 > perf record -ag python /path/to/uhd_packet_rx.py > > [let it run for a while, e.g. 30s, end it] > > perf report > > That should give you a quick insight in which function/code line your > processors where stuck the most time. Maybe that brings us one step forward. > > Best regards, > Marcus > > On 03.10.2017 14:41, Vladimir Rytikov wrote: > > Marcus, > > cpu: Intel Core i7-7820HQ (Quad Core 2.90GHz, 3.90GHz Turbo, 8MB) > chipset: Intel Mobile CM238 > RAM: 32GB (2x16GB) 2400MHz DDR4 > OS: ubuntu 16.04 TLS > almost no other software running - only GNU Radio. > > htop shows digital/packet/uhd_packet_rx.py takes ~192 % CPU. > header_payload2 - ~100% > pfb_clock_sync2 - ~90% > > and the flow graph visually is frozen. > I run volk_profile - it saved some files in .volk directory. > I restarted the test again after it. the same result. htop shows the same > numbers. > > the RF signal has correct shape at the freq doman - I am splitting the > signal to a spectrum analyzer- no overdriving the transmitter. the signal > inside the coax cable. 433 MHz freq as in the example. visually looking at > the Time tab - input signal within -1 to 1. The recv dies at the moment > when I press 'On' check box - it ungates receiver chain and polyphase clock > sync block. transmitter sends bursts every 2 seconds. > the only modifications comparing to the vanila example - Clock/Time > sources - I changed to Default instead of O/B GPSDO. > > -- Vladimir > > On Tue, Oct 3, 2017 at 4:57 AM, Marcus Müller via USRP-users < > usrp-users@lists.ettus.com> wrote: > >> Hi Vladimir, >> >> synchronization is usually among the most CPU-intense things a receiver >> does (only, if at all, contested by channel decoding for complex codes). >> So, the 100% CPU utilization don't sound totally unreasonable, depending on >> your system. >> >> That being said, I don't want to rule out bugs, but for the time being, >> I'd declare this issue as "unclear, probably insufficient compute power". >> >> Can you tell us a bit about your computer, in terms of CPU model, >> motherboard chipset, RAM configuration, OS? If you install and run "htop"¹, >> you'll see which block does how much without much complication, and maybe >> also significant non-GNU Radio CPU usage (for example, my mail client and >> my browser idling use *serious* amounts of CPU). >> >> Another thing worth trying is to close all software that might be using >> CPU (check with htop!) and then run "volk_profile"; this should test a lot >> of hand-written implementations for certain math operations, which might >> significantly speed up the polyphase clock sync. >> >> Best regards, >> >> Marcus >> >> >> ¹: I'm assuming you're using windows; after starting htop, press F2 for >> setup, go into the "Display Options", enable "Show custom thread names", >> press Esc >> >> On 03.10.2017 12:54, Vladimir Rytikov via USRP-users wrote: >> >> Hi, >> >> I am trying to run an example from GNU Radio - >> examples/digital/packet/uhd_packet_rx and uhd_packet_tx with real USPR >> radios connected via an attenuator and a coax cable. >> >> When I enable receiver by clicking on 'On' check box - the whole RX flow >> graph freezes. >> I think I manually adjusted transmit power and receive gain to be within >> reasonable ranges. >> >> By disabling different blocks one at a time I found is that polyphase >> clock sync inside packet_rx blocks kills the flow graph. It seems like the >> signal is gaited by Correlation Estimator and once it gets the correct sync >> word - the signal goes to Polyphase Clock Sync and CPU dies - it is 100% >> loaded all the time. >> >> If I run only loop back simulation the whole flow graph seems working >> fine. I wonder if noise or not very good signal destroys Polyphase Clock >> Sync. >> >> I installed GNU Radio via PyBOMS - 3.7.12git* version. Please help to >> find better idea on debugging it. >> >> with a different simple flow graphs Polyphase Clock Sync works fine even >> when I feed noise to it and signal later. >> >> Thanks, >> Vladimir >> >> >> _______________________________________________ >> USRP-users mailing listUSRP-users@lists.ettus.comhttp://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >> >> >> >> _______________________________________________ >> USRP-users mailing list >> USRP-users@lists.ettus.com >> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >> >> > > > _______________________________________________ > USRP-users mailing list > USRP-users@lists.ettus.com > http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com > >
SK
Steven Knudsen
Wed, Oct 4, 2017 3:16 PM

Hey Michael,

throwing in my 2 cents, I found the same last year and implemented separate logic to monitor RSSI and set thresholds. In my case I was receiving in consecutive slots from different radios, so the RSSI varied a lot. 

You also have to be careful with any loop tracking algorithms for the same reason… not always receiving from the same transmitter can mess things up.

steven

On October 4, 2017 at 08:21:29, Michael Wentz via USRP-users (usrp-users@lists.ettus.com) wrote:

Hi,

I ran into a similar problem several months ago - what I found was that the correlation estimator produced a huge number of false positives (and a tag for each of them) which caused the clock recovery block to be super overwhelmed. If I recall correctly, there were also some cases that these tags would send the clock recovery block into an infinite loop. We rolled back the correlation estimator to the previous version, which uses absolute thresholds, as a quick fix.

-Michael

On Tue, Oct 3, 2017 at 9:23 AM, Marcus Müller via USRP-users usrp-users@lists.ettus.com wrote:
That's very interesting, indeed! If I had to infer (sorry, not right now on a computer where I can test) from the thread name "header_payload2", I'd say that for some reason that I don't know, the header/payload demuxer in packet_rx "spins" on something.

If you want to, throw some more runtime analysis at this: 

sudo apt-get install linux-tools
sudo sysctl -w kernel.perf_event_paranoid=-1
perf record -ag python /path/to/uhd_packet_rx.py

[let it run for a while, e.g. 30s, end it]

perf report

That should give you a quick insight in which function/code line your processors where stuck the most time. Maybe that brings us one step forward.

Best regards,
Marcus

On 03.10.2017 14:41, Vladimir Rytikov wrote:
Marcus,

cpu: Intel Core i7-7820HQ (Quad Core 2.90GHz, 3.90GHz Turbo, 8MB)
chipset: Intel Mobile CM238
RAM: 32GB (2x16GB) 2400MHz DDR4
OS: ubuntu 16.04 TLS
almost no other software running - only GNU Radio.

htop shows digital/packet/uhd_packet_rx.py takes ~192 % CPU.
header_payload2 - ~100%
pfb_clock_sync2 - ~90%

and the flow graph visually is frozen.
I run volk_profile - it saved some files in .volk directory.
I restarted the test again after it. the same result. htop shows the same numbers.

the RF signal has correct shape at the freq doman - I am splitting the signal to a spectrum analyzer- no overdriving the transmitter. the signal inside the coax cable. 433 MHz freq as in the example. visually looking at the Time tab - input signal within  -1 to 1. The recv dies at the moment when I press 'On' check box - it ungates receiver chain and polyphase clock sync block. transmitter sends bursts every 2 seconds.
the only modifications comparing to the vanila example - Clock/Time sources - I changed to Default instead of O/B GPSDO.

-- Vladimir

On Tue, Oct 3, 2017 at 4:57 AM, Marcus Müller via USRP-users usrp-users@lists.ettus.com wrote:
Hi Vladimir,

synchronization is usually among the most CPU-intense things a receiver does (only, if at all, contested by channel decoding for complex codes). So, the 100% CPU utilization don't sound totally unreasonable, depending on your system.

That being said, I don't want to rule out bugs, but for the time being, I'd declare this issue as "unclear, probably insufficient compute power".

Can you tell us a bit about your computer, in terms of CPU model, motherboard chipset, RAM configuration, OS? If you install and run "htop"¹, you'll see which block does how much without much complication, and maybe also significant non-GNU Radio CPU usage (for example, my mail client and my browser idling use serious amounts of CPU).

Another thing worth trying is to close all software that might be using CPU (check with htop!) and then run "volk_profile"; this should test a lot of hand-written implementations for certain math operations, which might significantly speed up the polyphase clock sync.

Best regards,

Marcus

¹: I'm assuming you're using windows; after starting htop, press F2 for setup, go into the "Display Options", enable "Show custom thread names", press Esc

On 03.10.2017 12:54, Vladimir Rytikov via USRP-users wrote:
Hi,

I am trying to run an example from GNU Radio - examples/digital/packet/uhd_packet_rx and uhd_packet_tx with real USPR radios connected via an attenuator and a coax cable.

When I enable receiver by clicking on 'On' check box - the whole RX flow graph freezes.
I think I manually adjusted transmit power and receive gain to be within reasonable ranges.

By disabling different blocks one at a time I found is that polyphase clock sync inside packet_rx blocks kills the flow graph. It seems like the signal is gaited by Correlation Estimator and once it gets the correct sync word - the signal goes to Polyphase Clock Sync and CPU dies - it is 100% loaded all the time.

If I run only loop back simulation the whole flow graph seems working fine. I wonder if noise or not very good signal destroys Polyphase Clock Sync.

I installed GNU Radio via PyBOMS - 3.7.12git* version. Please help to find better idea on debugging it.

with a different simple flow graphs Polyphase Clock Sync works fine even when I feed noise to it and signal later.

Thanks,
Vladimir 


USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com


USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com


USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com


USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

Hey Michael, throwing in my 2 cents, I found the same last year and implemented separate logic to monitor RSSI and set thresholds. In my case I was receiving in consecutive slots from different radios, so the RSSI varied a lot.  You also have to be careful with any loop tracking algorithms for the same reason… not always receiving from the same transmitter can mess things up. steven On October 4, 2017 at 08:21:29, Michael Wentz via USRP-users (usrp-users@lists.ettus.com) wrote: Hi, I ran into a similar problem several months ago - what I found was that the correlation estimator produced a *huge* number of false positives (and a tag for each of them) which caused the clock recovery block to be super overwhelmed. If I recall correctly, there were also some cases that these tags would send the clock recovery block into an infinite loop. We rolled back the correlation estimator to the previous version, which uses absolute thresholds, as a quick fix. -Michael On Tue, Oct 3, 2017 at 9:23 AM, Marcus Müller via USRP-users <usrp-users@lists.ettus.com> wrote: That's very interesting, indeed! If I had to infer (sorry, not right now on a computer where I can test) from the thread name "header_payload2", I'd say that for some reason that I don't know, the header/payload demuxer in packet_rx "spins" on something. If you want to, throw some more runtime analysis at this:  sudo apt-get install linux-tools sudo sysctl -w kernel.perf_event_paranoid=-1 perf record -ag python /path/to/uhd_packet_rx.py [let it run for a while, e.g. 30s, end it] perf report That should give you a quick insight in which function/code line your processors where stuck the most time. Maybe that brings us one step forward. Best regards, Marcus On 03.10.2017 14:41, Vladimir Rytikov wrote: Marcus, cpu: Intel Core i7-7820HQ (Quad Core 2.90GHz, 3.90GHz Turbo, 8MB) chipset: Intel Mobile CM238 RAM: 32GB (2x16GB) 2400MHz DDR4 OS: ubuntu 16.04 TLS almost no other software running - only GNU Radio. htop shows digital/packet/uhd_packet_rx.py takes ~192 % CPU. header_payload2 - ~100% pfb_clock_sync2 - ~90% and the flow graph visually is frozen. I run volk_profile - it saved some files in .volk directory. I restarted the test again after it. the same result. htop shows the same numbers. the RF signal has correct shape at the freq doman - I am splitting the signal to a spectrum analyzer- no overdriving the transmitter. the signal inside the coax cable. 433 MHz freq as in the example. visually looking at the Time tab - input signal within  -1 to 1. The recv dies at the moment when I press 'On' check box - it ungates receiver chain and polyphase clock sync block. transmitter sends bursts every 2 seconds. the only modifications comparing to the vanila example - Clock/Time sources - I changed to Default instead of O/B GPSDO. -- Vladimir On Tue, Oct 3, 2017 at 4:57 AM, Marcus Müller via USRP-users <usrp-users@lists.ettus.com> wrote: Hi Vladimir, synchronization is usually among the most CPU-intense things a receiver does (only, if at all, contested by channel decoding for complex codes). So, the 100% CPU utilization don't sound totally unreasonable, depending on your system. That being said, I don't want to rule out bugs, but for the time being, I'd declare this issue as "unclear, probably insufficient compute power". Can you tell us a bit about your computer, in terms of CPU model, motherboard chipset, RAM configuration, OS? If you install and run "htop"¹, you'll see which block does how much without much complication, and maybe also significant non-GNU Radio CPU usage (for example, my mail client and my browser idling use *serious* amounts of CPU). Another thing worth trying is to close all software that might be using CPU (check with htop!) and then run "volk_profile"; this should test a lot of hand-written implementations for certain math operations, which might significantly speed up the polyphase clock sync. Best regards, Marcus ¹: I'm assuming you're using windows; after starting htop, press F2 for setup, go into the "Display Options", enable "Show custom thread names", press Esc On 03.10.2017 12:54, Vladimir Rytikov via USRP-users wrote: Hi, I am trying to run an example from GNU Radio - examples/digital/packet/uhd_packet_rx and uhd_packet_tx with real USPR radios connected via an attenuator and a coax cable. When I enable receiver by clicking on 'On' check box - the whole RX flow graph freezes. I think I manually adjusted transmit power and receive gain to be within reasonable ranges. By disabling different blocks one at a time I found is that polyphase clock sync inside packet_rx blocks kills the flow graph. It seems like the signal is gaited by Correlation Estimator and once it gets the correct sync word - the signal goes to Polyphase Clock Sync and CPU dies - it is 100% loaded all the time. If I run only loop back simulation the whole flow graph seems working fine. I wonder if noise or not very good signal destroys Polyphase Clock Sync. I installed GNU Radio via PyBOMS - 3.7.12git* version. Please help to find better idea on debugging it. with a different simple flow graphs Polyphase Clock Sync works fine even when I feed noise to it and signal later. Thanks, Vladimir  _______________________________________________ USRP-users mailing list USRP-users@lists.ettus.com http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com _______________________________________________ USRP-users mailing list USRP-users@lists.ettus.com http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com _______________________________________________ USRP-users mailing list USRP-users@lists.ettus.com http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com _______________________________________________ USRP-users mailing list USRP-users@lists.ettus.com http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
VR
Vladimir Rytikov
Sun, Oct 8, 2017 9:19 PM

thanks everyone for help. I looked closer to correlation estimator output
and it is the case. it does generate a lot of tags(false positive tags) in
my case and  overwhelms the rest of the flow graph. I looked at the
loopback flow graph - it has very different samples pattern than the radio
flow graph. It doesn't have any noise in between packets.
In any case I will need to look closer to the issue and debug it. I will
move the discussion to GNU Radio mail list once I get more details on this
issue.

-- Vladimir

On Wed, Oct 4, 2017 at 8:16 AM, Steven Knudsen ee.knud@gmail.com wrote:

Hey Michael,

throwing in my 2 cents, I found the same last year and implemented
separate logic to monitor RSSI and set thresholds. In my case I was
receiving in consecutive slots from different radios, so the RSSI varied a
lot.

You also have to be careful with any loop tracking algorithms for the same
reason… not always receiving from the same transmitter can mess things up.

steven

On October 4, 2017 at 08:21:29, Michael Wentz via USRP-users (
usrp-users@lists.ettus.com) wrote:

Hi,

I ran into a similar problem several months ago - what I found was that
the correlation estimator produced a huge number of false positives (and
a tag for each of them) which caused the clock recovery block to be super
overwhelmed. If I recall correctly, there were also some cases that these
tags would send the clock recovery block into an infinite loop. We rolled
back the correlation estimator to the previous version, which uses absolute
thresholds, as a quick fix.

-Michael

On Tue, Oct 3, 2017 at 9:23 AM, Marcus Müller via USRP-users <
usrp-users@lists.ettus.com> wrote:

That's very interesting, indeed! If I had to infer (sorry, not right now
on a computer where I can test) from the thread name "header_payload2", I'd
say that for some reason that I don't know, the header/payload demuxer in
packet_rx "spins" on something.

If you want to, throw some more runtime analysis at this:

sudo apt-get install linux-tools
sudo sysctl -w kernel.perf_event_paranoid=-1
perf record -ag python /path/to/uhd_packet_rx.py

[let it run for a while, e.g. 30s, end it]

perf report

That should give you a quick insight in which function/code line your
processors where stuck the most time. Maybe that brings us one step forward.

Best regards,
Marcus

On 03.10.2017 14:41, Vladimir Rytikov wrote:

Marcus,

cpu: Intel Core i7-7820HQ (Quad Core 2.90GHz, 3.90GHz Turbo, 8MB)
chipset: Intel Mobile CM238
RAM: 32GB (2x16GB) 2400MHz DDR4
OS: ubuntu 16.04 TLS
almost no other software running - only GNU Radio.

htop shows digital/packet/uhd_packet_rx.py takes ~192 % CPU.
header_payload2 - ~100%
pfb_clock_sync2 - ~90%

and the flow graph visually is frozen.
I run volk_profile - it saved some files in .volk directory.
I restarted the test again after it. the same result. htop shows the same
numbers.

the RF signal has correct shape at the freq doman - I am splitting the
signal to a spectrum analyzer- no overdriving the transmitter. the signal
inside the coax cable. 433 MHz freq as in the example. visually looking at
the Time tab - input signal within  -1 to 1. The recv dies at the moment
when I press 'On' check box - it ungates receiver chain and polyphase clock
sync block. transmitter sends bursts every 2 seconds.
the only modifications comparing to the vanila example - Clock/Time
sources - I changed to Default instead of O/B GPSDO.

-- Vladimir

On Tue, Oct 3, 2017 at 4:57 AM, Marcus Müller via USRP-users <
usrp-users@lists.ettus.com> wrote:

Hi Vladimir,

synchronization is usually among the most CPU-intense things a receiver
does (only, if at all, contested by channel decoding for complex codes).
So, the 100% CPU utilization don't sound totally unreasonable, depending on
your system.

That being said, I don't want to rule out bugs, but for the time being,
I'd declare this issue as "unclear, probably insufficient compute power".

Can you tell us a bit about your computer, in terms of CPU model,
motherboard chipset, RAM configuration, OS? If you install and run "htop"¹,
you'll see which block does how much without much complication, and maybe
also significant non-GNU Radio CPU usage (for example, my mail client and
my browser idling use serious amounts of CPU).

Another thing worth trying is to close all software that might be using
CPU (check with htop!) and then run "volk_profile"; this should test a lot
of hand-written implementations for certain math operations, which might
significantly speed up the polyphase clock sync.

Best regards,

Marcus

¹: I'm assuming you're using windows; after starting htop, press F2 for
setup, go into the "Display Options", enable "Show custom thread names",
press Esc

On 03.10.2017 12:54, Vladimir Rytikov via USRP-users wrote:

Hi,

I am trying to run an example from GNU Radio -
examples/digital/packet/uhd_packet_rx and uhd_packet_tx with real USPR
radios connected via an attenuator and a coax cable.

When I enable receiver by clicking on 'On' check box - the whole RX flow
graph freezes.
I think I manually adjusted transmit power and receive gain to be within
reasonable ranges.

By disabling different blocks one at a time I found is that polyphase
clock sync inside packet_rx blocks kills the flow graph. It seems like the
signal is gaited by Correlation Estimator and once it gets the correct sync
word - the signal goes to Polyphase Clock Sync and CPU dies - it is 100%
loaded all the time.

If I run only loop back simulation the whole flow graph seems working
fine. I wonder if noise or not very good signal destroys Polyphase Clock
Sync.

I installed GNU Radio via PyBOMS - 3.7.12git* version. Please help to
find better idea on debugging it.

with a different simple flow graphs Polyphase Clock Sync works fine even
when I feed noise to it and signal later.

Thanks,
Vladimir


USRP-users mailing listUSRP-users@lists.ettus.comhttp://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com


USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

thanks everyone for help. I looked closer to correlation estimator output and it is the case. it does generate a lot of tags(false positive tags) in my case and overwhelms the rest of the flow graph. I looked at the loopback flow graph - it has very different samples pattern than the radio flow graph. It doesn't have any noise in between packets. In any case I will need to look closer to the issue and debug it. I will move the discussion to GNU Radio mail list once I get more details on this issue. -- Vladimir On Wed, Oct 4, 2017 at 8:16 AM, Steven Knudsen <ee.knud@gmail.com> wrote: > Hey Michael, > > throwing in my 2 cents, I found the same last year and implemented > separate logic to monitor RSSI and set thresholds. In my case I was > receiving in consecutive slots from different radios, so the RSSI varied a > lot. > > You also have to be careful with any loop tracking algorithms for the same > reason… not always receiving from the same transmitter can mess things up. > > steven > > > On October 4, 2017 at 08:21:29, Michael Wentz via USRP-users ( > usrp-users@lists.ettus.com) wrote: > > Hi, > > I ran into a similar problem several months ago - what I found was that > the correlation estimator produced a *huge* number of false positives (and > a tag for each of them) which caused the clock recovery block to be super > overwhelmed. If I recall correctly, there were also some cases that these > tags would send the clock recovery block into an infinite loop. We rolled > back the correlation estimator to the previous version, which uses absolute > thresholds, as a quick fix. > > -Michael > > On Tue, Oct 3, 2017 at 9:23 AM, Marcus Müller via USRP-users < > usrp-users@lists.ettus.com> wrote: > >> That's very interesting, indeed! If I had to infer (sorry, not right now >> on a computer where I can test) from the thread name "header_payload2", I'd >> say that for some reason that I don't know, the header/payload demuxer in >> packet_rx "spins" on something. >> >> If you want to, throw some more runtime analysis at this: >> >> sudo apt-get install linux-tools >> sudo sysctl -w kernel.perf_event_paranoid=-1 >> perf record -ag python /path/to/uhd_packet_rx.py >> >> [let it run for a while, e.g. 30s, end it] >> >> perf report >> >> That should give you a quick insight in which function/code line your >> processors where stuck the most time. Maybe that brings us one step forward. >> >> Best regards, >> Marcus >> >> On 03.10.2017 14:41, Vladimir Rytikov wrote: >> >> Marcus, >> >> cpu: Intel Core i7-7820HQ (Quad Core 2.90GHz, 3.90GHz Turbo, 8MB) >> chipset: Intel Mobile CM238 >> RAM: 32GB (2x16GB) 2400MHz DDR4 >> OS: ubuntu 16.04 TLS >> almost no other software running - only GNU Radio. >> >> htop shows digital/packet/uhd_packet_rx.py takes ~192 % CPU. >> header_payload2 - ~100% >> pfb_clock_sync2 - ~90% >> >> and the flow graph visually is frozen. >> I run volk_profile - it saved some files in .volk directory. >> I restarted the test again after it. the same result. htop shows the same >> numbers. >> >> the RF signal has correct shape at the freq doman - I am splitting the >> signal to a spectrum analyzer- no overdriving the transmitter. the signal >> inside the coax cable. 433 MHz freq as in the example. visually looking at >> the Time tab - input signal within -1 to 1. The recv dies at the moment >> when I press 'On' check box - it ungates receiver chain and polyphase clock >> sync block. transmitter sends bursts every 2 seconds. >> the only modifications comparing to the vanila example - Clock/Time >> sources - I changed to Default instead of O/B GPSDO. >> >> -- Vladimir >> >> On Tue, Oct 3, 2017 at 4:57 AM, Marcus Müller via USRP-users < >> usrp-users@lists.ettus.com> wrote: >> >>> Hi Vladimir, >>> >>> synchronization is usually among the most CPU-intense things a receiver >>> does (only, if at all, contested by channel decoding for complex codes). >>> So, the 100% CPU utilization don't sound totally unreasonable, depending on >>> your system. >>> >>> That being said, I don't want to rule out bugs, but for the time being, >>> I'd declare this issue as "unclear, probably insufficient compute power". >>> >>> Can you tell us a bit about your computer, in terms of CPU model, >>> motherboard chipset, RAM configuration, OS? If you install and run "htop"¹, >>> you'll see which block does how much without much complication, and maybe >>> also significant non-GNU Radio CPU usage (for example, my mail client and >>> my browser idling use *serious* amounts of CPU). >>> >>> Another thing worth trying is to close all software that might be using >>> CPU (check with htop!) and then run "volk_profile"; this should test a lot >>> of hand-written implementations for certain math operations, which might >>> significantly speed up the polyphase clock sync. >>> >>> Best regards, >>> >>> Marcus >>> >>> >>> ¹: I'm assuming you're using windows; after starting htop, press F2 for >>> setup, go into the "Display Options", enable "Show custom thread names", >>> press Esc >>> >>> On 03.10.2017 12:54, Vladimir Rytikov via USRP-users wrote: >>> >>> Hi, >>> >>> I am trying to run an example from GNU Radio - >>> examples/digital/packet/uhd_packet_rx and uhd_packet_tx with real USPR >>> radios connected via an attenuator and a coax cable. >>> >>> When I enable receiver by clicking on 'On' check box - the whole RX flow >>> graph freezes. >>> I think I manually adjusted transmit power and receive gain to be within >>> reasonable ranges. >>> >>> By disabling different blocks one at a time I found is that polyphase >>> clock sync inside packet_rx blocks kills the flow graph. It seems like the >>> signal is gaited by Correlation Estimator and once it gets the correct sync >>> word - the signal goes to Polyphase Clock Sync and CPU dies - it is 100% >>> loaded all the time. >>> >>> If I run only loop back simulation the whole flow graph seems working >>> fine. I wonder if noise or not very good signal destroys Polyphase Clock >>> Sync. >>> >>> I installed GNU Radio via PyBOMS - 3.7.12git* version. Please help to >>> find better idea on debugging it. >>> >>> with a different simple flow graphs Polyphase Clock Sync works fine even >>> when I feed noise to it and signal later. >>> >>> Thanks, >>> Vladimir >>> >>> >>> _______________________________________________ >>> USRP-users mailing listUSRP-users@lists.ettus.comhttp://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >>> >>> >>> >>> _______________________________________________ >>> USRP-users mailing list >>> USRP-users@lists.ettus.com >>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >>> >>> >> >> >> _______________________________________________ >> USRP-users mailing list >> USRP-users@lists.ettus.com >> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >> >> > _______________________________________________ > USRP-users mailing list > USRP-users@lists.ettus.com > http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com > >
SK
Steven Knudsen
Sun, Oct 8, 2017 10:18 PM

FWIW, I found the dynamic threshold set in the corr-est to be far too low. I made some experiments using my custom sync sequence (a variant of m-sequences that gave me a couple of dB extra) and determined that except in degenerate cases, the core peak would be 8 times the average signal  for SNRs down to about 3 dB. So, I believe I set the threshold to be 4 times the average signal. This worked for me because in the TDMA system I had a good idea where the sync sequence might be. It may be okay for a non-TDMA system where you just look at a running history of samples, but I did not consider that scenario.

good luck,

steven

On October 8, 2017 at 15:19:07, Vladimir Rytikov (kk6ygb@gmail.com) wrote:

thanks everyone for help. I looked closer to correlation estimator output and it is the case. it does generate a lot of tags(false positive tags) in my case and  overwhelms the rest of the flow graph. I looked at the loopback flow graph - it has very different samples pattern than the radio flow graph. It doesn't have any noise in between packets.
In any case I will need to look closer to the issue and debug it. I will move the discussion to GNU Radio mail list once I get more details on this issue.

-- Vladimir

On Wed, Oct 4, 2017 at 8:16 AM, Steven Knudsen ee.knud@gmail.com wrote:
Hey Michael,

throwing in my 2 cents, I found the same last year and implemented separate logic to monitor RSSI and set thresholds. In my case I was receiving in consecutive slots from different radios, so the RSSI varied a lot. 

You also have to be careful with any loop tracking algorithms for the same reason… not always receiving from the same transmitter can mess things up.

steven

On October 4, 2017 at 08:21:29, Michael Wentz via USRP-users (usrp-users@lists.ettus.com) wrote:

Hi,

I ran into a similar problem several months ago - what I found was that the correlation estimator produced a huge number of false positives (and a tag for each of them) which caused the clock recovery block to be super overwhelmed. If I recall correctly, there were also some cases that these tags would send the clock recovery block into an infinite loop. We rolled back the correlation estimator to the previous version, which uses absolute thresholds, as a quick fix.

-Michael

On Tue, Oct 3, 2017 at 9:23 AM, Marcus Müller via USRP-users usrp-users@lists.ettus.com wrote:
That's very interesting, indeed! If I had to infer (sorry, not right now on a computer where I can test) from the thread name "header_payload2", I'd say that for some reason that I don't know, the header/payload demuxer in packet_rx "spins" on something.

If you want to, throw some more runtime analysis at this: 

sudo apt-get install linux-tools
sudo sysctl -w kernel.perf_event_paranoid=-1
perf record -ag python /path/to/uhd_packet_rx.py

[let it run for a while, e.g. 30s, end it]

perf report

That should give you a quick insight in which function/code line your processors where stuck the most time. Maybe that brings us one step forward.

Best regards,
Marcus

On 03.10.2017 14:41, Vladimir Rytikov wrote:
Marcus,

cpu: Intel Core i7-7820HQ (Quad Core 2.90GHz, 3.90GHz Turbo, 8MB)
chipset: Intel Mobile CM238
RAM: 32GB (2x16GB) 2400MHz DDR4
OS: ubuntu 16.04 TLS
almost no other software running - only GNU Radio.

htop shows digital/packet/uhd_packet_rx.py takes ~192 % CPU.
header_payload2 - ~100%
pfb_clock_sync2 - ~90%

and the flow graph visually is frozen.
I run volk_profile - it saved some files in .volk directory.
I restarted the test again after it. the same result. htop shows the same numbers.

the RF signal has correct shape at the freq doman - I am splitting the signal to a spectrum analyzer- no overdriving the transmitter. the signal inside the coax cable. 433 MHz freq as in the example. visually looking at the Time tab - input signal within  -1 to 1. The recv dies at the moment when I press 'On' check box - it ungates receiver chain and polyphase clock sync block. transmitter sends bursts every 2 seconds.
the only modifications comparing to the vanila example - Clock/Time sources - I changed to Default instead of O/B GPSDO.

-- Vladimir

On Tue, Oct 3, 2017 at 4:57 AM, Marcus Müller via USRP-users usrp-users@lists.ettus.com wrote:
Hi Vladimir,

synchronization is usually among the most CPU-intense things a receiver does (only, if at all, contested by channel decoding for complex codes). So, the 100% CPU utilization don't sound totally unreasonable, depending on your system.

That being said, I don't want to rule out bugs, but for the time being, I'd declare this issue as "unclear, probably insufficient compute power".

Can you tell us a bit about your computer, in terms of CPU model, motherboard chipset, RAM configuration, OS? If you install and run "htop"¹, you'll see which block does how much without much complication, and maybe also significant non-GNU Radio CPU usage (for example, my mail client and my browser idling use serious amounts of CPU).

Another thing worth trying is to close all software that might be using CPU (check with htop!) and then run "volk_profile"; this should test a lot of hand-written implementations for certain math operations, which might significantly speed up the polyphase clock sync.

Best regards,

Marcus

¹: I'm assuming you're using windows; after starting htop, press F2 for setup, go into the "Display Options", enable "Show custom thread names", press Esc

On 03.10.2017 12:54, Vladimir Rytikov via USRP-users wrote:
Hi,

I am trying to run an example from GNU Radio - examples/digital/packet/uhd_packet_rx and uhd_packet_tx with real USPR radios connected via an attenuator and a coax cable.

When I enable receiver by clicking on 'On' check box - the whole RX flow graph freezes.
I think I manually adjusted transmit power and receive gain to be within reasonable ranges.

By disabling different blocks one at a time I found is that polyphase clock sync inside packet_rx blocks kills the flow graph. It seems like the signal is gaited by Correlation Estimator and once it gets the correct sync word - the signal goes to Polyphase Clock Sync and CPU dies - it is 100% loaded all the time.

If I run only loop back simulation the whole flow graph seems working fine. I wonder if noise or not very good signal destroys Polyphase Clock Sync.

I installed GNU Radio via PyBOMS - 3.7.12git* version. Please help to find better idea on debugging it.

with a different simple flow graphs Polyphase Clock Sync works fine even when I feed noise to it and signal later.

Thanks,
Vladimir 


USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com


USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com


USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com


USRP-users mailing list
USRP-users@lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

FWIW, I found the dynamic threshold set in the corr-est to be far too low. I made some experiments using my custom sync sequence (a variant of m-sequences that gave me a couple of dB extra) and determined that except in degenerate cases, the core peak would be 8 times the average signal  for SNRs down to about 3 dB. So, I believe I set the threshold to be 4 times the average signal. This worked for me because in the TDMA system I had a good idea where the sync sequence might be. It may be okay for a non-TDMA system where you just look at a running history of samples, but I did not consider that scenario. good luck, steven On October 8, 2017 at 15:19:07, Vladimir Rytikov (kk6ygb@gmail.com) wrote: thanks everyone for help. I looked closer to correlation estimator output and it is the case. it does generate a lot of tags(false positive tags) in my case and  overwhelms the rest of the flow graph. I looked at the loopback flow graph - it has very different samples pattern than the radio flow graph. It doesn't have any noise in between packets. In any case I will need to look closer to the issue and debug it. I will move the discussion to GNU Radio mail list once I get more details on this issue. -- Vladimir On Wed, Oct 4, 2017 at 8:16 AM, Steven Knudsen <ee.knud@gmail.com> wrote: Hey Michael, throwing in my 2 cents, I found the same last year and implemented separate logic to monitor RSSI and set thresholds. In my case I was receiving in consecutive slots from different radios, so the RSSI varied a lot.  You also have to be careful with any loop tracking algorithms for the same reason… not always receiving from the same transmitter can mess things up. steven On October 4, 2017 at 08:21:29, Michael Wentz via USRP-users (usrp-users@lists.ettus.com) wrote: Hi, I ran into a similar problem several months ago - what I found was that the correlation estimator produced a *huge* number of false positives (and a tag for each of them) which caused the clock recovery block to be super overwhelmed. If I recall correctly, there were also some cases that these tags would send the clock recovery block into an infinite loop. We rolled back the correlation estimator to the previous version, which uses absolute thresholds, as a quick fix. -Michael On Tue, Oct 3, 2017 at 9:23 AM, Marcus Müller via USRP-users <usrp-users@lists.ettus.com> wrote: That's very interesting, indeed! If I had to infer (sorry, not right now on a computer where I can test) from the thread name "header_payload2", I'd say that for some reason that I don't know, the header/payload demuxer in packet_rx "spins" on something. If you want to, throw some more runtime analysis at this:  sudo apt-get install linux-tools sudo sysctl -w kernel.perf_event_paranoid=-1 perf record -ag python /path/to/uhd_packet_rx.py [let it run for a while, e.g. 30s, end it] perf report That should give you a quick insight in which function/code line your processors where stuck the most time. Maybe that brings us one step forward. Best regards, Marcus On 03.10.2017 14:41, Vladimir Rytikov wrote: Marcus, cpu: Intel Core i7-7820HQ (Quad Core 2.90GHz, 3.90GHz Turbo, 8MB) chipset: Intel Mobile CM238 RAM: 32GB (2x16GB) 2400MHz DDR4 OS: ubuntu 16.04 TLS almost no other software running - only GNU Radio. htop shows digital/packet/uhd_packet_rx.py takes ~192 % CPU. header_payload2 - ~100% pfb_clock_sync2 - ~90% and the flow graph visually is frozen. I run volk_profile - it saved some files in .volk directory. I restarted the test again after it. the same result. htop shows the same numbers. the RF signal has correct shape at the freq doman - I am splitting the signal to a spectrum analyzer- no overdriving the transmitter. the signal inside the coax cable. 433 MHz freq as in the example. visually looking at the Time tab - input signal within  -1 to 1. The recv dies at the moment when I press 'On' check box - it ungates receiver chain and polyphase clock sync block. transmitter sends bursts every 2 seconds. the only modifications comparing to the vanila example - Clock/Time sources - I changed to Default instead of O/B GPSDO. -- Vladimir On Tue, Oct 3, 2017 at 4:57 AM, Marcus Müller via USRP-users <usrp-users@lists.ettus.com> wrote: Hi Vladimir, synchronization is usually among the most CPU-intense things a receiver does (only, if at all, contested by channel decoding for complex codes). So, the 100% CPU utilization don't sound totally unreasonable, depending on your system. That being said, I don't want to rule out bugs, but for the time being, I'd declare this issue as "unclear, probably insufficient compute power". Can you tell us a bit about your computer, in terms of CPU model, motherboard chipset, RAM configuration, OS? If you install and run "htop"¹, you'll see which block does how much without much complication, and maybe also significant non-GNU Radio CPU usage (for example, my mail client and my browser idling use *serious* amounts of CPU). Another thing worth trying is to close all software that might be using CPU (check with htop!) and then run "volk_profile"; this should test a lot of hand-written implementations for certain math operations, which might significantly speed up the polyphase clock sync. Best regards, Marcus ¹: I'm assuming you're using windows; after starting htop, press F2 for setup, go into the "Display Options", enable "Show custom thread names", press Esc On 03.10.2017 12:54, Vladimir Rytikov via USRP-users wrote: Hi, I am trying to run an example from GNU Radio - examples/digital/packet/uhd_packet_rx and uhd_packet_tx with real USPR radios connected via an attenuator and a coax cable. When I enable receiver by clicking on 'On' check box - the whole RX flow graph freezes. I think I manually adjusted transmit power and receive gain to be within reasonable ranges. By disabling different blocks one at a time I found is that polyphase clock sync inside packet_rx blocks kills the flow graph. It seems like the signal is gaited by Correlation Estimator and once it gets the correct sync word - the signal goes to Polyphase Clock Sync and CPU dies - it is 100% loaded all the time. If I run only loop back simulation the whole flow graph seems working fine. I wonder if noise or not very good signal destroys Polyphase Clock Sync. I installed GNU Radio via PyBOMS - 3.7.12git* version. Please help to find better idea on debugging it. with a different simple flow graphs Polyphase Clock Sync works fine even when I feed noise to it and signal later. Thanks, Vladimir  _______________________________________________ USRP-users mailing list USRP-users@lists.ettus.com http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com _______________________________________________ USRP-users mailing list USRP-users@lists.ettus.com http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com _______________________________________________ USRP-users mailing list USRP-users@lists.ettus.com http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com _______________________________________________ USRP-users mailing list USRP-users@lists.ettus.com http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com