usrp-users@lists.ettus.com

Discussion and technical support related to USRP, UHD, RFNoC

View all threads

Controlling and Monitoring Latency on USRP X300/X310

JM
John Malsbury
Tue, Jan 20, 2015 11:40 PM

In a dated application with the N200 and UHD 3.5.2, I controlled and
measured latency by:

  1. Modifying UHD to propagate flow control message to the user layer
  2. Modifying gr-uhd and adding a custom block at the source end of the
    stream that would throttle the data flow.
  3. Setting the sample size written to the USRP.

The end result was that we could estimate the buffer state of the USRP N200
and and manage control latency.  I've made similar modifications in the
latest master.  This works fine for the N200 case, but breaks the X300
case.  While the indicators suggest we are meeting our latency requirement
at the start of operation, there is an underflow after 10-15 seconds of
operation.

Is there a better way to manage latency in "stock" UHD?  We're looking to
assure that bits at the input of the flowgraph always make it out of the
USRP DACs in <100 ms.  This is even true at relatively low sample rates.

-John

In a dated application with the N200 and UHD 3.5.2, I controlled and measured latency by: 1. Modifying UHD to propagate flow control message to the user layer 2. Modifying gr-uhd and adding a custom block at the source end of the stream that would throttle the data flow. 3. Setting the sample size written to the USRP. The end result was that we could estimate the buffer state of the USRP N200 and and manage control latency. I've made similar modifications in the latest master. This works fine for the N200 case, but breaks the X300 case. While the indicators suggest we are meeting our latency requirement at the start of operation, there is an underflow after 10-15 seconds of operation. Is there a better way to manage latency in "stock" UHD? We're looking to assure that bits at the input of the flowgraph always make it out of the USRP DACs in <100 ms. This is even true at relatively low sample rates. -John
SM
Sylvain Munaut
Tue, Jan 20, 2015 11:55 PM

Hi,

Is there a better way to manage latency in "stock" UHD?  We're looking to
assure that bits at the input of the flowgraph always make it out of the
USRP DACs in <100 ms.  This is even true at relatively low sample rates.

If you use a both a RX and a TX block in the same graph, you should be
able to use the timestamps / sample counter from the RX flow to 'know'
which time the radio currently has (even if you just drop the actual
RX data) and so you should be able to modulate your TX flow rate that
way to make sure you're not too much in advance.

Cheers,

Sylvain

Hi, > Is there a better way to manage latency in "stock" UHD? We're looking to > assure that bits at the input of the flowgraph always make it out of the > USRP DACs in <100 ms. This is even true at relatively low sample rates. If you use a both a RX and a TX block in the same graph, you should be able to use the timestamps / sample counter from the RX flow to 'know' which time the radio currently has (even if you just drop the actual RX data) and so you should be able to modulate your TX flow rate that way to make sure you're not too much in advance. Cheers, Sylvain
JM
John Malsbury
Wed, Jan 21, 2015 12:16 AM

I've used this approach before with less success.

I'm hoping to do this without changing much on the GR side except for maybe
device args.

-John

On Tue, Jan 20, 2015 at 3:55 PM, Sylvain Munaut 246tnt@gmail.com wrote:

Hi,

Is there a better way to manage latency in "stock" UHD?  We're looking to
assure that bits at the input of the flowgraph always make it out of the
USRP DACs in <100 ms.  This is even true at relatively low sample rates.

If you use a both a RX and a TX block in the same graph, you should be
able to use the timestamps / sample counter from the RX flow to 'know'
which time the radio currently has (even if you just drop the actual
RX data) and so you should be able to modulate your TX flow rate that
way to make sure you're not too much in advance.

Cheers,

Sylvain
I've used this approach before with less success. I'm hoping to do this without changing much on the GR side except for maybe device args. -John On Tue, Jan 20, 2015 at 3:55 PM, Sylvain Munaut <246tnt@gmail.com> wrote: > Hi, > > > Is there a better way to manage latency in "stock" UHD? We're looking to > > assure that bits at the input of the flowgraph always make it out of the > > USRP DACs in <100 ms. This is even true at relatively low sample rates. > > If you use a both a RX and a TX block in the same graph, you should be > able to use the timestamps / sample counter from the RX flow to 'know' > which time the radio currently has (even if you just drop the actual > RX data) and so you should be able to modulate your TX flow rate that > way to make sure you're not too much in advance. > > Cheers, > > Sylvain >
JM
John Malsbury
Wed, Jan 21, 2015 2:20 AM

Update.  The problems were caused in part by assumptions that were carried
over from the N200 implementation.  Of course, the assumptions were not
valid since the CHDR/VITA frames are a bit different.

Still, it seems like there might be a more generic way to solve this in the
driver...

-John

On Tue, Jan 20, 2015 at 4:16 PM, John Malsbury <jmalsbury.personal@gmail.com

wrote:

I've used this approach before with less success.

I'm hoping to do this without changing much on the GR side except for
maybe device args.

-John

On Tue, Jan 20, 2015 at 3:55 PM, Sylvain Munaut 246tnt@gmail.com wrote:

Hi,

Is there a better way to manage latency in "stock" UHD?  We're looking

to

assure that bits at the input of the flowgraph always make it out of the
USRP DACs in <100 ms.  This is even true at relatively low sample rates.

If you use a both a RX and a TX block in the same graph, you should be
able to use the timestamps / sample counter from the RX flow to 'know'
which time the radio currently has (even if you just drop the actual
RX data) and so you should be able to modulate your TX flow rate that
way to make sure you're not too much in advance.

Cheers,

Sylvain
Update. The problems were caused in part by assumptions that were carried over from the N200 implementation. Of course, the assumptions were not valid since the CHDR/VITA frames are a bit different. Still, it seems like there might be a more generic way to solve this in the driver... -John On Tue, Jan 20, 2015 at 4:16 PM, John Malsbury <jmalsbury.personal@gmail.com > wrote: > I've used this approach before with less success. > > I'm hoping to do this without changing much on the GR side except for > maybe device args. > > -John > > On Tue, Jan 20, 2015 at 3:55 PM, Sylvain Munaut <246tnt@gmail.com> wrote: > >> Hi, >> >> > Is there a better way to manage latency in "stock" UHD? We're looking >> to >> > assure that bits at the input of the flowgraph always make it out of the >> > USRP DACs in <100 ms. This is even true at relatively low sample rates. >> >> If you use a both a RX and a TX block in the same graph, you should be >> able to use the timestamps / sample counter from the RX flow to 'know' >> which time the radio currently has (even if you just drop the actual >> RX data) and so you should be able to modulate your TX flow rate that >> way to make sure you're not too much in advance. >> >> Cheers, >> >> Sylvain >> > >
JM
John Malsbury
Thu, Jan 22, 2015 1:01 AM

Another update, thanks to Ashish's responsiveness....

On the X300, the send_buff_size is used to determine how many outstanding
credits the flow control will allow.  By setting send_buff_size to a small
number thats > send_frame_size * num_send_frames, we can minimize the
buffered samples and therefor, bits-to-RF latency.

To control total buffering between the data source in the flowgraph and the
USRP, I will modify gr-uhd to count how many samples have passed to UHD.
Will use this figure and the relationship between USRP samp_rate and bit
rate to determine how many outstanding bits are in the flowgraph
(everything from output buffer of data src through the input buffer of the
UHD sink).

Ethernet latency is not really captured with this approach, but it is small
enough for me to ignore in this case..

Open to other ideas if someone has found a better way...

-John

Related code:

x300_io_impl.cpp
/*!

  • If the return value of this function is F, the last tx'd packet
  • has index N and the last ack'd packet has index M, the amount of
  • FC credit we have is C = F + M - N (i.e. we can send C more packets
  • before getting another ack).
    */
    static size_t get_tx_flow_control_window(size_t frame_size, const
    device_addr_t& tx_args)
    {
    double hw_buff_size = tx_args.cast<double>("send_buff_size",
    X300_TX_HW_BUFF_SIZE);
    std::cout << "send_buff_size: " << hw_buff_size << "\n";
    size_t window_in_pkts = (static_cast<size_t>(hw_buff_size) /
    frame_size);
    if (window_in_pkts == 0) {
    throw uhd::value_error("send_buff_size must be larger than the
    send_frame_size.");
    }
    return window_in_pkts;
    }

On Tue, Jan 20, 2015 at 6:20 PM, John Malsbury <jmalsbury.personal@gmail.com

wrote:

Update.  The problems were caused in part by assumptions that were carried
over from the N200 implementation.  Of course, the assumptions were not
valid since the CHDR/VITA frames are a bit different.

Still, it seems like there might be a more generic way to solve this in
the driver...

-John

On Tue, Jan 20, 2015 at 4:16 PM, John Malsbury <
jmalsbury.personal@gmail.com> wrote:

I've used this approach before with less success.

I'm hoping to do this without changing much on the GR side except for
maybe device args.

-John

On Tue, Jan 20, 2015 at 3:55 PM, Sylvain Munaut 246tnt@gmail.com wrote:

Hi,

Is there a better way to manage latency in "stock" UHD?  We're looking

to

assure that bits at the input of the flowgraph always make it out of

the

USRP DACs in <100 ms.  This is even true at relatively low sample

rates.

If you use a both a RX and a TX block in the same graph, you should be
able to use the timestamps / sample counter from the RX flow to 'know'
which time the radio currently has (even if you just drop the actual
RX data) and so you should be able to modulate your TX flow rate that
way to make sure you're not too much in advance.

Cheers,

Sylvain
Another update, thanks to Ashish's responsiveness.... On the X300, the send_buff_size is used to determine how many outstanding credits the flow control will allow. By setting send_buff_size to a small number thats > send_frame_size * num_send_frames, we can minimize the buffered samples and therefor, bits-to-RF latency. To control total buffering between the data source in the flowgraph and the USRP, I will modify gr-uhd to count how many samples have passed to UHD. Will use this figure and the relationship between USRP samp_rate and bit rate to determine how many outstanding bits are in the flowgraph (everything from output buffer of data src through the input buffer of the UHD sink). Ethernet latency is not really captured with this approach, but it is small enough for me to ignore in this case.. Open to other ideas if someone has found a better way... -John Related code: x300_io_impl.cpp /*! * If the return value of this function is F, the last tx'd packet * has index N and the last ack'd packet has index M, the amount of * FC credit we have is C = F + M - N (i.e. we can send C more packets * before getting another ack). */ static size_t get_tx_flow_control_window(size_t frame_size, const device_addr_t& tx_args) { double hw_buff_size = tx_args.cast<double>("send_buff_size", X300_TX_HW_BUFF_SIZE); std::cout << "send_buff_size: " << hw_buff_size << "\n"; size_t window_in_pkts = (static_cast<size_t>(hw_buff_size) / frame_size); if (window_in_pkts == 0) { throw uhd::value_error("send_buff_size must be larger than the send_frame_size."); } return window_in_pkts; } On Tue, Jan 20, 2015 at 6:20 PM, John Malsbury <jmalsbury.personal@gmail.com > wrote: > Update. The problems were caused in part by assumptions that were carried > over from the N200 implementation. Of course, the assumptions were not > valid since the CHDR/VITA frames are a bit different. > > Still, it seems like there might be a more generic way to solve this in > the driver... > > -John > > On Tue, Jan 20, 2015 at 4:16 PM, John Malsbury < > jmalsbury.personal@gmail.com> wrote: > >> I've used this approach before with less success. >> >> I'm hoping to do this without changing much on the GR side except for >> maybe device args. >> >> -John >> >> On Tue, Jan 20, 2015 at 3:55 PM, Sylvain Munaut <246tnt@gmail.com> wrote: >> >>> Hi, >>> >>> > Is there a better way to manage latency in "stock" UHD? We're looking >>> to >>> > assure that bits at the input of the flowgraph always make it out of >>> the >>> > USRP DACs in <100 ms. This is even true at relatively low sample >>> rates. >>> >>> If you use a both a RX and a TX block in the same graph, you should be >>> able to use the timestamps / sample counter from the RX flow to 'know' >>> which time the radio currently has (even if you just drop the actual >>> RX data) and so you should be able to modulate your TX flow rate that >>> way to make sure you're not too much in advance. >>> >>> Cheers, >>> >>> Sylvain >>> >> >> >