The question regarding time synchronization with B200's has been
 asked multiple times on this mailing list [3,4,5,6]. Many of
 these threads end with limited information in terms of how to
 successfully sync B200s and the expected accuracy. In [4], it is
 explained there is a random phase offset due the 3 fractional-N
 PLL's which supply the RF LO's. So I understand that there is
 some calibration which is needed, potentially across every tune.
 What I haven't seen, however, is an expected value for the these
 delays.

 Suppose I have a high quality timing reference driving the 10
 MHz / 1 PPS inputs on the B200s assuming matched length cables,
 etc. I split an input signal and receive it with three B200s
 attached to three separate PCs. I then perform a
 cross-correlation to determinte the TDoA between the signals.
 What is the expected value for these measurements? Is it just a
 phase term, or could the alignment be off by more? There was a
 nice presentation given in FOSDEM 2016 [1] regarding B200
 synchronization, but I did not see the uncorrected offset. [4]
 shows ~ 10uS offset.

 I have performed a similar experiment using the Ettus Octoclock
 Timing Distribution (w/o GPSDO) [7] and a Jackson Labs LC-XO
 GPSDO [8], which I believe is the same used in the Octoclock
 with GPSDO option. The LC-XO is connected to the Octoclock, and
 then equal length (not phased matched) cables are connected to
 the 1 PPS/ 10 MHz outputs to drive 3 B200s on separate
 computers. A separate B200 transmits a BPSK-modulated PN
 sequence with a RRC pulse shape and a bandwidth around 20 MHz.
 The data is recorded (after setting clocks and next_time_pps)
 and then cross-correlation is performed. The time delay between
 receivers is estimated by taking the maximum sample number of
 the cross-correlations. A sub-sample estimate is performed via
 parabolic interpolation. Our results show differences on the
 order of 10's of microseconds, but the actual number is random,
 which agrees with [4]. This leads me to believe that either

 1) The random offset due to the 3 fractional-N PLL's (or other
 RF front-end ambiguities) is more than a phase ambiguity
 2) There is something incorrect in the code for performing the
 time synchronization

 Has anyone successfully synchronized B200's in this way? If so,
 what is the expected value of the random delay (or phase
 offset?) between receivers? Is it on the order of nanoseconds,
 microseconds, etc? I'd be really interested in some hard numbers
 on these things. Of course, it depends on the particulars such
 as bandwidth, etc, but just some idea of what other folks are
 getting. I believe the B200 only samples the 1 PPS at the rate
 of the master clock so I would expect with calibration the B200s
 should be able to be synchronized on the order of maybe 100ns.
 Is this realistic?

 PWG

 [1] https://fosdem.org/2016/schedule/event/distributedsdr/
 [2]
 https://www.ettus.com/content/files/kb/mimo_and_sync_with_usrp_updated.pdf
 [3]
 http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/2015-January/012227.html
 [4]
 http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/2015-May/014025.html
 [5]
 http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/2015-December/017400.html
 [6]
 http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/2015-June/014656.html
 [7] https://www.ettus.com/product/details/OctoClock
 [8] http://www.jackson-labs.com/index.php/products/lc_xo



 _______________________________________________
 USRP-users mailing list
 USRP-users@lists.ettus.com
 http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

On 03/16/2016 05:21 PM, Paul Garver wrote: > 25MSPS with 50 MHz clock rate on UHD 3.8.5. > > PWG Since 3.8.5 there have been *several* fixes that I would categorize as "coherency management". Please try the latest UHD software and matching firmware. > > On 03/16/2016 03:46 PM, Marcus D. Leech wrote: >> On 03/16/2016 02:08 PM, Garver, Paul W wrote: >>> Hi Marcus, >>> >>> I didn't describe this step, but we do as you say. The receivers >>> call set_time_next_pps() to 0, confirm it is set properly, and then >>> schedule a start-of-streaming time to record with metadata. The >>> transmitter transmits a repeating signal on the second (it transmits >>> on (or close to) the PPS edge) which is about 0.5s in duration. For >>> each receiver, we loop through the headers and find the timestamp >>> closest to integer seconds and split the recorded files on those >>> integer seconds. Then the cross-correlation is performed to estimate >>> the TDoA between the sensors. Using this method the delays are still >>> on the order of microseconds. Are you saying this is expected >>> behavior? If so, how does the PPS help? >>> >>> PWG >>> >> What is your sample rate? What version of UHD are you using? >> >> >>> ------------------------------------------------------------------------ >>> *From: *"Marcus D. Leech" <mleech@ripnet.com> >>> *To: *usrp-users@lists.ettus.com >>> *Sent: *Wednesday, March 16, 2016 11:34:09 AM >>> *Subject: *Re: [USRP-users] Time Alignment Error Across Multiple >>> B200s with 10MHz/1PPS >>> >>> On 03/16/2016 09:56 AM, Paul Garver via USRP-users wrote: >>> >>> The question regarding time synchronization with B200's has been >>> asked multiple times on this mailing list [3,4,5,6]. Many of >>> these threads end with limited information in terms of how to >>> successfully sync B200s and the expected accuracy. In [4], it is >>> explained there is a random phase offset due the 3 fractional-N >>> PLL's which supply the RF LO's. So I understand that there is >>> some calibration which is needed, potentially across every tune. >>> What I haven't seen, however, is an expected value for the these >>> delays. >>> >>> Suppose I have a high quality timing reference driving the 10 >>> MHz / 1 PPS inputs on the B200s assuming matched length cables, >>> etc. I split an input signal and receive it with three B200s >>> attached to three separate PCs. I then perform a >>> cross-correlation to determinte the TDoA between the signals. >>> What is the expected value for these measurements? Is it just a >>> phase term, or could the alignment be off by more? There was a >>> nice presentation given in FOSDEM 2016 [1] regarding B200 >>> synchronization, but I did not see the uncorrected offset. [4] >>> shows ~ 10uS offset. >>> >>> I have performed a similar experiment using the Ettus Octoclock >>> Timing Distribution (w/o GPSDO) [7] and a Jackson Labs LC-XO >>> GPSDO [8], which I believe is the same used in the Octoclock >>> with GPSDO option. The LC-XO is connected to the Octoclock, and >>> then equal length (not phased matched) cables are connected to >>> the 1 PPS/ 10 MHz outputs to drive 3 B200s on separate >>> computers. A separate B200 transmits a BPSK-modulated PN >>> sequence with a RRC pulse shape and a bandwidth around 20 MHz. >>> The data is recorded (after setting clocks and next_time_pps) >>> and then cross-correlation is performed. The time delay between >>> receivers is estimated by taking the maximum sample number of >>> the cross-correlations. A sub-sample estimate is performed via >>> parabolic interpolation. Our results show differences on the >>> order of 10's of microseconds, but the actual number is random, >>> which agrees with [4]. This leads me to believe that either >>> >>> 1) The random offset due to the 3 fractional-N PLL's (or other >>> RF front-end ambiguities) is more than a phase ambiguity >>> 2) There is something incorrect in the code for performing the >>> time synchronization >>> >>> Has anyone successfully synchronized B200's in this way? If so, >>> what is the expected value of the random delay (or phase >>> offset?) between receivers? Is it on the order of nanoseconds, >>> microseconds, etc? I'd be really interested in some hard numbers >>> on these things. Of course, it depends on the particulars such >>> as bandwidth, etc, but just some idea of what other folks are >>> getting. I believe the B200 only samples the 1 PPS at the rate >>> of the master clock so I would expect with calibration the B200s >>> should be able to be synchronized on the order of maybe 100ns. >>> Is this realistic? >>> >>> PWG >>> >>> [1] https://fosdem.org/2016/schedule/event/distributedsdr/ >>> [2] >>> https://www.ettus.com/content/files/kb/mimo_and_sync_with_usrp_updated.pdf >>> [3] >>> http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/2015-January/012227.html >>> [4] >>> http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/2015-May/014025.html >>> [5] >>> http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/2015-December/017400.html >>> [6] >>> http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/2015-June/014656.html >>> [7] https://www.ettus.com/product/details/OctoClock >>> [8] http://www.jackson-labs.com/index.php/products/lc_xo >>> >>> >>> >>> _______________________________________________ >>> USRP-users mailing list >>> USRP-users@lists.ettus.com >>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >>> >>> There will be an unknown phase error among all your B200s. This can >>> be anywhere along the phase circle. >>> >>> The timing ambiguity is a different issue. Since you can only have >>> 1 B200 per multi_usrp object, time-alignment is not down automatically. >>> >>> Assuming that you've set the TOD registers on all your participating >>> B200s via set_time_next_pps() or set_time_unknown_pps(), and you've >>> set a common start-of-streaming time, then you'll have to look at >>> timestamps on all your streams, via the metadata, to understand what >>> the timing offsets are. Without those two steps, you can't >>> predict what the timing offset will be, since the N streams will all >>> be started >>> at some random time relative to each other. >>> >>> >>> >> >

For others who may be searching this same topic, I’ll provide an update. We had an error in our metadata recording code which was causing the mean time delay estimation between multiple B200s to be on the order of microseconds (~25uS) at 25 MSPS. Now, we have all three B200s sync’d within a sample on UHD 3.8.5 using a timing reference and 10 MHz/1PPS. The sample mean is around 40nS and the sample variance is extremely low (reads zero). To do better than this, I presume we will need to compensate for the random phase by calibrating on every run / retune. We use our custom recording tool in gr-analysis to record the times, which looks like it time-tags accurately as long as the correct segment size is given (default is OK). https://github.com/garverp/gr-analysis <https://github.com/garverp/gr-analysis> Does this seem reasonable? Does anyone else have time alignment results for the B200 they would care to share? PWG > On Mar 16, 2016, at 9:50 PM, Marcus D. Leech <mleech@ripnet.com> wrote: > > On 03/16/2016 05:21 PM, Paul Garver wrote: >> 25MSPS with 50 MHz clock rate on UHD 3.8.5. >> >> PWG > Since 3.8.5 there have been *several* fixes that I would categorize as "coherency management". Please try the latest UHD software > and matching firmware. > > >> >> On 03/16/2016 03:46 PM, Marcus D. Leech wrote: >>> On 03/16/2016 02:08 PM, Garver, Paul W wrote: >>>> Hi Marcus, >>>> >>>> I didn't describe this step, but we do as you say. The receivers call set_time_next_pps() to 0, confirm it is set properly, and then schedule a start-of-streaming time to record with metadata. The transmitter transmits a repeating signal on the second (it transmits on (or close to) the PPS edge) which is about 0.5s in duration. For each receiver, we loop through the headers and find the timestamp closest to integer seconds and split the recorded files on those integer seconds. Then the cross-correlation is performed to estimate the TDoA between the sensors. Using this method the delays are still on the order of microseconds. Are you saying this is expected behavior? If so, how does the PPS help? >>>> >>>> PWG >>>> >>> What is your sample rate? What version of UHD are you using? >>> >>> >>>> >>>> From: "Marcus D. Leech" <mleech@ripnet.com> <mailto:mleech@ripnet.com> >>>> To: usrp-users@lists.ettus.com <mailto:usrp-users@lists.ettus.com> >>>> Sent: Wednesday, March 16, 2016 11:34:09 AM >>>> Subject: Re: [USRP-users] Time Alignment Error Across Multiple B200s with 10MHz/1PPS >>>> >>>> On 03/16/2016 09:56 AM, Paul Garver via USRP-users wrote: >>>> The question regarding time synchronization with B200's has been asked multiple times on this mailing list [3,4,5,6]. Many of these threads end with limited information in terms of how to successfully sync B200s and the expected accuracy. In [4], it is explained there is a random phase offset due the 3 fractional-N PLL's which supply the RF LO's. So I understand that there is some calibration which is needed, potentially across every tune. What I haven't seen, however, is an expected value for the these delays. >>>> >>>> Suppose I have a high quality timing reference driving the 10 MHz / 1 PPS inputs on the B200s assuming matched length cables, etc. I split an input signal and receive it with three B200s attached to three separate PCs. I then perform a cross-correlation to determinte the TDoA between the signals. What is the expected value for these measurements? Is it just a phase term, or could the alignment be off by more? There was a nice presentation given in FOSDEM 2016 [1] regarding B200 synchronization, but I did not see the uncorrected offset. [4] shows ~ 10uS offset. >>>> >>>> I have performed a similar experiment using the Ettus Octoclock Timing Distribution (w/o GPSDO) [7] and a Jackson Labs LC-XO GPSDO [8], which I believe is the same used in the Octoclock with GPSDO option. The LC-XO is connected to the Octoclock, and then equal length (not phased matched) cables are connected to the 1 PPS/ 10 MHz outputs to drive 3 B200s on separate computers. A separate B200 transmits a BPSK-modulated PN sequence with a RRC pulse shape and a bandwidth around 20 MHz. The data is recorded (after setting clocks and next_time_pps) and then cross-correlation is performed. The time delay between receivers is estimated by taking the maximum sample number of the cross-correlations. A sub-sample estimate is performed via parabolic interpolation. Our results show differences on the order of 10's of microseconds, but the actual number is random, which agrees with [4]. This leads me to believe that either >>>> >>>> 1) The random offset due to the 3 fractional-N PLL's (or other RF front-end ambiguities) is more than a phase ambiguity >>>> 2) There is something incorrect in the code for performing the time synchronization >>>> >>>> Has anyone successfully synchronized B200's in this way? If so, what is the expected value of the random delay (or phase offset?) between receivers? Is it on the order of nanoseconds, microseconds, etc? I'd be really interested in some hard numbers on these things. Of course, it depends on the particulars such as bandwidth, etc, but just some idea of what other folks are getting. I believe the B200 only samples the 1 PPS at the rate of the master clock so I would expect with calibration the B200s should be able to be synchronized on the order of maybe 100ns. Is this realistic? >>>> >>>> PWG >>>> >>>> [1] https://fosdem.org/2016/schedule/event/distributedsdr/ <https://fosdem.org/2016/schedule/event/distributedsdr/> >>>> [2] https://www.ettus.com/content/files/kb/mimo_and_sync_with_usrp_updated.pdf <https://www.ettus.com/content/files/kb/mimo_and_sync_with_usrp_updated.pdf> >>>> [3] http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/2015-January/012227.html <http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/2015-January/012227.html> >>>> [4] http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/2015-May/014025.html <http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/2015-May/014025.html> >>>> [5] http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/2015-December/017400.html <http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/2015-December/017400.html> >>>> [6] http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/2015-June/014656.html <http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/2015-June/014656.html> >>>> [7] https://www.ettus.com/product/details/OctoClock <https://www.ettus.com/product/details/OctoClock> >>>> [8] http://www.jackson-labs.com/index.php/products/lc_xo <http://www.jackson-labs.com/index.php/products/lc_xo> >>>> >>>> >>>> >>>> _______________________________________________ >>>> USRP-users mailing list >>>> USRP-users@lists.ettus.com <mailto:USRP-users@lists.ettus.com> >>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com <http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com> >>>> There will be an unknown phase error among all your B200s. This can be anywhere along the phase circle. >>>> >>>> The timing ambiguity is a different issue. Since you can only have 1 B200 per multi_usrp object, time-alignment is not down automatically. >>>> >>>> Assuming that you've set the TOD registers on all your participating B200s via set_time_next_pps() or set_time_unknown_pps(), and you've >>>> set a common start-of-streaming time, then you'll have to look at timestamps on all your streams, via the metadata, to understand what >>>> the timing offsets are. Without those two steps, you can't predict what the timing offset will be, since the N streams will all be started >>>> at some random time relative to each other. >>>> >>>> >>>> >>> >> > >