syncronization algorithms question
Hello,
I make embedded sensor boards that are connected to a PC, and the sensor
board uses USB-CDC to communicate with the PC. USB-CDC means HID virtual
serial, that works at 921600 bauds, data is transmitted at 64 byte chunks,
and it does not have any of the problems of classical serial ports nor
needs to go over FTDI chip.
I have implemented a NTP like synchronization system where I calculate:
rtt = (t4_point - t1_point).count() - (t3_point - t2_point).count();
offset = ((t2_point - t1_point).count() + (t3_point - t4_point).count()) /
2;
Basically the computer sends a sync request that has t1, and the sensor
receives this at t2, and replies at t3, and the computer receives it at t4.
I transmit sync requests each 4 seconds, calculate the rtt and offset from
the computer side, and the calculated offset to the sensor, so RTC can be
trimmed.
My RTC frequency does not change, however each 64 seconds you can add or
subtract from the RTC counter at the end of the second. Effectively giving
you RTCTrimSet(32768 + OFFSET) - It is good as a frequency changing
oscillator but does the trick.
On the left column Offset, RTT, and RTC trim values of PC to sensor device.
On the right column we have values from RPI5 to another sensor device.
Units are nanoseconds. For the PC: I have them synced +-0.6ms and for the
RPI5 we have +-0.1ms. The horizontal axis, each point is measured each 4
seconds, so this is about 10 hours of data.
[image: Screenshot from 2025-05-29 19-19-50.png]
Here are my questions: Given that I calculate the offset with this method,
and I adjust the RTC by adding or removing subseconds each 64 seconds, what
kind of algorithm should I use to filter the RTC offset? or should I
directly adjust with the latest measured offset? Or even should I put a PI
algorithm? I have experimented with using a moving average filter vs direct
last measured RTC_OFFSET - and as expected even RTC_OFFSET method responds
faster avg(RTC_OFFSET) is more stable. Above graphs are both
avg(RTC_OFFSET). I am thinking that since RTC_OFFSET points to a difference
in time, theoretically it should be best to instruct the RTC to lead/lag by
that difference. But doing so causes noise. What other algorithms can be
used for these purposes? Maybe something that will calculate the trim not
based on past data, but also prediction of the next cycle of 64 secs?
What are your thoughts on this? What are your thoughts on the graph.
Any ideas / help / recommendations greatly appreciated.
Best Regards,
Can
It depends on how the stability of your oscillator compares to the accuracy
of the offset measurement. If the expected variance of your clock over
those 64 seconds is of the same order of magnitude as the noise in the
offset measurement, or worse, then you might as well just use the "hard
sync" approach of applying the measured offset immediately. If the
oscillator is stable over longer time periods then a PLL makes sense. A
second-order PLL is equivalent to a PI loop; the "I" term corrects for the
average frequency offset while the "P" time drives the phase offset to
zero. The more stable your oscillator, the smaller you can make the P and I
constants (over here we talk about using "longer time constants"); the
disciplined system will be dominated by the performance of the oscillator
over time periods shorter than the PLL time constants, and dominated by the
noise in your offset measurements over longer periods of time — which is
why it's not worth it to do anything fancy if your clock isn't stable
enough over the minimum adjustment interval.
I would guess that you can characterize the noise in the offset measurement
(if you haven't already) by taking batches of offset measurements much
faster than usual so that the clock doesn't have time to wander anywhere;
then if you take a long series of offset measurements with the correction
turned off you can get a TDEV plot of the oscillator performance plus
measurement noise; and by risking a few assumptions (independent variances,
and that the measurement error is white phase noise) you can figure
TDEV^2[clock] = TDEV^2[total] - TDEV^2[meas.err.] to separate the plots and
figure out where the crossover should be.
On Fri, May 30, 2025 at 8:43 AM Can Altineller via time-nuts <
time-nuts@lists.febo.com> wrote:
Hello,
I make embedded sensor boards that are connected to a PC, and the sensor
board uses USB-CDC to communicate with the PC. USB-CDC means HID virtual
serial, that works at 921600 bauds, data is transmitted at 64 byte chunks,
and it does not have any of the problems of classical serial ports nor
needs to go over FTDI chip.
I have implemented a NTP like synchronization system where I calculate:
rtt = (t4_point - t1_point).count() - (t3_point - t2_point).count();
offset = ((t2_point - t1_point).count() + (t3_point - t4_point).count()) /
2;
Basically the computer sends a sync request that has t1, and the sensor
receives this at t2, and replies at t3, and the computer receives it at t4.
I transmit sync requests each 4 seconds, calculate the rtt and offset from
the computer side, and the calculated offset to the sensor, so RTC can be
trimmed.
My RTC frequency does not change, however each 64 seconds you can add or
subtract from the RTC counter at the end of the second. Effectively giving
you RTCTrimSet(32768 + OFFSET) - It is good as a frequency changing
oscillator but does the trick.
On the left column Offset, RTT, and RTC trim values of PC to sensor device.
On the right column we have values from RPI5 to another sensor device.
Units are nanoseconds. For the PC: I have them synced +-0.6ms and for the
RPI5 we have +-0.1ms. The horizontal axis, each point is measured each 4
seconds, so this is about 10 hours of data.
[image: Screenshot from 2025-05-29 19-19-50.png]
Here are my questions: Given that I calculate the offset with this method,
and I adjust the RTC by adding or removing subseconds each 64 seconds, what
kind of algorithm should I use to filter the RTC offset? or should I
directly adjust with the latest measured offset? Or even should I put a PI
algorithm? I have experimented with using a moving average filter vs direct
last measured RTC_OFFSET - and as expected even RTC_OFFSET method responds
faster avg(RTC_OFFSET) is more stable. Above graphs are both
avg(RTC_OFFSET). I am thinking that since RTC_OFFSET points to a difference
in time, theoretically it should be best to instruct the RTC to lead/lag by
that difference. But doing so causes noise. What other algorithms can be
used for these purposes? Maybe something that will calculate the trim not
based on past data, but also prediction of the next cycle of 64 secs?
What are your thoughts on this? What are your thoughts on the graph.
Any ideas / help / recommendations greatly appreciated.
Best Regards,
Can
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
Hello,
Thank you for your explanatory answer. I made a PI, and also custom
plotting software that processes the logs, makes graphs with gnuplot, and a
web page, and a small python -m http.server to host the pages.
Well I have synchronization with in +- 3 * 1/32768 most of the time. I am
not sure if it can be made any better. The cheap crystal that I am using
skips like couple of steps each 64 seconds depending on temperature, etc.
One problem is that the system is not resistant to external changes, for
example then the ntp chrony daemon adjusts the hosts clock.
I am sending two giant png's one is the plot of my system.
This image is plot of offset, rtt, t1,t2,t3,t4 and t4-t1 vs t3-t2 - and
applied RTC trim each 64 seconds.
[image: ntp-local.png]
This graph is the chrony ntp logs, (open source software from github)
[image: ntp-echo.png]
You can see for some reason ntp was cut between 6 and 7 hrs, and my system
logs a great offset that it fails to compensate for.
I am thinking of replacing the crystal with a mems oscillator that has
0.1ppm.
Best Regards,
Can
On Sat, May 31, 2025 at 9:29 AM Andrew Rodland via time-nuts <
time-nuts@lists.febo.com> wrote:
It depends on how the stability of your oscillator compares to the accuracy
of the offset measurement. If the expected variance of your clock over
those 64 seconds is of the same order of magnitude as the noise in the
offset measurement, or worse, then you might as well just use the "hard
sync" approach of applying the measured offset immediately. If the
oscillator is stable over longer time periods then a PLL makes sense. A
second-order PLL is equivalent to a PI loop; the "I" term corrects for the
average frequency offset while the "P" time drives the phase offset to
zero. The more stable your oscillator, the smaller you can make the P and I
constants (over here we talk about using "longer time constants"); the
disciplined system will be dominated by the performance of the oscillator
over time periods shorter than the PLL time constants, and dominated by the
noise in your offset measurements over longer periods of time — which is
why it's not worth it to do anything fancy if your clock isn't stable
enough over the minimum adjustment interval.
I would guess that you can characterize the noise in the offset measurement
(if you haven't already) by taking batches of offset measurements much
faster than usual so that the clock doesn't have time to wander anywhere;
then if you take a long series of offset measurements with the correction
turned off you can get a TDEV plot of the oscillator performance plus
measurement noise; and by risking a few assumptions (independent variances,
and that the measurement error is white phase noise) you can figure
TDEV^2[clock] = TDEV^2[total] - TDEV^2[meas.err.] to separate the plots and
figure out where the crossover should be.
On Fri, May 30, 2025 at 8:43 AM Can Altineller via time-nuts <
time-nuts@lists.febo.com> wrote:
Hello,
I make embedded sensor boards that are connected to a PC, and the sensor
board uses USB-CDC to communicate with the PC. USB-CDC means HID virtual
serial, that works at 921600 bauds, data is transmitted at 64 byte
chunks,
and it does not have any of the problems of classical serial ports nor
needs to go over FTDI chip.
I have implemented a NTP like synchronization system where I calculate:
rtt = (t4_point - t1_point).count() - (t3_point - t2_point).count();
offset = ((t2_point - t1_point).count() + (t3_point - t4_point).count())
/
2;
Basically the computer sends a sync request that has t1, and the sensor
receives this at t2, and replies at t3, and the computer receives it at
t4.
I transmit sync requests each 4 seconds, calculate the rtt and offset
from
the computer side, and the calculated offset to the sensor, so RTC can be
trimmed.
My RTC frequency does not change, however each 64 seconds you can add or
subtract from the RTC counter at the end of the second. Effectively
giving
you RTCTrimSet(32768 + OFFSET) - It is good as a frequency changing
oscillator but does the trick.
On the left column Offset, RTT, and RTC trim values of PC to sensor
device.
On the right column we have values from RPI5 to another sensor device.
Units are nanoseconds. For the PC: I have them synced +-0.6ms and for the
RPI5 we have +-0.1ms. The horizontal axis, each point is measured each 4
seconds, so this is about 10 hours of data.
[image: Screenshot from 2025-05-29 19-19-50.png]
Here are my questions: Given that I calculate the offset with this
method,
and I adjust the RTC by adding or removing subseconds each 64 seconds,
what
kind of algorithm should I use to filter the RTC offset? or should I
directly adjust with the latest measured offset? Or even should I put a
PI
algorithm? I have experimented with using a moving average filter vs
direct
last measured RTC_OFFSET - and as expected even RTC_OFFSET method
responds
faster avg(RTC_OFFSET) is more stable. Above graphs are both
avg(RTC_OFFSET). I am thinking that since RTC_OFFSET points to a
difference
in time, theoretically it should be best to instruct the RTC to lead/lag
by
that difference. But doing so causes noise. What other algorithms can be
used for these purposes? Maybe something that will calculate the trim not
based on past data, but also prediction of the next cycle of 64 secs?
What are your thoughts on this? What are your thoughts on the graph.
Any ideas / help / recommendations greatly appreciated.
Best Regards,
Can
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe send an email to time-nuts-leave@lists.febo.com
