Relatively good resolution. Relative to what? :)

Deviation requires you have something to measure against. A “source of truth”. So what are you measuring deviation from?

From a system administration perspective , I want all my systems to be consistent. I’ll say that

right == consistently wrong

For many purposes. The inconsistency is what causes all manner of hair pulling for me as a system administrator across any meaningful fleet size. Very quickly you run into SSL and LDAP authentication issues.

I do frequently run date/time checks to generate the cloud of data points , and I do log NTPD client and server via snmp using librenms . I’m about to dive into netdata graphing as well.

I’m using a raspberry pi with gps hat for my master time source. Shortly I’ll be having a total of three systems (two using the same hat , one using the adafruit hat and being a pi2). I’ve got some interest in multiple way comparison and will follow this thread shortly. I’ll blog my setup and post a link.

Hi

One simple answer is that it’s not always 1:1 comparisons. The “three corner hat”
approach is indeed used to compare three devices at one time. There are some
issues with doing that. There are lots of papers on where the limits come from
and what you need to do ( = pick fairly similar clocks) to eliminate most of them.

Bob

Not all. But, yes, often. UTC itself is a wonderful example of making mutual measurements of several hundred atomic clocks and establishing a superior "paper" clock out of them collectively.

But most people have only one counter (one internal or external timebase reference) and one clock to be measured. So the measurements are one-to-one. If you have more references or more clocks, you're welcome to combine 2, or 3, or as many as you want. It gets complicated but in some cases this complexity is justified.

Right, in some cases this works. If you had 100 cesium clocks and combined them all, then the result would be a paper clock that's about sqrt(100) or 10x better than any one of them.

But in other cases, the improvement is not so good. For example, if you combined 100 quartz wristwatches you may find that the improvement is only a tiny bit better. The reason: it's likely that most of your watches all gain or lose time in proportion to temperature, a common mode effect. So your cloud result is more likely to be a 10x better paper thermometer than a 10x better paper clock.

It's important to look at the statistics to make sure that negatives and positives do in fact cancel. Depending on the type of clock noise and over a finite time span, this assumption is not at all guaranteed.

Also note, especially for short-term, that your success depends on how precisely can you compare the clocks and how long it takes to make an accurate comparison. If you measure too quickly or not accurately enough then the measurement noise will ruin your paper clock.

If you have some specific use case in mind, let us know. It might be easier to talk real numbers and real clocks than to discuss this topic in generalities.

/tvb

Thanks!

No, I don't really have a specific use case.

And your reply is very helpful in my continuing education!

I had wondered how to do something like this.
I always got stuck on how to know which signal came from which clock.
But it came up again in my mind when I was watching a waterfall display
of the FT-8 herd on 80 meters the other day.   FT-8 is an amateur
digital mode which has synchronized transmit times.  So I had
a visual solution of how to resolve times for multiple signals.

Is the statistical benefit of the paper-clock always sqrt(number of clocks)
if the common error modes are accounted for?

What kind of search should I do to find multi-way measurement strategies,
is that the correct terminology?

On 12/26/18 11:31 AM, Tom Van Baak wrote:

On Wed 2018-12-26T10:30:24-0600 Chris Howard hath writ:

Doing this was the reason for the creation of the Bureau International
de l'Heure (BIH) a century ago.  The initial announcement of their
work invited observatories around the world to participate via
correspondence sending the received times of radio time signals.
http://adsbit.harvard.edu/full/1922BuBIH...1....1.

A few years later they presented to the 1928 General Assembly of the
IAU a complete history of timekeeping and an inventory of their
equipment including the clocks at l'Observatoire de Paris which were
located down in the catacombs to maintain stable conditions
http://adsbit.harvard.edu/full/1929BuBIH...3..255.

The progression of issues of Bulletin Horaire shows the development of
technologies and techniques for intercomparing clocks from the age of
pendulum clocks with constant pressure cases into the age of atomic
chronometers.  At the retirement of two long-time staffers they
published plots of the improvement of timekeeping from 1922 to 1964.
https://www.ucolick.org/~sla/leapsecs/annastoyko.html

--
Steve Allen                    sla@ucolick.org              WGS-84 (GPS)
UCO/Lick Observatory--ISB 260  Natural Sciences II, Room 165  Lat  +36.99855
1156 High Street              Voice: +1 831 459 3046        Lng -122.06015
Santa Cruz, CA 95064          http://www.ucolick.org/~sla/  Hgt +250 m

Hi

There also are a lot of papers going back a ways by Jim Barnes and
David Alan (sometimes together and sometimes separately)  related
to multiple clocks driving a single “estimate” of what time it actually is.

Bob

Charles Wyble wrote:

I'd say three doesn't really get you good enough visibility.  It depends
somewhat on how good your GPS reception is and how stable the
environment, especially temperature.  At around five NTP servers with
suitable precision you start to see "interesting" things like asymmetric
latency in your local network and can more easily throw out the
inevitable spurs from degraded GPS reception (unless you have a really
good antenna location).

I'd suggest you also log at least the PPS timestamps to correlate to the
NTP logging.  NTP peer logging will be dominated by network latency and
jitter, provided you took care to tune the residual loop error to below
1µs.  I'm running a Perl script that also records the CPU temperature
and system utilization synchronized with the PPS.  All my logging is
into files at the moment, which puts some extra stress on the SD card
that several no-name cards have not survived for long.  I've salvaged an
SSD that I plan to connect via an USB to SATA converter and then set up
a proper time series database on one the boxes to feed all data into.
Alternatively you could log into a tmpfs and rotate onto SD card
whenever you've collected a full Flash block.

I currently have seven stratum-1 NTP servers (five different rasPi and
two TinkerBoard) on my LAN.  I've self-ovenized six of them (the
exception is the rasPi 1B+, which simply isn't powerful enought to pull
that off) to keep the crystal temperature very near the turnover point
of the f vs. T curve, which leaves me with just the jitter and drift of
the (apparent) system frequency most of the time.  The rasPi crystals
(or the interrupt system on the SoC) are a bit noisy with seemingly
unprovoked frequency jumps on a not too-long timescale, so that keeps
you to within a 5ppb window after removing the drift.  The TinkerBoard
doesn't have those jumps and I keep both routinely within 1ppb of the
expected drift curve.  I've experimented with both low and high thermal
mass designs, but so far I don't see a difference in timing performance
between the two.  The high-thermal mass design does smooth out the
external temperature swings more effectively, so with further
refinements to the oven controller it might eventually provide a usable
advantage.

--
Achim.

(on the road :-)

On 12/28/2018 7:45 AM, Achim Gratz wrote:

I built a dedicated server room in my house, with it's own air
conditioner. I've been working on overall instrumentation , especially
temperature. I've got collection points around the room. It stays fairly
consistent. The GPS reception seems pretty good so far with spot checks
of cgps and I'm logging gpsd in the librenms system.

Yeah I do all the logging over the network, specifically to avoid SD
card stress (and I do buy the "nice/recommend" cards, but that's
ultimately a marginal improvement). Could you share the snippets of the
PPS logging? I'm not 100% sure what you mean by the PPS timestamps.

Interesting. My ultimate application of this high precision timing is
driving TDMA wifi links as low cost as possible.

Charles Wyble wrote:

If the rasPi is a dedicated system and does not serve extra tasks, just
record its CPU temperature, no extra sensor needed.  The absolute
temperature will be an almost constant offset from the ambient, so the
changes are well preserved (except for short periods, e.g. when a cron
job runs).

Any temperature transient will show up in the timing error, however
small.  The FLL/PLL code in NTP will need to chase the changing crystal
frequency and then eliminate the already accrued timing error.  The
faster the transient, the larger the error.  So on-off aircon with
forced convection is pretty close to worst case.

Just reading from /dev/pps0 (system time and capture timestamp in my
case), really; additionally system load and CPU temperature and logging
the everything into a file.

I'm not familiar with the requirements for that.  I suspect that the
absolute timing between stations is actually pretty unimportant.  So the
only thing that matters is that the relative timing error between
beacons or syncs must keep below some threshold, which means the
frequency offset of the system must be kept within (probably not too
tight) bounds.  The latter is much easier to achieve and maintain.

Over wired Ethernet you can expect to synchronize a bunch of systems to
within a ~200µs envelope of absolute time and maybe a factor of 2x-3x
lower if you can control certain things more tightly than usual if you
run those system on a single hop switched LAN that have a GPS-based
stratum-1 NTP server.  For anything better than that you'd need PTP,
which unfortunately the rasPi is incapable of.  Or do you plan to use
the rasPi itself as the RX/TX?  The few  papers I've just looked up all
use Atheros 9k hardware.

--
Achim.

(on the road :-)