Hi

Any practical measure you feed into an ADEV computation will be a look at
“device A” versus “device B”. In this case one of them is your GPSDO. What
is the other device? ( = your TIC has a DUT input and a REF IN, it compares
one to the other ….).

If you feed your measurement system with the same signal on both inputs,
you get a “noise floor” measurement. This does have value since it represents
the best numbers you should ever see out of your system. Checking the
noise floor is highly recommended …..

======

One of the wonderful things about going to conferences is the ability to interact
with folks face to face. The question of “how many samples for ADEV?” was one
of my favorites. The range of answers turned out to be all over the place. On one
end, the practical answer of “I plot what I’ve got”. On the other end the statistician's
answer of 100 to 250. Both answers generally were also tagged with a lot of
“that depends” sort of stuff.

As a practical point, 100 samples for an ADEV plot with Tau = 100,000 is a pretty
long data run. Most of us don’t have ~4 months to do that sort of run. Indeed one
of the drivers for various “other” dev measurements has been to improve the
confidence with a lower number of samples.

Lots of zigs and zags ….

Bob

Hi again Bob - Seems I am destined to hog your time!

If my understanding and my questions appear foolish, sorry - I do have a
lot to learn to just understand the basics.

Maybe my setup is in fact comparing itself with itself?..!
I'll try to explain my hardware concisely -

The GPS 1PPS (with hardware saw-tooth compensation, not relevant here I
think) sets a flip-flop. The OCXO 10MHz is divided by 10, and that 1MHz
drives the reset of the flip-flop.
So the 1PPS rising edge sets the FF, the first rising edge of the OCXO
derived 1MHz clock resets it. The length of the pulse on the Q output of
the FF is now directly related to the stability of the 1MHz clock edge to
the 1PPs edge.
The max length is 1us ( the 1MHz clock period). It is this pulse length
that is measured (The TIC) and logged. The logged data is time stamped by
the falling edge of the GPS 1PPS. This data contains the essential Phase
difference between the PPS signal edge and the 1MHz clock edge, with a
regular jitter from the inherent PPS jitter from the GPS.
I then used TimeLab to read in that file, with sample interval set to
1second, and plot the Adev chart.
I understood that in my setup as described, my GPSDO , or the OXCO 10MHz
output, is now being compared to the GPS 'accurate' 1PPS - I am aware of
the PPS jitter, etc, but I also understand that that will be smoothed out
in the Adev derivation.

So, how much of my understanding of the above is bad?!

Secondly, I have not grasped the meaning of :

As a practical point, 100 samples for an ADEV plot with Tau = 100,000 is a
prettylong data run.  Most of us don’t have ~4 months to do that sort of
run

Could you explain what the 100 samples are, and what Tau=100,000 means?  I
am trying to put that in context with my data file -
eg, I have a file logged for maybe 32hours, with 145,000 logs, each one
second apart. Is that 145,000 samples? or just 145,000 data points? Or
is it Tau=145,000?
When I plot the Adev chart, it shows on the x axis that my 145,000 data
points plots as nearly 20,000 seconds - is that the 'samples' you mean?
And how did you equate that to needing ~4months?  Forgive my doltness - I
need to understand the relationships and terminology before it clicks!

I have gone through many examples and read heaps, but have not found a real
101 explanation of terms and data relationships yet..

Thank You!
Joe

On Mon, Oct 26, 2020 at 4:21 PM Bob kb8tq kb8tq@n1k.org wrote:

On 2020-10-26 15:17, Bob kb8tq wrote:

Already in the 70thies the overlapping ADEV was developed to increase
the estimated degrees of freedom (EDF) to bring confidence intervals
down. Some insist of separating them, but it's scaled the same way, thus
there is no bias, so it's just ADEV. Then David Howe went to discover
the measurement and found it to be missing half the data, and the
TotalDEV was developed and then he went further for Theo.

Oh, Bill Riley have developed a set of exercises to teach people to use
the Stable32. I have made a first round of reviews, and he have improved
them accordingly.

Cheers,
Magnus

Hi

Is the OCXO part of a GPSDO? In other words, is it steered to GPS?

In terms of ADEV, Tau is the time between phase samples. We talk about “1 second
ADEV”. In that case Tau is 1 second. You take phase samples at clock tick 0, then
again one second later at tick 1 and so on until you have a full data set.

If you look at other data spacing’s you get a different number for ADEV. 10, 100,
and 1,000 seconds are pretty typical. Going out to a bit over a day between samples
gets you to a tau of 100,000 seconds.

Tau is 1 second if you analyze that data for 1 sec ADEV. You will have 145,000 samples
in the data set.

If you go for Tau = 1000 seconds, you throw out 999 out of every 1000 samples. That turns
your data set into 145 samples. Same data, same “ADEV”, just at a different Tau.

If you have 145,000 data points spaced at 1 second apart and TimeLab or Stable32
shows it as 20,000 seconds … you have a problem with your data file. Something about
the time tags isn’t in line with what the program is expecting.

If you have points 100,000 seconds apart and want 100 points, it will take you 10,000,000
seconds to get that data. That’s 115.74 days of data collection. Four months would be 120
days …

A lot of this is simply getting the data into a form that this or that program “likes”. It’s
just a lot of tweaking and seeing if the result comes out right.

Bob

Bob, thanks for the very detailed explanation. It starts to make much more
sense.

To answer your question, yes, the 10MHz OCXO I am measuring is part of the
GPSDO and is steered to GPS 1PPS.
The process I explained ( the 1PPS and the 1MHz clock from the OCXO / 10
into the flip flop, etc) and the TIC, OCXO in question, and GPS 1PPS are
all part of said GPSDO.

It is then the same GPS 1PPS cycle which time stamps the sampled and
logged  phase data from the TIC, ie, there is not a second GPS 1PPS or
anything.
The 1 second phase output value from the TIC feeds a 32 tap IIR filter,
which then drives the EFC control DAC. At the same time that same 1 second
cycle phase value is logged, each second.
Does that make sense, and am I sampling appropriately doing it this way?

I hope what I have done is not the same as trying to measure a 10Mhz signal
on a frequency counter, while using the same 10MHz signal as the reference
clock for the frequency counter timebase...!!

I am concerned about your statement :

• If you have 145,000 data points spaced at 1 second apart and TimeLab or
  Stable32*

shows it as 20,000 seconds … you have a problem with your data file.
Something aboutthe time tags isn’t in line with what the program is
expecting.

The Timelab plot I attached in my first post shows on the bottom -
Input Freq = 10Mhz
Sample Interval = 1second
duration = 1day 15hours
Acquired points = 141879  ( this is the number of points in the file)
So 141879 seconds is 1day 15hours, but the plot X Axis shows 20000
seconds...

I cannot find anything wrong with the sample data - no missing samples, etc.
Is this not representing the averaging process?

Thank You again for your explanations!
Joe

On Mon, Oct 26, 2020 at 9:03 PM Bob kb8tq kb8tq@n1k.org wrote:

Hi

If the OCXO is locked to the GPS and the GPS PPS is used to measure
the OCXO …. you are measuring GPS against GPS. No different than using
a counter to look at its reference.

In order to evaluate a GPSDO, you need something to compare it to that
is independent of GPS. If your reference is connected to GPS in some way,
you eventually will just be comparing GPS to GPS.

Candidates for a comparison standard:

Good OCXO, probably good for tau = 1 sec out to tau = 100 sec
Rb, likely good for tau = 10 sec out to tau = 10K sec
Cs, probably good for tau = 100 sec on out.

Yes there are other more exotic comparison standards out there. These are the
ones that normal people seem to have around. In all cases, the standard should
be “free running” and not locked to anything else.

With each of the standards, you can get some that are better than others. In most
cases there is a “buy a bunch and sort them all out” process involved.

Assuming you are looking at a phase plot …. If it’s showing tau on an ADEV
plot, that’s something different entirely.

If you take a look at the formula used to compute ADEV, the first thing you do is to
get frequency from phase by taking delta phase over your tau. If you had 5 samples,
you now have 4 pieces of data ( 4 differences).

Next it tries to take the standard deviation of the data. That’s pretty much nonsense
below 4 pieces of data (and mostly nonsense even then).

So what it’s showing you is still more than you probably should be counting on to be
useful.

Bob

Hi

To be really clear:

ADEV can be used to measure anything and done any way you wish.

To have an ADEV that is a valid measure of a device, you need to do a
comparison to another totally independent device. That independent
device needs to have a better ADEV over the range of Tau that you are
after over that range of Tau.

Anything that tries to get ADEV out of a single device or cross connected
setup will run into problems.

Bob

Joe,

generate an ADEV plot.

Good. That's what you need. During normal operation those readings are
bounded by the PLL. So it's essentially a measurement of how well the
PLL is working, how aggressive the OCXO is steered, etc. ADEV isn't the
best way to process that kind of data because it's a boring, even
misleading, straight line going down forever.

Yes. Oops. But, here's an idea for you.

One useful technique is to have your GPSDO running fine and then
disable the disciplining. If you designed the GPSDO you'll know the
exact spot in the h/w or s/w to do this. From this point forward your
OCXO is still running, your GPS/1PPS receiver is still receiving, the
TIC is still comparing, and you are still logging TIC readings every
second. But now the DAC is frozen and the OCXO is free-running.

When you plot this data you will see phase slowly wandering away from
zero, you may see a slight drift in frequency, and mostly what you will
see is the "bathtub" ADEV shape that you were looking for. This method
works because as soon as your disable disciplining your OCXO becomes
independent of GPS and so the ADEV plot will be a measurement of an
oscillator instead of a measurement of a PLL.

This is not something you would do everyday, but especially now that you
are understanding how a GPSDO works and playing with Allan statistics
it's a educational exercise.

The Trimble Thunderbolt (aka TBolt) GPSDO has this disable-disciplining
feature. Note it's not "holdover"; that's something else entirely.

/tvb

Bob and Tom,
Thanks again for your time on this.  I understand a lot better, and have
just one issue I would like to 'harp' on a little, if you would allow..
In the simplistic example of a freq counter measuring its own reference, It
it easy to grasp and understand the incestous nature of the measurement.

I am having difficulty extending that concept to what I have though.  For
the purpose of understanding this concept, lets ignore jitter, and all
other 2nd order effects for now.

My OCXO is phase locked to a GPS 1PPS . The same GPS 1PPS is 'locked' to
a very accurate , very stable  (Cesium?) reference within the SAT
constellation.
I would say that if I use the GPS 1PPS ( which is the same as the Cesium
reference, in my example..) as my reference, then when I measure the phase
delta between the OCXO and that 1PPS
I am in fact measuring the phase delta between the OXCO and the accurate
Cesium reference.
I realise I am in fact measuring how well the OCXO is phase locked to the
Reference ( 1PPS , derived from the Cesium reference..) , but that should
still show what the frequency and phase offset is
between the OCXO and the Reference.
I am having difficulty seeing that this is in fact not independent - the
underlying raw reference for the measurement is the Cesium reference and I
can't get better than that.
Substituting a separate, equally good Cesium reference from which I derive
a 1PPS, is surely no different?

To simplify my confusion, I have attached a PDF block diagram - this shows
a 'perfect' 10MHz reference oscillator - perfect in accuracy, drift, phase
errors, etc - just perfect.
It is the reference for a PLL with the OCXO being controlled. The perfect
osc is divided down to present a 1PPS to the TIC. The OCXO is divided down
to present a 1MHz signal to the TIC.
The resultant phase delta is logged and used to plot Adev - basically what
I described above, but a perfect Osc instead of the GPS.
This surely is comparing the OCXO phase to the perfect osc phase,
regardless of what is controlling or steering the OCXO?

Tom, I am not sure what you mean by -

*The Trimble Thunderbolt (aka TBolt) GPSDO has this
disable-discipliningfeature. Note it's not "holdover"; that's something
else entirely.  *

I assume 'holdover' to be when the OCXO EFC voltage is just held fixed?
If so, I do not understand how disciplining can be disabled without the EFC
voltage just being held to a fixed value?
Can you explain the difference between 'disciplining-disabled' and
'holdover' please?

Chaps, thank you for indulging me on this - the basic concepts are the
formative grounding for beginning to understand this subject even a little
and I appreciate your assistance
and guidance in this!
Regards
Joe

On Tue, Oct 27, 2020 at 1:51 AM Tom Van Baak tvb@leapsecond.com wrote:

Joe,

Holdover is when there is no GPS reception and your GPSDO runs blind.
EFC may or may not be fixed depending on how clever the s/w tries to be.
But the main thing with holdover is that there is no GPS/1PPS present
and so no TIC readings either.

The "discipline-disable" mode is when GPS/1PPS is still valid and the
TIC is still giving you valid readings. The only thing you don't do is
change the EFC. The net result is that your GPSDO is measuring your OCXO
without adjusting your OCXO. Make sense? Try it out. Add a h/w switch or
comment out the line of code where you update the DAC. Watch your TIC
readings start to drift. And plot with TimeLab.

The TBolt user manual is a good read. [1]

/tvb

[1] http://leapsecond.com/pages/tbolt/manual.htm

On 10/26/2020 11:19 PM, Joe & Gisela Noci wrote:

Joe & Gisela Noci jgnoci@gmail.com
[image: Attachments]1:33 PM (12 minutes ago)
to Discussion
Hello Tom.
Well, you just opened another can of worms for me...
My GPSDO has many controls, one being that I can place it in DAC hold with
everything else still running - I can then adjust the DAC voltage manually
for 'zero' drift on the TIC output if desired.

See the Front panel screen - *Hold Mode on GPSDOa.jpg - * attached.
So I let the unit stabilize for some hours again, and enable HOLD mode and
then recorded the TIC output.
I record lots of other info as well - Oven temp, and many diagnostic data
elements as well.
I did this recording during a period where I know the ambient remains
reasonably stable for a while...(not long enough it would seem!)..

The plot Temp_vs_TIC.jpg shows ( a noisy..) TIC plot versus the oven temp
control - you can see how the ambient was quite stable for around 5000
seconds, and then became warmer ( top plot)
and how the Phase delta was likewise 'stable' during the 5000 seconds, and
then drifts with the increase on ambient ( bottom plot).

The plot Mod Allan Dev increasing temp.gif shows the turnaround
coinciding with the rising temp.
The plot Phase Delta over time - increasing temp.png shows the increase
in phase shift with temp as well..
So it all ties up, but..
The Adev plot is very poor ( in my eyes anyway..) sort of 4E-12... and
getting worse - but how much of that is because of the fact that the
OCXO ( Free running now) is being affected by ambient post the 5000 seconds
mark? In other words, if the ambient did not change, ie, remained close to
the first 5000 seconds, would Adev improve, all things being equal?
And if so I suppose the only way to do this sort of test is in a temp
controlled environment.

I must admit that I am now worried that this OCXO is in fact not well...I
see so many Adev sample plots of 'good' crystal oscillators giving Adev's
in the 1E-12 or better , within the first 2000 seconds...Or am I just
mis-interpreting reality still..

Thanks again to all!
regards

On Tue, Oct 27, 2020 at 11:08 AM Tom Van Baak tvb@leapsecond.com wrote:

Hi

What are we trying to measure with ADEV?

On a frequency source, it’s used to measure the noise of that source. Mostly we
put up plots of ADEV to show how quiet our source is. (Yes, we might also measure
noise floor or amplifier contributions ….).

That’s the statistics part. Now for the instrumentation:

There is no device that will directly measure frequency / time / phase to the accuracy
levels we are after. The closest we can get is to measure A against B and look at the
delta. We also could look at A vs B vs C and do some fancy math, that takes a bit
of setup and has some pretty significant limits. You still are doing a comparison.

With an A to B comparison, you need to know that one or the other device is much
lower noise than the other. Then the plot will (essentially) be the noise of the not so
quiet source. If that’s not true, you have a real tangle. It’s even worse if both our
sources have the same noise in them.

So how does this apply in your case?

The 1 pps out of your GPS module is far from a low noise source close in. Further
out it will wander more than a little due to ionosphere issues. Depending on how the
OCXO is locked, there  will be a cross over between “free run” and “follows GPS”.

The real ADEV of any GPSDO starts out with the free run OCXO noise + loop noise +
GPS noise. Hopefully (but not always) the OCXO noise is the big item close in. As you
get further out, GPS noise becomes the dominant contributor to the output noise. Again
this is a “hopefully” sort of thing. We test our designs because that’s what shows us
where improvement is needed …..

What to do?

You need an independent source or set of sources to compare against. You need to
feed them into something like a DMTD to measure what’s going on.

Bob

Thanks Bob.  Slowly it penetrates..
The 1PPS source is as you say far from being a good reference, so I will
cease that route!

I think I have to accept what I have - I believe it is good enough for my
Ham Radio work as a reference for my radios, at least into the low GHz
region.
If I manage less than 100Hz error in the 2.4GHz band, I am OK..

As I mentioned before, its very difficult for me to obtain or have
access to Reference Grade solutions, or for me to buy a handful of suitable
surplus sources, etc...
The usual Ebay sources don't work well to our part of the world, very
difficult, very costly, and most Ebay vendors won't even ship toNamibia!

Having learned a lot from you chaps, for which I am grateful, I think I now
at least know the size of the cliff above of me!

Thanks to all..
regards
Joe

On Tue, Oct 27, 2020 at 3:29 PM Bob kb8tq kb8tq@n1k.org wrote:

Hi,

I'd like to add a comment.

What is ADEV?

It is the replacement for standard deviation as we attempt to measure
the frequency of a source using a frequency estimation based on two
phase-samples with some time distant tau between them. Thus, it shows
the frequency stability for a 2-point frequency estimator using 2
frequency points of estimation of each stability measure (yeah, that's
the picky formulation, sorry).

Normal standard deviation will not provide meaningful results, or as
math people say, it does not converge. Without going into the pandora
box of math, some of which still remains properly and consistently
solved, this is a very tricky field and it was only during a few years
in the 60thies that a model that kind of works was formulated, resulting
the 2-point deviation measure, later called Allan's deviation and now
with the standardized name Allan deviation. It has since been expanded
upon in numerous ways, many misunderstood.

It's a response to the problem of the random noise types we have, but is
not very meaningful measure for systematic noises, albeit it is often
used for this without too much concern for it's correctness.

There is other ways to estimate frequency, and for that you need to use
other estimators, namely MDEV and PDEV. There is yet other estimators of
ADEV that provide better statistical certainty to provide the same
value. For some variants one has to use bias-functions, as they have
biases compared to ADEV, but those biases is predictable and hence one
can correct for them to produce an ADEV value.

I tried to convey some of this as I wrote the Wikipedia article on Allan
Deviation. I've not had time to keep track of it and the tons of edits
that have been done, but I should take the time to review it and refresh it.

Cheers,
Magnus

On 2020-10-27 13:46, Bob kb8tq wrote:

Hi

Indeed this is a very fundamental problem in doing this sort of work.
The test setups and references generally are way harder to set up (and
more costly) than the design and construction of this or that device.

Indeed, as part of a normal design process on a GPSDO, the testing /
tweaking phase is much longer and much more expensive than the
work leading to the first “working” batch of units.

Fun !!!

Bob

On another thread,  G3YSX described this technique :
Taken in conjunction with the Wikipedia entry it begins to make sense here.

...................................
"A note on this thread:  plotting phase against time will tell you a lot
about the effect of propagation on phase, but it is not the technique used
in the synchronisation world to understand the behaviour of oscillators.

They use Allen variance, TDEV and MTIE plots.

Fundamentally what these measurements do is to look at the error every
second, and then cumulatively work out what the error is every second,
every two seconds, every four seconds etc until the limit of the
measurement, but they do it using a sliding window system so in a one
hundred second time (say) they have one hundred one second measurements to
average, 99 two second measurements, 98 three second measurements…. 49 50
sec measurements etc. Then they plot this on a log scale.

This tells you a lot more about what is going on than the phase error vs
time measurements and revolutionised understanding of oscillator and hence
time measurement characteristics. In particular with the more sophisticated
plots developed after Allen initial work, the slope of the curve at various
places along the plot tells you the cause of the error in the system."

Andy
www.g4jnt.com

http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail
Virus-free.
www.avg.com
http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Tue, 27 Oct 2020 at 15:59, Bob kb8tq kb8tq@n1k.org wrote:

http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail
Virus-free.
www.avg.com
http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On 2020-10-27 17:38, Andy Talbot wrote:

Is "Allen variance" just the British way of spelling "Allan variance", or are they two different things ?

73  Alberto  I2PHD

On 10/27/20 11:00 AM, Alberto di Bene wrote:

It's David Allan's name, and references to Allen variance are just
misspellings (or bad spell check.. I've also seen alien variance :)).

There's also some playing fast and loose with variance vs deviation.
Most of the time, what people are plotting is deviation, which is the
square root of variance.  And hopefully labeled as such.

Alberto,

Thanks for asking. The correct spelling is Allan variance or Allan
deviation, named after David Allan, of NBS (NIST). Yes, on rare
occasions it is misspelled Allen, as in the hex key wrench. Note it's
spelled correctly in the Subject line of this thread. I'm not aware of
national explanations for the misspelling; it could simply be that allen
is more popular a name than allan and a spell checker allows either one.

The ADEV statistic itself is spelled Allan because that's his name: Dr.
David W. Allan. Similarly, the hex wrench is spelled Allen because of
Mr. William G. Allen.

More fun: When you google word search it displays a hit count:

"allan deviation" About        84,300 results (0.46 seconds)
"allen deviation" About         2,460 results (0.40 seconds)

"allan variance"  About        85,400 results (0.41 seconds)
"allen variance"  About        21,300 results (0.37 seconds)

"allen wrench"    About     4,820,000 results (0.52 seconds)
"allan"           About   213,000,000 results (0.77 seconds)
"allen"           About 1,190,000,000 results (0.87 seconds)

You can see that the word allen is 5x more popular than allan. That
alone may explain the occasion misspelling of ADEV. Also, apparently,
Allen wrench tools are 50x more popular than Allan deviation tools ;-)

More examples in popular vocabulary:

https://en.wikipedia.org/wiki/Hex_key
https://en.wikipedia.org/wiki/Allen_(brand)
https://en.wikipedia.org/wiki/Steve_Allen
https://en.wikipedia.org/wiki/Woody_Allen
https://en.wikipedia.org/wiki/Tim_Allen

vs.

https://en.wikipedia.org/wiki/David_W._Allan
https://en.wikipedia.org/wiki/Allan_variance

Some papers on the history of ADEV:

"Introduction to the Special Issue on Celebrating the 50th Anniversary
of the Allan Variance"
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7445935
(yes, open access! thank you, ieee)

"The Evolution of Frequency Stability Analysis Software", W.J. Riley
http://www.wriley.com/The%20Evolution%20of%20Frequency%20Stability%20Analysis%20Software.pdf

"BYU and CU Alumni Master's Thesis 50th Anniversary is Celebrated in
IEEE UFFC Special Issue"
https://itsabouttimebook.com/wp-content/uploads/2016/05/50th-anniversary-for-allan-variance.pdf

"Technical Background on the Fundamental Noise Problem"
https://ethw.org/w/images/e/e5/Allan_OH_-_FundamentalNoiseProblem.pdf

/tvb

On 10/27/2020 11:00 AM, Alberto di Bene wrote: