Update: I have finished routing the board (placement diagram at
http://www.lartmaker.nl/time-nuts/DMTD%20rev1.00%20assembly.pdf ) and
ordered a few prototype PCBs.

After the earlier discussions on the list I've grown sufficiently concerned
about the impact of 1/f converter noise that I have added headers to the
board to allow me to replace the D-flipflop sampler with an FPGA-based I/Q
downconverter. While the main PCBs are in production I'll draw a simple
daughterboard with dual ice40 UltraPlus FPGAs, If the FPGA solution turns
out to be necessary (or a noticeable improvement), I'll redraw the main PCB.

To be continued,

JDB.

On Sun, Sep 1, 2019 at 2:09 AM Jan-Derk Bakker jdbakker@gmail.com wrote:

Update: last Friday our students have populated a few prototype boards (
http://www.lartmaker.nl/time-nuts/dmtd-proto.jpg), and this weekend I
managed to get some early code running on it.

The good: Noise performance matches the datasheet (1.19LSB_RMS given,
1.21LSB_RMS measured). The 1/f corner is much lower than I feared (
http://www.lartmaker.nl/time-nuts/LTC2140-14%20noise%20corner.pdf ;
measured at a sample rate of 10ksps with terminated inputs, PSD averaged
over 500M samples in 1Msample chunks with a rectangular window), and direct
measurements at a 10Hz beat frequency should be possible. (I don't
completely trust this data yet; the absence of 50Hz lines in an unshielded
setup makes me suspicious.)

The bad: LF noise / drift is significant (
http://www.lartmaker.nl/time-nuts/LTC2140-14%20drift.pdf; vertical axis is
in LSB). On a 0dBFS signal with a 10Hz beat note an uncertainty of 0.25LSB
in the zero crossing point would translate to +/-1ps; what I'm seeing here
would eat my entire error budget and then some. I need to find a
computationally cheap way to filter this, or move to the FPGA after all.

The ugly: I can't get the FT2232H channel A to work in asynchronous
FIFO-mode. I have programmed the EEPROM wth the latest version of the
FT_PROG tool, but the TXE# line stays high (under Linux; I'm reading data
from /dev/ttyUSBx with ftdi_sio, a setup which FTDI hints should work). I
haven't looked too hard yet as I have the serial port as a backup, but it
would be nice to get this working.

To be continued,

JDB.
[listadmin: do let me know if you feel these updates are more noise than
signal]

On Sat, Sep 14, 2019 at 2:25 PM Jan-Derk Bakker jdbakker@gmail.com wrote:

Another update: Sampling now works at 100ksps. Samples are passed through a
two-stage CIC filter and are decimated to 2ksps for 200 samples per beat
note period. At system boot I am tuning the on-board VCTCXO (a Taitien
TT-series model) to 10Hz above the reference input and then leave the EFC
constant.

I've been running some tests with a 10MHz sine wave from an Abracon AOCJY1
OCXO into a resistive divider, feeding both channels of the DMTD through
identical SMA cables (Amphenol 135101-07-M0.50). At the ADC input this
yields a -12dBFS sine wave (PSD of the beat note:
http://www.lartmaker.nl/time-nuts/PSD%20of%20AOCJY1%20into%20the%20LTC2140.pdf
). Over a 34000s measurement period the ZCD as described upthread (least
squares fitting of the 32 samples nearest the zero crossing of the rising
flank, but without DC/drift correction) shows a time difference of 6.3ps
between the two channels, with a standard deviation of 1.6ps (full plot:
http://www.lartmaker.nl/time-nuts/DMTD%20Time%20between%20zero%20crossings%20with%20resistive%20divider%20(no%20offset%20correction).pdf
).

My preliminary conclusion is that the ZCD works as expected. I did find
that the zero crossing estimator has a bias which increases with the
fractional (in-between-samples) part of the crossing, caused by the
deviation of the sine wave from a straight line; a linear correction
reduces the contribution of this error to <0.1ps (although I need to
validate this against sine waves with varying amounts of harmonics).

I have yet to implement a robust drift cancellation method. The usual
techniques are either not linear phase (IIR HPF) or computationally
infeasible on an 8-bit platform at 2ksps (various FIR HPFs). I am
considering taking the average of the samples in the interval half a beat
note period before to half a beat note period after the ZC of the rising
edge (by detecting the ZCs of the falling edges); I need to work out the
most robust way to handle interpolation of fractional samples.

In the measurement data linked above I see intermittent bursts of increased
estimator error. These appear to happen in a 3600-second interval (and my
test setup is as unshielded as it gets). Might be interference, might be
something else.

After I've tackled drift cancellation and investigated the error bursts I
intend to repeat my measurement with one OCXO but with different cable
lengths from the splitter to the DMTD; if all works well there I'll mix and
match a few other frequency sources (dual 10811s, a Thunderbolt, a custom
GPSDO and a Rb-standard which I hope will fire up again after having been
in storage for a few years).

I'll be fairly busy on other projects for the next few weeks; I expect to
be able to get back to the DMTD by the end of this month.

To be continued,

JDB.
[have not made any further attempts to get raw samples out of the FTDI-chip
yet, either]

On Sun, Sep 22, 2019 at 11:32 PM Jan-Derk Bakker jdbakker@gmail.com wrote:

Hoi Jan-Derk,

As I am late to the party, I take the liberty to answer a few mails together

On Sun, 1 Sep 2019 02:09:19 +0200
Jan-Derk Bakker jdbakker@gmail.com wrote:

The design is quite decent and I don't see anything "obviously wrong". ;-)
The choice of the LTC2140 is good, there might be better options in
terms of noise, but the LTC2140 is good and available at a decent price.

The biggest change I would make, would be to use a higher sampling
frequency and use an FPGA with a CORDIC as phase detector. Especially
as your goal is to measure the phase difference of a distribution system,
where the frequency of both inputs is exactly the same.

The reason for this is rather simple. You are using a LMS fit over
32 samples around the zero crossing of a 10Hz signal with a ~10MHz
sampling clock. This means you have just a few samples over what would
be otherwise possible. A CORDIC does cover the whole sine wave. While
this does not give you n-times as many samples, it improves the SNR quite
considerably. I don't have exact numbers for how much, as I haven't
had the time to go through the math for this, yet.
The other advantage is, that you operate close to the 1/f corner frequency,
Ie the effect of 1/f noise hits you (almost) fully. Sampling the full
sine wave instead gives you the ability to work far away from the 1/f
corner and thus greatly reduces the effect of 1/f noise.

If you are interested, I have a parametrizable CORDIC core written
in VHDL ready for use. At the low sampling frequencies the LTC2140
works, a MAX10 would be already more than good enough. It probably
should be able to support even higher sampling frequencies in the
order of 100MHz (haven't synthesized it for a MAX10, but guesstimating
from the datasheet). The FPGA could also do the (digital) filtering
and down-sampling, so your uC wouldn't have to do any expensive
calculations.

As sampling period I would choose something that is such that the
signal ends up approximately centered within a niquist zone, but
with an as odd ratio as possible. Ie the reduced fraction a/b should
have as large a and b as possible. The reason for this comes from
number theory and the way how spurs form in DDS.
Alternatively, but with a higher effort, one can choose the sampling
rate such, that a/b has small a and b. This will result in a lot of
the spur energy to fall onto the signal and by that reduce the noise
floor which is mostly spur generated for this kind of application.
The difficulty lies here in having a sampling source that is locked to
the signal but has as low noise as possible, as the close in noise
of the sampling source now defines how well the spurs match up with
the passband signal. Again, I cannot say exactly how much better this
would be, as I haven't gone through the math yet.

On Sat, 5 Oct 2019 20:49:50 +0200
Jan-Derk Bakker jdbakker@gmail.com wrote:

I am a bit astonished by the high noise level you have. I would have expected
this to yield something below 1ps, judging from what we got from what Nicolas
acheived in his work on the sine exitation based TIC[1]. The apperture jitter
of the LTC2140 is spec'ed with 100fs, so that should be the first limit
to run against. Though, based on Sherman and Jördens' work[2], about
one fifth of that should be possible.

BTW: you want to keep even harmonics as low as possible, as these lead
to increase of 1/f noise in the system (see [3] for an explanation)

		Attila Kinali

[1] "Development of a High Precision Multi-Channel Time-to-Digital Converter",
by Nicolas Bucquey, 2016
https://www.ohwr.org/project/r19-tdc-del-a/uploads/a6a30e7969dd05bda87ee27a3112adb9/document.pdf

[2] "Oscillator metrology with software defined radio",
by Sherman and Jördens,2016
http://dx.doi.org/10.1063/1.4950898

[3] "A Physical Sine-to-Square Converter Noise Model", Attila Kinali, 2018
http://people.mpi-inf.mpg.de/~adogan/pubs/IFCS2018_comparator_noise.pdf

--
It is upon moral qualities that a society is ultimately founded. All
the prosperity and technological sophistication in the world is of no
use without that foundation.
-- Miss Matheson, The Diamond Age, Neal Stephenson

Dear Attila,

Thank you for your feedback, replies inline:

On Tue, Oct 15, 2019 at 6:01 PM Attila Kinali attila@kinali.ch wrote:
[snip]

That's the next step (after I've taken this 8-bit processor as far as it
can go). I'm working on a daughterboard with dual Lattice iCE40 UltraPlus
FPGAs, picked mainly because an open toolchain is available, but also for
their price and QFN48 package options (which I've not found in any other
FPGA family of similar density).

(note that for my purposes I do need to DMTD different frequencies, in
particular the 10MHz system master clock vs the slaved 50MHz clocks on the
individual SDR boards in the phased array).

It's not quite that bad, as the double CIC decimator already performs quite
a bit of averaging/filtering. The LMS fit is over 32 samples out of 200 per
period (after the CIC). I expect the largest improvement to come from the
increase in input sample rate.

This is definitely true, and at the moment my largest source of errors. As
an intermediate step I'm considering shifting the beat frequency up some
(say to 40...50Hz) and then I/Q demodulating in software.I expect this will
make the filtering of LF noise easier.

If you are interested, I have a parametrizable CORDIC core written

Thank you' I may take you up on that. So far I've been looking at the
(Verilog) CORDIC code in the Ettus USRP sources.

[snip]

This is actually better than I had expected, given the drift/LF noise I get
from the LTC2140 ( http://www.lartmaker.nl/time-nuts/LTC2140-14%20drift.pdf
). As I've mentioned upthread I'm looking for a robust way to cancel this
drift; my best plan so far is to calculate the signal average between
subsequent falling edges, and to use this to get the zero level for the
rising edge. (This is a problem which I would expect to have a closed form
solution, even when the period of the sine is not an integer multiple of
the sampling rate. Alas, my undergrad-level math seems to be failing me, so
I'm resorting to the blunt instrument of numerical approximations. I hope
to have more time for this in a week or two; in the meantime I'm very open
for hints.)

Thanks, that's good to keep in mind. What I've shown is the unfiltered
output of the OCXO under test; I've not attempted to do any analog
filtering on this.

Sincerely,

JDB.

Dear all,

Attila got me thinking with his remark:

I am a bit astonished by the high noise level you have. I would have

...and I realised that I'm expressing the error wrong. What I had
calculated was the standard deviation of the entire signal (noise +
drift/wander). For short time periods I do get sub-ps noise levels, even
without correcting for LF/DC errors.

It may make more sense to look at the Allan deviation. I've used tvb's
adev1 tool ( http://www.leapsecond.com/tools/adev1.htm ), with a manual
correction to accommodate the 10sps data. To investigate the effect of high
pass filtering and attenuating even-order harmonics, I've generated
1001-point high pass and band pass FIR filters with GMeteor (
http://gmeteor.sourceforge.net/ ; the odd size of 1001 points being the
largest size I could reliably get the program to generate filter solutions
for); the BPF was generated to have nulls at 20, 40, 60 and 80Hz. 10MHz was
generated with an 10811 that had been powered up for three days after
having been in storage for over a year, its output fed to a Mini-Circuits
ZFSC-2-1 splitter, connected to the DMTD with two 3in semi-rigid cables.
The test was run for 175k seconds (just over 2 days) in a very much
non-temperature controlled attic. The resulting ADEV can be found at
http://www.lartmaker.nl/time-nuts/DMTD%20self-noise%20ADEV.pdf ; the
measured time difference between the channels is
http://www.lartmaker.nl/time-nuts/DMTD%20self-noise.png ; the input
spectrum of channel 1 with and without the filters is
http://www.lartmaker.nl/time-nuts/DMTD%20self-noise%20input%20spectrum.pdf

For tau=1s the Allan deviation without any filtering is 9.5E-13. High pass
filtering to eliminate LF noise and drift improves this to 4.6E-13; adding
bandpass filtering yields 3.2E-13. Out to 1000s the adev decreases linearly
with tau, after that performance degrades (further testing in a temperature
controlled room should show whether this is intrinsic or not).

As the high pass filtering seems to have the largest impact, I plan to
implement this first (still based on averaging over a single period
centered around the rising flank of the sine; the on-board processor
doesn't have enough horsepower to run a 1001pt FIR at 2ksps). Next I want
to add code to phase lock the on-board VCTCXO to the reference input, this
should also make it easy to implement a notch filter to eliminate (even)
harmonics. After that I want to see if I can get a computationally
efficient arcsin/arctan applied to the data, to make it easier to extend
the number of samples used by the ZCD without running into linearity
issues. Meanwhile I'm working on a daughterboard with twin Lattice iCE40
FPGAs.

Any suggestions so far?

To be continued,

JDB.

On Tue, Oct 22, 2019 at 12:33 AM Jan-Derk Bakker jdbakker@gmail.com wrote:

Hi

If you phase lock the downconversion reference, the VCXO noise that now is uncorrelated
between the channels will become correlated. That may make things worse rather
than better

Bob

On Sun, 3 Nov 2019 01:41:08 +0100
Jan-Derk Bakker jdbakker@gmail.com wrote:

This looks good, but I would still expect the start of the ADEV plot,
ie the precision of the single measurements to be about half an order
to an order of magnitude lower. For comparison, commercial DMTD systems
operate at about 1e-13 at 1s, which is mostly dominated by the mixer
noise and temperature coefficient. I would expect you to be close to
1e-12 at 0.1s, even before filtering.

BTW: For this kind of measurement, TDEV is the more useful measure.

You care mostly about temperature gradients. If both paths are about the
same temperature, then they will get approximately the same change in delay.
Ie put a cardboard box over it and you will have a pretty stable system.

BTW: a rule of thumb for temp coefficients for quite a large group of
components (mixers, DDS, etc) is 1-5ps/°C,

You do want the notch filters in front of the ADCs. Once the harmonics
enter the system, they already contributed to noise conversion. Digital
filtering helps only if you can completely separate the harmonics from
the signal, which is not possible due to non-linearity of the ADC.
So, I would suggest to have at least some filtering in the frontend
and then clean up the rest in the digital domain.

I don't remember what uC you are using, but CORDIC would be the way to
go in most places, especially if you want lots of bits. You need something
in the order of 2-5 stages more than you want output bits and about as
many intermediate bits more.

		Attila Kinali

--
<JaberWorky> The bad part of Zurich is where the degenerates
throw DARK chocolate at you.

Hi Bob,

is uncorrelated

rather

Does the impact not depend on the loop bandwidth and VCXO performance? If I
read it correctly, in "Oscillator metrology with software defined radio" (
https://aip.scitation.org/doi/10.1063/1.4950898 /
https://arxiv.org/abs/1605.03505 , section B, first paragraph) the authors
implement a sampling DMTD on a USRP B210 SDR platform, with the local VCXO
PLL'ed to the input from their maser; I'm looking at a similar setup (with
the expected LO performance being significantly worse than the incoming
signals).

JDB.

On Sun, Nov 3, 2019 at 2:40 PM Bob kb8tq kb8tq@n1k.org wrote:

Dear Attila,

Is this not caused by the fact that I'm currently subsampling the ADC
(conversion rate 10Msps, rate into the microcontroller 100ksps by dropping
99 put of 100 samples)? In my simulations this accounts for almost exactly
20dB of noise (as expected). The FPGA board will make a filtered rate
reduction possible, of course.

(I'm also trying to find the balance between "perfect" and "good enough"
here. My original target was 1ps RMS; I'm hitting that even at 100ksps with
some light filtering. This DMTD is a means, not an end, and at some point I
will hit diminishing returns; engineering time spent on the DMTD cannot
also be spent on the phased array. Of course, I do try to get all low
hanging fruit before putting the DMTD into service.)

You do want the notch filters in front of the ADCs. Once the harmonics

Thanks, will put that on the list. This would have to be a separate
pluggable filter, as I need to support different input frequencies with the
same DMTD.

I don't remember what uC you are using, but CORDIC would be the way to

Yes, I'm looking at CORDIC. The CPU I'm using, an Atmel/Microchip XMega,
doesn't have a barrel shifter; to speed up the ZCD I'm playing with an
unrolled CORDIC where I use the hardware multiplier instead.

Sincerely,

JDB.

Hi

Yes, it only will be fully correlated well inside the loop bandwidth. The loop bandwidth
normally used for this sort of thing is >> 1 Hz. By the time you start doing ADEV, you
are in the correlated region.

Bob

Dear Bob,

I understand the general concern: perfect synchronization would potentially
turn the Dual Mixer Time Difference system into a Single Mixer Time
Difference setup. (Even with perfect synchronization, the dual-channel
architecture would serve to attenuate the common mode component of the ADC
aperture jitter and the rest of the front end).

In the Sherman & Roberts NIST paper I mentioned earlier the authors use a
3kHz PLL loop bandwidth without apparent ill effect. From my measurements
of the drift behavior of my VCTCXO I'm considering much lower bandwidths
(10..50mHz, possibly even lower when I substitute a VCOCXO for the offset
oscillator). While I feel this should be safe doing so, I admit I need to
look harder at the underlying math (and pointers are always welcome).

Sincerely,

JDB.

On Mon, Nov 4, 2019 at 3:00 AM Bob kb8tq kb8tq@n1k.org wrote:

Hi

I think you will find that with practical components and lock requirements that
bandwidths in the mili hertz range create more trouble than you might expect.
Without and oven involved, sub 1 Hz is challenging. It also puts the whole
“transition region” (and its noise hump) in the range of your typical ADEV plot.

Bob

Hi Jan-Derk

this is maybe a bit offtopic, but if you don't mind I would like to ask anyway:
is this a DMTD system you have built yourself? I saw a lot of messages on this topic in the archive, but didn't find where it started.
Since I would also like to do some clock stability measurements but don't have a HP 53131A or SR620, I thought the DMTD would be a good technique because it allows to measure picosecond phase errors without actually having a TIC with that resolution.
I only have a HP 5335A and 5316A, both of which don't have very high resolution and are thus perhaps not suitable to measure stability of OCXOs, but with a DMTD, it could be possible.
What oscillator do you use as the offset oscillator, and what is the reference you use?
I wonder whether it would be possible to use a HP 8662A or 8663A low phase noise signal generator as the offset oscillator. Because these can be adjusted in frequency for a beat signal which is convenient to measure. And the phase noise or stability of the offset oscillator is not of very high importance in DMTD measurements.

Best
Tobias
HB9FSX

From: time-nuts [time-nuts-bounces@lists.febo.com] on behalf of Jan-Derk Bakker [jdbakker@gmail.com]
Sent: Monday, November 04, 2019 10:33
To: Discussion of precise time and frequency measurement
Subject: Re: [time-nuts] A simple sampling DMTD

Dear Bob,

I understand the general concern: perfect synchronization would potentially
turn the Dual Mixer Time Difference system into a Single Mixer Time
Difference setup. (Even with perfect synchronization, the dual-channel
architecture would serve to attenuate the common mode component of the ADC
aperture jitter and the rest of the front end).

In the Sherman & Roberts NIST paper I mentioned earlier the authors use a
3kHz PLL loop bandwidth without apparent ill effect. From my measurements
of the drift behavior of my VCTCXO I'm considering much lower bandwidths
(10..50mHz, possibly even lower when I substitute a VCOCXO for the offset
oscillator). While I feel this should be safe doing so, I admit I need to
look harder at the underlying math (and pointers are always welcome).

Sincerely,

JDB.

On Mon, Nov 4, 2019 at 3:00 AM Bob kb8tq kb8tq@n1k.org wrote:

time-nuts mailing list -- time-nuts@lists.febo.com
To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.

Hello,

(It has been a more few years since I designed / developed DSP for spectrum analyzers for a major company, thanks for your patience.)

I recall that:

sample @ fs -> decimate (toss samples) by a factor of N

is equivalent to

sample @ fs / N

If we wanted a lower sample rate, we would:

sample @ fs  -> lowpass filter at (say) 80% of (fs/2) / N -> decimate

Otherwise all of the energy > 50% of (fs / N) gets aliased into your data.

Why not just sample at 100 kHz?

What am I missing ?

Thanks for considering,
Scott W7SLS

Dear Scott,

You are entirely correct, sampling at 100ksps is mathematically the same as
sampling at 10Msps and then decimating by a factor of 100. The reason I'm
doing it in this way is driven by two practical considerations:

• my ADC, the LTC2140 (selected for bandwidth, dynamic range, aperture
  jitter and availability) has a pipelined design and cannot be clocked as
  slow as 100ksps without degrading performance
• my CPU, the Atmel/Microchip XMega (selected for its peripherals, toolset
  support and familiarity for both me and my students) is an 8-bit AVR
  running at 30MHz and cannot directly process 10Msps

A simple D-flipflop based decimator was the easiest way to bridge this gap.
The DMTD design has connectors for a (forthcoming) FPGA daughterboard to
properly sinc-filter (and I/Q demodulate and...) the incoming samples.

Sincerely,

JD 'walk first, then run' B.

On Mon, Nov 4, 2019 at 6:38 PM W7SLS w7sls.scott@gmail.com wrote:

Dear Tobias,

Yeah, this thread has been going since late August. As far as I can tell
the best way to get it all in one go is at
https://www.mail-archive.com/time-nuts@lists.febo.com/msg04265.html ; the
main design considerations are listed in the first post.

As for the offset oscillator,I have two versions of the board built, one
with a Taitien TT-series VCTCXO (with which we've good experiences in a few
SDR/GPSDO-designs), and one with a Connor-Winfield DOC020V-series VCOCXO.
Both have ample EFC range to be shifted to 10MHz+10Hz. For the moment I'm
only doing self noise measurements, with an OCXO connected to both
measurement channels. I could not advise you on the suitability of the
HP8662/8663, not being familiar with these instruments.

Sincerely,

JDB.
[while I'm primarily developing this DMTD for use in my lab, I would still
be very happy to do an Open Hardware/Software release of the designs and/or
work with TAPR or others, should there be any demand]

On Mon, Nov 4, 2019 at 6:38 PM Tobias Pluess tobias.pluess@xwmail.ch
wrote: