time-nuts@lists.febo.com

Discussion of precise time and frequency measurement

View all threads

Weird TEC data

SN
Scott Newell
Wed, Aug 3, 2011 2:18 AM

Here's the latest plot from my TEC test rigs.  Y-axis is phase error
(in 60 Hz cycles), X-axis is time in MJD.  The plot starts at 7AM local time.

http://n5tnl.com/tec/tec_test_01.png

Same serial connected embedded hardware, timestamped on the receipt
of first character.  One machine (red line) is a Sun running debian,
the other (green line) is a Dell PC, also running debian.  Both
running ntp, of course.  One is located in a fairly large city (by
Arkansas standards!), the other is out in a small country
town.  Different utility providers.

The general shape and bumps in the plots track nicely, but I'm
wondering why there's so many cycles difference after 36 hours.

Comments?

Here's the latest plot from my TEC test rigs. Y-axis is phase error (in 60 Hz cycles), X-axis is time in MJD. The plot starts at 7AM local time. http://n5tnl.com/tec/tec_test_01.png Same serial connected embedded hardware, timestamped on the receipt of first character. One machine (red line) is a Sun running debian, the other (green line) is a Dell PC, also running debian. Both running ntp, of course. One is located in a fairly large city (by Arkansas standards!), the other is out in a small country town. Different utility providers. The general shape and bumps in the plots track nicely, but I'm wondering why there's so many cycles difference after 36 hours. Comments?
AK
Attila Kinali
Wed, Aug 3, 2011 9:19 AM

On Tue, 02 Aug 2011 21:18:17 -0500
Scott Newell newell@cei.net wrote:

The general shape and bumps in the plots track nicely, but I'm
wondering why there's so many cycles difference after 36 hours.

Could be a measurement error (like losing ticks or something).
Have you ploted the difference between both? For me it looks as
the difference is (nearly) linearly growing over time, but it's
hard to tell from such a plot.

		Attila Kinali

--
The trouble with you, Shev, is you don't say anything until you've saved
up a whole truckload of damned heavy brick arguments and then you dump
them all out and never look at the bleeding body mangled beneath the heap
-- Tirin, The Dispossessed, U. Le Guin

On Tue, 02 Aug 2011 21:18:17 -0500 Scott Newell <newell@cei.net> wrote: > The general shape and bumps in the plots track nicely, but I'm > wondering why there's so many cycles difference after 36 hours. Could be a measurement error (like losing ticks or something). Have you ploted the difference between both? For me it looks as the difference is (nearly) linearly growing over time, but it's hard to tell from such a plot. Attila Kinali -- The trouble with you, Shev, is you don't say anything until you've saved up a whole truckload of damned heavy brick arguments and then you dump them all out and never look at the bleeding body mangled beneath the heap -- Tirin, The Dispossessed, U. Le Guin
MD
Magnus Danielson
Wed, Aug 3, 2011 10:00 AM

On 08/03/2011 11:19 AM, Attila Kinali wrote:

On Tue, 02 Aug 2011 21:18:17 -0500
Scott Newellnewell@cei.net  wrote:

The general shape and bumps in the plots track nicely, but I'm
wondering why there's so many cycles difference after 36 hours.

Could be a measurement error (like losing ticks or something).
Have you ploted the difference between both? For me it looks as
the difference is (nearly) linearly growing over time, but it's
hard to tell from such a plot.

Indeed nice common view, but the difference plot should be there, as it
would better reveal what is going on. The difference plot is probably
best to do by itself since the common mode disturbance varies quite a
lot compared to the peak difference, so scaling would not be suitable to
be the same.

Cheers,
Magnus

On 08/03/2011 11:19 AM, Attila Kinali wrote: > On Tue, 02 Aug 2011 21:18:17 -0500 > Scott Newell<newell@cei.net> wrote: > > >> The general shape and bumps in the plots track nicely, but I'm >> wondering why there's so many cycles difference after 36 hours. > > Could be a measurement error (like losing ticks or something). > Have you ploted the difference between both? For me it looks as > the difference is (nearly) linearly growing over time, but it's > hard to tell from such a plot. Indeed nice common view, but the difference plot should be there, as it would better reveal what is going on. The difference plot is probably best to do by itself since the common mode disturbance varies quite a lot compared to the peak difference, so scaling would not be suitable to be the same. Cheers, Magnus
SN
Scott Newell
Thu, Aug 4, 2011 2:20 AM

At 05:00 AM 8/3/2011, Magnus Danielson wrote:

On 08/03/2011 11:19 AM, Attila Kinali wrote:

Could be a measurement error (like losing ticks or something).
Have you ploted the difference between both? For me it looks as
the difference is (nearly) linearly growing over time, but it's
hard to tell from such a plot.

Indeed nice common view, but the difference plot should be there, as
it would better reveal what is going on. The difference plot is
probably best to do by itself since the common mode disturbance
varies quite a lot compared to the peak difference, so scaling would
not be suitable to be the same.

Scaled the difference by 5 and updated to include another day:
http://n5tnl.com/tec/tec_test_02.png

--
newell  N5TNL

At 05:00 AM 8/3/2011, Magnus Danielson wrote: >On 08/03/2011 11:19 AM, Attila Kinali wrote: >> >>Could be a measurement error (like losing ticks or something). >>Have you ploted the difference between both? For me it looks as >>the difference is (nearly) linearly growing over time, but it's >>hard to tell from such a plot. > >Indeed nice common view, but the difference plot should be there, as >it would better reveal what is going on. The difference plot is >probably best to do by itself since the common mode disturbance >varies quite a lot compared to the peak difference, so scaling would >not be suitable to be the same. Scaled the difference by 5 and updated to include another day: http://n5tnl.com/tec/tec_test_02.png -- newell N5TNL
PK
Poul-Henning Kamp
Thu, Aug 4, 2011 6:12 AM

In message 916609.32990.qm@smtp106.prem.mail.ac4.yahoo.com, Scott Newell writes:

At 05:00 AM 8/3/2011, Magnus Danielson wrote:

Scaled the difference by 5 and updated to include another day:
http://n5tnl.com/tec/tec_test_02.png

The jumps in the difference looks a lot like transformer phase-lag
in the grid, but the real test is to collect more data and see if
you ever see the difference move the other way.

--
Poul-Henning Kamp      | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG        | TCP/IP since RFC 956
FreeBSD committer      | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

In message <916609.32990.qm@smtp106.prem.mail.ac4.yahoo.com>, Scott Newell writes: >At 05:00 AM 8/3/2011, Magnus Danielson wrote: >Scaled the difference by 5 and updated to include another day: >http://n5tnl.com/tec/tec_test_02.png The jumps in the difference looks a lot like transformer phase-lag in the grid, but the real test is to collect more data and see if you ever see the difference move the other way. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence.
BH
Bill Hawkins
Thu, Aug 4, 2011 8:27 AM

Could you please explain how transformer phase lags could jump?

As I understand synchronous generators tied to a common grid, it
is not possible for them to have large phase angle differences
under normal conditions. Losing a whole cycle would cause forces
that could damage the machine.

What that means is, that if the two locations representing the
red and green traces are on the same grid then there should be
less than one cycle difference between them at all times.

NTP can't be causing the jumps because the difference increases
with time. You would see the displayed time difference change
as well.

Since that is not the result that you have, it is time to
calibrate your equipment. I'd start with the line frequency sensor
looking for dropped cycles.

It's possible that different computers running different other
programs could drop different numbers of points. What are they
doing when the steep drops in difference occur?

Is anyone else running a similar experiment?

Bill Hawkins

-----Original Message-----
From: Poul-Henning Kamp
Sent: Thursday, August 04, 2011 1:13 AM

The jumps in the difference looks a lot like transformer phase-lag
in the grid, but the real test is to collect more data and see if
you ever see the difference move the other way.

Could you please explain how transformer phase lags could jump? As I understand synchronous generators tied to a common grid, it is not possible for them to have large phase angle differences under normal conditions. Losing a whole cycle would cause forces that could damage the machine. What that means is, that if the two locations representing the red and green traces are on the same grid then there should be less than one cycle difference between them at all times. NTP can't be causing the jumps because the difference increases with time. You would see the displayed time difference change as well. Since that is not the result that you have, it is time to calibrate your equipment. I'd start with the line frequency sensor looking for dropped cycles. It's possible that different computers running different other programs could drop different numbers of points. What are they doing when the steep drops in difference occur? Is anyone else running a similar experiment? Bill Hawkins -----Original Message----- From: Poul-Henning Kamp Sent: Thursday, August 04, 2011 1:13 AM The jumps in the difference looks a lot like transformer phase-lag in the grid, but the real test is to collect more data and see if you ever see the difference move the other way.
PK
Poul-Henning Kamp
Thu, Aug 4, 2011 8:31 AM

In message E9DFD046D648474891D49E82D6BD94E8@cyrus, "Bill Hawkins" writes:

Could you please explain how transformer phase lags could jump?

When one power plant ramps up and another ramps down, the power flow
reverses in some transformers and transmission lines.

If the shifts you see has this cause, you should see them in both
directions over time (weeks, months).

--
Poul-Henning Kamp      | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG        | TCP/IP since RFC 956
FreeBSD committer      | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

In message <E9DFD046D648474891D49E82D6BD94E8@cyrus>, "Bill Hawkins" writes: >Could you please explain how transformer phase lags could jump? When one power plant ramps up and another ramps down, the power flow reverses in some transformers and transmission lines. If the shifts you see has this cause, you should see them in both directions over time (weeks, months). -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence.
MD
Magnus Danielson
Thu, Aug 4, 2011 9:10 AM

On 08/04/2011 10:31 AM, Poul-Henning Kamp wrote:

In messageE9DFD046D648474891D49E82D6BD94E8@cyrus, "Bill Hawkins" writes:

Could you please explain how transformer phase lags could jump?

When one power plant ramps up and another ramps down, the power flow
reverses in some transformers and transmission lines.

If the shifts you see has this cause, you should see them in both
directions over time (weeks, months).

Notice how the big jumps down matches the beginning of the steep
direction down. It is interesting to note that there is also a tendency
for the diff curve to go down even for the upward direction.

Notice how the diff moves upwards only occasionally and with a much
smaller amplitude.

Could there be a systematic cycle slip one one of the sites?

I would hook up the probes on the same site such that they see the same
signal to see if there is a difference between them.

Cheers,
Magnus

On 08/04/2011 10:31 AM, Poul-Henning Kamp wrote: > In message<E9DFD046D648474891D49E82D6BD94E8@cyrus>, "Bill Hawkins" writes: > >> Could you please explain how transformer phase lags could jump? > > When one power plant ramps up and another ramps down, the power flow > reverses in some transformers and transmission lines. > > If the shifts you see has this cause, you should see them in both > directions over time (weeks, months). > Notice how the big jumps down matches the beginning of the steep direction down. It is interesting to note that there is also a tendency for the diff curve to go down even for the upward direction. Notice how the diff moves upwards only occasionally and with a much smaller amplitude. Could there be a systematic cycle slip one one of the sites? I would hook up the probes on the same site such that they see the same signal to see if there is a difference between them. Cheers, Magnus
SN
Scott Newell
Thu, Aug 4, 2011 2:48 PM

At 03:27 AM 8/4/2011, Bill Hawkins wrote:

What that means is, that if the two locations representing the
red and green traces are on the same grid then there should be
less than one cycle difference between them at all times.

That's what I was expecting.

NTP can't be causing the jumps because the difference increases
with time. You would see the displayed time difference change
as well.

Agreed.  Both machines are running Munin, which makes pretty graphs
of ntp data.  tock shows ntp kernel estimated error of (avg/min/max)
1.72ms / 538us / 6.43ms over the last week.  sparc is 1.37ms / 493us / 3.18ms.

I should copy the Munin data to the webserver tonight...it's also
doing live plots of the 60Hz error and frequency (1s, 60s, 600s, and
3600s moving averages).

Since that is not the result that you have, it is time to
calibrate your equipment. I'd start with the line frequency sensor
looking for dropped cycles.

tock might have dirty power--the embedded device shut down a couple
of times last month until I put a batt on it.  Then again, someone
may have turned it off for me.  sparc and its front end are on a
consumer grade UPS, tock and its front end are not.  Now there is a
big old Sola line conditioner that I could add to tock at lunch.

It's possible that different computers running different other
programs could drop different numbers of points. What are they
doing when the steep drops in difference occur?

I doubt that the computers could be at fault.  The front-end device
is counting cycles and transmitting it to the computer serial port
("+0123456789\r"), and then the computer timestamps the start char
("+") and reads the cycle count.

My new $10 webcam isn't setup yet (at sparc's location), so I don't
have my backup timelapse clock photo data source available.

I'm considering mods to the embedded hardware to increase noise
rejection--wait for 60 hz edge, start timer for (say) 15 ms, don't
look for next edge until timer expires.

--
newell  N5TNL

At 03:27 AM 8/4/2011, Bill Hawkins wrote: >What that means is, that if the two locations representing the >red and green traces are on the same grid then there should be >less than one cycle difference between them at all times. That's what I was expecting. >NTP can't be causing the jumps because the difference increases >with time. You would see the displayed time difference change >as well. Agreed. Both machines are running Munin, which makes pretty graphs of ntp data. tock shows ntp kernel estimated error of (avg/min/max) 1.72ms / 538us / 6.43ms over the last week. sparc is 1.37ms / 493us / 3.18ms. I should copy the Munin data to the webserver tonight...it's also doing live plots of the 60Hz error and frequency (1s, 60s, 600s, and 3600s moving averages). >Since that is not the result that you have, it is time to >calibrate your equipment. I'd start with the line frequency sensor >looking for dropped cycles. tock might have dirty power--the embedded device shut down a couple of times last month until I put a batt on it. Then again, someone may have turned it off for me. sparc and its front end are on a consumer grade UPS, tock and its front end are not. Now there is a big old Sola line conditioner that I could add to tock at lunch. >It's possible that different computers running different other >programs could drop different numbers of points. What are they >doing when the steep drops in difference occur? I doubt that the computers could be at fault. The front-end device is counting cycles and transmitting it to the computer serial port ("+0123456789\r"), and then the computer timestamps the start char ("+") and reads the cycle count. My new $10 webcam isn't setup yet (at sparc's location), so I don't have my backup timelapse clock photo data source available. I'm considering mods to the embedded hardware to increase noise rejection--wait for 60 hz edge, start timer for (say) 15 ms, don't look for next edge until timer expires. -- newell N5TNL