In response to Oz-in-DFW

Spot on; I hope I'd made it clear earlier, but perhaps I've been communicating poorly.  My goals w.r.t. to sync to UTC and w.r.t. holdover are very loose.

I do need time sync intra-cluster to be tight (sub millisecond, 100 nano as a stretch goal).  UTC sync can comparatively be terrible; 10-1 ms is fine, and I can live with "bad NTP, 100 ms" if I must.  From specs, /really/ good quartz is my limit and /good/ quartz is acceptable, so long as it doesn't mess with the intra-node PTP tightness.  I'm mostly looking at TCXO options. OCXO isn't out of the question, but rubidium doesn't seem to give $/value.

Thanks for the info.  PTP isn't as well understood/documented as NTP, so I've not been as certain about my decisions. Of course, that is fair for a relatively new standard.

Currently, I think my two best options are: 1) CDMA enabled PTP appliance (set and forget), or 2) PTP appliance running as stratum 2 from good NTP.

Thanks to everybody for the feedback.

-joe

Hi

Any CDMA device is a really bad idea. It’s perfectly legal to run CDMA several seconds out of sync with UTC. There are a number of networks that
do exactly this.

PTP is fine as long as all your network infrastructure (switches / hubs / routers) is PTP enabled. If any of your links are not fully PTP stamping devices
you will get into time issues as your network load varies.

Bob

"Tucek, Joseph" writes:

If you want intra-cluster at sub-millisecond, NTP is possible, and that
should be trivial with PTP.

I've been attending the ISPCS plugfests for the past few years' time and
I've been making sure that we can "take time" from upstream NTP or PTP,
and distribute that time via NTP or PTP.

Network Time Foundation "includes" the NTP Project, Ntimed (and PHK
plans to at least look at PTP support sometime), and 2 PTP projects -
PTPd, which is designed to be portable and generally useful, and Linux
PTP, which is designed to be optimized for the latest Linux kernels.

Either should be fine.

I saw you can't run an antenna wire from where you'll be, but perhaps a
lan cable?  That might go to either a GPS device or to a small NTP or
PTP device.

NTF is working to improve the products under its umbrella all the time,
and we're seriously resource-constrained.  OK, we're disturbingly
resource-constrained.  While the PTPd folks seem to have enough
developer resources and Richard Cochran has not complained about the
developer resources for Linux PTP, none of the projects have adequate
documentation writers.

Guess what I think would be a Swell Idea?

Harlan Stenn stenn@ntp.org
http://networktimefoundation.org - be a member!

Harlan,

On 05/14/2015 01:34 AM, Harlan Stenn wrote:

Work is silently being made to ensure that NTP vendors become NTF
members, and that way start to pay back for the code they use and at
least somewhat help solving the resource issues. Hope that you seen that
in your end.

Cheers,
Magnus

On 5/13/2015 4:44 PM, Tucek, Joseph wrote:

Sort of.  All I remember without combing through the notes were
subjective relative statements and no quantitative values.

There are four orders of magnitude between 1 millisecond and 100
nanoseconds - that's a heck of a stretch.  Is this what you really
mean?  What is the cluster doing that needs such good absolute time?
did you mean 100 microseconds as a stretch (one order of magnitude.)

10 e -1 milliseconds as in 100 microseconds?  This is achievable but
will require some care.  Even 10 ms is 'pretty darn good' for all but a
very few industrial applications. Most datacenter applications I've done
only guarantee 100 ms absolute.  Internal distribution is much better of
course.

It doesn't until it gets really bad.  Like shattered bad.

This begins to sound like you really don't know what you need and are
specing the best you can afford "to be sure." In my experience this is a
good way to get bitten, because you really are not sure.  Many
industrial applications require excellent relative accuracy within a
cluster. Time synchronizing chains of rotating machinery is surprisingly
demanding, but you almost don't care about absolute accuracy outside the
local clock domain. It's a rare application that /needs/ better than a
second absolute. Most of these are 'big science' projects or
infrastructure that covers a large geographical areas.  Time errors can
be catastrophic.  If you are working with one of the rare ones, you
really need to understand the real requirement and then design for that
with margin. If not, you should still be able to estimate the absolute
need and /then/ add margin to that.

PTP is both well understood and documented, but it sounds like it's not
for your industry and application. This tends to imply that it's not
time critical.

No, for a few reasons:

First, it's going away sooner than you think. Verizon says 2021, but
they are doing everything they can to accelerate that.  I'll be
surprised if it's still viable in 2018. Analog cellular was shut off
in February of 2008, but was barely useable in metro areas several
years before that. The operators shut off all but the absolute
minimum capacity to save costs and provide incentive to move to
newer technologies. Expect your cave to lose coverage much sooner
than 2021.

Second, while in-spec cellular is a good frequency reference, there
is no requirement for absolute time that you have access to.  The
time available over the air interface can be off by /minutes./ 
Typically it's within seconds of UTC and many operators now do far
better than that.  You are better off with WWVB or open Internet NTP
in terms of predictable accuracy.

Yes, or something you operate that is "network close" and has a GPS
reference.

--
mailto:oz@ozindfw.net
Oz
POB 93167
Southlake, TX 76092 (Near DFW Airport)

Hi Magnus,

I suspect you thought this was going only to me, but I'll use it for
what I hope is a short burst of illumination.

Magnus Danielson writes:

Yes, I'm hearing about this, and if it happens it will be most
helpful.

And while it will be genuinely helpful and a start, it won't be enough.

It is necessary, and not yet sufficient.

If 10 NTP vendors join NTF at the $50k/year level (as opposed to
whatever number at lower levels) NTF will almost have enough to cover
a partial combination of core staff, operating expenses, and equipment.
Core staff is just that - minimum core staff.  Throw documentation
writers, developers, Q/A test folks, sysadmin, a testing lab with a
scientist, testing gear, and sysadmin support, standards wrangling (Each
IETF meeting costs about $4k per person just to attend, and there is
ongoing work outside of the meetings.  There are IEEE and ITU meetings,
and others.), and things I'm forgetting to mention, and one discovers...

To really do the job that needs to be done we'll need a budget that is
closer to $3m/year, and we're working on ways to get there.

NTF's revenue stream has been steadily growing.  Last year NTF had
revenues of about $103k, and expenses of $104k (yes, we lost about
$1,000 last year).  This year we're continuing to grow (and NTF is still
not paying me).

Put another way, the ntp tarball is just a bit smaller than a bind-9
tarball. NTF does not have even 5% of the resources to support NTP that
ISC has to support BIND.

So yes, growing the membership at NTF is exactly what needs to happen,
and every new member noticeably helps.  And we need to do this and other
revenue-producing things even more.

Harlan Stenn stenn@ntp.org
http://networktimefoundation.org - be a member!