In message: 495F7285.3040003@erols.com
Chuck Harris cfharris@erols.com writes:
: christopher hoover wrote:
: > Hal Murray wrote:
: >> Two of my Linux systems hung.  One was running a 2.6.25 kernel and one
: >> 2.6.26.  A system running 2.6.23 worked fine.  I saw a couple of notes
: >> on
: >> comp.protocols.time.ntp about Linux systems locking up.  One said that
: >> it was
: >> a kernel bug in ntp.c but I haven't seen any details.
: >
: > None of mine (many dozens) hung.    This is typical:
: >
: > ch@snaggle:~$ uname -a
: > Linux snaggle.murgatroid.com 2.6.26-1-amd64 #1 SMP Mon Dec 15 19:40:58 UTC
: > 2008 x86_64 GNU/Linux
: > ch@snaggle:~$ dmesg | grep leap
: > [844362.415072] Clock: inserting leap second 23:59:60 UTC
: > ch@snaggle:~$
: >
: >
: > -ch
:
: None of my linux systems hung either!  My typical message was:
:
: $ dmesg | grep leap
: [6181904.453104] Clock: inserting leap second 23:59:60 UTC
:
: The message implies that linux clocks counted:
:
: 58..59..60..00..01
:
: Which would not be the POSIX way.

That message doesn't imply that at all...  Well, maybe it is the
implication, but POSIX CANNOT count that way.  It is number where
23:59:60 maps to, through normalization and *mktime, 00:00:00 (or maps
to 23:59:59 if you are ticking time, which isn't required to do the
mktime stuff).

Warner

In message: m31vvks3e9.fsf@lugabout.jhcloos.org
James Cloos cloos@jhcloos.com writes:
: >>>>> "Chuck" == Chuck Harris cfharris@erols.com writes:
:
: Chuck> The message implies that linux clocks counted:
:
: Chuck> 58..59..60..00..01
:
: Chuck> Which would not be the POSIX way.
:
: No, the linux clocks counted:
:
: 1230768021..1230768022..1230768023..1230768024..1230768025
:
: where 1230768024 is 2009-01-01 00:00:00 UTC.
:
: What gets output by any given userland apps depends on those apps.
:
: If one uses the Olsen tz database, the right zonefiles rather than the
: posix zonefiles, and a libc such as glibc, then one will have seen the
: seconds tick off 58..59..60..00..01.
:
: But that is purely a userland issue.
:
: If one uses the (lobotomized) posix zonefiles, one will have seen the
: seconds tick off 21..22..23..24..25.
:
: (Interesting coincidence there, that 1970 through 2008 (inclusive) has a
: number of days divisible by 5.  Which makes for a nice, even 1230768000
: seconds, were there no leap seconds.)

That doesn't match POSIX's mandated behavior...  time_t % 86400 == 0
at midnight is an invariant that's violated by the above sequence.

Warner

In message: m3vdswqmb5.fsf@lugabout.jhcloos.org
James Cloos cloos@jhcloos.com writes:
: >>>>> "Warner" == M Warner Losh imp@bsdimp.com writes:
:
: Warner> That doesn't match POSIX's mandated behavior...  time_t % 86400 == 0
: Warner> at midnight is an invariant that's violated by the above sequence.
:
: By which sequence?

The sequence where midnight % 86400 isn't 0.

: At every point in time where time_t % 86400 == 0 is true gmtime(2), when
: using no zoneinfo file or when using a posix zoneinfo file, will return
: a struct tm where the time of day is 00:00:00.
:
: (time_t)1230768024 was 2009-01-01 00:00:00 right/UTC
: (time_t)1230768000 was 2009-01-01 00:00:00 posix/UTC
:
: and the real 2009-01-01 00:00:00 UTC was exacly 1230768024 si seconds
: after 1970-01-01 00:00:00 UTC.

Correct.  The 'right' way here isn't POSIX compliant.  POSIX has no
leap seconds at all.  They don't exist.

: That this leap second was (time_t)1230768023 is is a simple fact of how
: long it has been since the start of 1970.
:
: That POSIX pretends that 2009 UTC started 24 seconds earlier than it
: did is a bug.

It is the standard.  It isn't desirable behavior, but it is what is
mandated by the standard.  You can argue it is a bug, and I might
agree with you, but that won't make it any less the standard.  And
explicitly part of the standard after much arguments in committee.  It
is one of the things horribly broken by POSIX.

Also, you need to run a hacked ntpd, because the ntp time stamps
follow the posix mandate, not the TAI-like second count...

: Anyone who has some requirement to exactly match POSIX as published can
: use the posix zoneinfo files and have time_t % 86400 == 0 as midnight
: “UTC” every day.  Their idea of time will be off, but they get to keep
: their fixed-sized intervals.

The trouble is that there's a lot of code that depends on this
behavior...

: Anyone who wants to track the real UTC can do so using the right
: zoneinfo files.  By doing so they explicitly choose to ignore
: that particular bit of nonsense in POSIX, but that is OK since it
: is their choice.
:
: Everyone gets to choose which they prefer.

Correct.

Of course, how do programs that run for years get updated leap second
information so they continue to produce the correct times?  I've never
understood how that part of the 'right' files works...

Warner

This is a minor little pedantic followup from an answer I gave before
to correct a minor little quibble about the historic UTC.  If you are
uninterested, hit 'd' now :)

In message: m3vdswqmb5.fsf@lugabout.jhcloos.org
James Cloos cloos@jhcloos.com writes:
: >>>>> "Warner" == M Warner Losh imp@bsdimp.com writes:
:
: Warner> That doesn't match POSIX's mandated behavior...  time_t % 86400 == 0
: Warner> at midnight is an invariant that's violated by the above sequence.
:
: By which sequence?
:
: At every point in time where time_t % 86400 == 0 is true gmtime(2), when
: using no zoneinfo file or when using a posix zoneinfo file, will return
: a struct tm where the time of day is 00:00:00.
:
: (time_t)1230768024 was 2009-01-01 00:00:00 right/UTC
: (time_t)1230768000 was 2009-01-01 00:00:00 posix/UTC
:
: and the real 2009-01-01 00:00:00 UTC was exacly 1230768024 si seconds
: after 1970-01-01 00:00:00 UTC.

Except I don't think that 1970-01-01 00:00:00 UTC isn't what you think
it is.  While there have been 24 leap seconds since 1972-01-01 when
the practice started, the divergence between TAI and UTC at 1970-01-01
00:00:00 was more like 8.1s[*] (it was fixed at 10.0 exactly in 1972).
So there really have been an additional 1.9s since then.  So the
number is closer to 1230768025.9s since 1970-01-01 00:00:00 UTC for
the second after the last leap second.

The time before 1972 was when the atomic clocks were run a little slow
to account for the variations in the earth's orbit.  See for example
the table from:

http://maia.usno.navy.mil/ser7/tai-utc.dat

So what you've done is created a new time scale that is a UTC from
1972 forward, but a simplified form of UTC prior to 1972 that didn't
match what UTC was doing then.  Yet another hazard of high precision
time keeping that few people get right (to pound on my leap seconds
are hard drum again).  An understandable simplification, to be true,
and one that's often made...

Warner

[*] I didn't do the math today, and the 8.1s figure is from my memory
only.  Someone with more time on their hands than I can compute the
exact offset...

In message: 26842.1231015338@critter.freebsd.dk
"Poul-Henning Kamp" phk@phk.freebsd.dk writes:
: In message 495FB615.9080200@rubidium.dyndns.org, Magnus Danielson writes:
:
: >> Having a message from ntp.c that says there was a leap
: >> to HH:MM:60 implies that HH:MM:60 is a valid time as far
: >> as ntp.c's author is concerned.
: >
: >It is valid UTC time, not valid POSIX time, which are two different things.
:
: Well, it is a valid POSIX time, but it means a second later than
: desired in this case, because the 60 is taken as 60 seconds, and
: folded into a minute-roll-over.

It is a valid POSIX struct tm time.  But it doesn't represent a leap
second.  Instead, like you say, it wraps to the next minute.

There are not POSIXly compliant time_t values that will map to the
leap second value (23:59:60).  It is not possible to represent the
leap second in a time_t.

: >> Having used unix since edition V, I am also aware of how unix
: >> systems count off seconds since the epoch 1/1/1970.  But that
: >> really is immaterial to the discussion.
:
: No, that is actually the crux of the matter...

Yes.  Such a simple definition gives so many possible ways to
interpret it.  The POSIX "there are no leap seconds" way.  The prior +
accumulated leap seconds.  And also the prior, plus the extra time
that accumulated between 1970 and 1972 in the UTC time scale...  But
that last one is kinda hard since it isn't an whole number of
seconds.

I've seen timescales try to use all of these definitions at one point
in my career or another (plus a boatload of others that seemed like a
good idea at the time to the inventors).

Warner

In message: m3mye8qen3.fsf@lugabout.jhcloos.org
James Cloos cloos@jhcloos.com writes:
: >>>>> "Warner" == M Warner Losh imp@bsdimp.com writes:
:
: Jim> By which sequence?
:
: Warner> The sequence where midnight % 86400 isn't 0.
:
: MY appologies, but that isn't narrowing it for me.  POSIX only cares
: about POSIX midnight, not UTC midnight, so the fact that it was already
: past PODIX midnight when the leap second and UTC midnight happened is
: irrelevant to POSIX.

posix midnight and utc midnight are the same things.  You had said
that the system time was returned as ....24 at UTC 2009-01-01
00:00:00, which isn't posixly correct.

: Warner> Also, you need to run a hacked ntpd, because the ntp time stamps
: Warner> follow the posix mandate, not the TAI-like second count...
:
: Huh?  rfc1305 says 32.32 bit fixed point seconds since 1900-01-01
: 00:00:00, and of course there is the newer 64.64 bit fixed point.
:
: The only question, then, is whether ntp and UTC agree on just when
: 1900-01-01T00:00:00 was. ;^)

ntpd needs to convert the time from the kernel to the 32.32 number.
To do that, by default it assumes a POSIXly correct time_t.  That's
the only point I was making.  If a different number is returned (one
with leap seconds added in), then ntpd has to cope.

: Warner> how do programs that run for years get updated leap second
: Warner> information so they continue to produce the correct times?  I've
: Warner> never understood how that part of the 'right' files works...
:
: That is libc-dependent.  I've not looked at the src in a while (many
: months), but IIRC glibc opens the file every time it needs it, so it
: will see a new zoneinfo database as soon as they are written to the
: filesystem.
:
: Other libcs might do things differently.  As an example, I don't think
: klibc uses the zoneinfo files at all; dietlibc and uclibc may not
: either.  And obviously the BSDs' libcs handle things quite differently
: than glibc.  (GNU on BSD kernels is not uncommon; I'm sure I've read
: about people doing one of the BSD userlands on a linux kernel, too.)
: But at least for glibc it just works.
:
: And, of course, said zoneinfo updates are needed anyway every time the
: politicians muck with the timezones.  Libc also has to deal with changes
: to the TZ environmental variable.  Another reason to re-read the
: zoneinfo files as necessary.  (It is possible that glibc only reopens if
: the stat(2) data changes; again, it has been a long time since I last
: read that part of the glibc src.)

Yea, I was wondering if it did a stat on every time conversion, or if
it batched them somehow.  Doing one on every conversion seems very
heavy-weight to me...  Hence my question of how do the updated
zoneinfo files happen, and how does the application get notified of
the changed of leap info in case they have already computed a time
that would be affected by the new leap information.

Warner

In message: 495FD637.5030105@erols.com
Chuck Harris cfharris@erols.com writes:
: Poul-Henning Kamp wrote:
: > In message 495FB615.9080200@rubidium.dyndns.org, Magnus Danielson writes:
: >
: >>> Having a message from ntp.c that says there was a leap
: >>> to HH:MM:60 implies that HH:MM:60 is a valid time as far
: >>> as ntp.c's author is concerned.
: >> It is valid UTC time, not valid POSIX time, which are two different things.
: >
: > Well, it is a valid POSIX time, but it means a second later than
: > desired in this case, because the 60 is taken as 60 seconds, and
: > folded into a minute-roll-over.
: >
: >>> Having used unix since edition V, I am also aware of how unix
: >>> systems count off seconds since the epoch 1/1/1970.  But that
: >>> really is immaterial to the discussion.
: >
: > No, that is actually the crux of the matter...
:
: Ok, that is news to me.  Are you saying that (pulling a number out of
: the air) time_t = 21120123 could be followed by 21120123 on a year where
: we added a leap second?

Yes.  Leap seconds don't exist in POSIX time_t, so the pedantically
proper value is undefined.  Most implementations repeat a second
(either 59 or the 00 second) to cope with this omission.

: It is my understanding that it cannot.  I believe that time_t must
: increment by one as each second passes, and must contain the number of
: seconds that have passed since the unix epoch on 1/1/1970... without
: regard for leap seconds.

That isn't what POSIX says.  POSIX is very clear that leap seconds do
not exist, and therefore the correct answer for the first second of a
year (or of any day) always ends in '00'.

: I was of the understanding that the problem was in how the UTC time was
: calculated from time_t, and the converse.

The problem is that the conversion of time_t to a 'broken down' UTC
time isn't 1:1 or onto.

Warner

In message: 495FF91C.60401@rubidium.dyndns.org
Magnus Danielson magnus@rubidium.dyndns.org writes:
: M. Warner Losh skrev:
: > In message: m3mye8qen3.fsf@lugabout.jhcloos.org
: >            James Cloos cloos@jhcloos.com writes:
: > : >>>>> "Warner" == M Warner Losh imp@bsdimp.com writes:
: > :
: > : Jim> By which sequence?
: > :
: > : Warner> The sequence where midnight % 86400 isn't 0.
: > :
: > : MY appologies, but that isn't narrowing it for me.  POSIX only cares
: > : about POSIX midnight, not UTC midnight, so the fact that it was already
: > : past PODIX midnight when the leap second and UTC midnight happened is
: > : irrelevant to POSIX.
: >
: > posix midnight and utc midnight are the same things.  You had said
: > that the system time was returned as ....24 at UTC 2009-01-01
: > 00:00:00, which isn't posixly correct.
:
: Um... no. That's the hacked POSIX interpretation, not the POSIX standard.
:
: We have at least three POSIX interpretations here.
:
: One which has UTC rubber seconds from 1970 to 1972 and from then true SI
: seconds from 1972.
: One which has true SI seconds from 1970.
: One which has UTC tracking in pieces and is slid "sideways" to make
: midnight match UTC midnight.
:
: The two first ones is interpretations of POSIX over UTC variations. The
: third one is a hack of POSIX to make it kind of work anyway with NTP.
: Only with the third interpretation POSIX midnight and UTC midnight is
: the same.
:
: Now, which of them is "right"?

Midnight must be xxxx00.  POSIX says so explicitly because leap
seconds do not exist in POSIX.  The committee has issued a very
explicit addendum to this effect.

Which one is more desirable?  Well, that's a matter of debate, but
which one is POSIXly correct isn't.

Warner

In message: 4960027E.1000103@erols.com
Chuck Harris cfharris@erols.com writes:
: Poul-Henning Kamp wrote:
: > In message 495FD637.5030105@erols.com, Chuck Harris writes:
: >
: >> Ok, that is news to me.  Are you saying that (pulling a number out of
: >> the air) time_t = 21120123 could be followed by 21120123 on a year where
: >> we added a leap second?
: >
: > Apart from the number, that is exactly what happens: The last
: > second of the (UTC) day is recycled twice.
:
: As far as I remember, and as far as I can tell, what you are saying
: violates both the unix and POSIX definition of time_t.
:
: So to check, I pulled out both of my K&R editions of "The C programming Language"
: and I did a quick google on time_t, and all of the sources I have found
: concur that time_t is the number of seconds since 1/1/1970 UTC without
: regard to leap seconds.

That's exactly what Poul is saying.  Without regard to leap seconds
means that they don't exist and do not count in POSIX.  A midnight
time_t % 86400 must == 0, or it isn't POSIX.

: When did this change?

It never was clearly defined before POSIX, and POSIX made at least 4
muddled attempts to define it.

Warner

In message: m3hc4gqdjb.fsf@lugabout.jhcloos.org
James Cloos cloos@jhcloos.com writes:
: >>>>> "Warner" == M Warner Losh imp@bsdimp.com writes:
:
: Warner> So what you've done is created a new time scale that is a UTC
: Warner> from 1972 forward, but a simplified form of UTC prior to 1972
: Warner> that didn't match what UTC was doing then.
:
: Grrr!  Except s/you/they/; I didn't invent.

Yea, I was speaking a bit rhetorically :-)

: So right isn't quite, err, right.  I wonder whether the Olsen db can
: be fixed to account for that?  right/UTC and posix/UTC currently are
: identical for all (time_t)LONG_MIN <= time_t < 78796800.

Yes.  I'd forgotten that the Olsen db doesn't deal with rubber seconds
at all.  It is a pain in the *** to try to do that, and of dubious
value.  I tried once to create a library that coped with them, but
gave up when I realized it wasn't a useful problem to solve.

: Thanks for the reminder.  I had forgotten that entirely.  (And am
: only just vaguely remembering that I used to know that fact.  [SIGH])

It is certainly underdocumented...

: Warner> Yet another hazard of high precision time keeping that few
: Warner> people get right
:
: Part of what makes this list's name so appropriate is just how hard
: it is, all things considered.  That is also what makes it enjoyable.

Yes.  Very enjoyable.  Of course, I could live without all this
complexity, frankly, and be happier.

: Warner> An understandable simplification, to be true, and one that's
: Warner> often made...
:
: Often, I'm sure, because not all sources document/remember that fact.

Yea.  In another life, I defined a datum as 'number of SI seconds
since 01-01-1972 00:00:00 UTC + 63072000'.  Which is what we're
talking about here, no?  This is number of seconds since 1970, with
the 'oddball' rubber seconds counting as SI seconds.

Warner