On Wed, Apr 17, 2013 at 08:31:49AM +0000, Mark C. Stephens wrote:

Interleaving makes sense for performance reasons and
usually happens at bit or byte level, but I'm not (yet)
convinced that the data was dumped correctly.

Assuming that the Z3801A and Z3805A ROM contents is
somewhat similar (1x512k = 4x128k), let's take a look
at the entropy of the data:

$ cat z3801a.bin | xz --stdout | wc -c
110020

$ cat z3805-*.bin | xz --stdout | wc -c
60436

This rough estimate on the amount of non redundant
data shows that the z3801a ROM contains almost twice
as much data than the z3805 ROM dumps together.

$ for n in z3805-*.bin; do xz --stdout $n | wc -c; done
25108
30724
24712
22812

This shows that the data is almost evenly distribute
between the four dumps, so some kind of interleaving
is obviously present and none of the files are purely
random or contain just padding or similar like:

$ dd if=/dev/urandom bs=1k count=128 | xz --stdout | wc -c
131140
$ dd if=/dev/zero bs=1k count=128 | xz --stdout | wc -c
152

Now looking at the data itself, it is simple to see
that there is a positional redundancy present:

$ for n in z3805-*.bin; do xxd -g 1 $n | head -1; done
0000000: 6f ff 00 05 04 05 00 05 10 05 00 05 28 05 00 05  o...........(...
0000000: 6f 10 00 05 04 fe 00 05 10 00 00 05 28 50 00 05  o...........(P..
0000000: 6f a0 00 05 04 00 00 05 10 42 00 05 28 42 00 05  o........B..(B..
0000000: 6f 00 00 05 04 ff 00 05 10 00 00 05 28 05 00 05  o...........(...

Columns 1, 3, 4, 5, 7, 8, 9 ... are all equal, only
the columns 2, 6, 10, 14 ... contain different values
across the files, which basically means that 3 out of
4 bytes are completely identical between the 4 dumps.
(this is, where I think something might have gone
wrong with the dumps :)

Anyway, extracting the data from the columns where the
dumps actually differ gives the following data [1]:

$ xxd col.data | head -4
0000000: ff10 a000 05fe 00ff 0500 4200 0550 4205  ..........B..PB.
0000010: 0510 0000 05b4 1105 0510 0000 05b4 0005  ................
0000020: 0510 1100 05b4 0005 0510 0000 05b4 1105  ................
0000030: 1310 0000 05b4 0005 0510 1100 05b4 0005  ................

Comparing this with the dump from z3801a ROM:

$ xxd z3801a.bin | head -4
0000000: 0010 fffe 0000 0550 0010 05b4 0010 05b4  .......P........
0000010: 0010 05b4 0010 05b4 0010 05b4 0010 05b4  ................
0000020: 0010 05b4 0010 05b4 0010 05b4 0010 05b4  ................
0000030: 0010 05b4 0010 05b4 0010 05b4 0010 05b4  ................

Suggests that there is some kind of reordering required
to get the similar structure present in the hardware
vector table.

The following mapping seems to do the trick:
Pick every odd byte from the data and switch every
two resulting bytes, or as pseudocode:

for (n = 0; n < 2; n++) {
    for (i = 0; i < dlen; i += 4) {
        j = i/2;

        split[n][j + 0] = data[i + n + 2];
        split[n][j + 1] = data[i + n + 0];
    }
}

The resulting data [2] seems to contain meaningful
strings and binary code similar to the z3801a ROM:

$ strings split.data | grep psos

@(#)$Header: psos_indep.h,v 1.5 95/03/24 15:24:19 da oe uxn d

@(#)$Header: psos_indep.h,v 1.5 95/03/24 15:24:19 da oe uxn d

HTH,
Herbert

[1] http://vserver.13thfloor.at/Stuff/VARIOUS/Z3805/col1.data
[2] http://vserver.13thfloor.at/Stuff/VARIOUS/Z3805/split1.data

It is possible I didn't take the dump correctly, or there is a problem with my burner.
I chose Binary when saving files.
Perhaps, the other possibility is those flash chips aren't the actual firmware, just storage.
The actual firmware (program) flash chip could be located elsewhere.
However, that would seem a bit strange to me, why socket transient memory and not the actual program itself?
How should I have taken the flash dumps?

-mark

-----Original Message-----
From: time-nuts-bounces@febo.com [mailto:time-nuts-bounces@febo.com] On Behalf Of Herbert Poetzl
Sent: Thursday, 18 April 2013 12:43 AM
To: Discussion of precise time and frequency measurement
Subject: Re: [time-nuts] Z3805A flash dump.

On Wed, Apr 17, 2013 at 08:31:49AM +0000, Mark C. Stephens wrote:

Interleaving makes sense for performance reasons and usually happens at bit or byte level, but I'm not (yet) convinced that the data was dumped correctly.

Assuming that the Z3801A and Z3805A ROM contents is somewhat similar (1x512k = 4x128k), let's take a look at the entropy of the data:

$ cat z3801a.bin | xz --stdout | wc -c
110020

$ cat z3805-*.bin | xz --stdout | wc -c
60436

This rough estimate on the amount of non redundant data shows that the z3801a ROM contains almost twice as much data than the z3805 ROM dumps together.

$ for n in z3805-*.bin; do xz --stdout $n | wc -c; done
25108
30724
24712
22812

This shows that the data is almost evenly distribute between the four dumps, so some kind of interleaving is obviously present and none of the files are purely random or contain just padding or similar like:

$ dd if=/dev/urandom bs=1k count=128 | xz --stdout | wc -c
131140
$ dd if=/dev/zero bs=1k count=128 | xz --stdout | wc -c
152

Now looking at the data itself, it is simple to see that there is a positional redundancy present:

$ for n in z3805-*.bin; do xxd -g 1 $n | head -1; done
0000000: 6f ff 00 05 04 05 00 05 10 05 00 05 28 05 00 05  o...........(...
0000000: 6f 10 00 05 04 fe 00 05 10 00 00 05 28 50 00 05  o...........(P..
0000000: 6f a0 00 05 04 00 00 05 10 42 00 05 28 42 00 05  o........B..(B..
0000000: 6f 00 00 05 04 ff 00 05 10 00 00 05 28 05 00 05  o...........(...

Columns 1, 3, 4, 5, 7, 8, 9 ... are all equal, only the columns 2, 6, 10, 14 ... contain different values across the files, which basically means that 3 out of
4 bytes are completely identical between the 4 dumps.
(this is, where I think something might have gone wrong with the dumps :)

Anyway, extracting the data from the columns where the dumps actually differ gives the following data [1]:

$ xxd col.data | head -4
0000000: ff10 a000 05fe 00ff 0500 4200 0550 4205  ..........B..PB.
0000010: 0510 0000 05b4 1105 0510 0000 05b4 0005  ................
0000020: 0510 1100 05b4 0005 0510 0000 05b4 1105  ................
0000030: 1310 0000 05b4 0005 0510 1100 05b4 0005  ................

Comparing this with the dump from z3801a ROM:

$ xxd z3801a.bin | head -4
0000000: 0010 fffe 0000 0550 0010 05b4 0010 05b4  .......P........
0000010: 0010 05b4 0010 05b4 0010 05b4 0010 05b4  ................
0000020: 0010 05b4 0010 05b4 0010 05b4 0010 05b4  ................
0000030: 0010 05b4 0010 05b4 0010 05b4 0010 05b4  ................

Suggests that there is some kind of reordering required to get the similar structure present in the hardware vector table.

The following mapping seems to do the trick:
Pick every odd byte from the data and switch every two resulting bytes, or as pseudocode:

for (n = 0; n < 2; n++) {
    for (i = 0; i < dlen; i += 4) {
        j = i/2;

        split[n][j + 0] = data[i + n + 2];
        split[n][j + 1] = data[i + n + 0];
    }
}

The resulting data [2] seems to contain meaningful strings and binary code similar to the z3801a ROM:

$ strings split.data | grep psos

@(#)$Header: psos_indep.h,v 1.5 95/03/24 15:24:19 da oe uxn d

@(#)$Header: psos_indep.h,v 1.5 95/03/24 15:24:19 da oe uxn d

HTH,
Herbert

[1] http://vserver.13thfloor.at/Stuff/VARIOUS/Z3805/col1.data
[2] http://vserver.13thfloor.at/Stuff/VARIOUS/Z3805/split1.data

time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.

I suspect your reader is not configured correctly. For example, look at the first 80 bytes of each BIN file; they should not be so identical.

After you fix that, then combining low/high bytes will give you what you want. See http://leapsecond.com/museum/z3801a/eeprom.htm for an example of the result.

/tvb

----- Original Message -----
From: "Mark C. Stephens" marks@non-stop.com.au
To: time-nuts@febo.com
Sent: Wednesday, April 17, 2013 1:31 AM
Subject: [time-nuts] Z3805A flash dump.

In message AA42DD206D5541258C6A474D42B3C0F1@pc52, "Tom Van Baak" writes:

If anybody feels like disassembling the code, I have started a python
based "reverse engineering" kit called "PyReveng" for such jobs:

https://github.com/bsdphk/PyRevEng

--
Poul-Henning Kamp      | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG        | TCP/IP since RFC 956
FreeBSD committer      | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

Hi Tom:

What CPU chip does the Z3805 use?

When working with the PIC microcontrollers it's possible to generate pseudo source code that can be assembled to yield
the actual binary image.
By placing a comment at the end of each line that contains the original binary hex you can audit that no mistakes have
been made.
Then it's just a process of adding names to replace the hex links and after some work you have a commented source listing.

But, it takes an assembler and source code generator for the CPU that's being used.

Have Fun,

Brooke Clarke
http://www.PRC68.com
http://www.end2partygovernment.com/2012Issues.html

Tom Van Baak wrote:

On Wed, Apr 17, 2013 at 03:38:56PM +0000, Mark C. Stephens wrote:

That's always a good choice :)

Unlikely, otherwise I would not have been able to
reconstruct the (maybe?) partial data from your dumps.
(see the files I linked in the previous reply)

I do not even know what ROM/Flash? chips are used
in this device, but from your reply I conclude that
they are socketed and you were able to read them
via some kind of EEPROM reader/burner?

If so, given that the 'reader' supports the chips
in question, I'd say your dumps are probably fine.

A good method is to dump each chip several times
(if possible in different ways) and compare the
results.

Assuming that everything was dumped correctly, it
might simply be that the device uses only a small
portion of the ROM/Flash for PSOS (operating system)
and the rest for data storage or checksumming.

A schematic how the chips connect to the CPU might
sched some light on the data stored/used as might
more information about the involved components.

best,
Herbert

Hi Tom,

I see my 'Reader' has different options.
It's a leaper 3C from Taiwan.
You can download the software here: http://www.vk2hmc.net/blog/?wpfb_dl=6 and run it in the demo mode.
I guess you could even load the ROM images I took and save them in different format.

Under parameters, there are a number of options selectable for data set.

8 Bit, 16 bit (even and odd) and 32bit (1st, 2nd, 3rd, 4th)
I was using 8 bit. Perhaps I should have been using one of the other options?

mark

-----Original Message-----
From: time-nuts-bounces@febo.com [mailto:time-nuts-bounces@febo.com] On Behalf Of Tom Van Baak
Sent: Thursday, 18 April 2013 2:21 AM
To: time-nuts@febo.com
Subject: Re: [time-nuts] Z3805A flash dump.

I suspect your reader is not configured correctly. For example, look at the first 80 bytes of each BIN file; they should not be so identical.

After you fix that, then combining low/high bytes will give you what you want. See http://leapsecond.com/museum/z3801a/eeprom.htm for an example of the result.

/tvb

----- Original Message -----
From: "Mark C. Stephens" marks@non-stop.com.au
To: time-nuts@febo.com
Sent: Wednesday, April 17, 2013 1:31 AM
Subject: [time-nuts] Z3805A flash dump.

time-nuts mailing list -- time-nuts@febo.com
To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.