Empathy List Archives

JK

Jörg Knobloch

Wed, Sep 6, 2017 8:56 PM

Hi,

don't tell Ben, but a clobber build takes me 105 minutes :-(

I've just done a clobber build of Firefox and that finished after 73
minutes. Sure, we compile a few C++ files more and shuffle a few JS
files around, but how does that account for 32 minutes more? Any ideas?

Jörg.

Hi, don't tell Ben, but a clobber build takes me 105 minutes :-( I've just done a clobber build of Firefox and that finished after 73 minutes. Sure, we compile a few C++ files more and shuffle a few JS files around, but how does that account for 32 minutes more? Any ideas? Jörg.

BB

Ben Bucksch

Wed, Sep 6, 2017 9:25 PM

Jörg Knobloch wrote on 06.09.17 22:56:

don't tell Ben, but a clobber build takes me 105 minutes :-(

What? Almost 2 hours???

On recent machines (Ryzen 7 CPU, SSD Samsung 960 Evo, with enough RAM),
it takes about 12 minutes.

BTW: esp. the SSD and the 4K screen (4K TV, connected via HDMI 2.0,
running as 1:1 desktop, costs 550 Eur) made a huge difference for me.

I've just done a clobber build of Firefox and that finished after 73
minutes. Sure, we compile a few C++ files more and shuffle a few JS
files around, but how does that account for 32 minutes more? Any ideas?

TB has always cost me 25-33% more build time than FF. E.g. FF took 45
minutes, while TB took about 60 minutes. By that ratio, if FF takes 73
minutes for you, TB would take 97 minutes - not too far from 105 minutes.

As to why: No idea. Sorry.

Ben

Jörg Knobloch wrote on 06.09.17 22:56: > don't tell Ben, but a clobber build takes me 105 minutes :-( What? Almost 2 hours??? On recent machines (Ryzen 7 CPU, SSD Samsung 960 Evo, with enough RAM), it takes about 12 minutes. BTW: esp. the SSD and the 4K screen (4K TV, connected via HDMI 2.0, running as 1:1 desktop, costs 550 Eur) made a huge difference for me. > I've just done a clobber build of Firefox and that finished after 73 > minutes. Sure, we compile a few C++ files more and shuffle a few JS > files around, but how does that account for 32 minutes more? Any ideas? TB has always cost me 25-33% more build time than FF. E.g. FF took 45 minutes, while TB took about 60 minutes. By that ratio, if FF takes 73 minutes for you, TB would take 97 minutes - not too far from 105 minutes. As to why: No idea. Sorry. Ben

PK

Philipp Kewisch

Wed, Sep 6, 2017 10:44 PM

On 9/6/17 11:25 PM, Ben Bucksch wrote:

Jörg Knobloch wrote on 06.09.17 22:56:

don't tell Ben, but a clobber build takes me 105 minutes :-(

What? Almost 2 hours???

2 hours is in line with the perf I was getting on a mid 2010 mbp with
mid range components. With a mid 2015 mbp top components I am also near
the 12-20 minutes Ben mentions.

Are you running with (s)ccache with a reasonable cache size? Are you
using an SSD?

Philipp

On 9/6/17 11:25 PM, Ben Bucksch wrote: > Jörg Knobloch wrote on 06.09.17 22:56: >> don't tell Ben, but a clobber build takes me 105 minutes :-( > > > What? Almost 2 hours??? 2 hours is in line with the perf I was getting on a mid 2010 mbp with mid range components. With a mid 2015 mbp top components I am also near the 12-20 minutes Ben mentions. Are you running with (s)ccache with a reasonable cache size? Are you using an SSD? Philipp

JK

Jörg Knobloch

Thu, Sep 7, 2017 12:12 AM

On 07/09/2017 00:44, Philipp Kewisch wrote:

2 hours is in line with the perf I was getting on a mid 2010 mbp with
mid range components. With a mid 2015 mbp top components I am also near
the 12-20 minutes Ben mentions.

Interesting. I'm using a machine with an AMD A10 Series A10-7860K 4 core
x 3.60GHz bought last year, know for it's less fortunate single thread
performance. Are we talking debug or opt builds, 32 or 64 bit and which
platform? Windows is quite slow compiling.

Are you running with (s)ccache with a reasonable cache size? Are you
using an SSD?

Yes, all machines have an SSD, but the compile process is CPU bound.
Apparently Kent does a clobber build with his i7 machine with a HDD in
around 20 minutes, or so he said.

I've never heard of ccache, but asking Dr. Google it doesn't appear to
exist on Windows. Besides, we're talking about clobber builds here, no?

Jörg.

On 07/09/2017 00:44, Philipp Kewisch wrote: > 2 hours is in line with the perf I was getting on a mid 2010 mbp with > mid range components. With a mid 2015 mbp top components I am also near > the 12-20 minutes Ben mentions. Interesting. I'm using a machine with an AMD A10 Series A10-7860K 4 core x 3.60GHz bought last year, know for it's less fortunate single thread performance. Are we talking debug or opt builds, 32 or 64 bit and which platform? Windows is quite slow compiling. > Are you running with (s)ccache with a reasonable cache size? Are you > using an SSD? Yes, all machines have an SSD, but the compile process is CPU bound. Apparently Kent does a clobber build with his i7 machine with a HDD in around 20 minutes, or so he said. I've never heard of ccache, but asking Dr. Google it doesn't appear to exist on Windows. Besides, we're talking about clobber builds here, no? Jörg.

JC

Joshua Cranmer 🐧

Thu, Sep 7, 2017 1:16 AM

On 9/6/2017 4:56 PM, Jörg Knobloch wrote:

Hi,

don't tell Ben, but a clobber build takes me 105 minutes :-(

I've just done a clobber build of Firefox and that finished after 73
minutes. Sure, we compile a few C++ files more and shuffle a few JS
files around, but how does that account for 32 minutes more? Any ideas?

The mozmill test installation that we do involves an unconditional
directory copy that could be optimized better. You're also on Windows
AIUI, which means that there's a lot more time spent copying files
rather than symlinking them.

--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist

On 9/6/2017 4:56 PM, Jörg Knobloch wrote: > Hi, > > don't tell Ben, but a clobber build takes me 105 minutes :-( > > I've just done a clobber build of Firefox and that finished after 73 > minutes. Sure, we compile a few C++ files more and shuffle a few JS > files around, but how does that account for 32 minutes more? Any ideas? The mozmill test installation that we do involves an unconditional directory copy that could be optimized better. You're also on Windows AIUI, which means that there's a lot more time spent copying files rather than symlinking them. -- Joshua Cranmer Thunderbird and DXR developer Source code archæologist

AH

Andrei Hajdukewycz

Thu, Sep 7, 2017 8:54 AM

On 2017-09-06 5:12 PM, Jörg Knobloch wrote:

On 07/09/2017 00:44, Philipp Kewisch wrote:

2 hours is in line with the perf I was getting on a mid 2010 mbp with
mid range components. With a mid 2015 mbp top components I am also near
the 12-20 minutes Ben mentions.

Interesting. I'm using a machine with an AMD A10 Series A10-7860K 4
core x 3.60GHz bought last year, know for it's less fortunate single
thread performance. Are we talking debug or opt builds, 32 or 64 bit
and which platform? Windows is quite slow compiling.

That CPU has roughly 40% of the multithreaded performance
https://www.cpubenchmark.net/high_end_cpus.html of an i7-7700K, 33% of
a Ryzen 7 1800X, and 60% of the single-threaded performance of a 7700K.

I did a first-time Windows build on my XPS 15(i7-7700HQ, 16GB DDR4, NVME
SSD) and it took 36 minutes. This was just a straight-forward 32-bit
non-debug build with no options other than the default and the mozconfig
stuff from the llvm-config thread. Based on the performance numbers, I'd
expect Ryzen 7 1800X(non-overclocked) to be about 72% faster than my
7700HQ, and it should be about 3x as fast as your very slow old AMD quad
core.

I'm not sure if your build options make your build very slow, or if the
benchmark used by that website just performs very differently from
compiling, but it seems odd that an i7-7700HQ is 3x faster than your
machine and Ryzen is almost 9 times faster.

On 2017-09-06 5:12 PM, Jörg Knobloch wrote: > On 07/09/2017 00:44, Philipp Kewisch wrote: >> 2 hours is in line with the perf I was getting on a mid 2010 mbp with >> mid range components. With a mid 2015 mbp top components I am also near >> the 12-20 minutes Ben mentions. > > Interesting. I'm using a machine with an AMD A10 Series A10-7860K 4 > core x 3.60GHz bought last year, know for it's less fortunate single > thread performance. Are we talking debug or opt builds, 32 or 64 bit > and which platform? Windows is quite slow compiling. That CPU has roughly 40% of the multithreaded performance <https://www.cpubenchmark.net/high_end_cpus.html> of an i7-7700K, 33% of a Ryzen 7 1800X, and 60% of the single-threaded performance of a 7700K. I did a first-time Windows build on my XPS 15(i7-7700HQ, 16GB DDR4, NVME SSD) and it took 36 minutes. This was just a straight-forward 32-bit non-debug build with no options other than the default and the mozconfig stuff from the llvm-config thread. Based on the performance numbers, I'd expect Ryzen 7 1800X(non-overclocked) to be about 72% faster than my 7700HQ, and it should be about 3x as fast as your very slow old AMD quad core. I'm not sure if your build options make your build very slow, or if the benchmark used by that website just performs very differently from compiling, but it seems odd that an i7-7700HQ is 3x faster than your machine and Ryzen is almost 9 times faster.

JK

Jörg Knobloch

Thu, Sep 7, 2017 12:20 PM

On 07/09/2017 10:54, Andrei Hajdukewycz wrote:

That CPU has roughly 40% of the multithreaded performance of an i7-7700K, 33% of a Ryzen 7 1800X, and 60% of the single-threaded performance of a 7700K.

I did a first-time Windows build on my XPS 15(i7-7700HQ, 16GB DDR4, NVME SSD) and it took 36 minutes. This was just a straight-forward 32-bit non-debug build with no options other than the default and the mozconfig stuff from the llvm-config thread. Based on the performance numbers, I'd expect Ryzen 7 1800X(non-overclocked) to be about 72% faster than my 7700HQ, and it should be about 3x as fast as your very slow old AMD quad core.

I'm not sure if your build options make your build very slow, or if the benchmark used by that website just performs very differently from compiling, but it seems odd that an i7-7700HQ is 3x faster than your machine and Ryzen is almost 9 times faster.

All very interesting. Calling a 2016 machine "old" is something we could dispute. It's the machine I have in my second residence. I'm planning on an upgrade soon. I don't build all the time, and when I do, I walk away and to something else - not chargeable time ;-)

Somehow this discussion has moved into a funny direction, like: "I can do it in that time". But that wasn't the original question. The original question was: Why is TB sooo much slower than FF? I can't see that compiling a few C++ files in mailnews/ can make such a difference. I've "touched" all the C++ files in mailnew/ and recompiled an that takes 5 minutes, that doesn't account for the 30 minutes difference between FF (75 minutes) and TB (105 minutes). OK, there is libical in calendar and more C++ stuff in ldap/ and db/ but even if you assume 10 minutes for more C++ code, there are still 20 minutes unaccounted for.

Jörg.

RM

Richard Marti

Thu, Sep 7, 2017 12:34 PM

On 07.09.2017 14:20, Jörg Knobloch wrote:

Somehow this discussion has moved into a funny direction, like: "I can
do it in that time". But that wasn't the original question. The original
question was: Why is TB sooo much slower than FF? I can't see that
compiling a few C++ files in mailnews/ can make such a difference. I've
"touched" all the C++ files in mailnew/ and recompiled an that takes 5
minutes, that doesn't account for the 30 minutes difference between FF
(75 minutes) and TB (105 minutes). OK, there is libical in calendar and
more C++ stuff in ldap/ and db/ but even if you assume 10 minutes for
more C++ code, there are still 20 minutes unaccounted for.

In the last weeks it happened for me one time on Windows and one time on
Linux that a build had unnaturally long (30%-50% longer) to my other
builds. Maybe this was such a build for you? I don't know what it was,
maybe it didn't used all threads I defined in mozconfig.

Richard

On 07.09.2017 14:20, Jörg Knobloch wrote: > Somehow this discussion has moved into a funny direction, like: "I can > do it in that time". But that wasn't the original question. The original > question was: Why is TB sooo much slower than FF? I can't see that > compiling a few C++ files in mailnews/ can make such a difference. I've > "touched" all the C++ files in mailnew/ and recompiled an that takes 5 > minutes, that doesn't account for the 30 minutes difference between FF > (75 minutes) and TB (105 minutes). OK, there is libical in calendar and > more C++ stuff in ldap/ and db/ but even if you assume 10 minutes for > more C++ code, there are still 20 minutes unaccounted for. In the last weeks it happened for me one time on Windows and one time on Linux that a build had unnaturally long (30%-50% longer) to my other builds. Maybe this was such a build for you? I don't know what it was, maybe it didn't used all threads I defined in mozconfig. Richard

BB

Ben Bucksch

Thu, Sep 7, 2017 2:30 PM

Jörg Knobloch wrote on 07.09.17 14:20:

I've "touched" all the C++ files in mailnew/ and recompiled an that
takes 5 minutes, that doesn't account for the 30 minutes difference
between FF (75 minutes) and TB (105 minutes). OK, there is libical in
calendar and more C++ stuff in ldap/ and db/ but even if you assume 10
minutes for more C++ code, there are still 20 minutes unaccounted for.

You could build single-threaded, pipe the mozbuild output, which IIRC
has time stamps, into a file and see where it spends the time.

Linking takes a while, too.

Ben

Jörg Knobloch wrote on 07.09.17 14:20: > I've "touched" all the C++ files in mailnew/ and recompiled an that > takes 5 minutes, that doesn't account for the 30 minutes difference > between FF (75 minutes) and TB (105 minutes). OK, there is libical in > calendar and more C++ stuff in ldap/ and db/ but even if you assume 10 > minutes for more C++ code, there are still 20 minutes unaccounted for. You could build single-threaded, pipe the mozbuild output, which IIRC has time stamps, into a file and see where it spends the time. Linking takes a while, too. Ben

AH

Andrei Hajdukewycz

Thu, Sep 7, 2017 5:57 PM

On 2017-09-07 5:20 AM, Jörg Knobloch wrote:

All very interesting. Calling a 2016 machine "old" is something we
could dispute. It's the machine I have in my second residence. I'm
planning on an upgrade soon. I don't build all the time, and when I
do, I walk away and to something else - not chargeable time ;-)

Yeah you're right, old isn't correct. Pre-Ryzen AMD was just not very
good :P

On 2017-09-07 5:20 AM, Jörg Knobloch wrote: > > All very interesting. Calling a 2016 machine "old" is something we > could dispute. It's the machine I have in my second residence. I'm > planning on an upgrade soon. I don't build all the time, and when I > do, I walk away and to something else - not chargeable time ;-) > Yeah you're right, old isn't correct. Pre-Ryzen AMD was just not very good :P

maildev@lists.thunderbird.net

Why are Thunderbird builds so slow?