BB
Ben Bucksch
Mon, Aug 14, 2017 9:12 PM
First off, I agree that search should be async, in any case. Even local
search in RAM would be too slow to block UI on it.
Joshua Cranmer 🐧 via Maildev wrote on 14.08.2017 22:21:
On 8/14/2017 3:02 PM, Ben Bucksch via Maildev wrote:
This is an interesting point. For simplicity, I would probably still
go for an in-memory copy. If you have a global corporate AB with
40000 entries, that's still not too bad to keep in RAM. (Assuming you
can delay-load the images :-) .)
If you assume a single entry takes up ~1KB (not too far off the mark,
I think, if you're talking corporate LDAP that's filling in most of
the fields)
I think 1KB per card is too much, and you don't need all fields.
But if that's an issue, we could define a delay-load API that does not
list all entries and only searches. The delay-load API would actually be
identical to the normal one, just that the card list would not be filled
on load.
Or reading mail, come to think of it--we do search for users in the
address book then.
You don't want an LDAP server query every time you read a mail, and I
don't think we do that right now.
First off, I agree that search should be async, in any case. Even local
search in RAM would be too slow to block UI on it.
Joshua Cranmer 🐧 via Maildev wrote on 14.08.2017 22:21:
> On 8/14/2017 3:02 PM, Ben Bucksch via Maildev wrote:
>> This is an interesting point. For simplicity, I would probably still
>> go for an in-memory copy. If you have a global corporate AB with
>> 40000 entries, that's still not too bad to keep in RAM. (Assuming you
>> can delay-load the images :-) .)
>
> If you assume a single entry takes up ~1KB (not too far off the mark,
> I think, if you're talking corporate LDAP that's filling in most of
> the fields)
I think 1KB per card is too much, and you don't need all fields.
But if that's an issue, we could define a delay-load API that does not
list all entries and only searches. The delay-load API would actually be
identical to the normal one, just that the card list would not be filled
on load.
> Or reading mail, come to think of it--we do search for users in the
> address book then.
You don't want an LDAP server query every time you read a mail, and I
don't think we do that right now.
EW
Enrico Weigelt, metux IT consult
Tue, Aug 15, 2017 2:38 AM
On 14.08.2017 20:07, R Kent James via Maildev wrote:
nsIAbCard is independent of database type. All of the current
nsIAbDirectory implementations are mork-specific, but at the C++ level
there is a mork-independent precursor nsAbDirProperty.cpp that partially
implements nsIAbDirectory that is mork independent. Also at the C++
level, nsAbCardProperty.cpp is a complete implementation of nsIAbCard
that is mork-independent.
Can nsIAbDirectory's be arbitrarily nested ?
In that case, I'd suggest making that the primary interface, with
some root directory holding the various datasources / address books
(local, ldap, ...). Things like contact collector then would also
just operate on an (configurable) path.
So it is possible to implement a mork-independent nsIAbDirectory that
works fine with the existing program, and there is no need to
reimplement nsIAbCard since the existing XPCOM object is fine.
I'm currently hacking on LookupAddressBook() and friends, trying to get
the alias resolution out of the composer into address book (so, composer
doesn't need to retrieve all records anymore, and we can use server-side
filtering, etc). Perhaps nsIAbDirectory would be suited for the result
list.
But there are gotchas. All of nsIAbDirectory appears to be sync. But in
fact, at the UI level, there are listeners that can be called async
after updates and is used by LDAP, but this functionality is very hidden
and only discoverable by pouring over the code.
What kind of updates ? When the server side database was updated ?
IMHO, those things should be kept behind the address book API (including
caching, etc) and all consumers (composer, UI, ...) should always call
this API.
But the real sync/async gotcha is how the rest of mailnews interfaces
with the address book. When data from the address book is needed, it is
accessed through sync methods. What that means is that any
nsIAbDirectory implementation has to read in an entire address book,
It should be cached behind the scenes (except for a data source which
has a fast access path on its own, eg. local RDMBS)
--mtx
On 14.08.2017 20:07, R Kent James via Maildev wrote:
> nsIAbCard is independent of database type. All of the current
> nsIAbDirectory implementations are mork-specific, but at the C++ level
> there is a mork-independent precursor nsAbDirProperty.cpp that partially
> implements nsIAbDirectory that is mork independent. Also at the C++
> level, nsAbCardProperty.cpp is a complete implementation of nsIAbCard
> that is mork-independent.
Can nsIAbDirectory's be arbitrarily nested ?
In that case, I'd suggest making that the primary interface, with
some root directory holding the various datasources / address books
(local, ldap, ...). Things like contact collector then would also
just operate on an (configurable) path.
> So it is possible to implement a mork-independent nsIAbDirectory that
> works fine with the existing program, and there is no need to
> reimplement nsIAbCard since the existing XPCOM object is fine.
I'm currently hacking on LookupAddressBook() and friends, trying to get
the alias resolution out of the composer into address book (so, composer
doesn't need to retrieve all records anymore, and we can use server-side
filtering, etc). Perhaps nsIAbDirectory would be suited for the result
list.
> But there are gotchas. All of nsIAbDirectory appears to be sync. But in
> fact, at the UI level, there are listeners that can be called async
> after updates and is used by LDAP, but this functionality is very hidden
> and only discoverable by pouring over the code.
What kind of updates ? When the server side database was updated ?
IMHO, those things should be kept behind the address book API (including
caching, etc) and all consumers (composer, UI, ...) should always call
this API.
> But the real sync/async gotcha is how the rest of mailnews interfaces
> with the address book. When data from the address book is needed, it is
> accessed through sync methods. What that means is that any
> nsIAbDirectory implementation has to read in an entire address book,
It should be cached behind the scenes (except for a data source which
has a fast access path on its own, eg. local RDMBS)
--mtx
EW
Enrico Weigelt, metux IT consult
Wed, Aug 16, 2017 5:18 PM
Hi folks,
<snip>
I've meanwhile made some pogress in unwinding things:
Moved out the references to AbManager and scanning the whole address
db in nsMsgCompose.cpp and encapsulated that in a new service
'AddrBookService', which is supposed as a hi-level frontend for all
address db consumers.
My next step is getting rid of all direct address book URIs in the
frontend code (eg: "moz-abmdbdirectory://abook.mab"). I see two
options for that:
a) introduce explicit getters (getPersonalAddressBook(), etc)
b) introduce a new - data source independent - namespace with
conventions for special-type directories (kind of aliases)
where also all data sources can be mounted into.
My experimental branch can be found here:
https://github.com/mozilla-cleanup/comm-esr52/tree/wip/mtx
--mtx
--
mit freundlichen Grüßen
Enrico, Sohn von Wilfried, a.d.F. Weigelt,
metux IT consulting
+49-151-27565287
Hi folks,
<snip>
I've meanwhile made some pogress in unwinding things:
Moved out the references to AbManager and scanning the whole address
db in nsMsgCompose.cpp and encapsulated that in a new service
'AddrBookService', which is supposed as a hi-level frontend for all
address db consumers.
My next step is getting rid of all direct address book URIs in the
frontend code (eg: "moz-abmdbdirectory://abook.mab"). I see two
options for that:
a) introduce explicit getters (getPersonalAddressBook(), etc)
b) introduce a new - data source independent - namespace with
conventions for special-type directories (kind of aliases)
where also all data sources can be mounted into.
My experimental branch can be found here:
https://github.com/mozilla-cleanup/comm-esr52/tree/wip/mtx
--mtx
--
mit freundlichen Grüßen
--
Enrico, Sohn von Wilfried, a.d.F. Weigelt,
metux IT consulting
+49-151-27565287
RK
R Kent James
Wed, Aug 16, 2017 9:09 PM
On 8/16/2017 10:18 AM, Enrico Weigelt, metux IT consult via Maildev wrote:
Hi folks,
<snip>
I've meanwhile made some pogress in unwinding things:
Moved out the references to AbManager and scanning the whole address
db in nsMsgCompose.cpp and encapsulated that in a new service
'AddrBookService', which is supposed as a hi-level frontend for all
address db consumers.
My next step is getting rid of all direct address book URIs in the
frontend code (eg: "moz-abmdbdirectory://abook.mab"). I see two
options for that:
a) introduce explicit getters (getPersonalAddressBook(), etc)
b) introduce a new - data source independent - namespace with
conventions for special-type directories (kind of aliases)
where also all data sources can be mounted into.
My experimental branch can be found here:
https://github.com/mozilla-cleanup/comm-esr52/tree/wip/mtx
I'm not really sure of your goals here. Let me describe some possible
goals in the AB that I see, and perhaps there is overlap between what
you want to do and those.
Direct references to moz-abmdbdirectory://abook.mab in the UI are bad,
because that assumes that the local address book is mork based. As part
of any effort to replace mork with something else (probably IndexedDB),
those references would have to be referred to a global of some sort,
probably kept in preferences. So a cleanup patch that replaced, for
example, this code in addrbookWidgets.xml with a lookup of a preference:
|const kPersonalAddressbookURI = "moz-abmdbdirectory://abook.mab";|
https://dxr.mozilla.org/comm-central/source/mailnews/addrbook/content/addrbookWidgets.xml#258
would be valid cleanup.
But there are frequently good reasons to use a string to refer to
complex objects rather than a reference counted XPCOM object, so we
would need to think hard about a long-term plan before it would make
sense to replace protocol URI specs as the major reference with
something else.
There have also been interest to replace the entire address book UI with
an alternate, with both CardBook and gNeandr's React-based project as
possibilities. Of all of the mailnews areas, the address book is
probably the best candidate for a wholesale rewrite in JS. That does not
need to stop feature development or bugfixes in the C++ code, but
generally the kind of work I would like to see (short of a complete
rewrite) is to replace major portions of the address book backend with
JS replacements. For example, a JS implementation of either a local
address book using IndexedDB, or an adaptation of existing CardDav code
to work with the existing AB UI and backend would be useful.
:rkent
On 8/16/2017 10:18 AM, Enrico Weigelt, metux IT consult via Maildev wrote:
> Hi folks,
>
> <snip>
>
> I've meanwhile made some pogress in unwinding things:
>
> Moved out the references to AbManager and scanning the whole address
> db in nsMsgCompose.cpp and encapsulated that in a new service
> 'AddrBookService', which is supposed as a hi-level frontend for all
> address db consumers.
>
> My next step is getting rid of all direct address book URIs in the
> frontend code (eg: "moz-abmdbdirectory://abook.mab"). I see two
> options for that:
>
> a) introduce explicit getters (getPersonalAddressBook(), etc)
> b) introduce a new - data source independent - namespace with
> conventions for special-type directories (kind of aliases)
> where also all data sources can be mounted into.
>
> My experimental branch can be found here:
>
> https://github.com/mozilla-cleanup/comm-esr52/tree/wip/mtx
I'm not really sure of your goals here. Let me describe some possible
goals in the AB that I see, and perhaps there is overlap between what
you want to do and those.
Direct references to moz-abmdbdirectory://abook.mab in the UI are bad,
because that assumes that the local address book is mork based. As part
of any effort to replace mork with something else (probably IndexedDB),
those references would have to be referred to a global of some sort,
probably kept in preferences. So a cleanup patch that replaced, for
example, this code in addrbookWidgets.xml with a lookup of a preference:
|const kPersonalAddressbookURI = "*moz-abmdbdirectory*://abook.mab";|
<https://dxr.mozilla.org/comm-central/source/mailnews/addrbook/content/addrbookWidgets.xml#258>
would be valid cleanup.
But there are frequently good reasons to use a string to refer to
complex objects rather than a reference counted XPCOM object, so we
would need to think hard about a long-term plan before it would make
sense to replace protocol URI specs as the major reference with
something else.
There have also been interest to replace the entire address book UI with
an alternate, with both CardBook and gNeandr's React-based project as
possibilities. Of all of the mailnews areas, the address book is
probably the best candidate for a wholesale rewrite in JS. That does not
need to stop feature development or bugfixes in the C++ code, but
generally the kind of work I would like to see (short of a complete
rewrite) is to replace major portions of the address book backend with
JS replacements. For example, a JS implementation of either a local
address book using IndexedDB, or an adaptation of existing CardDav code
to work with the existing AB UI and backend would be useful.
:rkent
JC
Joshua Cranmer 🐧
Wed, Aug 16, 2017 10:52 PM
On 8/16/2017 12:18 PM, Enrico Weigelt, metux IT consult via Maildev wrote:
Hi folks,
<snip>
I've meanwhile made some pogress in unwinding things:
Moved out the references to AbManager and scanning the whole address
db in nsMsgCompose.cpp and encapsulated that in a new service
'AddrBookService', which is supposed as a hi-level frontend for all
address db consumers.
My next step is getting rid of all direct address book URIs in the
frontend code (eg: "moz-abmdbdirectory://abook.mab"). I see two
options for that:
a) introduce explicit getters (getPersonalAddressBook(), etc)
b) introduce a new - data source independent - namespace with
conventions for special-type directories (kind of aliases)
where also all data sources can be mounted into.
My experimental branch can be found here:
https://github.com/mozilla-cleanup/comm-esr52/tree/wip/mtx
It's great to see people that are eager to contribute. With that in
mind, to echo some of Kent's comments and my own, it's not entirely
clear to what you end you are working towards and how useful this work
actually is.
To start with--and this is true for all open-source projects--you should
develop patches primarily on the trunk build if you want to ever see
them integrated upstream. It also helps to keep orthogonal patch trees
in different branches rather than relying on one branch for everything,
since it reduces the likelihood that unnecessary or undesirable changes
sneak into patches.
You should also familiarize yourself with our testing infrastructure.
It's clear to me that you haven't run tests since one of your incidental
changes breaks several tests. We've had a long-standing policy that all
new code should be well-tested, and there is very little code in our
tree that can't be automatically tested (although even then, I've
written basic frameworks for two of those places, and I've proposed new
test infrastructure for an entire class of presently untestable inputs).
See
https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Thunderbird_Automated_Testing
for more information, although it is slightly out-of-date.
As mentioned previously, our current long-term policy is to move much of
our codebase to JS. Adding a new service that is written in C++ is very
much something that raises eyebrows, and requires good justification.
Finally, if you're proposing to do large-scale projects or redesigns, it
is very helpful to first lay out what you wish to do before you embark
on code. The address book component is something that we've long had
flagged for a redesign for various reasons, and there is even another
team actively working on it. While that doesn't (and shouldn't) mean
that we ignore potential quicker improvements now, it does mean that we
ought to evaluate those improvements in the context of being able to
meet those goals (e.g., a new backend should potentially support more
than two emails per contact). You seem unaware and/or dismissive of this
rewrite project, which is not the appropriate attitude to be taking.
Taking a brief overview of what you've actually done... I don't see
anything useful yet in those patches worth contributing. All you've done
is shift around APIs that don't really hide anything, make anything
simpler, or eliminate redundant code. Indeed, one of my first thoughts
is "why are you trying to hide nsIAbManager, isn't that what you want
nsIAddrBookService to be?" If you really want to redesign the address
book API, do the following:
- Draw up an IDL proposal for the new interfaces
- Explain what is wrong with the current interfaces and why you need
to add a new interface in front of them.
- Show how the new interfaces would help, or at least not hinder,
desired address book improvements. Including a new local async
storage backend for the personal address book, writable LDAP,
unlimited emails per contact, CardDAV support, and pure JavaScript
implementation (except for necessary glue for system address books).
And please do that before asking us to look at the code, or better yet,
before writing the code, so that we can see and comment on where you
would like to have the codebase evolve.
--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist
On 8/16/2017 12:18 PM, Enrico Weigelt, metux IT consult via Maildev wrote:
> Hi folks,
>
> <snip>
>
> I've meanwhile made some pogress in unwinding things:
>
> Moved out the references to AbManager and scanning the whole address
> db in nsMsgCompose.cpp and encapsulated that in a new service
> 'AddrBookService', which is supposed as a hi-level frontend for all
> address db consumers.
>
> My next step is getting rid of all direct address book URIs in the
> frontend code (eg: "moz-abmdbdirectory://abook.mab"). I see two
> options for that:
>
> a) introduce explicit getters (getPersonalAddressBook(), etc)
> b) introduce a new - data source independent - namespace with
> conventions for special-type directories (kind of aliases)
> where also all data sources can be mounted into.
>
> My experimental branch can be found here:
>
> https://github.com/mozilla-cleanup/comm-esr52/tree/wip/mtx
It's great to see people that are eager to contribute. With that in
mind, to echo some of Kent's comments and my own, it's not entirely
clear to what you end you are working towards and how useful this work
actually is.
To start with--and this is true for all open-source projects--you should
develop patches primarily on the trunk build if you want to ever see
them integrated upstream. It also helps to keep orthogonal patch trees
in different branches rather than relying on one branch for everything,
since it reduces the likelihood that unnecessary or undesirable changes
sneak into patches.
You should also familiarize yourself with our testing infrastructure.
It's clear to me that you haven't run tests since one of your incidental
changes breaks several tests. We've had a long-standing policy that all
new code should be well-tested, and there is very little code in our
tree that can't be automatically tested (although even then, I've
written basic frameworks for two of those places, and I've proposed new
test infrastructure for an entire class of presently untestable inputs).
See
<https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Thunderbird_Automated_Testing>
for more information, although it is slightly out-of-date.
As mentioned previously, our current long-term policy is to move much of
our codebase to JS. Adding a new service that is written in C++ is very
much something that raises eyebrows, and requires good justification.
Finally, if you're proposing to do large-scale projects or redesigns, it
is very helpful to first lay out what you wish to do before you embark
on code. The address book component is something that we've long had
flagged for a redesign for various reasons, and there is even another
team actively working on it. While that doesn't (and shouldn't) mean
that we ignore potential quicker improvements now, it does mean that we
ought to evaluate those improvements in the context of being able to
meet those goals (e.g., a new backend should potentially support more
than two emails per contact). You seem unaware and/or dismissive of this
rewrite project, which is not the appropriate attitude to be taking.
Taking a brief overview of what you've actually done... I don't see
anything useful yet in those patches worth contributing. All you've done
is shift around APIs that don't really hide anything, make anything
simpler, or eliminate redundant code. Indeed, one of my first thoughts
is "why are you trying to hide nsIAbManager, isn't that what you want
nsIAddrBookService to be?" If you really want to redesign the address
book API, do the following:
1. Draw up an IDL proposal for the new interfaces
2. Explain what is wrong with the current interfaces and why you need
to add a new interface in front of them.
3. Show how the new interfaces would help, or at least not hinder,
desired address book improvements. Including a new local async
storage backend for the personal address book, writable LDAP,
unlimited emails per contact, CardDAV support, and pure JavaScript
implementation (except for necessary glue for system address books).
And please do that before asking us to look at the code, or better yet,
before writing the code, so that we can see and comment on where you
would like to have the codebase evolve.
--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist
EW
Enrico Weigelt, metux IT consult
Thu, Aug 17, 2017 8:26 AM
On 17.08.2017 00:52, Joshua Cranmer 🐧 via Maildev wrote:
To start with--and this is true for all open-source projects--you should > develop patches primarily on the trunk build if you want to ever see
them integrated upstream.
In general I agree.
But here, the situation is a bit different:
I'm basing on esr52, as for anything newer, rust is mandatory. Rust is
still too young and moving too fast to be supported by current stable
distros (which I'm targeting, primarily debian/devuan - and build /
deployment on production systems only via distro's package manager)
There's still a lot to do (had some experiments, but given up -
integrating cargo w/ package management is larger project on its own),
far too much for me alone.
It will still take a while until all of this is settled (I even doubt
there'll be backports to jessie anytime soon). Until then, I prefer
staying on esr52. (actually I already had a lot to do there to get
the big moz tree building at all).
I'd suspect that the bigger changes (which are likely to conflict w/
newer versions) can't be mainlined anytime soon anyways - needs more
deeper thoughts. So, perhaps see my patches as concept studies ;-)
You should also familiarize yourself with our testing infrastructure.
I'm not there yet ;-)
This branch is nowhere near stable, it's highly experimental.
I just posted it to let you guys know what I'm working on.
(OTOH, I'd appreciate, if anybody likes to try it and give feedback)
By the way: is there some automatic test system, I could just push to ?
I've already seen some docs on moz CI in general, but that seems to
require extra accounts, and I'm even unsure how that works w/ two trees
(moz and tbird).
It's clear to me that you haven't run tests since one of your incidental > changes breaks several tests.
Which ones ?
I'm currently running it (load > 25 - ugh!), and I see lots of timeouts,
in areas I didn't touch. Is that normal ?
As mentioned previously, our current long-term policy is to move much of > our codebase to JS.
Including all the data source and lo level protocol handling stuff ?
Finally, if you're proposing to do large-scale projects or redesigns, it
I hope not to do a large-scale project, but small steps.
is very helpful to first lay out what you wish to do before you embark
Please see my other mail.
You seem unaware and/or dismissive of this
rewrite project, which is not the appropriate attitude to be taking.
It seems that will take a while anyways, and I'm not sure whether a
complete rewrite is a good way to go (instead of step-by-step
evolution).
My primary concern here is integration with other (local) systems
and reduction of isolated data silos (w/o some cloud-sync-whatever).
Taking a brief overview of what you've actually done... I don't see
anything useful yet in those patches worth contributing. All you've done
is shift around APIs that don't really hide anything, make anything
simpler, or eliminate redundant code.
Well, it's a step-by-step approach to decouple things. In contrast
to nsIAbManager, this shall be completely independent of individual
data sources (their clients shall have no idea where the data is
actually coming from). I didn't want to change nsIAbManager (yet),
but isolate and later phase it out.
Note that's all experimental. I try to destill out smaller pieces
that could be mainline'able, but that's not the focus for now.
- Draw up an IDL proposal for the new interfaces
I'm not there yet. I'll first have to do some experiments and get
a better understanding how the things actually work under the hood.
--mtx
On 17.08.2017 00:52, Joshua Cranmer 🐧 via Maildev wrote:
> To start with--and this is true for all open-source projects--you should > develop patches primarily on the trunk build if you want to ever see
> them integrated upstream.
In general I agree.
But here, the situation is a bit different:
I'm basing on esr52, as for anything newer, rust is mandatory. Rust is
still too young and moving too fast to be supported by current stable
distros (which I'm targeting, primarily debian/devuan - and build /
deployment on production systems *only* via distro's package manager)
There's still *a lot* to do (had some experiments, but given up -
integrating cargo w/ package management is larger project on its own),
far too much for me alone.
It will still take a while until all of this is settled (I even doubt
there'll be backports to jessie anytime soon). Until then, I prefer
staying on esr52. (actually I already had a lot to do there to get
the big moz tree building at all).
I'd suspect that the bigger changes (which are likely to conflict w/
newer versions) can't be mainlined anytime soon anyways - needs more
deeper thoughts. So, perhaps see my patches as concept studies ;-)
> You should also familiarize yourself with our testing infrastructure.
I'm not there yet ;-)
This branch is nowhere near stable, it's highly experimental.
I just posted it to let you guys know what I'm working on.
(OTOH, I'd appreciate, if anybody likes to try it and give feedback)
By the way: is there some automatic test system, I could just push to ?
I've already seen some docs on moz CI in general, but that seems to
require extra accounts, and I'm even unsure how that works w/ two trees
(moz and tbird).
> It's clear to me that you haven't run tests since one of your incidental > changes breaks several tests.
Which ones ?
I'm currently running it (load > 25 - ugh!), and I see lots of timeouts,
in areas I didn't touch. Is that normal ?
> As mentioned previously, our current long-term policy is to move much of > our codebase to JS.
Including all the data source and lo level protocol handling stuff ?
> Finally, if you're proposing to do large-scale projects or redesigns, it
I hope not to do a large-scale project, but small steps.
> is very helpful to first lay out what you wish to do before you embark
Please see my other mail.
> You seem unaware and/or dismissive of this
> rewrite project, which is not the appropriate attitude to be taking.
It seems that will take a while anyways, and I'm not sure whether a
complete rewrite is a good way to go (instead of step-by-step
evolution).
My primary concern here is integration with other (local) systems
and reduction of isolated data silos (w/o some cloud-sync-whatever).
> Taking a brief overview of what you've actually done... I don't see
> anything useful yet in those patches worth contributing. All you've done
> is shift around APIs that don't really hide anything, make anything
> simpler, or eliminate redundant code.
Well, it's a *step-by-step* approach to decouple things. In contrast
to nsIAbManager, this shall be completely independent of individual
data sources (their clients shall have no idea where the data is
actually coming from). I didn't want to change nsIAbManager (yet),
but isolate and later phase it out.
Note that's all *experimental*. I try to destill out smaller pieces
that could be mainline'able, but that's not the focus for now.
> 1. Draw up an IDL proposal for the new interfaces
I'm not there yet. I'll first have to do some experiments and get
a better understanding how the things actually work under the hood.
--mtx
RK
R Kent James
Mon, Aug 21, 2017 5:47 PM
On 8/17/2017 1:26 AM, Enrico Weigelt, metux IT consult via Maildev wrote:
On 17.08.2017 00:52, Joshua Cranmer 🐧 via Maildev wrote:
To start with--and this is true for all open-source projects--you
should > develop patches primarily on the trunk build if you want to
ever see
them integrated upstream.
In general I agree.
But here, the situation is a bit different:
I'm basing on esr52, as for anything newer, rust is mandatory. Rust is
still too young and moving too fast to be supported by current stable
distros (which I'm targeting, primarily debian/devuan - and build /
deployment on production systems only via distro's package manager)
You are of course free to take the code for Thunderbird and use it in
any way that you please, but this maildev list is specifically about
making progress on core Thunderbird development. If you base on esr52,
or avoid rust, then you are not developing for core Thunderbird. We
would love to have you as a core Thunderbird developer, but the first
steps in that are to generally follow the practices of the project.
You also seem to want to make sweeping changes to the code before you
have successfully done any patches for Thunderbird. The general advice
for someone starting in a project is to start with some small patches,
and try to get those approved and landed to learn about the processes
and the culture of the project. If you are interested in being a core
developer, then I suggest that you start there.
If you are interested in issues associated with Thunderbird development
outside of specific decisions involving the core project, I suggest that
you try using the m.d.a.thunderbird newsgroup for that.
:rkent
On 8/17/2017 1:26 AM, Enrico Weigelt, metux IT consult via Maildev wrote:
> On 17.08.2017 00:52, Joshua Cranmer 🐧 via Maildev wrote:
>
>> To start with--and this is true for all open-source projects--you
>> should > develop patches primarily on the trunk build if you want to
>> ever see
> > them integrated upstream.
>
> In general I agree.
>
> But here, the situation is a bit different:
>
> I'm basing on esr52, as for anything newer, rust is mandatory. Rust is
> still too young and moving too fast to be supported by current stable
> distros (which I'm targeting, primarily debian/devuan - and build /
> deployment on production systems *only* via distro's package manager)
You are of course free to take the code for Thunderbird and use it in
any way that you please, but this maildev list is specifically about
making progress on core Thunderbird development. If you base on esr52,
or avoid rust, then you are not developing for core Thunderbird. We
would love to have you as a core Thunderbird developer, but the first
steps in that are to generally follow the practices of the project.
You also seem to want to make sweeping changes to the code before you
have successfully done any patches for Thunderbird. The general advice
for someone starting in a project is to start with some small patches,
and try to get those approved and landed to learn about the processes
and the culture of the project. If you are interested in being a core
developer, then I suggest that you start there.
If you are interested in issues associated with Thunderbird development
outside of specific decisions involving the core project, I suggest that
you try using the m.d.a.thunderbird newsgroup for that.
:rkent
EW
Enrico Weigelt, metux IT consult
Tue, Aug 22, 2017 12:19 PM
On 21.08.2017 19:47, R Kent James wrote:
Hi,
You are of course free to take the code for Thunderbird and use it in > any way that you please, but this maildev list is specifically about
making progress on core Thunderbird development. If you base on
esr52, > or avoid rust, then you are not developing for core Thunderbird.
First things first ;-)
I'll yet have to get a better of how things work under the hood and do
several experiments. Writing the code isn't the big deal - finding out
how to write it is. So, I'd guess, porting it to newer base later,
shouldn't be that big deal.
We would love to have you as a core Thunderbird developer, but the first > steps in that are to generally follow the practices of the project.
I'd love to, but it isn't that easy :(
I've started an import a few days ago, which ran over several days and
broke this morning with out of storage. Unfortunately I can't easily
add more storage to my notebook (I'm traveling much, so that's my
primary machine, and its also filled w/ lots of other projects).
IMHO, the infrastructure problem is a big stopper not just for me
(whenever I asked collegues about working on moz, they all don't like
to open that pandorra's box). So, perhaps we should focus on that first.
I've got a spare machine we could use - but currently lacking time to
set it up (it's idling for quite a while for that reason). If anybody
here could care about it, that would be a good start.
My idea goes like that:
- setup the machine w/ proxmox or docker for containers (maybe I'll add
another machine later ...)
- setup a minimal build environment template for moz/tb for easy cloning
- publish it to the docket registry, so everybody in the world can
clone it any time
For the scm problem my idea goes like this:
- setup an automatic hg->git mirror (perhaps @github), maybe even
cut-off history at latest stable release point
- do most of the work in git, only merge the really finished stuff
(what you call 'landing') into hg - only done by the mainainer(s).
- only the maintainers would need to cope with both sides (and need
full clones), the plain developer doesn't. (folks who still want
to use hg directly can still go the old route)
- for the huge mozilla-central tree, work w/ snapshots (or shallow
clone of the github mirror) and submit the likely few patches we'll
have manually (or let the maintainer(s) do it, who'll have full hg
clones anyways)
It's far from ideal, but should at least give us some usable approach
(the current one is practically unusable for me :().
For dist building / packaging, I'd suggest - as an intermediate
approach, until the rust issue is sorted out - going by foot:
- build moz/tb the upstream way (local build), but in minimal containers
of the corresponding target distros / releases
- put it into /opt/... instead of /usr/...
- add some scripts for directly creating packages out of the install
tree (bypassing the usual toolchains debuild etc)
Yes, this is horrible and never will make it into any distro directly,
but at least gives us a chance to put it into our own repos, so users
can actually install and use it. Buys us time until the rust problem
is solved by the distros.
--mtx
mit freundlichen Grüßen
Enrico, Sohn von Wilfried, a.d.F. Weigelt,
metux IT consulting
+49-151-27565287
On 21.08.2017 19:47, R Kent James wrote:
Hi,
> You are of course free to take the code for Thunderbird and use it in > any way that you please, but this maildev list is specifically about
> making progress on core Thunderbird development. If you base on
esr52, > or avoid rust, then you are not developing for core Thunderbird.
First things first ;-)
I'll yet have to get a better of how things work under the hood and do
several experiments. Writing the code isn't the big deal - finding out
how to write it is. So, I'd guess, porting it to newer base later,
shouldn't be that big deal.
> We would love to have you as a core Thunderbird developer, but the first > steps in that are to generally follow the practices of the project.
I'd love to, but it isn't that easy :(
I've started an import a few days ago, which ran over several days and
broke this morning with out of storage. Unfortunately I can't easily
add more storage to my notebook (I'm traveling much, so that's my
primary machine, and its also filled w/ lots of other projects).
IMHO, the infrastructure problem is a big stopper not just for me
(whenever I asked collegues about working on moz, they all don't like
to open that pandorra's box). So, perhaps we should focus on that first.
I've got a spare machine we could use - but currently lacking time to
set it up (it's idling for quite a while for that reason). If anybody
here could care about it, that would be a good start.
My idea goes like that:
* setup the machine w/ proxmox or docker for containers (maybe I'll add
another machine later ...)
* setup a minimal build environment template for moz/tb for easy cloning
* publish it to the docket registry, so everybody in the world can
clone it any time
For the scm problem my idea goes like this:
* setup an automatic hg->git mirror (perhaps @github), maybe even
cut-off history at latest stable release point
* do most of the work in git, only merge the really finished stuff
(what you call 'landing') into hg - only done by the mainainer(s).
* only the maintainers would need to cope with both sides (and need
full clones), the plain developer doesn't. (folks who still want
to use hg directly can still go the old route)
* for the *huge* mozilla-central tree, work w/ snapshots (or shallow
clone of the github mirror) and submit the likely few patches we'll
have manually (or let the maintainer(s) do it, who'll have full hg
clones anyways)
It's far from ideal, but should at least give us some usable approach
(the current one is practically unusable for me :().
For dist building / packaging, I'd suggest - as an intermediate
approach, until the rust issue is sorted out - going by foot:
* build moz/tb the upstream way (local build), but in minimal containers
of the corresponding target distros / releases
* put it into /opt/... instead of /usr/...
* add some scripts for directly creating packages out of the install
tree (bypassing the usual toolchains debuild etc)
Yes, this is horrible and never will make it into any distro directly,
but at least gives us a chance to put it into our own repos, so users
can actually install and use it. Buys us time until the rust problem
is solved by the distros.
--mtx
mit freundlichen Grüßen
--
Enrico, Sohn von Wilfried, a.d.F. Weigelt,
metux IT consulting
+49-151-27565287
BB
Ben Bucksch
Sun, Aug 27, 2017 5:05 PM
Hey Enrico,
Did you try
$ ./client.py checkout
$ ./mozilla/mach/bootstrap
Option 2 (Firefox Desktop, no artifacts)
$ nice ./make -f client.mk -s -j16
?
You may not like what it does (push stuff in ~/.cargo/ ), but then
symlinks are quickly made. In every project, you'll have to put up with
whatever build system they hammered together, be it good or bad. Mozilla
has one of the better ones I've seen.
Ben
Hey Enrico,
Did you try
$ ./client.py checkout
$ ./mozilla/mach/bootstrap
Option 2 (Firefox Desktop, no artifacts)
$ nice ./make -f client.mk -s -j16
?
You may not like what it does (push stuff in ~/.cargo/ ), but then
symlinks are quickly made. In every project, you'll have to put up with
whatever build system they hammered together, be it good or bad. Mozilla
has one of the better ones I've seen.
Ben
