Andrei Hajdukewycz wrote on 22.05.19 07:24:
There are going to be major, continuing changes to add-ons with
frequent breakage until legacy add-ons are completely gone. To me, the
faster we get there, the better for everyone.
May I ask for more clarification on this, please? ExQuilla is a legacy
extension and has tons of users. Those users completely depend on it.
ExQuilla has lots of XPCOM, all over the place. It's almost 100% XPCOM
code. I don't think it's feasible to port ExQuilla to WebExtensions.
That's why we wrote Owl from scratch, which took us almost a year and
we're not close to the coverage of ExQuilla. However, ExQuilla has a
large userbase and I am afraid to lose it. Owl is a young product,
doesn't have the same feature set and it will be difficult to get it on
par with ExQuilla. Exchange is a really really nasty business: Microsoft
doesn't adhere to any standards, not even those protocols they made up
themselves. Plus the server can be customized locally to no end - it
works for us, but not for the user, good luck finding out why, because
you don't have their login and cannot reproduce it. It's an utter pain
to support. ExQuilla already works for those users who use it, and I
have no idea what will happen once I migrate them to Owl. I'd probably
be flooded with 1000s of support emails, and no chance to debug all of
it. We need to fix all these bugs and incompatibilities as they come and
as we have the resources. But enough whining. This is just to let you
know the situation.
My plan was to keep ExQuilla alive for another 2 to 5 years, as long as
technically and financially possible, and slowly transition users to
Owl. This transition takes years, both for the users and for us.
It appears from the above sentence that you are actively trying to kill
legacy extensions. Is that the case? So far, I was under the impression
that Thunderbird makes efforts to keep them working as long as possible.
Disclaimer: I understand that there are changes beyond our control, like
the removal of overlays or removal of XBL. That's clear. I also
understand the change discussed here. Adding a manifest.json is easy
enough for extensions, and seems to be a big simplification for you, so
it seems like a reasonable trade-off (although I still think the 100s of
small extension authors should be supported better on this and not left
alone).
But your message seems to imply that you actively want to get rid of
legacy addons, instead of actively trying to keep them working. This
difference has very real implications. Could you please clarify that a bit?
We can do our best to minimize the pain or minimize the duration of
that pain but we can't do both, and I'm definitely in the camp of
minimizing duration over intensity.
Maybe the explanation above shines a little different light on this.
Humbly,
Ben Bucksch
On 2019-05-25 6:47 p.m., Ben Bucksch wrote:
The difference matters, from a extension developer perspective, from
an addon reviewer perspective, from a Thunderbird API support
perspective, and even from a user perspective (security sandbox, plus
longevity due to stable APIs). The ATN website must not confuse these
two types of extensions. bootstrapped addons (even with manifest.json)
should be treated the same they have been before, in addon reviews and
in end user display and in internal statistics. I concur with Jonathan
that this is urgent.
If you disagree that these are web extensions you'll need to take it up
with Geoff because, as the person who has implemented much of this, I've
already had this discussion with him and he says it's correct to call
and treat them as web extensions.
That doesn't mean they are treated identically to web extensions that
use the APIs(ones without the 'legacy' object in manifest.json), or the
same as web experiments -- they're not, and if they have a 'legacy'
object they do go in the 'legacy' queue. But the site will continue to
indicate that they're web extensions visually and in various internal
ways and it needs to do that because they use the web extension manifest
format.
Even if we decide to call these legacy extensions, it doesn't magically
change the code, and addons-server deeply considers add-ons with a
manifest.json to be web extensions. I'm not willing to try to modify
addons-server to change this. We have already modified it too much, in
my opinion, and it gets worse every time we try to patch it. It was
never designed for the things we're doing with it, and we don't have a
developer to dedicate to rewriting significant portions of it. I already
spend about 100% more of my time on addons-server than it should, it's
frankly become a huge time sink that vastly outpaces the value that our
users get out of it compared to pretty much anything else I work on.
Frankly we should be happy it is working at all.
Andrei Hajdukewycz wrote on 26.05.19 04:00:
On 2019-05-25 6:47 p.m., Ben Bucksch wrote:
The difference matters, from a extension developer perspective, from an addon reviewer perspective, from a Thunderbird API support perspective, and even from a user perspective (security sandbox, plus longevity due to stable APIs). The ATN website must not confuse these two types of extensions. bootstrapped addons (even with manifest.json) should be treated the same they have been before, in addon reviews and in end user display and in internal statistics. I concur with Jonathan that this is urgent.
Geoff ... says it's correct to call and treat them as web extensions.
I don't know in which context he said this. It may be OK to treat them the same in certain flows. But that doesn't mean that they *are* the same, or can be called the same.
addons-server deeply considers add-ons with a manifest.json to be web extensions.
That is probably OK from a technical implementation perspective. I can't speak for addon server's internals.
From a high-level perspective, what I think is necessary is:
However, given that you say that you have a "legacy" object with a "legacy" review queue, the first 2 requirements seem to be already met. The only thing left to do seems to be to adjust the end user display to show "restartless" instead of "WebExtension". I would not think that this is extremely difficult, given that it's just the page display.
We cannot display "WebExtension" to an end user for a bootstrapped extension, because that's essentially lying to them. The difference very much matters even for end users, for the reasons I explained.
Would it be possible to adjust the end user page display for bootstrapped addons to not claim that they are WebExtensions, but say "restartless" instead? I would hope that this is a mostly a display thing and feasible without changing much internals.
I'm not willing to try to modify addons-server to change this. We have already modified it too much, in my opinion, and it gets worse every time we try to patch it. It was never designed for the things we're doing with it, and we don't have a developer to dedicate to rewriting significant portions of it. I already spend about 100% more of my time on addons-server than it should...
Frankly we should be happy it is working at all.
I believe you that the addons server is a time sink and a pain in the ass, yes.
I am happy that things are working as far as they do! Thanks for all the work you put into it.
Ben
On 2019-05-25 7:18 p.m., Ben Bucksch wrote:
Would it be possible to adjust the end user page display for
bootstrapped addons to not claim that they are WebExtensions, but say
"restartless" instead? I would hope that this is a mostly a display
thing and feasible without changing much internals.
The bootstrapped web extensions currently show as restart required
add-ons actually, I think. Geoff is working on a patch to fix that
https://github.com/thundernest/addons-server/pull/89, it might require
database changes though.
Andrei Hajdukewycz wrote on 26.05.19 04:29:
On 2019-05-25 7:18 p.m., Ben Bucksch wrote:
Would it be possible to adjust the end user page display for bootstrapped addons to not claim that they are WebExtensions, but say "restartless" instead? I would hope that this is a mostly a display thing and feasible without changing much internals.
The bootstrapped web extensions currently show as restart required add-ons actually, I think. Geoff is working on a patch to fix that
Ah, OK. That's much better. Thank you!
Thanks to both of you for your work on this and similar questions.
Ben
On 2019-05-25 6:53 p.m., Ben Bucksch wrote:
Andrei Hajdukewycz wrote on 22.05.19 07:24:
There are going to be major, continuing changes to add-ons with
frequent breakage until legacy add-ons are completely gone. To me,
the faster we get there, the better for everyone.
May I ask for more clarification on this, please? ExQuilla is a legacy
extension and has tons of users. Those users completely depend on it.
Nothing I've written about forward-looking legacy support is official
policy. The fact is that I'm not the person writing the code that
supports add-ons in Thunderbird and I don't really know how long we can
or should support legacy add-ons.
It is my understanding that Firefox is relatively quickly removing
everything that supports legacy extensions, including many XPCOM
interfaces, however. And my personal opinion is that spending
Thunderbird employee time trying to repair or support removed Mozilla
code is a poor use of that time. I've also been told that literally any
legacy add-on can be rewritten as a Web Experiment, as well, although
the implementation is obviously very different. What I would personally
like to do is open up Web Experiments to all add-on developers, along
with a review policy that enables us to move those towards Thunderbird
APIs as much as possible over the next year or two.
It's up to Geoff or Magnus to decide if 5-year XPCOM extension support
is even possible, but my guess would be 'no'. To me 2 years should be
the max. Keeping an ATN based on today's code around in 2023 or 2024
does seem crazy to me, that's for sure.
On 5/26/19 4:58 AM, Andrei Hajdukewycz wrote:
On 2019-05-25 6:53 p.m., Ben Bucksch wrote:
Andrei Hajdukewycz wrote on 22.05.19 07:24:
There are going to be major, continuing changes to add-ons with
frequent breakage until legacy add-ons are completely gone. To me,
the faster we get there, the better for everyone.
May I ask for more clarification on this, please? ExQuilla is a
legacy extension and has tons of users. Those users completely depend
on it.
Nothing I've written about forward-looking legacy support is official
policy. The fact is that I'm not the person writing the code that
supports add-ons in Thunderbird and I don't really know how long we
can or should support legacy add-ons.
It is my understanding that Firefox is relatively quickly removing
everything that supports legacy extensions, including many XPCOM
interfaces, however. And my personal opinion is that spending
Thunderbird employee time trying to repair or support removed Mozilla
code is a poor use of that time. I've also been told that literally
any legacy add-on can be rewritten as a Web Experiment, as well,
although the implementation is obviously very different. What I would
personally like to do is open up Web Experiments to all add-on
developers, along with a review policy that enables us to move those
towards Thunderbird APIs as much as possible over the next year or two.
It's up to Geoff or Magnus to decide if 5-year XPCOM extension support
is even possible, but my guess would be 'no'. To me 2 years should be
the max. Keeping an ATN based on today's code around in 2023 or 2024
does seem crazy to me, that's for sure.
Hi Folks,
I don't think it is true that we are actively trying to remove
Thunderbird legacy extension support. In fact, the code written to make
xpcom and overlays work in WebExtensions with the legacy key in the
manifest shows that we are still providing authors with more time to
transition.
I specifically say transition because I believe it is not feasible to
continue supporting legacy or bootstrapped extensions in the future, for
similar reasons as Andrei mentioned. I don't think 5 years is feasible,
nor do I feel that we should commit to supporting legacy extensions for
a fixed period of time, even 2 years. This is simply so because most of
it is not in our hands. We also need to make sure that the paid time we
have is spent wisely, and going to lengths to support legacy extensions
does not seem great to me.
Back at the time of Thunderbird 52 we specifically said we'd continue to
support legacy extensions, but that we don't know how long this is
possible and that the decision for that is mainly dependent on what m-c
is doing. With the pace m-c is making changes, it is clear that we need
to provide alternatives for extension developers that are easy to
support. We should provide the means for developers to use WebExtensions
and WebExtension Experiments, and do what we can to encourage them to
work on APIs that could be re-used for other Thunderbird extensions.
My understanding of the plan for 60 and beyond had been, that we do the
best we can to provide a basic set of APIs up until Thunderbird 68, we
backport as much as we can to Thunderbird 60 to allow for transitioning,
and and that we encourage extension developers to support us with
writing APIs. Geoff has made some great progress in providing a basic
set of APIs, what we haven't been able to do is broadly encourage the
community to support us with further APIs.
We are getting a bit off topic on the original thread, and it doesn't
actually look like we are coming to a conclusion. I think what needs to
happen is that we take a step back, look at what high level problems we
would like to solve w.r.t. extensions, and then have a smaller circle of
people work out a plan on how to act on those problems.
What I think we can agree on is that we need a plan moving forward that
is considered official. Not everyone will be happy with the plan, and we
can't solve all the problems at once, but it will give extension
developers something to work towards while making the efforts for
Thunderbird developers more plannable. This is what we should have done
for 60-68, and I thought we had agreed on something specific, but this
thread makes clear we didn't communicate this officially in any way.
Philipp