switch to Python3 yet?

[misterhsp]

Are there any plans to switch to Python3 yet? Or does the syncserver work with Python3? Python2 will soon be finished.

[ekohl]

Duplicate of #165

[rfk]

I'm sorry to say, we don't have any specific plans to port this to Python3. For production use we are currently working on a re-write of the sync storage server in Rust, and I expect we will aim to deprecate the python versions of the servers once python3 reaches end of life.

[misterhsp]

What I want to know is if the syncserver already works with Python3?
Can you already see the first results in rust somewhere?

[vladikoff]

@misterhsp https://github.com/mozilla-services/syncstorage-rs

[misterhsp]

@vladikoff Does it already work or is it still at an early stage?

[rfk]

The rust version is not yet ready for use.

[return42]

The rust version is not yet ready for use.

OK, we have only a small gap closing py3 compatibility and what is mozilla's trendsetting decision: switching to another language? Throw away loyal developers and all community's know-how .. WTF is mozilla's intention? I'm so annoyed by mozilla arrogance.

[misterhsp]

ack.
The syncserver doesn't work with Python3, it can't be built with it. Not here anyway.

[fireglow]

I don't understand the anger here.
Mozilla is free to develop a tool for which they are the primary and biggest user in any language they wish.
To me as an end-user it matters very little which language the server is written in, I am in fact looking forward to the Rust version of syncserver.
I am very grateful to Mozilla that there is the possibility to self-host syncserver.

[thomcc]

OK, we have only a small gap closing py3 compatibility and what is mozilla's trendsetting decision: switching to another language?

I probably shouldn't comment, but for what it's worth, the decision to rewrite in a new language is not motivated by a desire to avoid porting to Py3, there are a lot of other factors, such as improved performance, the ability to use the same language for the server and client, in addition to the normal benefits you get when rewriting something (the ability to avoid suboptimal design decisions made in the previous implementation, for example).

[rfk]

I do understand the frustration here, as @return42 has contributed several patches to try to move us towards python3 compatibility in the past. I apologize that my earlier comment didn't provide much context, and wanted to say a few more words.

Reading back, it's easy to interpret my comment as "we'd rather rewrite in rust than bother with porting to python3" but that's not what's happening here.

We have several large operational changes we want to make to the production sync storage service, most notably switching to a new storage backend that promises easier scalability and less chance of data loss. We opted to pursue this as a clean re-write in rust for a variety of reasons, some technical (e.g. substantially lower process overhead per box) and some organizational (e.g. the consolidation points that @thomcc made above).

Given that active work, we don't have a lot of incentive to also spend time porting to python3. That may change if e.g. the rust work doesn't pan out for whatever reason., but it's not on our roadmap right now.

[fireglow]

@return42 I wasn't aware that you've invested time to port the syncserver to python3, my apologies.

[misterhsp]

Am 30.08.19 um 08:43 schrieb Christoph:

@return42 <https://github.com/return42> I wasn't aware that you've invested time to port the syncserver to python3, my apologies. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#179?email_source=notifications&email_token=AKGJLROQWTHUO2KBQKOSDFTQHC6S7A5CNFSM4IQEITX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5QXEWQ#issuecomment-526479962>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKGJLRKA25VVBGN6MAWG57DQHC6S7ANCNFSM4IQEITXQ>.

I didn't know that either, so I apologized.

[return42]

@thomcc and @rfk thanks for the clarifying and reconciling words. It helps me understand the decision of Mozilla better.

@fireglow Don't take me wrong. I'am not talking about some languages pros and cons. My fear is, that the community loses itself in forks. With the words from @thomcc and @rfk / from a wider POV its comprehensibly to make such a hard stepover. But one should also know that it also will come with lose of community members, know-how and testing experience .. changing the language (or rewrites at all) means also to build up a new community. From my POV a good community has it value for a project.

[rfk]

One concrete action for me here, is to make sure that we have a more explicit conversation about how the Rust work affects our self-hosting community, since right now it's very focused on the production sync servers. Thanks for bringing this up.

[strayer]

@rfk Sorry for being a little off-topic here, but I just wanted to chime in and and ask to really keep us self-hosted crowd in mind! I'm happily using syncserver for years now and hold Mozilla in high regard for this possibility. I don't mind changing my Dockerfile to a rust implementation as long as it can run on simple infrastructure like the python implementation (Docker and PostgreSQL in my case).

Is there any easy way to follow the development of the rust implementation?

As a side note: Thank you to all contributors, Mozilla and non-Mozilla alike, for this project!

[chris42]

So what is the status of the rust setup? Is it possible to be used for self hosting?
It is quite hard to figure that out of the README or issue status.

[jrconlin]

"In progress" is probably the best one I can give.

Our focus is on getting the spanner stuff working, mostly because that lets us service our existing customers. It's going well, but there have been a number of interesting bugs we've discovered that were either always in the code or weren't as big a problem under python, and that's complicated things a bit.

After we get that working and folks migrated over to the new system, we'll get time to work on the MySQL side. Granted, if folks want to help get that started earlier, we're happy to accept PRs.

[uwedisch]

If we put things together we have a syncserver written in outdated Python 2 that is able to use Rust implementations of syncstorage and tokenserver. No implementation of syncserver which is written in a current vendor-supported language.

This leads to the assumption that Mozilla itself uses on production systems for storing the passwords of millions of Firefox Sync users the nearly 9 months EOL and assumed insecure Python 2 implementation of syncserver. @rfk : I'm right with my assumption?

[rfk]

I'm right with my assumption?

No, you are not. The syncserver package is not used in production at all by Mozilla, it was built entirely as a convenience for self-hosting because the things that we need to do to run this service at scale (like a separate tokenserver db, and a sharded fleet of storage nodes) are nothing but annoying overhead for self-hosters.

[uwedisch]

I'm right with my assumption?

No, you are not. The syncserver package is not used in production at all by Mozilla, it was built entirely as a convenience for self-hosting because the things that we need to do to run this service at scale (like a separate tokenserver db, and a sharded fleet of storage nodes) are nothing but annoying overhead for self-hosters.

Ok, thank you @rfk , do you have any resilent news about syncserver for self-hosters?

[uwedisch]

Ok, but regarding the readme of tokenserver-rs, the tokenserver with Python 2 is still used. @rfk , right?

[jrconlin]

Tokenserver is currently python2 (specifically pypy, which has a longer shelf-life). We're working to make it rust, but have not had the opportunity to continue work on that. We're hoping to be able to focus on it more in the coming months.

[fireglow]

The syncserver package is not used in production at all by Mozilla, it was built entirely as a convenience for self-hosting

I didn't know that! That's pretty cool! 👍

[misterhsp]

How far is syncstorage-rs, is it usable yet? If so, do you have a Howto how to replace the syncserver in FXA-selfhosting?

https://mozilla-services.readthedocs.io/en/latest/howtos/run-sync-1.5.html
Still refers to the python-2.7 syncserver

[uwedisch]

Tokenserver is currently python2 ... We're working to make it rust, but have not had the opportunity to continue work on that.

I'm taking the GDPR point of view. Python 2 isn't state of the art since January, 1st 2020. I would expect a risk assesment and risk takeover from Mozilla management for Firefox Sync production systems because of TokenServer using Python 2.

[limolitz]

Well, this is Open Source. If you really want or need something done, I'm afraid you'll have to do it yourself. You're probably aware of the rough seas Mozilla is currently in, so I wouldn't expect too much priority on this issue.

(Note: I'm not affiliated with Mozilla in any way.)

[uwedisch]

If it is Open Source or not doesn't matter in case of GDPR and Mozilla services.

[chris42]

If it is Open Source or not doesn't matter in case of GDPR and Mozilla services.

So Mozilla will archive this project and put a warning on it not to use... good work @uwedisch

[limolitz]

As jrconlin said, pypy is still supporting Python2 (most likely 'forever') and even Python2.7 itself is still provided with security patches by Canonical (until 2023) and RedHat (until 2024).

[uwedisch]

Security patches are rolled out on Ubuntu with some unspecific priority. But I don't know for example a written commitment that a specific Ubuntu version will receive security updates for all packages until end of maintenance updates.

Example: Ubuntu 16.04 LTS receives maintenance updates until 2021, but dnsmasq is missing an update for Server Cache Snooping Remote Information Disclosure. DNSmasq 2.75 is shipped with some backported fixes, but this specific one is missing that is fixed with 2.79. But with our example DNSmasq is still supported from DNSmasq maintainers and fixes are still provided.

Do we have a commitment of Canonical or RedHat that they are keeping Phyton 2.7 "alive" until 2023 or 2024?
And what about the pinned requirements that are used from syncserver with versions as of 2018?

From GDPR point of view the self-hosters are not the problem. The problem is with Mozilla itself that is serving Firefox Sync to a huge bunch of customers with outdated components.

[limolitz]

We are going off topic now. This issue is about the self-hosted syncserver, and not about the hosted service.

[rfk]

From GDPR point of view the self-hosters are not the problem. The problem is with Mozilla itself that is serving
Firefox Sync to a huge bunch of customers with outdated components.

As noted above, this isn't really an appropriate thread for discussing broader service-security issues; I invite you to file a bug in bugzilla or even report a security issue if you believe there is a problem with how the production service is currently operating.

Please do not comment further on the production sync system in this thread.

I would expect a risk assesment and risk takeover from Mozilla management

I appreciate that there is little public evidence of this assessment having taken place, especially when you approach the system starting from the (yes, sadly very under-maintained by Mozilla) self-hosting docs, but we have had these conversations and we have made deliberate decisions about how to manage the python2 end-of-life in a way that we believe to be appropriate. It's possible that we're wrong of course, and feedback/discussions are welcome! But they're going to be much more productive via one of the forums linked above.

On a personal note I'd like to add: if you do intend to continue this discussion in a more appropriate forum, please approach it assuming some base level of competence and professionalism on behalf of the Mozilla staff involved. I assume it was not intentional, but it's easy to read some of the comments in this and related threads as assuming that we're too incompetent to have considered the basic security implications here, which makes it harder to engage with the discussion in a productive manner (especially at the moment when we're all a bit more emotionally frayed than usual).

[rfk]

Specifically regarding the future of self-hosting:

So Mozilla will archive this project and put a warning on it not to use.

I expect this is what will eventually happen to the syncserver repo, yes. Not as a result of any particular discussion in this or other threads, but because we're moving the production system to the rust codebase, and at some point that's almost certainly going to accumulate features/changes that diverge sufficiently from the python version that this repo ceases to be useful.

How far is syncstorage-rs, is it usable yet? If so, do you have a Howto how to replace the syncserver in FXA-selfhosting?

So, this is where we need to get to - revamping the self-hosting docs so that we guide folks over to the new rust codebases. The rust syncstorage server works and is in production use at Mozilla today, but there are probably a lot of rough edges around trying to run it in a self-hosted setup. The rust tokenserver, as noted above, is still in development.

I'm curious to hear from the folks in this thread, what would be helpful to you in trying to move from the python-based self-hosting setup to the new rust code? Do you think you would try to build the servers from source, or prefer to run via a published docker image? Would a rust equivalent of this syncserver repo be valuable, combining the storage server and tokenserver into a single package that's easier to run?

(I don't think we can promise anything in particular at this stage, but we can at least try to move the conversation towards a concrete picture of how we'd like things to be)

[chris42]

Thanks for grounding this discussion back towards the community setup of the server.
I personally prefer the Docker setup split into syncserver and tokenserver. Right now I only run the syncserver within a docker and still use Mozillas tokenserver (I always felt that the tokenserver setup is a bit big for a tiny userbase, that I have in private use).

The provided docker helped to get a running setup fast and being able to test. If someone wants to go from scratch, the Dockerfile normally is a good howto as well (Two birds one stone ;-))

[uwedisch]

I'm curious to hear from the folks in this thread, what would be helpful to you in trying to move from the python-based self-hosting setup to the new rust code? Do you think you would try to build the servers from source, or prefer to run via a published docker image? Would a rust equivalent of this syncserver repo be valuable, combining the storage server and tokenserver into a single package that's easier to run?

Thank you, that's the point. Personally I would vote for simplicity in administrative usage (to catch a much broader user base for self-hosting):

• Using a set of various Debian packages (or snaps) for Syncserver and Accountsserver
• Debian packages (or the snaps) should be included into the main distributions or at least distributed via a repository (but htis needs manual setup)
• Debian packages (or the snaps via auto update or the like) are always current and working with the supported Firefox versions
• Stripped down functionality of Syncserver, possibly without or with minimized Tokenserver
• Heavily stripped down Accountserver that is focusing on authentication and authorization only (no additional functionality)
• No need of installing an appliance or part of it (via Docker image or the like)
• No need of compiling anything, distribution is done binary

Personally I'm looking for a single server application that is doing password sync because I think it's the worst choice to give passwords (even in encrypted form) into hands of a third party. So, password sync should be simple in setup and usage. A solution that an advanced end user is able to setup and use.

[rfk]

Using a set of various Debian packages (or snaps) for Syncserver and Accountsserver

I hear you re: the management aspect of having things available in a more standardized packaging format. But just in terms of setting expectations, I don't think this is something that's ever likely to be supported directly by Mozilla, as the way we deploy these things in production is very heavily based on docker.

[Findus23]

I guess one of the most popular self-hosted services based on a rust webservice with a database is https://github.com/dani-garcia/bitwarden_rs.
So if the ways to self-host are similar (a docker container or building the binary oneself with cargo) then I think it should be pretty user-friendly (it is even easier as with bitwarden_rs as there is no webapp attached).
With those two methods provided, others could then also create their own debian (or other package manager) packages.

[sbraz]

Would a rust equivalent of this syncserver repo be valuable, combining the storage server and tokenserver into a single package that's easier to run?

I agree with @chris42, separating syncserver and tokenserver would be easiest for small databases. In any case, Docker images seem like a very reasonable solution IMO.

[rfk]

Cross-linking #226 for future reference, which has some good comments on the rough edges of the current setup when it comes to staying up to date. Whatever we do for the new rust stuff, should take those considerations into account.

[fabianwenk]

I'm curious to hear from the folks in this thread, what would be helpful to you in trying to move from the python-based self-hosting setup to the new rust code? Do you think you would try to build the servers from source, or prefer to run via a published docker image? Would a rust equivalent of this syncserver repo be valuable, combining the storage server and tokenserver into a single package that's easier to run?

As I am running my servers with FreeBSD, Docker is not an option. So being able to build from source is preferred. A rust equivalent syncserver repo would be valuable, but I guess it would also be doable with two separate repos. The gold option of course would be if it is available as a FreeBSD Port, so the software itself and all the needed dependencies could be kept updated with the regular FreeBSD Ports updates I am doing.

[return42]

I'm curious to hear from the folks in this thread, what would be helpful to you in trying to move from the python-based self-hosting setup to the new rust code?

Do you think you would try to build the servers from source, or prefer to run via a published docker image?

Open-Source development needs "source" and a build workflow, at least described in a README.

Would a rust equivalent of this syncserver repo be valuable, combining the storage server and tokenserver into a single package that's easier to run?

if it offers any advantages both can be in one repo, but it does not really matter. only important is that both is really open-source (not a binary or container or ..) and can be easily build..

[Vertux]

I'm curious to hear from the folks in this thread, what would be helpful to you in trying to move from the python-based self-hosting setup to the new rust code? Do you think you would try to build the servers from source, or prefer to run via a published docker image? Would a rust equivalent of this syncserver repo be valuable, combining the storage server and tokenserver into a single package that's easier to run?

I would prefer to build the server from source and yes, it would be great if the syncserver and the tokenserver would be combined in one package. I am looking forward to try out the new rust version.

[immanuelfodor]

In the meantime, there is https://github.com/jackyzy823/fxa-selfhosting if you want to self-host not only the sync stack but also FXA itself. It's a bit rough at first look but the init script works fine and everything is dockerized.