How to migrate to the Rust components?

[fireglow]

I was wondering, with the end-of-life of Python 2.7 approaching fast, if there is a migration path to the Rust syncstorage-rs component.
It's unclear to me how this all fits together, and what steps the self-hosters need to follow.
Has somebody already worked this out?

[cjones26]

I would second this--looks like Python 2.7 hits EOL tomorrow so I'm curious whether it's safe to continue utilizing Syncserver 1.5.

[jrconlin]

Trying not to sound authoritative here, but I understand the relative radio silence from us is a little disturbing. To that end, let me clarify what we're planning on doing.

First off, the team tasked with this (services-engineering) have been working on the new server (syncstorage-rs). The goal there is to make supporting Sync for millions of folks better. With the current python version, it's possible for us to lose your data if you don't have multiple clients doing regular refreshes. For us, that means using a different data storage back-end. This does not mean that we're requiring stand alone servers to do the same, and we're working hard to ensure that you can still use your SQL db of choice.

Needless to say, moving hundreds of millions of accounts to a new storage system is... tricky. We're moving slowly and carefully to make sure that we don't screw up. (We're also a pretty small team working on this, thus some of the radio silence since we're head down trying to get this working.)

What does this mean for you? Our current schedule is to try and get things up and running by the end of the second quarter of 2020 (ideally, before then). Python 2.7 is no longer officially supported as of Jan 01, 2020. That doesn't mean it stops working, just that it no longer gets new features, fixes or functions, and you get that nag message when you do anything with pip. Since you've looked over the code, you realize that Syncserver doesn't do anything crazy

In addition, pypy, will be extending it's support for python 2.7.* past 2020, which should give us all a bit more ramp.

So, the executive summary:

1. Syncserver 1.5 should continue to be safe under python 2.7 for the short term.
2. You may be able to run Syncserver 1.5 under Pypy 2.7 if some critical security bug hits python 2.7.
3. We're working on a new rust based syncserver and hope to have it available with migration scripts in the next 6 months.

Again, my apologies for no one over here spelling any of this out. One of our goals for next year is to be a bit better about that.

[cjones26]

Thanks for the detailed response, @jrconlin, works for me!

[fireglow]

@jrconlin thank you for your detailed response! Much appreciated.

Should I leave this issue open, so other might find it easier?

[jrconlin]

Yeah, let's leave this open for now, until the "Future of Sync" blog post gets written at least. I'll try to remember to cross link back to this bug with any future plan or work around the sync server.

But, yeah, there's zero intention of dropping support for stand-alone sync server support. It's going to be a bit tricky with mobile devices (we're fixing up a lot of the internal service support with the android-components and the application-services modules for newer browser versions), but we're going to try and strong-arm that too.

[imadevel]

Hi,
are there any updates on the roadmap? The hope for the new rust server 6 months from now seems to be vanished since the post was 25 months ago. I hope its still happening

[jrconlin]

We recently integrated the TokenServer into SyncStorage. This reduces the number of dependent systems and simplifies things. Next up is a refactor of the crates to improve build times as well as help make the stand alone work a bit easier. (Likewise, modernizing things since there have been a number of significant improvements in Rust and other frameworks.)

[martin-braun]

Hi, @jrconlin. I'm a little bit confused as Connecting to Firefox still states that we need this legacy https://github.com/mozilla-services/syncserver/ to connect to https://github.com/mozilla-services/syncstorage-rs/.

Is this correct or has the documentation not been updated?