New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support account migration #177

Open
hach-que opened this Issue Nov 22, 2016 · 195 comments

Comments

Projects
None yet
@hach-que

hach-que commented Nov 22, 2016

A lot of people seem to be jumping on https://mastodon.social right now, even though the end goal is to have users separated out across multiple federated instances. However, if people start putting up a lot of content and getting followers on the primary instance, this will be a disincentive to move providers.

I think this largely means that Mastodon needs to support two things:

  • Account import across providers, which should be authenticated on both ends to prevent people bulk copying another person's account, and
  • A "redirected account" field against user accounts, which indicates the full URL of the new user account

When users configure a redirect location against their account (whether explicitly on an account page, or implicitly set during a cross-provider account import), the instance on which the account is should implicitly and automatically redirect followers.

That is, if I have the account @hq on the primary instance (which I do), and I set up the account @hach-que on another Mastodon service, the @hq should:

  • Remain on the primary instance, and not be disabled in any way
  • Show posts from @hach-que after the redirection is set up
  • Disallow posting from the @hq account while ever the redirection is in place
  • Existing followers of @hq should start seeing posts from @hach-que instead
  • New followers of @hq should be allowed to follow the account (and internally, they are following @hq, but see posts from @hach-que)
  • Followers should not actually have their lists updated to follow @hach-que - in the event of a mistaken redirect, removing the redirect should act exactly as if there never was a redirect in the first place

This is just my 2c on how I think this should work, but I'm interested to hear other people's thoughts.

@nex3

This comment has been minimized.

Show comment
Hide comment
@nex3

nex3 Nov 22, 2016

A "redirected account" field would also be very useful for renaming accounts within a single server.

nex3 commented Nov 22, 2016

A "redirected account" field would also be very useful for renaming accounts within a single server.

@Gargron

This comment has been minimized.

Show comment
Hide comment
@Gargron

Gargron Nov 22, 2016

Member

Sadly there's nothing in the OStatus protocol currently that's like "go follow my new account instead of this one". Best I can offer is migrating content (but reblogs/favs would not come along) and who you were following.

Member

Gargron commented Nov 22, 2016

Sadly there's nothing in the OStatus protocol currently that's like "go follow my new account instead of this one". Best I can offer is migrating content (but reblogs/favs would not come along) and who you were following.

@hach-que

This comment has been minimized.

Show comment
Hide comment
@hach-que

hach-que Nov 22, 2016

I had a bit of a look at the OSocial spec, and it's not 100% clear on what extensions the protocol permits in the <author /> section. All the documentation and links on Portable Contacts (poco) from Wikipedia seem to be dead. Is it possible we can add a new poco tag that contains a redirect or forwarding URL?

edit: or I guess a <link> tag with a different rel attribute?

hach-que commented Nov 22, 2016

I had a bit of a look at the OSocial spec, and it's not 100% clear on what extensions the protocol permits in the <author /> section. All the documentation and links on Portable Contacts (poco) from Wikipedia seem to be dead. Is it possible we can add a new poco tag that contains a redirect or forwarding URL?

edit: or I guess a <link> tag with a different rel attribute?

@nex3

This comment has been minimized.

Show comment
Hide comment
@nex3

nex3 Nov 22, 2016

Unless I'm missing something, shouldn't this be protocol-independent? As @hach-que describes it, as far as followers of the original account are concerned, they're still following that account. So as long as the node that that account lives on is able to proxy posts from the new account, this could be managed in a way that's totally transparent to the protocol.

It gets more complicated if you want new followers to automatically follow the new account instead, but that doesn't match the user-visible behavior described above.

nex3 commented Nov 22, 2016

Unless I'm missing something, shouldn't this be protocol-independent? As @hach-que describes it, as far as followers of the original account are concerned, they're still following that account. So as long as the node that that account lives on is able to proxy posts from the new account, this could be managed in a way that's totally transparent to the protocol.

It gets more complicated if you want new followers to automatically follow the new account instead, but that doesn't match the user-visible behavior described above.

@nex3

This comment has been minimized.

Show comment
Hide comment
@nex3

nex3 Nov 23, 2016

I've been thinking about this more. It's an important prerequisite for making federation practical, which I know is something @Gargron cares about a lot. We're already seeing a lot of users making accounts and on mastodon.social in particular, and their investment in the community will (hopefully) only grow in the future.

Existing users are the most likely to understand the value of federation, but they're also the most likely to have a lot to lose by abandoning an existing account. At the same time, most people will only move to other servers if the barrier to entry is very low. Allowing accounts to be migrated without losing their investment in the community will remove the biggest part of that barrier.

nex3 commented Nov 23, 2016

I've been thinking about this more. It's an important prerequisite for making federation practical, which I know is something @Gargron cares about a lot. We're already seeing a lot of users making accounts and on mastodon.social in particular, and their investment in the community will (hopefully) only grow in the future.

Existing users are the most likely to understand the value of federation, but they're also the most likely to have a lot to lose by abandoning an existing account. At the same time, most people will only move to other servers if the barrier to entry is very low. Allowing accounts to be migrated without losing their investment in the community will remove the biggest part of that barrier.

@hikari-no-yume

This comment has been minimized.

Show comment
Hide comment
@hikari-no-yume

hikari-no-yume Dec 6, 2016

Contributor

I wonder if this is something other OStatus implementations would be interested in.

Contributor

hikari-no-yume commented Dec 6, 2016

I wonder if this is something other OStatus implementations would be interested in.

@marrus-sh

This comment has been minimized.

Show comment
Hide comment
@marrus-sh

marrus-sh Dec 7, 2016

Contributor

As an in-the-meantime stepping-stone for this, it would be nice if there was a simple import/export of who you're currently following, so that one could easily migrate that over to a new account. It's easy enough to post a “Changing accounts! Follow me at @█████” toot to let followers know you're changing accounts, but significantly harder to ensure you're following the same people on the new one.

Contributor

marrus-sh commented Dec 7, 2016

As an in-the-meantime stepping-stone for this, it would be nice if there was a simple import/export of who you're currently following, so that one could easily migrate that over to a new account. It's easy enough to post a “Changing accounts! Follow me at @█████” toot to let followers know you're changing accounts, but significantly harder to ensure you're following the same people on the new one.

@hoodieak

This comment has been minimized.

Show comment
Hide comment
@hoodieak

hoodieak Dec 15, 2016

i know that this is an old issue but i'm thinking more so about making a link for accounts, rather than a migration, namely allowing backwards-compatibility, making follow either mean the same account. this seems like it vaguely might be easier than migration? and i think both are very strong and important things to add to making the multiple instance setup for mastodon work

hoodieak commented Dec 15, 2016

i know that this is an old issue but i'm thinking more so about making a link for accounts, rather than a migration, namely allowing backwards-compatibility, making follow either mean the same account. this seems like it vaguely might be easier than migration? and i think both are very strong and important things to add to making the multiple instance setup for mastodon work

@hoodieak

This comment has been minimized.

Show comment
Hide comment
@hoodieak

hoodieak Dec 15, 2016

i would also suggest a profile option at the very least to show where a person has moved. it does not have to automatically move accounts, but making it easier for users to understand that a user has migrated is easy: could make whenever the new account posted, it would remind all followers of the original account to migrate from that they are posting elsewhere and link to the new account, as well as making a single button to unfollow the original and follow the new one.

hoodieak commented Dec 15, 2016

i would also suggest a profile option at the very least to show where a person has moved. it does not have to automatically move accounts, but making it easier for users to understand that a user has migrated is easy: could make whenever the new account posted, it would remind all followers of the original account to migrate from that they are posting elsewhere and link to the new account, as well as making a single button to unfollow the original and follow the new one.

@hoodieak

This comment has been minimized.

Show comment
Hide comment
@hoodieak

hoodieak Dec 15, 2016

if we get REALLY fancy and i would heavily suggest this: the old account could delete the first post to move accounts whenever it posts it again to update that the migrated account has updated a post, to remind users to migrate manually, while also retaining the repository of old posts as a result.

hoodieak commented Dec 15, 2016

if we get REALLY fancy and i would heavily suggest this: the old account could delete the first post to move accounts whenever it posts it again to update that the migrated account has updated a post, to remind users to migrate manually, while also retaining the repository of old posts as a result.

@leeeeeny

This comment has been minimized.

Show comment
Hide comment
@leeeeeny

leeeeeny Apr 3, 2017

Exporting/importing following/blocking lists is great but in my opinion it's not enough. As OP said, people who've tooted will probably stick with their first account instead of moving. I do think that too. I'd like to create an instance and move my account to it but i don't want to get a new account (plus we can't use multiple account at the same time like in TweetDeck). I think an account migration/backup tool is the most missing feature of Mastodon. Complicated sync over instances features are not needed, just a tool to export (CSV or whatever) and delete an account and import it back on another instance would allow people to control their account. Since the instances are managed by the instances' owners, they may be unreliable (reports not processed seriously, 50 % uptime). Then you just have to move.

leeeeeny commented Apr 3, 2017

Exporting/importing following/blocking lists is great but in my opinion it's not enough. As OP said, people who've tooted will probably stick with their first account instead of moving. I do think that too. I'd like to create an instance and move my account to it but i don't want to get a new account (plus we can't use multiple account at the same time like in TweetDeck). I think an account migration/backup tool is the most missing feature of Mastodon. Complicated sync over instances features are not needed, just a tool to export (CSV or whatever) and delete an account and import it back on another instance would allow people to control their account. Since the instances are managed by the instances' owners, they may be unreliable (reports not processed seriously, 50 % uptime). Then you just have to move.

@Maverynthia

This comment has been minimized.

Show comment
Hide comment
@Maverynthia

Maverynthia Apr 3, 2017

Why is it hard to migrate followers? Can't the new system just point to the people on the old system.
Such as if I follow @name on the old system, then when I migrate it then points to @name@mastodon.social or something? Or even just an auto follow and then they have to refollow you?

Maverynthia commented Apr 3, 2017

Why is it hard to migrate followers? Can't the new system just point to the people on the old system.
Such as if I follow @name on the old system, then when I migrate it then points to @name@mastodon.social or something? Or even just an auto follow and then they have to refollow you?

@gdurelle

This comment has been minimized.

Show comment
Hide comment
@gdurelle

gdurelle Apr 4, 2017

The idea is great but the example is a bit complicated, don't you think ?

If I want to migrate my account from mastodon.xyz to another instance, i would want to keep the same username.

Supporting username change should be a "second step" after this first step. (baby steps like neagan says ;))

What I see :
1 - I go the new instance says i want to migrate from mastodon.xyz,
2 - My username is temporary reserved on the new instance
3 - I indicate my login, then it redirects me to mastodon.xyz for authentication (OAuth2)
4 - Once authorized, the migration process begins by importing my parameters in the new instance (following, followers, toots, etc)
5 - delete my account from mastodon.xyz
6 - Enable me on the new instance
7 - update my following with my new address.

my 2cts. (gonna have a lot of cents if we all add up)

gdurelle commented Apr 4, 2017

The idea is great but the example is a bit complicated, don't you think ?

If I want to migrate my account from mastodon.xyz to another instance, i would want to keep the same username.

Supporting username change should be a "second step" after this first step. (baby steps like neagan says ;))

What I see :
1 - I go the new instance says i want to migrate from mastodon.xyz,
2 - My username is temporary reserved on the new instance
3 - I indicate my login, then it redirects me to mastodon.xyz for authentication (OAuth2)
4 - Once authorized, the migration process begins by importing my parameters in the new instance (following, followers, toots, etc)
5 - delete my account from mastodon.xyz
6 - Enable me on the new instance
7 - update my following with my new address.

my 2cts. (gonna have a lot of cents if we all add up)

@leeeeeny

This comment has been minimized.

Show comment
Hide comment
@leeeeeny

leeeeeny Apr 4, 2017

Yes that's a bit complicated but imo this is a very important feature to permit existing users balancing between the instances and avoid account loss in case of instance shutdown.

leeeeeny commented Apr 4, 2017

Yes that's a bit complicated but imo this is a very important feature to permit existing users balancing between the instances and avoid account loss in case of instance shutdown.

@Sadzeih

This comment has been minimized.

Show comment
Hide comment
@Sadzeih

Sadzeih Apr 4, 2017

I agree with everyone here. Account migration is necessary. The most important is toots and followers for me. Followings too but they can already be exported. Username would be a good thing too.

Sadzeih commented Apr 4, 2017

I agree with everyone here. Account migration is necessary. The most important is toots and followers for me. Followings too but they can already be exported. Username would be a good thing too.

@Motoma

This comment has been minimized.

Show comment
Hide comment
@Motoma

Motoma Apr 4, 2017

Contributor

Account migration is a necessity if you want federation to be a first-class citizen. Users need to be insulated from the risks associated with decentralization. If an instance goes away (crashes, is hacked, shut down by an oppressive regime) those users should not be SOL. Lacking that, users will flock to the instance with the highest likelihood of surviving.

Contributor

Motoma commented Apr 4, 2017

Account migration is a necessity if you want federation to be a first-class citizen. Users need to be insulated from the risks associated with decentralization. If an instance goes away (crashes, is hacked, shut down by an oppressive regime) those users should not be SOL. Lacking that, users will flock to the instance with the highest likelihood of surviving.

@cyphar

This comment has been minimized.

Show comment
Hide comment
@cyphar

cyphar Apr 4, 2017

Well, there's two ways (in my mind) of fixing the problem:

  1. You do it the GitHub way, where the old username is now forwarded to the new username. I'm not sure if OStatus has a concept of 301 redirects, but that's how you would do it. The downside is that if the server goes down before a client has got the redirect they're SoL.

  2. You handle it the Matrix way, which is that you have identity servers that decouple the "real" ID (@username@instance) from the person whose ID it belongs to. The downside is that Matrix implemented this in a very centralised way, which just punts the issue. Not to mention OStatus almost certainly doesn't have support for such a concept.

cyphar commented Apr 4, 2017

Well, there's two ways (in my mind) of fixing the problem:

  1. You do it the GitHub way, where the old username is now forwarded to the new username. I'm not sure if OStatus has a concept of 301 redirects, but that's how you would do it. The downside is that if the server goes down before a client has got the redirect they're SoL.

  2. You handle it the Matrix way, which is that you have identity servers that decouple the "real" ID (@username@instance) from the person whose ID it belongs to. The downside is that Matrix implemented this in a very centralised way, which just punts the issue. Not to mention OStatus almost certainly doesn't have support for such a concept.

@cyphar

This comment has been minimized.

Show comment
Hide comment
@cyphar

cyphar Apr 4, 2017

@TehShrike This issue is about moving between Mastodon instances, not from Twitter to Mastodon.

cyphar commented Apr 4, 2017

@TehShrike This issue is about moving between Mastodon instances, not from Twitter to Mastodon.

@TehShrike

This comment has been minimized.

Show comment
Hide comment
@TehShrike

TehShrike Apr 4, 2017

Ah, I googled-and-skimmed too quickly >_< I'll delete that post.

TehShrike commented Apr 4, 2017

Ah, I googled-and-skimmed too quickly >_< I'll delete that post.

@cdata

This comment has been minimized.

Show comment
Hide comment
@cdata

cdata Apr 4, 2017

My hand-wave-y two cents:

It would be nice if there was a decoupled notion of identity. For example: a keybase.io profile aggregates different accounts for a single person so that they can be easily identified across all networks.

If this were true, you could imagine a system where you follow someone on Mastodon by following an identity and all addresses (that is, instance/profile combinations) that are associated with it.

Of course, it isn't easy to maintain a trusted database of identities that is shareable across all federated nodes. But, it's possible. Perhaps we can discuss the feasibility of such an approach.

👋

cdata commented Apr 4, 2017

My hand-wave-y two cents:

It would be nice if there was a decoupled notion of identity. For example: a keybase.io profile aggregates different accounts for a single person so that they can be easily identified across all networks.

If this were true, you could imagine a system where you follow someone on Mastodon by following an identity and all addresses (that is, instance/profile combinations) that are associated with it.

Of course, it isn't easy to maintain a trusted database of identities that is shareable across all federated nodes. But, it's possible. Perhaps we can discuss the feasibility of such an approach.

👋

@stiell

This comment has been minimized.

Show comment
Hide comment
@stiell

stiell Apr 4, 2017

Blockstack might be a good candidate for providing a decentralised, global identity. With Blockstack you could keep the same ID (and thus your followers) even if the instance you are currently using goes down permanently.

stiell commented Apr 4, 2017

Blockstack might be a good candidate for providing a decentralised, global identity. With Blockstack you could keep the same ID (and thus your followers) even if the instance you are currently using goes down permanently.

@cyphar

This comment has been minimized.

Show comment
Hide comment
@cyphar

cyphar Apr 4, 2017

The problem with having separated identities is that it wouldn't be compatible with other OStatus implementations (Mastodon isn't the only implementation, GNU Social is another and there's quite a few more). Migration should be something that someone following a Mastodon user from GNU Social should be able to support.

cyphar commented Apr 4, 2017

The problem with having separated identities is that it wouldn't be compatible with other OStatus implementations (Mastodon isn't the only implementation, GNU Social is another and there's quite a few more). Migration should be something that someone following a Mastodon user from GNU Social should be able to support.

@cdata

This comment has been minimized.

Show comment
Hide comment
@cdata

cdata Apr 4, 2017

In the fullness of time, I agree that this should be something that is broadly compatible with GNU Social. In the isolation of the moment, I propose that evolution requires experimentation. Compatibility is a matter of adding sufficient layers to make the standardized things cooperate with the experimental thing.

My brain told my hand to stop waving. I'm sure it will stop soon.

cdata commented Apr 4, 2017

In the fullness of time, I agree that this should be something that is broadly compatible with GNU Social. In the isolation of the moment, I propose that evolution requires experimentation. Compatibility is a matter of adding sufficient layers to make the standardized things cooperate with the experimental thing.

My brain told my hand to stop waving. I'm sure it will stop soon.

@SirCmpwn

This comment has been minimized.

Show comment
Hide comment
@SirCmpwn

SirCmpwn Apr 4, 2017

Contributor

I think you guys are making a mountain out of a molehill here. I think this problem is pretty easy to solve. In order of least to most controversial: we should make it possible to import and export toots, we should make an automated migration tool (log into your old account via OAuth from your new instance), and we should toot out a notice of account change.

The migration toot would be a normal toot with a human-readable message ("My account is moving to (link)"), but would include something like this:

<mastodon:migrate participant="origin">https://new-account</mastodon:migrate>

The new account would toot something like this to confirm it ("Moved from my old account (link)"):

<mastodon:migrate participant="destination">https://old-account</mastodon:migrate>

Then, clients can recognize this attribute and perform the migration when they see the toot by unfollowing and following the new account (possibly with an another XML extension that indicates the user is responding to an account migration). Clients would likely want to hide the special toots from the feed as well. Clients that don't support it have the human-readable version as a fallback. Then we just send patches to GNU Social and other projects to recognize this behavior.

Contributor

SirCmpwn commented Apr 4, 2017

I think you guys are making a mountain out of a molehill here. I think this problem is pretty easy to solve. In order of least to most controversial: we should make it possible to import and export toots, we should make an automated migration tool (log into your old account via OAuth from your new instance), and we should toot out a notice of account change.

The migration toot would be a normal toot with a human-readable message ("My account is moving to (link)"), but would include something like this:

<mastodon:migrate participant="origin">https://new-account</mastodon:migrate>

The new account would toot something like this to confirm it ("Moved from my old account (link)"):

<mastodon:migrate participant="destination">https://old-account</mastodon:migrate>

Then, clients can recognize this attribute and perform the migration when they see the toot by unfollowing and following the new account (possibly with an another XML extension that indicates the user is responding to an account migration). Clients would likely want to hide the special toots from the feed as well. Clients that don't support it have the human-readable version as a fallback. Then we just send patches to GNU Social and other projects to recognize this behavior.

@wedza

This comment has been minimized.

Show comment
Hide comment
@wedza

wedza Jun 24, 2018

Why not integrate this into the registration process. When someone registers in a new instance, this instance will check if the mail address used to create the new account is already used on other public instances.

If the same mail address is detected on another account on another instance, the new account on the instance will propose to set up the redirection settings (to change from one instance to another, with all the parameters that @hach-que has purposed). The user can refuse and decide to do nothing, and start from an empty account.

This will make the process simpler for a broader audience who don't want to mess with theses redirections settings.

wedza commented Jun 24, 2018

Why not integrate this into the registration process. When someone registers in a new instance, this instance will check if the mail address used to create the new account is already used on other public instances.

If the same mail address is detected on another account on another instance, the new account on the instance will propose to set up the redirection settings (to change from one instance to another, with all the parameters that @hach-que has purposed). The user can refuse and decide to do nothing, and start from an empty account.

This will make the process simpler for a broader audience who don't want to mess with theses redirections settings.

@remram44

This comment has been minimized.

Show comment
Hide comment
@remram44

remram44 Jun 24, 2018

Contributor

check if the mail address used to create the new account is already used on other public instances.

How do you do that? How do you prevent other instances from lying? How do you prove to the old instance that this should happen? How do you deal with the old instance being broken, or malicious?

Contributor

remram44 commented Jun 24, 2018

check if the mail address used to create the new account is already used on other public instances.

How do you do that? How do you prevent other instances from lying? How do you prove to the old instance that this should happen? How do you deal with the old instance being broken, or malicious?

@Shywim

This comment has been minimized.

Show comment
Hide comment
@Shywim

Shywim Jun 24, 2018

One way would be the two instances sending each a validation email. The only case in which there is a problem is if the instance you want to move from is malicious, but you can't really prevent anything in this case.

Shywim commented Jun 24, 2018

One way would be the two instances sending each a validation email. The only case in which there is a problem is if the instance you want to move from is malicious, but you can't really prevent anything in this case.

@penguin42

This comment has been minimized.

Show comment
Hide comment
@penguin42

penguin42 Jun 24, 2018

remram44: Dealing with the old instance being broken is why I said above (and split out as issue 7715) is that what's really needed is live mirroring between accounts, so you setup a spare before one side fails.
To do migration, I guess you could use signed toots, with some form of message that advertises the change.

penguin42 commented Jun 24, 2018

remram44: Dealing with the old instance being broken is why I said above (and split out as issue 7715) is that what's really needed is live mirroring between accounts, so you setup a spare before one side fails.
To do migration, I guess you could use signed toots, with some form of message that advertises the change.

@trwnh

This comment has been minimized.

Show comment
Hide comment
@trwnh

trwnh Jun 24, 2018

Contributor

@remram44

My concern with a freely chosen GUID is that collisions are possible. A bad actor could use the same GUID

Random collision in 256-bit will take longer than the life of the universe. Intentional collision would be prevented by cryptography. At least, that's how zot does it -- see: what an identity consists of, #177 (comment) (uid and private key, plus address book if offline).

Why not use the old username@domain in place of the GUID? And just announce that the old handle is now at a new location?

That might work only for the purposes of migration, but for basically nothing else. It also requires validation of some sort with the server on that domain, which might be offline. It also will not be preserved when migrating to a new server/domain -- the point of the GUID is to identify the two accounts as the same without revealing a key. Verification happens in a different step.

@TehShrike

I'm fine with identity being tied to server. I just want [...] all my followers and follow lists [to] move [...] If [the old server] maintained a "forwarding address" to the new location, that would be cool too, but you can't rely on that staying up forever.

That's exactly what would ideally be avoided. If all you care about is having your address book move, then much of that is already possible. But IMO, a proper "migration" would unambiguously move your entire account, not just the details of it.

Changing the nature of identity on mastodon sounds much more difficult.

It will definitely be difficult, which is why the plan I described proposes a gradual change before it becomes official (in a v3.0.0 change). Right now, the fact that identity is tied to user/domain is a major blocking issue to any sort of migration. If that doesn't change, then Mastodon will never be able to gracefully handle anything more than the "account moved" notice we already have. Any existing instance will see your moved account as an entirely different account, which is extremely inconvenient.

@wedza

Why not integrate this into the registration process. When someone registers in a new instance, this instance will check if the mail address used to create the new account is already used on other public instances.

That doesn't really solve the issue. Signing up with the same email can already be done for multiple instances, to create separate accounts. Also, it's a really bad idea to expose the email address used by an account. But you do have a point that it probably should be exposed during registration -- in fact, Friendica and Hubzilla already do this, offering you options to import an account or to set up a new one. This has nothing to do with the email used, though; it just uses the ID/keys/posts/contacts.

@penguin42

what's really needed is live mirroring between accounts, so you setup a spare before one side fails.

That would easily be possible once the secure exchange / import logic is implemented. The only difference would be setting the new server as a secondary instance rather than as a new primary instance.

Contributor

trwnh commented Jun 24, 2018

@remram44

My concern with a freely chosen GUID is that collisions are possible. A bad actor could use the same GUID

Random collision in 256-bit will take longer than the life of the universe. Intentional collision would be prevented by cryptography. At least, that's how zot does it -- see: what an identity consists of, #177 (comment) (uid and private key, plus address book if offline).

Why not use the old username@domain in place of the GUID? And just announce that the old handle is now at a new location?

That might work only for the purposes of migration, but for basically nothing else. It also requires validation of some sort with the server on that domain, which might be offline. It also will not be preserved when migrating to a new server/domain -- the point of the GUID is to identify the two accounts as the same without revealing a key. Verification happens in a different step.

@TehShrike

I'm fine with identity being tied to server. I just want [...] all my followers and follow lists [to] move [...] If [the old server] maintained a "forwarding address" to the new location, that would be cool too, but you can't rely on that staying up forever.

That's exactly what would ideally be avoided. If all you care about is having your address book move, then much of that is already possible. But IMO, a proper "migration" would unambiguously move your entire account, not just the details of it.

Changing the nature of identity on mastodon sounds much more difficult.

It will definitely be difficult, which is why the plan I described proposes a gradual change before it becomes official (in a v3.0.0 change). Right now, the fact that identity is tied to user/domain is a major blocking issue to any sort of migration. If that doesn't change, then Mastodon will never be able to gracefully handle anything more than the "account moved" notice we already have. Any existing instance will see your moved account as an entirely different account, which is extremely inconvenient.

@wedza

Why not integrate this into the registration process. When someone registers in a new instance, this instance will check if the mail address used to create the new account is already used on other public instances.

That doesn't really solve the issue. Signing up with the same email can already be done for multiple instances, to create separate accounts. Also, it's a really bad idea to expose the email address used by an account. But you do have a point that it probably should be exposed during registration -- in fact, Friendica and Hubzilla already do this, offering you options to import an account or to set up a new one. This has nothing to do with the email used, though; it just uses the ID/keys/posts/contacts.

@penguin42

what's really needed is live mirroring between accounts, so you setup a spare before one side fails.

That would easily be possible once the secure exchange / import logic is implemented. The only difference would be setting the new server as a secondary instance rather than as a new primary instance.

@remram44

This comment has been minimized.

Show comment
Hide comment
@remram44

remram44 Jun 24, 2018

Contributor

Random collision in 256-bit will take longer than the life of the universe. Intentional collision would be prevented by cryptography.

Intentional collisions are exactly what I'm concerned about. "Prevented by cryptography" only happens if you use a public key hash instead of a GUID. Note that GUID means something specific: those are 128-bit values with a specific (versioned) schema.

Contributor

remram44 commented Jun 24, 2018

Random collision in 256-bit will take longer than the life of the universe. Intentional collision would be prevented by cryptography.

Intentional collisions are exactly what I'm concerned about. "Prevented by cryptography" only happens if you use a public key hash instead of a GUID. Note that GUID means something specific: those are 128-bit values with a specific (versioned) schema.

@trwnh

This comment has been minimized.

Show comment
Hide comment
@trwnh

trwnh Jun 25, 2018

Contributor

@remram44 Ah, my bad, I wasn't clear on the differences between guid/uuid/etc. I've been using the terms in the most generic sense possible (a globally unique user ID). Regardless, the (256-bit) UID alone will not be enough to constitute an identity; it is only part of it. A private key and address book make up the other parts (to prove ownership + rebuild connections).

Contributor

trwnh commented Jun 25, 2018

@remram44 Ah, my bad, I wasn't clear on the differences between guid/uuid/etc. I've been using the terms in the most generic sense possible (a globally unique user ID). Regardless, the (256-bit) UID alone will not be enough to constitute an identity; it is only part of it. A private key and address book make up the other parts (to prove ownership + rebuild connections).

@LeeteqXV

This comment has been minimized.

Show comment
Hide comment
@LeeteqXV

LeeteqXV Jun 25, 2018

LeeteqXV commented Jun 25, 2018

@trwnh

This comment has been minimized.

Show comment
Hide comment
@trwnh

trwnh Jul 20, 2018

Contributor

Addendum to the discussion with @remram44 about using public key hashes as UIDs: I don't think that would work in the current schema where every actor's public key is actually just the server's public key. Mastodon doesn't currently generate keypairs for every single actor. My proposal to start generating UIDs is to lay the groundwork for changes, not to be used immediately. That would allow cryptographic changes later on a server-to-server basis -- live migration is the more pressing thing to be tackled IMO, since offline migration requires per-actor keypairs.

Contributor

trwnh commented Jul 20, 2018

Addendum to the discussion with @remram44 about using public key hashes as UIDs: I don't think that would work in the current schema where every actor's public key is actually just the server's public key. Mastodon doesn't currently generate keypairs for every single actor. My proposal to start generating UIDs is to lay the groundwork for changes, not to be used immediately. That would allow cryptographic changes later on a server-to-server basis -- live migration is the more pressing thing to be tackled IMO, since offline migration requires per-actor keypairs.

@nightpool

This comment has been minimized.

Show comment
Hide comment
@nightpool

nightpool Jul 22, 2018

Collaborator

Mastodon doesn't currently generate keypairs for every single actor

what? this is incorrect. How are you coming to this conclusion?

Collaborator

nightpool commented Jul 22, 2018

Mastodon doesn't currently generate keypairs for every single actor

what? this is incorrect. How are you coming to this conclusion?

@trwnh

This comment has been minimized.

Show comment
Hide comment
@trwnh

trwnh Jul 22, 2018

Contributor

@nightpool examining the JSON output of two users on the same instance. I was mainly referring to server-server interactions and not client-server, because I have no idea how to examine what's happening privately. I may have overlooked something because I'm not perfectly familiar with how Mastodon generates all this stuff.

https://mastodon.social/@trwnh.json

"publicKey":{
"id":"https://mastodon.social/users/trwnh#main-key",
"owner":"https://mastodon.social/users/trwnh",
"publicKeyPem":"-----BEGIN PUBLIC KEY-----\nMIIBIjANBgk...-----END PUBLIC KEY-----\n"
}

https://mastodon.social/@gargron.json

"publicKey":{
"id":"https://mastodon.social/users/Gargron#main-key",
"owner":"https://mastodon.social/users/Gargron",
"publicKeyPem":"-----BEGIN PUBLIC KEY-----\nMIIBIjANBgk...-----END PUBLIC KEY-----\n"
}
Contributor

trwnh commented Jul 22, 2018

@nightpool examining the JSON output of two users on the same instance. I was mainly referring to server-server interactions and not client-server, because I have no idea how to examine what's happening privately. I may have overlooked something because I'm not perfectly familiar with how Mastodon generates all this stuff.

https://mastodon.social/@trwnh.json

"publicKey":{
"id":"https://mastodon.social/users/trwnh#main-key",
"owner":"https://mastodon.social/users/trwnh",
"publicKeyPem":"-----BEGIN PUBLIC KEY-----\nMIIBIjANBgk...-----END PUBLIC KEY-----\n"
}

https://mastodon.social/@gargron.json

"publicKey":{
"id":"https://mastodon.social/users/Gargron#main-key",
"owner":"https://mastodon.social/users/Gargron",
"publicKeyPem":"-----BEGIN PUBLIC KEY-----\nMIIBIjANBgk...-----END PUBLIC KEY-----\n"
}
@sireebob

This comment has been minimized.

Show comment
Hide comment
@sireebob

sireebob Jul 22, 2018

downloaded both and the public keys are different. some of the characters at the beginning and end are the same. that's standard

sireebob commented Jul 22, 2018

downloaded both and the public keys are different. some of the characters at the beginning and end are the same. that's standard

@nightpool

This comment has been minimized.

Show comment
Hide comment
@nightpool

nightpool Jul 22, 2018

Collaborator

@trwnh you're just seeing the framing of the internal binary encoding. the keys are completely different, they just share a similar structure (as do all pem-encoded asn.1 keys)

Collaborator

nightpool commented Jul 22, 2018

@trwnh you're just seeing the framing of the internal binary encoding. the keys are completely different, they just share a similar structure (as do all pem-encoded asn.1 keys)

@trwnh

This comment has been minimized.

Show comment
Hide comment
@trwnh

trwnh Jul 22, 2018

Contributor

@nightpool Ahhhh, okay yeah, I didn't check far enough, I just skimmed over the first and last few characters to see if they were the same. Whoops!

So, if keypairs are already generated for every actor/profile within Mastodon, that should be one less step to do. Assuming posts and private key are already included in the exported data (are they?) the last thing to do would be to pick/generate/assign/etc some sort of UID for each user, and then it should be "only" a matter of refactoring some logic and building the export/import tools.

I'm still not too sure of the advantages/disadvantages of picking a specific UID schema, per above -- the way I see it, defining a separate UID that isn't simply a hash of publicKey should allow for situations where the old key is repudiated or if the encryption hypothetically changed (e.g. in the future if/when more bits are needed for security) -- and the UID would still need to be connected to a keypair, so no changes there.

Contributor

trwnh commented Jul 22, 2018

@nightpool Ahhhh, okay yeah, I didn't check far enough, I just skimmed over the first and last few characters to see if they were the same. Whoops!

So, if keypairs are already generated for every actor/profile within Mastodon, that should be one less step to do. Assuming posts and private key are already included in the exported data (are they?) the last thing to do would be to pick/generate/assign/etc some sort of UID for each user, and then it should be "only" a matter of refactoring some logic and building the export/import tools.

I'm still not too sure of the advantages/disadvantages of picking a specific UID schema, per above -- the way I see it, defining a separate UID that isn't simply a hash of publicKey should allow for situations where the old key is repudiated or if the encryption hypothetically changed (e.g. in the future if/when more bits are needed for security) -- and the UID would still need to be connected to a keypair, so no changes there.

@blaine

This comment has been minimized.

Show comment
Hide comment
@blaine

blaine Aug 14, 2018

(I've skimmed previous posts, but there are a lot of them, apologies for any repetition)

fwiw, this is a big part of what we created webfinger for.

imho, the crypto approach is really complicated and anti-user. Do the same thing as HTTP, use [the equivalent of] a 301 redirect. If it's not in the standard, create a new one. ;-)

To be clear:

  1. I choose to move from me@mastodon.social to ohai@mydomain.elsewhere
  2. I tell the mastodon instance at mastodon.social to stop allowing posts there and to provide (in my mastondon.social profile) a redirect to ohai@mydomain.elsewhere.
  3. Any other instances where people follow me at me@mastodon.social should update their subscription to point to ohai@mydomain.elsewhere
  4. If I choose to migrate follower / followed lists, or even toots, that can be a separate API but (to the point of many here) migration is less important than portability, at least to start. Portability is a fundamental protocol requirement, portability is something that can be added later without core protocol support.

Note in the above, no new crypto is needed, all the security boundaries continue to work. There is an obvious hijacking exploit possible in the event that someone gains control of an account, so it would probably make sense to have a SHOULD clause for following user redirects that instructs federated servers to check back with the original server for some period of time (e.g., 30 days) to allow the original user to regain control and cancel the redirect. Again, this is an enhancement that could (should) be added later.

blaine commented Aug 14, 2018

(I've skimmed previous posts, but there are a lot of them, apologies for any repetition)

fwiw, this is a big part of what we created webfinger for.

imho, the crypto approach is really complicated and anti-user. Do the same thing as HTTP, use [the equivalent of] a 301 redirect. If it's not in the standard, create a new one. ;-)

To be clear:

  1. I choose to move from me@mastodon.social to ohai@mydomain.elsewhere
  2. I tell the mastodon instance at mastodon.social to stop allowing posts there and to provide (in my mastondon.social profile) a redirect to ohai@mydomain.elsewhere.
  3. Any other instances where people follow me at me@mastodon.social should update their subscription to point to ohai@mydomain.elsewhere
  4. If I choose to migrate follower / followed lists, or even toots, that can be a separate API but (to the point of many here) migration is less important than portability, at least to start. Portability is a fundamental protocol requirement, portability is something that can be added later without core protocol support.

Note in the above, no new crypto is needed, all the security boundaries continue to work. There is an obvious hijacking exploit possible in the event that someone gains control of an account, so it would probably make sense to have a SHOULD clause for following user redirects that instructs federated servers to check back with the original server for some period of time (e.g., 30 days) to allow the original user to regain control and cancel the redirect. Again, this is an enhancement that could (should) be added later.

@neuoy

This comment has been minimized.

Show comment
Hide comment
@neuoy

neuoy Aug 14, 2018

This makes sense, really. I'd just like to point out (you are probably aware of it) that the drawback of this method is that it doesn't work if an instance permanently goes down. Still better than nothing.

To have a more complete feature (and I'm not suggesting this would be required for a first step), I believe cryptography is required, so that a user is able to prove that he owns the account on the defunct instance. Other instances would know the account public key, only the user knows his private key. And as long as the instance is up, he should be able to update the key pair (in case it gets lost or compromised), which means the public key can't be the account unique ID.

neuoy commented Aug 14, 2018

This makes sense, really. I'd just like to point out (you are probably aware of it) that the drawback of this method is that it doesn't work if an instance permanently goes down. Still better than nothing.

To have a more complete feature (and I'm not suggesting this would be required for a first step), I believe cryptography is required, so that a user is able to prove that he owns the account on the defunct instance. Other instances would know the account public key, only the user knows his private key. And as long as the instance is up, he should be able to update the key pair (in case it gets lost or compromised), which means the public key can't be the account unique ID.

@trwnh

This comment has been minimized.

Show comment
Hide comment
@trwnh

trwnh Aug 15, 2018

Contributor

AFAIK, key generation is done on the server already, so there aren't any specific client-side issues to solve -- migration would involve copying a key from one server to another. If the instance is live, the user can authorize with another server. If the instance is offline, the user can upload a backup.

The point of changing the keys is a good one, and also why I think there should be a separate 256-bit user ID generated (the sooner the better). By the time Mastodon 3.0 hits, the majority of the ecosystem should have a user ID, and code should have been refactored to refer to this ID instead of to a user/domain pair. #177 (comment)

Contributor

trwnh commented Aug 15, 2018

AFAIK, key generation is done on the server already, so there aren't any specific client-side issues to solve -- migration would involve copying a key from one server to another. If the instance is live, the user can authorize with another server. If the instance is offline, the user can upload a backup.

The point of changing the keys is a good one, and also why I think there should be a separate 256-bit user ID generated (the sooner the better). By the time Mastodon 3.0 hits, the majority of the ecosystem should have a user ID, and code should have been refactored to refer to this ID instead of to a user/domain pair. #177 (comment)

@kevinmarks

This comment has been minimized.

Show comment
Hide comment
@kevinmarks

kevinmarks Aug 15, 2018

kevinmarks commented Aug 15, 2018

@trwnh

This comment has been minimized.

Show comment
Hide comment
@trwnh

trwnh Aug 15, 2018

Contributor

@kevinmarks Why would the users have to do any key management? Like I just said, all key management is done transparently on the server already, and the server is already managing everything for the user. User-facing crypto would be an entirely separate issue.

Contributor

trwnh commented Aug 15, 2018

@kevinmarks Why would the users have to do any key management? Like I just said, all key management is done transparently on the server already, and the server is already managing everything for the user. User-facing crypto would be an entirely separate issue.

@neuoy

This comment has been minimized.

Show comment
Hide comment
@neuoy

neuoy Aug 15, 2018

@kevinmarks @trwnh To make my thoughts clearer on that point, I also believe users should not have to handle their private key themselves, but having the possibility to do it would protect them against their instance unexpectedly shutting down. The private key would be proof of account ownership, they can delegate its handling to whoever they want (in exchange with password authentication for example). This would allow for additional assistance layers, such as instructing another instance to keep a backup for them.

I don't know any detail of the current protocol, this was just my two cents to answer @blaine about crypto not needed (in my opinion, it is needed, but only to solve the scenario where an instance goes down, which could be a second step in account migration functionality).

neuoy commented Aug 15, 2018

@kevinmarks @trwnh To make my thoughts clearer on that point, I also believe users should not have to handle their private key themselves, but having the possibility to do it would protect them against their instance unexpectedly shutting down. The private key would be proof of account ownership, they can delegate its handling to whoever they want (in exchange with password authentication for example). This would allow for additional assistance layers, such as instructing another instance to keep a backup for them.

I don't know any detail of the current protocol, this was just my two cents to answer @blaine about crypto not needed (in my opinion, it is needed, but only to solve the scenario where an instance goes down, which could be a second step in account migration functionality).

aaronpk pushed a commit to indieweb/wiki that referenced this issue Aug 16, 2018

@nicostuhlfauth

This comment has been minimized.

Show comment
Hide comment
@nicostuhlfauth

nicostuhlfauth Aug 17, 2018

Hi all,
I think especially in a decentralized network like Mastodon it's necessary to have the option to migrate to a new instance without loosing the own toots, favs and followers. It just needs to work, no matter if the old instance is still running or not.

If it's not possible, it automatically pushes more users to a few instances, which they believe to be stable and running for a long time.

So as there are many, many people interested in Mastodon and Twitter alternatives for now and they're just signing up without understanding the concept of instances, let's solve this issue asap.

nicostuhlfauth commented Aug 17, 2018

Hi all,
I think especially in a decentralized network like Mastodon it's necessary to have the option to migrate to a new instance without loosing the own toots, favs and followers. It just needs to work, no matter if the old instance is still running or not.

If it's not possible, it automatically pushes more users to a few instances, which they believe to be stable and running for a long time.

So as there are many, many people interested in Mastodon and Twitter alternatives for now and they're just signing up without understanding the concept of instances, let's solve this issue asap.

@lx4r

This comment has been minimized.

Show comment
Hide comment
@lx4r

lx4r Aug 17, 2018

I agree with @nicostuhlfauth that this should be solved immediately and would've proposed something similar to @blaine's solution, which sounds good to me. Problem: I don't know any Ruby, so now we just need an awesome person to implement this 🙂

lx4r commented Aug 17, 2018

I agree with @nicostuhlfauth that this should be solved immediately and would've proposed something similar to @blaine's solution, which sounds good to me. Problem: I don't know any Ruby, so now we just need an awesome person to implement this 🙂

@kevinmarks

This comment has been minimized.

Show comment
Hide comment
@kevinmarks

kevinmarks Aug 20, 2018

Rather than elaborate new id structures, what is stopping an instance doing a 301 redirect when a user that has migrated to another is requested?
Then the requesting instance can update the mapping from the local /user/[n] to the remote url to fetch and everything can carry on.
This was resolved in Websub with tests before shipping - is there some design difficulty with ActivityPub that precludes this?

kevinmarks commented Aug 20, 2018

Rather than elaborate new id structures, what is stopping an instance doing a 301 redirect when a user that has migrated to another is requested?
Then the requesting instance can update the mapping from the local /user/[n] to the remote url to fetch and everything can carry on.
This was resolved in Websub with tests before shipping - is there some design difficulty with ActivityPub that precludes this?

@remram44

This comment has been minimized.

Show comment
Hide comment
@remram44

remram44 Aug 20, 2018

Contributor

@kevinmarks the problems are migrating followers, and recovering from the instance going down permanently. Signaling that a user has moved is already implemented.

Contributor

remram44 commented Aug 20, 2018

@kevinmarks the problems are migrating followers, and recovering from the instance going down permanently. Signaling that a user has moved is already implemented.

@nicostuhlfauth

This comment has been minimized.

Show comment
Hide comment
@nicostuhlfauth

nicostuhlfauth Aug 21, 2018

By the way @remram44, do you know how the implemented moving works? Someone asked me a few days ago, but I found no documentation what actually happens when you enter your new user in the settings of the old server...

nicostuhlfauth commented Aug 21, 2018

By the way @remram44, do you know how the implemented moving works? Someone asked me a few days ago, but I found no documentation what actually happens when you enter your new user in the settings of the old server...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment