Support account migration #177

hach-que · Nov 22, 2016

A lot of people seem to be jumping on https://mastodon.social right now, even though the end goal is to have users separated out across multiple federated instances. However, if people start putting up a lot of content and getting followers on the primary instance, this will be a disincentive to move providers.

I think this largely means that Mastodon needs to support two things:

Account import across providers, which should be authenticated on both ends to prevent people bulk copying another person's account, and
A "redirected account" field against user accounts, which indicates the full URL of the new user account

When users configure a redirect location against their account (whether explicitly on an account page, or implicitly set during a cross-provider account import), the instance on which the account is should implicitly and automatically redirect followers.

That is, if I have the account @hq on the primary instance (which I do), and I set up the account @hach-que on another Mastodon service, the @hq should:

Remain on the primary instance, and not be disabled in any way
Show posts from @hach-que after the redirection is set up
Disallow posting from the @hq account while ever the redirection is in place
Existing followers of @hq should start seeing posts from @hach-que instead
New followers of @hq should be allowed to follow the account (and internally, they are following @hq, but see posts from @hach-que)
Followers should not actually have their lists updated to follow @hach-que - in the event of a mistaken redirect, removing the redirect should act exactly as if there never was a redirect in the first place

This is just my 2c on how I think this should work, but I'm interested to hear other people's thoughts.

A "redirected account" field would also be very useful for renaming accounts within a single server.

nex3 · Nov 22, 2016

A "redirected account" field would also be very useful for renaming accounts within a single server.

Sadly there's nothing in the OStatus protocol currently that's like "go follow my new account instead of this one". Best I can offer is migrating content (but reblogs/favs would not come along) and who you were following.

Gargron · Nov 22, 2016

Sadly there's nothing in the OStatus protocol currently that's like "go follow my new account instead of this one". Best I can offer is migrating content (but reblogs/favs would not come along) and who you were following.

I had a bit of a look at the OSocial spec, and it's not 100% clear on what extensions the protocol permits in the <author /> section. All the documentation and links on Portable Contacts (poco) from Wikipedia seem to be dead. Is it possible we can add a new poco tag that contains a redirect or forwarding URL?

edit: or I guess a <link> tag with a different rel attribute?

hach-que · Nov 22, 2016

I had a bit of a look at the OSocial spec, and it's not 100% clear on what extensions the protocol permits in the <author /> section. All the documentation and links on Portable Contacts (poco) from Wikipedia seem to be dead. Is it possible we can add a new poco tag that contains a redirect or forwarding URL?

edit: or I guess a <link> tag with a different rel attribute?

Unless I'm missing something, shouldn't this be protocol-independent? As @hach-que describes it, as far as followers of the original account are concerned, they're still following that account. So as long as the node that that account lives on is able to proxy posts from the new account, this could be managed in a way that's totally transparent to the protocol.

It gets more complicated if you want new followers to automatically follow the new account instead, but that doesn't match the user-visible behavior described above.

nex3 · Nov 22, 2016

Unless I'm missing something, shouldn't this be protocol-independent? As @hach-que describes it, as far as followers of the original account are concerned, they're still following that account. So as long as the node that that account lives on is able to proxy posts from the new account, this could be managed in a way that's totally transparent to the protocol.

It gets more complicated if you want new followers to automatically follow the new account instead, but that doesn't match the user-visible behavior described above.

I've been thinking about this more. It's an important prerequisite for making federation practical, which I know is something @Gargron cares about a lot. We're already seeing a lot of users making accounts and on mastodon.social in particular, and their investment in the community will (hopefully) only grow in the future.

Existing users are the most likely to understand the value of federation, but they're also the most likely to have a lot to lose by abandoning an existing account. At the same time, most people will only move to other servers if the barrier to entry is very low. Allowing accounts to be migrated without losing their investment in the community will remove the biggest part of that barrier.

nex3 · Nov 23, 2016

I've been thinking about this more. It's an important prerequisite for making federation practical, which I know is something @Gargron cares about a lot. We're already seeing a lot of users making accounts and on mastodon.social in particular, and their investment in the community will (hopefully) only grow in the future.

Existing users are the most likely to understand the value of federation, but they're also the most likely to have a lot to lose by abandoning an existing account. At the same time, most people will only move to other servers if the barrier to entry is very low. Allowing accounts to be migrated without losing their investment in the community will remove the biggest part of that barrier.

I wonder if this is something other OStatus implementations would be interested in.

hikari-no-yume · Dec 6, 2016

I wonder if this is something other OStatus implementations would be interested in.

As an in-the-meantime stepping-stone for this, it would be nice if there was a simple import/export of who you're currently following, so that one could easily migrate that over to a new account. It's easy enough to post a “Changing accounts! Follow me at @█████” toot to let followers know you're changing accounts, but significantly harder to ensure you're following the same people on the new one.

marrus-sh · Dec 7, 2016

As an in-the-meantime stepping-stone for this, it would be nice if there was a simple import/export of who you're currently following, so that one could easily migrate that over to a new account. It's easy enough to post a “Changing accounts! Follow me at @█████” toot to let followers know you're changing accounts, but significantly harder to ensure you're following the same people on the new one.

i know that this is an old issue but i'm thinking more so about making a link for accounts, rather than a migration, namely allowing backwards-compatibility, making follow either mean the same account. this seems like it vaguely might be easier than migration? and i think both are very strong and important things to add to making the multiple instance setup for mastodon work

hoodieak · Dec 15, 2016

i know that this is an old issue but i'm thinking more so about making a link for accounts, rather than a migration, namely allowing backwards-compatibility, making follow either mean the same account. this seems like it vaguely might be easier than migration? and i think both are very strong and important things to add to making the multiple instance setup for mastodon work

i would also suggest a profile option at the very least to show where a person has moved. it does not have to automatically move accounts, but making it easier for users to understand that a user has migrated is easy: could make whenever the new account posted, it would remind all followers of the original account to migrate from that they are posting elsewhere and link to the new account, as well as making a single button to unfollow the original and follow the new one.

hoodieak · Dec 15, 2016

i would also suggest a profile option at the very least to show where a person has moved. it does not have to automatically move accounts, but making it easier for users to understand that a user has migrated is easy: could make whenever the new account posted, it would remind all followers of the original account to migrate from that they are posting elsewhere and link to the new account, as well as making a single button to unfollow the original and follow the new one.

if we get REALLY fancy and i would heavily suggest this: the old account could delete the first post to move accounts whenever it posts it again to update that the migrated account has updated a post, to remind users to migrate manually, while also retaining the repository of old posts as a result.

hoodieak · Dec 15, 2016

if we get REALLY fancy and i would heavily suggest this: the old account could delete the first post to move accounts whenever it posts it again to update that the migrated account has updated a post, to remind users to migrate manually, while also retaining the repository of old posts as a result.

Exporting/importing following/blocking lists is great but in my opinion it's not enough. As OP said, people who've tooted will probably stick with their first account instead of moving. I do think that too. I'd like to create an instance and move my account to it but i don't want to get a new account (plus we can't use multiple account at the same time like in TweetDeck). I think an account migration/backup tool is the most missing feature of Mastodon. Complicated sync over instances features are not needed, just a tool to export (CSV or whatever) and delete an account and import it back on another instance would allow people to control their account. Since the instances are managed by the instances' owners, they may be unreliable (reports not processed seriously, 50 % uptime). Then you just have to move.

leeeeeny · Apr 3, 2017

Exporting/importing following/blocking lists is great but in my opinion it's not enough. As OP said, people who've tooted will probably stick with their first account instead of moving. I do think that too. I'd like to create an instance and move my account to it but i don't want to get a new account (plus we can't use multiple account at the same time like in TweetDeck). I think an account migration/backup tool is the most missing feature of Mastodon. Complicated sync over instances features are not needed, just a tool to export (CSV or whatever) and delete an account and import it back on another instance would allow people to control their account. Since the instances are managed by the instances' owners, they may be unreliable (reports not processed seriously, 50 % uptime). Then you just have to move.

Why is it hard to migrate followers? Can't the new system just point to the people on the old system.
Such as if I follow @name on the old system, then when I migrate it then points to @name@mastodon.social or something? Or even just an auto follow and then they have to refollow you?

Maverynthia · Apr 3, 2017

Why is it hard to migrate followers? Can't the new system just point to the people on the old system.
Such as if I follow @name on the old system, then when I migrate it then points to @name@mastodon.social or something? Or even just an auto follow and then they have to refollow you?

The idea is great but the example is a bit complicated, don't you think ?

If I want to migrate my account from mastodon.xyz to another instance, i would want to keep the same username.

Supporting username change should be a "second step" after this first step. (baby steps like neagan says ;))

What I see :
1 - I go the new instance says i want to migrate from mastodon.xyz,
2 - My username is temporary reserved on the new instance
3 - I indicate my login, then it redirects me to mastodon.xyz for authentication (OAuth2)
4 - Once authorized, the migration process begins by importing my parameters in the new instance (following, followers, toots, etc)
5 - delete my account from mastodon.xyz
6 - Enable me on the new instance
7 - update my following with my new address.

my 2cts. (gonna have a lot of cents if we all add up)

gdurelle · Apr 4, 2017

The idea is great but the example is a bit complicated, don't you think ?

If I want to migrate my account from mastodon.xyz to another instance, i would want to keep the same username.

Supporting username change should be a "second step" after this first step. (baby steps like neagan says ;))

What I see :
1 - I go the new instance says i want to migrate from mastodon.xyz,
2 - My username is temporary reserved on the new instance
3 - I indicate my login, then it redirects me to mastodon.xyz for authentication (OAuth2)
4 - Once authorized, the migration process begins by importing my parameters in the new instance (following, followers, toots, etc)
5 - delete my account from mastodon.xyz
6 - Enable me on the new instance
7 - update my following with my new address.

my 2cts. (gonna have a lot of cents if we all add up)

Yes that's a bit complicated but imo this is a very important feature to permit existing users balancing between the instances and avoid account loss in case of instance shutdown.

leeeeeny · Apr 4, 2017

Yes that's a bit complicated but imo this is a very important feature to permit existing users balancing between the instances and avoid account loss in case of instance shutdown.

I agree with everyone here. Account migration is necessary. The most important is toots and followers for me. Followings too but they can already be exported. Username would be a good thing too.

Sadzeih · Apr 4, 2017

I agree with everyone here. Account migration is necessary. The most important is toots and followers for me. Followings too but they can already be exported. Username would be a good thing too.

Account migration is a necessity if you want federation to be a first-class citizen. Users need to be insulated from the risks associated with decentralization. If an instance goes away (crashes, is hacked, shut down by an oppressive regime) those users should not be SOL. Lacking that, users will flock to the instance with the highest likelihood of surviving.

Motoma · Apr 4, 2017

Account migration is a necessity if you want federation to be a first-class citizen. Users need to be insulated from the risks associated with decentralization. If an instance goes away (crashes, is hacked, shut down by an oppressive regime) those users should not be SOL. Lacking that, users will flock to the instance with the highest likelihood of surviving.

Well, there's two ways (in my mind) of fixing the problem:

You do it the GitHub way, where the old username is now forwarded to the new username. I'm not sure if OStatus has a concept of 301 redirects, but that's how you would do it. The downside is that if the server goes down before a client has got the redirect they're SoL.
You handle it the Matrix way, which is that you have identity servers that decouple the "real" ID (@username@instance) from the person whose ID it belongs to. The downside is that Matrix implemented this in a very centralised way, which just punts the issue. Not to mention OStatus almost certainly doesn't have support for such a concept.

cyphar · Apr 4, 2017

Well, there's two ways (in my mind) of fixing the problem:

You do it the GitHub way, where the old username is now forwarded to the new username. I'm not sure if OStatus has a concept of 301 redirects, but that's how you would do it. The downside is that if the server goes down before a client has got the redirect they're SoL.
You handle it the Matrix way, which is that you have identity servers that decouple the "real" ID (@username@instance) from the person whose ID it belongs to. The downside is that Matrix implemented this in a very centralised way, which just punts the issue. Not to mention OStatus almost certainly doesn't have support for such a concept.

@TehShrike This issue is about moving between Mastodon instances, not from Twitter to Mastodon.

cyphar · Apr 4, 2017

@TehShrike This issue is about moving between Mastodon instances, not from Twitter to Mastodon.

Ah, I googled-and-skimmed too quickly >_< I'll delete that post.

TehShrike · Apr 4, 2017

Ah, I googled-and-skimmed too quickly >_< I'll delete that post.

My hand-wave-y two cents:

It would be nice if there was a decoupled notion of identity. For example: a keybase.io profile aggregates different accounts for a single person so that they can be easily identified across all networks.

If this were true, you could imagine a system where you follow someone on Mastodon by following an identity and all addresses (that is, instance/profile combinations) that are associated with it.

Of course, it isn't easy to maintain a trusted database of identities that is shareable across all federated nodes. But, it's possible. Perhaps we can discuss the feasibility of such an approach.

👋

cdata · Apr 4, 2017

My hand-wave-y two cents:

It would be nice if there was a decoupled notion of identity. For example: a keybase.io profile aggregates different accounts for a single person so that they can be easily identified across all networks.

If this were true, you could imagine a system where you follow someone on Mastodon by following an identity and all addresses (that is, instance/profile combinations) that are associated with it.

Of course, it isn't easy to maintain a trusted database of identities that is shareable across all federated nodes. But, it's possible. Perhaps we can discuss the feasibility of such an approach.

👋

Blockstack might be a good candidate for providing a decentralised, global identity. With Blockstack you could keep the same ID (and thus your followers) even if the instance you are currently using goes down permanently.

stiell · Apr 4, 2017

Blockstack might be a good candidate for providing a decentralised, global identity. With Blockstack you could keep the same ID (and thus your followers) even if the instance you are currently using goes down permanently.

The problem with having separated identities is that it wouldn't be compatible with other OStatus implementations (Mastodon isn't the only implementation, GNU Social is another and there's quite a few more). Migration should be something that someone following a Mastodon user from GNU Social should be able to support.

cyphar · Apr 4, 2017

The problem with having separated identities is that it wouldn't be compatible with other OStatus implementations (Mastodon isn't the only implementation, GNU Social is another and there's quite a few more). Migration should be something that someone following a Mastodon user from GNU Social should be able to support.

In the fullness of time, I agree that this should be something that is broadly compatible with GNU Social. In the isolation of the moment, I propose that evolution requires experimentation. Compatibility is a matter of adding sufficient layers to make the standardized things cooperate with the experimental thing.

My brain told my hand to stop waving. I'm sure it will stop soon.

cdata · Apr 4, 2017

In the fullness of time, I agree that this should be something that is broadly compatible with GNU Social. In the isolation of the moment, I propose that evolution requires experimentation. Compatibility is a matter of adding sufficient layers to make the standardized things cooperate with the experimental thing.

My brain told my hand to stop waving. I'm sure it will stop soon.

I think you guys are making a mountain out of a molehill here. I think this problem is pretty easy to solve. In order of least to most controversial: we should make it possible to import and export toots, we should make an automated migration tool (log into your old account via OAuth from your new instance), and we should toot out a notice of account change.

The migration toot would be a normal toot with a human-readable message ("My account is moving to (link)"), but would include something like this:

<mastodon:migrate participant="origin">https://new-account</mastodon:migrate>

The new account would toot something like this to confirm it ("Moved from my old account (link)"):

<mastodon:migrate participant="destination">https://old-account</mastodon:migrate>

Then, clients can recognize this attribute and perform the migration when they see the toot by unfollowing and following the new account (possibly with an another XML extension that indicates the user is responding to an account migration). Clients would likely want to hide the special toots from the feed as well. Clients that don't support it have the human-readable version as a fallback. Then we just send patches to GNU Social and other projects to recognize this behavior.

SirCmpwn · Apr 4, 2017

I think you guys are making a mountain out of a molehill here. I think this problem is pretty easy to solve. In order of least to most controversial: we should make it possible to import and export toots, we should make an automated migration tool (log into your old account via OAuth from your new instance), and we should toot out a notice of account change.

The migration toot would be a normal toot with a human-readable message ("My account is moving to (link)"), but would include something like this:

<mastodon:migrate participant="origin">https://new-account</mastodon:migrate>

The new account would toot something like this to confirm it ("Moved from my old account (link)"):

<mastodon:migrate participant="destination">https://old-account</mastodon:migrate>

Then, clients can recognize this attribute and perform the migration when they see the toot by unfollowing and following the new account (possibly with an another XML extension that indicates the user is responding to an account migration). Clients would likely want to hide the special toots from the feed as well. Clients that don't support it have the human-readable version as a fallback. Then we just send patches to GNU Social and other projects to recognize this behavior.

I'll probably start working on a toot export/import tool shortly, in any case.

SirCmpwn · Apr 4, 2017

I'll probably start working on a toot export/import tool shortly, in any case.

@TehShrike There IS this tool for what you are looking for: https://mastodon-bridge.herokuapp.com/ but that's only if they signed up on both.

Maverynthia · Apr 4, 2017

@TehShrike There IS this tool for what you are looking for: https://mastodon-bridge.herokuapp.com/ but that's only if they signed up on both.

The way to connect accounts is XFN - rel information on links. You use rel=me to say "this is also me" and if both urls do that to each other it's confirmed.
rel="friend" will do for following (there are others, but that is simple enough to use).

If you want to convey "I moved there" use rel="canonical me"

The added benefit of this is that you can do actual distributed verification tags - confirming people's twitter/github/medium/whatever accounts too.

kevinmarks · Apr 5, 2017

The way to connect accounts is XFN - rel information on links. You use rel=me to say "this is also me" and if both urls do that to each other it's confirmed.
rel="friend" will do for following (there are others, but that is simple enough to use).

If you want to convey "I moved there" use rel="canonical me"

The added benefit of this is that you can do actual distributed verification tags - confirming people's twitter/github/medium/whatever accounts too.

I'm not sure those fit ideally into this problem, @kevinmarks. They're for duplicated content, not a one-time notification that the content publisher has moved. Would make more sense if we were mirroring toots between multiple accounts but I think that's unrelated to the matter at hand.

SirCmpwn · Apr 5, 2017

I'm not sure those fit ideally into this problem, @kevinmarks. They're for duplicated content, not a one-time notification that the content publisher has moved. Would make more sense if we were mirroring toots between multiple accounts but I think that's unrelated to the matter at hand.

the point about rel's is that they are amenable to setting up bidirectional verification of the move. Then you can use a 301 redirect for moved URLs.

kevinmarks · Apr 5, 2017

the point about rel's is that they are amenable to setting up bidirectional verification of the move. Then you can use a 301 redirect for moved URLs.

I think I see what you mean. The one-time "I've moved" toots could use rel="me" (new account) and rel="canonical me" (old account) to indicate the move? I'm still not sure that's entirely in the spirit of these attributes but I don't see why it wouldn't work.

SirCmpwn · Apr 5, 2017

I think I see what you mean. The one-time "I've moved" toots could use rel="me" (new account) and rel="canonical me" (old account) to indicate the move? I'm still not sure that's entirely in the spirit of these attributes but I don't see why it wouldn't work.

No, they shouldn't be toots, they should be in the bio section

…

On 5 Apr 2017 1:31 am, "Drew DeVault" ***@***.***> wrote: I think I see what you mean. The one-time "I've moved" toots could use rel="me" (new account) and rel="canonical me" (old account) to indicate the move? I'm still not sure that's entirely in the spirit of these attributes but I don't see why it wouldn't work. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#177 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGCwJHPYajcxkpXohajzF5z9LASspUdks5rsuDxgaJpZM4K5Bba> .

kevinmarks · Apr 5, 2017

No, they shouldn't be toots, they should be in the bio section

…

On 5 Apr 2017 1:31 am, "Drew DeVault" ***@***.***> wrote: I think I see what you mean. The one-time "I've moved" toots could use rel="me" (new account) and rel="canonical me" (old account) to indicate the move? I'm still not sure that's entirely in the spirit of these attributes but I don't see why it wouldn't work. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#177 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGCwJHPYajcxkpXohajzF5z9LASspUdks5rsuDxgaJpZM4K5Bba> .

Aha, good idea. Makes sense.

SirCmpwn · Apr 5, 2017

Aha, good idea. Makes sense.

Doesn't importing a list of exported toots represent a pretty major potential vulnerability with regard to spamming? For instance, somebody could make a large synthetic export file with faked timestamps and flood the local timeline on a new account, or create new one-off follower accounts on several major nodes first and flood several federated timelines, at a scale larger than manual posting, if my understanding of how federation works is correct.

Don't get me wrong -- I think transferring full toot history is important for account migration, and I think we should have an export function for toots available to all users -- but I think full account transfer should probably be negotiated between federated servers (even if it's a separate protocol or an extension), and that toot history import should either: 1) be heavily limited in scale (say, no more than 100); 2) be omitted from inclusion in the local and federated timelines; or 3) have retroactively-applied rate limits at least as strict as the ones applied to people posting through the existing front end, so that any file with toot timestamps too close together gets rejected.

enkiv2 · Apr 5, 2017

Doesn't importing a list of exported toots represent a pretty major potential vulnerability with regard to spamming? For instance, somebody could make a large synthetic export file with faked timestamps and flood the local timeline on a new account, or create new one-off follower accounts on several major nodes first and flood several federated timelines, at a scale larger than manual posting, if my understanding of how federation works is correct.

Don't get me wrong -- I think transferring full toot history is important for account migration, and I think we should have an export function for toots available to all users -- but I think full account transfer should probably be negotiated between federated servers (even if it's a separate protocol or an extension), and that toot history import should either: 1) be heavily limited in scale (say, no more than 100); 2) be omitted from inclusion in the local and federated timelines; or 3) have retroactively-applied rate limits at least as strict as the ones applied to people posting through the existing front end, so that any file with toot timestamps too close together gets rejected.

A user opinion about the priority when migrating: make followers auto-follow the new account.

I believe that mastodon, as other microblogging platforms, is mainly focused on real-time.

I can deal with going back to a clean toots history but starting again from 0 followers would totally discourage me to migrate to another instance. And I know that a "I moved here re-follow me!" notification won't be enough to bring back half of the people that were in my network.

What is really precious in an account is its followers base.

thomaslule · Apr 5, 2017

A user opinion about the priority when migrating: make followers auto-follow the new account.

I believe that mastodon, as other microblogging platforms, is mainly focused on real-time.

I can deal with going back to a clean toots history but starting again from 0 followers would totally discourage me to migrate to another instance. And I know that a "I moved here re-follow me!" notification won't be enough to bring back half of the people that were in my network.

What is really precious in an account is its followers base.

What if use namecoin to store identity. It allows to attach to name some information, which could be url of actual user's account

autogestion · Jun 16, 2018

What if use namecoin to store identity. It allows to attach to name some information, which could be url of actual user's account

I feel like this issue should be much higher priority than it currently is, in the interest of offering a truly portable experience where you own your data.

At this point, rather than recycling old discussions and approaches, can we try to hammer out an actionable plan to implement portable identities?

Current blockers to account migration

1) Identification is bound to domain

A. Mastodon, throughout various places in its code and database, uses username/domain pairs to identify users.
B. The Mastodon API fetches users with GET /user/uid, where uid is a sequential integer like 55816 and only guaranteed to be unique on that domain. But this id is used only for API, and the local server converts the received profile into username/domain for its database.
C1. Therefore, there is no way to identify users in a way that isn't bound to at least the domain.
C2. Therefore, some other form of ID should be used, and Mastodon's codebase/database should be updated to use it internally, leaving username/domain only for lookup.
C3. Therefore, seamless translation should occur between all these types of identification, with username/domain being converted to UUID internally and mentions/accounts being mapped back to UUID.

As explained in #177 (comment), zot already has a workable scheme to address this, and an identity consists of a 256-bit GUID and a private key. For live migration, this is enough. For data exports, this should also include an address book, i.e. in Mastodon's terms, the follower/following lists.

The zot identity is managed by a primary hub, and if that hub goes offline, any hub with the private key can declare itself to be the new primary hub (initiated by the user). Users can import their GUID and private key to any server by either uploading their backup, or by live mirroring from the current primary hub. Your followers' primary hubs are notified of your new hub.

2) Old content

Previously-posted statuses are floating around with mentions that explicitly link to a certain domain/username. These will probably break. However, that's not much different than what currently happens if a server goes offline: any user on that server will become inaccessible, and all mention-links to that user will also become unresolvable. Therefore, this will be a breaking change, but ultimately one that needs to be made for the resilience and robustness of the future network.

Plan

Start generating GUIDs for Mastodon users in preparation for v.3.0.0, the sooner the better. We might as well use the "256-bits encoded as base64url" schema that zot uses, if there aren't any other options that work better for some reason or another.
Start refactoring the codebase to accept either username/domain pairs or the newly-generated GUIDs. If a GUID exists, it should be authoritative over username/domain. This will probably affect (at the very least) account generation, account lookup, mentions handling, and URI dereferencing. GUID should be served as ActivityPub id field for Actor objects. The URL should be served via url instead.
Write a database migration task to start storing accounts internally by GUID.
Start using GUID for follower/following lists. Ensure that the data export tool includes the GUID and user's private/public keys, as well as their posts, media, and follower/following lists.
Write a handler to announce updated username/domain for a given GUID to your follower/following lists, using a secure exchange between servers and probably the Update Activity. Write a handler to receive and parse this Update.
Write an import tool for the offline data export archive.
Write a live import tool to exchange data directly with your primary server.
When v3.0.0 is released, expose steps 6-7 to users. Since by this point the majority of users should already have a GUID due to step 1, and the logic for handling GUIDs should have percolated through the network thanks to steps 2-5.

Any insights from coders on this? Does this seem workable?

trwnh · Jun 21, 2018

I feel like this issue should be much higher priority than it currently is, in the interest of offering a truly portable experience where you own your data.

At this point, rather than recycling old discussions and approaches, can we try to hammer out an actionable plan to implement portable identities?

Current blockers to account migration

1) Identification is bound to domain

A. Mastodon, throughout various places in its code and database, uses username/domain pairs to identify users.
B. The Mastodon API fetches users with GET /user/uid, where uid is a sequential integer like 55816 and only guaranteed to be unique on that domain. But this id is used only for API, and the local server converts the received profile into username/domain for its database.
C1. Therefore, there is no way to identify users in a way that isn't bound to at least the domain.
C2. Therefore, some other form of ID should be used, and Mastodon's codebase/database should be updated to use it internally, leaving username/domain only for lookup.
C3. Therefore, seamless translation should occur between all these types of identification, with username/domain being converted to UUID internally and mentions/accounts being mapped back to UUID.

As explained in #177 (comment), zot already has a workable scheme to address this, and an identity consists of a 256-bit GUID and a private key. For live migration, this is enough. For data exports, this should also include an address book, i.e. in Mastodon's terms, the follower/following lists.

The zot identity is managed by a primary hub, and if that hub goes offline, any hub with the private key can declare itself to be the new primary hub (initiated by the user). Users can import their GUID and private key to any server by either uploading their backup, or by live mirroring from the current primary hub. Your followers' primary hubs are notified of your new hub.

2) Old content

Previously-posted statuses are floating around with mentions that explicitly link to a certain domain/username. These will probably break. However, that's not much different than what currently happens if a server goes offline: any user on that server will become inaccessible, and all mention-links to that user will also become unresolvable. Therefore, this will be a breaking change, but ultimately one that needs to be made for the resilience and robustness of the future network.

Plan

Start generating GUIDs for Mastodon users in preparation for v.3.0.0, the sooner the better. We might as well use the "256-bits encoded as base64url" schema that zot uses, if there aren't any other options that work better for some reason or another.
Start refactoring the codebase to accept either username/domain pairs or the newly-generated GUIDs. If a GUID exists, it should be authoritative over username/domain. This will probably affect (at the very least) account generation, account lookup, mentions handling, and URI dereferencing. GUID should be served as ActivityPub id field for Actor objects. The URL should be served via url instead.
Write a database migration task to start storing accounts internally by GUID.
Start using GUID for follower/following lists. Ensure that the data export tool includes the GUID and user's private/public keys, as well as their posts, media, and follower/following lists.
Write a handler to announce updated username/domain for a given GUID to your follower/following lists, using a secure exchange between servers and probably the Update Activity. Write a handler to receive and parse this Update.
Write an import tool for the offline data export archive.
Write a live import tool to exchange data directly with your primary server.
When v3.0.0 is released, expose steps 6-7 to users. Since by this point the majority of users should already have a GUID due to step 1, and the logic for handling GUIDs should have percolated through the network thanks to steps 2-5.

Any insights from coders on this? Does this seem workable?

Using a public key instead of some random hash would allow you to prove ownership of the identity, in the presence of bad third-party servers, or bad/gone previous server. Anyone can verify that you (possessor of the private key) did indeed initiate the change of domain.

remram44 · Jun 21, 2018

Using a public key instead of some random hash would allow you to prove ownership of the identity, in the presence of bad third-party servers, or bad/gone previous server. Anyone can verify that you (possessor of the private key) did indeed initiate the change of domain.

I'm confused. Does it mean username/domain will be changed as long as one moved?
The migration largely rely on notification from new primary hub, how can I find the moved user without the knowledge of new primary hub? Such as an old address printed on business card, a dump with outdated hub information, or a GUID string only.
Hand out my private key make me a bit unsatisfied, any details?

pingu8007 · Jun 22, 2018

I'm confused. Does it mean username/domain will be changed as long as one moved?
The migration largely rely on notification from new primary hub, how can I find the moved user without the knowledge of new primary hub? Such as an old address printed on business card, a dump with outdated hub information, or a GUID string only.
Hand out my private key make me a bit unsatisfied, any details?

I think in @trwnh's scheme you need the old domain to still exist and let you know where the new location is. Truly resilient discovery could be done through e.g. a DHT, mapping GUID to domain.

A private key is never handed out, it's private.

remram44 · Jun 22, 2018

I think in @trwnh's scheme you need the old domain to still exist and let you know where the new location is. Truly resilient discovery could be done through e.g. a DHT, mapping GUID to domain.

A private key is never handed out, it's private.

@pingu8007 The primary instance would send a notofication to your followers' instances -- see point 5 pf the plan.

@remram44 Are there any concerns with using the public key directly as a GUID, rather than as a separate field? I suppose functionally, a public key is ostensibly unique, but my understanding is that it would be inefficient to refer to every user by public key, even in unencrypted situations. Also, using a separate GUID allows for the encryption schema to change later.

Of course, I'm also unsure of the exact schema Mastodon uses for encryption right now. The existence of a private key for a user implies that public keys are generated for each user, but I have no idea if this is indeed the case, or if keys are generated per-server. Per-user keys would go a long way toward potential end-to-end support, but that's a separate issue. It may end up being necessary to establish provenance of your identity, though.

trwnh · Jun 23, 2018

@pingu8007 The primary instance would send a notofication to your followers' instances -- see point 5 pf the plan.

@remram44 Are there any concerns with using the public key directly as a GUID, rather than as a separate field? I suppose functionally, a public key is ostensibly unique, but my understanding is that it would be inefficient to refer to every user by public key, even in unencrypted situations. Also, using a separate GUID allows for the encryption schema to change later.

Of course, I'm also unsure of the exact schema Mastodon uses for encryption right now. The existence of a private key for a user implies that public keys are generated for each user, but I have no idea if this is indeed the case, or if keys are generated per-server. Per-user keys would go a long way toward potential end-to-end support, but that's a separate issue. It may end up being necessary to establish provenance of your identity, though.

You don't have to use the full public key. A hash or fingerprint works.

My concern with a freely chosen GUID is that collisions are possible. A bad actor could use the same GUID as you, and mess up your account for the people that hear about him before they hear about you.

Reading about your scheme again, I'm wondering if a GUID is necessary at all. Why not use the old username@domain in place of the GUID? And just announce that the old handle is now at a new location?

remram44 · Jun 23, 2018

You don't have to use the full public key. A hash or fingerprint works.

My concern with a freely chosen GUID is that collisions are possible. A bad actor could use the same GUID as you, and mess up your account for the people that hear about him before they hear about you.

Reading about your scheme again, I'm wondering if a GUID is necessary at all. Why not use the old username@domain in place of the GUID? And just announce that the old handle is now at a new location?

You don't have to use the full public key. A hash or fingerprint works.

For the record ed25519 public keys are only 32 bytes in size only twice as big as GUIDs (16 bytes).

Reading about your scheme again, I'm wondering if a GUID is necessary at all. Why not use the old username@domain in place of the GUID? And just announce that the old handle is now at a new location?

I think one of sub-problems to solve here is to handle old server down forever scenarios. Announcing new handle at old location is already possible with movedTo (e.g. this account).

wiktor-k · Jun 23, 2018

You don't have to use the full public key. A hash or fingerprint works.

For the record ed25519 public keys are only 32 bytes in size only twice as big as GUIDs (16 bytes).

Reading about your scheme again, I'm wondering if a GUID is necessary at all. Why not use the old username@domain in place of the GUID? And just announce that the old handle is now at a new location?

I think one of sub-problems to solve here is to handle old server down forever scenarios. Announcing new handle at old location is already possible with movedTo (e.g. this account).

I'm fine with identity being tied to server. I just want a way to formalize "My identity has transitioned from TehShrike@coolserver.com to TehShrike@toteselite.com" in a way that all my followers and follow lists move from coolserver.com to toteselite.com.

If coolserver.com maintained a "forwarding address" to the new location, that would be cool too, but you can't rely on that staying up forever.

Changing the nature of identity on mastodon sounds much more difficult.

TehShrike · Jun 23, 2018

I'm fine with identity being tied to server. I just want a way to formalize "My identity has transitioned from TehShrike@coolserver.com to TehShrike@toteselite.com" in a way that all my followers and follow lists move from coolserver.com to toteselite.com.

If coolserver.com maintained a "forwarding address" to the new location, that would be cool too, but you can't rely on that staying up forever.

Changing the nature of identity on mastodon sounds much more difficult.

It might be impossible to underestimate the long term value for the whole Fediverse (and beyond) of contributing to true, scaleable, decentralized identity handling. Server dependent solutions should therefore be avoided.

…

On Sat, Jun 23, 2018, 19:49 Josh Duff, ***@***.***> wrote: I'm fine with identity being tied to server. I just want a way to formalize "My identity has transitioned from ***@***.*** to ***@***.***" in a way that all my followers and follow lists move from coolserver.com to toteselite.com. If coolserver.com maintained a "forwarding address" to the new location, that would be cool too, but you can't rely on that staying up forever. Changing the nature of identity on mastodon sounds much more difficult. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#177 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ASzGVDFOIIFqv5hARL88X6YFk78-QxTxks5t_n-2gaJpZM4K5Bba> .

LeeteqXV · Jun 23, 2018

It might be impossible to underestimate the long term value for the whole Fediverse (and beyond) of contributing to true, scaleable, decentralized identity handling. Server dependent solutions should therefore be avoided.

…

On Sat, Jun 23, 2018, 19:49 Josh Duff, ***@***.***> wrote: I'm fine with identity being tied to server. I just want a way to formalize "My identity has transitioned from ***@***.*** to ***@***.***" in a way that all my followers and follow lists move from coolserver.com to toteselite.com. If coolserver.com maintained a "forwarding address" to the new location, that would be cool too, but you can't rely on that staying up forever. Changing the nature of identity on mastodon sounds much more difficult. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#177 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ASzGVDFOIIFqv5hARL88X6YFk78-QxTxks5t_n-2gaJpZM4K5Bba> .

Why not integrate this into the registration process. When someone registers in a new instance, this instance will check if the mail address used to create the new account is already used on other public instances.

If the same mail address is detected on another account on another instance, the new account on the instance will propose to set up the redirection settings (to change from one instance to another, with all the parameters that @hach-que has purposed). The user can refuse and decide to do nothing, and start from an empty account.

This will make the process simpler for a broader audience who don't want to mess with theses redirections settings.

wedza · Jun 24, 2018

Why not integrate this into the registration process. When someone registers in a new instance, this instance will check if the mail address used to create the new account is already used on other public instances.

If the same mail address is detected on another account on another instance, the new account on the instance will propose to set up the redirection settings (to change from one instance to another, with all the parameters that @hach-que has purposed). The user can refuse and decide to do nothing, and start from an empty account.

This will make the process simpler for a broader audience who don't want to mess with theses redirections settings.

check if the mail address used to create the new account is already used on other public instances.

How do you do that? How do you prevent other instances from lying? How do you prove to the old instance that this should happen? How do you deal with the old instance being broken, or malicious?

remram44 · Jun 24, 2018

check if the mail address used to create the new account is already used on other public instances.

How do you do that? How do you prevent other instances from lying? How do you prove to the old instance that this should happen? How do you deal with the old instance being broken, or malicious?

One way would be the two instances sending each a validation email. The only case in which there is a problem is if the instance you want to move from is malicious, but you can't really prevent anything in this case.

Shywim · Jun 24, 2018

One way would be the two instances sending each a validation email. The only case in which there is a problem is if the instance you want to move from is malicious, but you can't really prevent anything in this case.

remram44: Dealing with the old instance being broken is why I said above (and split out as issue 7715) is that what's really needed is live mirroring between accounts, so you setup a spare before one side fails.
To do migration, I guess you could use signed toots, with some form of message that advertises the change.

penguin42 · Jun 24, 2018

remram44: Dealing with the old instance being broken is why I said above (and split out as issue 7715) is that what's really needed is live mirroring between accounts, so you setup a spare before one side fails.
To do migration, I guess you could use signed toots, with some form of message that advertises the change.

@remram44

My concern with a freely chosen GUID is that collisions are possible. A bad actor could use the same GUID

Random collision in 256-bit will take longer than the life of the universe. Intentional collision would be prevented by cryptography. At least, that's how zot does it -- see: what an identity consists of, #177 (comment) (uid and private key, plus address book if offline).

Why not use the old username@domain in place of the GUID? And just announce that the old handle is now at a new location?

That might work only for the purposes of migration, but for basically nothing else. It also requires validation of some sort with the server on that domain, which might be offline. It also will not be preserved when migrating to a new server/domain -- the point of the GUID is to identify the two accounts as the same without revealing a key. Verification happens in a different step.

@TehShrike

I'm fine with identity being tied to server. I just want [...] all my followers and follow lists [to] move [...] If [the old server] maintained a "forwarding address" to the new location, that would be cool too, but you can't rely on that staying up forever.

That's exactly what would ideally be avoided. If all you care about is having your address book move, then much of that is already possible. But IMO, a proper "migration" would unambiguously move your entire account, not just the details of it.

Changing the nature of identity on mastodon sounds much more difficult.

It will definitely be difficult, which is why the plan I described proposes a gradual change before it becomes official (in a v3.0.0 change). Right now, the fact that identity is tied to user/domain is a major blocking issue to any sort of migration. If that doesn't change, then Mastodon will never be able to gracefully handle anything more than the "account moved" notice we already have. Any existing instance will see your moved account as an entirely different account, which is extremely inconvenient.

@wedza

Why not integrate this into the registration process. When someone registers in a new instance, this instance will check if the mail address used to create the new account is already used on other public instances.

That doesn't really solve the issue. Signing up with the same email can already be done for multiple instances, to create separate accounts. Also, it's a really bad idea to expose the email address used by an account. But you do have a point that it probably should be exposed during registration -- in fact, Friendica and Hubzilla already do this, offering you options to import an account or to set up a new one. This has nothing to do with the email used, though; it just uses the ID/keys/posts/contacts.

@penguin42

what's really needed is live mirroring between accounts, so you setup a spare before one side fails.

That would easily be possible once the secure exchange / import logic is implemented. The only difference would be setting the new server as a secondary instance rather than as a new primary instance.

trwnh · Jun 24, 2018

@remram44

My concern with a freely chosen GUID is that collisions are possible. A bad actor could use the same GUID

Random collision in 256-bit will take longer than the life of the universe. Intentional collision would be prevented by cryptography. At least, that's how zot does it -- see: what an identity consists of, #177 (comment) (uid and private key, plus address book if offline).

Why not use the old username@domain in place of the GUID? And just announce that the old handle is now at a new location?

That might work only for the purposes of migration, but for basically nothing else. It also requires validation of some sort with the server on that domain, which might be offline. It also will not be preserved when migrating to a new server/domain -- the point of the GUID is to identify the two accounts as the same without revealing a key. Verification happens in a different step.

@TehShrike

I'm fine with identity being tied to server. I just want [...] all my followers and follow lists [to] move [...] If [the old server] maintained a "forwarding address" to the new location, that would be cool too, but you can't rely on that staying up forever.

That's exactly what would ideally be avoided. If all you care about is having your address book move, then much of that is already possible. But IMO, a proper "migration" would unambiguously move your entire account, not just the details of it.

Changing the nature of identity on mastodon sounds much more difficult.

It will definitely be difficult, which is why the plan I described proposes a gradual change before it becomes official (in a v3.0.0 change). Right now, the fact that identity is tied to user/domain is a major blocking issue to any sort of migration. If that doesn't change, then Mastodon will never be able to gracefully handle anything more than the "account moved" notice we already have. Any existing instance will see your moved account as an entirely different account, which is extremely inconvenient.

@wedza

Why not integrate this into the registration process. When someone registers in a new instance, this instance will check if the mail address used to create the new account is already used on other public instances.

That doesn't really solve the issue. Signing up with the same email can already be done for multiple instances, to create separate accounts. Also, it's a really bad idea to expose the email address used by an account. But you do have a point that it probably should be exposed during registration -- in fact, Friendica and Hubzilla already do this, offering you options to import an account or to set up a new one. This has nothing to do with the email used, though; it just uses the ID/keys/posts/contacts.

@penguin42

what's really needed is live mirroring between accounts, so you setup a spare before one side fails.

That would easily be possible once the secure exchange / import logic is implemented. The only difference would be setting the new server as a secondary instance rather than as a new primary instance.

Random collision in 256-bit will take longer than the life of the universe. Intentional collision would be prevented by cryptography.

Intentional collisions are exactly what I'm concerned about. "Prevented by cryptography" only happens if you use a public key hash instead of a GUID. Note that GUID means something specific: those are 128-bit values with a specific (versioned) schema.

remram44 · Jun 24, 2018

Random collision in 256-bit will take longer than the life of the universe. Intentional collision would be prevented by cryptography.

Intentional collisions are exactly what I'm concerned about. "Prevented by cryptography" only happens if you use a public key hash instead of a GUID. Note that GUID means something specific: those are 128-bit values with a specific (versioned) schema.

@remram44 Ah, my bad, I wasn't clear on the differences between guid/uuid/etc. I've been using the terms in the most generic sense possible (a globally unique user ID). Regardless, the (256-bit) UID alone will not be enough to constitute an identity; it is only part of it. A private key and address book make up the other parts (to prove ownership + rebuild connections).

trwnh · Jun 25, 2018

@remram44 Ah, my bad, I wasn't clear on the differences between guid/uuid/etc. I've been using the terms in the most generic sense possible (a globally unique user ID). Regardless, the (256-bit) UID alone will not be enough to constitute an identity; it is only part of it. A private key and address book make up the other parts (to prove ownership + rebuild connections).

THIS: @trwnh wrote; "What this could mean for Mastodon: *accounts could be decoupled from their profiles, and profiles could be migratable across instances. Accounts could also manage multiple profiles. An analogy would be Tweetdeck allowing management of multiple Twitter profiles from the same account.* I think there's some real potential here to take *Hubzilla's work* on this topic and make it more simple and accessible, less confusing. I'm not sure how it would best be implemented, but I hope explaining the metaphors here can lead to something useful coming out of it."

…

On Mon, Jun 25, 2018, 04:50 trwnh, ***@***.***> wrote: @remram44 <https://github.com/remram44> Ah, my bad, I wasn't clear on the differences between guid/uuid/etc. I've been using the terms in the most generic sense possible (a globally unique user ID). Regardless, the (256-bit) UID alone will *not* be enough to constitute an identity; it is only part of it. A private key and address book make up the other parts (to prove ownership + rebuild connections). — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#177 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ASzGVPHX202nq-lBaTYtrL3OexJiSeVKks5uAFAIgaJpZM4K5Bba> .

LeeteqXV · Jun 25, 2018

THIS: @trwnh wrote; "What this could mean for Mastodon: *accounts could be decoupled from their profiles, and profiles could be migratable across instances. Accounts could also manage multiple profiles. An analogy would be Tweetdeck allowing management of multiple Twitter profiles from the same account.* I think there's some real potential here to take *Hubzilla's work* on this topic and make it more simple and accessible, less confusing. I'm not sure how it would best be implemented, but I hope explaining the metaphors here can lead to something useful coming out of it."

…

On Mon, Jun 25, 2018, 04:50 trwnh, ***@***.***> wrote: @remram44 <https://github.com/remram44> Ah, my bad, I wasn't clear on the differences between guid/uuid/etc. I've been using the terms in the most generic sense possible (a globally unique user ID). Regardless, the (256-bit) UID alone will *not* be enough to constitute an identity; it is only part of it. A private key and address book make up the other parts (to prove ownership + rebuild connections). — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#177 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ASzGVPHX202nq-lBaTYtrL3OexJiSeVKks5uAFAIgaJpZM4K5Bba> .

Addendum to the discussion with @remram44 about using public key hashes as UIDs: I don't think that would work in the current schema where every actor's public key is actually just the server's public key. Mastodon doesn't currently generate keypairs for every single actor. My proposal to start generating UIDs is to lay the groundwork for changes, not to be used immediately. That would allow cryptographic changes later on a server-to-server basis -- live migration is the more pressing thing to be tackled IMO, since offline migration requires per-actor keypairs.

trwnh · Jul 20, 2018

Addendum to the discussion with @remram44 about using public key hashes as UIDs: I don't think that would work in the current schema where every actor's public key is actually just the server's public key. Mastodon doesn't currently generate keypairs for every single actor. My proposal to start generating UIDs is to lay the groundwork for changes, not to be used immediately. That would allow cryptographic changes later on a server-to-server basis -- live migration is the more pressing thing to be tackled IMO, since offline migration requires per-actor keypairs.

Mastodon doesn't currently generate keypairs for every single actor

what? this is incorrect. How are you coming to this conclusion?

nightpool · Jul 22, 2018

Mastodon doesn't currently generate keypairs for every single actor

what? this is incorrect. How are you coming to this conclusion?

@nightpool examining the JSON output of two users on the same instance. I was mainly referring to server-server interactions and not client-server, because I have no idea how to examine what's happening privately. I may have overlooked something because I'm not perfectly familiar with how Mastodon generates all this stuff.

https://mastodon.social/@trwnh.json

"publicKey":{
"id":"https://mastodon.social/users/trwnh#main-key",
"owner":"https://mastodon.social/users/trwnh",
"publicKeyPem":"-----BEGIN PUBLIC KEY-----\nMIIBIjANBgk...-----END PUBLIC KEY-----\n"
}

https://mastodon.social/@gargron.json

"publicKey":{
"id":"https://mastodon.social/users/Gargron#main-key",
"owner":"https://mastodon.social/users/Gargron",
"publicKeyPem":"-----BEGIN PUBLIC KEY-----\nMIIBIjANBgk...-----END PUBLIC KEY-----\n"
}

trwnh · Jul 22, 2018

@nightpool examining the JSON output of two users on the same instance. I was mainly referring to server-server interactions and not client-server, because I have no idea how to examine what's happening privately. I may have overlooked something because I'm not perfectly familiar with how Mastodon generates all this stuff.

https://mastodon.social/@trwnh.json

"publicKey":{
"id":"https://mastodon.social/users/trwnh#main-key",
"owner":"https://mastodon.social/users/trwnh",
"publicKeyPem":"-----BEGIN PUBLIC KEY-----\nMIIBIjANBgk...-----END PUBLIC KEY-----\n"
}

https://mastodon.social/@gargron.json

"publicKey":{
"id":"https://mastodon.social/users/Gargron#main-key",
"owner":"https://mastodon.social/users/Gargron",
"publicKeyPem":"-----BEGIN PUBLIC KEY-----\nMIIBIjANBgk...-----END PUBLIC KEY-----\n"
}

downloaded both and the public keys are different. some of the characters at the beginning and end are the same. that's standard

sireebob · Jul 22, 2018

downloaded both and the public keys are different. some of the characters at the beginning and end are the same. that's standard

@trwnh you're just seeing the framing of the internal binary encoding. the keys are completely different, they just share a similar structure (as do all pem-encoded asn.1 keys)

nightpool · Jul 22, 2018

@trwnh you're just seeing the framing of the internal binary encoding. the keys are completely different, they just share a similar structure (as do all pem-encoded asn.1 keys)

@nightpool Ahhhh, okay yeah, I didn't check far enough, I just skimmed over the first and last few characters to see if they were the same. Whoops!

So, if keypairs are already generated for every actor/profile within Mastodon, that should be one less step to do. Assuming posts and private key are already included in the exported data (are they?) the last thing to do would be to pick/generate/assign/etc some sort of UID for each user, and then it should be "only" a matter of refactoring some logic and building the export/import tools.

I'm still not too sure of the advantages/disadvantages of picking a specific UID schema, per above -- the way I see it, defining a separate UID that isn't simply a hash of publicKey should allow for situations where the old key is repudiated or if the encryption hypothetically changed (e.g. in the future if/when more bits are needed for security) -- and the UID would still need to be connected to a keypair, so no changes there.

trwnh · Jul 22, 2018

@nightpool Ahhhh, okay yeah, I didn't check far enough, I just skimmed over the first and last few characters to see if they were the same. Whoops!

So, if keypairs are already generated for every actor/profile within Mastodon, that should be one less step to do. Assuming posts and private key are already included in the exported data (are they?) the last thing to do would be to pick/generate/assign/etc some sort of UID for each user, and then it should be "only" a matter of refactoring some logic and building the export/import tools.

I'm still not too sure of the advantages/disadvantages of picking a specific UID schema, per above -- the way I see it, defining a separate UID that isn't simply a hash of publicKey should allow for situations where the old key is repudiated or if the encryption hypothetically changed (e.g. in the future if/when more bits are needed for security) -- and the UID would still need to be connected to a keypair, so no changes there.

(I've skimmed previous posts, but there are a lot of them, apologies for any repetition)

fwiw, this is a big part of what we created webfinger for.

imho, the crypto approach is really complicated and anti-user. Do the same thing as HTTP, use [the equivalent of] a 301 redirect. If it's not in the standard, create a new one. ;-)

To be clear:

I choose to move from me@mastodon.social to ohai@mydomain.elsewhere
I tell the mastodon instance at mastodon.social to stop allowing posts there and to provide (in my mastondon.social profile) a redirect to ohai@mydomain.elsewhere.
Any other instances where people follow me at me@mastodon.social should update their subscription to point to ohai@mydomain.elsewhere
If I choose to migrate follower / followed lists, or even toots, that can be a separate API but (to the point of many here) migration is less important than portability, at least to start. Portability is a fundamental protocol requirement, portability is something that can be added later without core protocol support.

Note in the above, no new crypto is needed, all the security boundaries continue to work. There is an obvious hijacking exploit possible in the event that someone gains control of an account, so it would probably make sense to have a SHOULD clause for following user redirects that instructs federated servers to check back with the original server for some period of time (e.g., 30 days) to allow the original user to regain control and cancel the redirect. Again, this is an enhancement that could (should) be added later.

blaine · Aug 14, 2018

(I've skimmed previous posts, but there are a lot of them, apologies for any repetition)

fwiw, this is a big part of what we created webfinger for.

imho, the crypto approach is really complicated and anti-user. Do the same thing as HTTP, use [the equivalent of] a 301 redirect. If it's not in the standard, create a new one. ;-)

To be clear:

I choose to move from me@mastodon.social to ohai@mydomain.elsewhere
I tell the mastodon instance at mastodon.social to stop allowing posts there and to provide (in my mastondon.social profile) a redirect to ohai@mydomain.elsewhere.
Any other instances where people follow me at me@mastodon.social should update their subscription to point to ohai@mydomain.elsewhere
If I choose to migrate follower / followed lists, or even toots, that can be a separate API but (to the point of many here) migration is less important than portability, at least to start. Portability is a fundamental protocol requirement, portability is something that can be added later without core protocol support.

Note in the above, no new crypto is needed, all the security boundaries continue to work. There is an obvious hijacking exploit possible in the event that someone gains control of an account, so it would probably make sense to have a SHOULD clause for following user redirects that instructs federated servers to check back with the original server for some period of time (e.g., 30 days) to allow the original user to regain control and cancel the redirect. Again, this is an enhancement that could (should) be added later.

This makes sense, really. I'd just like to point out (you are probably aware of it) that the drawback of this method is that it doesn't work if an instance permanently goes down. Still better than nothing.

To have a more complete feature (and I'm not suggesting this would be required for a first step), I believe cryptography is required, so that a user is able to prove that he owns the account on the defunct instance. Other instances would know the account public key, only the user knows his private key. And as long as the instance is up, he should be able to update the key pair (in case it gets lost or compromised), which means the public key can't be the account unique ID.

neuoy · Aug 14, 2018

This makes sense, really. I'd just like to point out (you are probably aware of it) that the drawback of this method is that it doesn't work if an instance permanently goes down. Still better than nothing.

To have a more complete feature (and I'm not suggesting this would be required for a first step), I believe cryptography is required, so that a user is able to prove that he owns the account on the defunct instance. Other instances would know the account public key, only the user knows his private key. And as long as the instance is up, he should be able to update the key pair (in case it gets lost or compromised), which means the public key can't be the account unique ID.

AFAIK, key generation is done on the server already, so there aren't any specific client-side issues to solve -- migration would involve copying a key from one server to another. If the instance is live, the user can authorize with another server. If the instance is offline, the user can upload a backup.

The point of changing the keys is a good one, and also why I think there should be a separate 256-bit user ID generated (the sooner the better). By the time Mastodon 3.0 hits, the majority of the ecosystem should have a user ID, and code should have been refactored to refer to this ID instead of to a user/domain pair. #177 (comment)

trwnh · Aug 15, 2018

AFAIK, key generation is done on the server already, so there aren't any specific client-side issues to solve -- migration would involve copying a key from one server to another. If the instance is live, the user can authorize with another server. If the instance is offline, the user can upload a backup.

The point of changing the keys is a good one, and also why I think there should be a separate 256-bit user ID generated (the sooner the better). By the time Mastodon 3.0 hits, the majority of the ecosystem should have a user ID, and code should have been refactored to refer to this ID instead of to a user/domain pair. #177 (comment)

This naive faith in users' ability to do private key management is touching.

…

On Wed, 15 Aug 2018, 08:41 trwnh, ***@***.***> wrote: AFAIK, key generation is done on the server already, so there aren't any specific client-side issues to solve -- migration would involve copying a key from one server to another. If the instance is live, the user can authorize with another server. If the instance is offline, the user can upload a backup. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#177 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGCwDG5mDsb9TUoPXVBCK0gdpiCjjcXks5uQ9CmgaJpZM4K5Bba> .

kevinmarks · Aug 15, 2018

This naive faith in users' ability to do private key management is touching.

…

On Wed, 15 Aug 2018, 08:41 trwnh, ***@***.***> wrote: AFAIK, key generation is done on the server already, so there aren't any specific client-side issues to solve -- migration would involve copying a key from one server to another. If the instance is live, the user can authorize with another server. If the instance is offline, the user can upload a backup. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#177 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGCwDG5mDsb9TUoPXVBCK0gdpiCjjcXks5uQ9CmgaJpZM4K5Bba> .

@kevinmarks Why would the users have to do any key management? Like I just said, all key management is done transparently on the server already, and the server is already managing everything for the user. User-facing crypto would be an entirely separate issue.

trwnh · Aug 15, 2018

@kevinmarks Why would the users have to do any key management? Like I just said, all key management is done transparently on the server already, and the server is already managing everything for the user. User-facing crypto would be an entirely separate issue.

@kevinmarks @trwnh To make my thoughts clearer on that point, I also believe users should not have to handle their private key themselves, but having the possibility to do it would protect them against their instance unexpectedly shutting down. The private key would be proof of account ownership, they can delegate its handling to whoever they want (in exchange with password authentication for example). This would allow for additional assistance layers, such as instructing another instance to keep a backup for them.

I don't know any detail of the current protocol, this was just my two cents to answer @blaine about crypto not needed (in my opinion, it is needed, but only to solve the scenario where an instance goes down, which could be a second step in account migration functionality).

neuoy · Aug 15, 2018

@kevinmarks @trwnh To make my thoughts clearer on that point, I also believe users should not have to handle their private key themselves, but having the possibility to do it would protect them against their instance unexpectedly shutting down. The private key would be proof of account ownership, they can delegate its handling to whoever they want (in exchange with password authentication for example). This would allow for additional assistance layers, such as instructing another instance to keep a backup for them.

I don't know any detail of the current protocol, this was just my two cents to answer @blaine about crypto not needed (in my opinion, it is needed, but only to solve the scenario where an instance goes down, which could be a second step in account migration functionality).

Hi all,
I think especially in a decentralized network like Mastodon it's necessary to have the option to migrate to a new instance without loosing the own toots, favs and followers. It just needs to work, no matter if the old instance is still running or not.

If it's not possible, it automatically pushes more users to a few instances, which they believe to be stable and running for a long time.

So as there are many, many people interested in Mastodon and Twitter alternatives for now and they're just signing up without understanding the concept of instances, let's solve this issue asap.

nicostuhlfauth · Aug 17, 2018

Hi all,
I think especially in a decentralized network like Mastodon it's necessary to have the option to migrate to a new instance without loosing the own toots, favs and followers. It just needs to work, no matter if the old instance is still running or not.

If it's not possible, it automatically pushes more users to a few instances, which they believe to be stable and running for a long time.

So as there are many, many people interested in Mastodon and Twitter alternatives for now and they're just signing up without understanding the concept of instances, let's solve this issue asap.

I agree with @nicostuhlfauth that this should be solved immediately and would've proposed something similar to @blaine's solution, which sounds good to me. Problem: I don't know any Ruby, so now we just need an awesome person to implement this 🙂

lx4r · Aug 17, 2018

I agree with @nicostuhlfauth that this should be solved immediately and would've proposed something similar to @blaine's solution, which sounds good to me. Problem: I don't know any Ruby, so now we just need an awesome person to implement this 🙂

Rather than elaborate new id structures, what is stopping an instance doing a 301 redirect when a user that has migrated to another is requested?
Then the requesting instance can update the mapping from the local /user/[n] to the remote url to fetch and everything can carry on.
This was resolved in Websub with tests before shipping - is there some design difficulty with ActivityPub that precludes this?

kevinmarks · Aug 20, 2018

Rather than elaborate new id structures, what is stopping an instance doing a 301 redirect when a user that has migrated to another is requested?
Then the requesting instance can update the mapping from the local /user/[n] to the remote url to fetch and everything can carry on.
This was resolved in Websub with tests before shipping - is there some design difficulty with ActivityPub that precludes this?

@kevinmarks the problems are migrating followers, and recovering from the instance going down permanently. Signaling that a user has moved is already implemented.

remram44 · Aug 20, 2018

@kevinmarks the problems are migrating followers, and recovering from the instance going down permanently. Signaling that a user has moved is already implemented.

By the way @remram44, do you know how the implemented moving works? Someone asked me a few days ago, but I found no documentation what actually happens when you enter your new user in the settings of the old server...

nicostuhlfauth · Aug 21, 2018

By the way @remram44, do you know how the implemented moving works? Someone asked me a few days ago, but I found no documentation what actually happens when you enter your new user in the settings of the old server...

hikari-no-yume referenced this issue Nov 22, 2016
Closed
Account takeout/archive download #201

nex3 referenced this issue Nov 23, 2016
Closed
Ability to change usernames #214

hach-que referenced this issue Nov 24, 2016
Closed
Always show domain in username, even for same instance? #226

swaldie added the enhancement label Dec 6, 2016

ineffyble referenced this issue Jan 10, 2017
Closed
Account export/data liberation #454

Gargron added a commit that referenced this issue Mar 30, 2017

Import feature for following/blocking lists (addresses #62, #177, #201,
#454)

e8875c6

nightpool referenced this issue Jul 12, 2018
Open
Inform followers that your account has moved #8003
1 of 6 tasks complete

trwnh referenced this issue Jul 16, 2018
Closed
User inbox URLs are not updated from webfinger when they change #5873
2 of 2 tasks complete

novoid referenced this issue Jul 21, 2018
Open
Memacs module for Toots (Mastodon) #54

aaronpk pushed a commit to indieweb/wiki that referenced this issue Aug 16, 2018

tantek__ added "tootsuite/mastodon#177" to "See Also"

9bf7b80

kevinmarks referenced this issue Aug 26, 2018
Open
Support 301 redirection for account migration #8465

tootsuite/mastodon

Join GitHub today

Support account migration #177

Comments

hach-que commented Nov 22, 2016

This comment has been minimized.

nex3 commented Nov 22, 2016

This comment has been minimized.

Gargron commented Nov 22, 2016

This comment has been minimized.

hach-que commented Nov 22, 2016 • edited Edited 1 time hach-que edited Nov 22, 2016 (most recent)

This comment has been minimized.

nex3 commented Nov 22, 2016

hikari-no-yume referenced this issue Nov 22, 2016

nex3 referenced this issue Nov 23, 2016

This comment has been minimized.

nex3 commented Nov 23, 2016

hach-que referenced this issue Nov 24, 2016

swaldie added the enhancement label Dec 6, 2016

This comment has been minimized.

hikari-no-yume commented Dec 6, 2016

This comment has been minimized.

marrus-sh commented Dec 7, 2016

This comment has been minimized.

hoodieak commented Dec 15, 2016

This comment has been minimized.

hoodieak commented Dec 15, 2016

This comment has been minimized.

hoodieak commented Dec 15, 2016

ineffyble referenced this issue Jan 10, 2017

Gargron added a commit that referenced this issue Mar 30, 2017

This comment has been minimized.

leeeeeny commented Apr 3, 2017

This comment has been minimized.

Maverynthia commented Apr 3, 2017

This comment has been minimized.

gdurelle commented Apr 4, 2017

This comment has been minimized.

leeeeeny commented Apr 4, 2017

This comment has been minimized.

Sadzeih commented Apr 4, 2017

This comment has been minimized.

Motoma commented Apr 4, 2017 • edited Edited 1 time Motoma edited Apr 4, 2017 (most recent)

This comment has been minimized.

cyphar commented Apr 4, 2017

This comment has been minimized.

cyphar commented Apr 4, 2017 • edited Edited 1 time cyphar edited Apr 4, 2017 (most recent)

This comment has been minimized.

TehShrike commented Apr 4, 2017

This comment has been minimized.

cdata commented Apr 4, 2017

This comment has been minimized.

stiell commented Apr 4, 2017

This comment has been minimized.

cyphar commented Apr 4, 2017

This comment has been minimized.

cdata commented Apr 4, 2017 • edited Edited 1 time cdata edited Apr 4, 2017 (most recent)

This comment has been minimized.

SirCmpwn commented Apr 4, 2017 • edited Edited 1 time SirCmpwn edited Apr 4, 2017 (most recent)

This comment has been minimized.

SirCmpwn commented Apr 4, 2017

This comment has been minimized.

Maverynthia commented Apr 4, 2017

This comment has been minimized.

kevinmarks commented Apr 5, 2017

This comment has been minimized.

SirCmpwn commented Apr 5, 2017

This comment has been minimized.

kevinmarks commented Apr 5, 2017

This comment has been minimized.

SirCmpwn commented Apr 5, 2017

This comment has been minimized.

kevinmarks commented Apr 5, 2017

This comment has been minimized.

SirCmpwn commented Apr 5, 2017

This comment has been minimized.

enkiv2 commented Apr 5, 2017

This comment has been minimized.

thomaslule commented Apr 5, 2017

This comment has been minimized.

hach-que commented Nov 22, 2016 •

edited

Edited 1 time

hach-que edited Nov 22, 2016 (most recent)

Motoma commented Apr 4, 2017 •

edited

Edited 1 time

Motoma edited Apr 4, 2017 (most recent)

cyphar commented Apr 4, 2017 •

edited

Edited 1 time

cyphar edited Apr 4, 2017 (most recent)

cdata commented Apr 4, 2017 •

edited

Edited 1 time

cdata edited Apr 4, 2017 (most recent)

SirCmpwn commented Apr 4, 2017 •

edited

Edited 1 time

SirCmpwn edited Apr 4, 2017 (most recent)

trwnh commented Jun 21, 2018 •

edited

Edited 2 times

trwnh edited Jun 21, 2018 (most recent)

trwnh edited Jun 21, 2018

trwnh created Jun 21, 2018

remram44 commented Jun 23, 2018 •

edited

Edited 2 times

remram44 edited Jun 23, 2018 (most recent)

remram44 edited Jun 23, 2018

remram44 created Jun 23, 2018

wiktor-k commented Jun 23, 2018 •

edited

Edited 1 time

wiktor-k edited Jun 23, 2018 (most recent)

wiktor-k created Jun 23, 2018