[feature] Import from Mastodon

[xxxserxxx]

Is your feature request related to a problem ?

It's not per se a problem, but I imagine that it's not uncommon for people to test-drive Mastodon by signing up with a shared service before self-hosting. Being able to import a Mastodon export would facilitate relocations.

This is related to, but not a duplicate of:

• [feature] Database Migration Path for Mastodon => GoToSocial #128 (migration from mastodon #447) -- this request differs in that it is not a request for database migration from one self-hosted solution to another, but for account migration path from a shared-hosting account to a self-hosted gotosocial server. Although, I would not preclude using the feature to onboard multiple users from other Mastodon instances.
• Import export #194 -- is about import and export of gotosocial databases. This feature might, however, leverage Import export #194.

Describe the solution you'd like.

An extension to the CLI that accepts a Mastodon account export zip file as input, and which creates and configures an account and imports the data in the input file.

Describe alternatives you've considered.

gts admin export will dump the entire database; import will read such a dump. I don't know how much the formats diverge (although, from casual surface reading they're not the same schemas), but I will attempt to hack a transformer that will insert records into a GtS export, and import that. My Mastodon export is quite small, as I've only been using it for a limited time, and I'm expecting some light transformation scripting and probably some text editor work.

Having an Import API endpoint would allow users to upload and import their own files, rather than having to request an admin do it on the server. Having a web page to allow the upload would be the full-featured version.

I am guessing that, while the use case is probably not uncommon, it's also probably not a frequent thing for GtS admins to need to do. In other words, I think many GtS admins probably would like to do this once, but probably only exactly once. If that's true, the CLI solution is probably sufficient for most people, and the API and web pages would be overkill.

Additional context.

Mastodon has an export feature https://docs.joinmastodon.org/user/moving/#export which exports "in ActivityPub format" (per the Mastodon docs). The archive is a zip file containing all posted media attachments, the user avatar and header, and several json files representing various data and metadata:

media_attachments/files/...
outbox.json
likes.json
bookmarks.json
avatar.jpeg
header.png
actor.json

Schemas for the jsons are attached. I didn't find an explanation of the directory structure of the media attachments, but they match the URLs embedded in the post attachments array.

schemas.zip

[tsmethurst]

Thanks for opening this :)

Migration from Mastodon / other servers -> GtS is something we wanna implement as we move out of Alpha and towards Beta. Reason being that we don't necessarily want to encourage people to move their accounts to GtS yet because we're still squashing lots of bugs and implementing basic features.

We also eventually want to properly support the ActivityPub Move activity that Mastodon currently uses for migrating accounts from one instance to another. This is currently the easiest way to move your account and still keep your followers/following list. #130

However, since the Move activity currently doesn't let you keep posts + media, only followers/following, this issue you've opened will come in handy in the future.

Also worth linking to #2 here, since this is prettymuch the same thing but a bit more fleshed out.

[zladuric]

A quick question: does this answer mean that you don't support even the bare "Move account" functionality yet (without moving the data)?

I'm asking this in context of mastodon.technology shutting down, and my own notions of hosting my own anyway being ripe enough. In short, I want to move here, I can live with the lost history, but as I read these linked issues, this is not supported yet?

If that's true, what can I do to make this support happen in the near future?

[tsmethurst]

does this answer mean that you don't support even the bare "Move account" functionality yet (without moving the data)?

Yes that's right, it's not implemented yet :) We plan to implement it at the end of alpha phase (some time next year), when GtS is more stable and has more features implemented. Me and the other devs aren't comfortable just yet offering GtS as a ready-to-go alternative to other fedi implementations. But we'll get there :)

If that's true, what can I do to make this support happen in the near future?

We're probably not gonna push working on this feature forward, cuz there's lots of other stuff to do first. But when we do start working on it, it would be nice if we knew exactly how the Move activity works as implemented by Mastodon, so we can ensure compatibility. So it would be cool to gather links / info / schemas on this issue to speed implementation when we get around to it: #130 :)

[xxxserxxx]

Please note that what the ticket requests is to support a redirect, not a move.

Mastodon supports redirections, which are soft moves. The account is retained, but posting is restricted and users are redirected (by Mastodon) to a different account.

The request above is to import follows, bookmarks, etc. from an account -- something that is still necessary in a redirect.

More particularly, supporting Move activities would not address this ticket.

Note that the ticket never directly requests migration support: the ticket is to support cloning, not replacing, which is the user-side functional other half of a redirect.

[zladuric]

Good point, I was actually only looking for a redirect right now - a move would be great but redirect is the minimum of that I would like to have. I did look (and have already checked out #130) and it seemed like a lot, but supporting redirect would be simpler.

@tsmethurst does the comment above this one, about a move and redirect being different, make sense to you?

I mean, regarding the implementation effort, it looks similar up to a point, but they do seem to be distinct activities.

[tsmethurst]

does the comment above this one, about a move and redirect being different, make sense to you?

It's not entirely clear to me what's being requested, no... data import, Move, and recognizing Mastodon redirects are related but distinct things. Maybe it's better if we make separate tickets for all of these?

[edit] I realize this answer also doesn't really clear things up, my apologies, I haven't properly woken up yet today :')

[xxxserxxx]

The request is for the capability -- either through a tool, an API, or a page in gotosocial -- to import data from a Mastodon profile export into a gotosocial account. In particular, I'd like these data to be imported into gotosocial from a Mastodon zip.

• bookmarks
• following
• likes
• hashtags
• block and mute lists

I think gotosocial doesn't support hashtag-following? So, if there's something gotosocial doesn't support, it wouldn't be imported.

That's the minimum. This is the most difficult data to manually recreate. It would add some convenience if the tool were also able to create a new account and set the account settings from the zip (the avatar, header, default post language, follow-approval settings, etc), but that's not oenerous for users to re-set so I personally believe there's not much value.

If you like use cases:
As a User, I want to be able to easily populate a gotosocial account with the social relationships from a Mastodon profile export zip, so that I can have the same feed I had on Mastodon. Usually, this will be used in conjunction with Mastodon's "Redirect" function, which is a setting in a Mastodon account that differs from a Move in that it is undo-able.

[norayr]

i guess migration from mastodot to gts should be conducted in 2 main steps:

• convert the db to new, gts db.
• cp images of our instance users and put those in location which makes them visible by the same path.

i dont want to move to new account. i want to contitiue with the same account@domain.name but with gts.