Synchronization with other platforms

[xplosionmind]

Is your feature request related to a problem? Please describe.

No

Describe the solution you’d like

Adding a book to Bookwyrm (or editing its status) automatically synchronizes with other platforms. Ideally, this could be implemented vice-versa: any change occurred in other platforms, could be pinged and changed in Bookwyrm, too.

I opened the same issue for Inventaire, and for Open Library

Describe alternatives you’ve considered

There are many issues (#1134, #627, #296, #1678, and more) that request the option of importing and/or exporting to other book-focused platforms, among them GoodReads, Inventaire, Open Library, Calibre, and, of course, Bookwyrm.

Suppose a user has one account on more than one of those platforms (most of us), and they wants to add a book they just bought. Every single time this happens, they would have to go to every platform and import the book.

Additional context

Synchronization would include:

• book metadata
• reading statuses
  • started reading
  • currently reading
  • finished reading
  • paused (or stopped reading)
• collections (a.k.a. shelves)
• ???

[mouse-reeve]

Thank you! From a technical perspective, syncing book metadata is a very different task than syncing reading statuses and user data. I have talked to folks at OpenLibrary and the next step for better synchronizing data there is to create an OpenLibrary bot. I haven't looked in to what this would entail with Inventaire yet. But there is some prior work that needs to be done to make BookWyrm's own metadata editing process a little more robust first, so this is one of those things that is both important and on the roadmap, and also going to take some time. (To be clear, we do already have code in place for updating from OpenLibrary and Inventaire to BookWyrm, but not in the other direction.)

Personal data, like review and shelving, is a lot trickier, because it requires authenticating accounts across OL or Inventaire, and translating the format for the different platforms. For that to happen bi-directionally, we'll probably all have to make code changes. That's all possible (and I'm delighted to work with both crews), but is a handful of large tasks that depend a lot on the architecture of the external service.

I see this as a variety of subtasks:

• Sending book metadata changes to OpenLibrary
• Sending book metadata changes to Inventaire
• Cross-authenticating with OpenLibrary
• Syncing user data with OpenLibrary
• Cross-authenticating with Inventaire
• Syncing user data with Inventaire

Does that sound in line with what you're suggesting?

[xplosionmind]

Does that sound in line with what you're suggesting?

Totally, yes.

It appears to be an immense amount of work, too! Thanks a lot for your efforts and the great planning. Your perspective is really interesting and I am so psyched to see what is going to be next!

Anyways, I will leave to you the creation of individual, specific issues for the points you listed. Doing so, you can provide more technical detail and insights than a non-developer like me could provide.

Thanks again for your work!
Best,
Tommi

[skobkin]

Personal data, like review and shelving

I personally think that this part is much less important than metadata upload. Because the later allows all users of all platforms and instances to benefit from the actions of a single user who just wanted to add a book he just read.

So I'd say that for the benefit of entire community metadata upload features should have higher priority than personal reading status synchronization. It will save a lot of human-hours fot a lot of people if their books would be already imported by someone else even if that someone is on different instance.

[uriel1998]

Exactly what data is being imported and how from the Goodreads import? I have a lot of indie publisher books that maybe didn't make it into Goodreads, but I have all the appropriate metadata in my Calibre library.

To the point immediately above, the default exported fields in Calibre's CSV export include a description and most of what you need to populate the book fields locally. (There's even a local path to a cover file, but that'd be a PITA indeed.)

So if it's as simple as mangling the output CSV from Calibre into the Goodreads format, that's fairly easily doable with external tools. But it appears that if it's not found automagically during the import, the rest of the metadata (such as the book description under the "comments" field in the Calibre export) would be lost. I have no problem exporting my library to CSV and running another tool on it to populate metadata, but I really don't know what's going to be kept and lost. Am I just failing my Perception check?

[mouse-reeve]

@uriel1998 These are all great questions! While I completely see why it seems like it would be, CSV import is actually an unrelated feature to this issue, so I want to answer these in a separate place. Would you be able to open a new discussion with these same questions?

[ggrensteiner]

You might be able to build a plugin/integration for audiobookshelf. It has reader functionality for audio and ebook formats so the tracking would be automatic and it's metadata system is much more complete than your own.

https://github.com/advplyr/audiobookshelf

[mekarpeles]

Mek here from open library. Always happy to help invest effort to better sync data -- Open Library is continuing to release monthly data dumps of everything we have: https://openlibrary.org/developers/dumps

We also have an olclient https://github.com/internetarchive/openlibrary-client which can be used (authenticates using archive.org s3 credentials) which can be used to queue imports.

If book + author data is wrong on Open Library, we'd love to incorporate these corrections so anyone else may benefit from the data. We have lots of librarian tools to fix book + author data in the open library catalog. What I would love to see is edits being made to Open Library data and then these updates being streamed out to all the services which need them.

We're not really planning on importing patron data from other platforms at this time. Platforms e.g. Bookwyrm are doing a great job of providing features over ActivityPub and Open Library team only has so much bandwidth. Keep it up!

[zkat]

In case it helps round out an idea of the use-case for this:

I'm currently holding back on switching to Bookwyrm because of the lack of tooling integration. But Goodreads doesn't really lack this: Calibre and Kindle both have very good integrations with Goodreads.

If I were able to use my tooling to sync to Goodreads, I could then post to Bookwyrm through the sync plugin and leave Goodreads alone as "just" a conduit until tooling around Bookwyrm itself matures. I see it as a way to kill multiple birds with one stone, as far as managing the nascent ecosystem around Bookwyrm.

...of course I could also just look into writing a Bookwyrm plugin for Calibre (which would probably be highly desires), but I'm not sure I'm invested enough in Bookwyrm to dedicate time to that (yet).