💱 Converters for Imports

[fuksman]

Proposal

We want to begin exploring the development of new importers and exporters for Anytype. Our goal is to involve contributors right from the start of this project.

Anytype currently supports importing from Markdown, HTML, CSV, TXT, and Any-Block formats. Additionally, we offer a dedicated import feature for Notion.

Now, we want to test an approach where we don't create new direct imports from other apps. Instead, we rely on tools that convert data from different apps into Any-Block or simpler formats. We can then import this converted data into Anytype. The process would involve the following steps:

1. The user exports data from the external app or service.
2. The user opens the Importers Gallery and selects the importer for the app from which they want to import data.
3. The user uploads the exported file to the importer.
4. The importer converts the data to the Any-Block or a simpler format.
5. The user imports the converted data into Anytype.

In the future, as more importers are created and all corner cases are covered, this flow could be further simplified by deeply integrating importers into Anytype. This would allow users to skip manual conversion and uploading steps.

To begin, we suggest creating an importer for Evernote. Its data can be exported in HTML or ENEX (XML) formats and can be converted to Any-Block or Markdown with minimal effort.

Useful links

• Any-Block data format, with proto files in the root and JSON-schemas in json/models folder.
• Format description for Evernote's ENEX format.
• Here you can find export examples from Evernote made by @Shampra.
• space-pb.zip — space example in Any-Block PB format.
• space-json.zip — space example in Any-Block JSON format.

Remarks

Please feel free to ask questions or share your ideas in the comments section below.

You are welcome to create converters in any form and language of your choice. However, please note that Typescript solutions are more likely to be integrated into the Gallery and Anytype in the future.

We recommend to use pre-installed relations and object types as much as possible.

After converting the data to Any-Block or other data formats, you can test the result in the following ways:

• Import a single object by selecting File → Import → %chosen_format%.
• Import a collection of objects by creating a ZIP archive with objects/types/relations and importing it using File → Import → %chosen_format%.

P.S. There is no "Any-Block" button available in the current production version of Anytype. Instead, there is a "Protobuf" button that can be used to import ZIP archives with JSONs. The button has already been renamed in the main and will be released to production soon.

[Shampra]

Is it possible to get a json exemple of a Anytype page (with a bit of everything, except for collections/sets, which are not in Evernote)?
I create a anytype page similar to note example from Evernote but I can only export as protobuf (unreadable, so harder to understand/try things^^).
Or if you have a tips to "read" protobuf file?

[ra3orblade]

Current main/latest alpha has JSON export.

[Shampra]

If someone else need it, here an example (with a page equivalent to an Evernote note)
Anytype.20230912.164409.58.zip

[Shampra]

I've pehaps a bug (or a misundertanding) about export/import JSON :
https://community.anytype.io/t/export-import-doesnt-keep-correctly-featured-relation/11091

[Shampra]

Is there somewhere a list of accepted color for text and background?

[ra3orblade]

"textColor": [
		"grey",
		"yellow",
		"orange",
		"red",
		"pink",
		"purple",
		"blue",
		"ice",
		"teal",
		"lime"
	],

[Shampra]

Ouch, there's a bug blocking all Evernote imports :
https://community.anytype.io/t/import-json-doesnt-import-tag/11297

[fuksman]

Thanks for noticing this and the previous bug. We're working on them!

[Shampra]

Is there a debug log activable for imports?
Right now, I'm trying to import a note containing files: no errors are reported, but the files are not imported (and on the page, they are marked as failed to load).

More generally, I think it would be useful to enhance the error (or warning) messages for debugging or for the user. Currently, it's either "import failed" or "import successful" (which isn't always accurate).

[fuksman]

Hello @Shampra,

Try these settings.

[Shampra]

Thanks!
At least, by looking in a different way, it allowed me to discover one new bug 😁!

[fuksman]

🫣 thanks, will work on this one too

[Shampra]

No more error with these Flags :(.

I could use a little help.
How are the files imported?
I have a files folder next to my json file, with the files in it.
In the current blocks, I've these (same compared to an AT export) :
"file": {
"hash": "5afaf79789a776d81ec91ccbdc9fdaba",
"name": "file-sample_150kB.pdf",
"type": "PDF",
"mime": "application/pdf",
"size": "1142288",
"style": "Link",
"state": "Done"
}

There is a filekeys part but it seems to be internal (hash of small/thumb/...) so I didn't use it.

When I import, I've the block in the page but the file isn't downloaded...
What is the mecanism that say to AT it must import these files?

[Shampra]

Ok, it seems to be a bug.
I hadn't finished my AT export -> AT import test: the import didn't import the files.

Issue created here : anyproto/anytype-heart#456

[AnastasiaShemyakinskaya]

No more error with these Flags :(.

I could use a little help. How are the files imported? I have a files folder next to my json file, with the files in it. In the current blocks, I've these (same compared to an AT export) : "file": { "hash": "5afaf79789a776d81ec91ccbdc9fdaba", "name": "file-sample_150kB.pdf", "type": "PDF", "mime": "application/pdf", "size": "1142288", "style": "Link", "state": "Done" }

There is a filekeys part but it seems to be internal (hash of small/thumb/...) so I didn't use it.

When I import, I've the block in the page but the file isn't downloaded... What is the mecanism that say to AT it must import these files?

We don't use file folder in AT import yet. It's created for backup purpose for now. To import file we use file keys and hashes provided in objects. Hash is used as ID of file object in Anytype. Currently we have problems with importing files, which was deleted in account, because we delete them by their hashes. So we don't have these files internally after their deletion and don't have a mechanism to load them again after import. But we are going to work on that.

[Shampra]

Hi,
I try to understand how "simple table" work with JSON.
I must admit that I find it a bit confusing compared to the rest.

Is there any documentation somewhere?

For example, some text appears in two block elements (there is only one cell in the table with this text).
(652283500284c84504969cb2 is a column)

refered here :

Or pehaps it's a bug?
I copied and pasted a table from Evernote, then modified it.
By the way, I also have images referenced in the export that do not appear on the page (and are listed among the files in AT but are not visible): I had tried to paste an image into the table.

I'm going to try exporting another table.

[Shampra]

I'm going to try exporting another table.

Ok, there are bugs. With a "fresh" table exported, I was able to analyze the JSON structure without any issues.

Still no date for the improved tables?

[moverware]

Hey there!
I created a simple importer for Google Keep. Somewhat restrictive at the moment and doesn't import tags, images, or colors (mainly because I didn't use those) but does properly parse hyperlinks in the anyblock format :) I made this before finding out about this thread/community so I'm sure lots could be improved. Happy to take contributions or requests!

https://github.com/moverware/keep2Anytype

Also if there are any guides on how to upload images as part of the anyblock format that would be great. I see in the exported any-blocks that it has a ref to some image file ID, but not sure how to generate that in advance and include jpegs in the import jsons.

[Shampra]

For image, just see here, they are working on it :
https://github.com/orgs/anyproto/discussions/82#discussioncomment-7176599

[fuksman]

@any contributor @Shampra gallery

[fuksman]

@any contributor @moverware gallery

[tochinet]

Do you intend to provide the same channels for exporting notes than for importing ?
I like many ideas of any, but I want to be sure I can go back (to Joplin) in it doesn't click between us ...

[fuksman]

In Anytype, it is represented in Any-Block format, which can be imported and exported either in protobuf or JSON. Hence, if contributors create converters to generate Any-Block files, it is also possible to create converters from Any-Block to other formats.

[Mathspy]

If I export an object and then immediately imported it back it seems to create a new identical object instead of overriding the pre-existing one. Is there any intention of changing that behavior or making it customizable? I was intending to use exporting/importing feature to run custom updating code on my objects.