Idea: embed drawing source code into the exported PNG

[frantic]

I was thinking it would be cool if there was a way to export images with an ability to come back and edit them afterwards.

Excalidraw currently has two buttons to save data: one exports JSON, the other one exports PNG

Why have two buttons while we can get away with just one?

PNG allows text metadata or we could use stenography and embed the JSON into the exported pixels (e.g. as shades of white in the background or border).

[vjeux]

There's missing a mindblown reaction :p

[orta]

This is a clever idea

[frantic]

@vjeux what do you mean? It's right there…

[giovannigiordano]

@frantic what about a QR code? We could put it an a corner then remove it when importing the image.

[giovannigiordano]

I made a try with QR Code here.
I am converting a JSON to a QR Code with react-qr-svg, then exporting the created svg to a canvas to get an image, and then reading the image data with jsqr`.

[giovannigiordano]

@frantic I would prefer to add text metadata to the PNG. I have found https://github.com/mozilla/openbadges-backpack/wiki/badge-baking#baking which could be used to add metadata to the image.

[dwelle]

@giovannigiordano what about a QR code?

Maybe I'm missing something, but surely we don't want to be displaying a QR code in the corner of the exported images?

@frantic we could use stenography and embed the JSON into the exported pixels (e.g. as shades of white in the background or border).

It's steganography, btw (I get it wrong all the time, too). And I'd not add anything to the bitmap, certainly not anything that visibly changes it. Even if it's done right, and isn't discernible by naked eye, it would likely prevent image optimizations etc.

Thus, same as @giovannigiordano I believe the only way to go about this is metadata.

But still, is it a good idea? It'll add KBs (potentially 50-100KB+ for complex scenes), unnecessarily for most use cases. Keep in mind that this would increase the payload when the images are embedded in blog posts etc. (without running them through an optimizer that strips metadata).

On top of that, no one ever expects that a png file contains the original scene data, so we'd need to explain it.

And on top of that, when someone wants to just save a scene file, they'd now need to bundle in hundreds of KBs of bitmap data in form of PNG alongside it?

[lorisdev]

Maybe there can be a middle ground here.

The metadata could only contain the public unique url of the drawing (new feature discussed here #409)

Thus we would have the best of both world :

• ability to automagically drag'n'drop the png on the canvas and it would load the drawing from the backend
• a very negligible increase in png weight (SHA-1 hash value is 20byte)

[dwelle]

@lorisdev interesting idea, except the public urls are generated only manually. Moreoever, even if we decided to generate those urls on every canvas change, automatically, some users may not want to generate public urls at all (even if they're random).

Also, the url generation, I reckon, takes some time, which means it'd potentially slow down the export (you'd click on export and need to wait until the url is generated) --- this would probably be the least problem, though.

Also note that it'd not work if user isn't connected to the internet, or when the service suddenly stops working for some reason. I bet users would expect that when they save a scene to their hard drive, it stays on the hard drive, forever.

[frantic]

@dwelle thanks for the feedback!

I believe the extra file size can be addressed, see below.

Just to set the context: I'm mainly shooting for a "wow" and "just works" experience for people and replace 2 buttons with just one. The way I see the UX for it is something like this:

(I suggest we pre-select the checkbox)

Regarding the size, here's the 1666 bytes JSON for the image above:

{"version":1,"source":"https://www.excalidraw.com","elements":[{"id":"lfcLDbvi2EmWhJJsEh9aw","type":"rectangle","x":181,"y":449,"width":150,"height":64,"strokeColor":"#0693e3","backgroundColor":"#abb8c3","fillStyle":"hachure","strokeWidth":1,"roughness":1,"opacity":100,"isSelected":false,"seed":425135001},{"id":"MZlAOz4HRcs_MsRvZ-Y29","type":"ellipse","x":445,"y":316,"width":165,"height":99,"strokeColor":"#000000","backgroundColor":"#ffffff","fillStyle":"hachure","strokeWidth":2,"roughness":3,"opacity":100,"isSelected":false,"seed":1912762162},{"id":"qkuwhKWNU2_SE5o8iRmyO","type":"arrow","x":342,"y":432,"width":97,"height":-47,"strokeColor":"#000000","backgroundColor":"#ffffff","fillStyle":"hachure","strokeWidth":1,"roughness":1,"opacity":100,"isSelected":false,"seed":893191035},{"id":"pzZB31T_hLFW-e3rXFoDl","type":"text","x":200.5,"y":463.5,"width":113,"height":31,"strokeColor":"#0693e3","backgroundColor":"#abb8c3","fillStyle":"hachure","strokeWidth":1,"roughness":1,"opacity":100,"isSelected":false,"seed":1526850813,"text":"This is nice","font":"20px Virgil","baseline":22},{"id":"3R5VCZQb9xqu4H0YYvUIB","type":"rectangle","x":630,"y":584,"width":0,"height":0,"strokeColor":"#0693e3","backgroundColor":"#abb8c3","fillStyle":"hachure","strokeWidth":1,"roughness":1,"opacity":100,"isSelected":false,"seed":1065531265}],"appState":{"draggingElement":null,"resizingElement":null,"elementType":"selection","exportBackground":true,"currentItemStrokeColor":"#0693e3","currentItemBackgroundColor":"#abb8c3","currentItemFont":"20px Virgil","viewBackgroundColor":"#ffffff","scrollX":0,"scrollY":-2,"cursorX":95,"cursorY":54,"name":"excalidraw-202018114520"}}

Looking at the structure of it, it's easy to imagine how we could use Thrift or something like it to encode the data in a much more space-efficient way. Additionally, PNG supports zTXt chunks that is supposed to be compressed.

All in all, I'm sure we can make the source a tiny fraction of the resulting image.

[pshihn]

The idea of storing metadata in the image is really cool.

My main concern is this meta data can be lost if the image is modified in any way. A lot of editors/transformation-code/CMSes may strip out this metadata when the image is reprocessed.
So the user needs to remember that this particular instance of the files is the one that can be edited.
It may be fine for most use cases though, but thought I'd share my concern.

[dwelle]

@frantic right, that's better, but I'm not sure that storing metadata on PNG constitutes a "just works" UX. No one is gonna understand what's going on, because when you export a PNG, no other app then allows to reimport it to get to the original scene data --- that's not what people expect. So being the first (?), we'd need to make the explanation longer --- it's a big inferential leap.

There's also the question of where one is going to look for scene export. I would never look at PNG export to do that, and having a checkbox saying "make editable in the future" won't help much.

What happens when we support importing images into the scene? What takes precedence, the PNG bitmap, or the scene metadata? (Although this could be solved by having a prompt asking what the user wants to do in this case.)

Either way, it still doesn't address the issue of wanting to store the scene-only. Why force people to store PNGs just when they want to keep the scene data around.

What happens when we implement #253 or artboards with multiple slides/scenes? What do you export then, if we drop the support for exporting scene-only JSON files?

[frantic]

All good points, thanks for taking time to elaborate!

But consider the alternative—how many people are going to download the JSON along side the PNG to be able to edit the image in the future? I'd expect not many. Being able to edit the PNG to fix a typo would be a huge win.

As for scenes and more complicated projects, I think that's separate and it ties into other discussions about server persistence and accounts support. This is more about embedding the source for the subset of shapes being exported.

[dwelle]

All the above said, I still think this idea is cool.

IMO if it's implemented as such:

1. retain scene-only JSON export
2. keep the checkbox to opt-out of this (store to localStorage to remember)
3. ensure it's explained well

Then I'm personally all on board with this. Also note that I'm just voicing my concerns.

(Also, I was arguing on users' behalf, not my own. My personal use case is: export PNGs to upload to imgur which has its own optimizer where I reckon the metadata is stripped. And not having to keep separate JSON files is actually cool with me.)

[lipis]

Soon we could store these JSON files in the backend.. #409

[dwelle]

@frantic Didn't notice your last comment before posting my own.

@frantic As for scenes and more complicated projects, I think that's separate and it ties into other discussions about server persistence and accounts support. This is more about embedding the source for the subset of shapes being exported.

@lipis Soon we could store these JSON files in the backend.. #409

Still doesn't address the case where I'm not connected to the internet, or I elect not to log-in or don't want to persist potentially private data on more-or-less public server.

Also, the idea of Excalidraw was to be simple and fast, without forcing users to log-in. Adding a feature to bundle scene data into PNG shouldn't come at a cost of forcing users to have to sign-in to persist their (more-complex) scenes.

[lipis]

no need to sign in... only if they choose to share via URL.. otherwise you can always download the JSON, why bloating with PNG something that only Excalidraw could resolve..

[dwelle]

@lipis oh, I thought you were arguing to do away with the JSON export in favor of persisting on backend.

Anyway, I've stated my concerns, so I'll back off this issue for now, let others speak :).

[lipis]

Let's not embed stuff in PNG files..

[sansnom]

Hi, I think this is a killer feature at least for me. You can do this for example with draw.io. How they do that ? Simplify store the source as a base64 blob in the PNG metadata. I personally use this all the time because I keep loosing the source. :(

Why I think it is useful ? Because you don't have to keep two files. You can just import the PNG and you can start editing again ! It could be an option like draw.io for people who don't want it (space issue or other concerns) like @frantic has shown.

[dwelle]

Ok, I'm reopening this for discussion.

[pklaschka]

I believe this would be a great idea (maybe additionally/alternatively with an option to embed it in SVGs, which would probably be even easier).

A quick overview of why I think this would be a great feature (especially in SVGs):

I, in a few smaller software development teams (many projects are also open-source), am often responsible for keeping the project documentation up-to-date (I'm one of the few weird people who like writing documentation 😜).

Especially for internal documentation (and, depending on the formality of a project, also public documentation), I love to use Excalidraw because of its speed and not having to focus on getting everything pixel-perfect.

Storing both sources and results (image files) for diagrams in a repository with multiple people working on it can be quite a challenge (especially as in small teams, there often isn't a lot of time for writing documentation). Manually checking that everything's "in-sync" takes time, the instructions on how to edit a specific file are often not read by people who "just want to fix a quick mistake", it can be a challenge to get everyone to export the image to the right location again and so on.

Let's take the SVG "variant". If this embeds the source as a simple comment at the top including an "Only edit this file using Excalidraw" notice (cf. my proposal below), having that SVG file still be editable in Excalidraw would help in a lot of ways:

1. source and image files can't be out-of-sync anymore as they are one and the same file
2. it is easy to automatically check if a Pull Request "bricks" / "ejects" an editable file to a "pure" image file and write an automated comment about please fixing it before the PR gets reviewed
3. additional file size isn't really a concern of mine as, if needed, I could still strip it in the documentation build process
4. quickly looking at changed source files in GitHub/GitLab/... is a huge plus when trying to review PRs
5. every bit of making writing documentation less bureaucratical helps a lot when trying to get fellow developers to document their code 😉

Proposal for embedding the structure into an SVG export:

(I don't mean it should be done exactly this way, this is just meant as a quick idea on how this could be achieved)

<!-- Do not edit this file with editors other than Excalidraw -->
<!-- excalidraw-document: {...} -->
<svg>...</svg>

[maio]

Great idea. My use-case would be storing PNG (with metadata) into Git repository of a project, so that I or anyone else could modify it later on (+ it can be displayed in any markdown file as it's just PNG).

[freemansoft]

As someone else pointed out, draw.io supports this with png files. Create a diagram and export to .png. You can open the png with the drawing tool later and edit it. The save operation saves as png at that point, no export needed that time. It makes it easy to put editable diagrams in source repositories.

[dwelle]

Alright alright already 😅, here's a draft PR for the PNG: #2219

Try at https://excalidraw-git-embedscenetoimage.excalidraw.vercel.app/ (for now it works only with drag & drop)