wl-copy: how to craft a multi-mimetype copy

[ghost]

wl-paste allows me to pick between several mime types, as offered by the source application.
I'm having a hard time understanding how I can offer several mime types at the same time using wl-copy, for example text/html and text/plain with different contents.

[bugaevc]

There's (currently) no way to do this; there is always a single input wl-copy uses.

If you copy something wl-clipboard believes to be text, it'll also automatically offer text/plain and a bunch of other conventional types:

wl-clipboard/src/wl-copy.c

Lines 252 to 259 in c983120

	if (options.mime_type == NULL || mime_type_is_text(options.mime_type)) {
	/* Offer a few generic plain text formats */
	source_offer(copy_action->source, text_plain);
	source_offer(copy_action->source, text_plain_utf8);
	source_offer(copy_action->source, "TEXT");
	source_offer(copy_action->source, "STRING");
	source_offer(copy_action->source, "UTF8_STRING");
	}

but will serve the same content.

See #8 for an early plan to generate different versions automatically (I'm still not sure that's a good idea).

It would be possible to let the user specify several source/type pairs, but

• It's unclear how this would look in the command-line interface (wl-copy --type foo/bar --data /path/to/file1 --type baz --data /path/to/file2?). Ideas are welcome, even if I don't end up implementing this for other reasons
• This is getting too complicated for the command-line tool. I can't imagine why anyone who's running wl-copy manually would want this. A clipboard manager, on the other hand, indeed want something like this —

but at this point it's better to start using Wayland directly instead of spawning wl-copy; but then using Wayland directly is quite complicated and boilerplate-y (as evidenced by wl-clipboard itself!). Perhaps this would be a good use case for a potential libwl-clipboard.

Let's make an experiment. There already is a private libwl-clipboard in wl-clipboard 2.0; it's not public or stable, the API will change and so on, but it's fine for an experiment. Can you try using it (via cgo, probably) and see if it's convenient enough for you?

[ghost]

For the command line, maybe something like --type foo/bar::data --type foo/baz::data --type data (no prefix defaults to text). But yes, I too can't imagine people using it, particularly if there's a library that abstracts away the low-level implementation details.

I'll look into the library and see if I can understand it (I don't know any C or low-level stuff).

[pengux]

There is a valid use case for being able to set multiple mime types for the data that is copied. For example, when you copy an image from a browser, wl-paste --list-types returns:

text/html
text/_moz_htmlinfo
text/_moz_htmlcontext
image/png
image/bmp
image/x-bmp
image/x-MS-bmp
image/x-icon
image/x-ico
image/x-win-bitmap
image/vnd.microsoft.icon
application/ico
image/ico
image/icon
text/ico
image/jpeg
image/tiff
SAVE_TARGETS

While if you take a screenshot with grim and pipe it into wl-copy, only the image/png is returned. The lack of other mime-types makes pasting the data into a browser to sometimes break and only print binary text instead of triggering a file upload for example.

About the syntax for the flag, how about just a simple comma separate strings for adding multiple types to the data:

grim - | wl-copy -t image/png,text/html

[bugaevc]

when you copy an image from a browser, wl-paste --list-types returns

(Quick note: the exact list is of course browser-specific, your seems to come from Firefox)

The lack of other mime-types makes pasting the data into a browser to sometimes break and only print binary text instead of triggering a file upload for example.

This sounds like a bug in the browser (or even the webpage) you're experiencing this with. Besides, why do you think it is specifically the lack of MIME types other than image/png that causes it?

That is not to say that it's a bad idea to want to be able to copy multiple types into the clipboard...

About the syntax for the flag, how about just a simple comma separate strings for adding multiple types to the data:

grim - | wl-copy -t image/png,text/html

Where would that take the data from? In particular, what would the HTML data be when copying an image?

With a single type (here, image/png), the data is read from stdin (here, it comes from grim). With multiple types, there would have to be a way to specify the data for each of the types.

[pengux]

Yes, it comes from Firefox when copying an image. Pasting the image from the browser trigger file upload and it works as expected. It's only when the image coming from wl-copy that it cannot be detected properly.

The suggestion is just to extends what is currently supported with the -t flag, so you can pass in multiple MIME types instead of just one for the data source. The question of supporting multiple data sources would be a different issue?

[bugaevc]

Yes, it comes from Firefox when copying an image. Pasting the image from the browser trigger file upload and it works as expected. It's only when the image coming from wl-copy that it cannot be detected properly.

You should report this issue to Firefox.

The suggestion is just to extends what is currently supported with the -t flag, so you can pass in multiple MIME types instead of just one for the data source. The question of supporting multiple data sources would be a different issue?

So you're saying you would want wl-copy to advertise image/png and text/html, but actually serve the same data (the image binary) for both? That doesn't make sense to me.

wl-copy does just that for text formats, but that's simply because they're fairly interchangeable.

[pengux]

The issue is not because of Firefox though. For example, copying image from Gimp gives these mime types:

image/x-xcf
image/png
image/bmp
image/x-bmp
image/x-MS-bmp
image/x-icon
image/x-ico
image/x-win-bitmap
image/vnd.microsoft.icon
application/ico
image/ico
image/icon
text/ico
image/tiff
image/jpeg

and pasting this in the browser also works as expected. In this case the image formats are also interchangeable like the text formats.

[bugaevc]

The issue is not because of Firefox though. For example, copying image from Gimp gives these mime types

How is this relevant? No matter what GIMP is able to copy, Firefox should be able to paste image/png.

In this case the image formats are also interchangeable like the text formats.

No. Why would you think so?

$ wl-paste -t image/png | sha256sum
ac2aac0023c36c8811f435077c3d76449c1c9b05cb980ea2b058ed24638a7e03  -
$ wl-paste -t image/jpeg | sha256sum
ce0e689283d419d4c6c3014e17eb64f7c692b23416b1650a0545d8d9dfbcf5bc  -
$ wl-paste -t image/bmp | sha256sum
dca2e36091578012d28acead8d5c08af678b73d86f2dcc842269ad93a0b95b40  -

[pengux]

Please don't focus on Firefox too much as the behavior is website specific and not because of the browser. It behaves the same way in Chromium too. Here is the mime types for an image copied in Chromium for example:

text/x-moz-url
image/png
text/html

I haven't investigated the combination of the mime types which makes it behave correctly yet though, but only want to suggest that wl-copy could support multi-mimetype like other applications.

With your example, these text mime types also differs?

❯ wl-paste
åäö
~
❯ wl-paste -t text/plain | sha256sum
630ff374c5c09af7d61d838217ace9fc4d62beb390927390e3a7c781a60c293f  -

~
❯ wl-paste -t text/plain;charset=utf-8 | sha256sum
åäö

e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  -

Note that unicode chars would not make the content identical if the mime type differ. I guess this also applies for images and it's the receiving application who accepts in which of the offered mime types it want to receive the data.

[bugaevc]

Please don't focus on Firefox too much as the behavior is website specific and not because of the browser. It behaves the same way in Chromium too.

Well in any case, it's something between the browser and the website that's broken if you cannot just paste a PNG image. Maybe the browser should synthesize the types that the known to be broken websites can't live without.

Providing the same image binary as text/html makes zero sense. Making it into a <img src="data:..."> would make some sense, but whatever client prefers that over a simple image/png is just too broken.

Again, making it possible to copy multiple content types could be a somewhat useful feature — for copying rich text, mostly, but not for working around broken clients that cannot handle simple images.

wl-paste -t text/plain;charset=utf-8

You realize your shell parses this as

$ wl-paste -t text/plain
$ charset=utf-8

right? You need to quote the ; to prevent that.

[pengux]

Could it be that the other applications are setting multiple data parts with different mime types when the user is copying an image into the clipboard? For example, here is a list of mime types that a web application inspect using the browser's clipboard API

https://w3c.github.io/clipboard-apis/#mandatory-data-types-x

Does wl-clipboard support setting multiple data sources yet?

Yeah, sorry about the mistake in the commands, I realized it after a while. It does return the same data:

❯ echo "åäö" | wl-copy

~
❯ wl-paste -t "text/plain;charset=utf-8" | sha256sum
630ff374c5c09af7d61d838217ace9fc4d62beb390927390e3a7c781a60c293f  -

~
❯ wl-paste -t text/plain | sha256sum
630ff374c5c09af7d61d838217ace9fc4d62beb390927390e3a7c781a60c293f  -

[Bsplesk]

good day
Case:

1. copying a table from confluence (firefox web browser) Ctrl+C (confluence edit mode);
2. save clipboard to file $wl-paste > test.xml
3. read file to clipboard $wl-copy < test.xml
4. insert confluence/Ctrl+V (bad insert/text, formatting lost and there is no way to restore mime types):
   

$ wl-paste --list-types TIMESTAMP TARGETS MULTIPLE SAVE_TARGETS text/html text/plain;charset=utf-8 UTF8_STRING COMPOUND_TEXT TEXT STRING application/x-moz-custom-clipdata $wl-paste > test.xml $wl-copy < test.xml test.xml

1 Tets table 1 1 fgfgfggf 23 2 3 ggg

[bugaevc]

What format does Confluence natively copy/paste its contents in?

Does wl-copy offer text/html the second time? What if you name the file test.html instead of test.xml?

[Bsplesk]

What format does Confluence natively copy/paste its contents in?

Does wl-copy offer text/html the second time? What if you name the file test.html instead of test.xml?

rename: test.html

[Bsplesk]

What format does Confluence natively copy/paste its contents in?

Does wl-copy offer text/html the second time? What if you name the file test.html instead of test.xml?

xml

• after pasting

$ wl-paste --list-types 
application/xml
text/plain
text/plain;charset=utf-8
TEXT
STRING
UTF8_STRING

[bugaevc]

So your issue seems to be that xdg-mime doesn't recognize the file as HTML. Try doing wl-copy --type text/html explicitly then.

[Bsplesk]

there is not enough

[glowingsword]

Hi. But if I want to copy something with wl-copy(rich text from Writer or html from firefox) and store it with all mimetypes in set of files(or sqlite tables with mimetypes) how I can write all data with all mimetimes to clipboard later? With proposed feature(with crafting a multi-mimetype copy) it would be possible. But without this feature it can't be possible.

[lilydjwg]

I have two cases when I want to copy as multiple types.

1. Copy images. Wine seems to support only image/bmp, not image/png. So simple screenshoters don't work there. However in this case, I want to convert data on demand.
2. Copy file paths. You can copy file paths as text/uri-list, and paste them into file managers or applications accepting files (e.g. Telegram). However, you can't paste them into terminals or text editors (Firefox even pastes garbage). I'd like to copy a version of absolute paths as text/plain too. That's what thunar does.

[lukaslihotzki]

I have a proposal for supporting multi-mimetype copy without temporary files, using an external converter program:

wl-copy keeps taking a single buffer from stdin of a specific type (autodetected or -t). A new parameter -C is added to specify a path to a converter program. This converter is called on-demand when conversion is needed (when the requested type is not the buffer type). wl-copy executes this converter with stdin wired to the existing buffer FD, stdout to a new separate buffer FD, and the input and output MIME types as arguments, so a single program can handle multiple types. Then, the output buffer FD is passed to wayland.

These converters can be very simple adapters to existing tools. They help to keep the wl-copy command line short and expose all conversions through a unified interface to wl-copy. The supported types are output by the converter itself, so wl-copy has a correct knowledge of the supported types. Therefore, wl-copy calls the converter without output type on initialization to get the list of supported types. stdin is still wired to the input buffer FD, so these types can be dynamic. Lines starting with # are ignored by wl-copy.

A simple image converter ./convert-image.sh using ImageMagick would look like this:

case "$2" in
"") printf "%s\n" "# Usage: $0 INPUT_MIME OUTPUT_MIME" image/png image/bmp ;;
image/png) convert - png:- ;;
image/bmp) convert - bmp:- ;;
esac

Then, a user could call capture_screen | wl-copy -C ./convert-image.sh for example.

This would solve the use cases in the last comment with reasonable complexity in wl-copy, flexible and easy to use, much easier than using a library to access the wayland clipboard.

Different pre-existing types (so no on-demand conversion) are not the primary use case, but it still can be done by piping an archive to wl-copy that contains a file per type and use this converter program: if [ -z "$2" ]; then tar t; else tar Ox "$2"; fi