Submit AppImage MIME Type to xdg/shared-mime-info #144

elvisangelaccio · 2016-05-18T15:00:10Z

Hi,
do you have any plan about getting an "official" mimetype for .appimage files? Currently they are detected as executable files by the shared-mime-info database, which is fine. However there are use cases where a dedicated mimetype would be handy. For example, users of archiving tools like KDE's Ark or Gnome's file-roller may be interested in browsing the content of an appimage: https://bugs.kde.org/show_bug.cgi?id=363209

This new mimetype could inherit the application/x-executable mimetype, to express that appdata files are still executable files but they are also something more - e.g. they can be browsed with archiving tools.

The text was updated successfully, but these errors were encountered:

probonopd · 2016-05-18T18:55:09Z

Good point. Actually I have tried but never was successful; do you know why the following is not working?

<?xml version="1.0"?>
<mime-info xmlns='http://www.freedesktop.org/standards/shared-mime-info'>
  <mime-type type="application/x-appimage; version=1.0">
    <sub-class-of type="application/x-executable"/>
    <sub-class-of type="application/x-iso9660-image"/>
    <comment>AppImage</comment>
    <magic priority="50">
       <match value="ELF" type="string" offset="1">
       <match value="CD001" type="string" offset="32769">
       <match value="AppImage" type="string" offset="32809"/>
    </magic>
    <magic priority="40">
       <match value="ELF" type="string" offset="1">
       <match value="CD001" type="string" offset="32769">
       <match value="APPIMAGE" type="string" offset="32809"/>
    </magic>
    <glob pattern="*.appimage"/>
    <glob pattern="*.AppImage"/>
  </mime-type>
</mime-info>

sudo xdg-mime install --mode system appimagekit-appimage.xml
xdg-mime query filetype '/home/me/Downloads/test'
# always returns:
# application/x-executable

elvisangelaccio · 2016-05-18T19:30:18Z

@probonopd
Some of the <match> tags are not properly ended. You can check whether the installed XML is ok by running:

sudo update-mime-database /usr/share/mime

I was able to get the expected mimetype after fixing the above issues!

probonopd · 2016-05-18T19:42:52Z

Strange. Changed it to

<?xml version="1.0" encoding="UTF-8"?>
<mime-info xmlns="http://www.freedesktop.org/standards/shared-mime-info">
   <mime-type type="application/x-appimage; version=1.0">
      <sub-class-of type="application/x-executable" />
      <sub-class-of type="application/x-iso9660-image" />
      <comment>AppImage</comment>
      <magic priority="50">
         <match value="ELF" type="string" offset="1" />
         <match value="CD001" type="string" offset="32769" />
         <match value="AppImage" type="string" offset="32809" />
      </magic>
      <magic priority="40">
         <match value="ELF" type="string" offset="1" />
         <match value="CD001" type="string" offset="32769" />
         <match value="APPIMAGE" type="string" offset="32809" />
      </magic>
      <glob pattern="*.appimage" />
      <glob pattern="*.AppImage" />
   </mime-type>
</mime-info>

I get no more syntax complaints but it doesn't seem to work either. Ubuntu 16.04.

elvisangelaccio · 2016-05-18T19:51:17Z

No idea, works for me (even your updated version). I'm on archlinux with shared-mime-info 1.6

Anyway, when you submit a new mime to upstream shared-mime-info, they require you to provide a test file as well, and that is what matters. If the test passes, the mime is gonna be accepted :)

probonopd · 2016-05-18T20:02:13Z

Anyone, we need to iron this out and then submit together with a link to an example AppImage such as https://bintray.com/probono/AppImages/download_file?file_path=Leafpad-0.8.17-x86_64.AppImage according to https://cgit.freedesktop.org/xdg/shared-mime-info/tree/HACKING. I'm a bit lost here.

elvisangelaccio · 2016-05-18T20:04:01Z

I can do it if you want, I recently submitted a mimetype so I have fresh memory of the burocracy :p

probonopd · 2016-05-18T20:04:44Z

That would be great. Before you do please check that it works with and without the suffixes.

probonopd · 2016-05-18T20:05:36Z

...and that the AppImage can still be executed by double-clicking when chmod a+x is set. Worst case would be that it is no longer recognized as an ELF executable in GNOME Nautilus and the likes.

probonopd · 2016-05-18T20:26:47Z

Would making it a subclass of ISO automatically allow tools like Ark or file-roller to process it?

elvisangelaccio · 2016-05-18T20:37:09Z

Would making it a subclass of ISO automatically allow tools like Ark or file-roller to process it?

Not necessarily. I can only speak for Ark, which unfortunately hardcodes the list of supported mimetypes

probonopd · 2016-05-18T20:54:25Z

For whatever reason I had to log out and log in again and now it is working. AppImages still run when they have the executable bit set; however when the executable bit is not set they are tried to be opened with file-roller (which says it cannot handle the file). Is there a way around that?

elvisangelaccio · 2016-05-18T21:00:27Z

Yeah, same here with Dolphin+Ark. I guess they are indeed detected as ISO by the file-managers.

What should be the expected behavior instead? Just nothing happening?

elvisangelaccio · 2016-05-18T21:02:20Z

Another thing, the 2nd and 3rd magic offset are too big and I think the shared-mime-info maintainers will complain about this. Isn't enough to just match the ELF string?

probonopd · 2016-05-18T21:17:04Z

Yeah, same here with Dolphin+Ark. I guess they are indeed detected as ISO by the file-managers. What should be the expected behavior instead? Just nothing happening?

When we don't subclass it to application/x-iso9660-image, then it doesn't try to open the AppImage with file-roller. GNOME then asks "There is no application installed for “AppImage” files. Do you want to search for an application to open this file?".

I am actually not sure which behavior is better. Just found out that when we subclass it to application/x-iso9660-image, then in GNOME I can right-click to open with gnome-disk-image-mounter which conveniently mounts the AppImage. Not bad... I guess we need to test the behavior under multiple *DEs.

probonopd · 2016-05-18T21:18:43Z

Another thing, the 2nd and 3rd magic offset are too big and I think the shared-mime-info maintainers will complain about this. Isn't enough to just match the ELF string?

No, because then we would match every ELF. I guess every ISO is also matched this way?
Perhaps it would be sufficient to treat every ISO that is also an ELF as an AppImage, but I think the 3rd check doesn't really increase the offset by much and it doesn't hurt.

elvisangelaccio · 2016-05-18T21:27:07Z

Ah, now I remember. ISO files don't have any magic (because of too long offset required):

<mime-type type="application/x-cd-image">
    <_comment>raw CD image</_comment>
    <sub-class-of type="application/x-raw-disk-image"/>
    <alias type="application/x-iso9660-image"/>
    <!-- No magic, see https://bugs.freedesktop.org/show_bug.cgi?id=10049 -->
    <glob pattern="*.iso" weight="80"/>
    <glob pattern="*.iso9660"/>
  </mime-type>

There is also this other bug which breaks ISO files detection: https://bugs.freedesktop.org/show_bug.cgi?id=80877

I can try to submit the mime as is, and then let's see what the maintainers think...
In the worst case, we might have to go without magic?

probonopd · 2016-05-18T21:32:31Z

Do you know what is the largest acceptable offset?

elvisangelaccio · 2016-05-18T21:38:53Z

Nope, but I guess something around 2K bytes (the largest offset in the current database seems to be 2112).

elvisangelaccio · 2016-05-18T21:42:45Z

Draft of the patch here: https://gist.github.com/aelog/0db7e74520fe5dbfcb7d7e765e2ef986

probonopd · 2016-05-18T21:55:32Z

Quick experimentation suggests that ELF binaries have "GNU" at offset 608 but they continue to work if I change that to "AppImage". However I am entirely uncertain at this time whether this violates some standard... I guess I need to read up on the ELF format to find a way to sneak in a magic string or number into the ELF header reliably.

elvisangelaccio · 2016-05-18T21:57:43Z

Allright, no hurry from my side. Just ping me here when you think the mime is ready for submission :)

probonopd · 2016-05-19T06:58:24Z

Opened https://bugzilla.gnome.org/show_bug.cgi?id=766644 on file-roller: Process ISO files regardless of the filename extension

shoogle · 2016-06-15T07:30:23Z

@aelog I was thinking about doing this at some point but you beat me to it!
I just have a couple of small suggestions:

Glob matches are case-insensitive, so <glob pattern="*.AppImage" /> will be fine on its own.
Try this for the magic bits:

<magic priority="50">
  <match value="ELF" type="string" offset="1" >
    <match value="CD001" type="string" offset="32769">
      <match value="AppImage" type="string" offset="32809" />
      <match value="APPIMAGE" type="string" offset="32809" />
    </match>
  </match>
</magic>

The nested <match> tags should result in this logic:

Must match "ELF"
AND must match "CD001"
AND must match ("AppImage" OR "APPIMAGE").

shoogle · 2016-06-15T08:13:04Z

Ah, I see that the problem is with the ISO offsets being too large. It should be enough to just do the glob for ".AppImage" in the filename and match for "ELF" in the binary, providing the "ELF" magic is set to an appropriate priority. Looking at the shared-mime-info database I can see that "ELF" is already matched with priority 40 and 50, so I think the appropriate value is either 30 or 60. Somewhat counter-intuitively, I think it needs to be a higher priority to prevent matching against random ELF files, but you'd need to test it for yourself.

@probonopd it won't help detection of existing AppImages, but here are some options for embedding data in an ELF file to create your own magic bits.

elvisangelaccio · 2016-06-15T08:36:19Z

@shoogle I don't mind if you want to submit the mime. I was just waiting to hear from @probonopd about how he plans to solve this magic thing.

probonopd · 2016-06-15T08:54:53Z

Unfortunately it is not solved yet. We need to find a way to sneak in a magic further at the top of the ELF. Is there a way to get a certain magic into the ELF header and still be fully compliant?

0x2b3bfa0 · 2016-06-15T09:34:15Z

Maybe on the .data or .text sections of the ELF. Now I'm traveling, later
I'll try to figure. out how.

El mié., 15 jun. 2016 10:54, probonopd notifications@github.com escribió:

Unfortunately it is not solved yet. We need to find a way to sneak in a
magic further at the top of the ELF. Is there a way to get a certain magic
into the ELF header and still be fully compliant?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#144 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AK3C25N8WIawjwh76UJNQqmZQPytijYuks5qL73fgaJpZM4IhYFr
.

Un saludo.
Helio.

shoogle · 2016-06-15T11:11:36Z

Apparently bytes 8-15 of the header are unused, which gives just enough room to write "AppImage"!

printf 'AppImage' | dd bs=1 seek=8 count=8 conv=notrunc of=Program.AppImage

Which makes for some pretty neat Elf magic! 😄

<magic priority="50">
  <match value="ELF" type="string" offset="1" >
    <match value="AppImage" type="string" offset="8" />
  </match>
</magic>

probonopd · 2017-04-03T16:30:30Z

OK, good to hear @elvisangelaccio. I will continue to investigate. I also have the optional appimaged daemon running on this system.

elvisangelaccio · 2017-04-03T16:33:29Z

Ah wait, I don't think I have appimaged installed, could this make a difference?

probonopd · 2017-04-03T16:36:48Z

Possibly. It sets the +x bit.

probonopd · 2017-04-03T16:38:05Z

On a related note, do you think you could help getting the magic for the type 2 (ELF+squashfs) AppImage format magic in?

elvisangelaccio · 2017-04-03T16:50:26Z

Oh yes, I can give it a try when I have time, shouldn't be harder than the other one :)

elvisangelaccio · 2017-04-06T09:23:41Z

@probonopd I need a type 2 test file (e.g. leafpad, like we did for type 1)

Also, do you like application/vnd.appimage for the mime name? This is for consistency with snap (application/vnd.snap) which also inherits from the squashfs mimetype (application/vnd.squashfs).

probonopd · 2017-04-06T18:17:43Z

Sure, here you go @elvisangelaccio
https://bintray.com/probono/AppImages/download_file?file_path=Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage

elvisangelaccio · 2017-04-07T09:00:37Z

Thanks! https://bugs.freedesktop.org/show_bug.cgi?id=100608

elvisangelaccio · 2017-04-10T09:44:49Z

@probonopd Since both mimetypes have the same glob pattern (*.appimage or *.AppImage), one of them necessarily will fail the "lookup by filename", and applications will need to check the magic value to resolve the ambiguity. Which one of the two appimage formats would you rather be the "default"? (i.e. the output mimetype when looking up an appimage only by filename).

probonopd · 2017-04-10T21:27:01Z

@elvisangelaccio good catch. Magic bytes should always override the filename extension, and the filename extension should point to type 1 (ISO9660) AppImage, since we expect all type 2 (squashfs) AppImages to have the correct magic bytes set.

Still, as the magic bytes should always override the filename extension, a type 2 (squashfs) AppImage with the AppImage filename extension would correctly be identified as type 2.

probonopd · 2017-06-05T22:41:23Z

@elvisangelaccio any news on this? Should we also submit it for the file magic number database (and where is the upstream project for that)?

TheAssassin · 2017-06-05T22:44:05Z

We should also submit our magic numbers here: https://en.wikipedia.org/wiki/List_of_file_signatures

elvisangelaccio · 2017-06-06T08:21:47Z

@probonopd Not yet, I just pinged the freedesktop guy on the bugzilla ticket...

elvisangelaccio · 2017-06-06T08:39:57Z

@probonopd Oh, that was quick. Please let me know what do you think about https://bugs.freedesktop.org/show_bug.cgi?id=100608#c4

TheAssassin · 2017-06-06T10:07:38Z

Looks like you're suggesting vnd.appimage as the MIME type for v2. That doesn't sound like a great idea, since there are going to be future versions of the AppImage specification You should rather introduce some versioned type, e.g. appimage.type2.

The benefit of being able to distinguish between different types of AppImages for end user applications is so they can perform integrations like displaying the icon etc. which is a significantly different operation for different types of AppImages.

probonopd · 2017-06-06T15:23:02Z

Could we not remove the type specific sub-class-of, add application/x-iso9660-appimage as an alias of the more generic application/vnd.appimage, and add the magic to the first type as well?

Type 1 uses ISO9660 whereas type 2 is an entirely different format. They are described in the AppImageSpec.

elvisangelaccio · 2017-06-06T15:40:35Z

What about future versions of the spec (as pointed out by @TheAssassin)? Are we going to need a new mimetype every time?

probonopd · 2017-06-06T16:48:27Z

Yes, but we are not foreseeing changes to the spec that would require a type 3 for a very long time, since the type 2 spec is quite generic.

elvisangelaccio · 2017-09-07T18:51:58Z

@probonopd It's done! https://cgit.freedesktop.org/xdg/shared-mime-info/commit/?id=c643cab25b8a4ea17e73eae5bc318c840f0e3d4b

probonopd · 2017-09-08T05:48:29Z

Thank you @elvisangelaccio.

TheAssassin · 2017-11-23T15:35:00Z

Thanks @elvisangelaccio. I think I can close it safely now.

probonopd · 2018-07-19T17:06:44Z

What does "vnd" mean?

Can we explain what is going on here?

TheAssassin · 2018-07-19T17:48:05Z

@probonopd vnd seems to be used by the XDG folks for all "containerized" applications.

probonopd · 2018-07-19T19:04:19Z

ZOMG. AppImages are not "containerized", are they?

TheAssassin · 2018-07-19T19:07:40Z

Did you even bother looking this up on the Internet using a search engine?

https://www.startpage.com/do/dsearch?query=mime+vnd&cat=web&pl=opensearch

https://stackoverflow.com/questions/5351093/what-is-the-meaning-of-vnd-in-mime-types

vnd -> vendor specific

probonopd · 2018-07-19T20:36:33Z

vnd indicates vendor-specific MIME types, which means they are MIME types that were introduced by corporate bodies rather than e.g. an Internet consortium.

Heh, so we are a "corporate body" now. Well, we are not an "Internet consortium" I guess.

probonopd changed the title ~~Dedicated mimetype for .appimage files?~~ Submit AppImage MIME Type to xdg/shared-mime-info May 18, 2016

estan mentioned this issue Jun 4, 2016

Clarify file suffix requirements for AppImages #153

Closed

TheAssassin added the enhancement label Jun 5, 2017

TheAssassin closed this as completed Nov 23, 2017

probonopd mentioned this issue Mar 24, 2019

Where to embed AppImage magic bytes in the future? AppImage/AppImageSpec#26

Open

Kagami mentioned this issue Oct 23, 2023

Cannot run foreign architecture AppImages directly with binfmt_misc support #965

Open

Submit AppImage MIME Type to xdg/shared-mime-info #144

Submit AppImage MIME Type to xdg/shared-mime-info #144

Comments

elvisangelaccio commented May 18, 2016

probonopd commented May 18, 2016 • edited Loading

elvisangelaccio commented May 18, 2016 • edited Loading

probonopd commented May 18, 2016

elvisangelaccio commented May 18, 2016

probonopd commented May 18, 2016 • edited Loading

elvisangelaccio commented May 18, 2016

probonopd commented May 18, 2016

probonopd commented May 18, 2016 • edited Loading

probonopd commented May 18, 2016

elvisangelaccio commented May 18, 2016

probonopd commented May 18, 2016 • edited Loading

elvisangelaccio commented May 18, 2016

elvisangelaccio commented May 18, 2016 • edited Loading

probonopd commented May 18, 2016

probonopd commented May 18, 2016 • edited Loading

elvisangelaccio commented May 18, 2016

probonopd commented May 18, 2016

elvisangelaccio commented May 18, 2016 • edited Loading

elvisangelaccio commented May 18, 2016

probonopd commented May 18, 2016

elvisangelaccio commented May 18, 2016

probonopd commented May 19, 2016

shoogle commented Jun 15, 2016

shoogle commented Jun 15, 2016 • edited Loading

elvisangelaccio commented Jun 15, 2016

probonopd commented Jun 15, 2016

0x2b3bfa0 commented Jun 15, 2016

shoogle commented Jun 15, 2016

probonopd commented Apr 3, 2017

elvisangelaccio commented Apr 3, 2017

probonopd commented Apr 3, 2017

probonopd commented Apr 3, 2017

elvisangelaccio commented Apr 3, 2017

elvisangelaccio commented Apr 6, 2017 • edited Loading

probonopd commented Apr 6, 2017

elvisangelaccio commented Apr 7, 2017

elvisangelaccio commented Apr 10, 2017 • edited Loading

probonopd commented Apr 10, 2017 • edited Loading

probonopd commented Jun 5, 2017 • edited Loading

TheAssassin commented Jun 5, 2017

elvisangelaccio commented Jun 6, 2017

elvisangelaccio commented Jun 6, 2017

TheAssassin commented Jun 6, 2017

probonopd commented Jun 6, 2017

elvisangelaccio commented Jun 6, 2017

probonopd commented Jun 6, 2017

elvisangelaccio commented Sep 7, 2017

probonopd commented Sep 8, 2017

TheAssassin commented Nov 23, 2017

probonopd commented Jul 19, 2018

TheAssassin commented Jul 19, 2018

probonopd commented Jul 19, 2018

TheAssassin commented Jul 19, 2018

probonopd commented Jul 19, 2018

probonopd commented May 18, 2016 •

edited

Loading

elvisangelaccio commented May 18, 2016 •

edited

Loading

probonopd commented May 18, 2016 •

edited

Loading

probonopd commented May 18, 2016 •

edited

Loading

probonopd commented May 18, 2016 •

edited

Loading

elvisangelaccio commented May 18, 2016 •

edited

Loading

probonopd commented May 18, 2016 •

edited

Loading

elvisangelaccio commented May 18, 2016 •

edited

Loading

shoogle commented Jun 15, 2016 •

edited

Loading

elvisangelaccio commented Apr 6, 2017 •

edited

Loading

elvisangelaccio commented Apr 10, 2017 •

edited

Loading

probonopd commented Apr 10, 2017 •

edited

Loading

probonopd commented Jun 5, 2017 •

edited

Loading