Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Sony RAW (.ARW) #3612

Closed
uhthomas opened this issue Apr 3, 2023 · 19 comments
Closed

Support Sony RAW (.ARW) #3612

uhthomas opened this issue Apr 3, 2023 · 19 comments
Labels

Comments

@uhthomas
Copy link
Contributor

uhthomas commented Apr 3, 2023

Feature request

What are you trying to achieve?

I'd like to use Immich with Sony RAW, which uses sharp to process media.

immich-app/immich#2156

Related issues link to libvips/libvips#1304 and were closed, but it seems that it's possible to use magickload?

When you searched for similar feature requests, what did you find that might be related?

#1745

#2867

What would you expect the API to look like?

?

What alternatives have you considered?

N/A

Please provide sample image(s) that help explain this feature

N/A

@lovell
Copy link
Owner

lovell commented Apr 3, 2023

Are you using a custom libvips compiled with support for ImageMagick, that itself has been compiled with support for the specific RAW format you need?

If you're not using a custom libvips then the suggestions in #2867 are worth investigating, perhaps https://github.com/justinkambic/libraw.js will do what you need.

@lovell
Copy link
Owner

lovell commented Apr 27, 2023

@uhthomas Were you able to make any progress with this?

@lovell
Copy link
Owner

lovell commented May 11, 2023

Closing due to inactivity but please feel free to reopen with more details if further help is required.

@lovell lovell closed this as completed May 11, 2023
@uhthomas
Copy link
Contributor Author

Sorry for the lack of activity. It looks like there is a draft PR for immich to support ARW.

immich-app/immich#2198

The PR seems to be using the darktable cli.

Is it possible for sharp to support raw files directly @lovell?

@lovell
Copy link
Owner

lovell commented May 11, 2023

You'll need to use a globally-installed libvips compiled with support for ImageMagick.

It looks like Immich is built on Alpine Linux, so could switch from vips to vips-magick here:
https://github.com/immich-app/immich/blob/d25ddfc46bf059471a89f51c73f51617b6c9ccaf/server/Dockerfile#L26

@uhthomas
Copy link
Contributor Author

You'll need to use a globally-installed libvips compiled with support for ImageMagick.

It looks like Immich is built on Alpine Linux, so could switch from vips to vips-magick here: https://github.com/immich-app/immich/blob/d25ddfc46bf059471a89f51c73f51617b6c9ccaf/server/Dockerfile#L26

This sounds great, thank you so much for the help. Assuming this is installed, are any code changes required or should it "just work"?

@uhthomas
Copy link
Contributor Author

uhthomas commented Jun 5, 2023

@lovell I've installed vips-magick and made sure to update to the new version of imagemagick which includes support for libraw, and consequently sony raw files.

I can see this works as expected with vips-tools, but not sharp.

/usr/src/app/upload/library/admin/2021/2021-07-22 # vips thumbnail _DSC2642.arw test.jpg 100

(vips:94): VIPS-WARNING **: 12:38:01.299: Unknown field with tag 50341 (0xc4a5) encountered

(vips:94): VIPS-WARNING **: 12:38:01.384: Unknown field with tag 50341 (0xc4a5) encountered

(vips:94): VIPS-WARNING **: 12:38:02.791: Unknown field with tag 50341 (0xc4a5) encountered

error from sharp:

immich_microservices     | [Nest] 40  - 06/05/2023, 12:31:56 PM    WARN [MediaService] Failed to generate jpeg thumbnail using sharp, trying with exiftool-vendored (asset=03a24205-0e0e-48c9-abfc-5c00463827e5): Input file contains unsupported image format

Any ideas on what could be happening?

@uhthomas
Copy link
Contributor Author

uhthomas commented Jun 5, 2023

Looks like the error comes from here.

throw vips::VError("Input file contains unsupported image format");

@uhthomas
Copy link
Contributor Author

uhthomas commented Jun 5, 2023

Again, I can see imagemagick is happy and vips is also pretty happy. Seems strange for the C library not to work.

/usr/src/app/upload/library/admin/2021/2021-07-22 # vipsheader -a _DSC2642.arw

(vipsheader:129): VIPS-WARNING **: 12:41:27.105: Unknown field with tag 50341 (0xc4a5) encountered
_DSC2642.arw: 9568x6376 ushort, 3 bands, rgb16, magickload
width: 9568
height: 6376
bands: 3
format: ushort
coding: none
interpretation: rgb16
xoffset: 0
yoffset: 0
xres: 1
yres: 1
filename: _DSC2642.arw
vips-loader: magickload
magick-date:create: 2023-06-05T12:18:20+00:00
magick-date:modify: 2023-06-05T12:18:20+00:00
magick-date:timestamp: 2023-06-05T12:41:27+00:00
magick-dng:camera.model.name: ILCE-7RM4
magick-dng:create.date: 2021-07-22T18:01:26+00:00
magick-dng:exposure.time: 1/8000
magick-dng:f.number: 1.4
magick-dng:focal.length: 35.0 mm
magick-dng:focal.length.in.35mm.format: 35 mm
magick-dng:gps.altitude: 0.0 m
magick-dng:gps.latitude: 0 deg 0' 0.00" N
magick-dng:gps.longitude: 0 deg 0' 0.00" W
magick-dng:iso.setting: 100
magick-dng:lens: 35.0-0.0mm f/1.4-0.0
magick-dng:lens.f.stops: 0.00
magick-dng:lens.type:
magick-dng:make: Sony
magick-dng:max.aperture.at.max.focal: 0.0
magick-dng:max.aperture.at.min.focal: 1.4
magick-dng:max.aperture.value: 1.4
magick-dng:max.focal.length: 0.0 mm
magick-dng:min.focal.length: 35.0 mm
magick-dng:serial.number: 05074736
magick-dng:software: ILCE-7RM4 v1.20
magick-dng:wb.rb.levels: 2464.000000 1796.000000 1024.000000 1024.000000
magick-xmp:Rating: 0
xmp-data: 4097 bytes of binary data
magick-format: ARW
n-pages: 1
orientation: 1

@uhthomas
Copy link
Contributor Author

uhthomas commented Jun 5, 2023

Is it possible that libvips is not detecting imagemagick correctly? Seems odd given I have these following packages installed:

apk add --no-cache vips vips-cpp vips-heif vips-magick libraw-dev ffmpeg perl imagemagick vips vips-tools

#3346 (comment)

@lovell
Copy link
Owner

lovell commented Jun 5, 2023

Did you see sharp.format? This provides runtime information about available formats.

When using sharp with a globally-installed libvips, please check the output of npm install --verbose --foreground-scripts sharp to ensure it has been detected.

@uhthomas
Copy link
Contributor Author

uhthomas commented Jun 5, 2023

I'm not really sure what I should be looking for from the requested info. I hope it's helpful though.

immich_server            | {
immich_server            |   jpeg: {
immich_server            |     id: 'jpeg',
immich_server            |     input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server            |     output: { file: true, buffer: true, stream: true, alias: [Array] }
immich_server            |   },
immich_server            |   png: {
immich_server            |     id: 'png',
immich_server            |     input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server            |     output: { file: true, buffer: true, stream: true }
immich_server            |   },
immich_server            |   webp: {
immich_server            |     id: 'webp',
immich_server            |     input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server            |     output: { file: true, buffer: true, stream: true }
immich_server            |   },
immich_server            |   tiff: {
immich_server            |     id: 'tiff',
immich_server            |     input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server            |     output: { file: true, buffer: true, stream: true, alias: [Array] }
immich_server            |   },
immich_server            |   magick: {
immich_server            |     id: 'magick',
immich_server            |     input: { file: false, buffer: false, stream: false },
immich_server            |     output: { file: false, buffer: false, stream: false }
immich_server            |   },
immich_server            |   openslide: {
immich_server            |     id: 'openslide',
immich_server            |     input: { file: false, buffer: false, stream: false },
immich_server            |     output: { file: false, buffer: false, stream: false }
immich_server            |   },
immich_server            |   dz: {
immich_server            |     id: 'dz',
immich_server            |     input: { file: false, buffer: false, stream: false },
immich_server            |     output: { file: true, buffer: true, stream: true }
immich_server            |   },
immich_server            |   ppm: {
immich_server            |     id: 'ppm',
immich_server            |     input: { file: false, buffer: false, stream: false },
immich_server            |     output: { file: false, buffer: false, stream: false }
immich_server            |   },
immich_server            |   fits: {
immich_server            |     id: 'fits',
immich_server            |     input: { file: false, buffer: false, stream: false },
immich_server            |     output: { file: false, buffer: false, stream: false }
immich_server            |   },
immich_server            |   gif: {
immich_server            |     id: 'gif',
immich_server            |     input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server            |     output: { file: true, buffer: true, stream: true }
immich_server            |   },
immich_server            |   svg: {
immich_server            |     id: 'svg',
immich_server            |     input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server            |     output: { file: false, buffer: false, stream: false }
immich_server            |   },
immich_server            |   heif: {
immich_server            |     id: 'heif',
immich_server            |     input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server            |     output: { file: true, buffer: true, stream: true, alias: [Array] }
immich_server            |   },
immich_server            |   pdf: {
immich_server            |     id: 'pdf',
immich_server            |     input: { file: false, buffer: false, stream: false },
immich_server            |     output: { file: false, buffer: false, stream: false }
immich_server            |   },
immich_server            |   vips: {
immich_server            |     id: 'vips',
immich_server            |     input: { file: true, buffer: false, stream: false, fileSuffix: [Array] },
immich_server            |     output: { file: true, buffer: false, stream: false }
immich_server            |   },
immich_server            |   jp2k: {
immich_server            |     id: 'jp2k',
immich_server            |     input: { file: false, buffer: false, stream: false },
immich_server            |     output: { file: false, buffer: false, stream: false, alias: [Array] }
immich_server            |   },
immich_server            |   jxl: {
immich_server            |     id: 'jxl',
immich_server            |     input: { file: false, buffer: false, stream: false },
immich_server            |     output: { file: false, buffer: false, stream: false }
immich_server            |   },
immich_server            |   raw: {
immich_server            |     id: 'raw',
immich_server            |     input: { file: false, buffer: true, stream: true },
immich_server            |     output: { file: false, buffer: true, stream: true }
immich_server            |   }
immich_server            | 
/usr/src/app # npm install --verbose --foreground-scripts sharp
npm verb cli /usr/local/bin/node /usr/local/bin/npm
npm info using npm@9.5.1
npm info using node@v18.16.0
npm verb title npm install sharp
npm verb argv "install" "--loglevel" "verbose" "--foreground-scripts" "sharp"
npm verb logfile logs-max:10 dir:/root/.npm/_logs/2023-06-05T14_35_08_361Z-
npm verb logfile /root/.npm/_logs/2023-06-05T14_35_08_361Z-debug-0.log
npm http fetch GET 200 https://registry.npmjs.org/sharp 357ms (cache hit)
npm verb reify failed optional dependency /usr/src/app/node_modules/fsevents
npm verb reify failed optional dependency /usr/src/app/node_modules/exiftool-vendored.exe
npm verb reify failed optional dependency /usr/src/app/node_modules/@msgpackr-extract/msgpackr-extract-win32-x64
npm verb reify failed optional dependency /usr/src/app/node_modules/@msgpackr-extract/msgpackr-extract-linux-arm64
npm verb reify failed optional dependency /usr/src/app/node_modules/@msgpackr-extract/msgpackr-extract-linux-arm
npm verb reify failed optional dependency /usr/src/app/node_modules/@msgpackr-extract/msgpackr-extract-darwin-x64
npm verb reify failed optional dependency /usr/src/app/node_modules/@msgpackr-extract/msgpackr-extract-darwin-arm64
npm http fetch POST 200 https://registry.npmjs.org/-/npm/v1/security/advisories/bulk 648ms
npm http fetch GET 200 https://registry.npmjs.org/@nestjs%2fcore 17ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/engine.io 26ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/socket.io-parser 29ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/request 30ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/@nestjs%2fplatform-express 10ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/@openapitools%2fopenapi-generator-cli 19ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/@nestjs%2fwebsockets 28ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/socket.io 3ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/local-reverse-geocoder 2ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/@nestjs%2fplatform-socket.io 5ms (cache hit)

up to date, audited 1025 packages in 2s

120 packages are looking for funding
  run `npm fund` for details

10 vulnerabilities (8 moderate, 2 high)

To address all issues possible, run:
  npm audit fix --force

Some issues need review, and may require choosing
a different dependency.

Run `npm audit` for details.
npm verb exit 0
npm info ok

@lovell
Copy link
Owner

lovell commented Jun 5, 2023

immich_server | magick: {
immich_server | id: 'magick',
immich_server | input: { file: false, buffer: false, stream: false },

This tells me that *magick support is not available at runtime.

Perhaps run npm install --verbose --foreground-scripts sharp in a new, empty directory that is nowhere near an existing node_modules directory or package.json file, e.g. /tmp.

@uhthomas
Copy link
Contributor Author

I installed sharp in /tmp and I don't think it changed much?

/usr/src/app # cd /tmp
/tmp # npm install --verbose --foreground-scripts sharp
npm verb cli /usr/local/bin/node /usr/local/bin/npm
npm info using npm@9.5.1
npm info using node@v18.16.0
npm verb title npm install sharp
npm verb argv "install" "--loglevel" "verbose" "--foreground-scripts" "sharp"
npm verb logfile logs-max:10 dir:/root/.npm/_logs/2023-06-12T14_30_34_457Z-
npm verb logfile /root/.npm/_logs/2023-06-12T14_30_34_457Z-debug-0.log
npm http fetch GET 200 https://registry.npmjs.org/sharp 323ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/color 78ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/node-addon-api 127ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/tar-fs 161ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/detect-libc 177ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/tunnel-agent 174ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/simple-get 183ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/prebuild-install 194ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/semver 208ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/pump 93ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/napi-build-utils 100ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/color-string 108ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/expand-template 107ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/minimist 110ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/color-convert 116ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/tar-stream 153ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/simple-concat 170ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/safe-buffer 167ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/once 176ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/chownr 172ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/decompress-response 179ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/lru-cache 196ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/github-from-package 267ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/rc 311ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/node-abi 335ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/mkdirp-classic 779ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/mkdirp-classic 790ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/color-name 65ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/simple-swizzle 65ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/color-name 70ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/is-arrayish 56ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/end-of-stream 86ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/ini 89ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/deep-extend 162ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/strip-json-comments 161ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/wrappy 63ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/yallist 57ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/mimic-response 84ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/bl 101ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/readable-stream 99ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/inherits 204ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/fs-constants 712ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/util-deprecate 73ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/buffer 91ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/string_decoder 171ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/ieee754 81ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/base64-js 83ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/node-addon-api/-/node-addon-api-6.1.0.tgz 97ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/semver/-/semver-7.5.1.tgz 102ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/node-abi/-/node-abi-3.45.0.tgz 123ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/sharp/-/sharp-0.32.1.tgz 137ms (cache miss)
npm info run sharp@0.32.1 install node_modules/sharp (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)

> sharp@0.32.1 install
> (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)

sharp: Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.14.2/libvips-8.14.2-linuxmusl-x64.tar.brll/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
sharp: Integrity check passed for linuxmusl-x64arp@0.32.1 install node_modules/sharp (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
prebuild-install info begin Prebuild-install version 7.1.1install node_modules/sharp (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
prebuild-install info looking for local prebuild @ prebuilds/sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz
prebuild-install info looking for cached prebuild @ /root/.npm/_prebuilds/564a4a-sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz
prebuild-install http request GET https://github.com/lovell/sharp/releases/download/v0.32.1/sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz
prebuild-install http 200 https://github.com/lovell/sharp/releases/download/v0.32.1/sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
prebuild-install info downloading to @ /root/.npm/_prebuilds/564a4a-sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz.102-57bfe0507e169.tmp
prebuild-install info renaming to @ /root/.npm/_prebuilds/564a4a-sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz&& node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
prebuild-install info unpacking @ /root/.npm/_prebuilds/564a4a-sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz
prebuild-install info unpack resolved to /tmp/node_modules/sharp/build/Release/sharp-linuxmusl-x64.node
prebuild-install info install Successfully installed prebuilt binary!
npm info run sharp@0.32.1 install { code: 0, signal: null }

added 45 packages in 5s

10 packages are looking for funding
  run `npm fund` for details
npm verb exit 0
npm info ok

@lovell
Copy link
Owner

lovell commented Jun 12, 2023

sharp: Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.14.2/libvips-8.14.2-linuxmusl-x64.tar.br

sharp is unable to detect the globally-installed libvips. Have you installed the vips-dev package?

@uhthomas
Copy link
Contributor Author

omg you solved it 😭

I created this minimal reproduction repository to work through why it wasn't working and simply adding vips-dev fixed it. I hope the repo is helpful in future as it should be an easy general purpose way to print diagnostic information on sharp runtime formats.

https://github.com/uhthomas/sharp3612

Thank you so much. I'll look at adding Sony ARW support to immich right now. You're the best!

@uhthomas
Copy link
Contributor Author

Just to double check, vips-cpp isn't required by sharp right?

@lovell
Copy link
Owner

lovell commented Jun 12, 2023

It is required but you don't have to explicitly install it as vips-cpp is already (correctly) a dependency of vips-dev.

@uhthomas
Copy link
Contributor Author

Makes sense, thank you. Are any other packages other than vips-magick required for it to be fully functional do you know? Sharp says the format magick is supported, but it fails to actually do anything with a .ARW image.

immich_server            |   magick: {
immich_server            |     id: 'magick',
immich_server            |     input: { file: true, buffer: true, stream: true },
immich_server            |     output: { file: true, buffer: true, stream: true }
immich_server            |   },
immich_microservices     | [Nest] 29  - 06/12/2023, 7:09:29 PM    WARN [MediaService] Failed to generate jpeg thumbnail using sharp, trying with exiftool-vendored (asset=6b3fb2d6-59ba-465e-809d-e319139b6127): Input file contains unsupported image format

Ignore the above, but thought it was helpful for context. Installing imagemagick-dev seems to have fixed it. Awesome!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants