-
-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support Sony RAW (.ARW) #3612
Comments
Are you using a custom libvips compiled with support for ImageMagick, that itself has been compiled with support for the specific RAW format you need? If you're not using a custom libvips then the suggestions in #2867 are worth investigating, perhaps https://github.com/justinkambic/libraw.js will do what you need. |
@uhthomas Were you able to make any progress with this? |
Closing due to inactivity but please feel free to reopen with more details if further help is required. |
You'll need to use a globally-installed libvips compiled with support for ImageMagick. It looks like Immich is built on Alpine Linux, so could switch from |
This sounds great, thank you so much for the help. Assuming this is installed, are any code changes required or should it "just work"? |
@lovell I've installed I can see this works as expected with vips-tools, but not sharp. /usr/src/app/upload/library/admin/2021/2021-07-22 # vips thumbnail _DSC2642.arw test.jpg 100
(vips:94): VIPS-WARNING **: 12:38:01.299: Unknown field with tag 50341 (0xc4a5) encountered
(vips:94): VIPS-WARNING **: 12:38:01.384: Unknown field with tag 50341 (0xc4a5) encountered
(vips:94): VIPS-WARNING **: 12:38:02.791: Unknown field with tag 50341 (0xc4a5) encountered error from sharp: immich_microservices | [Nest] 40 - 06/05/2023, 12:31:56 PM WARN [MediaService] Failed to generate jpeg thumbnail using sharp, trying with exiftool-vendored (asset=03a24205-0e0e-48c9-abfc-5c00463827e5): Input file contains unsupported image format Any ideas on what could be happening? |
Looks like the error comes from here. Line 516 in 93fafb0
|
Again, I can see imagemagick is happy and vips is also pretty happy. Seems strange for the C library not to work. /usr/src/app/upload/library/admin/2021/2021-07-22 # vipsheader -a _DSC2642.arw
(vipsheader:129): VIPS-WARNING **: 12:41:27.105: Unknown field with tag 50341 (0xc4a5) encountered
_DSC2642.arw: 9568x6376 ushort, 3 bands, rgb16, magickload
width: 9568
height: 6376
bands: 3
format: ushort
coding: none
interpretation: rgb16
xoffset: 0
yoffset: 0
xres: 1
yres: 1
filename: _DSC2642.arw
vips-loader: magickload
magick-date:create: 2023-06-05T12:18:20+00:00
magick-date:modify: 2023-06-05T12:18:20+00:00
magick-date:timestamp: 2023-06-05T12:41:27+00:00
magick-dng:camera.model.name: ILCE-7RM4
magick-dng:create.date: 2021-07-22T18:01:26+00:00
magick-dng:exposure.time: 1/8000
magick-dng:f.number: 1.4
magick-dng:focal.length: 35.0 mm
magick-dng:focal.length.in.35mm.format: 35 mm
magick-dng:gps.altitude: 0.0 m
magick-dng:gps.latitude: 0 deg 0' 0.00" N
magick-dng:gps.longitude: 0 deg 0' 0.00" W
magick-dng:iso.setting: 100
magick-dng:lens: 35.0-0.0mm f/1.4-0.0
magick-dng:lens.f.stops: 0.00
magick-dng:lens.type:
magick-dng:make: Sony
magick-dng:max.aperture.at.max.focal: 0.0
magick-dng:max.aperture.at.min.focal: 1.4
magick-dng:max.aperture.value: 1.4
magick-dng:max.focal.length: 0.0 mm
magick-dng:min.focal.length: 35.0 mm
magick-dng:serial.number: 05074736
magick-dng:software: ILCE-7RM4 v1.20
magick-dng:wb.rb.levels: 2464.000000 1796.000000 1024.000000 1024.000000
magick-xmp:Rating: 0
xmp-data: 4097 bytes of binary data
magick-format: ARW
n-pages: 1
orientation: 1 |
Is it possible that libvips is not detecting imagemagick correctly? Seems odd given I have these following packages installed: apk add --no-cache vips vips-cpp vips-heif vips-magick libraw-dev ffmpeg perl imagemagick vips vips-tools |
Did you see sharp.format? This provides runtime information about available formats. When using sharp with a globally-installed libvips, please check the output of |
I'm not really sure what I should be looking for from the requested info. I hope it's helpful though. immich_server | {
immich_server | jpeg: {
immich_server | id: 'jpeg',
immich_server | input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server | output: { file: true, buffer: true, stream: true, alias: [Array] }
immich_server | },
immich_server | png: {
immich_server | id: 'png',
immich_server | input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server | output: { file: true, buffer: true, stream: true }
immich_server | },
immich_server | webp: {
immich_server | id: 'webp',
immich_server | input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server | output: { file: true, buffer: true, stream: true }
immich_server | },
immich_server | tiff: {
immich_server | id: 'tiff',
immich_server | input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server | output: { file: true, buffer: true, stream: true, alias: [Array] }
immich_server | },
immich_server | magick: {
immich_server | id: 'magick',
immich_server | input: { file: false, buffer: false, stream: false },
immich_server | output: { file: false, buffer: false, stream: false }
immich_server | },
immich_server | openslide: {
immich_server | id: 'openslide',
immich_server | input: { file: false, buffer: false, stream: false },
immich_server | output: { file: false, buffer: false, stream: false }
immich_server | },
immich_server | dz: {
immich_server | id: 'dz',
immich_server | input: { file: false, buffer: false, stream: false },
immich_server | output: { file: true, buffer: true, stream: true }
immich_server | },
immich_server | ppm: {
immich_server | id: 'ppm',
immich_server | input: { file: false, buffer: false, stream: false },
immich_server | output: { file: false, buffer: false, stream: false }
immich_server | },
immich_server | fits: {
immich_server | id: 'fits',
immich_server | input: { file: false, buffer: false, stream: false },
immich_server | output: { file: false, buffer: false, stream: false }
immich_server | },
immich_server | gif: {
immich_server | id: 'gif',
immich_server | input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server | output: { file: true, buffer: true, stream: true }
immich_server | },
immich_server | svg: {
immich_server | id: 'svg',
immich_server | input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server | output: { file: false, buffer: false, stream: false }
immich_server | },
immich_server | heif: {
immich_server | id: 'heif',
immich_server | input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server | output: { file: true, buffer: true, stream: true, alias: [Array] }
immich_server | },
immich_server | pdf: {
immich_server | id: 'pdf',
immich_server | input: { file: false, buffer: false, stream: false },
immich_server | output: { file: false, buffer: false, stream: false }
immich_server | },
immich_server | vips: {
immich_server | id: 'vips',
immich_server | input: { file: true, buffer: false, stream: false, fileSuffix: [Array] },
immich_server | output: { file: true, buffer: false, stream: false }
immich_server | },
immich_server | jp2k: {
immich_server | id: 'jp2k',
immich_server | input: { file: false, buffer: false, stream: false },
immich_server | output: { file: false, buffer: false, stream: false, alias: [Array] }
immich_server | },
immich_server | jxl: {
immich_server | id: 'jxl',
immich_server | input: { file: false, buffer: false, stream: false },
immich_server | output: { file: false, buffer: false, stream: false }
immich_server | },
immich_server | raw: {
immich_server | id: 'raw',
immich_server | input: { file: false, buffer: true, stream: true },
immich_server | output: { file: false, buffer: true, stream: true }
immich_server | }
immich_server | /usr/src/app # npm install --verbose --foreground-scripts sharp
npm verb cli /usr/local/bin/node /usr/local/bin/npm
npm info using npm@9.5.1
npm info using node@v18.16.0
npm verb title npm install sharp
npm verb argv "install" "--loglevel" "verbose" "--foreground-scripts" "sharp"
npm verb logfile logs-max:10 dir:/root/.npm/_logs/2023-06-05T14_35_08_361Z-
npm verb logfile /root/.npm/_logs/2023-06-05T14_35_08_361Z-debug-0.log
npm http fetch GET 200 https://registry.npmjs.org/sharp 357ms (cache hit)
npm verb reify failed optional dependency /usr/src/app/node_modules/fsevents
npm verb reify failed optional dependency /usr/src/app/node_modules/exiftool-vendored.exe
npm verb reify failed optional dependency /usr/src/app/node_modules/@msgpackr-extract/msgpackr-extract-win32-x64
npm verb reify failed optional dependency /usr/src/app/node_modules/@msgpackr-extract/msgpackr-extract-linux-arm64
npm verb reify failed optional dependency /usr/src/app/node_modules/@msgpackr-extract/msgpackr-extract-linux-arm
npm verb reify failed optional dependency /usr/src/app/node_modules/@msgpackr-extract/msgpackr-extract-darwin-x64
npm verb reify failed optional dependency /usr/src/app/node_modules/@msgpackr-extract/msgpackr-extract-darwin-arm64
npm http fetch POST 200 https://registry.npmjs.org/-/npm/v1/security/advisories/bulk 648ms
npm http fetch GET 200 https://registry.npmjs.org/@nestjs%2fcore 17ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/engine.io 26ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/socket.io-parser 29ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/request 30ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/@nestjs%2fplatform-express 10ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/@openapitools%2fopenapi-generator-cli 19ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/@nestjs%2fwebsockets 28ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/socket.io 3ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/local-reverse-geocoder 2ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/@nestjs%2fplatform-socket.io 5ms (cache hit)
up to date, audited 1025 packages in 2s
120 packages are looking for funding
run `npm fund` for details
10 vulnerabilities (8 moderate, 2 high)
To address all issues possible, run:
npm audit fix --force
Some issues need review, and may require choosing
a different dependency.
Run `npm audit` for details.
npm verb exit 0
npm info ok |
This tells me that *magick support is not available at runtime. Perhaps run |
I installed sharp in /usr/src/app # cd /tmp
/tmp # npm install --verbose --foreground-scripts sharp
npm verb cli /usr/local/bin/node /usr/local/bin/npm
npm info using npm@9.5.1
npm info using node@v18.16.0
npm verb title npm install sharp
npm verb argv "install" "--loglevel" "verbose" "--foreground-scripts" "sharp"
npm verb logfile logs-max:10 dir:/root/.npm/_logs/2023-06-12T14_30_34_457Z-
npm verb logfile /root/.npm/_logs/2023-06-12T14_30_34_457Z-debug-0.log
npm http fetch GET 200 https://registry.npmjs.org/sharp 323ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/color 78ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/node-addon-api 127ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/tar-fs 161ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/detect-libc 177ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/tunnel-agent 174ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/simple-get 183ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/prebuild-install 194ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/semver 208ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/pump 93ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/napi-build-utils 100ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/color-string 108ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/expand-template 107ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/minimist 110ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/color-convert 116ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/tar-stream 153ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/simple-concat 170ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/safe-buffer 167ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/once 176ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/chownr 172ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/decompress-response 179ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/lru-cache 196ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/github-from-package 267ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/rc 311ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/node-abi 335ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/mkdirp-classic 779ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/mkdirp-classic 790ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/color-name 65ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/simple-swizzle 65ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/color-name 70ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/is-arrayish 56ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/end-of-stream 86ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/ini 89ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/deep-extend 162ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/strip-json-comments 161ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/wrappy 63ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/yallist 57ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/mimic-response 84ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/bl 101ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/readable-stream 99ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/inherits 204ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/fs-constants 712ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/util-deprecate 73ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/buffer 91ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/string_decoder 171ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/ieee754 81ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/base64-js 83ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/node-addon-api/-/node-addon-api-6.1.0.tgz 97ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/semver/-/semver-7.5.1.tgz 102ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/node-abi/-/node-abi-3.45.0.tgz 123ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/sharp/-/sharp-0.32.1.tgz 137ms (cache miss)
npm info run sharp@0.32.1 install node_modules/sharp (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
> sharp@0.32.1 install
> (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
sharp: Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.14.2/libvips-8.14.2-linuxmusl-x64.tar.brll/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
sharp: Integrity check passed for linuxmusl-x64arp@0.32.1 install node_modules/sharp (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
prebuild-install info begin Prebuild-install version 7.1.1install node_modules/sharp (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
prebuild-install info looking for local prebuild @ prebuilds/sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz
prebuild-install info looking for cached prebuild @ /root/.npm/_prebuilds/564a4a-sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz
prebuild-install http request GET https://github.com/lovell/sharp/releases/download/v0.32.1/sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz
prebuild-install http 200 https://github.com/lovell/sharp/releases/download/v0.32.1/sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
prebuild-install info downloading to @ /root/.npm/_prebuilds/564a4a-sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz.102-57bfe0507e169.tmp
prebuild-install info renaming to @ /root/.npm/_prebuilds/564a4a-sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz&& node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
prebuild-install info unpacking @ /root/.npm/_prebuilds/564a4a-sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz
prebuild-install info unpack resolved to /tmp/node_modules/sharp/build/Release/sharp-linuxmusl-x64.node
prebuild-install info install Successfully installed prebuilt binary!
npm info run sharp@0.32.1 install { code: 0, signal: null }
added 45 packages in 5s
10 packages are looking for funding
run `npm fund` for details
npm verb exit 0
npm info ok |
sharp is unable to detect the globally-installed libvips. Have you installed the |
omg you solved it 😭 I created this minimal reproduction repository to work through why it wasn't working and simply adding https://github.com/uhthomas/sharp3612 Thank you so much. I'll look at adding Sony ARW support to immich right now. You're the best! |
Just to double check, |
It is required but you don't have to explicitly install it as |
Makes sense, thank you. Are any other packages other than
Ignore the above, but thought it was helpful for context. Installing |
Feature request
What are you trying to achieve?
I'd like to use Immich with Sony RAW, which uses sharp to process media.
immich-app/immich#2156
Related issues link to libvips/libvips#1304 and were closed, but it seems that it's possible to use
magickload
?When you searched for similar feature requests, what did you find that might be related?
#1745
#2867
What would you expect the API to look like?
?
What alternatives have you considered?
N/A
Please provide sample image(s) that help explain this feature
N/A
The text was updated successfully, but these errors were encountered: