Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dcrawMS.exe breaks scanning the library and makes DB huge in size + JPEG ISO exception #554

Closed
ExSport opened this issue May 5, 2015 · 6 comments

Comments

@ExSport
Copy link
Contributor

ExSport commented May 5, 2015

I have some .CR2 files. UMS tries to extract thumb when scanning library or manually browse them and store them in DB.
In my case scanning is stucked because no output is generated, no timeout, so UMS waits endlessly for the output. Killing process manually unpause this stuck but next file again pause whole scan process.
Running dcrawMS.exe -e -c photo.cr2 generates in cmd "Will not write an image to the terminal!" and without -c param thumb jpg file is generated with size larger than 4MB. If it is stored in library, DB enormeously grows (killing processes manually). I tried to update exe file with latest v9.25 64bit with no change.
I can provide sample somewhere if needed. Also it is the question if thumbs should not be cached outside DB as separate files as it generates huge DB (limited by java heapsize set) and also making small thumb instead of full thumb generated by dcrawMS (15MB cr2 includes 4MB jpeg as thumb=jpeg in full size, not real small thumb)

@ExSport
Copy link
Contributor Author

ExSport commented May 5, 2015

Btw. when I disabled image thumb generation, dcrawMS is still loaded so it freezes scan process on cr2 files again

@ExSport
Copy link
Contributor Author

ExSport commented May 5, 2015

It freezes on (log from library scan with unticked image thumbs generation):

TRACE 22:21:38.427 [Library Scanner] Matched format RAW to "E:\_MIX\_RAID\_FOTO (preber)_\_Fotky z Icka(externi)\fotooooooooooooooooo\DCIM2\100CANON\IMG_1103.CR2"
DEBUG 22:21:38.427 [Library Scanner] Starting C:\Program Files (x86)\Universal Media Server\win32\dcrawMS.exe -i -v E:\_MIX\_RAID\_FOTO (preber)_\_Fotky z Icka(externi)\fotooooooooooooooooo\DCIM2\100CANON\IMG_1103.CR2
DEBUG 22:21:39.987 [dcrawMS.exe-176-1] 
DEBUG 22:21:39.987 [dcrawMS.exe-176-1] Filename: E:\_MIX\_RAID\_FOTO (preber)_\_Fotky z Icka(externi)\fotooooooooooooooooo\DCIM2\100CANON\IMG_1103.CR2
DEBUG 22:21:39.987 [dcrawMS.exe-176-1] Timestamp: Sat Aug 02 02:24:43 2008
DEBUG 22:21:39.987 [dcrawMS.exe-176-1] Camera: Canon EOS 400D DIGITAL
DEBUG 22:21:39.987 [dcrawMS.exe-176-1] Owner: unknown
DEBUG 22:21:39.987 [dcrawMS.exe-176-1] ISO speed: 800
DEBUG 22:21:39.987 [dcrawMS.exe-176-1] Shutter: 1/25.0 sec
DEBUG 22:21:39.987 [dcrawMS.exe-176-1] Aperture: f/4.5
DEBUG 22:21:39.987 [dcrawMS.exe-176-1] Focal length: 31.0 mm
DEBUG 22:21:39.987 [dcrawMS.exe-176-1] Embedded ICC profile: no
DEBUG 22:21:39.987 [dcrawMS.exe-176-1] Number of raw images: 1
DEBUG 22:21:39.987 [dcrawMS.exe-176-1] Thumb size:  3888 x 2592
DEBUG 22:21:39.987 [dcrawMS.exe-176-1] Full size:   3948 x 2622
DEBUG 22:21:39.987 [dcrawMS.exe-176-1] Image size:  3906 x 2602
DEBUG 22:21:39.987 [dcrawMS.exe-176-1] Output size: 3906 x 2602
DEBUG 22:21:39.987 [dcrawMS.exe-176-1] Raw colors: 3
DEBUG 22:21:39.987 [dcrawMS.exe-176-1] Filter pattern: RG/GB
DEBUG 22:21:39.987 [dcrawMS.exe-176-1] Daylight multipliers: 2.423860 0.921348 1.151113
DEBUG 22:21:39.987 [dcrawMS.exe-176-1] Camera multipliers: 2270.000000 1024.000000 1802.000000 1024.000000
DEBUG 22:21:39.989 [Library Scanner] Starting C:\Program Files (x86)\Universal Media Server\win32\dcrawMS.exe -e -c E:\_MIX\_RAID\_FOTO (preber)_\_Fotky z Icka(externi)\fotooooooooooooooooo\DCIM2\100CANON\IMG_1103.CR2
TRACE 22:21:39.992 [Library Scanner] Successfully initialized buffer to 5,242,880 bytes.
DEBUG 22:21:39.992 [Library Scanner] Attaching thread: dcrawMS.exe-177
TRACE 22:22:57.066 [UPNPHelper] Receiving a M-SEARCH from [192.168.1.17:37609]
TRACE 22:23:03.916 [UPNPHelper] Receiving a NOTIFY from [192.168.1.16:39851]
TRACE 22:24:57.069 [UPNPHelper] Receiving a M-SEARCH from [192.168.1.17:37609]
TRACE 22:25:03.919 [UPNPHelper] Receiving a NOTIFY from [192.168.1.16:42527]

EDIT: Added next few lines at the end too see it is really stucked

@ExSport
Copy link
Contributor Author

ExSport commented May 5, 2015

Sum things up:
Whatever settings is used [disabled/enabled image thumbs], "-i -v" & "-e -c" params are used.
The first one correctly shows metadata but second one will always block Library Scanning totally.
I see three problems here. One is it stucks, second one is why thumbs are exported when thumbs are disabled and third one is that size of thumb is huge and whole image is put in DB (killing dcrawMS process unfreezes scan and stores whole 4MB in DB!)

@ExSport
Copy link
Contributor Author

ExSport commented May 6, 2015

Another problem connected to JPEG files without ISO value:
Adding it here to not create another issue for this.

TRACE 21:11:37.864 [Library Scanner] Matched format JPG to "K:\_MIX\FOTO&VIDEO\Mamka mobil SD zaloha\DCIM\Camera\IMG_20130224_123121.jpg"
ERROR 21:11:37.873 [Library Scanner] Error adding child: "IMG_20130224_123121.jpg"
java.lang.ArrayIndexOutOfBoundsException: 0
    at net.pms.dlna.DLNAMediaInfo.parse(DLNAMediaInfo.java:773) ~[ums.jar:5.1.3]
    at net.pms.configuration.FormatConfiguration.parse(FormatConfiguration.java:371) ~[ums.jar:5.1.3]
    at net.pms.formats.Format.parse(Format.java:270) ~[ums.jar:5.1.3]
    at net.pms.dlna.RealFile.resolve(RealFile.java:191) ~[ums.jar:5.1.3]
    at net.pms.dlna.DLNAResource.run(DLNAResource.java:1214) [ums.jar:5.1.3]
    at net.pms.dlna.RealFile.isValid(RealFile.java:64) ~[ums.jar:5.1.3]
    at net.pms.dlna.DLNAResource.addChild(DLNAResource.java:550) [ums.jar:5.1.3]
    at net.pms.dlna.DLNAResource.addChild(DLNAResource.java:510) [ums.jar:5.1.3]
    at net.pms.dlna.MapFile.manageFile(MapFile.java:106) [ums.jar:5.1.3]
    at net.pms.dlna.MapFile.analyzeChildren(MapFile.java:165) [ums.jar:5.1.3]
    at net.pms.dlna.RootFolder.scan(RootFolder.java:251) [ums.jar:5.1.3]
    at net.pms.dlna.RootFolder.scan(RootFolder.java:261) [ums.jar:5.1.3]
    at net.pms.dlna.RootFolder.scan(RootFolder.java:261) [ums.jar:5.1.3]
    at net.pms.dlna.RootFolder.scan(RootFolder.java:261) [ums.jar:5.1.3]
    at net.pms.dlna.RootFolder.scan(RootFolder.java:261) [ums.jar:5.1.3]
    at net.pms.dlna.RootFolder.scan(RootFolder.java:209) [ums.jar:5.1.3]
    at net.pms.dlna.DLNAMediaDatabase.run(DLNAMediaDatabase.java:772) [ums.jar:5.1.3]
    at java.lang.Thread.run(Unknown Source) [na:1.8.0_45]
DEBUG 21:11:37.873 [Library Scanner] GlobalIdRepo: id not found: -1

Error is on line [iso = isoValues[0];]:

                            tf = jpegmeta.findEXIFValue(TiffConstants.EXIF_TAG_ISO);
                            if (tf != null) {
                                // Galaxy Nexus jpg pictures may contain multiple values, take the first
                                int[] isoValues = tf.getIntArrayValue();
                                iso = isoValues[0];
                            }

@ExSport ExSport changed the title dcrawMS.exe breaks scanning the library and makes DB huge in size dcrawMS.exe breaks scanning the library and makes DB huge in size + JPEG ISO nullpoint exception May 6, 2015
@ExSport ExSport changed the title dcrawMS.exe breaks scanning the library and makes DB huge in size + JPEG ISO nullpoint exception dcrawMS.exe breaks scanning the library and makes DB huge in size + JPEG ISO exception May 6, 2015
valib added a commit that referenced this issue May 31, 2015
valib added a commit that referenced this issue Jun 1, 2015
Nadahar pushed a commit to Nadahar/UniversalMediaServer that referenced this issue Sep 16, 2015
Nadahar pushed a commit to Nadahar/UniversalMediaServer that referenced this issue Sep 16, 2015
@Sami32
Copy link
Contributor

Sami32 commented May 17, 2017

@ExSport @Nadahar Can this issue be closed ? as the latest UMS snapshot should have fixed it.

@Nadahar
Copy link
Contributor

Nadahar commented May 17, 2017

@ExSport @Sami32 The code that handles this is completely rewritten, so this issue report isn't relevant anymore. There might be new undiscovered issues, but they sure arent' these 😉

@Nadahar Nadahar closed this as completed May 17, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants