Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to modify a JPEGImage10Metadata comment value #845

Closed
steinarb opened this issue Oct 21, 2023 · 17 comments
Closed

Unable to modify a JPEGImage10Metadata comment value #845

steinarb opened this issue Oct 21, 2023 · 17 comments
Labels
Trouble-shooting

Comments

@steinarb
Copy link
Contributor

steinarb commented Oct 21, 2023 •
edited

I am trying to modify the JFIF comment (note: not an EXIF tag, but the comment in the JFIF metadata) of an image and am running into the issue that the JPEGImage10Metadata object is readonly.

I am trying the approach of getting the metadata as a tree, walking the tree, replacing an attribute and setting the metadata as a tree, as outlined here: https://stackoverflow.com/a/9508479

Is there another way?

I have looked for ways to copy-and-mutate the JPEGImage10Metadata object, but not found any?

(I can see the existing value in the debugger so that it feels kind of annoying not to be able to change it... I've even considered using reflection... 😄 )

Here is the code that I'm trying to run:

    File downloadImageUrlToTempFile(AlbumEntry albumEntry, Path tempDir) {
        var imageUrl = albumEntry.getImageUrl();
        if (imageUrl == null || imageUrl.isEmpty()) {
            throw new OldAlbumException(String.format("Unable to download album entry matching id=%d, imageUrl is missing", albumEntry.getId()));
        }

        var fileName = findFileNamePartOfUrl(imageUrl);
        var tempfile = tempDir.resolve(fileName).toFile();
        IIOImage image = null;
        ImageWriter writer = null;
        try {
            HttpURLConnection connection = getConnectionFactory().connect(imageUrl);
            connection.setRequestMethod("GET");
            try(var inputStream = ImageIO.createImageInputStream(connection.getInputStream())) {
                var readers = ImageIO.getImageReaders(inputStream);
                if (readers.hasNext()) {
                    var reader = readers.next();
                    writer = ImageIO.getImageWriter(reader);
                    reader.setInput(inputStream);
                    image = reader.readAll(0, null);
                }
            }
        } catch (IOException e) {
            throw new OldAlbumException(String.format("Unable to download album entry matching id=%d from url=\"%s\"", albumEntry.getId(), albumEntry.getImageUrl()), e);
        }

        var metadata = image.getMetadata();
        var metadataAsTree = metadata.getAsTree("javax_imageio_1.0");
        findJfifCommentNode(metadataAsTree)
            .ifPresent(node -> node.setAttribute("value", albumEntry.getDescription()));
        try {
            metadata.setFromTree("javax_imageio_1.0", metadataAsTree);
        } catch (IIOInvalidTreeException e) {
            throw new OldAlbumException(String.format("Failed to replace comment in local copy of album entry matching id=%d from url=\"%s\"", albumEntry.getId(), albumEntry.getImageUrl()), e);
        }

        try (var outputStream = ImageIO.createImageOutputStream(new FileOutputStream(tempfile))){
            writer.setOutput(outputStream);
            writer.write(image);
            Files.setLastModifiedTime(tempfile.toPath(), FileTime.from(albumEntry.getLastModified().toInstant()));
            return tempfile;
        } catch (IOException e) {
            throw new OldAlbumException(String.format("Unable to save local copy of album entry matching id=%d from url=\"%s\"", albumEntry.getId(), albumEntry.getImageUrl()), e);
        }
    }

    Optional<IIOMetadataNode> findJfifCommentNode(Node metadataAsTree) {
        return StreamSupport.stream(iterable(metadataAsTree.getChildNodes()).spliterator(), false)
            .filter(n -> "Text".equals(n.getNodeName()))
            .findFirst()
            .flatMap(n -> StreamSupport.stream(iterable(n.getChildNodes()).spliterator(), false).findFirst());
    }

    public static Iterable<IIOMetadataNode> iterable(final NodeList nodeList) {
        return () -> new Iterator<IIOMetadataNode>() {

                private int index = 0;

                @Override
                public boolean hasNext() {
                    return index < nodeList.getLength();
                }

                @Override
                public IIOMetadataNode next() {
                    if (!hasNext())
                        throw new NoSuchElementException();
                    return (IIOMetadataNode) nodeList.item(index++);
                }
            };
    }
@haraldk
Copy link
Owner

haraldk commented Oct 21, 2023

Hi Steinar,

Short answer: JPEGImage10Metadata is (currently) read-only, see the isReadOnly method. Invoking mergeTree/setFromTree on an instance, will fail.

Long answer: You should still be able to achieve what you want, see #668. 😀
Especially this comment where I try to outline a workaround. I think it worked for jAlbum.

Try that first, but if you can't make it work, let me know, and we'll see if I can help.

@steinarb
Copy link
Contributor Author

Thanks, Harald! The workaround of using the default metadata of the writer and replacing the content with the modified metadata tree of the writer worked. Working code example below.

Next up for me is to find the exif segment of the metadata, if present, or create one of there isn't one, and then insert 3 values:

  1. last modified date
  2. the image description
  3. the user comment

Can I do this in a simple way from the existing metadata object (where I can see the exif data in the debugger)?
Or do I have to read the exif segment from the input stream?
And when/if I have the exif segent can I easily add it to the default metadata object?
I assume I will have to insert it into the tree?

    File downloadImageUrlToTempFile(AlbumEntry albumEntry, Path tempDir) {
        var imageUrl = albumEntry.getImageUrl();
        if (imageUrl == null || imageUrl.isEmpty()) {
            throw new OldAlbumException(String.format("Unable to download album entry matching id=%d, imageUrl is missing", albumEntry.getId()));
        }

        var fileName = findFileNamePartOfUrl(imageUrl);
        var tempfile = tempDir.resolve(fileName).toFile();
        IIOImage image = null;
        ImageWriter writer = null;
        try {
            HttpURLConnection connection = getConnectionFactory().connect(imageUrl);
            connection.setRequestMethod("GET");
            try(var inputStream = ImageIO.createImageInputStream(connection.getInputStream())) {
                var readers = ImageIO.getImageReaders(inputStream);
                if (readers.hasNext()) {
                    var reader = readers.next();
                    writer = ImageIO.getImageWriter(reader);
                    reader.setInput(inputStream);
                    image = reader.readAll(0, null);
                }
            }
        } catch (IOException e) {
            throw new OldAlbumException(String.format("Unable to download album entry matching id=%d from url=\"%s\"", albumEntry.getId(), albumEntry.getImageUrl()), e);
        }

        var metadata = image.getMetadata();
        var metadataAsTree = metadata.getAsTree("javax_imageio_1.0");
        findJfifCommentNode(metadataAsTree)
            .ifPresent(node -> node.setAttribute("value", albumEntry.getDescription()));

        try (var outputStream = ImageIO.createImageOutputStream(new FileOutputStream(tempfile))){
            writer.setOutput(outputStream);
            var param = writer.getDefaultWriteParam();
            var modifiedMetadata = writer.getDefaultImageMetadata(ImageTypeSpecifiers.createFromRenderedImage(image.getRenderedImage()), param);
            modifiedMetadata.setFromTree("javax_imageio_1.0", metadataAsTree);
            image.setMetadata(modifiedMetadata);
            writer.write(image);
            Files.setLastModifiedTime(tempfile.toPath(), FileTime.from(albumEntry.getLastModified().toInstant()));
            return tempfile;
        } catch (IOException e) {
            throw new OldAlbumException(String.format("Unable to save local copy of album entry matching id=%d from url=\"%s\"", albumEntry.getId(), albumEntry.getImageUrl()), e);
        }
    }

    Optional<IIOMetadataNode> findJfifCommentNode(Node metadataAsTree) {
        return StreamSupport.stream(iterable(metadataAsTree.getChildNodes()).spliterator(), false)
            .filter(n -> "Text".equals(n.getNodeName()))
            .findFirst()
            .flatMap(n -> StreamSupport.stream(iterable(n.getChildNodes()).spliterator(), false).findFirst());
    }

    public static Iterable<IIOMetadataNode> iterable(final NodeList nodeList) {
        return () -> new Iterator<IIOMetadataNode>() {

                private int index = 0;

                @Override
                public boolean hasNext() {
                    return index < nodeList.getLength();
                }

                @Override
                public IIOMetadataNode next() {
                    if (!hasNext())
                        throw new NoSuchElementException();
                    return (IIOMetadataNode) nodeList.item(index++);
                }
            };
    }

@steinarb
Copy link
Contributor Author

steinarb commented Oct 22, 2023 •
edited

Adding exif to the tree is possibly covered here? #586 (comment)

(Seems a little cumbersome...?) Is it easier if the exif segment already exists I wonder?

(Edit: yes it easier if the exif segment exists, it says so in the text of the comment above the code example which (I think describes) how to create an exif segment from scratch)

@steinarb
Copy link
Contributor Author

steinarb commented Oct 22, 2023 •
edited

Tried the approach outlined in #586 (comment) and https://stackoverflow.com/q/36029295 but I got the following error message when setting the metadata:

Caused by: javax.imageio.metadata.IIOInvalidTreeException: Invalid node: markerSequence

Here is what I've tried to do: https://gist.github.com/steinarb/0ab59ad7c17b7ac15b757dd67368dad9

Here is the complete stack trace: https://gist.github.com/steinarb/a2ae1019aa90a4ade2ca49387f6135b9

I'm sure there is a simple error copying from the examples, but I'm unable to sport the error.

@haraldk
Copy link
Owner

haraldk commented Oct 22, 2023

Sorry, no time right now, but: javax_imageio_1.0 is the format neutral metadata format. You need the “JPEG native” format to access/write Exif, ie javax_imageio_jpeg_image_1.0.

@haraldk
Copy link
Owner

haraldk commented Oct 24, 2023 •
edited

Did changing to the javax_imageio_jpeg_image_1.0 metadata format help? 😀

Your code does a lot, so it would probably help to organize things a little different to make things easier to understand... But I'm fairly sure this was the reason for the exception (as there is no markerSequece in the "plug-in neutral" format).

The "plug-in neutral" format is good for getting a few essential values in a standard way, but it's not so good if you need to mess with format specific things like Exif in JPEG. I believe the JPEG native format always has a markerSequence node, but I guess it doesn't hurt to be on the safe side...

javax_imageio_1.0 DTD
javax_imageio_jpeg_image_1.0 DTD

@steinarb
Copy link
Contributor Author

steinarb commented Oct 24, 2023 via email

@steinarb
Copy link
Contributor Author

Ok, now I'm back to getting java.io.EOFException.

The exception comes when I'm trying to read the metadata of the transformed image in the test and it comes when trying to read a stream containing just "Exif" followed by to 0-byte values.

I.e. it looks like the line doesn't do anything:

            new TIFFWriter().write(entries, new MemoryCacheImageOutputStream(bytes));

The size of the bytes ByteArrayOutputStream doesn't change in the above line.

This is the current version of the complete code: https://gist.github.com/steinarb/251e02cda3de13c9cc2845a2c9020f4a#file-oldalbumserviceprovider-java-L35

@steinarb
Copy link
Contributor Author

I did this instead, and then I got further:

            try(var imageOutputStream = new MemoryCacheImageOutputStream(bytes)) {
                new TIFFWriter().write(entries, imageOutputStream);
            }

Now the EXIF metadata parses without throwing an EOFException.

Current problem: my replacement of the JFIF comment didn't survive the addition of modifying EXIF comments: https://gist.github.com/steinarb/da81e5d71a6d9e44058ef02598972f3a#file-oldalbumserviceprovider-java-L30

@haraldk
Copy link
Owner

haraldk commented Oct 25, 2023 •
edited

The latest version (with try-with-resources) closes (and implicitly flushes) the stream, to commit from the memory cache to the underlying ByteArrayOuputStream. The previous code would probably also have worked if you explicitly invoked flush() on the MemoryCacheImageOutputStream.

Not sure about why the comment didn't survive. As mentioned, there's too much "going on" for me to fully understand what's happening. Try writing this as a test case, and I'll be able to help you better. 😀

Are you sure the findJfifCommentNode method works as intended? Are you sure there was an existing comment in the file to replace in the first place? Are you sure the ifPresent block is even executed?

PS: This code won't work as intended:

        var markerSequence = (IIOMetadataNode) metadataAsTree.getElementsByTagName("markerSequence").item(0);
        if (markerSequence == null) {
            markerSequence = new IIOMetadataNode("markerSequence");
            metadataAsTree.appendChild(markerSequence);
        }

If there is no "markerSequence" node, the node list will be empty, and there won't be an item 0... You'll either get a node or an exception, markerSequence will never be null.

@steinarb
Copy link
Contributor Author

steinarb commented Oct 26, 2023 via email

@haraldk
Copy link
Owner

haraldk commented Oct 26, 2023

I'm trying to find the right node in the new tree. As said I can't see
the tree nodes well in the debugger and the debugger goes a bit crazy
when I try to navigate them.

One thing this odd XML-ish node structure is good for, is that it's.. well.. XML. 😀

You can dump the whole tree like this:

 new XMLSerializer(System.out, "UTF-8").serialize(tree, false);

(using com.twelvemonkeys.xml. XMLSerializer, other serializers will probably do fine too..)

The only thing you won't see there it the userObjects, because the javax.imageio API invented userObject instead of reusing userData, not really sure why...

I found that the comment was put into a node named "com" by stepping
through the build of the native tree in the debugger.

But I found a node named "com" directly under markerSequence and tried
replacing it, but that probably wasn't the right one.

According to the DTD I referenced above, the com (JFIF comment) marker is a child of markerSequence. It can't appear anywhere else.

@steinarb
Copy link
Contributor Author

steinarb commented Oct 26, 2023 via email

@steinarb
Copy link
Contributor Author

Here's the tidied up code: https://gist.github.com/steinarb/ec1c22513a64870a880f21e134ea33fb

Now it will reside in a git branch waiting for the OSGi-fied version of Twelvemonkey to be released. 😃

Note that I'm currently just storing the description in the JFIF comment (i.e. the way I did things back in 1996), instead of in the EXIF usercomment.

It's trivial to store it there as well. I take suggestions as what to do (store it in the exif user comment? continue to use the JFIF comment? Use both?).

Do other programs use any of these fields?

I know google photo uses the lastmodifieddate EXIF field.

Does any applications out there use the EXIF ImageDescription?

Does any applications out there use either the JFIF comment (it has been around for a while...)?
Does any applications out there use the EXIF UserComment?

@haraldk
Copy link
Owner

haraldk commented Oct 30, 2023

Now it will reside in a git branch waiting for the OSGi-fied version of Twelvemonkey to be released. 😃

I thought I mentioned it, but 3.10.0 with your OSGi support was released last week.

I take suggestions as what to do (store it in the exif user comment? continue to use the JFIF comment? Use both?).
Do other programs use any of these fields?

Wikipedia lists a few programs, but that's just a small subset. I don't have a list of applications, but mostly use only Preview on MacOS, which displays a lot of TIFF and Exif/GPS metadata. You should probably just test with the software you use, or the software you want to support.

For Exif vs JFIF comment, it's a bit of apples and oranges discussion. The COM marker is just a generic "comment", while the TIFF/Exif tags have a specific semantic meaning, allowing for much richer metadata. I don't see a problem using both. There's also IPTC metadata and XMP metadata worth considering, if you want to jump into the meta data rabbit hole... 😀

@steinarb
Copy link
Contributor Author

steinarb commented Oct 30, 2023 via email •
edited

@haraldk
Copy link
Owner

haraldk commented Oct 31, 2023

Sounds good!

I'll close this issue as resolved, unless you have more questions about JPEG metadata? If so just reopen, or create a new, more specific issue.

@haraldk haraldk closed this as completed Oct 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Trouble-shooting
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@haraldk @steinarb