v0.5.3

[sylikc]

Released to PyPI

• Quite a few docstring changes
• ExifToolHelper's get_tags() and set_tags() checks tag names to prevent inadvertent write behavior.
• Renamed a few of the errors to make sure the errors are explicit
  • ProcessStateError -> ExifToolProcessStateError
  • new ExifToolExecuteException for all execute() related errors which returns stdout, stderr, returncode, cmd of offending command
    • OutputEmpty -> ExifToolOutputEmptyError
    • OutputNotJSON -> ExifToolJSONInvalidError
• ExifToolHelper() has some static helper methods which can be used when extending the class (ExifToolAlpha.set_keywords_batch() demonstrates a sample usage).
• setup.py tweaked to make it Beta rather than Alpha
• ExifToolAlpha.get_tag() updated to make it more robust.
• Fixed ujson compatibility
• Cleaned up and refactored testing.

---

This discussion was created from the release v0.5.3.

[Fox214]

Hi,

Not sure what's going on and not sure where to ask so I am doing so here.

Using this via cygwin on windows, (python 3.6) exiftool is install and Its reading tags just fine, trying to use the execute method to change the date taken. Doing the following:

with exiftool.ExifTool() as et: et._set_logger(logger) et.execute('-DateTimeOriginal="{}" -overwrite_original'.format(cdate), f)

Its not updating the date takent though (in fact its clearing it out),

Running this at the command line works no problem:
exiftool -P -DateTimeOriginal="2012:10:20 14:40:23" /cygdrive/p/Photos/Events/Birthdays/s41463s1128187_2_0.jpg

Here's the logger output (trying a few different things, curious it does not like -P:
3/26/2022 09:40:55 PM Method 'execute': Command sent = ['-P -overwrite_original -DateTimeOriginal="2011:10:20 14:40:23-4:00"', '/cygdrive/p/Photos/Events/Birthdays/s41463s1128187_2_0.jpg', '-echo4', '=${status}=post606172', '-execute606172']
03/26/2022 09:40:55 PM ExifTool.execute: IN params = ('-P -overwrite_original -DateTimeOriginal="2011:10:20 14:40:23-4:00"', '/cygdrive/p/Photos/Events/Birthdays/s41463s1128187_2_0.jpg')
03/26/2022 09:40:55 PM ExifTool.execute: OUT stdout = ""
03/26/2022 09:40:55 PM ExifTool.execute: OUT stderr = "Warning: Tag 'P' is not defined
Nothing to do.
"
03/26/2022 09:40:55 PM ExifTool.execute: OUT status = 1
03/26/2022 09:40:55 PM Method 'terminate': Exiftool terminated successfully.
03/26/2022 09:42:10 PM Method 'execute': Command sent = ['-DateTimeOriginal="2011:10:20 14:40:23-4:00"', '/cygdrive/p/Photos/Events/Birthdays/s41463s1128187_2_0.jpg', '-echo4', '=${status}=post641800', '-execute641800']
03/26/2022 09:42:10 PM ExifTool.execute: IN params = ('-DateTimeOriginal="2011:10:20 14:40:23-4:00"', '/cygdrive/p/Photos/Events/Birthdays/s41463s1128187_2_0.jpg')
03/26/2022 09:42:10 PM ExifTool.execute: OUT stdout = "1 image files updated
"
03/26/2022 09:42:10 PM ExifTool.execute: OUT stderr = ""
03/26/2022 09:42:10 PM ExifTool.execute: OUT status = 0
03/26/2022 09:42:10 PM Method 'terminate': Exiftool terminated successfully.
03/26/2022 09:43:07 PM Method 'execute': Command sent = ['-DateTimeOriginal="2011:10:20 14:40:23"', '/cygdrive/p/Photos/Events/Birthdays/s41463s1128187_2_0.jpg', '-echo4', '=${status}=post323644', '-execute323644']
03/26/2022 09:43:08 PM ExifTool.execute: IN params = ('-DateTimeOriginal="2011:10:20 14:40:23"', '/cygdrive/p/Photos/Events/Birthdays/s41463s1128187_2_0.jpg')
03/26/2022 09:43:08 PM ExifTool.execute: OUT stdout = "1 image files updated
"
03/26/2022 09:43:08 PM ExifTool.execute: OUT stderr = ""
03/26/2022 09:43:08 PM ExifTool.execute: OUT status = 0
03/26/2022 09:43:08 PM Method 'terminate': Exiftool terminated successfully.
03/26/2022 09:43:59 PM Method 'execute': Command sent = ['-DateTimeOriginal="2011:10:20 14:40:23" -overwrite_original', '/cygdrive/p/Photos/Events/Birthdays/s41463s1128187_2_0.jpg', '-echo4', '=${status}=post125005', '-execute125005']
03/26/2022 09:44:00 PM ExifTool.execute: IN params = ('-DateTimeOriginal="2011:10:20 14:40:23" -overwrite_original', '/cygdrive/p/Photos/Events/Birthdays/s41463s1128187_2_0.jpg')
03/26/2022 09:44:00 PM ExifTool.execute: OUT stdout = "1 image files updated
"
03/26/2022 09:44:00 PM ExifTool.execute: OUT stderr = ""
03/26/2022 09:44:00 PM ExifTool.execute: OUT status = 0
03/26/2022 09:44:00 PM Method 'terminate': Exiftool terminated successfully.

[sylikc]

Hi @Fox214 asking here is good, or creating an issue works too. So, first off, thanks for sharing all the relevant debugging info...

The way PyExifTool is used, and called is slightly different than what you'd expect. I have got to put together some examples in the docs...

So your call was originally

with exiftool.ExifTool() as et:
    et._set_logger(logger)
    et.execute('-DateTimeOriginal="{}" -overwrite_original'.format(cdate), f)

but it needs to be called this way:

with exiftool.ExifTool() as et:
    et.logger(logger)
    et.execute('-DateTimeOriginal="{}"'.format(cdate), '-overwrite_original', f)

The main difference being any time there is a "space" in the command, it needs to be another parameter to execute()

If you wanted to specify the -P flag, you do it the same way... here it is written again, btw, using f-strings (Python3 feature)

with exiftool.ExifTool() as et:
    et.logger(logger)
    et.execute('-P', f'-DateTimeOriginal="{cdate}"', '-overwrite_original', f)

[sylikc]

@Fox214 I just updated the documentation. Let me know it it helps you understand how to call ExifTool. https://sylikc.github.io/pyexiftool/examples.html#input-parameters

[sylikc]

Oh I was looking at this code to write some stuff in the FAQ and now this actually looks a little funny. The call shouldn't be et._set_logger but just use the property directly as defined ExifTool.logger:

et.logger = logger

I edited my response above with the public attribute rather than using the private method.

[Fox214]

Thank you for that, I do think the doc update is helpful regardless, the 03/26/2022 09:43:07 PM only sent 1 parameter though and after updating I still am not seeing the date correctly reflected in the file.

So this is failing: (it says image updated in the debug)
with exiftool.ExifTool() as et: et._set_logger(logger) et.execute('-DateTimeOriginal="{}"'.format(cdate),'-P', f)
but this works:
p = subprocess.Popen(['/usr/bin/Image-ExifTool-12.30/exiftool', '-DateTimeOriginal="{}"'.format(cdate),'-P', f], stdout=subprocess.PIPE) out, err = p.communicate()

any ideas? If you give me a file/line to edit I can go try and debug for you

[sylikc]

btw, please use triple ` 's to show code... it's showing flat for me

import exiftool

from datetime import datetime

f = "rose.jpg"
cdate = "2011:10:20 14:40:23"
now = datetime.strftime(datetime.now(), "%Y:%m:%d %H:%M:%S")

with exiftool.ExifTool() as et:
    print(et.execute('-P', f'-DateTimeOriginal="{now}"', '-overwrite_original', f))
    print(et.execute('-DateTimeOriginal', f))
    print(et.execute('-P', f'-DateTimeOriginal="{cdate}"', '-overwrite_original', f))
    print(et.execute('-DateTimeOriginal', f))

works fine for me. this is the output, showing the time was modified properly

1 image files updated

[EXIF]          Date/Time Original              : "2022:03:27 20:41:39"

1 image files updated

[EXIF]          Date/Time Original              : "2011:10:20 14:40:23"

[Fox214]

I just added print('proc_args',proc_args) to line 734 of exiftool.py v0.5.3 and got this:
proc_args ['/usr/bin/Image-ExifTool-12.30/exiftool', '-stay_open', 'True', '-@', '-', '-common_args', '-G', '-n']

[sylikc]

that's correct

[Fox214]

Ok, so this is strange. When I update via pyexiftool I can see the Date/Time Original tag via your method above (and calling exiftool via cmd line) but not right clicking on the details in windows explorer (or in the windows explorer preview) but if I update via subprocess or cmd line myself, I can see the data in windows.

Any ideas why that would be?

[sylikc]

i'm not sure why that would be... more testing would be in order on your end. PyExifTool is using exiftool directly under the hood, so the results should be exactly the same as running on the command line with exiftool.

I'm also primarily on Windows, but not via Cygwin. You could have some strange interaction with Cygwin? I'm not sure... The tags inside can only reliably be checked with the command line tool. I'm not sure which DateTimeOriginal that Windows reads in explorer...

[Fox214]

I think its the quotes, I remove them and it appears to work:

et.execute('-P', f'-DateTimeOriginal="{cdate}"', '-overwrite_original', f)

to:
et.execute('-P', f'-DateTimeOriginal={cdate}', '-overwrite_original', f)

the first yielding:
Date/Time Original : "2011:10:20 14:40:23"

the 2nd:
Date/Time Original : 2011:10:20 14:40:23

Thanks so much for your help.

[sylikc]

ok. thanks for the update. i'll update that in the examples.

[MagnusOxlund]

I noticed you're working on error messages in this update. Do you have any plans to make the errors more verbose—maybe passing on Exiftool's error messages from stderr with the exception?

I've recently had to debug some instances of exiftool.exceptions.ExifToolExecuteError with the message execute returned a non-zero exit status: 1, which isn't much to go by. Digging around the docs led me to last_stderr:

with ExifToolHelper() as et:
    try:
        et.set_tags(filepath, metadata)
    except Exception as e:
        amended_args = tuple([f'{e.args[0]}\n{e.stderr}', *e.args[1:]])
        e.args = amended_args
        raise

Traceback (most recent call last):
  File "/test.py", line 15, in <module>
    et.set_tags(filepath, metadata)
  File "/lib/python3.10/site-packages/exiftool/helper.py", line 444, in set_tags
    return self.execute(*exec_params)
  File "/lib/python3.10/site-packages/exiftool/helper.py", line 119, in execute
    raise ExifToolExecuteError(self._last_status, self._last_stdout, self._last_stderr, params)
exiftool.exceptions.ExifToolExecuteError: execute returned a non-zero exit status: 1
Error: Not a valid PNG (looks more like a JPEG) - /testing/rose.png

Maybe it would serve a greater purpose if put in front of the user?

[sylikc]

I suppose the challenge would be figuring out when and where the CLI tool produces useful errors and piping them through? Or are you standing firm on not doing any error message plumbing? I can certainly understand not wanting to adopt the responsibility of forwarding every error message as might be implied by forwarding some, admittedly more so after having peeked at the Perl source code.

Well, I'm open to code ideas to cover at least general error cases. I would say the logic of parsing upstream error messages is a bit problematic. First off, upstream code might change unexpectedly, and then PyExifTool might have to keep up with the nuances of it. I'm actually glad that the newer versions of exiftool return the exit-status at all. It didn't exist in older versions, and that exit status at least says "hey there was something wrong" but not necessarily that "this is a big error". But as you can see in the sample, stderr is not useful sometimes. Sometimes it is...

Actually, ExifToolAlpha has code that someone tried awhile back to figure out error conditions. I left it in Alpha for now (though it's certainly pending removal)
https://github.com/sylikc/pyexiftool/blob/master/exiftool/experimental.py#L67-L74
https://github.com/sylikc/pyexiftool/blob/master/exiftool/experimental.py#L311-L325

Just to be clear on the change in 0.5.3, is the update that ExifToolExecuteException provides access to stderr, etc., as per the release notes?

Yes, that is correct. I added that feature in v0.5.3. It was specifically to assist debugging exit-status problems.

The example above is actually a snippet from exit-status=1 problems I had when I started using v0.5.2 (this version added the exit-status checking by default). I had an exif metadata backup code, that was never a problem before... and then it started crashing because of that "Unknown file type" error.

The code snippet was just for illustration. I wouldn't use it in production without at least checking for .args[0] and .stderr, and it doesn't seem like a wise thing to attempt anyway.

Ha. Ok, just checking. I apologize for assuming so.

[sylikc]

In addition, parsing for execute_json might seem easy, but changes to common_args affects the output. But let's take this simple example I just ran into to demonstrate why I am hesitant to try to write code relating to error parsing:

logging.basicConfig(level=logging.DEBUG)
with ExifToolHelper(logger=logging.getLogger(__name__)) as et:
	et.get_tags("bad.file.bin", "FileName")

returns

DEBUG:__main__:ExifToolHelper.execute: IN  params = ('-j', '-FileName', 'bad.file.bin')
DEBUG:__main__:ExifToolHelper.execute: OUT stdout = "[{
  "SourceFile": "bad.file.bin",
  "File:FileName": "bad.file.bin"
}]
"
DEBUG:__main__:ExifToolHelper.execute: OUT stderr = ""
DEBUG:__main__:ExifToolHelper.execute: OUT status = 1

This is the same bad.file.bin from the previous example... and I'm getting the FileName... but wait, the exit status is 1 but no error is anywhere to be found.

Getting the full tag list shows the error, but the output from exiftool didn't help.

[MagnusOxlund]

...upstream code might change unexpectedly, and then PyExifTool might have to keep up with the nuances of it.

Yeah, it would require parsing the source and filtering out the relevant messages on each update. Just from glancing at it, I can't tell which messages get sent where—I don't speak Perl—so I wouldn't be able to map the messages to their corresponding errors.

Actually, ExifToolAlpha has code that someone tried awhile back to figure out error conditions. I left it in Alpha for now (though it's certainly pending removal) https://github.com/sylikc/pyexiftool/blob/master/exiftool/experimental.py#L67-L74 https://github.com/sylikc/pyexiftool/blob/master/exiftool/experimental.py#L311-L325

I guess that's the curse of wrappers, keeping up with while also trying to improve on a moving target 🙂 Maybe a disclaimer on the transparency of CLI error messages and a referral to the logger option in the docs is the better way to handle it?

Ha. Ok, just checking. I apologize for assuming so.

Not at all 🙂 A PEP seeking to implement a feature closely resembling that code was just accepted:

The Steering Council reviewed your updates and is Accepting the latest iteration of PEP 678: Enriching Exceptions with Notes. 😁

[sylikc]

I'll update something on the docs on the FAQ. I'll probably cite the examples we have in this discussion.

I read thru that PEP. Interesting stuff... this is pretty much exactly what use case it's for!

I've definitely looked through the Perl code... but it is so insanely complicated that I don't have a clue how Phil Harvey can even maintain it. I recall a few months ago stepping through trying to figure out how a certain flag works... nearly impossible.

[MagnusOxlund]

I'll update something on the docs on the FAQ.

That's wonderful. I'm sure it's going to be appreciated. Thank you for indulging me.

I don't have a clue how Phil Harvey can even maintain it.

Yeah, I hope whoever takes over one day has a better sense of how he's structuring it than I do 😉