Track.Date does not work with iTunes (maybe also Track.Description?)

[sandreas]

The problem

I recently came across a problem with setting the Recording Date (©day) via Track.Date - the track is saved, but it does not show up in iTunes.

After some experiments, I tried the following piece of code:

        var d = DateTime.Parse("2020-01-31");
// does not show up in iTunes
        var t = new Track(
            "Santa Claus_atldotnet.mp4");
        t.Date = d;
        t.Save();
        
// shows up in iTunes
        var t2 = new Track(
            "Santa Claus_atldotnet_af.mp4");
        t2.AdditionalFields["©day"] = d.ToString("yyyy-MM-dd");
        t2.Save();
        return 0;

Where the Track.AdditionalFields part shows up in iTunes, but the Track.Date does not.

Unfortunately, this is not an acceptable workaround, because

• writing ©day to AdditionalFields sometimes KILLS every other existing entry in AdditionalFields (maybe due to the fact that ©day is already mapped via Track.Date)
• or does not store the date correctly (e.g. 2020-01-31 is stored as 2020-01-01)
• or if it has been stored like this once, is completely ignored (AdditionalFields never shows up ©day, because it is already mapped to Date - this might also be the case for other mapped fields)

Another tool (mp4v2) also then reports that there are multiple ©day fields and that the file is bogus.

I'm gonna attach 4 files for experiments:

1. Original (without ©day) - Santa Claus_noday.mp4
2. Track.Date variant that does not work with iTunes - Santa Claus_atldotnet.mp4
3. AdditionalFields variant, that does work but has side effects - Santa Claus_atldotnet_af.mp4
4. mp4tags (mp4v2) variant, tagged with a tool that works like expected - Santa Claus_mp4tags.mp4

Hope you can work with these? Thanks

[sandreas]

BTW: @wader pointed me to his project fq, this looks really promising for debugging these kind of files...

[wader]

Can't stay away from broken media files so poked around a bit:

$ fq -o line_bytes=8 -s 'map(grep_by(.type=="ilst") | .boxes | map({key: .type, value: .data}) | from_entries) as [$a,$b] | diff($a;$b) | .. | scalars | parent' Santa\ Claus_atldotnet.mp4 Santa\ Claus_mp4tags.mp4
        │00 01 02 03 04 05 06 07│01234567│.boxes[3].boxes[3].boxes[1].boxes[1].boxes[9]{}: box
0x9e50c8│               00 00 00│     ...│  size: 28
0x9e50d0│1c                     │.       │
0x9e50d0│   a9 64 61 79         │ .day   │  type: "�day"
0x9e50d0│               00 00 00│     ...│  data: raw bits
0x9e50d8│14 64 61 74 61 00 00 00│.data...│
0x9e50e0│01 00 00 00 00 32 30 32│.....202│
0x9e50e8│30                     │0       │
        │00 01 02 03 04 05 06 07│01234567│.boxes[5].boxes[3].boxes[1].boxes[1].boxes[13]{}: box
0x9f3ac8│         00 00 00 22   │   ..." │  size: 34
0x9f3ac8│                     a9│       .│  type: "�day"
0x9f3ad0│64 61 79               │day     │
0x9f3ad0│         00 00 00 1a 64│   ....d│  data: raw bits
0x9f3ad8│61 74 61 00 00 00 01 00│ata.....│
0x9f3ae0│00 00 00 32 30 32 30 2f│...2020/│
0x9f3ae8│30 31 2f 33 31         │01/31   │

Looks like in Santa Claus_atldotnet.mp4 the day is "2020" and in Santa Claus_atldotnet_af.mp4 "2020/01/31".

Maybe can explain fq query a bit: It reads and decodes two files and slurps them into one input array (-s), then map:s each decode file by grepping for the "ilst" box, map the array of subboxes into {key: "a", value: "b"} array and turn that into an object {"a": "b", ....}, now bind each object in the array to $a (object for Santa Claus_atldotnet.mp4) and $b (object for Santa Claus_atldotnet_af.mp4), use diff to find difference, recuse all values in the diff object and filter out all non-scalars (only want "edge" values), for each get the parent (the mp4 box in this case).

Other useful commands might be fq 'grep_by(.type=="meta") | dd' Santa\ Claus_atldotnet.mp4, find meta box and dd will fancy hexdump with truncating anything.

This reminds me that fq should probably have a bit better mp4 metadata support, values are treated as raw bits now.

[Zeugma440]

damn these iTunes specifics

I'm on vacations right now, I can't act for a couple more days. Please keep these sample files online until the 15th. Thanks 🙏

[sandreas]

damn these iTunes specifics

I would agree, if not Apple did find elegant solutions to problems the others did not care about. Some implementations are really annoying, but most things are pretty well thought through, if you dig deeper.

I'm on vacations right now, I can't act for a couple more days.

Don't worry, i already thought of that :-) Enjoy your vacation - I'll keep a local copy of the files, if github deletes them, but I don't think so. Thank you for your quick feedback, I really appreciate it.

Just my investigations / guess:

• ©day can contain a year only or a full date in this format: Y/m/d
• atldotnet strips the DateTime of Track.Date to Y only
• If I use AdditionalFields with ©day, then it writes the string as is
  • It does not recognize this as Track.Date because it is the wrong format
  • AdditionalFields and mapped properties (like Track.Date) are not in sync, because they are treated differently, so AdditionalFields does not show ©day when reading, maps it to Track.Date, can't handle it and if I add ©day to AdditionalFields again, it creates 2 ©day atoms, which makes the file bogus

[sandreas]

Short update: Same seems to apply for Track.Description - does not show in iTunes. A workaround with mp4tags -description "my description" file.m4b also works like expected.

Samples:

• atldotnet description tag (not shown in iTunes)
• mp4tags description tag (shown in iTunes)
• no description at all

[Zeugma440]

Thanks to both of you for the detailed feedback.

• MP4's ©day field can contain either a plain year or a date. When assigning a value to Track.Date, ATL mistakenly mapped that value as a year instead of mapping it as an entire date. That has been fixed : a75c9c7

• I also changed the field used to write Track.Description, which is now desc instead of ©des : cdf5561

If you're fine with that solution, I'll publish a new release.

Cheers

[sandreas]

Hey, hope you enjoyed your vacation.

Thanks to both of you for the detailed feedback.

You're welcome! I rarely had a maintainer that responsive and co-operative...

MP4's ©day field can contain either a plain year or a date. When assigning a value to Track.Date, ATL mistakenly mapped that value as a year instead of mapping it as an entire date. That has been fixed : a75c9c7

Awesome! Thank you. Are you sure that ToShortDateString() is valid in all cases?

The value of the current DateTime object is formatted using the pattern defined by the DateTimeFormatInfo.ShortDatePattern property associated with the current culture.

Depending on the current culture, the Date format would be different, e.g. in Germany (where I live) it would be dd.MM.yyyy - I doubt that iTunes will recognize culture specific date formats. In my opinion it should always be yyyy/MM/dd, so I would use value.ToString("yyyy/MM/dd"). See

atldotnet/ATL/Entities/MetadataHolder.cs

Line 169 in 227eddd

tagData.IntegrateValue(Field.RECORDING_YEAR_OR_DATE, (value > DateTime.MinValue) ? value.ToShortDateString() : null);

I also changed the field used to write Track.Description, which is now desc instead of ©des : cdf5561

Cool, that fixes this problem... I tried this out with AdditionalFields["desc"] and it worked in iTunes like expected :-)

If you're fine with that solution, I'll publish a new release.

I'm fine with a new release, if the date format question is clarified.

I still have one question, that should not block the release, but maybe worth another issue:

There was a strange thing regarding the Description... I had:

• NO Description value
• a LongDescription value was set (AdditionalFields["ldes"])

Accessing the Description somehow pointed to the LongDescription value. That was very confusing, because after deleting the value, it still had one, but changed its contents.

Another file, another problem was that setting Description=null or Description="" did not delete the value... even Description=" " (space) did not delete the value like expected (or set it to ).

Is there an internal Fallback or special handling of this?

[sandreas]

@Zeugma440 Sorry, I updated my comment above. Summary

• The desc change is working in iTunes
• The ©day change will pretty likely fail in non-US countries with a wrong date format (e.g. dd.MM.yyyy in Germany)

[Zeugma440]

I rarely had a maintainer that responsive and co-operative...

You're very welcome 😉 I know what it is to have one's project depending on a 3rd party library for complex operations.

Are you sure that ToShortDateString() is valid in all cases?

Good catch, thanks. I just updated it according to your suggestion.

There was a strange thing regarding the Description... I had:

Could you upload the file where that happens ? As far as I understand, there are 3 fields to play with (desc, ©des and ldes), and I am not sure which ones your file ended up with.

[sandreas]

I know what it is to have one's project depending on a 3rd party library for complex operations.

True, as always :-) tone heavily relies on atldotnet and without it, I could stop further development :-)

Could you upload the file where that happens ? As far as I understand, there are 3 fields to play with (desc, ©des and ldes), and I am not sure which ones your file ended up with.

I'll add a separate issue for this one, because I have to investigate this - as it was not a specific file or track, but everytime I try the Track.Description to "" or null, it just keeps the description as is and SOMETIMES it falls back to the ldes... very strange and hard to reproduce. So this one should not be in the way of 4.9.0.

[Zeugma440]

Thanks. v4.09 has just been released

[sandreas]

Thanks. v4.09 has just been released

Awesome. BTW I also released a small Metadata library with extensions and improvements for atldotnet today. Just if you need some inspiration :-)

Moreover I created a new Metadata Format tone.json, containing all the possible metadata items used in atldotnet. If you would like to try it out, you can use tone - in the next days I'll release a v0.0.7 including your fixes.

[Zeugma440]

https://github.com/sandreas/DotnetLibAudioMetadata looks like it has added value, but the welcome page doesn't say much.

Could you take some time to document it, and maybe open a Discussions tab over there for further chat ?

[Zeugma440]

I also advise you to create a SonarCloud account to analyze tone. Sonar is a great way to get objective metrics about code quality. I've been using it for my two pet projects and it definitely helped in improving my coding practices.

[sandreas]

I also advise you to create a SonarCloud account to analyze tone. Sonar is a great way to get objective metrics about code quality.

Good idea, I know SonarQube but my Projects never felt big enough to let loose this monster ;) Maybe I plan this after having improved my github build workflows .