Dates importing incorrectly (or not at all)

[primetime-sudo]

The bug

I uploaded 35k photos to a new install. Most of them imported with proper dates, but there's a very large number (~4-5k) that imported with no dates in the field

There is a lot of metadata in the actual image

As you can see there is no date shown in Immich, however the date for all these thousands of photos is showing as 3/13/2019. This is a date where I moved a ton of files to this drive I'm guessing, so it's showing as the creation or modification date. Shouldn't Immich be reading from the metadata of the file, not the creation or modification dates?

The OS that Immich Server is running on

Ubuntu 22.04

Version of Immich Server

1.97.0

Version of Immich Mobile App

?

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

version: "3.8"

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: [ "start.sh", "immich" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - /mnt/storage/storage/media/photos:/mnt/storage/storage/media/photos:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.yml
    #   service: hwaccel
    command: [ "start.sh", "microservices" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - /mnt/storage/storage/media/photos:/mnt/storage/storage/media/photos:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:c5a607fb6e1bb15d32bbcf14db22787d19e428d59e31a5da67511b49bb0f1ccc
    restart: always

  database:
    container_name: immich_postgres
    image: tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

volumes:
  pgdata:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/mnt/storage/storage/media/immich-photos

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=[[redacted]]

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis

Reproduction steps

1.Upload photo
2.Look at photo in immich
3.See wrong dates
...

Additional information

No response

[alextran1502]

Hello, please use EXIF tool to check for the date of the images.

Also, have the jobs finished processing?

[alextran1502]

These are the date/time tags from EXIF that are read for the file, sorted by priority.

You can try selecting an incorrect file and then returning the metadata extraction on that one to see if it fixes the issue.

Also, anything you see in the logs?

Did the server get restarted at any point after you importing/uploading your photos?

[primetime-sudo]

Can you elaborate on "returning the metadata extraction" part?

Yes, the server has restarted a few times after import. It crashed my server several times while uploading photos.

Considering I have a very large number of incorrect files, would it be best for me to delete every single thing from Immich and try to correct the files' metadata prior to importing? The issue is that the incorrect files span over hundreds of different folders so it's not like I can just delete everything from 3/13 and selectively upload those. Would be absolutely open to the best way to do this!

[alextran1502]

You can select the photos on the timeline, then from the context menu on the upper top right, run the metadata extraction again.

Or you can run the metadata extraction job for all in the job page.

You server restarting means that some of the job queue was lost during restart process

What is your server spec and did you change any of the concurrency level?

[primetime-sudo]

Intel Core i5 Quad (i5-6500) 3.20 GHz, 8 GB RAM, 500GB SSD. Nothing else was really going on the server at the time. I changed concurrency from 2 to 10, that crashed very fast, so I restarted the machine and didn't change concurrency again. Still crashed a few times but it wasn't as quick to do so. Weird thing was that it didn't just crash the docker container, but the whole machine became unresponsive.

And thank you, I'll try the metadata extraction via this method.

[maelstrm-code]

Hello, please use EXIF tool to check for the date of the images.

Also, have the jobs finished processing?

Hi Alex,

I'm having a similar issue and would appreciate your insights.

My file appears to have appropriate metadata - does it look like there something wrong with the file's metadata?
Alternatively are there some extra metadata tags it might be worth adding to Immich to parse the value?

Per the path, this has been placed in "1991\04-April" which aligns with the file modified time (an odd value on the file) rather than the Media Create date of 2014-12-24.

ExifTool Version Number         : 13.00
File Name                       : IMG_5677.MOV
Directory                       : D:/Photos/library/User/1991/04-April
File Size                       : 5.9 MB
File Modification Date/Time     : 1991:04:17 21:05:03+10:00
File Access Date/Time           : 2024:10:31 14:23:02+11:00
File Creation Date/Time         : 2024:10:22 17:32:10+11:00
File Permissions                : -rw-rw-rw-
File Type                       : MOV
File Type Extension             : mov
MIME Type                       : video/quicktime
Major Brand                     : Apple QuickTime (.MOV/QT)
Minor Version                   : 0.0.0
Compatible Brands               : qt
Media Data Size                 : 5916882
Media Data Offset               : 36
Movie Header Version            : 0
Create Date                     : 2014:12:24 02:53:08
Modify Date                     : 2014:12:24 02:53:09
Time Scale                      : 600
Duration                        : 1.18 s
Preferred Rate                  : 1
Preferred Volume                : 100.00%
Preview Time                    : 0 s
Preview Duration                : 0 s
Poster Time                     : 0 s
Selection Time                  : 0 s
Selection Duration              : 0 s
Current Time                    : 0 s
Next Track ID                   : 3
Track Header Version            : 0
Track Create Date               : 2014:12:24 02:53:08
Track Modify Date               : 2014:12:24 02:53:09
Track ID                        : 1
Track Duration                  : 1.18 s
Track Layer                     : 0
Track Volume                    : 0.00%
Image Width                     : 1280
Image Height                    : 720
Clean Aperture Dimensions       : 1280x720
Production Aperture Dimensions  : 1280x720
Encoded Pixels Dimensions       : 1280x720
Graphics Mode                   : ditherCopy
Op Color                        : 32768 32768 32768
Compressor ID                   : avc1
Source Image Width              : 1280
Source Image Height             : 720
X Resolution                    : 72
Y Resolution                    : 72
Compressor Name                 : H.264
Bit Depth                       : 24
Video Frame Rate                : 239.831
Matrix Structure                : 1 0 0 0 1 0 0 0 1
Media Header Version            : 0
Media Create Date               : 2014:12:24 02:53:08
Media Modify Date               : 2014:12:24 02:53:09
Media Time Scale                : 44100
Media Duration                  : 1.23 s
Media Language Code             : und
Balance                         : 0
Handler Class                   : Data Handler
Handler Vendor ID               : Apple
Handler Description             : Core Media Data Handler
Audio Format                    : mp4a
Audio Bits Per Sample           : 16
Audio Sample Rate               : 44100
Layout Flags                    : Mono
Audio Channels                  : 1
Purchase File Format            : mp4a
Software Version                : 8.1
Content Create Date             : 2014:12:24 13:23:08+10:30
Model                           : iPhone 6
Make                            : Apple
Handler Type                    : Metadata Tags
Creation Date (und-AU)          : 2014:12:24 13:23:08+10:30
Model (und-AU)                  : iPhone 6
Software (und-AU)               : 8.1
Make (und-AU)                   : Apple
Creation Date                   : 2014:12:24 13:23:08+10:30
Software                        : 8.1
Image Size                      : 1280x720
Megapixels                      : 0.922
Avg Bitrate                     : 40.1 Mbps
Rotation                        : 180

[primetime-sudo]

Another example, this time it actually shows a date:

And it looks like the exif data may in fact be wrong? For some reason there's no "date taken" showing here so I can understand why this one is showing this way. To be fair to Immich, I believe this to be a majority of the problems.

ExifTool Version Number : 12.77
File Name : IMG_2469.JPG
Directory : Z:/[[redacted]]/iphone
File Size : 474 kB
File Modification Date/Time : 2019:03:13 15:01:12-07:00
File Access Date/Time : 2024:03:04 13:54:09-08:00
File Creation Date/Time : 2019:03:13 15:01:12-07:00
File Permissions : -rw-rw-rw-
File Type : JPEG
File Type Extension : jpg
MIME Type : image/jpeg
Exif Byte Order : Big-endian (Motorola, MM)
Orientation : Horizontal (normal)
X Resolution : 72
Y Resolution : 72
Resolution Unit : inches
Y Cb Cr Positioning : Centered
Exif Version : 0221
Components Configuration : Y, Cb, Cr, -
Flashpix Version : 0100
Color Space : sRGB
Exif Image Width : 1440
Exif Image Height : 1440
Scene Capture Type : Standard
Compression : JPEG (old-style)
Thumbnail Offset : 298
Thumbnail Length : 10218
Image Width : 1440
Image Height : 1440
Encoding Process : Baseline DCT, Huffman coding
Bits Per Sample : 8
Color Components : 3
Y Cb Cr Sub Sampling : YCbCr4:2:0 (2 2)
Image Size : 1440x1440
Megapixels : 2.1
Thumbnail Image : (Binary data 10218 bytes, use -b option to extract)
-- press ENTER --

[werner-j]

Hello.

Since I'm struggling as well with the EXIF data, and many files that my Synology Photos was still able to recognize Immich is not able to. These files have the change, creation and inode change date in the metadata, and immich won't properly put them to the timeline. Hence, many files are left as being taken 1st of January...

I created a little shell script that prepares my files for processing with immich and adds a recognized tag from the above thread, if none is present. It uses thereby several other possible tags, last resort is change date but the GPSDate might also serve.

I hope this helps someone, until immich is able to handle more exif date/time tags for the files (for me this is > 100 photos that are left without date, compared to Synology Photos) in a sample. In total I have around a thousand without proper date.

https://github.com/werner-j/immich-datetime-fix/

This is by no means perfect and just started out of the need as shell script. But it works for me so far.

[Fethbita]

For those who are looking for a solution, the above script works but since it uses exiftool in a bash script, it is quite slow for thousands of files. Check exiftool common mistake 3. I instead ran the following command that only copies and fixes files that do not have a date (the files that have a date are not copied), which is much faster:

exiftool -r \
  -if 'not $SubSecDateTimeOriginal and not $DateTimeOriginal and not $SubSecCreateDate and not $CreationDate and not $CreateDate and not $SubSecMediaCreateDate and not $MediaCreateDate' \
  '-SubSecCreateDate="1970:01:01 00:00:01"' \
  '-SubSecCreateDate<FileModifyDate' \
  '-SubSecCreateDate<Filename' \
  '-SubSecCreateDate<GPSDateStamp' \
  '-SubSecCreateDate<SubSecModifyDate' \
  '-SubSecCreateDate<ModifyDate' \
  '-SubSecCreateDate<GPSDateTime' \
  '-Directory=output_folder' \
  input_folder

One difference is that this does not copy the files if the files are not valid image/video files (for example .AAE files) and if possible uses the date from the filename (exiftool FAQ entry).

[adamef93]

For those who are looking for a solution, the above script works but since it uses exiftool in a bash script, it is quite slow for thousands of files. Check exiftool common mistake 3. I instead ran the following command that only copies and fixes files that do not have a date (the files that have a date are not copied), which is much faster:

exiftool -r \
  -if 'not $SubSecDateTimeOriginal and not $DateTimeOriginal and not $SubSecCreateDate and not $CreationDate and not $CreateDate and not $SubSecMediaCreateDate and not $MediaCreateDate' \
  '-SubSecCreateDate="1970:01:01 00:00:01"' \
  '-SubSecCreateDate<FileModifyDate' \
  '-SubSecCreateDate<Filename' \
  '-SubSecCreateDate<GPSDateStamp' \
  '-SubSecCreateDate<SubSecModifyDate' \
  '-SubSecCreateDate<ModifyDate' \
  '-SubSecCreateDate<GPSDateTime' \
  '-Directory=output_folder' \
  input_folder

One difference is that this does not copy the files if the files are not valid image/video files (for example .AAE files) and if possible uses the date from the filename (exiftool FAQ entry).

Came across this solution while trying to import a bunch of photos from a Facebook export. All the videos have correct date information while the photos don't and there doesn't seem to be any rhyme or reason as to why. I ran this script and got tons of errors such as Month '42' out of range 1..12 while trying to process a small subfolder of FB photos. The same error occurs with different values (like another one was Day '45' out of range 1..31). Any idea what might be causing this?

The script one reply above yours didn't work for me at all. It didn't even find any images to process

[Fethbita]

The script tries to find the creation date in “FileModifyDate”, “Filename”, “GPSDateStamp”… and if it can, sets that to the SubSecCreateDate which i’m much can use. If your photos don’t have that information, then it won’t work. But if they do, maybe you can remove the problematic line, so if the date is parsed from the filename, and the filename is not actually a date, you can remove the line that has “Filename” in it and try again on a fresh copy.

[adamef93]

Ah, that's probably it then because the file names are all random strings. I've also since found out that FB removes metadata on photos upon upload, but apparently doesn't do the same with videos. I found a tool last night that can apparently reconcile metadata with JSON data from the export kinda like Google Takeout, but I originally exported as HTML. Waiting for another export to finish processing so I can try again