When using R2, pruning mistakenly delete backups in different folders within the same bucket

[WonderfulSoap]

I have two database instance. So I ran two mysql-backup programs to back them up to to different directories within the same bucket.

• DB_DUMP_TARGET=s3://db-backup-test-bucket/node1
• DB_DUMP_TARGET=s3://db-backup-test-bucket/node2

And I found that when I start the second mysql-backup program, it mistakenly deleted the backups generated by the first mysql-backup program.

Below are the two docker compose files I used to reproduce the issue. The only difference between the two docker compose files is that DB_DUMP_TARGET specifies different paths (I chose a 1 minute interval and a maximum retention of 3 backup files for testing)

docker-compose-node1.yaml It will backup db into s3://db-backup-test-bucket/node1

version: "3.7"

services:
  mysql:
    image: mysql:8.4
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: password
      MYSQL_DATABASE: app
      MYSQL_USER: app
      MYSQL_PASSWORD: password
    volumes:
        - ./data:/var/lib/mysql

  dbbackup:
    image: databack/mysql-backup:1.2.1
    command: dump
    restart: always
    depends_on:
      - mysql
    environment:
      - DB_SERVER=mysql
      - DB_USER=root
      - DB_PASS=password
      - DB_DUMP_FREQUENCY=1
      - DB_DUMP_RETENTION=3c
      - DB_DUMP_TARGET=s3://db-backup-test-bucket/node1 # the only difference between two docker-compose.yaml
      - DB_DEBUG=true
      - AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxx
      - AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxx
      - AWS_ENDPOINT_URL=https://xxxxxxxxxxxxxxx.r2.cloudflarestorage.com
      - AWS_REGION=auto

docker-compose-node2.yaml It will backup db into s3://db-backup-test-bucket/node2

version: "3.7"

services:
  mysql:
    image: mysql:8.4
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: password
      MYSQL_DATABASE: app
      MYSQL_USER: app
      MYSQL_PASSWORD: password
    volumes:
        - ./data:/var/lib/mysql

  dbbackup:
    image: databack/mysql-backup:1.2.1
    command: dump
    restart: always
    depends_on:
      - mysql
    environment:
      - DB_SERVER=mysql
      - DB_USER=root
      - DB_PASS=password
      - DB_DUMP_FREQUENCY=1
      - DB_DUMP_RETENTION=3c
      - DB_DUMP_TARGET=s3://db-backup-test-bucket/node2 # the only difference between two docker-compose.yaml
      - DB_DEBUG=true
      - AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxx
      - AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxx
      - AWS_ENDPOINT_URL=https://xxxxxxxxxxxxxxx.r2.cloudflarestorage.com
      - AWS_REGION=auto

First, use docker compose -f docker-compose-node1.yaml up -d to start the node1. After three minutes, three backup files correctly appeared in the ./node1 folder of the bucket.

Then use docker compose -f docker-compose-node2.yaml up -d to start the node2. After node2 start up, the mysql-backup program in the node2 will delete backup files in folder ./node1. It seems that the MySQL backup program erroneously deleted backup files in other folders. Could you please confirm this?
Here is the debug log after starting the second docker-compose.yaml. It shows that the program appears to have incorrectly scanned the node1 folder and deleted its contents.

dbbackup-1  | time="2025-04-16T12:14:52Z" level=debug msg="starting dump"
dbbackup-1  | time="2025-04-16T12:14:52Z" level=info msg="beginning dump 2025-04-16T12:14:52Z" run=ae058a41-cc28-4354-a5e8-a9705a59f040
dbbackup-1  | time="2025-04-16T12:14:52Z" level=debug msg="uploading via protocol s3 from db_backup_2025-04-16T12:14:52Z.tgz to db_backup_2025-04-16T12:14:52Z.tgz" run=ae058a41-cc28-4354-a5e8-a9705a59f040
dbbackup-1  | time="2025-04-16T12:14:53Z" level=debug msg="completed copying 617 bytes" run=ae058a41-cc28-4354-a5e8-a9705a59f040
dbbackup-1  | time="2025-04-16T12:14:53Z" level=info msg="beginning prune" run=ae058a41-cc28-4354-a5e8-a9705a59f040
dbbackup-1  | time="2025-04-16T12:14:53Z" level=debug msg="pruning target s3://db-backup-test-bucket/node2" run=ae058a41-cc28-4354-a5e8-a9705a59f040
dbbackup-1  | time="2025-04-16T12:14:53Z" level=debug msg="checking filename that is standard backup pattern: node1/db_backup_2025-04-16T12:13:16Z.tgz" run=ae058a41-cc28-4354-a5e8-a9705a59f040
dbbackup-1  | time="2025-04-16T12:14:53Z" level=debug msg="checking filename that is standard backup pattern: node1/db_backup_2025-04-16T12:13:55Z.tgz" run=ae058a41-cc28-4354-a5e8-a9705a59f040
dbbackup-1  | time="2025-04-16T12:14:53Z" level=debug msg="checking filename that is standard backup pattern: node1/db_backup_2025-04-16T12:14:14Z.tgz" run=ae058a41-cc28-4354-a5e8-a9705a59f040
dbbackup-1  | time="2025-04-16T12:14:53Z" level=debug msg="checking filename that is standard backup pattern: node2/db_backup_2025-04-16T12:14:52Z.tgz" run=ae058a41-cc28-4354-a5e8-a9705a59f040
dbbackup-1  | time="2025-04-16T12:14:53Z" level=debug msg="Adding candidate file node1/db_backup_2025-04-16T12:13:16Z.tgz:" run=ae058a41-cc28-4354-a5e8-a9705a59f040
dbbackup-1  | time="2025-04-16T12:14:54Z" level=debug msg="pruning 1 files from target s3://db-backup-test-bucket/node2" run=ae058a41-cc28-4354-a5e8-a9705a59f040
dbbackup-1  | time="2025-04-16T12:15:51Z" level=info msg="beginning dump 2025-04-16T12:15:51Z" run=e3fb2527-67e4-4f49-ba77-b5e799f19854
dbbackup-1  | time="2025-04-16T12:15:51Z" level=debug msg="uploading via protocol s3 from db_backup_2025-04-16T12:15:51Z.tgz to db_backup_2025-04-16T12:15:51Z.tgz" run=e3fb2527-67e4-4f49-ba77-b5e799f19854
dbbackup-1  | time="2025-04-16T12:15:52Z" level=debug msg="completed copying 617 bytes" run=e3fb2527-67e4-4f49-ba77-b5e799f19854
dbbackup-1  | time="2025-04-16T12:15:52Z" level=info msg="beginning prune" run=e3fb2527-67e4-4f49-ba77-b5e799f19854
dbbackup-1  | time="2025-04-16T12:15:52Z" level=debug msg="pruning target s3://db-backup-test-bucket/node2" run=e3fb2527-67e4-4f49-ba77-b5e799f19854
dbbackup-1  | time="2025-04-16T12:15:52Z" level=debug msg="checking filename that is standard backup pattern: node1/db_backup_2025-04-16T12:14:14Z.tgz" run=e3fb2527-67e4-4f49-ba77-b5e799f19854
dbbackup-1  | time="2025-04-16T12:15:52Z" level=debug msg="checking filename that is standard backup pattern: node1/db_backup_2025-04-16T12:15:13Z.tgz" run=e3fb2527-67e4-4f49-ba77-b5e799f19854
dbbackup-1  | time="2025-04-16T12:15:52Z" level=debug msg="checking filename that is standard backup pattern: node2/db_backup_2025-04-16T12:14:52Z.tgz" run=e3fb2527-67e4-4f49-ba77-b5e799f19854
dbbackup-1  | time="2025-04-16T12:15:52Z" level=debug msg="checking filename that is standard backup pattern: node2/db_backup_2025-04-16T12:15:51Z.tgz" run=e3fb2527-67e4-4f49-ba77-b5e799f19854
dbbackup-1  | time="2025-04-16T12:15:52Z" level=debug msg="Adding candidate file node1/db_backup_2025-04-16T12:14:14Z.tgz:" run=e3fb2527-67e4-4f49-ba77-b5e799f19854
dbbackup-1  | time="2025-04-16T12:15:53Z" level=debug msg="pruning 1 files from target s3://db-backup-test-bucket/node2" run=e3fb2527-67e4-4f49-ba77-b5e799f19854
dbbackup-1  | time="2025-04-16T12:16:49Z" level=info msg="beginning dump 2025-04-16T12:16:49Z" run=c10aff59-3355-4f01-bfba-a109de1c1599
dbbackup-1  | time="2025-04-16T12:16:49Z" level=debug msg="uploading via protocol s3 from db_backup_2025-04-16T12:16:49Z.tgz to db_backup_2025-04-16T12:16:49Z.tgz" run=c10aff59-3355-4f01-bfba-a109de1c1599
dbbackup-1  | time="2025-04-16T12:16:50Z" level=debug msg="completed copying 616 bytes" run=c10aff59-3355-4f01-bfba-a109de1c1599
dbbackup-1  | time="2025-04-16T12:16:50Z" level=info msg="beginning prune" run=c10aff59-3355-4f01-bfba-a109de1c1599
dbbackup-1  | time="2025-04-16T12:16:50Z" level=debug msg="pruning target s3://db-backup-test-bucket/node2" run=c10aff59-3355-4f01-bfba-a109de1c1599
dbbackup-1  | time="2025-04-16T12:16:50Z" level=debug msg="checking filename that is standard backup pattern: node1/db_backup_2025-04-16T12:15:13Z.tgz" run=c10aff59-3355-4f01-bfba-a109de1c1599
dbbackup-1  | time="2025-04-16T12:16:50Z" level=debug msg="checking filename that is standard backup pattern: node1/db_backup_2025-04-16T12:16:12Z.tgz" run=c10aff59-3355-4f01-bfba-a109de1c1599
dbbackup-1  | time="2025-04-16T12:16:50Z" level=debug msg="checking filename that is standard backup pattern: node2/db_backup_2025-04-16T12:15:51Z.tgz" run=c10aff59-3355-4f01-bfba-a109de1c1599
dbbackup-1  | time="2025-04-16T12:16:50Z" level=debug msg="checking filename that is standard backup pattern: node2/db_backup_2025-04-16T12:16:49Z.tgz" run=c10aff59-3355-4f01-bfba-a109de1c1599
dbbackup-1  | time="2025-04-16T12:16:50Z" level=debug msg="Adding candidate file node1/db_backup_2025-04-16T12:15:13Z.tgz:" run=c10aff59-3355-4f01-bfba-a109de1c1599
dbbackup-1  | time="2025-04-16T12:16:51Z" level=debug msg="pruning 1 files from target s3://db-backup-test-bucket/node2" run=c10aff59-3355-4f01-bfba-a109de1c1599

[deitch]

Well, ouch. That isn't good!

Thanks for posting the debug log, that should help.

[deitch]

Good catch. We missed the path in one place. Going to put out a fix, would appreciate if you test it as well before cutting a patch release.

[deitch]

Fix is in. There should be a digest-based tag ready in a few minutes, which you can use to test. If it passes, we can cut a patch release.

[deitch]

It is out. databack/mysql-backup:25cb740969e8d823fe35cdd290560d6eaa3bc795

[WonderfulSoap]

@deitch After testing, the multi-directory is now being processed correctly. What a excellent work!

[deitch]

Very good to know. I cut v1.2.2. Should be out within a few minutes as CI runs.