Support systems with Docker #189
Comments
|
Hi there, I'm pretty much a passer by looking for projects I use that I can contribute to. Anyway this looks like an interesting one to look into so I am just going to butt in here. First since you can see the path I am assuming you are doing backups in the command line via I installed docker and while pulling a container ran the same thing with the following output occuring: ------------------------------------------------------------------------------
Creating new snapshot...(RSYNC)
Saving to device: /dev/sdb2, mounted at path: /
Linking from snapshot: 2018-05-08_19-34-05
Synching files with rsync...
Created control file: /timeshift/snapshots/2018-05-08_19-38-01/info.json
RSYNC Snapshot saved successfully (8s)
Tagged snapshot '2018-05-08_19-38-01': ondemand
------------------------------------------------------------------------------Would you be able to provide a more detailed output as well? |
|
@divanvisagie However, when I stop docker manually with On my work laptop that has Ubuntu 18.04, the directory that timeshift chooses for updates, when the docker is running, is actually different: |
|
That's rather interesting, from poking around in the code it looks like the path is determined from an lsblk: https://github.com/teejee2008/timeshift/blob/master/src/Utility/Device.vala#L236 Would you be able to run the command with the debug flag on and post the output both with docker running and not running? So thats That should spit out a lot of detail like this: D: Main()
D:
D: Running Timeshift v18.4
D:
D: Session log file: /var/log/timeshift/2018-05-10_19-25-54_list-snapshots.log
D: Distribution: ManjaroLinux 17.1.10 (Hakoila)
D: DIST_ID: ManjaroLinux
D: Main: check_dependencies()
D: Main: add_default_exclude_entries()
D: Main: add_default_exclude_entries(): exit
D: update_partitions()
D: df -T -B1
D: Device: get_disk_space_using_df(): 4
D: Device: get_mounted_filesystems_using_mtab(): 4
D: Device: get_filesystems(): 11
D: partition list updated
D: detect_system_devices()
D: /boot/efi is mapped to device: /dev/sdb1, UUID=5922-523D
D: / is mapped to device: /dev/sdb2, UUID=9c2b7cc3-cb09-40c6-babb-20ed0ccfda75
D: Searching subvolume for system at path: /
D: Users: root divan
D: Encrypted home users:
D: Encrypted home dirs:
D: Encrypted private dirs:
D: Main: load_app_config()
App config loaded: /etc/timeshift.json
D: IconManager: init()
D: bin_path: /usr/bin/timeshift
D: found images directory: /usr/share/timeshift/images
D: Main(): ok
D: AppConsole: parse_arguments()
D: Main: initialize_repo()
D: backup_uuid=9c2b7cc3-cb09-40c6-babb-20ed0ccfda75
D: backup_parent_uuid=
D: Setting snapshot device from config file
D: repo: creating from uuid
D: SnapshotRepo: from_uuid(): RSYNC
D: uuid=9c2b7cc3-cb09-40c6-babb-20ed0ccfda75
D: SnapshotRepo: init_from_device()
D:
D: SnapshotRepo: unlock_and_mount_devices()
D: device=/dev/sdb2
D: SnapshotRepo: unlock_and_mount_device()
D: device=/dev/sdb2
D: Device: get_mounted_filesystems_using_mtab(): 4
D: SnapshotRepo: load_snapshots()
D: loading snapshots from '/timeshift/snapshots': 14 found
D: SnapshotRepo: unlock_and_mount_device(): exit
D: Selected snapshot device: /dev/sdb2
D: Free space: 156.3 GB
D: SnapshotRepo: check_status()
D: SnapshotRepo: available()
D: is_available: ok
D: SnapshotRepo: has_snapshots()
D: SnapshotRepo: has_space()
D: df -T -B1 '/dev/sdb2'
D: Device: get_disk_space_using_df(): 1
D: SnapshotRepo: check_status(): exit
D: SnapshotRepo: init_from_device(): exit
D: SnapshotRepo: from_uuid(): exit
D: Main: initialize_repo(): exit
D: AppConsole: start_application()
D: SnapshotRepo: check_status()
D: SnapshotRepo: available()
D: is_available: ok
D: SnapshotRepo: has_snapshots()
D: SnapshotRepo: has_space()
D: df -T -B1 '/dev/sdb2'
D: Device: get_disk_space_using_df(): 1
D: SnapshotRepo: check_status(): exit
Device : /dev/sdb2
UUID : 9c2b7cc3-cb09-40c6-babb-20ed0ccfda75
Path : /
Mode : RSYNC
Device is OK
14 snapshots, 156.3 GB free
D: SnapshotRepo: has_snapshots()
Num Name Tags Description
------------------------------------------------------------------------------
# I deleted this to make it anonymous, you should to if you have comments
D: exit_app()
D: crontab -l
D: Failed to read cron tab
D: crontab -l
D: Failed to read cron tab
D: Cron task exists: /etc/cron.d/timeshift-hourly
D: unmount_target_device()
D: clean_logs()
``
|
|
With docker running: After having stopped docker, it looks like this: |
|
Also, here is a bit more information about my system. If I stop the docker service, the second mount point is gone. |
|
There are two situations when running docker daemon: 1 - Docker using AUFS storage driver From Docker's documentation: "...AUFS cannot use the following backing filesystems: aufs, btrfs, or ecryptfs. This means that the filesystem which contains /var/lib/docker/aufs cannot be one of these filesystem types..." 2 - Docker using Btrfs storage driver From Docker's documentation: "...btrfs requires a dedicated block storage device such as a physical disk. This block device must be formatted for Btrfs and mounted into /var/lib/docker/..." It should explain the different behaviors on Ubuntu X Linux Mint. When doing backups, TimeShift should recursively snapshot the subvolumes in /var/lib/docker, (if Docker is using Btrfs). @ilya40umov Q2 - Can you confirm if there are Btrfs subvolumes in /var/lib/docker with |
|
@rafaelsoaresbr On LM machine that I have with me right now:
P.S. I'll try comparing these results with the Ubuntu installation later this week. |
|
BTW I've installed Linux Mint 19 on |
|
@ilya40umov |
Wish I knew that before I started upgrading Mint 18 to Mint 19. :) But is it a problem? I don't mind timeshift using my docker directory. |
I retract my previous opinion. The snapshots are not available when using the live USB installer to run TimeShift. Since this is a part of Linux Mint now, and docker is indispensible for webdevelopers, perhaps @clefebvre can take a look at this. |
|
Used to have this problem as well w/ Docker & Timeshift, where snapshots were not available. I am now able to run Docker, Timeshift, and Ubuntu together, cohesively, and successfully restored multiple snapshots, using LVM (can check LVM option during OS install). Note, without LVM option checked, the snapshots are not available. (haven't installed btrfs) Snapshots are available directly in /timeshift/snapshots directory, and no need to run a live USB installer. Works normal, and have reproduced this multiple times now on various installs. This has been confirmed to work on Ubuntu / Xubuntu 16.04, and I would imagine other versions should work as well. |
|
I posted a bounty for this issue if anyone wants to take it on. |
|
Just use LVM option when you install. |
|
@Darkladder That's a handy workaround to know, but it would not help the users who already have installations in place. There's also #227 which seems related. |
|
Hi, folks I submitted the pull request #289 . It solved the issue with Timeshift+Docker on my station (Linux Mint 19 + Docker + Timeshift). Timeshift was putting snapshots on the /var/lib/docker/plugins. If this does not work for you, I think I can help you narrow down the Timeshift issue. Fell free to fork my changes in https://github.com/bazoocaze/timeshift, compile, test, and then submit the output of the
|
|
Is it possible to let the user select the backup-folder in "Settings"? |
|
@DariusQuintus that's not really optimal, as the static location makes restoring a snapshot from the Live USB installation media a breeze. |
|
@Redsandro Hm. Okay. I don't know about the underlying concept of timeshift. But regarding usability it would be nice to specify a destination folder during backups and a source folder during recovery. That could possibly prevent further mtab-related issues. |
|
@DariusQuintus I disagree, it is actually nice that it just works. During early stages of Linux Mint 19 I had to recover an image from the Live USB dozens of times. I'm very glad I didn't have to 'pick' the snapshot folder every time, find out I changed the snapshot folder half way through making snapshots, etc. We don't write issue reports to Debian to ask if we can customize the location of That said, I do think that the snapshot directory should have been FSH compliant by using However, this kind of changes should not be done during a release cycle. Linux Mint is LTS. |
|
@Redsandro Unfortunately, because of the timeshift bug on binding mounts #227, the snapshot folder is dangling between folders, so it's not static nor fixed for the affected users. On my station, the snapshot folder is dangling between In fact, the users asking for "a way to select the backup-folder" are searching for a solution for this bug, that is, the snapshot folder is not static/fixed in the correct location on systems using bind mount for some use case. I submitted #289 for a workaround for timeshift incorrectly selecting the Docker folder for snapshots. |
|
@bazoocaze I detect some passive aggression in your response. There is no need for that. Because the above was a response and not just a statement, there is some implicit context:
So far my response to UX. I also outlined two further important rules concerning directories and bugfixes. If you're just looking for a solution and are not interested in a discussion about UX, FHS or LTS, then here's the workaround I'm using: |
|
@Redsandro Sadly, stopping the docker deamon does not work. Because docker already altered mtab, at least on my machine: The problem seems to be that timeshift reads mtab, finds the device (in my case "sdb2"), uses "/", finds another entry and finally uses "/var/lib/docker". The only temporary solution I came around is, to install docker on a separate filesystem (which could be wise anyway). Considering your anwser, I would suggest integrating an option into the UI of timeshift (and of course a corresponding command for CLI) where the user can use the "standard location" or a "custom location". |
|
@DariusQuintus this is interesting. The |
|
I get the same result as @DariusQuintus, stopping docker doesn't work for me. Although stopping docker does remove some mtab entries, |
I'm sorry for that. It was not my intention to sound aggressive and mean. Sorry! |
|
#289 merged |
|
@jakewarren how's about the bounty? |
|
@bazoocaze I'll be honest with you, I haven't been through the entire process on BountySource so I am not entirely sure how it works.. I believe this issue will have to be closed and then you can submit a claim. I tested v18.8 and the fix seems to be working for me. 👍 Thank you! |
|
BTW I can confirm that the latest Timeshift is behaving correctly on my Ubuntu laptop and I no longer need to stop docker daemon as a workaround. |
|
I'm running Timeshift 18.4 on Linux Mint 19 and is working as expected on a btrfs volume. |
|
Hi, @rafaelsoaresbr There is a bug in timeshift (#227) that makes timeshift selects (and prioritize) bind mount points to determine the snapshots directory. So if you select a file system as the destination snapshot device, and if you have a bind mount point on that file system, then timeshift will incorrectly select the bind mount point as the path for the snapshots directory. Docker create such bind mounts in There is a workaround for Docker since #289 . |
The #289 workaround makes timeshift ignores mount points containing '/docker' substring when determining the path for the snapshots directory. |
|
Gotcha, thank you.
Edit: alpine image works, hello-world don't. |
|
@rafaelsoaresbr Docker images/overlays are not supported by the Timeshift snapshot engine as per the readme. I guess you must rebuild hello-world container. |
|
Works as expected on EXT4 w/ LVM. LVM or BTRFS should work as intended with Docker installed. If you're not using LVM or BTRFS, you should expect issues. The reason why it works with these two format types, apparently, is due to abstracted volume handling provided via LVM or BTRFS. It's a wild guess, but it works. |
|
@Darkladder As I said in my previous comment, Btrfs isn't working as expected, but it's ok, you just need to pull and/or rebuild your docker images. |
|
There are docker image now |
Hey, thanks for creating this amazing software! (It has saved my day a few times already).
I know it's currently unsupported, but since Docker is being used by lots of the developers nowadays, it would be really nice to have support systems that are running Docker daemon (no need to support timeshift inside of Docker).
E.g. right now if docker daemon is running, Timeshift shows Path being
/var/lib/docker/plugins.It's probably due to Docker mounting plugins directory
As a workaround right now I manually stop docker service before taking a snapshot / doing restore, but as a result I can't do scheduled backups.
The text was updated successfully, but these errors were encountered: