Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for VSS (windows) #340

Open
frederich opened this Issue Nov 4, 2015 · 62 comments

Comments

Projects
None yet
@frederich
Copy link

frederich commented Nov 4, 2015

The restic backup runs. In the mean time my work is going on. Restic come to the point where it saves my Outlook archive. But Outlook is still open, restic fails.

[2Kscanned 77389 directories, 924469 files in 7:49 GiBerror for e:\home\jf\archive\outlook\archive.pst: SaveFile() chunker.Next(): read e:\home\jf\archive\outlook\archive.pst: Der Prozess kann nicht auf die Datei zugreifen, da ein anderer Prozess e
@fd0

This comment has been minimized.

Copy link
Member

fd0 commented Nov 4, 2015

Hm, what do other programs on Windows typically do? It looks like restic is denied opening the file...

@episource

This comment has been minimized.

Copy link
Contributor

episource commented Nov 4, 2015

The way to go is Volume Shadow Copy Service (VSS) [1, 2]. It creates (at least) crash consistent copy-on-write snapshots and allows safely copying opened files. Additionally The Volume Shadow Copy service notifies compatible applications (like MS SQL Server) that a VSS Snapshot is to be taken so that they can provide consistent data to be included in the snapshot [1].

VSS support could be integrated into restic, but it's also possible to create a VSS snapshot externally and use that as a backup source. [3-7] might be of interest.

[1] https://technet.microsoft.com/en-us/library/ee923636.aspx
[2] https://en.wikipedia.org/wiki/Shadow_Copy
[3] https://technet.microsoft.com/en-us/library/cc754968.aspx
[4] http://flaming-grackles.tumblr.com/post/41091667883/some-notes-on-reinstating-shadow-copies-on-windows
[5] http://www.heise.de/ct/inhalt/2009/09/180/
[6] http://blogs.msdn.com/b/adioltean/archive/2008/02/28/a-simple-way-to-access-shadow-copies-in-vista.aspx
[7] https://technet.microsoft.com/en-us/library/cc772172.aspx

@fd0

This comment has been minimized.

Copy link
Member

fd0 commented Nov 4, 2015

Thanks for linking the documentation! Integrating automatic VSS creation looks like a big endeavour, though.

@episource

This comment has been minimized.

Copy link
Contributor

episource commented Nov 4, 2015

If someone is willing to integrate VSS directly into restic, the VSS API description can be found here.

@fd0 fd0 added enhancement feature and removed bug enhancement labels Apr 16, 2017

@fd0 fd0 changed the title file already in use, backup fails Add support for VSS (windows) Apr 16, 2017

@turnkey-commerce

This comment has been minimized.

Copy link
Contributor

turnkey-commerce commented Jun 11, 2017

To more easily integrate restic with automatic VSS creation I wrote a Windows PowerShell script that can be run to automate this without the complications mentioned above. It is available in the attached zip:
restic-backup-windows-vss.zip

If restic has a repository for external scripts but perhaps it could be included there.

The meat of the script is in these lines excerpted from the script (omitting the config part):

$ShadowPath = $rootVolume + 'shadowcopy\'

# Create a volume shadow copy and make it accessible
$s1 = (Get-WmiObject -List Win32_ShadowCopy).Create($rootVolume, "ClientAccessible")
$s2 = Get-WmiObject Win32_ShadowCopy | Where-Object { $_.ID -eq $s1.ShadowID }
$device  = $s2.DeviceObject + "\"

# Create a symbolic link to the shadow copy
cmd /c mklink /d $ShadowPath "$device"

# Run Restic on the data files in the shadowcopy

ForEach ($folderToBackup in $foldersToBackup) {
    cmd /c $resticExe backup -r $resticRepository ($ShadowPath + $folderToBackup)
}

# Delete the shadow copy and remove the symbolic link
$s2.Delete()
cmd /c rmdir $ShadowPath
@fd0

This comment has been minimized.

Copy link
Member

fd0 commented Jun 11, 2017

Thank you very much! There's no contrib repo of some kind, but I'm sure that interested users will find this issue and your script.

@turnkey-commerce

This comment has been minimized.

Copy link
Contributor

turnkey-commerce commented Jun 11, 2017

You're welcome and thanks for sharing this useful application!

@bwmarrin

This comment has been minimized.

Copy link

bwmarrin commented Jun 12, 2017

That's a nice script @turnkey-commerce :) I've been using normal non-powershell batch file to do this with rsync, but your version is nicer.

On the topic of restic though. Adding VSS support somehow would be really really OMG really nice :) Without that, it's hard to break into the Windows Enterprise segment of backups where VSS is both expected and really needed. Of course, shell script work arounds like above will help mitigate that, especially if decently documented somewhere :)

@Alwaysin

This comment has been minimized.

Copy link

Alwaysin commented Feb 16, 2018

I'm seconding this! We need this at my company!

@jcasale

This comment has been minimized.

Copy link

jcasale commented Mar 25, 2018

Keep in mind when implementing this feature, if you are attempting to access a file in a shadow copy whose provider has data on other volumes in addition to the one you snapshot, the writer will be excluded from the shadow copy unless you add each volume that contains data.

The WMI example won't suffice as the Create signature only facilitates a single volume.

@aventrax

This comment has been minimized.

Copy link

aventrax commented Jun 1, 2018

The script proposed by @turnkey-commerce work great. I have been able to schedule it on machine startup to backup some stuff including locked files. Anyway, I think that VSS must be used by a user with admin privileges (in elevated mode) to work, so restic should be able to manage this as "sudo" does on linux.

Moreover, It's true that the VSS support and a small set of remote commands for clients and repository (API), would be necessary to implement this in a company.

@fd0 : Great job with Restic, I discovered it a few weeks ago and it works great!

@bjoe2k4

This comment has been minimized.

Copy link

bjoe2k4 commented Jun 10, 2018

I am trying to use the Powershell script by @turnkey-commerce, which works great, but i am having trouble writing the restic output to a logfile. No matter what i do, i always end up having a jammed logfile. For example, when replacing line 33 with:

cmd /c $resticExe backup -r $resticRepository ($ShadowPath + $folderToBackup) >> backup.log

there goes something horribly wrong:


�[2K[0:01] 0 files 0 B, total 1 files 11 B, 0 errors

�[2K[0:01] 15 files 199 B, total 25 files 995.782 KiB, 0 errors

�[2K
�[2K[0:01] 15 files 199 B, total 25 files 995.782 KiB, 0 errors
... and so on...

However, when i run a bare restic backup command on cmd.exe and redirect that to a logfile (here having -v -v options) all is fine.

open repository
lock repository
load index files
using parent snapshot 2709b9ff
start scan on [C:\ProgramData]
start backup on [C:\ProgramData]
unchanged /C/ProgramData/.asv2sfi
unchanged /C/ProgramData/.cld2sfi
unchanged /C/ProgramData/.dys1sfi
... and so on...
@mgumz

This comment has been minimized.

Copy link

mgumz commented Jun 24, 2018

@bjoe2k4 try the cmd /c ... part without all the rest and see if your "logging problems" persists.

i suspect that you are just observing something that is unrelated to the VSS part. if you can confirm this by trying out what i suggested, then feel free to open another issue.


towards the backups-on-windows issue: i am not sure that adding the support for vss directly into restic is actually worth the effort: usually one creates one vss-snapshot, then one calls restic multiple times to back things up, maybe some more operations on the snapshot itself and only afterwards one destroys the snapshot (or even keep it). i can not see a way to integrate vss into restic (called multiple times) with that "create the snapshot once" workflow - without adding lots of complexity to restic. the powershell script of @turnkey-commerce is the best fit for the whole problem, actually. similar means are needed to operate on zfs or btrfs snapshots as well.

the real issue would then be to add a section to the restic documentation which describes how the "snapshot of a filesystem" issue can be solved for multiple filesystems. then, obviously, people need to keep that section up-to-date but it's still better than to add complexity to restic itself. imho.

@fd0

This comment has been minimized.

Copy link
Member

fd0 commented Jun 24, 2018

Good point, thanks! I'm also not sure it's a good idea to add this to restic, so I'll move this to the "feature request" stage.

@mgumz

This comment has been minimized.

Copy link

mgumz commented Jun 24, 2018

Well, it is more a "improve the documentation" thing than a feature. Especially if you think about the use cases and the given different filesystems which are able to create a snapshot. Tomorrow someone appears and wants snapshots of ReFS on Windows, UFS2 snapshots on FreeBSD, APFS from Apple etc :)

@bjoe2k4

This comment has been minimized.

Copy link

bjoe2k4 commented Jun 24, 2018

@bjoe2k4 try the cmd /c ... part without all the rest and see if your "logging problems" persists.

The problems persist. I agree that this has nothing to do with VSS, I rather didn't want to open a new issue for this minor problem, as I was expecting some error on my side.

In the meantime I garthered new insights and also a workaround regarding my logging problems. It seems to be a interop problem with powershell and restic. I will open a new issue when i find time.

I also do agree that "managing snapshots of a filesystem" is outside the scope of restic.

@matthijskooijman

This comment has been minimized.

Copy link

matthijskooijman commented Aug 26, 2018

I wonder if this isn't something to put into restic after all. I can see that restic should not be trying to automatically manage snapshots on all kinds of systems. However:

  • I believe that Windows is a lot more picky about reading opened files, making it more important to use snapshotting to get a complete backup (so it's about completeness, not just consistency).
  • VSS is, AFAICS, a generic FS-agnostic feature, that handles this issue completely for the Windows platform, which makes it an implementation with a high bang-for-buck.
  • VSS is, AFAICS from the powershell example, fairly straightforward to get to work (unlike snapshotting on unix system snapshot which AFAIK are either filesystem-specific or operate on the block level, requiring a lot more insight in how the system is set up).

Additionally, doing this natively in restic seems a lot more reliable than from a script (in particular in terms of proper cleanup and guaranteeing a snapshot is removed after usage).

@matthijskooijman

This comment has been minimized.

Copy link

matthijskooijman commented Aug 31, 2018

One more consideration I found when looking more into VSS, is the handling or cleanup. Ideally, you want a non-persistent shadow copy, meaning it gets automatically cleaned up when it is released, which seems like an important feature to prevent shadow copies from piling up and taking up disk space. I assume this also means they are removed when the backup process somehow crashes or otherwise terminates without explicitly cleaning up, though I could not find definitive info on that (though non-persistent shadow copies are always cleaned up on reboot). See auto-release shadow copy on https://docs.microsoft.com/en-us/windows/desktop/vss/vssgloss-ahttps://docs.microsoft.com/en-us/windows/desktop/vss/vssgloss-a and all types on https://docs.microsoft.com/en-us/windows/desktop/api/Vss/ne-vss-_vss_snapshot_context

The powershell script shown above uses a "ClientAccessible" context. I'm not really sure what that means exactly, but it results in a persistent snapshot according to vssadmin list shadows (and I also confirmed that killing restic with ^C leaves the shadow copy behind). I tried modifying it to "Backup" (based on this doc) or "Volatile" (based on this doc), but both of these return code 5, which means "Unsupported shadow copy context". I'm not sure if I'm just using the wrong vocabulary, or non-persistent types are not supported. This post suggests that this type might not be supported for workstations, but I somehow doubt that (also because these docs say that VSS_CTX_BACKUP is the default, and also the only supported value on Windows XP).

A related observation is that you might need a persistent shadow copy to expose a shadow copy on a drive letter or folder. I somewhat suspect that the mklink approach used by the powershell script might also work on non-persistent snapshots. If it does not, that might be another reason to support VSS in restic - so restic can access the shadow copy directly, without needing any drive letter or link (though I think that accessing VSS files is a matter of using a magic filename, like \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy123, but that needs some kind of path-mapping like #1376 suggests)

Note even with the link approach in the above Powershell script there is still a caveat: When trying to backup the entire volume, the path to the link is passed to restic, which does not follow it (which relates to #542).

@fd0

This comment has been minimized.

Copy link
Member

fd0 commented Sep 5, 2018

Another issue that makes me wonder if it is a good idea to have VSS integrated into restic is: how would users configure it? From what you're describing it sounds like there's a high chance that we will run into a configuration nightmare...

@matthijskooijman

This comment has been minimized.

Copy link

matthijskooijman commented Sep 5, 2018

how would users configure it?

I believe that no real configuration is needed. I think Restic could:

  • Figure out what volumes are involved in the backup
  • Create a shadow copy of the backup
  • Backup files from the shadow copy (translating pathnames internally, so there is no need to symlink or mount the shadow copy anywhere)
  • Remove the shadow copy (ideally, it should be autoremoved, but I'm not sure that's always possible, see #340 (comment))

I suspect this is also what most other backup solutions do, though I haven't got much experience (I'm not really a Windows user, though I do have an interest in backup someone else's Windows laptop).

@cfbao

This comment has been minimized.

Copy link

cfbao commented Sep 5, 2018

I agree with @matthijskooijman that no user configuration is required, and his list of steps is also how I imagine restic would work with VSS.

The complications mentioned earlier is more about implementation, in particular the various Windows tools restic may or may not use. Once it's implemented, it should be just one more flag for Windows users to specify if they want to use VSS.

restic only needs to report appropriate errors when things go south, e.g. lack of privilege, volume not eligible for VSS... It's usually expected that such errors are handled outside of the backup program.

@jarmo

This comment has been minimized.

Copy link

jarmo commented Sep 5, 2018

I totally understand everyone's painpoints of Restic not having VSS support built-in and I have to admit that this was also a con for myself as well before starting using Restic. However, I've used Duplicati for years now and I find one of its biggest problems to be that it supports everything. This has had a negative impact to Duplicati since it has gotten really complex over the years - complexity affects its codebase and its UI/GUI too. This means that every new version update might break something. I've had to wait weeks before getting anything restored and I just don't want to have that kind of risk when using a backup software. That's why I started to look for an alternative and found Restic which seems (at this time) to work better and I'm on halfway of switching to it fully. Of course there is a possibility to create complex software, which still works perfectly, but the more features it has the more complex it gets and the more harder it gets to keep it running smoothly. Just my two cents, why I prefer it not to have any OS/file-system specific support out of Go-s own world.

@aventrax

This comment has been minimized.

Copy link

aventrax commented Sep 17, 2018

My slight modified version of @turnkey-commerce script

#
# Restic VSS Backup Script v0.1
#

# Environmental Variables
$env:RESTIC_PASSWORD = "...."
$env:RESTIC_REPOSITORY = "...."



# Paths to backup
$paths = @{}
$paths["C:\"] = @(
    'nginx',
    'test',
    'test with spaces'
)
#$paths["D:\"] = @(
#    'Software'
#)



# -- DO NOT MODIFY ABOVE -- #

ForEach ($item in $paths.GetEnumerator()) {

    # Create the Shadow Copy
    $s1 = (Get-WmiObject -List Win32_ShadowCopy).Create($item.Key, "ClientAccessible")
    $s2 = Get-WmiObject -Class Win32_ShadowCopy | Where-Object { $_.ID -eq $s1.ShadowID }

    $device  = $s2.DeviceObject + "\"
    $ShadowPath = $item.Key + 'VSS\'
    
    # Create a symbolic link to the shadow copy
    cmd /c mklink /d $ShadowPath "$device"

    # Build the new list of folders (incuding the ShadowPath)
    $folder_list = New-Object System.Collections.Generic.List[System.Object]
    ForEach ($path in $item.Value) {
        $p = $ShadowPath + $path
        $folder_list.Add($p)
    }

    # Print the command line
    cmd /c echo Launching: $resticExe -q backup $folder_list

    # Launch Restic
    cmd /c restic.exe -q backup $folder_list
    
    # Delete the shadow copy and remove the symbolic link
    $s2.Delete()
    cmd /c rmdir $ShadowPath
}
@jirib

This comment has been minimized.

Copy link

jirib commented Oct 7, 2018

Another issue that makes me wonder if it is a good idea to have VSS integrated into restic is: how would users configure it? From what you're describing it sounds like there's a high chance that we will run into a configuration nightmare...

For example burp[1] backup uses VSS by default on Windows and nothing has to be done (though the code seems to be originating from Bacula). They also provide a tool to strip VSS from backup files so one can restore files backuped on Windows onto Linux.

[1] https://github.com/grke/burp/search?q=vss&unscoped_q=vss

@bherila

This comment has been minimized.

Copy link

bherila commented Oct 7, 2018

They also provide a tool to strip VSS from backup files so one can restore files backuped on Windows onto Linux.

VSS just provides a consistent snapshot of the fs, it doesn't modify the files in any so I'm not sure what "stripping" does? Still agree that transparent VSS usage would be the friendliest way of using restic on Windows in general however I understand that it may not be the highest priority since it is a platform-specific feature. There are other things that (last I checked anyway) don't work too well in Windows, like FUSE mounting, which I would also love to see working!

@zemb

This comment has been minimized.

Copy link

zemb commented Oct 11, 2018

Can it be documented properly in the guide?

Someone mentioned having VSS in Restic is too complex to develop and maintain. Think about this:

As an ordinary user, I have to read through the comments, find out which solution/workaround is the most actual one and what is the problem with it (otherwise, it would be included in the application, wouldn't it?). Users who do not know how to run existing PowerShell scripts have a bad luck.

Then I have to find other workarounds in other issues like #2004 (comment) and combine them into the script above. Users who do not know how to alter scripts have a bad luck. But this is a recent issue so I am maybe too dramatic, too early.

Users who run Restic according to the Documentation, think they have backups but do not know that they do not contain all of their files, have a bad luck, but they find out only when they want to restore them.

The files that are constantly open are being worked on, and may be exactly those that need regular backups.

Anyway, thank you so much for the tool and the help here and the workarounds.

@mholt

This comment has been minimized.

Copy link
Contributor

mholt commented Nov 23, 2018

After a lot of reading, then many hours of tinkering, and still not knowing fully what is going on, I was able to successfully create a shadow copy using, I think, pure Go, with the help of a couple libraries and some amazing luck.

Here's what I did:

Running cmd as Administrator:

C:\> vssadmin list shadows

vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2013 Microsoft Corp.

No items found that satisfy the query.

Then I ran my Go program:

C:\> createshadowcopy.exe

Then again:

C:\> vssadmin list shadows
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2013 Microsoft Corp.

Contents of shadow copy set ID: {8786dacd-2567-466e-8d13-7711260f1c0e}
   Contained 1 shadow copies at creation time: 11/23/2018 1:30:35 PM
      Shadow Copy ID: {b9a647ff-9f8b-412e-a2f5-1b34d1ed9aad}
         Original Volume: (C:)\\?\Volume{6e7146fd-fb7a-494c-9421-346633d03dc5}\
         Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy2
         Originating Machine: MYMACHINE
         Service Machine: MYMACHINE
         Provider: 'Microsoft Software Shadow Copy provider 1.0'
         Type: ClientAccessible
         Attributes: Persistent, Client-accessible, No auto release, No writers, Differential

It's worth noting that the Go program that did this was compiled from my Mac for Windows, then copied to a Windows VM, so it does still cross-compile.

The Go program also obtained the VSS ID in return.

The Go program must be run as Administrator, however.

Would you like to proceed?

I do have one thing to ask as a favor in return, though, to the Windows users in this thread: How are you using restic to back up mapped network drives, while being run as Administrator? Is there a better way to access drives than by their letters? (UNC paths?) And how do you present those to users? (i.e. Windows Explorer just shows drive letters I think...)

References:

@turnkey-commerce

This comment has been minimized.

Copy link
Contributor

turnkey-commerce commented Nov 23, 2018

@mholt That is very cool indeed! I would think it would be best if your tool could be imported as a package from restic so it can create and also presumably destroy the shadow copy of the local volume(s) that restic is backing up.

To answer your question, local volumes are usually referred to with drive letters and remote volumes are usually referred to with UNC paths. Local shared volumes can be referred to either way on the local system if they are mapped to a letter drive.

So if you're trying to backup network drives you would normally use UNC. Windows explorer is really using that when browsing the network and you can reveal what it is using by clicking in the path bar at the top to change it to a UNC path.

@bherila

This comment has been minimized.

Copy link

bherila commented Nov 23, 2018

Since it's difficult to reliably VSS snapshot the remote machine, I typically recommend running restic on the file server if possible. This is ideal because the file server can be backed up consistently even if other people are using the file share remotely.

@ProactiveServices

This comment has been minimized.

Copy link
Contributor

ProactiveServices commented Nov 23, 2018

@mholt Keep in mind that restic cannot currently restore backups made from UNC paths. ISTR that when I tested a restore from a VSS path I had the same problem.

@mholt

This comment has been minimized.

Copy link
Contributor

mholt commented Nov 23, 2018

@turnkey-commerce

That is very cool indeed! I would think it would be best if your tool could be imported as a package from restic so it can create and also presumably destroy the shadow copy of the local volume(s) that restic is backing up.

Thanks. No need for a separate package though. If that PR happens in the linked issue over at the WMI package, it'll only be a few lines of code: just enough for its own function. If that PR doesn't happen, it's still just an extra function or two. Maybe its own code file. Or a fork of that package with the change.

To answer your question, local volumes are usually referred to with drive letters and remote volumes are usually referred to with UNC paths. Local shared volumes can be referred to either way on the local system if they are mapped to a letter drive.

Interesting, so how does this work with the possibility that drive letters can change, and how does a program which is running as Administrator back up some user's C:\ or Z:\ or (whatever):\ drives?

@ProactiveServices

Keep in mind that restic cannot currently restore backups made from UNC paths. ISTR that when I tested a restore from a VSS path I had the same problem.

Alright, well, that can probably be fixed too.

@turnkey-commerce

This comment has been minimized.

Copy link
Contributor

turnkey-commerce commented Nov 23, 2018

@mholt

Interesting, so how does this work with the possibility that drive letters can change, and how does a program which is running as Administrator back up some user's C:\ or Z:\ or (whatever):\ drives?

Hopefully the drive letter mappings don't change often (especially the C: drive) and a program running as administrator would see the same drive letters as the user on the machine.

@mholt

This comment has been minimized.

Copy link
Contributor

mholt commented Nov 24, 2018

@turnkey-commerce

and a program running as administrator would see the same drive letters as the user on the machine.

Hmm, this is not my experience. When running as administrator, I can't access all the drive letters the user has.

@turnkey-commerce

This comment has been minimized.

Copy link
Contributor

turnkey-commerce commented Nov 24, 2018

@mholt

Hmm, this is not my experience. When running as administrator, I can't access all the drive letters the user has.

Sorry, you're right, I was thinking they are global but shows that I work on single-user machines too much. Most of the times I have had to do something like that I create a UNC mapped drive using symbolic links (mklink /D...) which allows it to be accessible as both a standard and elevated user. However, that's not really drive letters so my bad. Drive letters are indeed a messy thing on Windows!

@mholt

This comment has been minimized.

Copy link
Contributor

mholt commented Nov 26, 2018

Using my change in StackExchange/wmi#45, all that is required to make a shadow copy in Go is this:

var vol string
connectArgs := []interface{}{nil, "root/CIMV2"}

_, err := wmi.CallMethod(connectArgs, "Win32_ShadowCopy", "Create", []interface{}{
	"C:\\",
	"ClientAccessible",
	&vol,
})
if err != nil {
	return err
}

fmt.Println("Shadow copy volume:", vol)

No idea how to clean it up (delete the shadow copy) yet.

@joonas-fi

This comment has been minimized.

Copy link

joonas-fi commented Nov 28, 2018

I'm currently working on a super small library in Go taking cross-platform snapshots. Huge shout-out to @turnkey-commerce for giving me important pointers in #340 (comment) !

On Windows can be done entirely from command line, without PowerShell or (direct use of WMI) crap. This means that all we need on Go side is import "os/exec".

First, create snapshot:

> wmic shadowcopy call create Volume="D:"
Executing (Win32_ShadowCopy)->create()
Method execution successful.
Out Parameters:
instance of __PARAMETERS
{
        ReturnValue = 0;
        ShadowID = "{984628B9-4972-4AF3-8748-E9EC2C810DEC}";
};

Microsoft being dick as usual, we have to go through WMI console because they deliberately cripple non-server versions of Windows by only removing the vssadmin create command. The implementation for snapshots clearly is there, but Microsoft actively wants to make things hard for people by requiring the use of workarounds for non-server OS versions.

You'll get back a GUID (because entreprise software, right?): {984628B9-4972-4AF3-8748-E9EC2C810DEC}

You can use this ID to query vssadmin for the volume path:

> vssadmin list shadows /Shadow={984628B9-4972-4AF3-8748-E9EC2C810DEC}
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2013 Microsoft Corp.

Contents of shadow copy set ID: {2caa3819-940a-42ef-a39f-f01f4c75260d}
   Contained 1 shadow copies at creation time: 28/11/2018 15.08.49
      Shadow Copy ID: {984628b9-4972-4af3-8748-e9ec2c810dec}
         Original Volume: (D:)\\?\Volume{10eaffff-0000-0000-0000-602219000000}\
         Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy2
         Originating Machine: joonas3
         Service Machine: joonas3
         Provider: 'Microsoft Software Shadow Copy provider 1.0'
         Type: ClientAccessible
         Attributes: Persistent, Client-accessible, No auto release, No writers, Differential

From that output we need to locate \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy2

This can be symlinked so the files can be regularly accessed (NOTE: we had add the last backslash - that's important!):

> mklink /D d:\shadow \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy2\
symbolic link created for d:\shadow <<===>> \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy2\

We can now use regular tools of accessing the files in the snapshot:

> dir d:\shadow
 Volume in drive D is Data-SSD
 Volume Serial Number is A876-6BA6

 Directory of d:\shadow

23/11/2018  14.04    <DIR>          Data
               0 File(s)              0 bytes
               1 Dir(s)  179 510 530 048 bytes free

Now we can delete the snapshot and the symlink:

> vssadmin delete shadows /Quiet /Shadow={984628b9-4972-4af3-8748-e9ec2c810dec}
> rmdir D:\shadow

The /Quiet is needed because otherwise it asks you if you really want to remove the snapshot.

@bherila

This comment has been minimized.

Copy link

bherila commented Nov 28, 2018

This is excellent; however, calling the API (as opposed to using wmic or vssadmin) has its advantages:

@joonas-fi

This comment has been minimized.

Copy link

joonas-fi commented Nov 28, 2018

@bherila good point. I feel less dirty about not having to use WMI from Golang though.

I quickly open sourced my package so that if it helps in any way, even for inspiration: https://github.com/function61/bup/tree/master/pkg/fssnapshot

@joonas-fi

This comment has been minimized.

Copy link

joonas-fi commented Nov 29, 2018

My package now transparently supports Linux (via LVM) also. I won't spam this issue more.

@nioncode

This comment has been minimized.

Copy link

nioncode commented Dec 22, 2018

In a way, I feel like restic should not be responsible to cover snapshots on all systems. On Windows, it just needs VSS, but on Linux it might use either LVM or file system specific snapshot technologies (e.g. on btrfs).

However, since there is a universal way for doing snapshots on Windows and this can be added relatively easy through command line invocations, I think it makes sense to include VSS support in restic. On Linux, it is quite trivial to create snapshots before running restic and deleting them afterwards (or keep them for a local history of files).

@cdhowie

This comment has been minimized.

Copy link
Contributor

cdhowie commented Dec 23, 2018

I feel like restic should not be responsible to cover snapshots on all systems.

I think it makes sense to have restic support snapshots everywhere there is a consistent and unified service that will take care of the details. On Windows, this is VSS. On Linux, this doesn't exist yet, but it might some day when systemd decides it needs to do yet another thing.

@MB512

This comment has been minimized.

Copy link

MB512 commented Jan 27, 2019

I found this short looking documentation from MS to create a snapshot for backup. Maybe this could help someone.

https://docs.microsoft.com/en-us/windows/desktop/VSS/simple-shadow-copy-creation-for-backup

@mholt

This comment has been minimized.

Copy link
Contributor

mholt commented Feb 8, 2019

If someone wants to help wrap up my pull request over at StackExchange/wmi#45 then we can probably make this possible in restic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.