Skip to content

Commit

Permalink
Merge pull request #4705 from MichaelEischer/snapshot-statistics
Browse files Browse the repository at this point in the history 
Store snapshot statistics & print snapshot size
  • Loading branch information
@MichaelEischer
MichaelEischer committed Mar 28, 2024
2 parents 71b6284 + e71660c commit 7f9ad1c
Show file tree
Hide file tree
Showing 16 changed files with 369 additions and 160 deletions.
12 changes: 12 additions & 0 deletions changelog/unreleased/issue-693
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
Enhancement: Support printing snapshot size in `snapshots` command

The `snapshots` command now supports printing the snapshot size for snapshots
created using this or a future restic version. For this, the `backup` command
now stores the backup summary statistics in the snapshot.

The text output of the `snapshots` command only shows the snapshot size. The
other statistics are only included in the JSON output. To inspect these
statistics use `restic snapshots --json` or `restic cat snapshot <snapshotID>`.

https://github.com/restic/restic/issues/693
https://github.com/restic/restic/pull/4705
6 changes: 4 additions & 2 deletions cmd/restic/cmd_backup.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -451,6 +451,7 @@ func runBackup(ctx context.Context, opts BackupOptions, gopts GlobalOptions, ter
}

timeStamp := time.Now()
backupStart := timeStamp
if opts.TimeStamp != "" {
timeStamp, err = time.ParseInLocation(TimeFormat, opts.TimeStamp, time.Local)
if err != nil {
Expand Down Expand Up @@ -640,6 +641,7 @@ func runBackup(ctx context.Context, opts BackupOptions, gopts GlobalOptions, ter
snapshotOpts := archiver.SnapshotOptions{
Excludes: opts.Excludes,
Tags: opts.Tags.Flatten(),
BackupStart: backupStart,
Time: timeStamp,
Hostname: opts.Host,
ParentSnapshot: parentSnapshot,
Expand All @@ -649,7 +651,7 @@ func runBackup(ctx context.Context, opts BackupOptions, gopts GlobalOptions, ter
if !gopts.JSON {
progressPrinter.V("start backup on %v", targets)
}
_, id, err := arch.Snapshot(ctx, targets, snapshotOpts)
_, id, summary, err := arch.Snapshot(ctx, targets, snapshotOpts)

// cleanly shutdown all running goroutines
cancel()
Expand All @@ -663,7 +665,7 @@ func runBackup(ctx context.Context, opts BackupOptions, gopts GlobalOptions, ter
}

// Report finished execution
progressReporter.Finish(id, opts.DryRun)
progressReporter.Finish(id, summary, opts.DryRun)
if !gopts.JSON && !opts.DryRun {
progressPrinter.P("snapshot %s saved\n", id.Str())
}
Expand Down
14 changes: 14 additions & 0 deletions cmd/restic/cmd_snapshots.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import (
"strings"

"github.com/restic/restic/internal/restic"
"github.com/restic/restic/internal/ui"
"github.com/restic/restic/internal/ui/table"
"github.com/spf13/cobra"
)
Expand Down Expand Up @@ -163,6 +164,11 @@ func PrintSnapshots(stdout io.Writer, list restic.Snapshots, reasons []restic.Ke
keepReasons[*id] = reasons[i]
}
}
// check if any snapshot contains a summary
hasSize := false
for _, sn := range list {
hasSize = hasSize || (sn.Summary != nil)
}

// always sort the snapshots so that the newer ones are listed last
sort.SliceStable(list, func(i, j int) bool {
Expand Down Expand Up @@ -198,6 +204,9 @@ func PrintSnapshots(stdout io.Writer, list restic.Snapshots, reasons []restic.Ke
tab.AddColumn("Reasons", `{{ join .Reasons "\n" }}`)
}
tab.AddColumn("Paths", `{{ join .Paths "\n" }}`)
if hasSize {
tab.AddColumn("Size", `{{ .Size }}`)
}
}

type snapshot struct {
Expand All @@ -207,6 +216,7 @@ func PrintSnapshots(stdout io.Writer, list restic.Snapshots, reasons []restic.Ke
Tags []string
Reasons []string
Paths []string
Size string
}

var multiline bool
Expand All @@ -228,6 +238,10 @@ func PrintSnapshots(stdout io.Writer, list restic.Snapshots, reasons []restic.Ke
multiline = true
}

if sn.Summary != nil {
data.Size = ui.FormatBytes(sn.Summary.TotalBytesProcessed)
}

tab.AddRow(data)
}

Expand Down
56 changes: 29 additions & 27 deletions doc/045_working_with_repos.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,41 +18,43 @@ Working with repositories
Listing all snapshots
=====================

Now, you can list all the snapshots stored in the repository:
Now, you can list all the snapshots stored in the repository. The size column
only exists for snapshots created using restic 0.17.0 or later. It reflects the
size of the contained files at the time when the snapshot was created.

.. code-block:: console
$ restic -r /srv/restic-repo snapshots
enter password for repository:
ID Date Host Tags Directory
----------------------------------------------------------------------
40dc1520 2015-05-08 21:38:30 kasimir /home/user/work
79766175 2015-05-08 21:40:19 kasimir /home/user/work
bdbd3439 2015-05-08 21:45:17 luigi /home/art
590c8fc8 2015-05-08 21:47:38 kazik /srv
9f0bc19e 2015-05-08 21:46:11 luigi /srv
ID Date Host Tags Directory Size
-------------------------------------------------------------------------
40dc1520 2015-05-08 21:38:30 kasimir /home/user/work 20.643GiB
79766175 2015-05-08 21:40:19 kasimir /home/user/work 20.645GiB
bdbd3439 2015-05-08 21:45:17 luigi /home/art 3.141GiB
590c8fc8 2015-05-08 21:47:38 kazik /srv 580.200MiB
9f0bc19e 2015-05-08 21:46:11 luigi /srv 572.180MiB
You can filter the listing by directory path:

.. code-block:: console
$ restic -r /srv/restic-repo snapshots --path="/srv"
enter password for repository:
ID Date Host Tags Directory
----------------------------------------------------------------------
590c8fc8 2015-05-08 21:47:38 kazik /srv
9f0bc19e 2015-05-08 21:46:11 luigi /srv
ID Date Host Tags Directory Size
-------------------------------------------------------------------
590c8fc8 2015-05-08 21:47:38 kazik /srv 580.200MiB
9f0bc19e 2015-05-08 21:46:11 luigi /srv 572.180MiB
Or filter by host:

.. code-block:: console
$ restic -r /srv/restic-repo snapshots --host luigi
enter password for repository:
ID Date Host Tags Directory
----------------------------------------------------------------------
bdbd3439 2015-05-08 21:45:17 luigi /home/art
9f0bc19e 2015-05-08 21:46:11 luigi /srv
ID Date Host Tags Directory Size
-------------------------------------------------------------------
bdbd3439 2015-05-08 21:45:17 luigi /home/art 3.141GiB
9f0bc19e 2015-05-08 21:46:11 luigi /srv 572.180MiB
Combining filters is also possible.

Expand All @@ -64,21 +66,21 @@ Furthermore you can group the output by the same filters (host, paths, tags):
enter password for repository:
snapshots for (host [kasimir])
ID Date Host Tags Directory
----------------------------------------------------------------------
40dc1520 2015-05-08 21:38:30 kasimir /home/user/work
79766175 2015-05-08 21:40:19 kasimir /home/user/work
ID Date Host Tags Directory Size
------------------------------------------------------------------------
40dc1520 2015-05-08 21:38:30 kasimir /home/user/work 20.643GiB
79766175 2015-05-08 21:40:19 kasimir /home/user/work 20.645GiB
2 snapshots
snapshots for (host [luigi])
ID Date Host Tags Directory
----------------------------------------------------------------------
bdbd3439 2015-05-08 21:45:17 luigi /home/art
9f0bc19e 2015-05-08 21:46:11 luigi /srv
ID Date Host Tags Directory Size
-------------------------------------------------------------------
bdbd3439 2015-05-08 21:45:17 luigi /home/art 3.141GiB
9f0bc19e 2015-05-08 21:46:11 luigi /srv 572.180MiB
2 snapshots
snapshots for (host [kazik])
ID Date Host Tags Directory
----------------------------------------------------------------------
590c8fc8 2015-05-08 21:47:38 kazik /srv
ID Date Host Tags Directory Size
-------------------------------------------------------------------
590c8fc8 2015-05-08 21:47:38 kazik /srv 580.200MiB
1 snapshots
Expand Down
41 changes: 40 additions & 1 deletion doc/075_scripting.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,9 @@ Summary is the last output line in a successful backup.
+---------------------------+---------------------------------------------------------+
| ``tree_blobs`` | Number of tree blobs |
+---------------------------+---------------------------------------------------------+
| ``data_added`` | Amount of data added, in bytes |
| ``data_added`` | Amount of (uncompressed) data added, in bytes |
+---------------------------+---------------------------------------------------------+
| ``data_added_packed`` | Amount of data added (after compression), in bytes |
+---------------------------+---------------------------------------------------------+
| ``total_files_processed`` | Total number of files processed |
+---------------------------+---------------------------------------------------------+
Expand Down Expand Up @@ -551,11 +553,48 @@ The snapshots command returns a single JSON object, an array with objects of the
+---------------------+--------------------------------------------------+
| ``program_version`` | restic version used to create snapshot |
+---------------------+--------------------------------------------------+
| ``summary`` | Snapshot statistics, see "Summary object" |
+---------------------+--------------------------------------------------+
| ``id`` | Snapshot ID |
+---------------------+--------------------------------------------------+
| ``short_id`` | Snapshot ID, short form |
+---------------------+--------------------------------------------------+

Summary object

The contained statistics reflect the information at the point in time when the snapshot
was created.

+---------------------------+---------------------------------------------------------+
| ``backup_start`` | Time at which the backup was started |
+---------------------------+---------------------------------------------------------+
| ``backup_end`` | Time at which the backup was completed |
+---------------------------+---------------------------------------------------------+
| ``files_new`` | Number of new files |
+---------------------------+---------------------------------------------------------+
| ``files_changed`` | Number of files that changed |
+---------------------------+---------------------------------------------------------+
| ``files_unmodified`` | Number of files that did not change |
+---------------------------+---------------------------------------------------------+
| ``dirs_new`` | Number of new directories |
+---------------------------+---------------------------------------------------------+
| ``dirs_changed`` | Number of directories that changed |
+---------------------------+---------------------------------------------------------+
| ``dirs_unmodified`` | Number of directories that did not change |
+---------------------------+---------------------------------------------------------+
| ``data_blobs`` | Number of data blobs |
+---------------------------+---------------------------------------------------------+
| ``tree_blobs`` | Number of tree blobs |
+---------------------------+---------------------------------------------------------+
| ``data_added`` | Amount of (uncompressed) data added, in bytes |
+---------------------------+---------------------------------------------------------+
| ``data_added_packed`` | Amount of data added (after compression), in bytes |
+---------------------------+---------------------------------------------------------+
| ``total_files_processed`` | Total number of files processed |
+---------------------------+---------------------------------------------------------+
| ``total_bytes_processed`` | Total number of bytes processed |
+---------------------------+---------------------------------------------------------+


stats
-----
Expand Down

0 comments on commit 7f9ad1c

Please sign in to comment.