Skip to content

Update backup instructions #583

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Apr 18, 2024
Merged

Update backup instructions #583

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
dd8f056
updated desktop backup instructions
NoahMaizels Apr 18, 2024
ef1a787
updated backup instructions
NoahMaizels Apr 18, 2024
1062db9
improve swarm desktop backup instructions
NoahMaizels Apr 18, 2024
b384beb
remove clef, improve backup instructions
NoahMaizels Apr 18, 2024
d1c1234
revise command
NoahMaizels Apr 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
145 changes: 35 additions & 110 deletions docs/bee/working-with-bee/backups.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@ id: backups

## Files

A full Bee node backup includes the `keys`, `localstore`, `statestore`, and `password` files. The node should be stopped before taking a backup and not restarted until restoring the node from the backup to prevent the node from getting out of sync with the network.
A full Bee node backup includes the `kademlia-metrics`, `keys`, `localstore`, `password`, `stamperstore`, `statestore`, and `password` files. The node should be stopped before taking a backup and not restarted until restoring the node from the backup to prevent the node from getting out of sync with the network.

Node key and state data is found in the data directory specified in its [configuration](configuration).
A node's data including keys and stamp data are found in the data directory specified in its [configuration](configuration).

Key data in backup files allows access to Bee node's Gnosis account. If lost or stolen it could lead to the loss of all assets in that account. Multiple backups should be kept in secure locations.
Key data in backup files allows access to Bee node's Gnosis account. If lost or stolen it could lead to the loss of all assets in that account. Furthermore the `stamperstore` contains postage stamp data, and postage stamps will not be recoverable if it is lost.

:::info
Don't forget - it's not a backup until you're sure the backup files work! Make sure to test restoring from backup files to prevent loss of assets due to data loss or corruption.
Expand All @@ -29,6 +29,8 @@ It may also be useful to include the `bee.yaml` config file in a backup so that
/etc/bee
```

This guide uses the default package manager location for the data folder, make sure to change the commands to match your data folder's location if it in a different directory.

### Binary package install

If you installed Bee using the [automated shell script](/docs/bee/installation/install#shell-script-install) or by [building Bee from source](/docs/bee/installation/build-from-source), your data directory will typically be located at:
Expand All @@ -47,83 +49,80 @@ Use `docker cp` to retrieve the contents of these folders:
docker cp bee_bee_1:/home/bee/.bee/ bee
```

## Data types
### Data types

The data directory contains three directories. Its default location depends on the node install method used.

Shell script install:
For shell script install the location is `/home/<user>/.bee` and for package manager installs it is `/var/lib/bee`. The directory structure is as follows:

```
/home/<user>/.bee
├── keys
│   ├── libp2p.key
│   ├── pss.key
│   └── swarm.key
├── localstore
├── kademlia-metrics
│   └── ...
└── statestore
└── ...
```

Package manager install:

```
/var/lib/bee
├── keys
│   ├── libp2p.key
│   ├── libp2p_v2.key
│   ├── pss.key
│   └── swarm.key
├── localstore
│   ├── indexstore
│   └── sharky
├── password
├── stamperstore
│   └── ...
└── statestore
└── ...
│   └── ...
```


### Keys

The `keys` directory contains three key files: `libp2p_v2.key`, `pss.key`, and `swarm.key`. These keys are generated during the Bee node's initialisation and are the most important data to retain for a backup.
The `keys` directory contains three key files: `libp2p.key`, `libp2p_v2.key`, `pss.key`, `swarm.key`,. These keys are generated during the Bee node's initialisation and are required for maintaining access to your node.

:::danger
The `swarm.key` file allows access to Bee node's Gnosis account. If the key is lost or stolen it could lead to the loss of all assets secured by that key. Multiple backups should be kept in secure locations to prevent loss of assets or unauthorized access.
The `swarm.key` file allows access to Bee node's Gnosis Chain account. If the key is lost or stolen it could lead to the loss of all assets secured by that key.
:::

:::info
To use `swarm.key` to manage the Gnosis account for a node through Metamask or other wallets, [exportSwarmKeys](https://github.com/ethersphere/exportSwarmKey) can be used to convert `swarm.key` to a compatible format.
:::

### Statestore
### Statestore and Localstore.

The `statestore` directory retains information related to the node,
including SWAP balances, info on peers, block list, postage stamps, and more.
The `statestore` retains data related to its operation, and the `localstore` contains chunks locally which are frequently requested, pinned in the node, or are in the node's neighbourhood of responsibility.

:::info
As the data in `statestore` and `localstore` continually changes during normal operation of a node, when taking a backup the node should first be stopped and not re-connected to the Swarm network until restoring from the backup (otherwise the `statestore` and `localstore` files will get out of sync with the network). It is possible to restore using out of sync `statestore` and `localstore` files, however it may lead to data loss or unexpected behavior related to chunk uploads, postage stamps, and more.
:::

### Localstore
## Back-up your node data

The `localstore` directory contains chunks locally which are frequently requested, pinned in the node, or are in the node's neighbourhood of responsibility.

## Backup Your node
Copy entire `bee` data folder to fully backup your node. This will do a full backup of `kademlia-metrics`, `keys`, `localstore`, `stamperstore`, `password`, and `statestore`, files into a newly created `/backup` directory. Make sure to save the backup directory to a safe location.


Copy entire `bee` data folder to fully backup node. This will do a full backup of `statestore`. `localstore`, and `key` files into the newly created `/backup` directory. Make sure to save the backup directory to a safe location.
```
mkdir backup
sudo cp -r /var/lib/bee/ backup
```

### Export keys
### Back-up your password

Depending on your configuration, your password may not be located in the `/bee` data directory which was copied in the previous step. If it has been specified in an environment variable or in your [`bee.yaml` configuration file](/docs/bee/working-with-bee/configuration#default-data-and-config-directories), make sure to copy it and save it together with the rest of your backup files or write it down in a safe place.

If you only need to export your node's blockchain keys, you need to export the `swarm.key` UTC / JSON keystore file and the `password` file used to encrypt it. First create a directory for your keys and then export, make sure to save the newly created `keystore` directory in a safe location.

### Back-up blockchain keys only

If you only need to export your node's blockchain keys, you need to export the `swarm.key` UTC / JSON keystore file and the `password` file used to encrypt it. First create a directory for your keys and then copy your keys to that directory.


```bash
mkdir keystore
sudo cp -r /var/lib/bee/keys/swarm.key /var/lib/bee/password keystore

```

## Metamask Import

If you wish to import your blockchain account to a wallet such as Metamask, you can simply print out your keystore file and password and use those data to import into the wallet:

### View key and password for wallet import

```bash
Expand All @@ -135,15 +134,11 @@ sudo cat /var/lib/bee/password
Note that `swarm.key` is in UTC / JSON keystores format and is encrypted by default by your password file inside the `/bee` directory. Make sure to export both the `swarm.key` file and the `password` file in order to secure your wallet. If you need your private key exported from the keystore file, you may use one of a variety of Ethereum wallets which support exporting private keys from UTC files (such as [Metamask](https://metamask.io/), however we offer no guarantees for any software, make sure you trust it completely before using it).
:::

## Metamask Import

If you wish to import your blockchain account to a wallet such as Metamask, you can simply print out your keystore file and password and use those data to import into the wallet:

### Get private key from keystore and password

To import to Metamask:

1. Get your `swarm.key` and `password` as described in the section above.
1. View and copy your `swarm.key` and `password` as shown above
2. Go to Metamask and click "Account 1" --> "Import Account"
3. Choose the "Select Type" dropdown menu and choose "JSON file"
4. Paste the password (Make sure to do this first)
Expand All @@ -169,13 +164,7 @@ Before restoring, make sure to check for any old node data at `/var/lib/bee` fro

1. Install Bee. See [install](/docs/bee/installation/install/) page for more info.

1. Change ownership of `bee` data folder.

```
sudo chown -R /var/lib/bee
```

1. Delete statestore, keys, localstore, and password files.
1. Delete `/bee` folder which was generated during install

```
sudo rm -r /var/lib/bee
Expand All @@ -184,7 +173,7 @@ Before restoring, make sure to check for any old node data at `/var/lib/bee` fro
1. Navigate to backup directory and copy files to data folder.

```
cp -r /<path-to-backup>/bee /var/lib/
sudo cp -r /<path-to-backup>/. /var/lib/bee
```

1. Revert ownership of the data folder.
Expand All @@ -196,67 +185,3 @@ Before restoring, make sure to check for any old node data at `/var/lib/bee` fro
sudo systemctl restart bee
sudo journalctl --lines=100 --follow --unit bee
```

## Import Clef Keys

[Bee Clef](https://github.com/ethersphere/bee-clef) is the deprecated external signer for Bee. Node operators are recommended to remove Clef from their setup in order to ensure their nodes continue to operate smoothly through future updates. Below is a guide for importing Clef keys into a non-Clef setup. See the [#node-operators Discord channel](https://discord.com/channels/799027393297514537/811553590170353685) for more support from other node operators and channel admins.

### Step 1: From /bee-clef directory, print out keystore and password and keystore and save for the next step:

*Notice: You may need to `sudo chown -R user:user /bee-clef` if you do not already have access*

:::danger
This is a throwaway account so we are sharing the keystore and password for this guide, never share your keystore or password like this!
:::

![](/img/clef_import_01.png)

Follow the steps shown in the screenshot above to print out clef keystore.

#### Keystore (throwaway example):
Copy the output shown in the screenshot:

```json
{"address":"1199e0674661ed795ead3182d5a5407e5d609612","crypto":{"cipher":"aes-128-ctr","ciphertext":"bcd47f40e970658f10be215833acebc60e5826935484be75009bb43c71472a7a","cipherparams":{"iv":"84137379119a5547888cfc8ea425ce7d"},"kdf":"scrypt","kdfparams":{"dklen":32,"n":4096,"p":6,"r":8,"salt":"d555e66cbe0779d33eb088f50a345c6da5eed6f84ac6f418f9b0b731d721d2ac"},"mac":"48e7b3967d804ab54cd3610070d74a45f7f2eca4195eb6e82f289c4ce3332383"},"id":"e2e2eef3-8fd2-40f5-8f5d-a54c716110db","version":3}
```

#### Password

Again copy the output, make sure you are printing the password file in `/passwords` whose title matches the address of the account you are exporting (i.e., after the `bee-0_` portion of the password file name the address of the wallet is printed starting with `1199e...` )

`nd_UePSEWHLHa8drvl4TYR1W2QPNlKov`


### Step 2: Import Clef Keys

#### Check your settings

Make sure you have removed clef related entries from your settings, such as `clef-signer-enable` and `clef-signer-endpoint`

#### Import keys

Navigate to your bee install location. For our example it is installed at the default `/var/lib/bee`:

*Notice: You may need to `sudo chown -R user:user /bee` if you do not already have access*

:::danger
From here you may have accounts here which you have not yet backed up. As the following steps to import your bee-clef keys will erase any keys you have currently, make sure to [backup](https://docs.ethswarm.org/docs/bee/working-with-bee/backups#keys) any keys you already have here.
:::

#### Import password

From `/var/lib/bee` open your password file using `vi` and overwrite it with the password you copied from step 1, and save the changes. *(Make sure you have the original password backed up if you want to save the account already there.)*

![](/img/clef_import_02.png)

#### Import keystore

First navigate to `/var/lib/bee/keys` and delete the `libp2p_v2.key` and `pss.key` files using `rm libp2p_v2.key pss.key`.

Next, open the `swarm.key` file using `vi` and paste the keystore you copied from the first step, then save and close the file.

Finally, give permissions back to `bee` with:
```
sudo chown -R bee:bee /bee
```
Now your import is complete, simply restart your bee to allow it to regenerate your `pss.key` and `libp2p_v2.key` files
50 changes: 39 additions & 11 deletions docs/desktop/backup-restore.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,48 +5,76 @@ id: backup-restore

## Create a Backup

To create a backup of your Bee node in Swarm Desktop, begin by navigating to the ***Settings*** tab in the app and copying the location of the data directory as indicated in the ***Data DIR*** field:
To create a backup of your Bee node in Swarm Desktop, start by shutting down your node.

Right click the Bee icon in the System tray and select `Stop Bee` and then `Quit` to close and exit from Swarm Desktop:

![](/img/backup2.png)


Next navigate to the `Settings` tab in the app and copy the location of the data directory as indicated in the `Data DIR` field:

![](/img/backup1.png)

Navigate to the directory you just copied and create copies of the ***localstore***, ***statestore***, and ***keys*** folders and store them in a secure and private location. The ***keys*** folder is the most important folder as it contains your Bee node's private keys, and you will lose access to your Bee node and its assets if those keys are lost.
Navigate to the directory you just copied and create copies of all the files in that directory (`\data-dir`), including `localstore`, `statestore`, `stamperstore`, `kademlia-metrics` and `keys` folders and store them in a secure and private location.

![](/img/backup7.png)

In addition to the data folders, you will also need the password found in the ***config.yaml*** file in order to restore a Bee node from backup. Move up one directory from ***Data DIR*** to the ***Data*** directory, and create a copy of the ***config.yaml*** file and save it along with the other folders you just backed up:
In addition to the data folders, you will also need the password found in the `config.yaml` file in order to restore a Bee node from backup. Move up one directory from `Data DIR` to the `Data` directory, and create a copy of the `config.yaml` file and save it along with the other folders you just backed up:

![](/img/backup4.png)

Alternatively you may open the ***config.yaml*** and save the password as a text file along with the rest of your backup files:
Alternatively you may open the `config.yaml` and save the password as a text file along with the rest of your backup files:

![](/img/backup5.png)

Your completed backup should have three folders and one file (***localstore***, ***statestore***, ***keys*** and ***config.yaml***/***password.txt*** ) and look like this:
Your completed backup should contain all the files from your data directory as well as your password (either in your `config.yaml` file or as a separate file or written down.)

![](/img/backup8.png)

### Back-up Gnosis Chain Key Only

If you only wish to back-up your Gnosis Chain key, navigate to the `\data-dir\keys` directory, and copy the `swarm.key` to a safe location:

![](/img/backup9.png)


You also need the password found in the `config.yaml` file in order to access your Gnosis Chain account. Move up one directory from `Data DIR` to the `Data` directory, and create a copy of the `config.yaml` file and save it along with the other folders you just backed up:

![](/img/backup4.png)

Alternatively you may open the `config.yaml` and save the password as a text file along with the rest of your backup files:

![](/img/backup5.png)

## Restore from Backup

To restore from backup, begin with a [new install](/docs/desktop/install) of Swarm Desktop. Once the installation process is finished, navigate to the ***Settings*** tab in the app and copy the install file directory as indicated in the ***Data DIR*** field:
To restore from backup, begin with a [new install](/docs/desktop/install) of Swarm Desktop. Once the installation process is finished, navigate to the `Settings` tab in the app and copy the install file directory as indicated in the `Data DIR` field:

![](/img/backup1.png)

Before navigating to the directory you just copied, right click the Bee icon in the System tray and select ***Stop Bee*** and then ***Quit*** to close and exit from Swarm Desktop:
Before navigating to the directory you just copied, right click the Bee icon in the System tray and select `Stop Bee` and then `Quit` to close and exit from Swarm Desktop:

![](/img/backup2.png)

Next open your file explorer and navigate to the directory you just copied. Delete the ***localstore***, ***statestore***, and ***keys*** (if present) folders, and replace them with your own backup copies of these folders:
Next open your file explorer and navigate to the directory you just copied. Delete any files present in the directory, and replace them with your own backup copies (excluding the `config.yaml` / password file):

![](/img/backup3.png)
![](/img/backup7.png)

Move up one directory from `Data DIR` to `Data`, and replace delete the `config.yaml` file and replace it with the `config.yaml` file from your backup.

Move up one directory from ***Data DIR*** to ***Data***, and open the ***config.yaml*** file in a text editor such as VS Code or a plaintext editor:
Alternatively if you have saved just the password and not the entire config file, open the default `config.yaml` file in a text editor such as VS Code or a plain text editor:

![](/img/backup4.png)

![](/img/backup5.png)

Replace the ***password*** string with your own password from the ***config.yaml*** backup.
Replace the `password` string with your own password which you saved from the `config.yaml` backup.

Restart Swarm Desktop and check to see if the backup was restored successfully:

![](/img/backup6.png)

### Restore Gnosis Chain Account

If you only wish to access your Gnosis Chain account, you can [follow these instructions](/docs/bee/working-with-bee/backups#metamask-import) for exporting to Metamask in order to access your account.
4 changes: 2 additions & 2 deletions openapi/SwarmCommon.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ components:
$ref: "#/components/schemas/PublicKey"

BigInt:
description: Numeric string that represents integer which might exceeds `Number.MAX_SAFE_INTEGER` limit (2^53-1)
description: Numeric string that represents integer which might exceed `Number.MAX_SAFE_INTEGER` limit (2^53-1)
type: string
example: "1000000000000000000"

Expand Down Expand Up @@ -1172,4 +1172,4 @@ components:
content:
application/problem+json:
schema:
$ref: "#/components/schemas/ProblemDetails"
$ref: "#/components/schemas/ProblemDetails"
Loading