Skip to content

Commit

Permalink
update docs, config, libs, ioutil, sshUser
Browse files Browse the repository at this point in the history
  • Loading branch information
gedge committed Feb 14, 2023
1 parent 4fbdcbb commit 2a8aedc
Show file tree
Hide file tree
Showing 11 changed files with 119 additions and 120 deletions.
69 changes: 35 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,19 +18,19 @@ git clone git@github.com:ONSdigital/dp-cli.git

**Required:**

The DP CLI uses Go Modules so requires a go version of **1.11** or later (ideally the latest)
The DP CLI uses Go Modules so requires a go version of **1.19** or later (ideally the latest)

Ensure `session-manager-plugin` is installed by running the following command

```shell
which session-manager-plugin
```
which session-manager-plugin
```

if not installed, follow this [doc](https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-install-plugin.html#install-plugin-macos)

**Optional:**
#### Optional but common requirements

The following are only required for some functionality of this tool.
The following are only required for some common functionality of this tool.

In order to use the `dp ssh` sub-command you will need:

Expand All @@ -42,12 +42,14 @@ In order to use the `dp ssh` sub-command you will need:

- [`dp-ci`](https://github.com/ONSdigital/dp-ci) cloned locally:

```bash
```shell
git clone git@github.com:ONSdigital/dp-ci
```

Note: Make sure `dp-setup` and `dp-ci` are on the branch `main` (or `awsb` for `dp-setup`), locally. This is necessary as they have the required SSH configuration and the relevant inventories.

#### Optional and less common CMD requirements

In order to use the `dp import cmd` sub-command (e.g. when you are using **Neo4j**; however, `import` is currently *not needed* if you are using Neptune) you will need:

- [`dp-code-list-scripts`](https://github.com/ONSdigital/dp-code-list-scripts) cloned locally:
Expand All @@ -64,8 +66,10 @@ In order to use the `dp import cmd` sub-command (e.g. when you are using **Neo4j

### Configuration

Configuration is defined in a YAML file. By default the CLI expects the config file to be `~/.dp-cli-config.yml`.
The config file location can be customised by setting the `DP_CLI_CONFIG` environment variable to your chosen path.
Configuration is defined in a YAML file:

- By default the CLI expects the config file to be `~/.dp-cli-config.yml`.
- The config file location can be customised by setting the `DP_CLI_CONFIG` environment variable to your chosen path.

The [sample config file](./config/example_config.yml) should be tailored to suit you. For example:

Expand All @@ -74,27 +78,17 @@ cp -i config/example_config.yml ~/.dp-cli-config.yml
vi ~/.dp-cli-config.yml
```

update the paths and `ssh-user`:
update the paths and `user-name`:

```yaml
dp-setup-path: path to your local dp-setup
dp-ci-path: path to your local dp-ci
dp-hierarchy-builder-path: path to your local dp-hierarchy-builder-path
dp-code-list-scripts-path: path to your local dp-code-list-scripts-path
ssh-user: Your first and last name concatenated eg. JaneBloggs"
user-name: Your first and last name concatenated eg. JaneBloggs"
```

and if this is a first time setup, comment out `prod` from environments, thus:

```yaml
#- name: prod
# profile: dp-prod
# user: ubuntu
```

You can uncomment them when you get `prod` access.

*Note*: **ssh-user** is a string mainly now used to put your name against SecurityGroup changes.
You can uncomment the `environments` values as and when you get access to them.

### Brew Installation

Expand Down Expand Up @@ -204,6 +198,7 @@ Use the available commands for more info on the functionality available.
```shell
$ dp ssh sandbox
ssh to sandbox
...
```

If the SSH or SCP command fails, ensure that the `dp remote allow` command has been run for the environment you want to connect to.
Expand All @@ -214,7 +209,11 @@ If the SSH or SCP command fails, ensure that the `dp remote allow` command has b

Ensure you have `region=eu-west-2` in your AWS configuration.

Depending on the command you're trying to run, and what you're trying to access, ensure your `AWS_PROFILE` is set correctly and there is no prod/sandbox/ci config added in the `~/.aws/credentials` file.
Depending on the command you're trying to run, and what you're trying to access:

- ensure your `AWS_PROFILE` is set correctly
- and there is no dp-prod/dp-sandbox/dp-ci config in the `~/.aws/credentials` file.

Example:

```yaml
Expand All @@ -223,35 +222,35 @@ export AWS_PROFILE=dp-staging

#### Remote Allow security group rule already exists error

```sh
```shell
$ dp remote allow sandbox
[dp] allowing access to sandbox
Error: error adding rules to bastionSG: InvalidPermission.Duplicate: the specified rule "peer: X.X.X.X/32, TCP, from port: 22, to port: 22, ALLOW" already exists
status code: 400, request id: 26a61345-8391-4c65-bfd7-4f0052892b6b
```

The error occurs when rules have previously been added and the command is run again.
Use `dp remote deny $env` to clear out existing rules and try again.
Use (e.g.) `dp remote deny sandbox` to clear out existing rules and try again.

*This error should no longer appear* - the code should now avoid re-adding existing rules.
Note: *This error should no longer appear* - the code should now avoid re-adding existing rules.
However, it is possible that the rule has been added with a description that does not match your username.
If so, you will have to use the AWS web UI/console to remove any offending security group rules.
If so, you will have to use the AWS web UI/console to remove any offending Security Group rules.

### Advanced use

#### ssh commands

You can run ssh commands from the command-line, for example to determine the time on a given host:

```sh
```shell
$ dp ssh sandbox web 1 date
[...motd banner...]
[result of date command]
[result of date command when run on remote host]
```

:warning: However, if you wish to include *flags* in the (remote) command, you must tell `dp` to stop looking for flags - use the `--` flag:

```sh
```shell
$ dp ssh sandbox web 1 -- ls -la
[...]
```
Expand Down Expand Up @@ -301,9 +300,11 @@ When creating new releases, please be sure to:

```ini
[profile dp-sandbox]
sso_start_url = https://ons.awsapps.com/start
sso_account_id = 1234556253 #replace this with correct account id
sso_role_name = AdministratorAccess
sso_region = eu-west-2
region = eu-west-2
sso_start_url = https://ons.awsapps.com/start
sso_account_id = 1234556253 # replace this with correct account id (from above URL)
sso_role_name = AdministratorAccess
sso_region = eu-west-2
region = eu-west-2
```

Repeat the above for any other environments (with the appropriate profile-name and account-id changes).
4 changes: 2 additions & 2 deletions ansible/groups.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import (
"bufio"
"bytes"
"io"
"io/ioutil"
"os"
"path/filepath"
"strings"
)
Expand All @@ -26,7 +26,7 @@ func GetGroupsForEnvironment(dpSetUpPath, environment string) ([]string, error)

func readFile(dpSetUpPath, environment string) ([]byte, error) {
hostsPath := filepath.Join(dpSetUpPath, "ansible/inventories", environment, "hosts")
return ioutil.ReadFile(hostsPath)
return os.ReadFile(hostsPath)
}

func parseFile(fileBytes []byte) ([]string, error) {
Expand Down
Loading

0 comments on commit 2a8aedc

Please sign in to comment.