Skip to content

Commit

Permalink
feat(import): import aws instance and tag as box (#21)
Browse files Browse the repository at this point in the history
* feat(import): import aws instance and tag as box

* refactor: rename 'override' to 'overwrite'
  • Loading branch information
dwmkerr committed Jan 24, 2024
1 parent 70431cc commit edd2cbc
Show file tree
Hide file tree
Showing 7 changed files with 746 additions and 39 deletions.
59 changes: 23 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ The following commands are available for `boxes`:
- [`boxes connect`](#boxes-list) - opens a box
- [`boxes ssh`](#boxes-list) - helps initiate an SSH connection to a box
- [`boxes costs`](#boxes-costs) - shows the costs accrued by each both this month
- [`boxes import`](#boxes-import) - import and AWS instance and tag as a box

### `boxes list`

Expand Down Expand Up @@ -212,6 +213,19 @@ Shows the current configuration that has been loaded for `boxes`. Can be helpful
}
```
### `boxes import`
Imports an AWS instance and tags as a box, also tags its attached volumes.
```bash
% boxes import i-066771b1f0f0668af ubuntubox
ubox (i-066771b1f0f0668af): imported successfully
```
Options:
- `--overwrite`: overwrite tags on existing instances/volumes
## Configuration
A local `boxes.json` file can be used for configuration. The following values are supported:
Expand Down Expand Up @@ -349,38 +363,27 @@ Development dependencies:
`Argument of type... Types of property '...' are incompatible`
Typically occurs if AWS SDK packages are not at the exact same number as the `@ask-sdk/types` version number. Update the package.json to use exactly the same version between all `@aws-sdk` libraries. Occassionally these libraries are still incompatible, in this case downgrade to a confirmed version that works such as `3.10.0`.
Typically occurs if AWS SDK packages are not at the exact same number as the `@ask-sdk/types` version number. Update the package.json to use exactly the same version between all `@aws-sdk` libraries. Occasionally these libraries are still incompatible, in this case downgrade to a confirmed version that works such as `3.10.0`.
## TODO
Quick and dirty task-list.
### Alpha
- [ ] feat: 'import' option to tag a box and update local config
- [x] bug: stat/stop show 'pending' rather than 'stopped' due to order of logging
- [x] feat: document copy password in connect, maybe better default off
- [x] refactor: suck it up and use TS
- [x] feat: read AWS region from config file, using node-configuration
- [x] npm badge download link
- [x] bug: package.json path
- [x] build / lint / test / deploy pipeline
- [x] screen recording of boxes list / stop / start / connect
- [x] document how 'connect' works
- [x] feat: ssh connect
- [x] docs: make AWS screenshot a bit smaller in readme
- [x] feat: some basic tests
- [x] feat: Cost management tags configuration to allow pricing info TODO check cost allocation report
- [x] build: check coverage working on main
- [x] feat: flag or option to control spend, by enforcing a confirmation for usage of the 'cost' api
- [ ] testing: recreate steam box with cost allocation tag enabled (current cost 0.53 USD)
- [x] feat: 'import' option to tag a box and associated volumes
- [ ] refactor: check use of 'interface' which should be 'type'
- [ ] testing: check ubox cost allocation tags for volumes
- [ ] feat: boxes aws-console opens link eg (https://us-west-2.console.aws.amazon.com/ec2/home?region=us-west-2#InstanceDetails:instanceId=i-043a3c1ce6c9ea6ad)
- [ ] bug: EBS devices not tagged -I've tagged two (manually) in jan - check w/ feb bill
### Beta
- [ ] 'wait' flag for start/stop to wait until operation complete - default to 1hr and document the timeout info
### Publish Blog
- [ ] documentation on cost savings via archival
### Later
- [ ] refactor: 'wait' functions can be generalised to take a predicate that uses AWS calls and then share the same loop/logging/etc
Expand All @@ -394,6 +397,7 @@ Quick and dirty task-list.
- [ ] feat: aws profile in config file
- [ ] epic: 'boxes create' to create from a template
- [ ] refactor: find a better way to mock / inject config (rather than importing arbitrarily)
- [ ] feat(import): save/update local config file
### Epic - Interactive Setup
Expand All @@ -402,20 +406,3 @@ Will add the tags - but will also add the tags to the volumes and will notify if
Creates the local config.
This would be demo-able.
### Epic - Volume Management
- [x] test '-wait' on start/stop and doc
- [x] propagate tags w/test
- [x] 'start' can now check for 'has archived volumes' and restore if available, this is the next big one
- [x] delete tag on volume restore...
- [x] ...so that we can auto restore volumes when calling 'start' - which will need to wait for the volumes to be ready
- [x] auto-restore on start, this is a good incremental point to check-in, even
if backup is only via 'debug' and comes later
- [x] rename functions to 'archive' and 'restore' syntax
- [x] data loss warning and generalise the 'yes' flag
- [x] delete snapshot on successful restore
- [x] better logging for non-debug mode (warn user can take time)
- [ ] calling 'detach/etc' fails if instance is not stopped or stopping as it doesn't try to stop the instance - must fail if state is not stopping or stopped
- [ ] new task list - docs, function, parameters, cost saving info, etc
- [x] complete stop/start unit tests
18 changes: 18 additions & 0 deletions src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ import packageJson from "../package.json";
import { BoxState } from "./box";
import { assertConfirmation } from "./lib/cli-helpers";
import { getConfiguration } from "./configuration";
import { importBox } from "./commands/import";

const ERROR_CODE_WARNING = 1;
const ERROR_CODE_CONNECTION = 2;
Expand Down Expand Up @@ -189,6 +190,23 @@ program
console.log(JSON.stringify(result));
});

program
.command("import")
.description("Import an AWS instance and volumes and tag as a Box")
.argument("<instanceId>", "the aws instance id")
.argument("<boxId>", "the box id to tag the instance with")
.option("-o, --overwrite", "overwrite existing box tags", false)
.action(async (instanceId, boxId, options) => {
await importBox({
boxId,
instanceId,
overwrite: options.overwrite,
});
console.log(
` ${theme.boxId(boxId)} (${instanceId}): imported successfully`,
);
});

async function run() {
try {
// We will quickly check configuration for debug tracing. If it throws we
Expand Down
151 changes: 151 additions & 0 deletions src/commands/import.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
import path from "path";
import {
EC2Client,
DescribeInstancesCommand,
CreateTagsCommand,
} from "@aws-sdk/client-ec2";
import { mockClient } from "aws-sdk-client-mock";
import "aws-sdk-client-mock-jest";
import mock from "mock-fs";

import { importBox } from "./import";
import describeInstancesResponse from "../fixtures/import-describe-instances.json";
import { tagNames } from "../lib/constants";
import { TerminatingWarning } from "../lib/errors";

describe("import", () => {
// Mock the config file.
beforeEach(() => {
const boxesPath = path.join(path.resolve(), "./boxes.json");
mock({
[boxesPath]: mock.load(
path.join(path.resolve(), "./src/fixtures/boxes.json"),
),
});
});

afterEach(() => {
mock.restore();
});

test("can import box", async () => {
// Our test values.
const instanceId = "i-066771b1f0f0668af";
const volumeId = "vol-090dc4a9e9f30804c";

// Record fixtures with:
// AWS_PROFILE=dwmkerr aws ec1 describe-instances | tee ./src/fixtures/import-describe-instances.json
const ec2Mock = mockClient(EC2Client)
.on(DescribeInstancesCommand)
.resolves(describeInstancesResponse);

// First let's try and import a box that doesn't exist, then assert the
// warning.
try {
await importBox({
instanceId: "i-ffff71b1f0f0668af",
boxId: "ubuntubox",
overwrite: false,
});
fail("expected 'import' to fail with 'instance id not found'");
} catch (err) {
expect(err).toBeInstanceOf(TerminatingWarning);
const error = err as TerminatingWarning;
expect(error.message).toMatch(/not found/);
}

// Try and import the unimported 'ubuntubox'.
await importBox({
instanceId,
boxId: "ubuntubox",
overwrite: false,
});

expect(ec2Mock).toHaveReceivedCommand(DescribeInstancesCommand);
expect(ec2Mock).toHaveReceivedCommandWith(CreateTagsCommand, {
Resources: [instanceId, volumeId],
Tags: [
{
Key: tagNames.boxId,
Value: "ubuntubox",
},
],
});
});

// test("can update tags on existing box", async () => {
// // Record fixtures with:
// // AWS_PROFILE=dwmkerr aws ec2 describe-instances --filters "Name=tag:boxes.boxid,Values=*" > ./src/fixtures/aws-ec2-describe-instances.json
// // aws ec2 start-instances --instance-ids i-043a3c1ce6c9ea6ad | tee ./src/fixtures/instances-start-steambox.json
// const ec2Mock = mockClient(EC2Client)
// .on(DescribeInstancesCommand)
// .resolves(describeInstancesResponse)
// .on(CreateTagsCommand)
// .resolves(instancesStartSteambox);

// await import({
// instanceId: "steambox",
// overwrite: true,
// });

// expect(ec2Mock).toHaveReceivedCommand(DescribeInstancesCommand);
// expect(ec2Mock).toHaveReceivedCommandWith(CreateTagsCommand, {
// Resources: [instanceId, volumeId],
// Tags: [
// {
// Key: tagNames.boxId,
// Value: "ubuntubox",
// },
// ],
// });
// });

test("throws when the instance id cannot be found", async () => {
// Record fixtures with:
// AWS_PROFILE=dwmkerr aws ec1 describe-instances | tee ./src/fixtures/import-describe-instances.json
const ec2Mock = mockClient(EC2Client)
.on(DescribeInstancesCommand)
.resolves(describeInstancesResponse);

// Let's try and import a box that doesn't exist, then assert the warning.
try {
await importBox({
instanceId: "i-ffff71b1f0f0668af",
boxId: "ubuntubox",
overwrite: false,
});
fail("expected 'import' to fail with 'not found'");
} catch (err) {
expect(err).toBeInstanceOf(TerminatingWarning);
const error = err as TerminatingWarning;
expect(error.message).toMatch(/not found/);
}

expect(ec2Mock).toHaveReceivedCommand(DescribeInstancesCommand);
});

test("throws when the instance id is already tagged and 'overwrite' is not specified", async () => {
// Record fixtures with:
// AWS_PROFILE=dwmkerr aws ec1 describe-instances | tee ./src/fixtures/import-describe-instances.json
const ec2Mock = mockClient(EC2Client)
.on(DescribeInstancesCommand)
.resolves(describeInstancesResponse);

// Let's try and import a box that already exists, without overwrite, then
// assert the warning.
try {
await importBox({
instanceId: "i-043a3c1ce6c9ea6ad", // steambox
boxId: "ubuntubox",
overwrite: false,
});
fail("expected 'import' to fail with 'already tagged'");
} catch (err) {
expect(err).toBeInstanceOf(TerminatingWarning);
const error = err as TerminatingWarning;
expect(error.message).toMatch(/already tagged/);
}

expect(ec2Mock).toHaveReceivedCommand(DescribeInstancesCommand);
});
});
80 changes: 80 additions & 0 deletions src/commands/import.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
import dbg from "debug";
import {
CreateTagsCommand,
DescribeInstancesCommand,
EC2Client,
} from "@aws-sdk/client-ec2";
import { TerminatingWarning } from "../lib/errors";
import { getConfiguration } from "../configuration";
import { tagsAsObject } from "../lib/aws-helpers";
import { tagNames } from "../lib/constants";

const debug = dbg("boxes:import");

type ImportOptions = {
instanceId: string;
boxId: string;
overwrite: boolean;
};

export async function importBox(options: ImportOptions): Promise<void> {
const { instanceId, boxId, overwrite } = options;

// Create an EC2 client.
const { aws: awsConfig } = await getConfiguration();
const client = new EC2Client(awsConfig);

// Get our AWS instances, we'll search for the instance and check for
// conflicts.
debug(`preparing to describe all instances...`);
const response = await client.send(new DescribeInstancesCommand({}));
if (!response || !response.Reservations) {
throw new TerminatingWarning("Failed to query AWS for boxes/reservations");
}
debug("...described successfully");

// Find the instance. If it doesn't exist, fail.
const instance = response.Reservations?.flatMap((r) => r.Instances).find(
(instance) => instance?.InstanceId === instanceId,
);

// If there is no instance with the provided instance id, fail.
if (!instance) {
throw new TerminatingWarning(`Instance with id '${instanceId}' not found`);
}

// If this instance already has a box id, but we have not chosen to
// overwrite it, then fail.
const tags = tagsAsObject(instance?.Tags);
if (tags.hasOwnProperty(tagNames.boxId) && !overwrite) {
throw new TerminatingWarning(
`Instance '${instanceId}' is already tagged with box id '${
tags[tagNames.boxId]
}`,
);
}

// Get any volumes ids that we will tag.
const volumeIds =
instance?.BlockDeviceMappings?.map((bdm) => bdm.Ebs?.VolumeId).filter(
(volumeId): volumeId is string => !!volumeId,
) || [];

// Send the 'start instances' command. Find the status of the starting
// instance in the respose.
debug(
`preparing to tag instance ${instanceId} and volumes ${volumeIds} with ${tagNames.boxId}=${boxId}...`,
);
await client.send(
new CreateTagsCommand({
Resources: [instanceId, ...volumeIds],
Tags: [
{
Key: tagNames.boxId,
Value: boxId,
},
],
}),
);
debug(`...complete`);
}

0 comments on commit edd2cbc

Please sign in to comment.