Skip to content

Commit

Permalink
feat: archive volumes debug commands
Browse files Browse the repository at this point in the history
Many fixes, improvements, test updates. The 'archive' commands now work when invoked via the `debug` command and will gradually be incorporated into the user-facing commands
  • Loading branch information
dwmkerr committed Jan 24, 2024
1 parent 5cfd11b commit 2d699a2
Show file tree
Hide file tree
Showing 43 changed files with 2,108 additions and 110 deletions.
13 changes: 8 additions & 5 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
node_modules
jspm_packages
package-lock.json
boxes.json
artifacts/
.DS_Store
artifacts/
build/
node_modules/
jspm_packages/
package-lock.json

# Ignore any local boxes config - but track the test fixture version.
boxes.json
!src/fixtures/boxes.json
60 changes: 48 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,3 @@
# todo

test ssh, new torrent script with logging, keep existing script for testing
detach this eve
region in config

test a burner box for michelle's book

# boxes

[![main](https://github.com/dwmkerr/boxes/actions/workflows/main.yml/badge.svg)](https://github.com/dwmkerr/boxes/actions/workflows/main.yml) ![npm (scoped)](https://img.shields.io/npm/v/%40dwmkerr/boxes) [![codecov](https://codecov.io/gh/dwmkerr/boxes/graph/badge.svg?token=uGVpjGFbDf)](https://codecov.io/gh/dwmkerr/boxes)
Expand Down Expand Up @@ -63,6 +55,10 @@ $ boxes start steambox
steambox (i-098e8d30d5e399b03): stopped -> pending
```

Options:

- `--wait`: wait for instance to complete startup

### `boxes stop`

Run `boxes start <id>` to stop a box:
Expand All @@ -72,6 +68,10 @@ $ boxes stop steambox
steambox (i-098e8d30d5e399b03): running -> stopping
```

Options:

- `--wait`: wait for instance to complete shutdown

### `boxes info`

Run `boxes info <id>` to show detailed info on a box:
Expand Down Expand Up @@ -287,7 +287,15 @@ If you are developing and would like to run the `boxes` command without relinkin
npm run build:watch
```
This will keep the `./build` folder up-to-date and the `boxes` command will use the latest compiled code.
This will keep the `./build` folder up-to-date and the `boxes` command will use the latest compiled code. This will *sometimes* work but it might miss certain changes, so `relink` is the safer option. `build:watch` works well if you are making small changes to existing files, but not if you are adding new files (it seems).
### Debugging
The [`debug`](https://github.com/debug-js/debug) library is used to make it easy to provide debug level output. Debug logging to the console can be enabled with:
```bash
DEBUG='boxes*' boxes list
```
### Error Handling
Expand Down Expand Up @@ -341,7 +349,7 @@ Typically occurs if AWS SDK packages are not at the exact same number as the `@a
Quick and dirty task-list.
## Alpha
### Alpha
- [x] feat: document copy password in connect, maybe better default off
- [ ] refactor: suck it up and use TS
Expand All @@ -362,10 +370,13 @@ Quick and dirty task-list.
- [ ] feat: boxes aws-console opens link eg (https://us-west-2.console.aws.amazon.com/ec2/home?region=us-west-2#InstanceDetails:instanceId=i-043a3c1ce6c9ea6ad)
- [ ] bug: EBS devices not tagged -I've tagged two (manually) in jan - check w/ feb bill
## Beta
### Beta
- [ ] 'wait' flag for start/stop to wait until operation complete - default to 1hr and document the timeout info
## Later
### Later
- [ ] refactor: make 'debug' command local/debug build only?
- [ ] feat: 'import' command to take an instance ID and create local box config for it and tag the instance
- [ ] docs: cost allocation tags blog post
- [ ] docs: create and share blogpost
Expand All @@ -374,3 +385,28 @@ Quick and dirty task-list.
- [ ] feat: autocomplete
- [ ] feat: aws profile in config file
- [ ] epic: 'boxes create' to create from a template
- [ ] refactor: find a better way to mock / inject config (rather than importing arbitrarily)
### Epic - Interactive Setup
Run `boxes init` - lets you choose a region, select instances, give a name.
Will add the tags - but will also add the tags to the volumes and will notify if the cost explorer tag is not setup.
Creates the local config.
This would be demo-able.
### Epic - Volume Management
- [x] test '-wait' on start/stop and doc
- [ ] 'start' can now check for 'has archived volumes' and restore if available, this is the next big one
- [ ] propagate tags w/test
- [ ] delete tag on volume restore...
- [ ] ...so that we can auto restore volumes when calling 'start' - which will need to wait for the volumes to be ready
- [ ] auto-restore on start, this is a good incremental point to check-in, even
if backup is only via 'debug' and comes later
- [ ] data loss warning and generalise the 'yes' flag
- [x] delete snapshot on successful restore
- [ ] better logging for non-debug mode (warn user can take time)
- [ ] new task list - docs, function, parameters, cost saving info, etc
- [ ] calling 'detach/etc' fails if instance is not stopped or stopping as it doesn't try to stop the instance - must fail if state is not stopping or stopped
- [ ] complete stop/start unit tests
5 changes: 3 additions & 2 deletions jest.config.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import type { Config } from "jest";

/*
* For a detailed explanation regarding each configuration property, visit:
* https://jestjs.io/docs/configuration
Expand Down Expand Up @@ -28,8 +29,8 @@ const config: Config = {
// Only look for tests in the src folder, i.e. excluded build.
roots: ["./src/"],

// Initial config/setup function for jest
globalSetup: "./src/jest-global-setup.ts",
// Jest setup function.
setupFilesAfterEnv: ["./src/jest.setup.ts"],

// Automatically clear mock calls, instances, contexts and results before every test
clearMocks: true,
Expand Down
8 changes: 7 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,12 @@
"build": "tsc",
"build:watch": "tsc -w",
"start": "ts-node ./src/cli.ts",
"start:debug": "NODE_OPTIONS='--experimental-vm-modules' node --inspect-brk node_modules/.bin/ts-node ./src/cli.ts",
"test": "NODE_OPTIONS='--experimental-vm-modules' jest",
"lint": "eslint .",
"lint:fix": "eslint --fix .",
"test:debug": "NODE_OPTIONS='--experimental-vm-modules' node --inspect-brk node_modules/.bin/jest --runInBand",
"test:watch": "NODE_OPTIONS='--experimental-vm-modules' node node_modules/.bin/jest --runInBand --watch",
"test:watch": "NODE_OPTIONS='--experimental-vm-modules' node node_modules/.bin/jest --runInBand --watch --no-coverage",
"test:cov": "NODE_OPTIONS='--experimental-vm-modules' jest --coverage",
"tsc": "tsc",
"relink": "npm run build && npm unlink boxes && npm link boxes"
Expand All @@ -36,17 +37,22 @@
"clipboardy": "^4.0.0",
"colors": "^1.4.0",
"commander": "^11.1.0",
"debug": "^4.3.4",
"open": "^9.1.0"
},
"devDependencies": {
"@aws-sdk/types": "3.10.0",
"@types/debug": "^4.1.12",
"@types/jest": "^29.5.11",
"@types/mock-fs": "^4.13.4",
"@types/node": "^20.11.5",
"@typescript-eslint/eslint-plugin": "^6.19.0",
"aws-sdk-client-mock-jest": "^3.0.0",
"eslint": "^8.53.0",
"eslint-config-prettier": "^9.0.0",
"eslint-plugin-prettier": "^5.0.1",
"jest": "^29.7.0",
"mock-fs": "^5.2.0",
"ts-jest": "^29.1.1",
"ts-node": "^10.9.2",
"typescript": "^5.3.3"
Expand Down
1 change: 1 addition & 0 deletions src/box.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,5 +34,6 @@ export interface Box {
name: string;
state: BoxState;
instanceId: string | undefined;
hasArchivedVolumes: boolean;
instance: Instance | undefined;
}
45 changes: 38 additions & 7 deletions src/cli.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
#!/usr/bin/env node

import dbg from "debug";
import { Command } from "commander";
import { list, info } from "./commands";
import { debug } from "./commands/debug";
import { start } from "./commands/start";
import { stop } from "./commands/stop";
import { ssh } from "./commands/ssh";
Expand All @@ -12,6 +14,10 @@ import theme from "./theme";
import { TerminatingWarning } from "./lib/errors";
import packageJson from "../package.json";
import { BoxState } from "./box";
import { assertConfirmation } from "./lib/cli-helpers";

// While we're developing, debug output is always enabled.
dbg.enable("boxes*");

const ERROR_CODE_WARNING = 1;
const ERROR_CODE_CONNECTION = 2;
Expand All @@ -30,12 +36,15 @@ program
const boxes = await list();
boxes.forEach((box) => {
theme.printBoxHeading(box.boxId, box.state);
theme.printBoxDetail("Name", box.name);
theme.printBoxDetail("Name", box.name || "<unknown>");
// Only show DNS details if they exist (i.e. if the box is running).
if (box.instance?.PublicDnsName && box.instance?.PublicIpAddress) {
theme.printBoxDetail("DNS", box.instance.PublicDnsName);
theme.printBoxDetail("IP", box.instance.PublicIpAddress);
}
if (box.hasArchivedVolumes) {
theme.printBoxDetail("Archived Volumes", "true");
}
});
});

Expand Down Expand Up @@ -75,8 +84,12 @@ program
.command("start")
.description("Start a box")
.argument("<boxId>", 'id of the box, e.g: "steambox"')
.action(async (boxId) => {
const { instanceId, currentState, previousState } = await start(boxId);
.option("-w, --wait", "wait for box to complete startup", false)
.action(async (boxId, options) => {
const { instanceId, currentState, previousState } = await start({
boxId,
wait: options.wait,
});
console.log(
` ${theme.boxId(boxId)} (${instanceId}): ${theme.state(
previousState,
Expand All @@ -88,12 +101,12 @@ program
.command("stop")
.description("Stop a box")
.argument("<boxId>", 'id of the box, e.g: "steambox"')
.option("--detach-volumes", "detach EBS volumes (experimental)", false)
.option("-w, --wait", "wait for box to complete startup", false)
.action(async (boxId, options) => {
const { instanceId, currentState, previousState } = await stop(
const { instanceId, currentState, previousState } = await stop({
boxId,
options.detachVolumes,
);
wait: options.wait,
});
console.log(
` ${theme.boxId(boxId)} (${instanceId}): ${theme.state(
previousState,
Expand All @@ -108,6 +121,14 @@ program
.option("-y, --year <year>", "month of year", undefined)
.option("-m, --month <month>", "month of year", undefined)
.action(async (options) => {
// Demand confirmation.
await assertConfirmation(
options,
"yes",
`The AWS cost explorer charges $0.01 per call.
To accept charges, re-run with the '--yes' parameter.`,
);

const boxes = await list();
const costs = await getCosts({
yes: options.yes,
Expand Down Expand Up @@ -146,6 +167,16 @@ program
console.log(JSON.stringify(configuration, null, 2));
});

program
.command("debug")
.description("Additional commands used for debugging")
.argument("<command>", 'debug command to use, e.g. "test-detach"')
.argument("<parameters...>", 'parameters for the command, e.g. "one two"')
.action(async (command, parameters) => {
const result = await debug(command, parameters);
console.log(JSON.stringify(result));
});

async function run() {
try {
await program.parseAsync();
Expand Down
7 changes: 5 additions & 2 deletions src/commands/connect.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
// used in the 'connect' function below.
// import clipboard from "clipboardy";
import dbg from "debug";
import { getBoxes } from "../lib/get-boxes";
import { getConfiguration } from "../configuration";
import { TerminatingWarning } from "../lib/errors";

const debug = dbg("boxes:connect");

export async function connect(
boxId: string,
openConnection: boolean,
Expand Down Expand Up @@ -36,9 +37,11 @@ export async function connect(

// Expand the url string, which'll look something like this:
// http://${host}:9091/transmission/web/
debug(`expanding from: ${boxConfig.connectUrl}`);
const expandedUrl = boxConfig.connectUrl
.replace("${host}", box.instance.PublicDnsName)
.replace("${username}", boxConfig.username);
debug(`expanded result: ${boxConfig.connectUrl}`);

// If the user has asked for the password to be copied, put it on the
// clipboard.
Expand Down
61 changes: 61 additions & 0 deletions src/commands/debug.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
import { TerminatingWarning } from "../lib/errors";
import {
getDetachableVolumes,
recreateVolumesFromSnapshotTag,
snapshotTagDeleteVolumes,
} from "../lib/volumes";

// eslint-disable-next-line @typescript-eslint/no-explicit-any
function logJson(val: any) {
console.log(JSON.stringify(val, null, 2));
}

export async function debug(command: string, parameters: string[]) {
console.log(`debug: command - ${command} with parameters ${parameters}`);
if (command === "test-detach") {
console.log("debug: test-detach");
const instanceId = parameters[0];
if (!instanceId) {
console.error("instanceid is required as the first parameter");
return;
}

const detachableVolumes = getDetachableVolumes(instanceId);

return detachableVolumes;
} else if (command === "test-detach-snapshot-tag") {
console.log("debug: test-detach");
const instanceId = parameters[0];
if (!instanceId) {
console.error("instanceid is required as the first parameter");
return;
}

// TODO fix tag settings
const tags = [{ key: "boxes.boxid", value: "torrentbox" }];
console.log("Getting detachable volumes...");
const detachableVolumes = await getDetachableVolumes(instanceId);
logJson(detachableVolumes);
console.log("Snapshotting / tagging...");
const result = await snapshotTagDeleteVolumes(
instanceId,
detachableVolumes,
tags,
);
logJson(result);

return result;
} else if (command === "test-restore-volumes") {
console.log("debug: test-restore-volumes");
const instanceId = parameters[0];
if (!instanceId) {
console.error("instanceid is required as the first parameter");
return;
}
const result = await recreateVolumesFromSnapshotTag(instanceId);
console.log(result);
} else {
throw new TerminatingWarning(`unknown debug command ${command}`);
}
return {};
}
9 changes: 0 additions & 9 deletions src/commands/getCosts.ts
Original file line number Diff line number Diff line change
@@ -1,24 +1,15 @@
import { getBoxesCosts } from "../lib/get-boxes-costs";
import { TerminatingWarning } from "../lib/errors";

type BoxCosts = Record<string, string>;

export async function getCosts({
yes,
year,
month,
}: {
yes: boolean;
year: string;
month: string;
}): Promise<BoxCosts> {
// If the user hasn't passed the 'yes' parameter to confirm, ask now.
if (yes !== true) {
const message = `The AWS cost explorer charges $0.01 per call.
To accept charges, re-run with the '--yes' parameter.`;
throw new TerminatingWarning(message);
}

// Parse the year/month number if provided.
const yearNumber = year ? parseInt(year, 10) : undefined;
const monthNumber = month ? parseInt(month, 10) : undefined;
Expand Down

0 comments on commit 2d699a2

Please sign in to comment.