Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: broken daemon build #170

Merged
merged 12 commits into from
Jun 12, 2024
3 changes: 1 addition & 2 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
dist/
node_modules/
__tests__/
.git/
.github/
Expand All @@ -8,5 +7,5 @@ flake.*
node_modules/
packages/daemon/dist/
packages/daemon/node_modules/
packages/wallet-service
packages/common/node_modules/
packages/wallet-service
21 changes: 7 additions & 14 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
# arising from the use of this software.
# This software cannot be redistributed unless explicitly agreed in writing with the authors.


# Build phase
FROM node:20-alpine AS builder

Expand All @@ -19,19 +18,13 @@ RUN corepack enable
# Use the same version as flake's
RUN yarn set version 4.1.0

# This will install dependencies for the sync-daemon, devDependencies included:
RUN yarn workspaces focus sync-daemon

RUN yarn workspace sync-daemon build

# This will remove all dependencies and install production deps only:
RUN yarn workspaces focus sync-daemon --production
# This will install dependencies for all packages, except for the lambdas since
# they are ignored in .dockerignore
RUN yarn install

FROM node:20-alpine

WORKDIR /app
RUN yarn workspace sync-daemon run build

COPY --from=builder /app/packages/daemon/dist .
COPY --from=builder /app/packages/daemon/node_modules ./node_modules
# This will remove all dev dependencies and install production deps only
RUN yarn workspaces focus -A --production
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the -A flag because otherwise we wouldn't be able to use wallet-lib as a peer dependency and would have issues with bitcore-lib

This is not installing dependencies from the wallet-service lambdas as we are ignoring its package in .dockerignore


CMD ["node", "index.js"]
CMD ["yarn", "workspace", "sync-daemon", "run", "start"]
11 changes: 8 additions & 3 deletions packages/common/__tests__/utils/alerting.utils.mock.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
export const mockedAddAlert = jest.fn();
export default jest.mock('@src/utils/alerting.utils', () => ({
addAlert: mockedAddAlert.mockReturnValue(Promise.resolve()),
}));
export default jest.mock('@src/utils/alerting.utils', () => {
const originalModule = jest.requireActual('@src/utils/alerting.utils');

return {
...originalModule,
addAlert: mockedAddAlert.mockReturnValue(Promise.resolve()),
};
});
11 changes: 10 additions & 1 deletion packages/common/__tests__/utils/nft.utils.test.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
// @ts-ignore: Using old wallet-lib version, no types exported
import hathorLib from '@hathor/wallet-lib';
import { mockedAddAlert } from './alerting.utils.mock';
import { Severity } from '@src/types';
import { NftUtils } from '@src/utils/nft.utils';
import { Severity } from '@src/types';
import { getHandlerContext, getTransaction } from '../events/nftCreationTx';
import {
LambdaClient as LambdaClientMock,
Expand Down Expand Up @@ -37,6 +37,15 @@ jest.mock('@aws-sdk/client-lambda', () => {
};
});

jest.mock('@src/utils/index.utils', () => {
const originalModule = jest.requireActual('@src/utils/index.utils');

return {
...originalModule,
assertEnvVariablesExistence: jest.fn(),
};
});

const network = new hathorLib.Network('testnet');
const logger = new Logger();

Expand Down
9 changes: 9 additions & 0 deletions packages/common/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
/**
* Copyright (c) Hathor Labs and its affiliates.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/

export * from './src/types';
export * from './src/utils/index.utils';
2 changes: 2 additions & 0 deletions packages/common/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@
"name": "@wallet-service/common",
"version": "1.5.0",
"packageManager": "yarn@4.1.0",
"main": "dist/index.js",
"scripts": {
"build": "tsc --declaration",
"test": "jest --runInBand --collectCoverage --detectOpenHandles --forceExit"
},
"peerDependencies": {
Expand Down
9 changes: 0 additions & 9 deletions packages/common/src/utils/alerting.utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,8 @@

import { SendMessageCommand, SQSClient } from '@aws-sdk/client-sqs';
import { Severity } from '../types';
import { assertEnvVariablesExistence } from './index.utils';
import { Logger } from 'winston';

assertEnvVariablesExistence([
'NETWORK',
'APPLICATION_NAME',
'ACCOUNT_ID',
'ALERT_MANAGER_REGION',
'ALERT_MANAGER_TOPIC',
]);

Comment on lines -13 to -20
Copy link
Collaborator Author

@andreabadesso andreabadesso Jun 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a config method that already checks for those variables and this check shouldn't be in a lib

/**
* Adds a message to the SQS alerting queue
*
Expand Down
4 changes: 4 additions & 0 deletions packages/common/src/utils/index.utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@
* LICENSE file in the root directory of this source tree.
*/

export * from './alerting.utils';
export * from './nft.utils';
export * from './wallet.utils';

/**
* Validates if a list of env variables are set in the environment. Throw if at least
* one of them is missing
Expand Down
1 change: 1 addition & 0 deletions packages/common/src/utils/wallet.utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/

// @ts-ignore
import { constants } from '@hathor/wallet-lib';

Expand Down
9 changes: 6 additions & 3 deletions packages/common/tsconfig.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
{
"compilerOptions": {
"composite": true,
"target": "ES2022",
"module": "CommonJS",
"declaration": true,
"sourceMap": true,
"strict": true,
"esModuleInterop": true,
Expand All @@ -10,12 +12,13 @@
"outDir": "./dist",
"types": ["node", "jest"],
"paths": {
"@src/*": ["src/*"],
"@tests/*": ["__tests__/*"],
"@events/*": ["__tests__/events/*"]
"@src/*": ["./src/*"],
"@tests/*": ["./__tests__/*"],
"@events/*": ["./__tests__/events/*"]
}
},
"include": [
"index.ts",
"src/**/*.ts",
"__tests__/**/*.ts"
],
Expand Down
4 changes: 2 additions & 2 deletions packages/daemon/__tests__/db/index.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,9 @@ import {
createOutput,
XPUBKEY,
} from '../utils';
import { isAuthority } from '@wallet-service/common/src/utils/wallet.utils';
import { isAuthority } from '@wallet-service/common';
import { DbTxOutput, StringMap, TokenInfo, WalletStatus } from '../../src/types';
import { Authorities, TokenBalanceMap } from '@wallet-service/common/src/types';
import { Authorities, TokenBalanceMap } from '@wallet-service/common';

// Use a single mysql connection for all tests
let mysql: Connection;
Expand Down
4 changes: 0 additions & 4 deletions packages/daemon/__tests__/guards/guards.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,6 @@ jest.mock('../../src/config', () => {
};
});

jest.mock('@wallet-service/common/src/utils/index.utils', () => ({
assertEnvVariablesExistence: jest.fn(),
}));

import getConfig from '../../src/config';

const TxCache = {
Expand Down
4 changes: 0 additions & 4 deletions packages/daemon/__tests__/integration/balances.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,6 @@ jest.mock('../../src/config', () => {
};
});

jest.mock('@wallet-service/common/src/utils/index.utils', () => ({
assertEnvVariablesExistence: jest.fn(),
}));

import getConfig from '../../src/config';

// @ts-ignore
Expand Down
4 changes: 0 additions & 4 deletions packages/daemon/__tests__/machines/SyncMachine.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,6 @@ import getConfig from '../../src/config';
const { TX_CACHE_SIZE, FULLNODE_PEER_ID, STREAM_ID } = getConfig();
const { VERTEX_METADATA_CHANGED, NEW_VERTEX_ACCEPTED, REORG_STARTED } = EventFixtures;

jest.mock('@wallet-service/common/src/utils/index.utils', () => ({
assertEnvVariablesExistence: jest.fn(),
}));

const TxCache = new LRU(TX_CACHE_SIZE);

beforeAll(async () => {
Expand Down
4 changes: 0 additions & 4 deletions packages/daemon/__tests__/services/services.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -90,10 +90,6 @@ jest.mock('../../src/utils', () => ({
getFullnodeHttpUrl: jest.fn(),
}));

jest.mock('@wallet-service/common/src/utils/index.utils', () => ({
assertEnvVariablesExistence: jest.fn(),
}));

beforeEach(() => {
jest.clearAllMocks();
});
Expand Down
7 changes: 4 additions & 3 deletions packages/daemon/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@
},
"scripts": {
"lint": "eslint .",
"build": "tsc",
"build": "tsc -b",
"start": "node dist/index.js",
"watch": "tsc -w",
"test_images_up": "docker-compose -f ./__tests__/integration/scripts/docker-compose.yml up -d",
"test_images_down": "docker-compose -f ./__tests__/integration/scripts/docker-compose.yml down",
Expand Down Expand Up @@ -45,12 +46,12 @@
"typescript": "^4.9.5"
},
"peerDependencies": {
"@hathor/wallet-lib": "0.39.0"
"@hathor/wallet-lib": "0.39.0",
"@wallet-service/common": "1.5.0"
},
"dependencies": {
"@aws-sdk/client-lambda": "3.540.0",
"@aws-sdk/client-sqs": "3.540.0",
"@wallet-service/common": "1.5.0",
"assert": "^2.1.0",
"aws-sdk": "^2.1454.0",
"axios": "^1.6.2",
Expand Down
2 changes: 1 addition & 1 deletion packages/daemon/src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,9 @@ const requiredEnvs = [
'ACCOUNT_ID',
'ALERT_MANAGER_TOPIC',
'ALERT_MANAGER_REGION',
'APPLICATION_NAME',
];


export const checkEnvVariables = () => {
const missingEnv = requiredEnvs.filter(envVar => process.env[envVar] === undefined);

Expand Down
4 changes: 2 additions & 2 deletions packages/daemon/src/db/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ import {
TxInput,
TokenBalanceMap,
TxOutputWithIndex,
} from '@wallet-service/common/src/types';
import { isAuthority } from '@wallet-service/common/src/utils/wallet.utils';
} from '@wallet-service/common';
import { isAuthority } from '@wallet-service/common';
import {
AddressBalanceRow,
AddressTxHistorySumRow,
Expand Down
4 changes: 2 additions & 2 deletions packages/daemon/src/services/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
import hathorLib from '@hathor/wallet-lib';
import axios from 'axios';
import { get } from 'lodash';
import { NftUtils } from '@wallet-service/common/src/utils/nft.utils';
import { NftUtils } from '@wallet-service/common';
import {
StringMap,
Wallet,
Expand All @@ -25,7 +25,7 @@ import {
Transaction,
TokenBalanceMap,
TxOutputWithIndex,
} from '@wallet-service/common/src/types';
} from '@wallet-service/common';
import {
prepareOutputs,
getAddressBalanceMap,
Expand Down
2 changes: 1 addition & 1 deletion packages/daemon/src/types/wallet.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
* LICENSE file in the root directory of this source tree.
*/

import { TokenBalanceMap } from '@wallet-service/common/src/types';
import { TokenBalanceMap } from '@wallet-service/common';

export enum WalletStatus {
CREATING = 'creating',
Expand Down
2 changes: 1 addition & 1 deletion packages/daemon/src/utils/aws.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import { SendMessageCommand, SendMessageCommandOutput, SQSClient, MessageAttribu
import { StringMap } from '../types';
import getConfig from '../config';
import logger from '../logger';
import { addAlert } from '@wallet-service/common/src/utils/alerting.utils';
import { addAlert } from '@wallet-service/common';

export function buildFunctionName(functionName: string): string {
const { STAGE } = getConfig();
Expand Down
2 changes: 1 addition & 1 deletion packages/daemon/src/utils/wallet.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ import {
TxInput,
TxOutput,
TokenBalanceMap,
} from '@wallet-service/common/src/types';
} from '@wallet-service/common';
import {
fetchAddressBalance,
fetchAddressTxHistorySum,
Expand Down
6 changes: 6 additions & 0 deletions packages/daemon/tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,12 @@
"outDir": "./dist",
"types": ["node", "jest"]
},
// The common module is not deployed to npm and is an internal tool,
// we want it to have its own tsconfig, so this is instructing typescript
// to compile its tsconfig.json file
"references": [
{ "path": "../../node_modules/@wallet-service/common" }
],
Comment on lines +16 to +18
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes tsconfig compile the common module tsconfig

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question(non-blocking): Should we add these instructions to a document on the repository itself?

There seem to be many of those small details that can break the production environment and delay future maintenance releases if left unchecked.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just found out that typescript implements its own JSON parser, which allows for comments in JSON

Added a comment in tsconfig.json in 536ee04

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also created an issue so we document these details better

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! It will surely help with future maintenance

"include": [
"src/**/*.ts"
],
Expand Down
2 changes: 1 addition & 1 deletion yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -15051,7 +15051,6 @@ __metadata:
"@types/ws": "npm:^8.5.5"
"@typescript-eslint/eslint-plugin": "npm:^6.7.3"
"@typescript-eslint/parser": "npm:^6.7.3"
"@wallet-service/common": "npm:1.5.0"
assert: "npm:^2.1.0"
aws-sdk: "npm:^2.1454.0"
axios: "npm:^1.6.2"
Expand All @@ -15074,6 +15073,7 @@ __metadata:
xstate: "npm:^4.38.2"
peerDependencies:
"@hathor/wallet-lib": 0.39.0
"@wallet-service/common": 1.5.0
languageName: unknown
linkType: soft

Expand Down
Loading