Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(enum-updater): add enum updater tool #33681

Open
wants to merge 17 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add additional unit tests
  • Loading branch information
paulhcsun committed Mar 4, 2025
commit eb169bbc6303974e031e2246f4534bb6e3374b15
89 changes: 45 additions & 44 deletions .github/workflows/enum-auto-updater.yml
Original file line number Diff line number Diff line change
@@ -30,50 +30,51 @@ jobs:
cd tools/@aws-cdk/enum-updater
./bin/update-enums
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't it should be update-missing-enums instead?


- name: Commit & Push changes
if: steps.git-check.outputs.changes == 'true'
run: |
git config --global user.name 'aws-cdk-automation'
git config --global user.email 'aws-cdk-automation@users.noreply.github.com'

# Iterate through each module directory that has changes
for module in $(git diff --name-only | grep -E '^packages/(@aws-cdk|aws-cdk-lib)/.*' | sed 's|/.*||' | sort -u); do
moduleName=$(basename $module)

# Check for existing PR with the same name
prExists=$(gh pr list --state open --search "chore(${moduleName#aws-}): add new enum values for ${moduleName#aws-}" --json number,title -q '.[].number')

# If a PR exists, close it and continue
if [[ -n "$prExists" ]]; then
echo "PR already exists for module ${moduleName#aws-}, closing the existing PR."
gh pr close "$prExists" --confirm # Close the PR by its number
fi

# Create a new branch for the module
branchName="enum-update/${moduleName#aws-}"
git checkout -b "$branchName"

# Stage, commit, and push changes for the module
git add "packages/$module" # Add only changes for this module
git commit -m "chore(${moduleName#aws-}): add new enum values for ${moduleName#aws-}"
git push origin "$branchName"

# Create a new pull request
gh pr create --title "chore(${moduleName#aws-}): add new enum values for ${moduleName#aws-}" \
--body "This PR updates the enum values for ${moduleName#aws-}." \
--base main \
--head "$branchName"
done
- name: Check for changes
id: git-check
run: |
if [[ -n "$(git status --porcelain)" ]]; then
echo "changes=true" >> $GITHUB_OUTPUT
else
echo "changes=false" >> $GITHUB_OUTPUT
fi

- name: Commit & Push changes
if: steps.git-check.outputs.changes == 'true'
run: |
git config --global user.name 'aws-cdk-automation'
git config --global user.email 'aws-cdk-automation@users.noreply.github.com'

# Iterate through each module directory that has changes
for module in $(git diff --name-only | grep -E '^packages/(@aws-cdk|aws-cdk-lib)/.*' | sed 's|/.*||' | sort -u); do
moduleName=$(basename $module)

# Check for existing PR with the same name
prExists=$(gh pr list --state open --search "chore(${moduleName#aws-}): add new enum values for ${moduleName#aws-}" --json number,title -q '.[].number')

# If a PR exists, close it and continue
if [[ -n "$prExists" ]]; then
echo "PR already exists for module ${moduleName#aws-}, closing the existing PR."
gh pr close "$prExists" --confirm # Close the PR by its number
fi

# Create a new branch for the module
branchName="enum-update/${moduleName#aws-}"
git checkout -b "$branchName"

# Stage, commit, and push changes for the module
git add "packages/$module" # Add only changes for this module
git commit -m "chore(${moduleName#aws-}): add new enum values for ${moduleName#aws-}"
git push origin "$branchName"

# Create a new pull request
gh pr create --title "chore(${moduleName#aws-}): add new enum values for ${moduleName#aws-}" \
--body "This PR updates the enum values for ${moduleName#aws-}." \
--base main \
--head "$branchName"
--label "contribution/core,pr-linter/exempt-integ-test,pr-linter/exempt-readme,pr-linter/exempt-test" \
--reviewer "aws-cdk-team" \
done

# - name: Commit & Push changes
# if: steps.git-check.outputs.changes == 'true'
# run: |
# git config --global user.name 'aws-cdk-automation'
# git config --global user.email 'aws-cdk-automation@users.noreply.github.com'

# git checkout -B ${{ github.event.pull_request.head.ref }}
# git add .
# git commit -m "chore: update analytics metadata blueprints"
# git push origin ${{ github.event.pull_request.head.ref }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Original file line number Diff line number Diff line change
@@ -7,7 +7,7 @@ import * as extract from 'extract-zip';
const ENUMS_URL = "https://raw.githubusercontent.com/aws/aws-cdk/main/packages/aws-cdk-lib/core/lib/analytics-data-source/enums/module-enums.json";
const ENUM_LIKE_CLASSES_URL = "https://raw.githubusercontent.com/aws/aws-cdk/main/packages/aws-cdk-lib/core/lib/analytics-data-source/enums/module-enumlikes.json";
const AWS_SDK_MODELS_URL = "https://github.com/awslabs/aws-sdk-rust/archive/refs/heads/main.zip";
const MODULE_MAPPING = path.join(__dirname, "module_mapping.json");
const MODULE_MAPPING = path.join(__dirname, "module-mapping.json");
const STATIC_MAPPING_FILE_NAME = "static-enum-mapping.json";
const PARSED_CDK_ENUMS_FILE_NAME = "cdk-enums.json";
export const PARSED_SDK_ENUMS_FILE_NAME = "sdk-enums.json";
@@ -409,7 +409,7 @@ function calculateValueMatchPercentage(cdkValues: Set<string>, sdkValues: Set<st
* - `enumName`: The name of the best-matching SDK enum.
* - `matchPercentage`: The percentage of matching values.
*/
function findMatchingEnum(
export function findMatchingEnum(
cdkEnumName: string,
cdkValues: (string | number)[],
sdkServices: string[],
@@ -471,7 +471,7 @@ function isValidMatch(cdkValues: Set<string>, sdkValues: Set<string>): boolean {
* @param {Record<string, string[]>} manualMappings - The manually defined service mappings.
* @returns {Promise<void>}
*/
async function generateAndSaveStaticMapping(
export async function generateAndSaveStaticMapping(
cdkEnums: CdkEnums,
sdkEnums: SdkEnums,
manualMappings: Record<string, string[]>
223 changes: 223 additions & 0 deletions tools/@aws-cdk/enum-updater/test/missing-enum-updater.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
import { MissingEnumsUpdater } from '../lib/missing-enum-updater';
import { Project, SourceFile, EnumDeclaration, ClassDeclaration, SyntaxKind } from 'ts-morph';
import * as path from 'path';
import * as fs from 'fs';

// Mock dependencies
jest.mock('ts-morph');
jest.mock('fs');
jest.mock('path');
jest.mock('tmp');

describe('MissingEnumsUpdater', () => {
let updater: MissingEnumsUpdater;
let mockSourceFile: jest.Mocked<SourceFile>;
let mockEnumDeclaration: jest.Mocked<EnumDeclaration>;
let mockClassDeclaration: jest.Mocked<ClassDeclaration>;

beforeEach(() => {
jest.clearAllMocks();

// Setup Project mock
(Project as jest.Mock).mockImplementation(() => ({
addSourceFilesAtPaths: jest.fn(),
getSourceFiles: jest.fn().mockReturnValue([]),
getSourceFile: jest.fn().mockReturnValue(mockSourceFile)
}));

// Mock file system operations
(fs.readdirSync as jest.Mock).mockReturnValue([]);
(fs.statSync as jest.Mock).mockReturnValue({ isDirectory: () => false });
(fs.readFileSync as jest.Mock).mockReturnValue('{}');
(path.resolve as jest.Mock).mockImplementation((...args) => args.join('/'));

updater = new MissingEnumsUpdater('./test-dir');
});

describe('constructor', () => {
it('should initialize with correct project settings', () => {
expect(Project).toHaveBeenCalledWith({
tsConfigFilePath: expect.stringMatching(/tsconfig\.json$/),
manipulationSettings: expect.any(Object)
});
});
});
describe('readTypescriptFiles', () => {
it('should skip specified directories', () => {
const mockFiles = ['node_modules', 'dist', 'test', 'valid.ts'];
(fs.readdirSync as jest.Mock).mockReturnValue(mockFiles);
(path.join as jest.Mock).mockImplementation((...args) => args.join('/'));
(fs.statSync as jest.Mock).mockImplementation((filePath) => ({
isDirectory: () => !filePath.includes('.ts')
}));

const result = (updater as any).readTypescriptFiles('./test-dir');
expect(result).toEqual(['./test-dir/valid.ts']);
});

it('should filter out invalid typescript files', () => {
const mockFiles = ['file.ts', 'file.generated.ts', 'file.d.ts', 'file.test.ts'];
(fs.readdirSync as jest.Mock).mockReturnValue(mockFiles);
(path.join as jest.Mock).mockImplementation((...args) => args.join('/'));
(fs.statSync as jest.Mock).mockReturnValue({ isDirectory: () => false });

const result = (updater as any).readTypescriptFiles('./test-dir');
expect(result).toEqual(['./test-dir/file.ts']);
});
});

describe('updateEnum', () => {
beforeEach(() => {
mockEnumDeclaration = {
getFullText: jest.fn().mockReturnValue('enum Test {\n VALUE1 = "value1"\n}'),
replaceWithText: jest.fn(),
} as any;

mockSourceFile = {
getEnum: jest.fn().mockReturnValue(mockEnumDeclaration),
saveSync: jest.fn(),
} as any;

// Update Project mock implementation
(Project as jest.Mock).mockImplementation(() => ({
addSourceFilesAtPaths: jest.fn(),
getSourceFiles: jest.fn().mockReturnValue([]),
getSourceFile: jest.fn().mockReturnValue(mockSourceFile)
}));

updater = new MissingEnumsUpdater('./test-dir');
});

it('should update enum with missing values', () => {
const missingValue = {
cdk_path: 'path/to/enum',
missing_values: ['value2']
};

(updater as any).updateEnum('TestEnum', missingValue);

expect(mockEnumDeclaration.replaceWithText).toHaveBeenCalled();
expect(mockSourceFile.saveSync).toHaveBeenCalled();
});

it('should throw error if source file not found', () => {
// Update Project mock to return null for getSourceFile
(Project as jest.Mock).mockImplementation(() => ({
addSourceFilesAtPaths: jest.fn(),
getSourceFiles: jest.fn().mockReturnValue([]),
getSourceFile: jest.fn().mockReturnValue(null)
}));

updater = new MissingEnumsUpdater('./test-dir');

expect(() => {
(updater as any).updateEnum('TestEnum', {
cdk_path: 'invalid/path',
missing_values: ['value']
});
}).toThrow('Source file not found');
});
});

describe('updateEnumLike', () => {
beforeEach(() => {
mockClassDeclaration = {
forEachChild: jest.fn(),
addProperty: jest.fn().mockReturnValue({
setOrder: jest.fn(),
addJsDoc: jest.fn()
}),
} as any;

mockSourceFile = {
getClass: jest.fn().mockReturnValue(mockClassDeclaration),
saveSync: jest.fn(),
} as any;

// Update Project mock implementation
(Project as jest.Mock).mockImplementation(() => ({
addSourceFilesAtPaths: jest.fn(),
getSourceFiles: jest.fn().mockReturnValue([]),
getSourceFile: jest.fn().mockReturnValue(mockSourceFile)
}));

updater = new MissingEnumsUpdater('./test-dir');
});

it('should update enum-like class with missing values', () => {
const missingValue = {
cdk_path: 'path/to/class',
missing_values: ['new-value']
};

// Mock PropertyDeclaration
const mockProperty = {
getText: jest.fn().mockReturnValue('public static readonly EXISTING = new TestClass("existing")'),
getInitializer: jest.fn().mockReturnValue({
getKind: () => SyntaxKind.NewExpression
}),
getName: jest.fn().mockReturnValue('EXISTING'),
getInitializerIfKind: jest.fn().mockReturnValue({
getArguments: jest.fn().mockReturnValue(['existing'])
})
} as any;

mockClassDeclaration.forEachChild.mockImplementation(callback => callback(mockProperty));

(updater as any).updateEnumLike('testModule', 'TestClass', missingValue);

expect(mockClassDeclaration.addProperty).toHaveBeenCalled();
expect(mockSourceFile.saveSync).toHaveBeenCalled();
});
});
describe('execute', () => {
it('should execute the update process', async () => {
const mockMissingValuesPath = '/tmp/missing-values.json';

// Mock the methods
const analyzeMissingEnumValuesSpy = jest.spyOn(updater as any, 'analyzeMissingEnumValues')
.mockResolvedValue(mockMissingValuesPath);
const updateEnumLikeValuesSpy = jest.spyOn(updater as any, 'updateEnumLikeValues')
.mockImplementation(() => {});
const updateEnumValuesSpy = jest.spyOn(updater as any, 'updateEnumValues')
.mockImplementation(() => {});

await updater.execute();

expect(analyzeMissingEnumValuesSpy).toHaveBeenCalled();
expect(updateEnumLikeValuesSpy).toHaveBeenCalledWith(mockMissingValuesPath);
expect(updateEnumValuesSpy).toHaveBeenCalledWith(mockMissingValuesPath);
});
});

describe('removeAwsCdkPrefix', () => {
it('should remove aws-cdk prefix', () => {
expect((updater as any).removeAwsCdkPrefix('aws-cdk/path/to/file')).toBe('path/to/file');
expect((updater as any).removeAwsCdkPrefix('path/to/file')).toBe('path/to/file');
});
});

describe('getParsedEnumValues and getParsedEnumLikeValues', () => {
beforeEach(() => {
const mockFileContent = JSON.stringify({
module1: {
enum1: { enumLike: false, values: ['value1'] },
enum2: { enumLike: true, values: ['value2'] }
}
});
(fs.readFileSync as jest.Mock).mockReturnValue(mockFileContent);
});

it('should return only regular enums', () => {
const result = (updater as any).getParsedEnumValues();
expect(result.module1.enum1).toBeDefined();
expect(result.module1.enum2).toBeUndefined();
});

it('should return only enum-likes', () => {
const result = (updater as any).getParsedEnumLikeValues();
expect(result.module1.enum1).toBeUndefined();
expect(result.module1.enum2).toBeDefined();
});
});
});
Loading
Oops, something went wrong.
Loading
Oops, something went wrong.