Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(config): validate reconfigure branch #24699

Merged
merged 38 commits into from
Nov 3, 2023
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
d87eedd
feat: implement reconfigureLogic
RahulGautamSingh Sep 28, 2023
48b4ecf
refactor
RahulGautamSingh Sep 28, 2023
18e95ca
Apply suggestions from code review
RahulGautamSingh Sep 28, 2023
45bc7c8
rename function name
RahulGautamSingh Sep 29, 2023
044c755
ignore reconfigure branch while pruning
RahulGautamSingh Sep 29, 2023
2e94c72
add tests
RahulGautamSingh Oct 3, 2023
d28455f
more tests
RahulGautamSingh Oct 3, 2023
67e18f1
fix coverage
RahulGautamSingh Oct 3, 2023
b7c1dc1
Apply Suggestions
RahulGautamSingh Oct 4, 2023
0d466ab
use JSON5.parse
RahulGautamSingh Oct 4, 2023
6c6153e
Apply suggestions from code review
RahulGautamSingh Oct 4, 2023
bec3c67
Merge branch 'rename-second-parsePreset' of https://github.com/RahulG…
RahulGautamSingh Oct 4, 2023
803bbdc
fix formatting
RahulGautamSingh Oct 4, 2023
16d7cc3
don't cache configFileName
RahulGautamSingh Oct 4, 2023
c712e27
add docs
RahulGautamSingh Oct 4, 2023
e3fa7a2
Update docs/usage/config-validation.md
RahulGautamSingh Oct 4, 2023
98026f6
Apply suggestions from code review
RahulGautamSingh Oct 8, 2023
47fc0f4
refactor
RahulGautamSingh Oct 8, 2023
c46e3ff
fix lint issues
RahulGautamSingh Oct 8, 2023
88dafb9
Update docs/usage/config-validation.md
RahulGautamSingh Oct 8, 2023
ef3bb71
fix coverage
RahulGautamSingh Oct 8, 2023
a535bc5
Merge branch 'rename-second-parsePreset' of https://github.com/RahulG…
RahulGautamSingh Oct 8, 2023
abcf4cb
Apply suggestions from code review
RahulGautamSingh Oct 14, 2023
ca14c35
add tests
RahulGautamSingh Oct 15, 2023
250ba9e
merge
RahulGautamSingh Oct 15, 2023
3296472
fix lint issues
RahulGautamSingh Oct 15, 2023
bb71f60
Merge branch 'main' into rename-second-parsePreset
RahulGautamSingh Oct 15, 2023
eae1621
Merge branch 'main' into rename-second-parsePreset
rarkins Oct 23, 2023
aa59a7d
Update docs/usage/config-validation.md
RahulGautamSingh Oct 23, 2023
b2b3b7a
Apply suggestions from code review
RahulGautamSingh Oct 23, 2023
caae616
Merge branch 'main' into rename-second-parsePreset
RahulGautamSingh Oct 23, 2023
06969d8
skips validation if status check found
RahulGautamSingh Oct 24, 2023
e0ea724
Apply suggestions from code review
RahulGautamSingh Oct 26, 2023
6e051f6
apply suggestions
RahulGautamSingh Oct 26, 2023
1a203e4
fix logging message
RahulGautamSingh Oct 27, 2023
f551b9d
fix test name
RahulGautamSingh Oct 27, 2023
3db751a
Merge branch 'main' into rename-second-parsePreset
RahulGautamSingh Oct 30, 2023
ce96698
fix: set cache and branch status, if file is unparsable
RahulGautamSingh Oct 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 7 additions & 0 deletions lib/util/cache/repository/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,12 @@ export interface OnboardingBranchCache {
configFileParsed?: string;
}

export interface ReconfigureBranchCache {
reconfigureBranchSha?: string;
configFileName?: string;
isConfigValid?: boolean;
}

export interface PrCache {
/**
* Fingerprint of the PR body
Expand Down Expand Up @@ -129,6 +135,7 @@ export interface RepoCacheData {
};
prComments?: Record<number, Record<string, string>>;
onboardingBranchCache?: OnboardingBranchCache;
reconfigureBranchCache?: ReconfigureBranchCache;
}

export interface RepoCache {
Expand Down
2 changes: 2 additions & 0 deletions lib/workers/repository/finalize/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import * as repositoryCache from '../../../util/cache/repository';
import { clearRenovateRefs } from '../../../util/git';
import { configMigration } from '../config-migration';
import { PackageFiles } from '../package-files';
import { validateReconfigureBranch } from '../reconfigure';
import { pruneStaleBranches } from './prune';
import {
runBranchSummary,
Expand All @@ -16,6 +17,7 @@ export async function finalizeRepo(
config: RenovateConfig,
branchList: string[]
): Promise<void> {
await validateReconfigureBranch(config);
await configMigration(config, branchList);
await repositoryCache.saveCache();
await pruneStaleBranches(config, branchList);
Expand Down
7 changes: 7 additions & 0 deletions lib/workers/repository/finalize/prune.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,13 @@ describe('workers/repository/finalize/prune', () => {
expect(git.getBranchList).toHaveBeenCalledTimes(0);
});

it('ignores reconfigure branch', async () => {
delete config.branchList;
git.getBranchList.mockReturnValueOnce(['renovate/reconfigure']);
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
await cleanup.pruneStaleBranches(config, config.branchList);
expect(git.getBranchList).toHaveBeenCalledTimes(0);
});

it('returns if no renovate branches', async () => {
config.branchList = [];
git.getBranchList.mockReturnValueOnce([]);
Expand Down
6 changes: 4 additions & 2 deletions lib/workers/repository/finalize/prune.ts
Original file line number Diff line number Diff line change
Expand Up @@ -109,8 +109,10 @@ export async function pruneStaleBranches(
return;
}
// TODO: types (#22198)
let renovateBranches = getBranchList().filter((branchName) =>
branchName.startsWith(config.branchPrefix!)
let renovateBranches = getBranchList().filter(
(branchName) =>
branchName.startsWith(config.branchPrefix!) &&
branchName !== `${config.branchPrefix!}reconfigure`
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
);
if (!renovateBranches?.length) {
logger.debug('No renovate branches found');
Expand Down
2 changes: 1 addition & 1 deletion lib/workers/repository/init/merge.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ import {
import { OnboardingState } from '../onboarding/common';
import type { RepoFileConfig } from './types';

async function detectConfigFile(): Promise<string | null> {
export async function detectConfigFile(): Promise<string | null> {
const fileList = await scm.getFileList();
for (const fileName of configFileNames) {
if (fileName === 'package.json') {
Expand Down
141 changes: 141 additions & 0 deletions lib/workers/repository/reconfigure/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
import JSON5 from 'json5';
import upath from 'upath';
import type { RenovateConfig } from '../../../config/types';
import { validateConfig } from '../../../config/validation';
import { logger } from '../../../logger';
import { platform } from '../../../modules/platform';
import { scm } from '../../../modules/platform/scm';
import { getCache } from '../../../util/cache/repository';
import { readLocalFile } from '../../../util/fs';
import { getBranchCommit } from '../../../util/git';
import { detectConfigFile } from '../init/merge';
import {
deleteReconfigureBranchCache,
setReconfigureBranchCache,
} from './reconfigure-cache';

export async function validateReconfigureBranch(
config: RenovateConfig
): Promise<void> {
logger.debug('validateReconfigureBranch()');
const context = `renovate/config-validation`;

Check warning on line 21 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L20-L21

Added lines #L20 - L21 were not covered by tests

const branchName = `${config.branchPrefix}reconfigure`;
const branchExists = await scm.branchExists(branchName);

Check warning on line 24 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L23-L24

Added lines #L23 - L24 were not covered by tests

// this is something, the user initiates so skip if no branch exists
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
if (!branchExists) {
logger.debug('No reconfigure branch found');
deleteReconfigureBranchCache(); // inorder to remove cache when the branch deleted
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
return;

Check warning on line 30 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L28-L30

Added lines #L28 - L30 were not covered by tests
}

// look for config file
// 1. check reconfigure branch cache and use the conifgFileName if it exists
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
// 2. checkout reconfigure branch and look for the config file, don't assume default config fileName
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
const branchSha = getBranchCommit(branchName);
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
const cache = getCache();
let configFileName: string | null = null;
const branchCache = cache.reconfigureBranchCache;

Check warning on line 39 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L36-L39

Added lines #L36 - L39 were not covered by tests

// only use cached information if it is valid
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
if (branchCache?.reconfigureBranchSha === branchSha) {
logger.debug('Cache is valid. Skipping validation check');
return;

Check warning on line 44 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L43-L44

Added lines #L43 - L44 were not covered by tests
}
logger.debug('Cache is outdated. Performing validation check');

Check warning on line 46 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L46

Added line #L46 was not covered by tests

if (!configFileName) {
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
try {
await scm.checkoutBranch(branchName);
configFileName = await detectConfigFile();

Check warning on line 51 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L49-L51

Added lines #L49 - L51 were not covered by tests
} catch (err) {
logger.debug(

Check warning on line 53 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L53

Added line #L53 was not covered by tests
{ err },
'Error while searching for config file in reconfigure branch'
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
);
}
}

if (!configFileName) {
logger.warn('No config file found in reconfigure branch');
await platform.setBranchStatus({

Check warning on line 62 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L61-L62

Added lines #L61 - L62 were not covered by tests
branchName,
context,
description: 'Validation Failed - No config file found',
state: 'red',
secustor marked this conversation as resolved.
Show resolved Hide resolved
});
await scm.checkoutBranch(config.baseBranch!);
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
return;

Check warning on line 69 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L68-L69

Added lines #L68 - L69 were not covered by tests
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
}

let configFileRaw: string | null = null;
try {
configFileRaw = await readLocalFile(configFileName, 'utf8');

Check warning on line 74 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L72-L74

Added lines #L72 - L74 were not covered by tests
} catch (err) {
logger.debug('Error while reading config file');

Check warning on line 76 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L76

Added line #L76 was not covered by tests
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
}

if (!configFileRaw) {
logger.debug('Empty config file');
await platform.setBranchStatus({

Check warning on line 81 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L80-L81

Added lines #L80 - L81 were not covered by tests
branchName,
context,
description: 'Validation Failed - Empty config file',
state: 'red',
});
setReconfigureBranchCache(branchSha!, configFileName, false);
await scm.checkoutBranch(config.baseBranch!);
return;

Check warning on line 89 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L87-L89

Added lines #L87 - L89 were not covered by tests
}

let configFileParsed: any;
try {
const fileType = upath.extname(configFileName);

Check warning on line 94 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L93-L94

Added lines #L93 - L94 were not covered by tests
if (fileType === '.json') {
configFileParsed = JSON.parse(

Check warning on line 96 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L96

Added line #L96 was not covered by tests
(await readLocalFile(configFileName, 'utf8'))!
);
// ? should we allow package.json
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
// no need to confirm renovate field in package.json we already do it in `detectConfigFile()`
if (configFileName === 'package.json') {
configFileParsed = configFileParsed.renovate;

Check warning on line 102 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L102

Added line #L102 was not covered by tests
}
} else {
configFileParsed = JSON5.parse(

Check warning on line 105 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L105

Added line #L105 was not covered by tests
(await readLocalFile(configFileName, 'utf8'))!
);
}
} catch (err) {
logger.debug({ err }, 'Error while reading config file');
await scm.checkoutBranch(config.baseBranch!);
return;

Check warning on line 112 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L110-L112

Added lines #L110 - L112 were not covered by tests
}

// perform validation and provide a passing or failing check run based on result
const validationResult = await validateConfig(configFileParsed);

Check warning on line 116 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L116

Added line #L116 was not covered by tests

// failing check
if (validationResult.errors.length > 0) {
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
await platform.setBranchStatus({

Check warning on line 120 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L120

Added line #L120 was not covered by tests
branchName,
context,
description: 'Validation Failed',
state: 'red',
});
setReconfigureBranchCache(branchSha!, configFileName, false);
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
await scm.checkoutBranch(config.baseBranch!);
return;

Check warning on line 128 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L126-L128

Added lines #L126 - L128 were not covered by tests
}

// passing check
await platform.setBranchStatus({

Check warning on line 132 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L132

Added line #L132 was not covered by tests
branchName,
context,
description: 'Validation Successfull',
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
state: 'green',
});
setReconfigureBranchCache(branchSha!, configFileName, true);

Check warning on line 138 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L138

Added line #L138 was not covered by tests
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved

await scm.checkoutBranch(config.baseBranch!);

Check warning on line 140 in lib/workers/repository/reconfigure/index.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/index.ts#L140

Added line #L140 was not covered by tests
}
37 changes: 37 additions & 0 deletions lib/workers/repository/reconfigure/reconfigure-cache.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
import is from '@sindresorhus/is';
import { logger } from '../../../logger';
import { getCache } from '../../../util/cache/repository';

export function setReconfigureBranchCache(
reconfigureBranchSha: string,
configFileName: string,
isConfigValid: boolean
): void {
// do not update cache if commit is null/undefined
if (!is.nonEmptyString(reconfigureBranchSha)) {
logger.debug('Onboarding cache not updated');
return;

Check warning on line 13 in lib/workers/repository/reconfigure/reconfigure-cache.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/reconfigure-cache.ts#L12-L13

Added lines #L12 - L13 were not covered by tests
}

const cache = getCache();
const reconfigureBranchCache = {

Check warning on line 17 in lib/workers/repository/reconfigure/reconfigure-cache.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/reconfigure-cache.ts#L16-L17

Added lines #L16 - L17 were not covered by tests
reconfigureBranchSha,
configFileName,
isConfigValid,
};
if (cache.reconfigureBranchCache) {
logger.debug({ reconfigureBranchCache }, 'Update Reconfigure Branch Cache');

Check warning on line 23 in lib/workers/repository/reconfigure/reconfigure-cache.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/reconfigure-cache.ts#L23

Added line #L23 was not covered by tests
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
} else {
logger.debug({ reconfigureBranchCache }, 'Create Reconfigure Branch Cache');

Check warning on line 25 in lib/workers/repository/reconfigure/reconfigure-cache.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/reconfigure-cache.ts#L25

Added line #L25 was not covered by tests
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
}
cache.reconfigureBranchCache = reconfigureBranchCache;

Check warning on line 27 in lib/workers/repository/reconfigure/reconfigure-cache.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/reconfigure-cache.ts#L27

Added line #L27 was not covered by tests
}

export function deleteReconfigureBranchCache(): void {
const cache = getCache();

Check warning on line 31 in lib/workers/repository/reconfigure/reconfigure-cache.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/reconfigure-cache.ts#L31

Added line #L31 was not covered by tests

if (cache?.reconfigureBranchCache) {
logger.debug('Delete Reconfigure Branch Cache');
RahulGautamSingh marked this conversation as resolved.
Show resolved Hide resolved
delete cache.reconfigureBranchCache;

Check warning on line 35 in lib/workers/repository/reconfigure/reconfigure-cache.ts

View check run for this annotation

Codecov / codecov/patch

lib/workers/repository/reconfigure/reconfigure-cache.ts#L34-L35

Added lines #L34 - L35 were not covered by tests
}
}