Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(manager/composer): support git-tags hostRules for github.com when updating artifacts #18004

Merged
merged 26 commits into from
Jan 13, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
c2b61dc
feat(manager/composer): support git-tags hostRules for github.com whe…
etremblay Sep 27, 2022
a03002c
Handle github token with x-access-token
etremblay Sep 27, 2022
0d3f540
Unit test also
etremblay Sep 27, 2022
87c6b6c
Merge branch 'main' into composer-private-repos-take2
etremblay Oct 3, 2022
58348bc
Merge branch 'main' into composer-private-repos-take2
etremblay Oct 5, 2022
3fff06d
Dont check for ghp_ prefix
etremblay Oct 5, 2022
f47008d
lint
etremblay Oct 5, 2022
725ee94
Improve personal access token logic and log debug if not PAT
etremblay Oct 5, 2022
4dcac6a
Merge branch 'main' into composer-private-repos-take2
etremblay Oct 5, 2022
de8c3ec
Merge branch 'main' into composer-private-repos-take2
etremblay Oct 19, 2022
79367aa
Update lib/modules/manager/composer/utils.ts
etremblay Oct 27, 2022
185474d
Update lib/modules/manager/composer/artifacts.ts
etremblay Oct 27, 2022
1a7afd5
Merge branch 'main' into composer-private-repos-take2
etremblay Oct 27, 2022
48496d2
Fix import and support fine-grained token
etremblay Oct 27, 2022
849d51e
Update docs/usage/getting-started/private-packages.md
etremblay Nov 8, 2022
965b72f
Update docs/usage/getting-started/private-packages.md
etremblay Nov 8, 2022
8579452
Update docs/usage/getting-started/private-packages.md
etremblay Nov 8, 2022
81e7877
Merge branch 'main' into composer-private-repos-take2
etremblay Nov 8, 2022
e5b9bce
Update docs/usage/getting-started/private-packages.md
etremblay Nov 8, 2022
c8d4bf4
Merge branch 'main' into composer-private-repos-take2
etremblay Dec 19, 2022
e1ab6ff
prettier
etremblay Dec 19, 2022
7614ad4
Fix code coverage
etremblay Dec 19, 2022
7cc5d75
Merge branch 'main' into composer-private-repos-take2
etremblay Jan 12, 2023
310d01b
Merge branch 'main' into composer-private-repos-take2
etremblay Jan 13, 2023
7853ed8
Replace regex by startWith and move logs to takePersonalAccessTokenIf…
etremblay Jan 13, 2023
18d5660
Merge branch 'main' into composer-private-repos-take2
etremblay Jan 13, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions docs/usage/getting-started/private-packages.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,25 @@ The following details the most common/popular manager artifacts updating and how

Any `hostRules` token for `github.com` or `gitlab.com` are found and written out to `COMPOSER_AUTH` in env for Composer to parse.
Any `hostRules` with `hostType=packagist` are also included.
For dependencies on `github.com` without a Packagist server: use a Personal Access Token for `hostRule` with `hostType=git-tags`, do not use an application token.
Avoid adding a `hostRule` with `hostType=github` because:

- it overrides the default Renovate application token for everything else
- it causes unwanted side effects

The repository in `composer.json` should have the `vcs` type with a `https` URL.
For example:

```json
{
"repositories": [
{
"type": "vcs",
"url": "https://github.com/organization/private-repository"
}
]
}
```

### gomod

Expand Down
124 changes: 122 additions & 2 deletions lib/modules/manager/composer/artifacts.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import * as docker from '../../../util/exec/docker';
import type { StatusResult } from '../../../util/git/types';
import * as hostRules from '../../../util/host-rules';
import * as _datasource from '../../datasource';
import { GitTagsDatasource } from '../../datasource/git-tags';
import { PackagistDatasource } from '../../datasource/packagist';
import type { UpdateArtifactsConfig } from '../types';
import * as composer from '.';
Expand Down Expand Up @@ -112,7 +113,13 @@ describe('modules/manager/composer/artifacts', () => {
hostRules.add({
hostType: 'github',
matchHost: 'api.github.com',
token: 'github-token',
token: 'ghp_github-token',
});
// This rule should not affect the result the Github rule has priority to avoid breaking changes.
hostRules.add({
hostType: GitTagsDatasource.id,
matchHost: 'github.com',
token: 'ghp_git-tags-token',
});
hostRules.add({
hostType: 'gitlab',
Expand Down Expand Up @@ -164,14 +171,127 @@ describe('modules/manager/composer/artifacts', () => {
cwd: '/tmp/github/some/repo',
env: {
COMPOSER_AUTH:
'{"github-oauth":{"github.com":"github-token"},"gitlab-token":{"gitlab.com":"gitlab-token"},"gitlab-domains":["gitlab.com"],"http-basic":{"packagist.renovatebot.com":{"username":"some-username","password":"some-password"},"artifactory.yyyyyyy.com":{"username":"some-other-username","password":"some-other-password"}},"bearer":{"packages-bearer.example.com":"abcdef0123456789"}}',
'{"github-oauth":{"github.com":"ghp_git-tags-token"},' +
'"gitlab-token":{"gitlab.com":"gitlab-token"},' +
'"gitlab-domains":["gitlab.com"],' +
'"http-basic":{' +
'"packagist.renovatebot.com":{"username":"some-username","password":"some-password"},' +
'"artifactory.yyyyyyy.com":{"username":"some-other-username","password":"some-other-password"}' +
'},' +
'"bearer":{"packages-bearer.example.com":"abcdef0123456789"}}',
COMPOSER_CACHE_DIR: '/tmp/renovate/cache/others/composer',
},
},
},
]);
});

it('git-tags hostRule for github.com set github-token in COMPOSER_AUTH', async () => {
hostRules.add({
hostType: GitTagsDatasource.id,
matchHost: 'github.com',
token: 'ghp_token',
});
fs.readLocalFile.mockResolvedValueOnce('{}');
const execSnapshots = mockExecAll();
fs.readLocalFile.mockResolvedValueOnce('{}');
const authConfig = {
...config,
registryUrls: ['https://packagist.renovatebot.com'],
};
git.getRepoStatus.mockResolvedValueOnce(repoStatus);
expect(
await composer.updateArtifacts({
packageFileName: 'composer.json',
updatedDeps: [],
newPackageFileContent: '{}',
config: authConfig,
})
).toBeNull();

expect(execSnapshots).toMatchObject([
{
options: {
env: {
COMPOSER_AUTH: '{"github-oauth":{"github.com":"ghp_token"}}',
},
},
},
]);
});

it('Skip github application access token hostRules in COMPOSER_AUTH', async () => {
hostRules.add({
hostType: 'github',
matchHost: 'api.github.com',
token: 'ghs_token',
});
hostRules.add({
hostType: GitTagsDatasource.id,
matchHost: 'github.com',
token: 'ghp_token',
});
fs.readLocalFile.mockResolvedValueOnce('{}');
const execSnapshots = mockExecAll();
fs.readLocalFile.mockResolvedValueOnce('{}');
const authConfig = {
...config,
registryUrls: ['https://packagist.renovatebot.com'],
};
git.getRepoStatus.mockResolvedValueOnce(repoStatus);
expect(
await composer.updateArtifacts({
packageFileName: 'composer.json',
updatedDeps: [],
newPackageFileContent: '{}',
config: authConfig,
})
).toBeNull();
expect(execSnapshots).toMatchObject([
{
options: {
env: {
COMPOSER_AUTH: '{"github-oauth":{"github.com":"ghp_token"}}',
},
},
},
]);
});

it('github hostRule for github.com with x-access-token set github-token in COMPOSER_AUTH', async () => {
viceice marked this conversation as resolved.
Show resolved Hide resolved
hostRules.add({
hostType: 'github',
matchHost: 'https://api.github.com/',
token: 'x-access-token:ghp_token',
});
fs.readLocalFile.mockResolvedValueOnce('{}');
const execSnapshots = mockExecAll();
fs.readLocalFile.mockResolvedValueOnce('{}');
const authConfig = {
...config,
registryUrls: ['https://packagist.renovatebot.com'],
};
git.getRepoStatus.mockResolvedValueOnce(repoStatus);
expect(
await composer.updateArtifacts({
packageFileName: 'composer.json',
updatedDeps: [],
newPackageFileContent: '{}',
config: authConfig,
})
).toBeNull();

expect(execSnapshots).toMatchObject([
{
options: {
env: {
COMPOSER_AUTH: '{"github-oauth":{"github.com":"ghp_token"}}',
},
},
},
]);
});

it('returns updated composer.lock', async () => {
fs.readLocalFile.mockResolvedValueOnce('{}');
const execSnapshots = mockExecAll();
Expand Down
19 changes: 16 additions & 3 deletions lib/modules/manager/composer/artifacts.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,26 +18,39 @@ import {
import { getRepoStatus } from '../../../util/git';
import * as hostRules from '../../../util/host-rules';
import { regEx } from '../../../util/regex';
import { GitTagsDatasource } from '../../datasource/git-tags';
import { PackagistDatasource } from '../../datasource/packagist';
import type { UpdateArtifact, UpdateArtifactsResult } from '../types';
import type { AuthJson, ComposerLock } from './types';
import {
extractConstraints,
findGithubToken,
getComposerArguments,
getPhpConstraint,
requireComposerDependencyInstallation,
takePersonalAccessTokenIfPossible,
} from './utils';

function getAuthJson(): string | null {
const authJson: AuthJson = {};

const githubCredentials = hostRules.find({
const githubToken = findGithubToken({
hostType: 'github',
url: 'https://api.github.com/',
});
if (githubCredentials?.token) {

const gitTagsGithubToken = findGithubToken({
hostType: GitTagsDatasource.id,
url: 'https://github.com',
});

const selectedGithubToken = takePersonalAccessTokenIfPossible(
githubToken,
gitTagsGithubToken
);
if (selectedGithubToken) {
authJson['github-oauth'] = {
'github.com': githubCredentials.token.replace('x-access-token:', ''),
'github.com': selectedGithubToken,
};
}

Expand Down
Loading