Skip to content

Implement QL Test support #173

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 22 commits into from
Mar 9, 2020
Merged

Implement QL Test support #173

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
444aca3
Implement QL Test support (using `odasa` for now)
dbartol Nov 20, 2019
ddf42d8
Fix LGTM alerts
dbartol Nov 20, 2019
85f7ff1
Attempt to fix pure test failure
dbartol Nov 20, 2019
be72e9b
Activate when Test Explorer view is opened
dbartol Nov 21, 2019
e37807c
Better error message when `odasa` is not configured properly
dbartol Nov 21, 2019
cbe3c05
`odasa` -> `Semmle Core`
dbartol Nov 21, 2019
7d325e3
Fix file-watching bug in QLTest discovery
dbartol Nov 21, 2019
9e6100f
Merge remote-tracking branch 'upstream/master' into dbartol/QLTest
dbartol Nov 22, 2019
2f9a314
Merge from master
dbartol Dec 4, 2019
55d1a4a
Merge from master
dbartol Dec 16, 2019
de2a6cc
Make QLTest use `codeql test run` instead of `odasa qltest`
dbartol Jan 3, 2020
207743e
Merge remote-tracking branch 'upstream/master' into dbartol/QLTest
dbartol Jan 3, 2020
0947a35
Implement cancellation of test run
dbartol Jan 6, 2020
edc1f1c
Merge remote-tracking branch 'upstream/master' into dbartol/QLTest
dbartol Jan 13, 2020
90a9753
Merge from master
dbartol Feb 15, 2020
1b7d0da
Fix typo
dbartol Feb 15, 2020
afcc05f
Remove unused interface
dbartol Feb 15, 2020
f2c525b
Fix references to renamed type
dbartol Feb 18, 2020
984ba73
Merge from master
dbartol Mar 3, 2020
370444c
Sort tests by name
dbartol Mar 3, 2020
39805bc
Remove unnecessary? import hack
dbartol Mar 6, 2020
7fc501f
Revert "Remove unnecessary? import hack"
dbartol Mar 6, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 4 additions & 7 deletions .vscode/launch.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@
"request": "launch",
"runtimeExecutable": "${execPath}",
"args": [
"--extensionDevelopmentPath=${workspaceRoot}/dist/vscode-codeql",
"--disable-extensions"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite understand the reason for removing --disable-extensions? I thought we had it in here to ensure that integration test failures couldn't be because some other extension was confusing things.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our extension now depends on the "VS Code Test Explorer" extension. If we launch VS Code with --disable-extensions, it disables the "VS Code Test Explorer" extension, and therefore refuses to load our own extension due to the disabled dependency.

"--extensionDevelopmentPath=${workspaceRoot}/dist/vscode-codeql"
],
"stopOnEntry": false,
"sourceMaps": true,
Expand All @@ -29,8 +28,7 @@
"runtimeExecutable": "${execPath}",
"args": [
"--extensionDevelopmentPath=${workspaceRoot}/dist/vscode-codeql",
"--extensionTestsPath=${workspaceRoot}/extensions/ql-vscode/out/test",
"--disable-extensions"
"--extensionTestsPath=${workspaceRoot}/extensions/ql-vscode/out/test"
],
"stopOnEntry": false,
"sourceMaps": true,
Expand All @@ -51,8 +49,7 @@
"runtimeExecutable": "${execPath}",
"args": [
"--extensionDevelopmentPath=${workspaceRoot}/dist/vscode-codeql",
"--extensionTestsPath=${workspaceRoot}/extensions/ql-vscode/out/vscode-tests/no-workspace/index",
"--disable-extensions"
"--extensionTestsPath=${workspaceRoot}/extensions/ql-vscode/out/vscode-tests/no-workspace/index"
],
"stopOnEntry": false,
"sourceMaps": true,
Expand All @@ -70,7 +67,7 @@
"args": [
"--extensionDevelopmentPath=${workspaceRoot}/dist/vscode-codeql",
"--extensionTestsPath=${workspaceRoot}/extensions/ql-vscode/out/vscode-tests/minimal-workspace/index",
"${workspaceRoot}/extensions/ql-vscode/test/data",
"${workspaceRoot}/extensions/ql-vscode/test/data"
],
"stopOnEntry": false,
"sourceMaps": true,
Expand Down
38 changes: 30 additions & 8 deletions common/config/rush/shrinkwrap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,8 @@ dependencies:
vscode-jsonrpc: 4.0.0
vscode-languageclient: 5.2.1
vscode-test: 1.2.0
vscode-test-adapter-api: 1.7.0
vscode-test-adapter-util: 0.7.0
webpack: 4.39.1
webpack-cli: 3.3.6
packages:
Expand Down Expand Up @@ -6122,6 +6124,21 @@ packages:
dev: false
resolution:
integrity: sha512-lTmS6AlAlMHOvPQemVwo3CezxBp0sNB95KNPkqp3Nxd5VFEnuG1ByM0zlRWos0zjO3ZWtkvhal0COgiV1xIA4A==
/vscode-test-adapter-api/1.7.0:
dev: false
engines:
vscode: ^1.23.0
resolution:
integrity: sha512-X0rTcoDhDBmpmJuev2C5+GHGZD41nmcRYoSe7iw5e9/aIPTOFve1T1F5x9gb+zXoNQnkXSDibyMkeHDKtIkqCg==
/vscode-test-adapter-util/0.7.0:
dependencies:
tslib: 1.10.0
vscode-test-adapter-api: 1.7.0
dev: false
engines:
vscode: ^1.24.0
resolution:
integrity: sha512-eAsB8koXct5JytvUcV62wLEBCQfsoclauzMLEFT6H0qBr1h8LyRc+dGDcs48pO28yFOo6VV+5AwCRLxTKh7TzQ==
/vscode-test/1.2.0:
dependencies:
http-proxy-agent: 2.1.0
Expand Down Expand Up @@ -6433,7 +6450,7 @@ packages:
child-process-promise: 2.2.1
fs-extra: 8.1.0
glob: 7.1.4
glob-promise: 3.4.0
glob-promise: /glob-promise/3.4.0/glob@7.1.4
gulp: 4.0.2
gulp-sourcemaps: 2.6.5
gulp-typescript: /gulp-typescript/5.0.1/typescript@3.7.2
Expand All @@ -6449,7 +6466,7 @@ packages:
dev: false
name: '@rush-temp/build-tasks'
resolution:
integrity: sha512-ta2kXnX7phnKrO7rxdJl5A9Vtd8B4RDyoae3vhdI1d+COeITaXDd9xdPxo8lvduPSJTw2+HnzOgOu2pMAKSjTw==
integrity: sha512-WBog5Gepo348LYeu7FAZMORJpgx0CNFM+IjgLcKNFYnN9YOCRQogUkRL7ZmBt8WXoew1S5OyVltmEM8XPy8i+w==
tarball: 'file:projects/build-tasks.tgz'
version: 0.0.0
'file:projects/semmle-bqrs.tgz':
Expand All @@ -6461,7 +6478,7 @@ packages:
dev: false
name: '@rush-temp/semmle-bqrs'
resolution:
integrity: sha512-24GdnvMbGfQIWMfgDhift+kYJDnG7dX03NrpX4ajZ2rckteysvq2/K7XI1OXGvUuqrt3m0/+GRDHpSI9XKDJJA==
integrity: sha512-ufPu8zLXf9JvPCRycWLiFTDb5rZ7bqxQZuiFjy1DAxnatEG5VJITPSXwDFVc11qpjJpaFd4hI+4QtOda7d5zww==
tarball: 'file:projects/semmle-bqrs.tgz'
version: 0.0.0
'file:projects/semmle-io-node.tgz':
Expand All @@ -6473,7 +6490,7 @@ packages:
dev: false
name: '@rush-temp/semmle-io-node'
resolution:
integrity: sha512-Bj0ax/bASrHV7tamOuXZZdd3UOB4NBKdjdszIRaDvDRTu8RlEst+TVoUhkfy30qb2/6ePp3/juOJyyiBJN7u8Q==
integrity: sha512-jB3C3WWEI991Kr3knPKUwqqNi040WmYCubLJJG7AK1nz3V1YjmyLHIAdjqwOgDNXYKCQPC6tlaEgljbs2Q/kIQ==
tarball: 'file:projects/semmle-io-node.tgz'
version: 0.0.0
'file:projects/semmle-io.tgz':
Expand All @@ -6484,7 +6501,7 @@ packages:
dev: false
name: '@rush-temp/semmle-io'
resolution:
integrity: sha512-NtyviDSevxbd+hj4J66LucOzo8LU2hJ1Jh0eHw0Qu3tRZPUT8HcQlseyy29AvZR8n8eppfEZiAm/JdiHfmRPMA==
integrity: sha512-6DFvjDclWTihDToSf31Hh+wQNhLGkA37l4QajeW/w6gS4NHjSrFL1qBlS7dLUY80VC/8nQJH9foe3r6dfEfQYw==
tarball: 'file:projects/semmle-io.tgz'
version: 0.0.0
'file:projects/semmle-vscode-utils.tgz':
Expand All @@ -6496,19 +6513,20 @@ packages:
dev: false
name: '@rush-temp/semmle-vscode-utils'
resolution:
integrity: sha512-5y5r8SDoN9Fp44naC9gUe8rOexeckXg2T0h9QCJAIcEgnFqOxzRc6Rv9gbMUStFKNh+rFlvmYmgPAdg5QkfgUg==
integrity: sha512-yE5S1wsnrsJ8lTt9O9ALedlvH37M9sWQha7sL5iQ3P6dn2KsyUItGsJDnFoh2f0wy3TpZuj3p/KTEDBbgjXBGg==
tarball: 'file:projects/semmle-vscode-utils.tgz'
version: 0.0.0
'file:projects/typescript-config.tgz':
dev: false
name: '@rush-temp/typescript-config'
resolution:
integrity: sha512-XuUIySaNoooIduvehnlKYaHqZJmmQoCqB1RtKhNszjCYZaSSJAnKVucViWBf5oNLKSNP7NchrD7gcoBlQ3xYvw==
integrity: sha512-kSFyvKy63jUHFVXQEzALiYfsTdn7J+Y7PcqtUVo9GndU5b5Xh3rBpVbZD1QN8+y8GfT0m/sdZZQVyH0h+On11Q==
tarball: 'file:projects/typescript-config.tgz'
version: 0.0.0
'file:projects/vscode-codeql.tgz':
dependencies:
'@types/chai': 4.1.7
'@types/child-process-promise': 2.2.1
'@types/classnames': 2.2.9
'@types/fs-extra': 8.0.0
'@types/glob': 7.1.1
Expand Down Expand Up @@ -6559,13 +6577,15 @@ packages:
vscode-jsonrpc: 4.0.0
vscode-languageclient: 5.2.1
vscode-test: 1.2.0
vscode-test-adapter-api: 1.7.0
vscode-test-adapter-util: 0.7.0
webpack: 4.39.1
webpack-cli: /webpack-cli/3.3.6/webpack@4.39.1
xml2js: 0.4.19
dev: false
name: '@rush-temp/vscode-codeql'
resolution:
integrity: sha512-DE97bdxda65gVLZne73QzBpj2hyCbyzvQiRZxrJqDP1rkF62EGNohBSmlEQs8H2Jp8hxh5RhPhm/yUx70G7KEA==
integrity: sha512-1zj1CO19sanU2olFV3Kwf9OrG+92KO25WR6K1vnVHmScMj8H7aegRrjwyNeHi/IzxXDCAp68CZK+d4vEWBEa+g==
tarball: 'file:projects/vscode-codeql.tgz'
version: 0.0.0
registry: 'https://registry.npmjs.org/'
Expand Down Expand Up @@ -6640,5 +6660,7 @@ specifiers:
vscode-jsonrpc: ^4.0.0
vscode-languageclient: ^5.2.1
vscode-test: ^1.0.0
vscode-test-adapter-api: ~1.7.0
vscode-test-adapter-util: ~0.7.0
webpack: ^4.38.0
webpack-cli: ^3.3.2
55 changes: 51 additions & 4 deletions extensions/ql-vscode/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,14 @@
"categories": [
"Programming Languages"
],
"extensionDependencies": [
"hbenl.vscode-test-explorer"
],
"activationEvents": [
"onLanguage:ql",
"onView:codeQLDatabases",
"onView:codeQLQueryHistory",
"onView:test-explorer",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this? If you have the test explorer open but no CodeQL workspace then I'm not sure we need to activate yet.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need it for the same reason we need the other onView: events: If I open a workspace that contains CodeQL tests, but have not yet opened a .ql file, I should still be able to browse and run my tests.

"onCommand:codeQL.checkForUpdatesToCLI",
"onCommand:codeQL.chooseDatabase",
"onCommand:codeQL.setCurrentDatabase",
Expand Down Expand Up @@ -95,14 +99,20 @@
"description": "Number of threads for running queries."
},
"codeQL.runningQueries.timeout": {
"type": ["integer", "null"],
"type": [
"integer",
"null"
],
"default": null,
"minimum": 0,
"maximum": 2147483647,
"description": "Timeout (in seconds) for running queries. Leave blank or set to zero for no timeout."
},
"codeQL.runningQueries.memory": {
"type": ["integer", "null"],
"type": [
"integer",
"null"
],
"default": null,
"minimum": 1024,
"description": "Memory (in MB) to use for running queries. Leave blank for CodeQL to choose a suitable value based on your system's available memory."
Expand All @@ -111,6 +121,22 @@
"type": "boolean",
"default": false,
"description": "Enable debug logging and tuple counting when running CodeQL queries. This information is useful for debugging query performance."
},
"codeQL.tests.semmleCoreDistributionPath": {
"scope": "window",
"type": "string",
"default": "",
"description": "Location of the Semmle Core distribution"
},
"codeQL.tests.semmleCoreLicensePath": {
"scope": "window",
"type": "string",
"description": "Location of the directory containing the Semmle Core license file"
},
"codeQL.tests.numberOfThreads": {
"scope": "window",
"type": "integer",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default to 1?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed to be 1, allowed range [1, 1024], just like the number of threads for queries.

"description": "Number of threads to use for CodeQL test execution"
}
}
},
Expand Down Expand Up @@ -178,6 +204,14 @@
{
"command": "codeQLQueryResults.previousPathStep",
"title": "CodeQL: Show Previous Step on Path"
},
{
"command": "codeQLTests.showOutputDifferences",
"title": "CodeQL: Show Test Output Differences"
},
{
"command": "codeQLTests.acceptOutput",
"title": "CodeQL: Accept Test Output"
}
],
"menus": {
Expand Down Expand Up @@ -213,6 +247,16 @@
"command": "codeQLQueryHistory.removeHistoryItem",
"group": "9_qlCommands",
"when": "view == codeQLQueryHistory"
},
{
"command": "codeQLTests.showOutputDifferences",
"group": "qltest@1",
"when": "view == test-explorer && viewItem == testWithSource"
},
{
"command": "codeQLTests.acceptOutput",
"group": "qltest@2",
"when": "view == test-explorer && viewItem == testWithSource"
}
],
"explorer/context": [
Expand Down Expand Up @@ -306,6 +350,7 @@
"format": "tsfmt -r"
},
"dependencies": {
"child-process-promise": "^2.2.1",
"classnames": "~2.2.6",
"fs-extra": "^8.1.0",
"glob-promise": "^3.4.0",
Expand All @@ -318,10 +363,13 @@
"tmp": "^0.1.0",
"unzipper": "~0.10.5",
"vscode-jsonrpc": "^4.0.0",
"vscode-languageclient": "^5.2.1"
"vscode-languageclient": "^5.2.1",
"vscode-test-adapter-api": "~1.7.0",
"vscode-test-adapter-util": "~0.7.0"
},
"devDependencies": {
"@types/chai": "^4.1.7",
"@types/child-process-promise": "^2.2.1",
"@types/classnames": "~2.2.9",
"@types/fs-extra": "^8.0.0",
"@types/glob": "^7.1.1",
Expand All @@ -341,7 +389,6 @@
"@types/xml2js": "~0.4.4",
"build-tasks": "^0.0.1",
"chai": "^4.2.0",
"child-process-promise": "^2.2.1",
"css-loader": "~3.1.0",
"glob": "^7.1.4",
"gulp": "^4.0.2",
Expand Down
22 changes: 22 additions & 0 deletions extensions/ql-vscode/src/cli.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,13 @@ export interface SourceInfo {
sourceLocationPrefix: string;
}

/**
* The expected output of `codeql resolve qlpacks`.
*/
export interface ResolvedQLPacks {
[index: string]: string[];
}

/**
* This class manages a cli server started by `codeql execute cli-server` to
* run commands without the overhead of starting a new java
Expand Down Expand Up @@ -303,6 +310,21 @@ export class CodeQLCliServer implements Disposable {
return await this.runJsonCodeQlCliCommand<QuerySetup>(['resolve', 'library-path'], subcommandArgs, "Resolving library paths");
}

/**
* Finds all available QL packs.
* @param workspaces The current open workspaces
* @param searchPath Overrides the default QL pack search path
*/
async resolveQLPacks(workspaces: string[], searchPath?: string[]): Promise<ResolvedQLPacks> {
const subcommandArgs = [
'--additional-packs', workspaces.join(path.delimiter)
];
if (searchPath !== undefined) {
subcommandArgs.push('--search-path', searchPath.join(path.delimiter));
}
return await this.runJsonCodeQlCliCommand<ResolvedQLPacks>(['resolve', 'qlpacks'], subcommandArgs, 'Resolving QL packs');
}

/**
* Gets the metadata for a query.
* @param queryPath The path to the query.
Expand Down
Loading