Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse manifest in Rust instead of Python #1330

Merged
merged 11 commits into from
Apr 23, 2024
Merged

Conversation

orpuente-MS
Copy link
Contributor

@orpuente-MS orpuente-MS commented Mar 28, 2024

This PR regains some rust real-state. It delegates the parsing of the manifest in pip/qsharp/_qsharp.py/init() to rust, avoiding duplicate manifest-parsing logic in the project.

This change is needed to unblock #1313

Copy link

Benchmark for 9f91728

Click to view benchmark
Test Base PR %
Array append evaluation 352.0±2.34µs 338.2±2.69µs -3.92%
Array literal evaluation 193.4±2.71µs 173.7±1.07µs -10.19%
Array update evaluation 436.1±2.67µs 421.8±3.52µs -3.28%
Deutsch-Jozsa evaluation 5.1±0.05ms 5.0±0.05ms -1.96%
Large file parity evaluation 33.6±0.21ms 33.6±0.39ms 0.00%
Large input file 28.6±0.29ms 28.6±0.87ms 0.00%
Large nested iteration 34.5±0.54ms 33.2±0.27ms -3.77%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1485.3±21.06µs 1494.8±33.20µs +0.64%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.7±0.36ms 7.6±0.10ms -1.30%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1400.3±29.80µs 1406.2±30.38µs +0.42%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 21.2±0.18ms 21.2±0.19ms 0.00%
Standard library 16.4±0.04ms 16.4±0.05ms 0.00%
Teleport evaluation 82.0±3.89µs 82.3±3.88µs +0.37%

Copy link

Benchmark for 9cebb9a

Click to view benchmark
Test Base PR %
Array append evaluation 352.5±2.32µs 352.7±2.92µs +0.06%
Array literal evaluation 192.4±1.05µs 196.2±2.11µs +1.98%
Array update evaluation 434.5±2.39µs 437.3±2.09µs +0.64%
Deutsch-Jozsa evaluation 5.1±0.05ms 5.1±0.06ms 0.00%
Large file parity evaluation 33.6±0.09ms 33.6±0.06ms 0.00%
Large input file 30.3±1.10ms 30.5±1.15ms +0.66%
Large nested iteration 34.6±0.25ms 34.7±0.29ms +0.29%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1487.9±30.16µs 1485.8±30.37µs -0.14%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.7±0.09ms 7.7±0.10ms 0.00%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1404.2±40.09µs 1403.6±39.44µs -0.04%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 21.3±0.22ms 21.3±0.33ms 0.00%
Standard library 16.7±0.20ms 16.8±0.27ms +0.60%
Teleport evaluation 82.5±3.62µs 84.0±7.07µs +1.82%

Copy link

Benchmark for c481cc1

Click to view benchmark
Test Base PR %
Array append evaluation 354.5±7.53µs 356.8±4.69µs +0.65%
Array literal evaluation 193.0±2.29µs 193.5±4.75µs +0.26%
Array update evaluation 436.1±4.42µs 434.5±3.38µs -0.37%
Deutsch-Jozsa evaluation 5.1±0.05ms 5.1±0.06ms 0.00%
Large file parity evaluation 33.7±0.08ms 33.9±0.98ms +0.59%
Large input file 35.0±1.36ms 33.3±1.10ms -4.86%
Large nested iteration 34.5±0.24ms 34.6±0.34ms +0.29%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1526.5±62.99µs 1522.3±40.40µs -0.28%
Perform Runtime Capabilities Analysis (RCA) on large file sample 8.0±0.16ms 7.8±0.12ms -2.50%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1440.1±112.32µs 1420.1±58.83µs -1.39%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 21.7±0.25ms 21.5±0.25ms -0.92%
Standard library 18.1±0.86ms 17.7±0.72ms -2.21%
Teleport evaluation 82.0±3.78µs 82.3±3.70µs +0.37%

Copy link
Member

@minestarks minestarks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm afraid this change will interfere with how the notebook language service works (for which I'm realizing there's no integration tests, so nothing in the CI broke).

    # Return the configuration information to provide a hint to the
    # language service through the cell output.
    return Config(target_profile, language_features)

This line is critical for the language service - it causes the qsharp.json configuration to be written out to the notebook cell output, which the language service will read to configure compiler options. (Can give you a demo to show you what I mean, but you can try by changing languageFeatures and re-running qsharp.init to see how the squiggles change in the editor).

I think this requirement forces us to parse the qsharp.json at the Python level. Even if we parse it in Rust, we'd need to return the contents of the manifest back to Python... which will be too convoluted to be worth it IMO.

pip/src/interpreter.rs Outdated Show resolved Hide resolved
@orpuente-MS
Copy link
Contributor Author

I'm afraid this change will interfere with how the notebook language service works (for which I'm realizing there's no integration tests, so nothing in the CI broke).

    # Return the configuration information to provide a hint to the
    # language service through the cell output.
    return Config(target_profile, language_features)

This line is critical for the language service - it causes the qsharp.json configuration to be written out to the notebook cell output, which the language service will read to configure compiler options. (Can give you a demo to show you what I mean, but you can try by changing languageFeatures and re-running qsharp.init to see how the squiggles change in the editor).

I think this requirement forces us to parse the qsharp.json at the Python level. Even if we parse it in Rust, we'd need to return the contents of the manifest back to Python... which will be too convoluted to be worth it IMO.

I implemented the solution we talked about. Passing the raw manifest around and parsing it when it gets to rust land.

Copy link

github-actions bot commented Apr 4, 2024

Benchmark for b1d25bf

Click to view benchmark
Test Base PR %
Array append evaluation 339.9±2.30µs 339.9±2.67µs 0.00%
Array literal evaluation 181.6±26.53µs 175.5±5.65µs -3.36%
Array update evaluation 420.0±2.31µs 420.2±1.87µs +0.05%
Core + Standard library compilation 18.2±0.86ms 18.5±0.92ms +1.65%
Deutsch-Jozsa evaluation 5.0±0.05ms 5.0±0.05ms 0.00%
Large file parity evaluation 33.7±0.12ms 33.7±0.32ms 0.00%
Large input file compilation 12.2±0.47ms 11.9±0.63ms -2.46%
Large input file compilation (interpreter) 48.6±1.64ms 47.7±1.80ms -1.85%
Large nested iteration 33.1±0.68ms 33.0±0.43ms -0.30%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1538.7±166.07µs 1512.3±72.77µs -1.72%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.9±0.15ms 7.9±0.16ms 0.00%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1416.4±50.91µs 1421.7±59.85µs +0.37%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 21.8±0.30ms 21.7±0.34ms -0.46%
Teleport evaluation 80.5±3.57µs 80.4±3.55µs -0.12%

Copy link

github-actions bot commented Apr 4, 2024

Benchmark for 0386608

Click to view benchmark
Test Base PR %
Array append evaluation 355.2±10.09µs 340.4±2.22µs -4.17%
Array literal evaluation 199.7±6.43µs 180.6±2.33µs -9.56%
Array update evaluation 443.4±17.00µs 424.3±8.98µs -4.31%
Core + Standard library compilation 19.7±0.86ms 18.0±1.67ms -8.63%
Deutsch-Jozsa evaluation 5.1±0.06ms 5.1±0.15ms 0.00%
Large file parity evaluation 33.9±0.12ms 34.0±1.08ms +0.29%
Large input file compilation 12.6±0.38ms 11.2±0.13ms -11.11%
Large input file compilation (interpreter) 49.6±1.07ms 43.5±1.19ms -12.30%
Large nested iteration 34.2±0.33ms 33.1±0.37ms -3.22%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1518.1±55.44µs 1495.7±51.02µs -1.48%
Perform Runtime Capabilities Analysis (RCA) on large file sample 8.2±0.42ms 7.7±0.10ms -6.10%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1460.0±167.82µs 1424.4±146.61µs -2.44%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 22.0±0.63ms 21.2±0.88ms -3.64%
Teleport evaluation 86.9±9.25µs 83.7±5.42µs -3.68%

Copy link
Member

@minestarks minestarks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good but there seems to be a type error when sending the manifest through the language service.

I think this is the kind of scenario that would benefit from an integration test. LMK if you need help running/authoring those.

wasm/src/language_service.rs Outdated Show resolved Hide resolved
pip/src/interpreter.rs Outdated Show resolved Hide resolved
Co-authored-by: Mine Starks <16928427+minestarks@users.noreply.github.com>
Copy link

github-actions bot commented Apr 8, 2024

Benchmark for 6a1889f

Click to view benchmark
Test Base PR %
Array append evaluation 355.8±3.86µs 349.0±10.05µs -1.91%
Array literal evaluation 192.1±4.76µs 201.7±6.61µs +5.00%
Array update evaluation 434.6±6.55µs 430.3±6.00µs -0.99%
Core + Standard library compilation 16.6±0.10ms 16.5±0.32ms -0.60%
Deutsch-Jozsa evaluation 5.1±0.12ms 4.9±0.11ms -3.92%
Large file parity evaluation 33.7±0.24ms 32.8±0.83ms -2.67%
Large input file compilation 11.1±0.12ms 11.0±0.24ms -0.90%
Large input file compilation (interpreter) 42.7±0.88ms 41.6±0.99ms -2.58%
Large nested iteration 34.8±0.24ms 33.9±0.73ms -2.59%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1492.3±42.73µs 1480.5±39.95µs -0.79%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.5±0.21ms 7.6±0.17ms +1.33%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1403.1±36.10µs 1373.4±45.14µs -2.12%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 21.2±0.23ms 20.8±0.46ms -1.89%
Teleport evaluation 81.6±3.03µs 78.9±3.99µs -3.31%

Copy link

Benchmark for f7dfd63

Click to view benchmark
Test Base PR %
Array append evaluation 334.1±4.36µs 331.8±1.27µs -0.69%
Array literal evaluation 180.5±1.92µs 187.6±10.32µs +3.93%
Array update evaluation 418.5±3.50µs 416.7±2.46µs -0.43%
Core + Standard library compilation 16.9±0.17ms 17.0±0.33ms +0.59%
Deutsch-Jozsa evaluation 5.1±0.06ms 5.1±0.05ms 0.00%
Large file parity evaluation 33.7±0.28ms 33.6±0.41ms -0.30%
Large input file compilation 11.5±0.22ms 11.4±0.20ms -0.87%
Large input file compilation (interpreter) 44.6±1.01ms 44.3±1.00ms -0.67%
Large nested iteration 32.7±0.81ms 32.5±0.57ms -0.61%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1497.3±27.07µs 1499.2±47.56µs +0.13%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.9±0.08ms 7.8±0.09ms -1.27%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1411.0±29.66µs 1413.1±41.05µs +0.15%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 21.4±0.20ms 21.2±0.21ms -0.93%
Teleport evaluation 82.9±3.36µs 83.3±5.22µs +0.48%

Copy link

Benchmark for e500567

Click to view benchmark
Test Base PR %
Array append evaluation 345.5±1.76µs 333.6±3.01µs -3.44%
Array literal evaluation 199.1±4.44µs 180.8±2.50µs -9.19%
Array update evaluation 429.7±2.27µs 415.8±1.57µs -3.23%
Core + Standard library compilation 18.8±0.96ms 17.2±0.46ms -8.51%
Deutsch-Jozsa evaluation 5.2±0.05ms 5.1±0.05ms -1.92%
Large file parity evaluation 33.9±0.18ms 33.6±0.19ms -0.88%
Large input file compilation 11.6±0.36ms 12.0±0.45ms +3.45%
Large input file compilation (interpreter) 47.8±2.03ms 47.9±1.43ms +0.21%
Large nested iteration 33.9±0.31ms 32.6±1.09ms -3.83%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1551.7±157.40µs 1508.3±54.02µs -2.80%
Perform Runtime Capabilities Analysis (RCA) on large file sample 8.1±0.14ms 7.8±0.21ms -3.70%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1450.7±102.56µs 1424.0±50.65µs -1.84%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 22.2±0.35ms 21.4±0.27ms -3.60%
Teleport evaluation 82.6±3.53µs 81.7±3.54µs -1.09%

@orpuente-MS
Copy link
Contributor Author

Looking good but there seems to be a type error when sending the manifest through the language service.

I think this is the kind of scenario that would benefit from an integration test. LMK if you need help running/authoring those.

I fixed the type mismatch. I will add the integration test to #1313 instead of this PR. This is one is ready for review.

Copy link

Benchmark for 43ac472

Click to view benchmark
Test Base PR %
Array append evaluation 337.8±2.55µs 340.5±1.55µs +0.80%
Array literal evaluation 180.7±1.03µs 198.2±1.08µs +9.68%
Array update evaluation 421.5±1.66µs 425.1±2.67µs +0.85%
Core + Standard library compilation 17.3±0.33ms 17.7±0.49ms +2.31%
Deutsch-Jozsa evaluation 5.0±0.05ms 5.1±0.05ms +2.00%
Large file parity evaluation 33.6±0.20ms 33.6±0.26ms 0.00%
Large input file compilation 11.6±0.31ms 12.0±0.31ms +3.45%
Large input file compilation (interpreter) 43.8±1.35ms 48.9±1.42ms +11.64%
Large nested iteration 33.3±1.21ms 33.2±0.17ms -0.30%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1495.8±34.95µs 1501.1±39.83µs +0.35%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.8±0.12ms 7.8±0.09ms 0.00%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1411.0±44.66µs 1429.3±74.49µs +1.30%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 21.2±0.18ms 21.6±0.62ms +1.89%
Teleport evaluation 81.1±4.19µs 80.8±4.10µs -0.37%

Copy link
Member

@minestarks minestarks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks!

@orpuente-MS orpuente-MS added this pull request to the merge queue Apr 23, 2024
@orpuente-MS orpuente-MS removed this pull request from the merge queue due to a manual request Apr 23, 2024
@orpuente-MS orpuente-MS added this pull request to the merge queue Apr 23, 2024
Merged via the queue into main with commit 28817e4 Apr 23, 2024
17 checks passed
@orpuente-MS orpuente-MS deleted the oscarpuente/compiler-options branch April 23, 2024 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants