Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compile-time variable imports #11657

Merged
merged 23 commits into from
Sep 15, 2023
Merged

Compile-time variable imports #11657

merged 23 commits into from
Sep 15, 2023

Conversation

jeskew
Copy link
Contributor

@jeskew jeskew commented Aug 28, 2023

Partially addresses #10121

This PR updates the compileTimeImports experimental feature to support variables in addition to type declarations. var statements can be the target of an @export() decorator (provided that they only contain references to other variables and not to resources, modules, or parameters), and an imported variable can be used just like one declared in the importing template.

Some aspects of this feature that should get special scrutiny:

  • Variables in an ARM template don't have anywhere to put metadata, so I've added a template-level metadata property that contains the names (and descriptions, if any) of any exported variables.
  • Imported variables are analyzed twice (once to determine their type, and then separately when they are migrated to the importing template in TemplateWriter), so there is room to improve efficiency
  • The translation from a "copy" property to a for loop expression is not perfectly roundtrippable:
    • copyIndex(<var name>, 1) will be transformed to add(copyIndex(<var name>), 1)
    • The count copy property will be transformed to length(range(0, X)), where X is the original count ARM expression.
Microsoft Reviewers: Open in CodeFlow

@jeskew jeskew requested a review from a team August 28, 2023 21:52
@github-actions
Copy link
Contributor

github-actions bot commented Aug 28, 2023

Test Results (osx-x64)

       33 files  ±0         33 suites  ±0   1h 24m 19s ⏱️ - 6m 41s
10 459 tests +9  10 459 ✔️ +9  0 💤 ±0  0 ±0 
12 677 runs  +6  12 677 ✔️ +6  0 💤 ±0  0 ±0 

Results for commit 627f88d. ± Comparison against base commit 6904c4a.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 28, 2023

Test Results (win-x64)

       33 files  ±0         33 suites  ±0   41m 51s ⏱️ + 6m 29s
10 467 tests +9  10 467 ✔️ +9  0 💤 ±0  0 ±0 
12 684 runs  +6  12 684 ✔️ +6  0 💤 ±0  0 ±0 

Results for commit 627f88d. ± Comparison against base commit 6904c4a.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 28, 2023

Test Results (linux-musl-x64)

       33 files  ±0         33 suites  ±0   34m 18s ⏱️ - 4m 15s
10 455 tests +9  10 455 ✔️ +9  0 💤 ±0  0 ±0 
12 673 runs  +6  12 673 ✔️ +6  0 💤 ±0  0 ±0 

Results for commit 627f88d. ± Comparison against base commit 6904c4a.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 28, 2023

Test Results (linux-x64)

       33 files  ±0         33 suites  ±0   30m 57s ⏱️ -36s
10 455 tests +9  10 455 ✔️ +9  0 💤 ±0  0 ±0 
12 673 runs  +6  12 673 ✔️ +6  0 💤 ±0  0 ±0 

Results for commit 627f88d. ± Comparison against base commit 6904c4a.

♻️ This comment has been updated with latest results.

# Conflicts:
#	src/Bicep.Core/Semantics/ImportedTypeSymbol.cs
#	src/Bicep.Core/Semantics/WildcardImportSymbol.cs
#	src/Bicep.LangServer/Handlers/BicepHoverHandler.cs
{
AssertSyntaxType(originalSymbolName, nameof(originalSymbolName), typeof(StringSyntax), typeof(IdentifierSyntax), typeof(SkippedTriviaSyntax));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the use case for supporting a string in the originalSymbolName field?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Names in ARM templates don't have to be valid Bicep identifiers. You can have and use a variable named '@!$^%#, and there are only a few ways we could deal with that when importing from a compiled template:

  1. Raise a diagnostic on the import statement indicating that the imported template has issues, even though it doesn't violate any of ARM's rules.
  2. Ignore any import whose name isn't a valid identifier in Bicep.
  3. Allow import statements to use a string for the original symbol name but require that such imports alias the symbol to a valid identifier (e.g., import {'a-b' as ab} from 'template.json'.

I didn't feel great about option 2 because it silently drops exported symbols without explaining why to the user. Option 1 has a couple problems that made me think it's not a viable solution:

  1. There's no guarantee that the author of the template with the import statement can edit the imported artifact (they may be importing a template spec or registry module to which they have read privileges only).
  2. The user will have to search the imported template for the problem, since the diagnostic will be associated with the import statement in the consuming template rather than the syntax that exports the symbol from the offending template.

I didn't see any drawbacks to option 3 besides some additional complexity in the parser. Similar syntax is supported in js imports, so there was a good example to follow.

_ => throw new InvalidOperationException($"Unrecognized ARM LanguageExpression of type {expression.GetType().Name} encountered."),
};

private static string SerializeExpression(FunctionExpression functionExpression)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there anything in the deployments engine to do this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not that I could find, although there probably should be. I can open a PR to copy this code into a public method in one of the Azure.Deployments. nuget packages, but removing this code from Bicep would need to happen in a future PR.

return collector.symbolsReferenced;
}

internal static IEnumerable<DeclaredSymbol> CollectSymbolsReferencedRecursive(IBinder binder, DeclaredSymbol symbol)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised this functionality doesn't already exist in the codebase!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do something very similar in CyclicCheckVisitor, but I wasn't able to reuse that code path since there are some special cases where we permit syntactic cycles.

# Conflicts:
#	src/Bicep.Core/Emit/CompileTimeImports/ImportClosureInfo.cs
#	src/Bicep.Core/Semantics/ImportedTypeSymbol.cs
#	src/Bicep.Core/Semantics/SemanticDiagnosticVisitor.cs
#	src/Bicep.Core/Semantics/WildcardImportSymbol.cs
#	src/Bicep.Core/Workspaces/SourceFileGrouping.cs
#	src/Bicep.LangServer/Completions/BicepCompletionProvider.cs
#	src/Bicep.LangServer/Handlers/BicepDefinitionHandler.cs
#	src/Bicep.LangServer/Handlers/BicepHoverHandler.cs
# Conflicts:
#	src/Bicep.Core/Semantics/ArmTemplateSemanticModel.cs
#	src/Bicep.Core/Semantics/ISemanticModel.cs
#	src/Bicep.Core/Semantics/SemanticModel.cs
#	src/Bicep.Core/Semantics/TemplateSpecSemanticModel.cs
#	src/Bicep.Core/Workspaces/SourceFileGrouping.cs
Copy link
Member

@anthony-c-martin anthony-c-martin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚢 looks great!

You'll need to merge the latest from main into this branch for the renamed CI jobs to be picked up.

@jeskew jeskew enabled auto-merge (squash) September 15, 2023 13:17
@github-actions
Copy link
Contributor

github-actions bot commented Sep 15, 2023

Test this change out locally with the following install scripts (Action run 6198858998)

VSCode
  • Mac/Linux
    bash <(curl -Ls https://aka.ms/bicep/nightly-vsix.sh) --run-id 6198858998
  • Windows
    iex "& { $(irm https://aka.ms/bicep/nightly-vsix.ps1) } -RunId 6198858998"
Azure CLI
  • Mac/Linux
    bash <(curl -Ls https://aka.ms/bicep/nightly-cli.sh) --run-id 6198858998
  • Windows
    iex "& { $(irm https://aka.ms/bicep/nightly-cli.ps1) } -RunId 6198858998"

@jeskew jeskew merged commit 659815a into main Sep 15, 2023
45 of 46 checks passed
@jeskew jeskew deleted the jeskew/variable-imports branch September 15, 2023 13:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants