Recognize individual expressions when nested

[radeksimko]

Background (current state)

Through the logic in hcl-lang/schema and related schema in terraform-schema we currently provide completion, hover and semantic highlighting for the outermost expression (with a few exceptions for HCL objects and tuples) and generally don't take into account (potentially infinite) nesting.

Completion

resource "aws_instance" "example" {
  ami = # HERE
  depends_on = [ HERE ]
}

For ami we can provide completion of any string candidates, for completion within depends_on brackets we can provide references to other resources and data sources.

Hover

resource "aws_instance" "example" {
  ami = "test"
  depends_on = [ aws_vpc.test ]
}

When hovering over "test" we can match the string constant against schema (string type) and show relevant popup ("test" string). Similarly when hovering over [ aws_vpc.test ] we can match it against schema (set of references) and show relevant popup (set of references).

Semantic Tokens

resource "aws_instance" "example" {
  ami = "test"
  depends_on = [ aws_vpc.test ]
}

Here "test" is expected to be string (per schema) and aws_vpc.test is expected to be a reference (per schema), so LS will report relevant semantic tokens in both cases.

Use-cases

User would benefit from recognition of arbitrarily nested expressions in all relevant functions of the LS, as described in more detail below.

Completion

resource "aws_instance" "example" {
  ami = "ami-${HERE}"
  depends_on = [
    aws_vpc.test,
    # HERE
  ]
}

within ${} we should provide any compatible expressions (presumably all primitive types).
for the second position we should provide any available references other than aws_vpc.test.

Hover

resource "aws_instance" "example" {
  ami = "ami-${data.aws_ami.example.test.image_id}"
  depends_on = [
    aws_vpc.test,
  ]
}

when hovering over data.aws_ami.example.test.id we display popup with details about the reference (string The ID of the AMI.)
when hover over aws_vpc.test we display popup with details about the reference (resource).

Semantic Tokens

resource "aws_instance" "example" {
  ami = "ami-${data.aws_ami.example.test.image_id}"
  cpu_core_count = 2 * var.core_count
}

"ami-${data.aws_ami.example.test.image_id}" is highlighted as a template expression with ami- highlighted as string and data.aws_ami.example.test.image_id as reference.

2 * var.core_count is highlighted as multiplication expression with 2 highlighted as number and var.core_count as a reference.

Proposal

• Add (failing) tests to cover existing nested expressions
  • completion
    • inside hclsyntax.TupleConsExpr (attr = [ var.foo, _ ])
    • inside hclsyntax.ObjectConsItem (attr = { foo = _ }) in the context of object/map as value & object({}) type declaration
    • inside hclsyntax.ObjectConsExpr (attr = { _ }) in the context of object/map as value & object({}) type declaration
    • inside hclsyntax.TemplateExpr (attr = "${_}")
    • inside hclsyntax.FunctionCallExpr (attr = fun(_)) in the context of type declarations such as list(), set(), map() & built-in functions
  • hover
    • inside hclsyntax.TupleConsExpr (attr = [ var.foo, _ ])
    • inside hclsyntax.ObjectConsItem (attr = { foo = _ }) in the context of object/map as value & object({}) type declaration
    • inside hclsyntax.ObjectConsExpr (attr = { _ }) in the context of object/map as value & object({}) type declaration
    • inside hclsyntax.TemplateExpr (attr = "${_}")
    • inside hclsyntax.FunctionCallExpr (attr = fun(_)) in the context of type declarations such as list(), set(), map() & built-in functions
  • semantic tokens
    • inside hclsyntax.TupleConsExpr (attr = [ var.foo, _ ])
    • inside hclsyntax.ObjectConsItem (attr = { foo = _ }) in the context of object/map as value & object({}) type declaration
    • inside hclsyntax.ObjectConsExpr (attr = { _ }) in the context of object/map as value & object({}) type declaration
    • inside hclsyntax.TemplateExpr (attr = "${_}")
    • inside hclsyntax.FunctionCallExpr (attr = fun(_)) in the context of type declarations such as list(), set(), map() & built-in functions
• hashicorp/hcl-lang
  • Deprecate schema.TupleConsExpr
  • Create Expression interface in decoder such that completion for any expression type can be delegated and each type can also delegate completion of nested expressions to other expression types
  • Implement decoder.NewExpression(expr hcl.Expression, cons schema.ExprConstraints) Expression
  • Implement schema.ObjectTypeDeclItemExpr
  • Implement new completion candidate kinds?
    • lang.PrimitiveTypeDeclExpr
    • lang.ComplexTypeDeclExpr
    • lang.ObjectTypeDeclItemExpr
  • Implement schema.AnyExprOfType{Type: cty.Type} to act as an alias to
    • schema.LiteralTypeExpr
    • schema.TraversalExpr
    • (future) dynamically express function calls which return the Type
    • (future) dynamically express templates which return the Type
    • (future) dynamically express conditional expressions which return the Type
    • (future) dynamically express operator expressions which return the Type
  • Implement LiteralValueExpr as decoder.Expression
  • Implement ScopeTraversalExpr as decoder.Expression (for type declarations, references and keywords)
  • Implement TupleConsExpr as decoder.Expression
  • Implement ObjectConsExpr as decoder.Expression
  • Implement (basic) FunctionCallExpr as decoder.Expression (for type declarations)
  • Implement (basic) TemplateExpr as decoder.Expression
  • Implement (basic) TemplateWrapExpr as decoder.Expression
  • Plumb all new decoder.Expression types into PathDecoder - CandidatesAtPos
  • Plumb all new decoder.Expression types into PathDecoder - HoverAtPos
  • Plumb all new decoder.Expression types into PathDecoder - SemanticTokensInFile

type Expression interface {
	CandidatesAtPos(ctx context.Context, pos hcl.Pos) []lang.Candidate
	HoverAtPos(ctx context.Context, pos hcl.Pos) *lang.HoverData
	SemanticTokens(ctx context.Context) []lang.SemanticToken
	// TODO: Validate()
}

Prototype WIP: hashicorp/hcl-lang@d21a863

[radeksimko]

Most of the basic combinations were implemented as part of recent PRs (#1237, hashicorp/hcl-lang#232, hashicorp/hcl-lang#203, hashicorp/hcl-lang#199, hashicorp/hcl-lang#186, hashicorp/hcl-lang#185, hashicorp/hcl-lang#184) which were also released in v0.31.0.

Finished & remaining work related to nested expressions is tracked in the following issues:

• UX around expression types

  • Support step-based reference completion #492
  • Recognize individual expressions when nested #496
  • Provide function signature help #857
  • Reflect convertible types in type comparison #583
  • Show interpolated expression values on hover #495
  • Reflect nullable variables in Terraform 1.1.0+ #894
  • Reflect new resource. address and _ escaping block in 0.15.4+ #893

• Local References

  • schema: Introduce Extensions to BodySchema hcl-lang#118
  • Support self.* references in provisioner, connection and lifecycle blocks #859
  • Support count.index references in blocks w/ count #860
  • Support each.* references in blocks w/ for_each #861
  • Support for dynamic block #530
  • Go-to-definition/go-to-references for count.index/count #1093
  • Go-to-definition/go-to-references for each.*/for_each #1095
  • Go-to-definition/go-to-references for self.* in provisioner blocks etc. #1096

• Expression Types Support

  • Support for type declarations (for variable type) #490
  • Solidify handling of optional/default object attributes #917
  • Support built-in functions #37
  • Support ${...} template expressions in attribute values #522
  • Support splat expressions (*) #526
  • Support for-in-if expressions #527
  • Support conditional expressions (condition ? true : false) #528
  • Support operator expressions (+, -, !=, !...) and parentheses #529
  • Support more complex index expressions (elem[key]) #531
  • Support relative references / traversals #532

There is also a meta-issue tracking all of the above in hashicorp/vscode-terraform#641

Therefore I am going to close this issue.

[github-actions]

This functionality has been released in v0.31.0 of the language server.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.