[AVM Question/Feedback]: Output of what-if (if any) can't be trusted

[o-l-a-v]

Check for previous/existing GitHub issues

• I have checked for previous/existing GitHub issues

Description

AVM suffers the same problem as CARML: Bicep / ARM what-if doesn't handle nested modules, and thus shows little to no output. Which means it's spray and pray for deploying stuff with AVM, as what-if can't be trusted.

Is anything being done about this from AVM point of view?

Related issues:

• Resources in nested template not shown if reference is used in the parameter values arm-template-whatif#157

[neilmca-inc]

I don't think it matters where you get your modules from - the related issue you mention is pretty much where it is right now - awaiting a fix - but I don't think it's coming anytime soon to be a reliable yardstick

[o-l-a-v]

Sure, I agree. Yet doc doesn't mention a single issue about what-if:

• https://azure.github.io/Azure-Verified-Modules

So from AVM perspective it's like this issue doesn't exist.

[AlexanderSehr]

Hey @o-l-a-v,
thank you for your issue. I responded to a very similar issue recently in this comment: Azure/bicep-registry-modules#1292 (comment)

The long and short of it is that it is like @neilmca-inc said - it's not a module-specific issue (in this case AVM), but an overall flaw of the WhatIf functionality that should be raised here: https://github.com/Azure/arm-template-whatif (if not already existing).

It's unfortuantely not an issue we can do much about from our end as we cannot influence the language feature itself. And because of that I don't think it was considered to document the issue on our end either. That said, I of course agree with you, that it's quite a pitty that this is an issue in the first place and I hope the Product Group will get around to invest into it again soon.

cc: @jtracey93 @eriqua @ChrisSidebotham @matebarabas

[o-l-a-v]

Thanks @AlexanderSehr. AVM can however, and should in my opinion, inform users of AVM about this shortcoming. As in mention this limitation front and center in the documentation.

One of the main drivers for IaC is predictability by the ability to preview / plan / what-if. Be upfront about that shortcoming if choosing Bicep. Link to some issue people can vote on. And maybe suggest using AVM Terraform modules for now, if predictability is important.

One more thing: There already exists an issue in the repo you mentioned, I mentioned it in the issue description.

[AlexanderSehr]

Thanks @AlexanderSehr. AVM can however, and should in my opinion, inform users of AVM about this shortcoming. As in mention this limitation front and center in the documentation.

One of the main drivers for IaC is predictability by the ability to preview / plan / what-if. Be upfront about that shortcoming if choosing Bicep. Link to some issue people can vote on. And maybe suggest using AVM Terraform modules for now, if predictability is important.

One more thing: There already exists an issue in the repo you mentioned, I mentioned it in the issue description.

Hey @o-l-a-v I don't per se disagree - what-if IS very important and trust me I'd wish it would work better than it does too. I'm jut (in general) sceptical to documenting issues of external features that don't directly tie into a module's capabilties. In other words, it's on the 'solution'-side of the story, less than the CI/CD-side of this repository.

Back in CARML we used to have a page for 'Solution creation' which would have been a perfect place to call this issue out. The only Wiki the BRM repository has is the AVM Wiki (which to my knowledge doesn't have a counterpart as of today).

Regardless, if we were to call it out, where would you'd like to see this? In the AVM Wiki, here on the Public-Bicep-Registry CI/CD? Subjectively, I think the former would make more sense in the form of e.g., a 'Known issues' section. Please share your thoughts.

cc: @jtracey93 @eriqua @ChrisSidebotham @matebarabas

[o-l-a-v]

I like the idea of a "known issues/limitations" page on AVM wiki. Where issue inherited by using tools outside AVM are mentioned too, like ARM/Bicep what-if.

[AlexanderSehr]

Hey @o-l-a-v , thanks for the feedback. If you don't mind, I'll transfer the issue to the AVM repository where we'd work on the Wiki and can add the disclaimer.

[mbilalamjad]

@AlexanderSehr just double checking if you managed to add that disclaimer

[AlexanderSehr]

Hey @mbilalamjad,
not yet. I want to discuss with the team first where best to put it, but did not get around following up on it yet. It's on the agenda though :)

[watfordsuzy]

Just ran into this disaster today, quite unexpected. Should be highlighted in red in big bold print.

Gone back to our flat files (ouch) in order to get visibility into what's going on with deployments.

[AlexanderSehr]

Hey folks, just a quick update. We just agreed on an approach for "Known Issues" in the docs and @jtracey93 will add the described issue there.

Also, we'll ping the PG to see how they're progressing on this issue.

[jtracey93]

As promised we have now put a "known issues" page in AVM and this topic is mentioned there: https://azure.github.io/Azure-Verified-Modules/help-support/known-issues/#bicep-what-if-compatibility-with-modules

cc: @AlexanderSehr

[AlexanderSehr]

Will close the issue via the linked PR. @o-l-a-v let us know if you have any feedback 🚀

[o-l-a-v]

@jtracey93 and @AlexanderSehr: Thanks, I'm happy with the outcome. 🙂

Let's hope Bicep what-if is at a better place in three months:

• Resources in nested template not shown if reference is used in the parameter values arm-template-whatif#157 (comment)