Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reenable TemplateEngine tests #42267

Merged
merged 4 commits into from
Jul 23, 2024
Merged

Conversation

v-wuzhai
Copy link
Member

@v-wuzhai v-wuzhai commented Jul 22, 2024

fixes: #42260

@v-wuzhai v-wuzhai requested a review from a team as a code owner July 22, 2024 05:58
@dotnet-issue-labeler dotnet-issue-labeler bot added Area-Infrastructure untriaged Request triage from a team member labels Jul 22, 2024
@akoeplinger
Copy link
Member

I'm not sure this is a good long-term strategy, this package is being updated regularly so we'll run into this again in the future:
image

I checked some of the other tests and they mostly use packages that haven't been updated in years, maybe we should just switch to one of these?

@v-wuzhai
Copy link
Member Author

looks like there is already a fixed version test: https://github.com/dotnet/sdk/blob/main/test/dotnet-new.Tests/DotnetNewDetailsTest.Approval.cs#L47

@akoeplinger
Copy link
Member

Yeah, the test is supposed to test not passing a version but I think that won't really work with a package that is frequently updated. In other tests we use NUnit3.DotNetNew.Template which was last updated in 2022, so we could switch to that.

Package version: 4.0.3038
Details: Collection of templates for Azure Functions
NUnit3.DotNetNew.Template
Package version: 1.9.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any way to change the test comparison logic to not actually check the details and package version? If we don't control the template, those could change at any time and break the test (even if we do control it, it might need to change). Having a test gate on a dynamic set of values like that is not great.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that almost all the test failures are due to version issues, so we can simply check if it is the latest version.

test/dotnet-new.Tests/DotnetNewDetailsTest.Approval.cs Outdated Show resolved Hide resolved
throw new Exception("No versions found.");
}

return versions.Last();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI the last entry is not necessarily the latest stable version, e.g. when we have previews, but looks like we don't need to care about that since templates don't support preview versions: dotnet/templating#1423

@marcpopMSFT marcpopMSFT merged commit c58f4e9 into main Jul 23, 2024
37 checks passed
@marcpopMSFT marcpopMSFT deleted the dev/Jason/Reenable-TemplateEngine-tests branch July 23, 2024 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Infrastructure untriaged Request triage from a team member
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Two TemplateEngine tests are failing because a new version got pushed
3 participants