Support all natural languages which Gherkin provides

[gnuechtel]

• Support execution of all non-English feature files
• Add tests for Spanish and German
• Copy translation file from https://github.com/cucumber/cucumber/blob/master/gherkin/gherkin-languages.json
• Store non-English step files and Gherkin tests as UTF-8 with BOM

Fix #1165

[gnuechtel]

Merge with #1168 here:

https://github.com/cgnuechtel/Pester/tree/merge-CreateScenarioForEveryExample-GherkinMultipleLanguages

Just for your information ...

A merge conflict is expected in this pull request or #1168

[nohwnd]

@fourpastmidnight / @Jaykul could you give this a look and tell me if I should merge please :)

[fourpastmidnight]

@cgnuechtel, @nohwnd: I really like the changes in this PR. Adding in the languages file, in addition to getting us support for feature files in different languages, also gets us the ability to use the "Business Need" and "Ability" feature keyword synonyms and the "Example" scenario keyword synonym (which I myself was going to add!).

There are only two things which I would like to potentially see changed:

1. The location of the gherkin-languages.json file
2. The mechanism by which the gherkin-languages.json file is obtained/kept up to date

The location of the gherkin-languages.json file

Currently, Gherkin.dll is distributed in the lib/{core,legacy} folder. I think this folder could use a bit of organization and that the JSON languages file should really be located here, too, e.g.:

─ lib
  └─ Gherkin
    ├ core
    │  └ Gherkin.dll
    ├ legacy
    │  ├ Gherkin.dll
    │  └ Newtonsoft.Json.dll
    └ gherkin-languages.json

I believe this organization:

1. Better reflects the source of the libraries
2. Better groups together the dependencies
3. Could potentially better support a mechanism to automatically keep the languages file up to date.

A Mechanism by Which to Keep the Languages File Up To Date

I would love to see some mechanism by which Pester can attempt to keep the languages file up to date. It could be something as simple as attempting to download the language file via Invoke-WebRequest from the Cucumber GitHub repo via the build scripts when building and releasing a new version of Pester (see release.ps1 and associated scripts). A failure to retrieve the file should not result in a failure to build a release of Pester--Pester should just be released with the existing languages file. Though a warning should be emitted by the build script in case, for example, the Cucumber project decides to move the location of the file.

It should be noted that this file is located in several places within the Cucumber repository. It appears that https://github.com/cucumber/gherkin/gherkin-languages.json may be the canonical source.

[fourpastmidnight]

P.S. I would merge this prior to #1168.

[gnuechtel]

@fourpastmidnight Thanks for your suggestion! I have been changed the file structure. Who changes release.ps1? I could try it, but how to use it?

[gnuechtel]

One thing to note: I also introduced a small bug with this pull request which I have been fixed again. The description was not displayed in the output of the feature and scenario. I did not find a way to test it easily nor I saw any existing test for the output.

Have anybody an idea to test the output?

[nohwnd]

@cgnuechtel the release.ps1 is something I put together recently. Up until the previous version the module was released on TeamCity (TC) automatically, but I don't want to put my signature there on the "public" server, so I took all the scripts we have on TC and put them together in the release.ps1 so it's easier for me to do the release. You could simply add another script to the root of the repo, call it updateGherkinLanguageFile.ps1 and link it from the release.ps1 script as I do with all the other scripts, just make it throw if it cannot get the file, otherwise I won't have any chance to solve the problem if there is just a warning.

To re-organize the gherkin folder you will need to also change the nuget definition so the package remains deployable. You can try that out by commenting out the sign step in the release.ps1 so the script will only delete unneed files (like tests) and them packs them up in the build folder into nuget and powershell gallery packages. (It will delete files in your repo so make sure all changes you want to keep are comitted/staged.)

Or you can just change the folder structure, and add the download script and I will handle the release script and packaging. Your call :)

[gnuechtel]

@fourpastmidnight

It should be noted that this file is located in several places within the Cucumber repository. It appears that https://github.com/cucumber/gherkin/gherkin-languages.json may be the canonical source.

That resource is not available.
Please look at https://github.com/cucumber/gherkin (or https://github.com/cucumber-attic/gherkin):
"The Gherkin project has moved to the cucumber/cucumber monorepo."

@nohwnd

I will create updateGherkinLanguageFile.ps1 which checks for https://raw.githubusercontent.com/cucumber/cucumber/master/gherkin/gherkin-languages.json

[gnuechtel]

@fourpastmidnight and @nohwnd Please check again. Everything should be implemented now

[Jaykul]

Regarding the changes to add text to the output -- have we verified this doesn't break the NUnit XML output (or parsing of it by 3rd parties like AppVeyor or Azure DevOps Services)?

[fourpastmidnight]

@cgnuechtel Sorry about that bum link--I "munged" it together myself and obviously got it wrong, but it appears you got the file I was looking for. Thanks for that.

I need to review this further the changes (on the surface, they look good).

@Jaykul I didn't personally verify whether or not the NUnit XML is broken or not--though, I don't know why making textual changes would break that. Apparently the AppVeyor build is passing--is there something else we need to be concerned with? And again, nothing has been verified wrt Azure DevOps. Maybe we could setup an Azure DevOps pipeline build? Is there anything in particular that you think would break wrt to an Azure DevOps pipeline and this PR? @nohwnd Any thoughts on setting up an Azure DevOps pipeline build? Do we need it? Would it provide any benefits over the 3 builds we already have setup for Pester?

[Jaykul]

The only reason I'm asking is because you now have text that used to always be a one-liner, but is now sometimes many lines, and I'm wondering what happens in the XML.

The only way I can think of to test it is to output the XML from a multi-line test and verify you can [xml](get-content ... ) on it.

[nohwnd]

Yes I plan to mig migrate from the community build server to azure pipelines soon. Actually I am just deploying a server to run powershell 2 on it to try out the setup. build.powershell.org proves to be difficult to work with and I would rather a self hosted server to sign my code and do the deployments.

[Jaykul]

@nohwnd I haven't set this up myself yet, but ... @onovotny's SignService is on my list for code-signing -- I'm hoping I can use that for signing, and then actually deploy a VM just to run tests on PowerShell 2.

For my purposes, I don't want to build in PS2 (way too frustrating to make build tools/scripts work there), but I do want to run my tests there to make sure the module works. I also don't want to leave a VM running all the time just for that -- although, now that I think about it ... I did just get the extended Azure credits ...

[nohwnd]

@Jaykul I was thinking to have a orchestrator server and a VM where I will run the tests via remoting, but in the end I "hacked" azure pipelines agent into running over powershell 6 on powershell 2 enabled server. Imho making the agent pick up a portable copy of PowerShell 6 and then run it "natively" on Server 2008 is the simplest option. Then it seems that the only step needed is to make the tasks invoker run without -Sta switch that PowerShell 6 does not have.

[gnuechtel]

What is missing to complete this pull request?

[fourpastmidnight]

I need to pull down the latest on this and check it out. I just haven’t gotten a chance yet. Of one concern that @Jaykul bored was the NUnit output. Have you tested that with your changes? I’m not currently making use of Nunit output, so I would have a hard time verifying this myself. Other than that, I just need to perform a once over on it. I just haven’t quite got to it yet after the holidays.

…

Sent from my Windows 10 phone From: Christian Gnüchtel Sent: Thursday, January 3, 2019 19:34 To: pester/Pester Cc: Craig E. Shea; Mention Subject: Re: [pester/Pester] Support all natural languages which Gherkinprovides (#1166) What is missing to complete this pull request? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[gnuechtel]

@fourpastmidnight I have been added NUnit XML tests for all languages under test

[nohwnd]

@cgnuechtel I don't think anything is missing, it just has been a bit overwhelming during holidays. I will have a look on it this evening, to see if everything is correctly packaged. Hopefully there will be another beta release today.

[nohwnd]

@cgnuechtel I tried merging this with master but get 26 failed tests (including ~8 version and style tests) most of them failing on counts. Generating scenarios for every example probably broke some stuff. Could you merge it please?

[nohwnd]

Thank you.