Add way to disambiguate multiple tags in feature files #79
Comments
alexviz7
changed the title
|
Hey @alexviz7, thanks for creating the issue. I don't think I fully understand it yet. Would you mind providing a minimal reproducible example containing your tag configuration? You don't have to include step definitions or your real steps. I'm just a little bit confused still, especially about the
This error indicates that Xray wasn't properly configured. You seem to be using a server instance, which requires the Please keep in mind that the Cucumber integration isn't fully fleshed out yet, I haven't had the time to investigate all the corner cases out there. But I'm very grateful for the issue already! |
|
My e2e.ts: require('cy-verify-downloads').addCustomCommand();
import 'cypress-real-events';
import 'cypress-fail-fast';
import 'cypress-xray-plugin/register';
import './commands';My cypress config: async function setupNodeEvents(
on: Cypress.PluginEvents,
config: Cypress.PluginConfigOptions
): Promise<Cypress.PluginConfigOptions> {
await addCucumberPreprocessorPlugin(on, config);
const bundler = createBundler({
plugins: [nodepolyfills(), createEsbuildPlugin(config)],
});
await configureXrayPlugin({
jira: {
projectKey: 'PRJ',
url: 'https://devstack.jira.com/'
},
cucumber: {
featureFileExtension: '.feature',
uploadFeatures: true,
},
xray: {
statusPassed: 'PASS',
statusFailed: 'FAIL',
statusSkipped: 'ABORTED',
testType: 'Cucumber',
uploadResults: true
},
plugin: {
overwriteIssueSummary: true
},
});
await addXrayResultUpload(on);
require('cypress-fail-fast/plugin')(on, config);
on('file:preprocessor',bundler);
on('file:preprocessor', async (file) => {
await syncFeatureFile(file);
const cucumberPlugin = createBundler({
plugins: [createEsbuildPlugin(config)],
});
return cucumberPlugin(file);
});
on('task', {
verifyDownloadTasks,
isFileExist,
findFiles,
getClipboard: () => {
return clipboardy.readSync();
},
});
// Make sure to return the config object as it might have been modified by the plugin.
return config;
}A scenario example: Feature: Language
Background: Login
Given user opens login page
When user fills username with "user"
And user fills password with "psw"
And user clicks on login
@PRJ-2451 @happypath @DE
Scenario: Change Language to German
When user checks if language is set to German
Then the page shows values in "de"So my cypress command prior the cypress-xray-plugin was Both commands failed but with different fails. The first has the following error: The second command without the tag has the following error: And beneath that the error: Did I make the issue more clear to you? |
|
Yes, thank you very much for the detailed example! I managed to set up a corresponding project myself.
You've really set up everything properly. I think the issue is that you're separating the environment variables with a space, not a comma: https://docs.cypress.io/guides/guides/environment-variables#Option-4---env. Cypress/the plugin then never get to see the trailing options.
Multiple tags are pretty problematic, true. But what should the plugin do? In such a situation, the plugin is facing a difficult choice:
Right now there's no way to tell it which tag to use. Simply throwing the above error is currently the user-friendliest way, I think. But maybe you've got an alternative solution/proposal? I'm definitely open for it. Regarding the errors of on('file:preprocessor', async (file) => {
await syncFeatureFile(file);
const cucumberPlugin = createBundler({
plugins: [nodepolyfills(), createEsbuildPlugin(config)],
// ^^^^^^^^^^^^^^^
});
return cucumberPlugin(file);
});This fixed the error for me, although I used a different polyfill package I think (I could not figure out which one you're using). |
|
Wow, thank you it solved the issues with the cy-verify-downloads and the run command with the comma. If its an issue of the multiple tags though, then the only solution is to create separate tickets and tags for the other language. |
|
Yeah, the multiple tags found error is not going to disappear, I'm afraid. The plugin just can't decide for you which tag to use. await configureXrayPlugin({
cucumber: {
featureFileExtension: '.feature',
uploadFeatures: true,
tagPrefix: "PRJ"
}Would this help you? After changing your tags, you would then be able to provide/switch tags through Feature: Language
Background: Login
Given user opens login page
When user fills username with "user"
And user fills password with "psw"
And user clicks on login
@PRJ-2451 @PRJ-EN-6472
Scenario: Change Language to German
When user checks if language is set to German
Then the page shows values in "de"Regarding the upload: I cannot reproduce the error, unfortunately. I don't think the reporter is at fault, it should not influence the plugin. The 405 response is interesting, Xray's documentation states that it only returns the following codes on import:
Can you set https://devstack.jira.com/rest/raven/latest/import/feature?projectKey=PRJIf this is correct, can you try replacing https://devstack.jira.com/rest/raven/1.0/import/feature?projectKey=PRJYou can do it directly in the You can always roll back any such changes by running Maybe this helps already (not a permanent solution, obviously - only for investigative purposes). Otherwise I would need to add better debugging/logging output, I think. ⚠ Do not simply copy and paste entire debug payloads here for further analysis, as they will contain your credentials! ⚠ |
|
Hi I didnt try the debugging( maybe you can explain me where to set it), Now all test scenarios run even if they have same tags. This of course means I cannot use the cypress cucumber reporter.. Update: |
|
Another issue i found out, that when I run the test with tag=EN I have new test tickets created even though all the scenarios have ticket numbers taged on them. Now i have to delete duzen dublicates from jira. |
|
I'm glad to hear it was just the URL that required changing.
Would you mind elaborating on this problem? I've never had to reopen issues before. Ideally, please provide steps to reproduce the problem.
Yes, this is what I was trying to explain earlier in the comments (#79 (comment), #79 (comment)). There's just no way currently for the plugin to know which issue you would like to reuse when providing multiple tags. Please consider the proposal I made in the last comment and if it would be an adequate solution for you. |
I investigated it. The situation is as follows:
The big issue is that Cypress does not allow multiple hooks. The last function that hooks into the event will replace all the other ones: cypress-io/cypress#22428. In your case, the Unless I (or Cypress) come up with a solution to this problem, you will need to decide between:
Both at the sime time is not possible, unless you create the output directory manually. But it's possible that there's other stuff happening inside the If you don't want to keep using the
This problem was fixed in Can you check if you're on |
Ok thanks, I ll take it in consideration which is more useful for reporting.
Yes i have this installed:
Also the tagPrefix is not recognized. |
Are the scenarios which result in multiple issues complaining about multiple tags? If not, could you provide a set of scenarios which result in multiple issue creation? I just need something to recreate your problem, the description is not enough, sorry.
Yes, the option does not exist yet. It was just something I came up with to discuss with you if it could be a solution to your problem. |
|
Sorry I cannot give you right now test examples from my project cause they have sensitive data of the project which I do not have time to alter right now. But on both languages I had this issue with multiple tickets with examples, so its not about the language tag. I wanted to ask you, the reports in xray will show the failed test step or just the failed code snippet and screenshot? |
|
The plugin does not retrieve Cucumber step information as of now. It will only upload screenshots/videos and set the corresponding status in Xray. |
That is a pitty because in the xray documentation I found that this is possible https://docs.getxray.app/display/XRAYCLOUD/Testing+using+Cypress+and+Cucumber+in+JavaScript |
|
Thank you for the link, I actually did not know about that. This is very useful and highly interesting. I will report my findings in #87. I will leave this PR open for some time, so that you can provide more examples whenever you find the time. Apart from that, I'm sorry that the Cucumber integration isn't as fleshed out as the other parts of the plugin. The plugin was created more or less out of necessity, and Cucumber has not really been the main focus of the project which kickstarted the plugin. |
This is not true and can be achieved using cypress-on-fix.
This documentation is misleading, for reason mention above. |
|
Thank you @badeball, I did not know about |
csvtuda
changed the title
No inconvenience at all, I just want everyone to have the best experience. |
So according to this plugin it is possible to create a cucumber report and work with the xray plugin. |
|
@alexviz7 I think I'm going to close this issue, things like handling multiple Cucumber tags are out of this plugin's scope because everybody is probably requiring a different kind of solution to disambiguate the correct tag. If you (or future readers) still have issues, you should try to handle multiple tags manually as you see fit, for example by writing custom feature file parsers/preprocessors: #86 (comment). Something like this: import { AstBuilder, GherkinClassicTokenMatcher, Parser } from "@cucumber/gherkin";
import { pretty } from "@cucumber/gherkin-utils";
import { IdGenerator, Tag } from "@cucumber/messages";
import path from "path";
import fs from "fs";
/**
* This function does three things basically:
* 1. Parse the feature file
* 2. Apply tag handling logic
* 3. Create new feature file in temporary directory
*
* @param featurePath - path to the original feature file
* @param config - the Cypress configuration
* @returns the path to a feature file with updated tags
*/
function removeUnnecessaryTags(featurePath: string, config: Cypress.PluginConfigOptions): string {
if (!featurePath.endsWith(".feature")) {
return featurePath;
}
// 1. Parse feature file
// See: https://github.com/cucumber/gherkin#library
const uuidFn = IdGenerator.uuid();
const builder = new AstBuilder(uuidFn);
const matcher = new GherkinClassicTokenMatcher();
const parser = new Parser(builder, matcher);
const gherkinDocument = parser.parse(fs.readFileSync(featurePath, { encoding: "utf-8" }));
if (!gherkinDocument.feature) {
return featurePath;
}
// 2. Apply tag handling logic
// Children can be: background, scenarios, ...
const children = gherkinDocument.feature.children;
for (const child of children) {
if (child.scenario) {
child.scenario.tags = child.scenario.tags.filter((tag: Tag) => {
// Add tag filtering logic here.
});
}
}
// 3. Create new feature file in temporary directory
const root = path.join(projectRoot, "cypress", "e2e");
const filedir = path.basename(path.dirname(featurePath));
const filename = path.basename(featurePath);
const tmpFilepath = path.join(root, filedir, "generated", filename);
fs.mkdirSync(path.dirname(tmpFilepath), { recursive: true });
fs.writeFileSync(tmpFilepath, pretty(gherkinDocument, "gherkin"));
return tmpFilepath;
} |
csvtuda
added
rejected
Hello,
I have already implemented a Cypress Cucumber Test Automation Framework running with all test scenarios each of them have one or more tags. Some Scenarios have the same Ticket Tag because I test them on different languages. On Bamboo I had solved this issue with a language Tag, so therefore I had 2 separate builds running one for each tag/language. Now I want to do the same with the cypress-xray-plugin, but I get a fail when running the command
npx cypress run --env tags="@EN" JIRA_API_TOKEN="PATToken"with the error:Error: Failed to configure Xray uploader: no viable Xray configuration was found or the configuration you provided is not supported.What did I do wrong?
If I leave the tag out I get an error that I have multiple tests with same tag. Also I have an issue with another plugin
cy-verify-downloadswhich works perfectly in open mode.The text was updated successfully, but these errors were encountered: