green check marks missing after run on tests with duplicate names

[gregveres]

Environment

1. node -v: v12.14.1
2. npm -v: 6.12.0 (but I use yarn 1.22.4)
3. npm ls jest or npm ls react-scripts (if you haven’t ejected):

ui@0.1.0 C:\Users\gregv\source\repos\SquashSpider\SquashSpider\SkyCourt.UI\ui
`-- @vue/cli-plugin-unit-jest@4.3.0
  `-- jest@24.9.0 

I use vue, not react

1. your vscode-jest settings if customized:
   -no customizations. Just installed and run.

2. Operating system: windows 10

Prerequisite

• are you able to run jest test from command line? yes
• how do you run your tests from command line? (for example: npm run test or node_modules/.bin/jest) yarn test:unit

it is defined in package.json as:
"test:unit": "vue-cli-service test:unit",

Steps to Reproduce

I have noticed that sometimes when I save a unit test file, causing jest to run, the green check marks are only shown on some unit tests. They aren't shown on all the unit tests.

Today, I noticed this happen again after I copied one unit test from one describe block and pasted it into another describe block. I didn't change the test name because it did the same test but under different conditions. I have this type of setup occur a lot in the way I run unit tests. For instance, here is the situation I had today:

my class being tested has two functions: showAll and hideExtra.
I wrote a test called '... resets the showSeparator flags - true on all but last' in a describe block called 'showAll'. This test tests that after calling showAll, the showSeparator flags are correctly set.
Then I copied that test and pasted it into a describe block called 'hideExtra'. I modified the test but left the name the same because the test name was still completely accurate, when hideExtra is called, it should reset the showSeparator flags.

When I saved the unit test file, the tests ran and the output indicated that the tests passed without issue, but the green check mark was not shown on both of these tests, even though the green checkmark had been on the showAll version of the test before saving.

I then renamed the hideExtra version's name and saved again. This time the tests ran to completion and the green check mark showed up.

To prove to myself that it is a problem with the names being the same, I changed the name back to being the same on the hideExtra test again and after the run, both the tests were indicated as not having been run (no green check - just the empty circle).

This would explain why I have been seeing this behaviour all over in my code because this is a common practice for me.

Relevant Debug Info

Expected Behavior

I would expect both of these tests to show the green check mark after they run and pass. I would expect the extension to use the full path to the test (ie, the full chain of describe blocks down to the test name) instead of just the test name since jest has no restriction on all tests having unique names

Actual Behavior

the green check mark is replaced with the open circle for tests that have the same name in the same unit test file.

[gregveres]

Now that I know what to look for, every time I see this happening, it is because there are tests in the file with the same names. For instance, I will setup different situations for my vue components and then call expect(wrapper.element).ToMatchSnapshot(). I put this in a test called "matches snapshot" where the describe block describes the situation that was setup. So there might be three or four of these "matches snapshot" tests in a single file if a component can be configured for three or four different situations.

[dinofx]

I'm seeing this as well. I'm new to this extension. Is this a recent regression (just wondering if I should grab an older version until it's fixed)?

describe('foo', () => {
  it('should show green checkbox', () => {
    assert.equal(1, 1);
  });
});

describe('bar', () => {
  it('should show green checkbox', () => {
    assert.equal(1, 1);
  });
});

Neither success is reported. If one or more fails, that seems to be displayed, so I guess the problem is mostly cosmetic.

[gregveres]

It is mostly cosmetic, but you lose that confidence that the test was run when it stays as the circle rather than switching to the checkmark.

[connectdotz]

unfortunately, it's not just cosmetic nor regression.

This is a known issue for duplicate or dynamic test descriptions. vscode-jest will try to match tests with jest result by test descriptions; if failed, it will then try to match by line numbers (jest reports test line numbers from its own sourcemap). Due to various transpile related issues, javascript sourcemap is notoriously unreliable, therefore it is not uncommon the line numbers match would fail. In that situation, the test status will be left as "unknown", i.e. the empty circle...

I think the easiest workaround right now is probably to give each test a unique description. Hopefully, we will have a better solution soon.

[gregveres]

I dealt with this in Majestic. There we track the full path of the test to uniquely identify which tests are which.

[connectdotz]

agree, was just thinking that we should have taken the describe context into consideration... this can indeed resolve the duplicate descriptions use case (PR welcome!), however it won't help the dynamic descriptions, such as forEach or template literal strings... do you know how Majestic handles those?

[a-x-]

duplicate of #484

[connectdotz]

this should be fixed, feel free to give v4-alpha a try.