Issue Verification

---

tags: [conventions]

When an issue has been given the 4 - installed lifecycle label, the next step is for the Product Engineers (PEs) to verify that the installed code fulfills the acceptance criteria without introducing any new bugs or regressions.

To view issues that are installed and need to be picked up for verification along with those assigned to yourself, use this GitHub filter:

sort:updated-desc is:issue is:open label:"4 - installed" (no:assignee OR assignee:@me)

Process

Use this list as a general, but not exact, guideline for verifying issues.

1. Assign the issue to yourself.

2. Read through the issue to gain an full understanding of the situation and look for any possible side affects.

3. Test the original state, looking at the incorrect behavior and general behavior to be aware of the full context.

4. Test the new state, ensuring the fix or feature is in place and meets the acceptance criteria. Additionally, look carefully for any regressions or bugs. Think critically about what else the change could affect. See the testing new state section for more details.

5. Once verified, change the lifecycle label from 4 - installed to 5 - verified and ensure the correct milestone is assigned.

   

6. Close the issue with a comment, providing what version it was verified with and as much context as possible. Typically, this may follow a format of "Verified with X.X.X-next.XX via {{CodePen URL}}" OR "Verified with X.X.X-next.XX locally" with the inclusion of a photo/video/GIF that shows the change.

Testing New State

Where to Test

There are a a few common methods used to test behavior changes:

1. CodePen: many times the issue author will include a link to a CodePen or Component Sample that displays the wrong/unexpected behavior. Cloning the CodePen or opening the Sample as a CodePen is often the easiest way to test the new state. In the new CodePen, change the Calcite CDN import to the @next version and make any requires code changes.
2. Locally: if CodePen is not a good environment for the situation, you can test changes with the Component Demos pages on the dev branch and include a photo/video in the closing comment that displays the change.
3. Documentation: for changes affecting the public documentation of components, run the Calcite Documentation Site locally using a @next version of Calcite. To do so, adjust the @esri/calcite-components--internal-doc version in the package.json file to a specific "X.X.X.-next.XX" version and run npm install. This will fetch the specified version of documentation and set the version for Component Samples. To revert, undo the version change in the package.json file and run npm install again.

Note

Package deployments (available via NPM/CDN) are only triggered by merging commits that use the fix and feat conventional commit types. If the commit for the PR to be verified is not of one of these types, check if a Tag version is displayed at the top of the PR or refer to the Calcite Repo Tags page to see if a Tag release has happened since the PR merge. See images below for examples of each method.

If the PR is not included in a release, verification can be done locally or be delayed until the next Tag release.

Testing Checklist

1. Using exclusively mouse interactions, test acceptance criteria and prior behavior.
2. Using exclusively keyboard interactions, test acceptance criteria and prior behavior.
3. Test for any possible regressions newly introduced bugs.
4. When it might be affected, test with assistive technologies. For VM access, see requesting a VM.
5. If a new component property is introduced, ensure that the property has been added to the possible arguments configuration in the component's Storybook stories.ts file.
6. For code refactoring changes with no intended change in behavior, there is no specific proof to display, but ensure that the component functions as expected without any regressions or bugs.
7. If impacted, test right to left languages.

Testing New Components

When testing/verifying new components, the process changes. Use the list below as a general guide.

1. Test the general functionality/behavior of the component.
2. Compare styling/visual behavior with the final Figma specifications.
3. Confirm the properties, attributes, methods, and events are working as intended.
4. Make sure there are tests covering any required props/attributes.
5. Have 1 or 2 other PEs do a pass over the component to reduce the changes of missing anything.

Missing Criteria or Unexpected Behavior (bounce back)

During verification, if you find acceptance criteria are not met or there is unexpected behavior, follow the steps below:

Note

The bounce back label indicates that work has been done (merged), but more work is needed to be verified and completed. It helps quickly identify the issue's state, allowing for easier and more successful planning when moving issues in/out of milestones.

1. Investigate if the behavior was present prior to the PR merge.

• If yes, create a new issue and continue with verification, mentioning the finding and new issue in the verification comment.
• If no, continue through the steps.

2. Add a comment with the following:

• The Calcite version number used to test the issue.
• What acceptance criteria has been met.
• What acceptance criteria has not been met.
• Any new behavior or findings that were discovered during testing.
• A tag of the developer and/or designer who worked on the issue, along with @Esri/calcite-pes.
• CodePens, screenshots, or recordings where applicable.
• If needed, a question asking if the findings are expected or how to proceed.

3. Removed the 4 - installed label.
4. Add the 3 - in development and bounce back labels.