Unique identifiers for all test examples #771

jwfriese · 2018-02-07T16:20:00Z

This PR addresses #770 by doing two things:

Assigning unique identifiers to all examples based on the description chain of the example, adding numbers at the end to deduplicate if necessary.
Elevating pending elements to full-fledged example objects that print these unique identifiers but still do not run their content.

There are a couple specific things that I would appreciate feedback on:

For the sake of trying to change one thing at a time (at least in spirit), I changed the Filter.pending flag to Filter.excluded. This is because I did not know whether there was some strong reason that xit and the like were removed entirely from the Quick test list as opposed to just neutering their content. I'll lean on @modocache and @jeffh to hopefully provide any history they can. It makes sense to me and would simplify the code to just treat x-type examples the same as pending, but I don't want to make that decision alone.

Also, @shx7, will you need those xit, xdescribe, etc examples to print Pending: blah_blah_blah as well? If so, then we'll definitely have to make some more changes. Fortunately, I already experimented with that and it will be easy for me to do. I just need to know whether that's what we want.
Tests only start appending digits once its apparent they are duplicated. In other words, a set of tests which happen to be duplicated will have the following identifiers:

test_with_duplicated_identifier
test_with_duplicated_identifier_2
test_with_duplicated_identifier_3

Notice that the first one does not have a 1 appended to it. @shx7 will that satisfy the ask?

I still have a couple things to do on this PR, but wanted to start a review of the code and a conversation about its progress ASAP. I'll maintain the list here:

Prepend unique identifiers with test class name
(maybe) Refactor xit and its like elements to use the new pending code

Cheers, y'all.

QuickBot · 2018-02-07T16:38:40Z

	1 Warning
⚠️	Big PR

Generated by 🚫 danger

jwfriese · 2018-12-30T20:11:24Z

I've started updating this PR as well, similar to the other two (#767 and #766).

houndci-bot · 2019-01-01T18:59:05Z

Tests/QuickTests/QuickTests/FunctionalTests/ExampleTests.swift

+        ) {}
+        groupContainingExamplesDuplicatedAcrossGroupsA.appendExample(exampleDupedAcrossSpecA)
+
+        let groupContainingExamplesDuplicatedAcrossGroupsB = ExampleGroup(


Identifier Name Violation: Variable name should be between 3 and 40 characters long: 'groupContainingExamplesDuplicatedAcrossGroupsB' (identifier_name)

houndci-bot · 2019-01-01T18:59:05Z

Tests/QuickTests/QuickTests/FunctionalTests/ExampleTests.swift

+
+        rootExampleGroup.appendExampleGroup(groupContainingExamplesWithDuplicateDescriptions)
+
+        let groupContainingExamplesDuplicatedAcrossGroupsA = ExampleGroup(


Identifier Name Violation: Variable name should be between 3 and 40 characters long: 'groupContainingExamplesDuplicatedAcrossGroupsA' (identifier_name)

houndci-bot · 2019-01-01T18:59:05Z

Tests/QuickTests/QuickTests/FunctionalTests/ExampleTests.swift

+        ) {}
+        rootExampleGroup.appendExample(rootExample)
+
+        let groupContainingExamplesWithDuplicateDescriptions = ExampleGroup(


Identifier Name Violation: Variable name should be between 3 and 40 characters long: 'groupContainingExamplesWithDuplicateDescriptions' (identifier_name)

houndci-bot · 2019-01-01T18:59:05Z

Tests/QuickTests/QuickTests/FunctionalTests/ExampleTests.swift

+import Nimble
+
+class ExampleTests: XCTestCase {
+    func test_uniqueIdentifier_uniquelyIdentifiesTheExampleAcrossASpec() {


Function Body Length Violation: Function body should span 40 lines or less excluding comments and whitespace: currently spans 84 lines (function_body_length)

houndci-bot · 2019-01-01T18:59:05Z

Sources/Quick/DSL/World+DSL.swift

+            raiseError("'pending' cannot be used inside 'afterEach', 'pending' may only be used inside 'context' or 'describe'. ")
+        }
+        guard currentExampleMetadata == nil else {
+            raiseError("'pending' cannot be used inside 'it', 'pending' may only be used inside 'context' or 'describe'. ")


Line Length Violation: Line should be 120 characters or less: currently 123 characters (line_length)

houndci-bot · 2019-01-01T18:59:05Z

Sources/Quick/DSL/World+DSL.swift

+            raiseError("'pending' cannot be used inside 'beforeEach', 'pending' may only be used inside 'context' or 'describe'. ")
+        }
+        if aftersCurrentlyExecuting {
+            raiseError("'pending' cannot be used inside 'afterEach', 'pending' may only be used inside 'context' or 'describe'. ")


Line Length Violation: Line should be 120 characters or less: currently 130 characters (line_length)

houndci-bot · 2019-01-01T18:59:05Z

Sources/Quick/DSL/World+DSL.swift

+    @nonobjc
+    internal func pending(_ description: String, file: String, line: UInt, closure: @escaping () -> Void) {
+        if beforesCurrentlyExecuting {
+            raiseError("'pending' cannot be used inside 'beforeEach', 'pending' may only be used inside 'context' or 'describe'. ")


Line Length Violation: Line should be 120 characters or less: currently 131 characters (line_length)

jwfriese · 2019-01-01T19:01:14Z

@ikesyo - The file with the "c99 identifier" behavior is the only one that had to significantly change as part of my work to rebase this back into a happy state with master. I think that my accommodations as part of the merge keep in good faith with the changes that originally were made in there, but please feel free to suggest or discuss any concerns you have or things you'd like to see done differently in there.

jwfriese · 2019-01-03T20:24:54Z

For what it's worth, I don't think that CI failure is legit. It passed for all but one macOS config, and both Linux configs. Something went wrong with Ruby land in there.

ikesyo · 2019-02-03T16:35:06Z

This is related to #490 and #491.

jessesquires · 2022-04-13T17:52:32Z

Hey there! 👋🏼

I'm a new maintainer for this project and I'm trying to get the next release out ASAP and also clean up old issues and old PRs. I'm closing all PRs older than 1-2 years, but that doesn't mean these changes are necessarily being rejected. If you are still interested in contributing these changes, please submit a new PR branching from the top of main.

We appreciate your work here and acknowledge the time and effort you put in to contribute! You're awesome. 💯 However, we are all volunteers here with limited capacity working for free. Unfortunately, that means we must close out stale issues and PRs to move the project forward in a reasonable way.

We apologize for the inconvenience and appreciate your understanding! 😎 ✌🏼

jwfriese added 4 commits January 1, 2019 08:55

Assign unique identifiers to all spec examples and print it when pending

a31faef

Simplify example identifier deduping code

d85f032

Import XCTest in the new test file

7b1d0d1

Move c99ExtendedIdentifier extension from NSString to String

6f95c8c

jwfriese force-pushed the 770-more-specific-pending-test-output branch from d27a99a to 6f95c8c Compare January 1, 2019 18:58

houndci-bot reviewed Jan 1, 2019

View reviewed changes

jwfriese mentioned this pull request Oct 8, 2019

Better pending test case output #770

Closed

ikesyo linked an issue Oct 8, 2020 that may be closed by this pull request

Using pending causes custom Configuration hooks not to run #490

Open

Base automatically changed from master to main March 2, 2021 13:54

jessesquires closed this Apr 13, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unique identifiers for all test examples #771

Unique identifiers for all test examples #771

jwfriese commented Feb 7, 2018 •

edited

QuickBot commented Feb 7, 2018 •

edited

jwfriese commented Dec 30, 2018

houndci-bot Jan 1, 2019

houndci-bot Jan 1, 2019

houndci-bot Jan 1, 2019

houndci-bot Jan 1, 2019

houndci-bot Jan 1, 2019

houndci-bot Jan 1, 2019

houndci-bot Jan 1, 2019

jwfriese commented Jan 1, 2019

jwfriese commented Jan 3, 2019

ikesyo commented Feb 3, 2019

jessesquires commented Apr 13, 2022


		rootExampleGroup.appendExampleGroup(groupContainingExamplesWithDuplicateDescriptions)

		let groupContainingExamplesDuplicatedAcrossGroupsA = ExampleGroup(

Unique identifiers for all test examples #771

Unique identifiers for all test examples #771

Conversation

jwfriese commented Feb 7, 2018 • edited

QuickBot commented Feb 7, 2018 • edited

jwfriese commented Dec 30, 2018

houndci-bot Jan 1, 2019

Choose a reason for hiding this comment

houndci-bot Jan 1, 2019

Choose a reason for hiding this comment

houndci-bot Jan 1, 2019

Choose a reason for hiding this comment

houndci-bot Jan 1, 2019

Choose a reason for hiding this comment

houndci-bot Jan 1, 2019

Choose a reason for hiding this comment

houndci-bot Jan 1, 2019

Choose a reason for hiding this comment

houndci-bot Jan 1, 2019

Choose a reason for hiding this comment

jwfriese commented Jan 1, 2019

jwfriese commented Jan 3, 2019

ikesyo commented Feb 3, 2019

jessesquires commented Apr 13, 2022

jwfriese commented Feb 7, 2018 •

edited

QuickBot commented Feb 7, 2018 •

edited