Skip to content

Commit

Permalink
[NTV-648] Agnostic Snapshot Testing Between M1 and Intel (#1794)
Browse files Browse the repository at this point in the history
* trial new snapshots on m1 snapshot testing framework with 0.98 perceptualPrecision

* intel failed comparing against m1 recordings of this test. Trying a subview recording.

* replaced FBSnapshotVerifyView with assertSnapshot everywhere in Kickstarter-Framework-iOSTests

* re-recorded all snapshots with snapshot-testing framework.

This is a huge commit. But only because it's a lot of files. The idea being it replaces everything in the `_64` folder which had our old tests that were references in the `FBSnapshotTestCase` dependency. These new screenshots are not going to be in one folder. They will be local to the view they represent, similar to the organization of the `Kickstarter-Framework-iOS` target file structure. It's more ordered than having one giant reference folder because if you need to check the snapshots, it's easier find any given views' subfolder in XCode. Opening that views subfolder in XCode will give you access to the screenshots in that folder, but through Finder. The image files themselves are not included in the XCode project, so it doesn't bloat our project either.

* updating a missed set of tests

* updated tests not to overwrite each other when using the same assert.

* some tests running on Intel did require approximate comparisons, so added perceptual precision to only those classes.

Tests now recorded on M1 pass 100% on Intel.

* added test for facebook reset password view controller

recorded on M1, should pass on CI and locally on Intel.

* removed _64 directory, where old screenshots were, remove ios-snapshot-test-case as dependency, updated TestCase class, removed project schema variables.

* still using old trait controllers, combos, devices, languages and other features. Mainly because we still cannot enforce a simulator to fail a record if its' not iPhone 8.

It'll just have to be common knowledge that iPhone is our recording simulator. Will update the readme. It's already kind of a guarantee because if we record with another simulator, we'd have failing tests one existing tests when the whole suite is run.

* updated snapshots on FacebookResetPasswordViewController

* Added preferred simulator check, so no tests pass if simulator for scheme is not iPhone 8 iOS 14.5. Same for recording.
  • Loading branch information
msadoon committed Feb 22, 2023
1 parent 40d368f commit 29008cd
Show file tree
Hide file tree
Showing 4,127 changed files with 867 additions and 331 deletions.
The diff you're trying to view is too large. We only load the first 3000 changed files.
Expand Up @@ -2,6 +2,7 @@
@testable import KsApi
@testable import Library
import Prelude
import SnapshotTesting
import XCTest

private let creator = User.template |> \.avatar.small .~ ""
Expand Down Expand Up @@ -139,7 +140,7 @@ internal final class ActivitiesViewControllerTests: TestCase {

self.scheduler.run()

FBSnapshotVerifyView(vc.view, identifier: "lang_\(language)_device_\(device)")
assertSnapshot(matching: parent.view, as: .image, named: "lang_\(language)_device_\(device)")
}
}
}
Expand Down Expand Up @@ -176,7 +177,7 @@ internal final class ActivitiesViewControllerTests: TestCase {

self.scheduler.run()

FBSnapshotVerifyView(vc.view, identifier: "lang_\(language)_device_\(device)")
assertSnapshot(matching: parent.view, as: .image, named: "lang_\(language)_device_\(device)")
}
}
}
Expand Down Expand Up @@ -211,7 +212,7 @@ internal final class ActivitiesViewControllerTests: TestCase {

self.scheduler.run()

FBSnapshotVerifyView(vc.view, identifier: "lang_\(language)_device_\(device)")
assertSnapshot(matching: parent.view, as: .image, named: "lang_\(language)_device_\(device)")
}
}
}
Expand Down Expand Up @@ -243,7 +244,7 @@ internal final class ActivitiesViewControllerTests: TestCase {

self.scheduler.run()

FBSnapshotVerifyView(vc.view, identifier: "lang_\(language)_device_\(device)")
assertSnapshot(matching: parent.view, as: .image, named: "lang_\(language)_device_\(device)")
}
}
}
Expand Down
Expand Up @@ -2,6 +2,7 @@
@testable import KsApi
@testable import Library
import Prelude
import SnapshotTesting
import UIKit

/** FIXME: There is a problem with testing the payment sheet. Somehow SPM is incorrectly linking the SDK resources explained here:
Expand All @@ -23,7 +24,7 @@ final class AddNewCardViewControllerTests: TestCase {
controller.configure(with: .settings)
let (parent, _) = traitControllers(device: device, orientation: .portrait, child: controller)

FBSnapshotVerifyView(parent.view, identifier: "lang_\(language)_device_\(device)")
assertSnapshot(matching: parent.view, as: .image, named: "lang_\(language)_device_\(device)")
}
}
}
Expand All @@ -35,7 +36,7 @@ final class AddNewCardViewControllerTests: TestCase {
controller.configure(with: .pledge)
let (parent, _) = traitControllers(device: device, orientation: .portrait, child: controller)

FBSnapshotVerifyView(parent.view, identifier: "lang_\(language)_device_\(device)")
assertSnapshot(matching: parent.view, as: .image, named: "lang_\(language)_device_\(device)")
}
}
}
Expand All @@ -53,7 +54,7 @@ final class AddNewCardViewControllerTests: TestCase {
controller.configure(with: .settings)
let (parent, _) = traitControllers(device: device, orientation: .portrait, child: controller)

FBSnapshotVerifyView(parent.view, identifier: "locale_\(locale)_device_\(device)")
assertSnapshot(matching: parent.view, as: .image, named: "lang_\(locale)_device_\(device)")
}
}
}
Expand Down
Expand Up @@ -2,6 +2,7 @@
@testable import KsApi
@testable import Library
import Prelude
import SnapshotTesting
import XCTest

internal final class BackerDashboardProjectsViewControllerTests: TestCase {
Expand Down Expand Up @@ -67,7 +68,7 @@ internal final class BackerDashboardProjectsViewControllerTests: TestCase {
)
self.scheduler.run()

FBSnapshotVerifyView(parent.view, identifier: "lang_\(language)_device_\(device)")
assertSnapshot(matching: parent.view, as: .image, named: "lang_\(language)_device_\(device)")
}
}
}
Expand All @@ -90,7 +91,7 @@ internal final class BackerDashboardProjectsViewControllerTests: TestCase {
)
self.scheduler.run()

FBSnapshotVerifyView(parent.view, identifier: "lang_\(language)_device_\(device)")
assertSnapshot(matching: parent.view, as: .image, named: "lang_\(language)_device_\(device)")
}
}
}
Expand All @@ -113,7 +114,7 @@ internal final class BackerDashboardProjectsViewControllerTests: TestCase {
)
self.scheduler.run()

FBSnapshotVerifyView(parent.view, identifier: "lang_\(language)_device_\(device)")
assertSnapshot(matching: parent.view, as: .image, named: "lang_\(language)_device_\(device)")
}
}
}
Expand Down
Expand Up @@ -2,6 +2,7 @@
@testable import KsApi
@testable import Library
import Prelude
import SnapshotTesting
import UIKit

final class BetaToolsViewControllerTests: TestCase {
Expand All @@ -26,7 +27,7 @@ final class BetaToolsViewControllerTests: TestCase {

self.scheduler.run()

FBSnapshotVerifyView(parent.view)
assertSnapshot(matching: parent.view, as: .image)
}
}
}
Expand Up @@ -2,6 +2,7 @@ import Foundation
@testable import Kickstarter_Framework
@testable import KsApi
@testable import Library
import SnapshotTesting
import XCTest

final class CancelPledgeViewControllerTests: TestCase {
Expand Down Expand Up @@ -35,7 +36,11 @@ final class CancelPledgeViewControllerTests: TestCase {
controller.configure(with: data)
let (parent, _) = traitControllers(device: device, orientation: .portrait, child: controller)

FBSnapshotVerifyView(parent.view, identifier: "lang_\(language)_device_\(device)")
assertSnapshot(
matching: parent.view,
as: .image(perceptualPrecision: 0.98),
named: "lang_\(language)_device_\(device)"
)
}
}
}
Expand Down
@@ -1,6 +1,7 @@
@testable import Kickstarter_Framework
@testable import KsApi
@testable import Library
import SnapshotTesting
import XCTest

internal final class CategorySelectionViewControllerTests: TestCase {
Expand Down Expand Up @@ -32,7 +33,11 @@ internal final class CategorySelectionViewControllerTests: TestCase {

self.scheduler.run()

FBSnapshotVerifyView(parent.view, identifier: "lang_\(language)_device_\(device)")
assertSnapshot(
matching: parent.view,
as: .image(perceptualPrecision: 0.98),
named: "lang_\(language)_device_\(device)"
)
}
}
}
Expand Down
Expand Up @@ -2,6 +2,7 @@
@testable import KsApi
@testable import Library
import Prelude
import SnapshotTesting
import XCTest

final class ChangeEmailViewControllerTests: TestCase {
Expand Down Expand Up @@ -29,7 +30,7 @@ final class ChangeEmailViewControllerTests: TestCase {

scheduler.advance()

FBSnapshotVerifyView(parent.view, identifier: "lang_\(language)_device_\(device)")
assertSnapshot(matching: parent.view, as: .image, named: "lang_\(language)_device_\(device)")
}
}
}
Expand All @@ -45,7 +46,7 @@ final class ChangeEmailViewControllerTests: TestCase {

scheduler.advance()

FBSnapshotVerifyView(parent.view, identifier: "lang_\(language)_device_\(device)")
assertSnapshot(matching: parent.view, as: .image, named: "lang_\(language)_device_\(device)")
}
}
}
Expand All @@ -68,7 +69,7 @@ final class ChangeEmailViewControllerTests: TestCase {

scheduler.advance()

FBSnapshotVerifyView(parent.view, identifier: "lang_\(language)_device_\(device)")
assertSnapshot(matching: parent.view, as: .image, named: "lang_\(language)_device_\(device)")
}
}
}
Expand All @@ -86,7 +87,7 @@ final class ChangeEmailViewControllerTests: TestCase {

scheduler.advance()

FBSnapshotVerifyView(parent.view, identifier: "lang_\(language)_device_\(device)")
assertSnapshot(matching: parent.view, as: .image, named: "lang_\(language)_device_\(device)")
}
}
}
Expand Down
Expand Up @@ -2,6 +2,7 @@
@testable import KsApi
@testable import Library
import Prelude
import SnapshotTesting
import XCTest

final class ChangePasswordViewControllerTests: TestCase {
Expand All @@ -23,7 +24,7 @@ final class ChangePasswordViewControllerTests: TestCase {
let controller = ChangePasswordViewController.instantiate()
let (parent, _) = traitControllers(device: device, orientation: .portrait, child: controller)

FBSnapshotVerifyView(parent.view, identifier: "lang_\(language)_device_\(device)")
assertSnapshot(matching: parent.view, as: .image, named: "lang_\(language)_device_\(device)")
}
}
}
Expand Down
Expand Up @@ -2,6 +2,7 @@
@testable import KsApi
import Library
import Prelude
import SnapshotTesting
import XCTest

final class CommentRepliesViewControllerTests: TestCase {
Expand Down Expand Up @@ -40,7 +41,11 @@ final class CommentRepliesViewControllerTests: TestCase {

self.scheduler.run()

FBSnapshotVerifyView(parent.view, identifier: "CommentReplies - lang_\(language)_device_\(device)")
assertSnapshot(
matching: parent.view,
as: .image,
named: "CommentReplies - lang_\(language)_device_\(device)"
)
}
}
}
Expand Down Expand Up @@ -70,7 +75,11 @@ final class CommentRepliesViewControllerTests: TestCase {
parent.view.frame.size.height = 1_100

self.scheduler.run()
FBSnapshotVerifyView(parent.view, identifier: "CommentReplies - lang_\(language)_device_\(device)")
assertSnapshot(
matching: parent.view,
as: .image,
named: "CommentReplies - lang_\(language)_device_\(device)"
)
}
}
}
Expand Down Expand Up @@ -100,7 +109,11 @@ final class CommentRepliesViewControllerTests: TestCase {
parent.view.frame.size.height = 1_100

self.scheduler.run()
FBSnapshotVerifyView(parent.view, identifier: "CommentReplies - lang_\(language)_device_\(device)")
assertSnapshot(
matching: parent.view,
as: .image,
named: "CommentReplies - lang_\(language)_device_\(device)"
)
}
}
}
Expand Down Expand Up @@ -132,7 +145,11 @@ final class CommentRepliesViewControllerTests: TestCase {

self.scheduler.run()

FBSnapshotVerifyView(parent.view, identifier: "CommentReplies - lang_\(language)_device_\(device)")
assertSnapshot(
matching: parent.view,
as: .image,
named: "CommentReplies - lang_\(language)_device_\(device)"
)
}
}
}
Expand Down Expand Up @@ -166,7 +183,11 @@ final class CommentRepliesViewControllerTests: TestCase {

self.scheduler.advance()

FBSnapshotVerifyView(parent.view, identifier: "CommentReplies - lang_\(language)_device_\(device)")
assertSnapshot(
matching: parent.view,
as: .image,
named: "CommentReplies - lang_\(language)_device_\(device)"
)
}
}
}
Expand Down Expand Up @@ -196,7 +217,11 @@ final class CommentRepliesViewControllerTests: TestCase {

self.scheduler.advance()

FBSnapshotVerifyView(parent.view, identifier: "CommentReplies - lang_\(language)_device_\(device)")
assertSnapshot(
matching: parent.view,
as: .image,
named: "CommentReplies - lang_\(language)_device_\(device)"
)
}
}
}
Expand Down

0 comments on commit 29008cd

Please sign in to comment.