Automated framed screenshots #28
Conversation
circleci seems to be getting the following error, which I'm not getting on my local machine
|
// - Audio tab home | ||
// - Local Network tab home | ||
// - Video tab home | ||
// |
dcordero
Apr 1, 2018
Collaborator
This header comment does not follow the format of the project, specially regarding the license part. Please check an existing file as reference.
This header comment does not follow the format of the project, specially regarding the license part. Please check an existing file as reference.
|
||
override func tearDown() { | ||
super.tearDown() | ||
} |
dcordero
Apr 1, 2018
Collaborator
Could this method be removed?
Could this method be removed?
// | ||
// Screenshots taken | ||
// - Video playback (Jellyfish) | ||
// |
dcordero
Apr 1, 2018
Collaborator
Same here, this header comment does not follow the project format
Same here, this header comment does not follow the project format
|
||
override func tearDown() { | ||
super.tearDown() | ||
} |
dcordero
Apr 1, 2018
Collaborator
Same here, I think this method could be removed
Same here, I think this method could be removed
// | ||
// Created by Mike Choi on 3/30/18. | ||
// Copyright © 2018 VideoLAN. All rights reserved. | ||
// |
dcordero
Apr 1, 2018
Collaborator
☝️ 🤓
@@ -23,6 +23,7 @@ def iOS_pods | |||
pod 'MediaLibraryKit-prod' | |||
pod 'MobileVLCKit', '3.0.2' | |||
pod 'GTMAppAuth' | |||
pod 'SimulatorStatusMagic', :configurations => ['Debug'] |
dcordero
Apr 1, 2018
Collaborator
This is adding the dependency to the main application targets increasing their bundle sizes. Could it be possible to define it only in the uitests target?
This is adding the dependency to the main application targets increasing their bundle sizes. Could it be possible to define it only in the uitests target?
mkchoi212
Apr 1, 2018
Author
Collaborator
Moving pod 'SimulatorStatusMagic'
to target 'VLC for iOSUITests'
won't allow VLCAppDelegate.m
to import the module, right?
Moving pod 'SimulatorStatusMagic'
to target 'VLC for iOSUITests'
won't allow VLCAppDelegate.m
to import the module, right?
carolanitz
Apr 2, 2018
•
Member
correct! @dcordero correct me if I'm wrong here please.
I think instead of importing it directly you have an extension on Appdelegate for the tests and that extension is only part of the testmodule. You would also not need the debug configuration that way. Though I'm not sure how that works with classmethods. I would take a look at Artsys App here
correct! @dcordero correct me if I'm wrong here please.
I think instead of importing it directly you have an extension on Appdelegate for the tests and that extension is only part of the testmodule. You would also not need the debug configuration that way. Though I'm not sure how that works with classmethods. I would take a look at Artsys App here
Great work, the screenshots look awesome !!! I have added a few comments here and there |
The error in circleci means that XCTest is been imported from an Application target. Are the those new swift files included in the test target or the app target? That could be the reason of the error. |
This looks already soooo great!! I had some inline comments. I can't wait until this is ready to be merged! :D |
@@ -10,7 +10,6 @@ xcuserdata | |||
# Fastlane | |||
fastlane/report.xml | |||
fastlane/Preview.html | |||
fastlane/screenshots |
carolanitz
Apr 2, 2018
Member
According to the fastlane docs it's recommended to not store the screenshots in the repository
https://docs.fastlane.tools/best-practices/source-control/
According to the fastlane docs it's recommended to not store the screenshots in the repository
https://docs.fastlane.tools/best-practices/source-control/
carolanitz
Apr 3, 2018
Member
awesome that's even better! Great find :)
awesome that's even better! Great find :)
@@ -23,6 +23,7 @@ def iOS_pods | |||
pod 'MediaLibraryKit-prod' | |||
pod 'MobileVLCKit', '3.0.2' | |||
pod 'GTMAppAuth' | |||
pod 'SimulatorStatusMagic', :configurations => ['Debug'] |
carolanitz
Apr 2, 2018
•
Member
correct! @dcordero correct me if I'm wrong here please.
I think instead of importing it directly you have an extension on Appdelegate for the tests and that extension is only part of the testmodule. You would also not need the debug configuration that way. Though I'm not sure how that works with classmethods. I would take a look at Artsys App here
correct! @dcordero correct me if I'm wrong here please.
I think instead of importing it directly you have an extension on Appdelegate for the tests and that extension is only part of the testmodule. You would also not need the debug configuration that way. Though I'm not sure how that works with classmethods. I would take a look at Artsys App here
|
||
class VLCiOSTestMenu: XCTestCase { | ||
let app = XCUIApplication() | ||
let moreTab = XCUIApplication().tabBars.buttons.element(boundBy: 4) |
carolanitz
Apr 2, 2018
Member
Is it possible to get the tab by localized title instead ?
Is it possible to get the tab by localized title instead ?
mkchoi212
Apr 2, 2018
•
Author
Collaborator
There is no localizable string for this :(
I think the "More" tab is using Apple's built-in localized strings.
There is no localizable string for this :(
I think the "More" tab is using Apple's built-in localized strings.
} | ||
|
||
func localizedString(key: String) -> String { | ||
let localizationBundle = Bundle(path: Bundle(for: VLCiOSTestMenu.self).path(forResource: deviceLanguage, ofType: ".lproj")!)! |
carolanitz
Apr 2, 2018
Member
force unwrap is not great. How do you feel about making the tests fail if the localization file can't be loaded?
force unwrap is not great. How do you feel about making the tests fail if the localization file can't be loaded?
carolanitz
Apr 2, 2018
Member
Additionally it should be enough to load the bundle once for every language. That would speed up the test
Additionally it should be enough to load the bundle once for every language. That would speed up the test
} | ||
|
||
func testNavigationToVideoTab() { | ||
app.tabBars.buttons["Video"].tap() |
carolanitz
Apr 2, 2018
Member
Is there no localizable string ?
Is there no localizable string ?
func localizedString(key: String) -> String { | ||
let localizationBundle = Bundle(path: Bundle(for: VLCiOSTestMenu.self).path(forResource: deviceLanguage, ofType: ".lproj")!)! | ||
let result = NSLocalizedString(key, bundle: localizationBundle, comment: "") | ||
return result |
carolanitz
Apr 2, 2018
Member
seeing that this is used here as well it might be even good to have a testhelpers method ? We should definitely not duplicate the code here
seeing that this is used here as well it might be even good to have a testhelpers method ? We should definitely not duplicate the code here
@@ -0,0 +1,13 @@ | |||
devices([ | |||
"iPhone X", | |||
"iPhone 8" |
carolanitz
Apr 2, 2018
Member
I think we will need all the devices :D
I think we will need all the devices :D
languages([ | ||
"en", | ||
"de", | ||
"fr", |
carolanitz
Apr 2, 2018
Member
we will also need all the languages that we support. And yes I know this will take forever
we will also need all the languages that we support. And yes I know this will take forever
mkchoi212
Apr 3, 2018
Author
Collaborator
To create the framed screenshots, we need localized keyword.string
and title.string
for each language. Maybe this should be a progressive effort once we officially decide on what text to put on the framed screenshots??
To create the framed screenshots, we need localized keyword.string
and title.string
for each language. Maybe this should be a progressive effort once we officially decide on what text to put on the framed screenshots??
carolanitz
Apr 3, 2018
Member
absolutely let's just ask Louis!
absolutely let's just ask Louis!
} | ||
} | ||
|
||
// Please don't remove the lines below |
carolanitz
Apr 2, 2018
Member
I'm expecting that you double checked and understood this file btw :)
I'm expecting that you double checked and understood this file btw :)
we should switch to the accessibility element identifiers. that will resolve the bundle and localization issues. Also the simulator magic only as part of the Testtarget and adding it in the setup for the tests should resolve the import conundrum |
XCUIDevice.shared.orientation = .portrait | ||
setupSnapshot(app) | ||
helper = TestHelper(lang: deviceLanguage, target: VLCiOSTestMenu.self) | ||
app.launch() |
carolanitz
Apr 3, 2018
Member
This here would actually be a better place for the status magic (but remember to disable it again in tearDown()).
This here would actually be a better place for the status magic (but remember to disable it again in tearDown()).
import XCTest | ||
|
||
struct TestHelper { | ||
let localizationBundle: Bundle |
carolanitz
Apr 3, 2018
Member
something that I didn't think about in my earlier review but that a friend pointed out is to use accessibility element identifiers. They don’t need to be localized! Since we need to make the app accessibility friendly anyway this will kill two birds with one stone :)
something that I didn't think about in my earlier review but that a friend pointed out is to use accessibility element identifiers. They don’t need to be localized! Since we need to make the app accessibility friendly anyway this will kill two birds with one stone :)
mkchoi212
Apr 4, 2018
Author
Collaborator
So it looks like none of the elements in the app has its accessibilityIdentifier
set. How should we go about this? Update every single element's accessibilityIdentifier
?
So it looks like none of the elements in the app has its accessibilityIdentifier
set. How should we go about this? Update every single element's accessibilityIdentifier
?
carolanitz
Apr 4, 2018
Member
yes exactly. It needs to be added
yes exactly. It needs to be added
helper.tap(tabDescription: audio, app: app) | ||
XCTAssertNotNil(app.navigationBars[audio]) | ||
|
||
snapshot("audio_tab") |
carolanitz
Apr 3, 2018
Member
btw could we separate the Unit Tests from the snapshot taking?
btw could we separate the Unit Tests from the snapshot taking?
fastlane/test_output | ||
|
||
fastlane/screenshots/*/*.png | ||
fastlane/screenshots/screenshots.html |
carolanitz
Apr 4, 2018
•
Member
will you hate me if I ask for an alphabetically ordered .gitignore 🙊 ? At least within the fastlane section
will you hate me if I ask for an alphabetically ordered .gitignore
mkchoi212
Apr 5, 2018
Author
Collaborator
Not at all 😄
Not at all
looks already better. Could you use the accessibilityidentifier on the tabbaritems and other views as well? We need to avoid all use of the localization bundle otherwise renaming a tab would lead to failing tests for no reason |
"iPad Pro (12.9-inch)", | ||
"iPad Pro (12.9-inch) (2nd generation)", | ||
"iPad Pro (10.5-inch)" | ||
]) |
carolanitz
Apr 9, 2018
Member
👍
the tests fail on iPad btw but the more tab will disappear anyway soon. And one more question. What are the screenshots doing for a language that is not defined? can we make sure that it uses the screenshots from the English version? |
@carolanitz Right now, |
the merge conflicts just need to be resolved and I think this is good to go |
Fixed all the merge conflicts! |
Perfect status bar displays 100% battery life and 9:41 as the time. This allows for consistent, and perfect screenshoting, 100% of the time
To take framed screenshots, run `fastlane screenshots` Note that the list of supported devices and languages is random for now.
Notice that screenshots for fastlane are taken in the test file by calling `snapshot("IMG_NAME")`
Note that only German, French, and English have been added so far. Design/content for the framed screenshots are BASIC and NEEDS IMPROVEMENT.
iOS test cases are now compatible with both iPhone and iPad.
Note that a new target has been created in the Podfile specifically for iOS UITests.
Fixed bugs regarding change of localization during screenshoting
Note that string literals are being used all over the project to create identifiers. A new file containing all identifier literals should be created in the future.
Fastlane does not support default langauges. Therefore, in order to generate screenshots for all languages, default English keyword.strings and title.strings have been copied for all langauges.
All commits have been squashed! |
@@ -3431,7 +3408,6 @@ | |||
ALWAYS_EMBED_SWIFT_STANDARD_LIBRARIES = YES; | |||
CLANG_ANALYZER_NONNULL = YES; | |||
CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE; | |||
CLANG_CXX_LANGUAGE_STANDARD = "gnu++14"; |
mkchoi212
Apr 18, 2018
Author
Collaborator
@carolanitz, this fixes the misconfigured in the xcodeproj problem that you discovered!
@carolanitz, this fixes the misconfigured in the xcodeproj problem that you discovered!
Major feat in this PR are
fastlane screenshots
with localizationI wasn't sure which screens to capture and what kind of text to put in the frames but here's what I came up with. Here's some demo screenshots in German; thanks to Google Translate😃