Guide how to see screenshots in html report

[luisxiaomai]

Hi ,

I run my xcuitest via below command, but it can't see any screenshot in html report.

xcodebuild test options | xcpretty --report html --screenshot

Please help how can get screenshots in html report.

Thanks.
Luis

[kirillzh]

Shouldn't it be --screenshots?

[supermarin]

closing, wrong flag used. please report back if it's still broken with --screenshots

[luisxiaomai]

@supermarin @kirillzh with flag --screenshots doesn't work, it still only display a html without any screenshot.

[nabeel-ibrahim]

I am having the same issue, I cannot see the screenshots in my html reports with the --screenshots flag

[KateOgar]

Hi! and i also have the same problem. Is there some workaround here?

[luisxiaomai]

@supermarin, any update for this bug? Some users also met this issue? Hope it can be fixed asap, thanks.

[zaidkazi]

any update on this?

[smaljaar]

have been seeing the same issue

[Krivoblotsky]

+1
By the way, we didn't want to wait and implemented our own solution for generating html pages with embedded screenshots. Maybe you'll find it useful too, please check it out:
https://github.com/MacPaw/xcsummary

[supermarin]

Sorry for the silence here - everyone is welcome to look into this and submit a PR.
I'm just out of bandwidth to fix this right now.

[supermarin]

Everyone here - mind confirming if @mattcotton's #269 fixes the issue?

[smaljaar]

I tried running
xcodebuild test [params] | xcpretty --report html --screenshots
but no screenshots visible in generated html

[mattcotton]

@smaljaar how are you generating the screenshots in the first place, and where are you storing them? Can you give me an example filename?

[smaljaar]

@mattcotton Thanks for your response. That's a good point because I don't know. I thought xcpretty would find the screenshots that XCTest generates while running tests by itself.

Now, from looking at your fix and your question to me, I'm guessing you have to provide the --path and create screenshots explicitly in your test run, and provide those screenshots with a specific name format. Then xcpretty will pick it up. Am I on the right track?

I'll experiment with it, but I think it would be helpful if it was more explicitly documented how to get screenshots in the html report.

[mattcotton]

@smaljaar no problem. I believe that the original PR was done with KIF testing in mind - KIF provides the ability to specify the location of captured screenshots.

For the --screenshots flag to work, the images need to be stored in the same directory as the report output (build/reports). My fix merely parses the filename for the Test Suite name so that they're matched to the tests correctly.

[mattcotton]

It actually looks like I have a further issue with #269 - it only works with the Test Suite names for tests written in Objective-C. It looks like tests written in Swift prepend the namespace to the Suite name which breaks the screenshot matching.

I'll submit another PR later to resolve @supermarin

[supermarin]

Thank you for investigating @mattcotton!
It might be worth adding tests for this so we cover all cases and don't break it in the future

[mattcotton]

No problem @supermarin!

I've submitted #270 which fixes my above comment - I didn't see yours regarding adding test cases but can add them if required? There are already test cases around screenshots, but based on a different file name scheme.

I seem to have a failing test on #270 which I didn't have when running locally - any ideas? There is quite a difference in performance - 1.6s on my machine vs 2.3 on Travis...

[amirsh-jain]

hello @mattcotton @supermarin

So here is my use case : I only capture screenshots on failure with a custom name and in some custom directory. (I am writing my tests in SWIFT)

But I understand to make --screenshots work I need to have all my screenshots in build/reports and with a "some specific" name.

Now my question is : what should be the name look like? can you provide a sample? My test suite may have multiple methods. So is it like TestSuiteName_MethodName? I am looking for specific format --screenshots parses. Please suggest.

Thanks in advance

[aditjain]

Can some one please respond to question above? Its critical at this point for me.

[mattcotton]

@amirsh-jain yes the screenshots need to be in the build/reports directory - I may make this configurable in future. As described in #280, at the moment the screenshot filename needs to begin with the test method name.

[mattcotton]

#280 now contains support for screenshots in a "TESTSUITE_TESTMETHOD" format (separated by an underscore)

[pawankums]

@mattcotton Could you please give an example of how to implement it... i am giving xcodebuild test [params] | xcpretty --report html --screenshots command but there are no screenshots in the report. Thanks for your help

[tsaakyan]

Does this look right? I dont see any screenshots in my html reports...

xcodebuild test
-workspace 'Test.xcworkspace'
-scheme 'UI Tests'
-destination 'platform=iOS Simulator,name=iPhone SE,OS=10.3'
-only-testing:'WelcomesitialUITests' | xcpretty -r html -o ./build/reports/Ads_WelcomesitialUITests.html --screenshots

[mattcotton]

@pawankums, @tsaakyan both commands look fine, however, are you sure that your screenshots are stored in build/reports? At the moment this is the only supported location, and screenshot file names must begin with the test method name (optionally pretending the test suite name followed by an underscore)

[tsaakyan]

@mattcotton do you mean change screenshot save location?
Right now it saves screenshots to my Desktop by default. Should I change it to build/reports location?

[mattcotton]

Yes exactly, at the moment the location that the report searches for screenshots isn't user configurable. I may submit a PR to add this.

For now, save your screenshots to build/reports and give it another try.

[pawankums]

@mattcotton how to save the screenshots in xctest using swift. I used to think that XCtest takes the snapshots automatically. Please let me know, how can i save a screenshot in the build/reports location.

[mattcotton]

If you supply the derivedDataPath argument to your xcodebuild test command, you can pass in build/reports as the path.

E.g. Xcodebuild [...] -derivedDataPath 'build/reports' test

XCTest screenshots will then be stored in the correct location.

[pawankums]

@mattcotton thanks, i was able to save the screenshots in the build/reports location but still i can't see them in the report for the failed scenarios.

[mattcotton]

@pawankums can you give me and example of how the screenshots are named?

[pawankums]

@mattcotton screenshots are getting saved in build/reports/Logs/Test/Attchments/ as Screenshot_1E2229E4-6C31-49A3-9182-ED57F287E90B.png

[tsaakyan]

Would be nice if there would be a tutorial on how to set this up.

[APshenkin]

+1 for this issue. Xcode generates screenshots to build/reports/Logs/Test/Attchments/ as Screenshot_[UID].png

I've tried to generate screenshots itself using https://github.com/zmeyc/UITestUtils to build/reports with correct name, but still there is no screenshots in report(

[APshenkin]

Found a solution for me. Will write it here, maybe it will help you too, guys.
Note: I use XCFIT https://github.com/Shashikant86/XCFit to write tests and UITestUtils https://github.com/zmeyc/UITestUtils to take screenshots if test fails.

1. Target name was PartitaProtocolBDDTests
2. Added new variable to info.plist in test target SAVEPATH = $(PROJECT_DIR)
3. Added this code in teardown method

if testRun!.failureCount > 0 {
            var name = self.name
            var srcPath = ""
            name = name?.replacingOccurrences(of: "[", with: "").replacingOccurrences(of: "]", with: "").replacingOccurrences(of: "-", with: "").replacingOccurrences(of: " ", with: "_")
            if let path = Bundle.main.path(forResource: "Info", ofType: "plist", inDirectory: "Plugins/PartitaProtocolBDDTests.xctest") {
                let dictRoot = NSDictionary(contentsOfFile: path)
                if let dict = dictRoot {
                    srcPath = dict["SAVEPATH"] as! String
                }
            }
            saveScreenshot(srcPath + "/build/reports/" + "PartitaProtocolBDDTests." + name! + ".png")
        }

4. Launched tests with this command:

xcodebuild -workspace Partita.xcworkspace -scheme Partita  -sdk iphonesimulator -destination 'platform=iOS Simulator,name=iPhone 7,OS=10.3' test| xcpretty -r html --screenshots

then the screenshot name will be PartitaProtocolBDDTests.[TESTSUITE]_[TESTMETHOD].png e.g. PartitaProtocolBDDTests.SmokeTests_testUserCanClickStartButton.png
The same as in html report. Now i have my screenshot

hope it helps

Also screenshots will be available in xcode 9, see https://developer.apple.com/documentation/xctest/xcuiscreenshot

[aditjain]

Hey @mattcotton I am little confused with your two different comment. About the file name format, In one comment above you mentioned
"now contains support for screenshots in a "TESTSUITE_TESTMETHOD" format (separated by an underscore)".

And in other you mentioned
"At the moment this is the only supported location, and screenshot file names must begin with the test method name (optionally pretending the test suite name followed by an underscore)"

can you confirm which one is correct?

[mattcotton]

@aditjain xcpretty now supports screenshots named in either TESTSUITE_TESTMETHOD format, or just TESTMETHOD.

@pawankums, @APshenkin as I've mentioned previously, the current implementation was written with KIF screenshots in mind, which names them in the above format and allows you to specify the location.

At some point I will write this up and provide the ability to specify the location. However we have to have some standard naming scheme so that we can match to tests.

[phynet]

This thread is one year almost, but I wanted to ask: does the screenshot have to be taken by us in each test case? xcpretty doesn't take it from build/reports/Logs/Test/Attchments/ I was wondering...
Thanks!

[cvillalobosg]

Hey guys, we found a workaround for this issue, hopefully it will be useful to you.

1. We are saving the screenshots on teardown (only on failure) and attaching to test report as explained here, the only change we made was naming the screenshot (right before the "add" statement): attachment.name = self.invocation?.selector.description

This will name the screenshots something like: "testMyFeature_1_84B147E1-9981-4BX3-845F-AC49A4BF2E0X.png"

2. When running our tests with the "xcodebuild" command, we are using the "-resultBundlePath" option to place the test output files in a custom location: xcodebuild blablabla -resultBundlePath myDir

3. XCPretty is looking for the screenshots in the "build/reports" directory. So we created a small bash script to move all the screenshots from "myDir" (resultBundlePath dir) to "build/reports". When generating the xcpretty report, if you use the "-o" option to create the report in a custom location, the screenshots will also need to be placed there.

Hope this helps!

[ganjarpanji]

@cvillalobos84 where do you put the script? and when its called? since report generated in one command with test i think this not possible?
xcodebuild blablabla -resultBundlePath myDir | xcpretty -flag for screenshot and html

[Gman9855]

Hey guys, we found a workaround for this issue, hopefully it will be useful to you.

1. We are saving the screenshots on teardown (only on failure) and attaching to test report as explained here, the only change we made was naming the screenshot (right before the "add" statement): attachment.name = self.invocation?.selector.description

This will name the screenshots something like: "testMyFeature_1_84B147E1-9981-4BX3-845F-AC49A4BF2E0X.png"

1. When running our tests with the "xcodebuild" command, we are using the "-resultBundlePath" option to place the test output files in a custom location: xcodebuild blablabla -resultBundlePath myDir
2. XCPretty is looking for the screenshots in the "build/reports" directory. So we created a small bash script to move all the screenshots from "myDir" (resultBundlePath dir) to "build/reports". When generating the xcpretty report, if you use the "-o" option to create the report in a custom location, the screenshots will also need to be placed there.

Hope this helps!

At what point do we call the script?

[mkj-is]

We found a simple way to attach screenshots to HTML reports in current version of xcpretty using --screenshots. It looks it can help some of you.

I just want to note that the correct name of the screenshot is:

ExampleAppUITests.ExampleTestCase_testMethod_anything else.png

We also made simple extension to XCTestCase which takes the screenshot and saves it to correct location for xcpretty: (No need for third-party testing frameworks and shell scripts.)

XCTestCase+TakeScreenshot.swift

[ganjarpanji]

We found a simple way to attach screenshots to HTML reports in current version of xcpretty using --screenshots. It looks it can help some of you.

I just want to note that the correct name of the screenshot is:

ExampleAppUITests.ExampleTestCase_testMethod_anything else.png

We also made simple extension to XCTestCase which takes the screenshot and saves it to correct location for xcpretty: (No need for third-party testing frameworks and shell scripts.)

XCTestCase+TakeScreenshot.swift

u mean with this extension the screenshot will be automatically included to the xcpretty report? no manual call for takescreenshot method?

[mkj-is]

u mean with this extension the screenshot will be automatically included to the xcpretty report? no manual call for takescreenshot method?

No, it will not be included automatically on fail. We also use the report for quick visual checks before we release the app, so we also use the method manually.

I can think of two solutions to this issue:

1. Write custom assert that takes screenshot if the assert fails. (That's what we use – I added the solution to the gist so you can see how we use it.)
2. Override tearDown method similarly to Guide how to see screenshots in html report #251 (comment).

[ghost]

How can I add Automatic Screenshot from Xcode 11 (when UI test is failed) to report.html? Nothing works from above.

[kenji21]

take a look at https://github.com/TitouanVanBelle/XCTestHTMLReport

[jaiswas33]

@here Do we have any update on this?, None of the above solutions worked for me