feat: Filtering test configurations (iOS TestPlans) #1448

zuziakaxel · 2021-01-04T11:59:10Z

This PR adds to new options for flank:

skip-test-configuration
only-test-configuration

Flank users can filter test configurations (inside of a test plan).

Test Plan

How do we know the code works?

From project root:

. .env
cd ./test_runner/
flankScripts shell buildFlank 
flank ios run -c ./src/test/kotlin/ftl/fixtures/ios_test_plan.yml

additionally, edit os_test_plan.yml for testing different configurations.

Checklist

Documented - updated main docs
Unit tested

test_runner/src/main/kotlin/ftl/config/ios/IosFlankConfig.kt

piotradamczyk5 · 2021-01-07T13:17:48Z

@Mergifyio rebase

mergify · 2021-01-07T13:19:02Z

Command rebase: success

Branch has been successfully rebased

github-actions · 2021-01-07T13:19:17Z

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

github-actions · 2021-01-08T10:45:55Z

Timestamp: 2021-01-21 14:13:08
Buildscan url for ubuntu-workflow run 501290257

test_runner/src/main/kotlin/ftl/ios/xctest/XcTestData.kt

jan-goral

Implementation looks great 👍

zuziakaxel · 2021-01-18T09:02:25Z

I have read the CLA Document and I hereby sign the CLA

piotradamczyk5

Please update flank.ios.yml

test_runner/src/main/kotlin/ftl/args/ValidateIosArgs.kt

test_runner/src/test/kotlin/ftl/ios/xctest/XcTestDataTest.kt

test_runner/src/main/kotlin/ftl/args/CreateIosArgs.kt

pawelpasterz · 2021-01-19T09:03:27Z

Please provide a detailed Test Plan. Thanks! 👍

mergify · 2021-01-19T15:13:49Z

Title does not follow the guidelines of Conventional Commits.
Please adjust title before merge and use one of following prefix:

build - Changes that affect the build system or external dependencies (dependencies update)
ci - Changes to our CI configuration files and scripts (basically directory .github/workflows)
docs - Documentation only changes
feat - A new feature
fix - A bug fix
chore - Changes which does not touch the code (ex. manual update of release notes). It will not generate release notes changes
refactor - A code change that contains refactor
style - Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc)
test - Adding missing tests or correcting existing tests and also changes for our test app
perf - A code change that improves performance (I do not think we will use it)

jan-goral · 2021-01-20T13:11:21Z

I have read the CLA Document and I hereby sign the CLA

mergify · 2021-01-20T13:11:59Z

Title does not follow the guidelines of Conventional Commits.
Please adjust title before merge and use one of following prefix:

build - Changes that affect the build system or external dependencies (dependencies update)
ci - Changes to our CI configuration files and scripts (basically directory .github/workflows)
docs - Documentation only changes
feat - A new feature
fix - A bug fix
chore - Changes which does not touch the code (ex. manual update of release notes). It will not generate release notes changes
refactor - A code change that contains refactor
style - Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc)
test - Adding missing tests or correcting existing tests and also changes for our test app
perf - A code change that improves performance (I do not think we will use it)

mergify · 2021-01-20T13:44:43Z

Title does not follow the guidelines of Conventional Commits.
Please adjust title before merge and use one of following prefix:

build - Changes that affect the build system or external dependencies (dependencies update)
ci - Changes to our CI configuration files and scripts (basically directory .github/workflows)
docs - Documentation only changes
feat - A new feature
fix - A bug fix
chore - Changes which does not touch the code (ex. manual update of release notes). It will not generate release notes changes
refactor - A code change that contains refactor
style - Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc)
test - Adding missing tests or correcting existing tests and also changes for our test app
perf - A code change that improves performance (I do not think we will use it)

Sloox

LGTM

piotradamczyk5

Still valid comment about missing documentation in flank.ios.yml

build.gradle.kts

adamfilipow92

tested with default ios_test_plan.yml
Code looks good

Pls remove this spacing and update docs

…ipTestConfiguration

…tion validation

Co-authored-by: Jan Góral <60390247+jan-gogo@users.noreply.github.com>

adamfilipow92 · 2021-01-21T13:58:34Z

build.gradle.kts

-            "**/*Generated.kt" // we can expand this list
+            "**/*Generated.kt",
+            "**/*Test.kt",
+            "**/Test*.kt" // we can expand this list


I think we enable linter for tests in this pr: #1515
88b6c52

adamfilipow92

👍

pawelpasterz · 2021-01-22T06:31:06Z

IT are not updated with new options...

zuziakaxel added Feature iOS labels Jan 4, 2021

zuziakaxel linked an issue Jan 4, 2021 that may be closed by this pull request

iOS: Support for filtering Test Configurations #1379

Closed

zuziakaxel added the Tiger 🐯 label Jan 4, 2021

jan-goral assigned zuziakaxel and jan-goral Jan 4, 2021

jan-goral reviewed Jan 4, 2021

View reviewed changes

test_runner/src/main/kotlin/ftl/config/ios/IosFlankConfig.kt Outdated Show resolved Hide resolved

test_runner/src/main/kotlin/ftl/config/ios/IosFlankConfig.kt Show resolved Hide resolved

bootstraponline force-pushed the 1379-filtering-test-configurations branch from 628e113 to f21a4c8 Compare January 7, 2021 13:19

jan-goral reviewed Jan 8, 2021

View reviewed changes

test_runner/src/main/kotlin/ftl/ios/xctest/XcTestData.kt Outdated Show resolved Hide resolved

jan-goral reviewed Jan 14, 2021

View reviewed changes

zuziakaxel force-pushed the 1379-filtering-test-configurations branch from 4931cb8 to b843395 Compare January 18, 2021 09:01

github-actions bot added a commit that referenced this pull request Jan 18, 2021

@axelzuziak-gogo has signed the CLA from Pull Request #1448

02b5377

zuziakaxel marked this pull request as ready for review January 18, 2021 09:05

piotradamczyk5 reviewed Jan 18, 2021

View reviewed changes

test_runner/src/main/kotlin/ftl/args/ValidateIosArgs.kt Show resolved Hide resolved

zuziakaxel mentioned this pull request Jan 18, 2021

Question: is the latest version of Flank and Firebase Test Lab support Test Plans? #1511

Closed

Sloox suggested changes Jan 18, 2021

View reviewed changes

test_runner/src/main/kotlin/ftl/args/ValidateIosArgs.kt Show resolved Hide resolved

test_runner/src/test/kotlin/ftl/ios/xctest/XcTestDataTest.kt Show resolved Hide resolved

test_runner/src/main/kotlin/ftl/args/CreateIosArgs.kt Outdated Show resolved Hide resolved

zuziakaxel added this to the Sprint -1 milestone Jan 19, 2021

bootstraponline force-pushed the 1379-filtering-test-configurations branch from b843395 to 8ec53c3 Compare January 19, 2021 15:12

github-actions bot added a commit that referenced this pull request Jan 20, 2021

@jan-gogo has signed the CLA from Pull Request #1448

d1217c1

jan-goral requested review from piotradamczyk5 and Sloox January 20, 2021 14:29

Sloox approved these changes Jan 20, 2021

View reviewed changes

piotradamczyk5 suggested changes Jan 20, 2021

View reviewed changes

adamfilipow92 reviewed Jan 20, 2021

View reviewed changes

build.gradle.kts Outdated Show resolved Hide resolved

adamfilipow92 reviewed Jan 20, 2021

View reviewed changes

zuziakaxel and others added 12 commits January 21, 2021 14:23

Added two new IosFlankConfig parameters: onlyTestConfiguration and sk…

cc2b36e

…ipTestConfiguration

Added tests for IosArgs's only-test-configuration/skip-test-configura…

5533e7d

…tion validation

Code formatting

349b43d

Added filtering logic and tests to XCTestData

0d70fa9

Update test_runner/src/main/kotlin/ftl/ios/xctest/XcTestData.kt

5ef4fea

Co-authored-by: Jan Góral <60390247+jan-gogo@users.noreply.github.com>

Added usage description to docs

ae3fbcb

Make onlyTestConfiguration & skipTestConfiguration non-nullable

8a367b8

Move filter test configuration condition to getter

e9f4f73

Fix ios_test_plan.yml

33c87c1

Update build.gradle.kts

940b80d

Remove blank characters

c65a1ac

Add missing options to flank.ios.yml

0dfd609

jan-goral force-pushed the 1379-filtering-test-configurations branch from fa41579 to 0dfd609 Compare January 21, 2021 13:26

jan-goral requested review from piotradamczyk5 and adamfilipow92 January 21, 2021 13:30

adamfilipow92 reviewed Jan 21, 2021

View reviewed changes

Fix lint task

ad258c1

jan-goral requested a review from adamfilipow92 January 21, 2021 14:09

adamfilipow92 approved these changes Jan 21, 2021

View reviewed changes

piotradamczyk5 approved these changes Jan 21, 2021

View reviewed changes

mergify bot merged commit b428402 into master Jan 21, 2021

mergify bot deleted the 1379-filtering-test-configurations branch January 21, 2021 14:30

github-actions bot locked and limited conversation to collaborators Jan 21, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Filtering test configurations (iOS TestPlans) #1448

feat: Filtering test configurations (iOS TestPlans) #1448

zuziakaxel commented Jan 4, 2021 •

edited by jan-goral

piotradamczyk5 commented Jan 7, 2021

mergify bot commented Jan 7, 2021

github-actions bot commented Jan 7, 2021 •

edited

github-actions bot commented Jan 8, 2021 •

edited

jan-goral left a comment

zuziakaxel commented Jan 18, 2021

piotradamczyk5 left a comment

pawelpasterz commented Jan 19, 2021

mergify bot commented Jan 19, 2021

jan-goral commented Jan 20, 2021

mergify bot commented Jan 20, 2021

mergify bot commented Jan 20, 2021

Sloox left a comment

piotradamczyk5 left a comment

adamfilipow92 left a comment •

edited

adamfilipow92 Jan 21, 2021 •

edited

adamfilipow92 left a comment

pawelpasterz commented Jan 22, 2021

feat: Filtering test configurations (iOS TestPlans) #1448

feat: Filtering test configurations (iOS TestPlans) #1448

Conversation

zuziakaxel commented Jan 4, 2021 • edited by jan-goral

Test Plan

Checklist

piotradamczyk5 commented Jan 7, 2021

mergify bot commented Jan 7, 2021

github-actions bot commented Jan 7, 2021 • edited

github-actions bot commented Jan 8, 2021 • edited

jan-goral left a comment

Choose a reason for hiding this comment

zuziakaxel commented Jan 18, 2021

piotradamczyk5 left a comment

Choose a reason for hiding this comment

pawelpasterz commented Jan 19, 2021

mergify bot commented Jan 19, 2021

jan-goral commented Jan 20, 2021

mergify bot commented Jan 20, 2021

mergify bot commented Jan 20, 2021

Sloox left a comment

Choose a reason for hiding this comment

piotradamczyk5 left a comment

Choose a reason for hiding this comment

adamfilipow92 left a comment • edited

Choose a reason for hiding this comment

adamfilipow92 Jan 21, 2021 • edited

Choose a reason for hiding this comment

adamfilipow92 left a comment

Choose a reason for hiding this comment

pawelpasterz commented Jan 22, 2021

zuziakaxel commented Jan 4, 2021 •

edited by jan-goral

github-actions bot commented Jan 7, 2021 •

edited

github-actions bot commented Jan 8, 2021 •

edited

adamfilipow92 left a comment •

edited

adamfilipow92 Jan 21, 2021 •

edited