Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(ci): reduce flakiness in add_users, siblings smoke test #6883

Merged
merged 10 commits into from
Dec 29, 2022
Merged

fix(ci): reduce flakiness in add_users, siblings smoke test #6883

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
f33deed
reduce flakiness in add_users, siblings
anshbansal Dec 29, 2022
11a72a4
modify analytics test for tracking to be sent
anshbansal Dec 29, 2022
3db0350
Merge branch 'master' into aseem-reduce-smoke-test-flakiness
anshbansal Dec 29, 2022
d6e07ca
update versions pulled in background
anshbansal Dec 29, 2022
890785c
fix problem with venv caching - wrong path
anshbansal Dec 29, 2022
bf467a4
Merge branch 'master' into aseem-reduce-smoke-test-flakiness
anshbansal Dec 29, 2022
7d6841b
add 60s sleep
anshbansal Dec 29, 2022
c839dfc
add test less flaky
anshbansal Dec 29, 2022
33bc1a2
remove pre-fetch of images
anshbansal Dec 29, 2022
d352887
fix check for cypress key
anshbansal Dec 29, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 6 additions & 12 deletions .github/workflows/docker-unified.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -467,14 +467,6 @@ jobs:
with:
python-version: "3.7"
cache: 'pip'
- name: Pull images in background
run: |
docker pull acryldata/datahub-actions:head &
docker pull confluentinc/cp-kafka:7.2.0 &
docker pull elasticsearch:7.9.3 &
docker pull mariadb:10.5.8 &
docker pull confluentinc/cp-schema-registry:7.2.0 &
docker pull confluentinc/cp-zookeeper:7.2.0 &
- name: Install dependencies
run: ./metadata-ingestion/scripts/install_deps.sh
- name: Build datahub cli
Expand Down Expand Up @@ -521,23 +513,25 @@ jobs:
pip install pip -U # only 22.2 and above contains the --dry-run flag

# only the last line of the output is the packages that will be installed
pip install --dry-run -r ./smoke/requirements.txt | tail -n 1 > /tmp/would_be_installed.txt
pip install --dry-run -r ./smoke-test/requirements.txt | tail -n 1 > /tmp/would_be_installed.txt
CHECKSUM=$(shasum /tmp/would_be_installed.txt | awk '{print $1}')
echo "packages_checksum=$CHECKSUM" >> $GITHUB_OUTPUT
- uses: actions/cache@v3
id: cache-venv
with:
path: ./smoke-test/venv/
key: smoke-venv-${{ runner.os }}-${{ steps.packages_checksum.outputs.packages_checksum }}
- name: check background image download status
run: |
jobs
- name: run quickstart
env:
DATAHUB_TELEMETRY_ENABLED: false
DATAHUB_VERSION: ${{ needs.setup.outputs.unique_tag }}
run: |
./smoke-test/run-quickstart.sh
- name: sleep 60s
run: |
# we are doing this because gms takes time to get ready
# and we don't have a better readiness check when bootstrap is done
sleep 60s
- name: Smoke test
env:
RUN_QUICKSTART: false
Expand Down
4 changes: 2 additions & 2 deletions docker/docker-compose.dev.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,8 @@ services:
args:
APP_ENV: dev
depends_on:
- broker:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this : at the end was causing error

ERROR: The Compose file './docker-compose.dev.yml' is invalid because:
services.kafka-setup.depends_on contains {"broker": null}, which is an invalid type, it should be a string

- schema-registry:
- broker
- schema-registry

datahub-gms:
image: linkedin/datahub-gms:debug
Expand Down
3 changes: 1 addition & 2 deletions smoke-test/tests/cypress/cypress/integration/analytics/analytics.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,12 @@ describe('analytics', () => {
cy.goToChart("urn:li:chart:(looker,cypress_baz1)");
cy.waitTextVisible("Baz Chart 1");
cy.openEntityTab("Dashboards");
cy.wait(1000);

cy.goToAnalytics();
cy.wait(1000);
cy.contains("Section Views across Entity Types").scrollIntoView({
ensureScrollable: false
})
cy.wait(1000);
cy.waitTextPresent("dashboards");
});
})
3 changes: 2 additions & 1 deletion smoke-test/tests/cypress/cypress/integration/mutations/add_users.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,9 @@ describe("add_user", () => {

cy.clickOptionWithText("Invite Users")

cy.waitTextVisible('signup?invite_token').then(($elem) => {
cy.waitTextVisible(/signup\?invite_token=\w+/).then(($elem) => {
Copy link
Collaborator Author

@anshbansal anshbansal Dec 29, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sometimes the UI was loading signup?invite_token= without any token before the url with token shows up and that got copied which led to invalid link. So adding regex that there should be something at the end of =

const inviteLink = $elem.text();
cy.log(inviteLink);
cy.logout();
cy.visit(inviteLink);
let name = tryToSignUp();
Expand Down
22 changes: 5 additions & 17 deletions smoke-test/tests/cypress/cypress/integration/siblings/siblings.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ describe('siblings', () => {
cy.visit('/dataset/urn:li:dataset:(urn:li:dataPlatform:dbt,cypress_project.jaffle_shop.customers,PROD)/?is_lineage_mode=false');

// navigate to the bq entity
cy.get('[data-testid="compact-entity-link-urn:li:dataset:(urn:li:dataPlatform:bigquery,cypress_project.jaffle_shop.customers,PROD)"]').click({ force: true });
cy.clickOptionWithTestId('compact-entity-link-urn:li:dataset:(urn:li:dataPlatform:bigquery,cypress_project.jaffle_shop.customers,PROD)');

// check merged platforms is not shown
cy.get('[data-testid="entity-header-test-id"]').contains('dbt & BigQuery').should('not.exist');
Expand All @@ -62,16 +62,16 @@ describe('siblings', () => {
cy.visit('/dataset/urn:li:dataset:(urn:li:dataPlatform:dbt,cypress_project.jaffle_shop.customers,PROD)/?is_lineage_mode=false');

// navigate to the bq entity
cy.get('[data-testid="compact-entity-link-urn:li:dataset:(urn:li:dataPlatform:bigquery,cypress_project.jaffle_shop.customers,PROD)"]').click({ force: true });
cy.clickOptionWithTestId('compact-entity-link-urn:li:dataset:(urn:li:dataPlatform:bigquery,cypress_project.jaffle_shop.customers,PROD)');

cy.contains('Add Term').click();
cy.clickOptionWithText('Add Term');

cy.selectOptionInTagTermModal('CypressTerm');

cy.visit('/dataset/urn:li:dataset:(urn:li:dataPlatform:dbt,cypress_project.jaffle_shop.customers,PROD)/?is_lineage_mode=false');

cy.get('a[href="/glossaryTerm/urn:li:glossaryTerm:CypressNode.CypressTerm"]').within(() => cy.get('span[aria-label=close]').click());
cy.contains('Yes').click();
cy.clickOptionWithText('Yes');

cy.contains('CypressTerm').should('not.exist');
});
Expand All @@ -88,18 +88,6 @@ describe('siblings', () => {
cy.get('.test-search-result-sibling-section').get('.test-mini-preview-class:contains(raw_orders)').should('have.length', 2);
});

it('will combine results in search', () => {
Copy link
Collaborator Author

@anshbansal anshbansal Dec 29, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deleted as this was a duplicate test as the one above it (expand the diff to see the duplicate)

cy.login();
cy.visit('/search?page=1&query=%2522raw_orders%2522');

cy.contains('Showing 1 - 2 of 2 results');

cy.get('.test-search-result').should('have.length', 1);
cy.get('.test-search-result-sibling-section').should('have.length', 1);

cy.get('.test-search-result-sibling-section').get('.test-mini-preview-class:contains(raw_orders)').should('have.length', 2);
});

it('will combine results in lineage', () => {
cy.login();
cy.visit('dataset/urn:li:dataset:(urn:li:dataPlatform:dbt,cypress_project.jaffle_shop.stg_orders,PROD)/?is_lineage_mode=true');
Expand All @@ -122,7 +110,7 @@ describe('siblings', () => {
cy.login();
cy.visit('dataset/urn:li:dataset:(urn:li:dataPlatform:dbt,cypress_project.jaffle_shop.stg_orders,PROD)/?is_lineage_mode=true');

cy.get('[data-testid="compress-lineage-toggle"]').click({ force: true });
cy.clickOptionWithTestId('compress-lineage-toggle');

// check the subtypes
cy.get('text:contains(View)').should('have.length', 2);
Expand Down
2 changes: 1 addition & 1 deletion smoke-test/tests/cypress/cypress/support/commands.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ Cypress.Commands.add("enterTextInTestId", (id, text) => {
})

Cypress.Commands.add("clickOptionWithTestId", (id) => {
cy.get(selectorWithtestId(id)).click({
cy.get(selectorWithtestId(id)).first().click({
force: true,
});
})
Expand Down
2 changes: 1 addition & 1 deletion smoke-test/tests/cypress/integration_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ def test_run_cypress(frontend_session, wait_for_healthchecks):
record_key = os.getenv("CYPRESS_RECORD_KEY")
tag_arg = ""
test_strategy = os.getenv("TEST_STRATEGY", None)
if record_key is not None:
if record_key:
record_arg = " --record "
tag_arg = f" --tag {test_strategy} "
else:
Expand Down
2 changes: 1 addition & 1 deletion smoke-test/tests/managed-ingestion/managed_ingestion_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ def _ensure_ingestion_source_present(

if num_execs is not None:
ingestion_source = res_data["data"]["ingestionSource"]
assert ingestion_source["executions"]["total"] == num_execs
assert ingestion_source["executions"]["total"] > num_execs
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because sometimes the APIs took time to respond and by then another execution happened.


return res_data

Expand Down