Skip to content

Conversation

jtsodapop
Copy link
Contributor

@jtsodapop jtsodapop commented Mar 31, 2022

Update to test cases and availability to run on-prem given your local environment has the following var:

LABELBOX_TEST_API_KEY_ONPREM - take the api key from the onprem instance

LABELBOX_TEST_ONPREM_HOSTNAME example - https://app.replicated-bb2f865.labelbox.dev

@jtsodapop
Copy link
Contributor Author

jtsodapop commented Apr 1, 2022

It is difficult to make this consistently pass all test cases. The test results are very flaky for on-prem instance.

Example test case:
client.upload_data() method will intermittently fail and we will get a NoneType error (likely due to the result not being available yet)

Running several times will allow the test case to pass, but it is not guaranteed with X number of times.

image

image

With no changes, the same test can fail and pass in succession.

Edit 4/8:
With rerunfailures, able to make this more consistent.

@jtsodapop jtsodapop changed the title [WIP][AL-1859] Update SDK Tests for On-Prem [AL-1859] Update SDK Tests for On-Prem Apr 7, 2022
@jtsodapop jtsodapop requested a review from msokoloff1 April 8, 2022 19:29
…unt of time for label import. this should be updated since we changed how label creation is done via tests now
@msokoloff1 msokoloff1 self-requested a review April 11, 2022 13:13
Copy link
Contributor

@msokoloff1 msokoloff1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One nit, otherwise looks good.

@jtsodapop jtsodapop merged commit f1bb208 into develop Apr 11, 2022
@jtsodapop jtsodapop deleted the jt/al-1859 branch April 11, 2022 13:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants