Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Example-ORMs pulls files from 3rd party vendors on every CI run, and fails if upstream is unavailable #52342

Open
knz opened this issue Aug 4, 2020 · 5 comments
Labels
A-testing Testing tools and infrastructure C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. S-3-productivity Severe issues that impede the productivity of CockroachDB developers. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)

Comments

@knz
Copy link
Contributor

knz commented Aug 4, 2020

The Java tests in the Examples-ORMs pull their dependencies from repo.maven.apache.org upon every run.
The Python tests pull their dependencies from pipy servers.
And so on.

This is happening on every run. The files are not cached

This is causing unnecessary ingress costs on our side, and expensive egress cost on the side of 3rd party.

Lately we've had an uptick of failures due to the upstream server refusing our requests. (I wouldn't blame them personally, from their perspective we're an impolite leecher with a missing/incorrect proxy config.)

This is really a variation of #51543, with the problem focused on Examples-ORMs instead of the cdc roachtests.

Jira issue: CRDB-3960

@knz knz added C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. C-test-failure Broken test (automatically or manually discovered). S-3-productivity Severe issues that impede the productivity of CockroachDB developers. A-testing Testing tools and infrastructure labels Aug 4, 2020
@knz knz added this to To triage in SQL Sessions - Deprecated via automation Aug 4, 2020
@knz knz changed the title Example-ORMs pulls files from 3rd party vendors on every CI run Example-ORMs pulls files from 3rd party vendors on every CI run, and fails if upstream is unavailable Aug 4, 2020
@knz
Copy link
Contributor Author

knz commented Aug 4, 2020

cc @rafiss @jlinder

@rafiss
Copy link
Collaborator

rafiss commented Aug 4, 2020

Example failure:

https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_ExampleORMs/2146876?buildTab=log&focusLine=3&linesState=378

     > Could not resolve dom4j:dom4j:1.6.1.
        > Could not get resource 'https://repo.maven.apache.org/maven2/dom4j/dom4j/1.6.1/dom4j-1.6.1.pom'.
           > Could not GET 'https://repo.maven.apache.org/maven2/dom4j/dom4j/1.6.1/dom4j-1.6.1.pom'.
              > Connect to repo.maven.apache.org:443 [repo.maven.apache.org/199.232.64.215] failed: connect timed out

@rafiss rafiss moved this from To triage to Backlog in SQL Sessions - Deprecated Aug 4, 2020
@jbowens
Copy link
Collaborator

jbowens commented Aug 13, 2020

Here's another:

https://teamcity.cockroachdb.com/viewLog.html?tab=buildLog&logTab=tree&filter=debug&expand=all&buildId=2178687&_focus=573

[11:13:39][Step 2/3] :compileJava FAILED
[11:13:39][Step 2/3] 
[11:13:39][Step 2/3] FAILURE: Build failed with an exception.
[11:13:39][Step 2/3] 
[11:13:39][Step 2/3] * What went wrong:
[11:13:39][Step 2/3] Could not resolve all files for configuration ':compileClasspath'.
[11:13:39][Step 2/3] > Could not resolve com.fasterxml:classmate:1.3.0.
[11:13:39][Step 2/3]   Required by:
[11:13:39][Step 2/3]       project : > org.hibernate:hibernate-core:5.2.4.Final
[11:13:39][Step 2/3]    > Could not resolve com.fasterxml:classmate:1.3.0.
[11:13:39][Step 2/3]       > Could not get resource 'https://repo.maven.apache.org/maven2/com/fasterxml/classmate/1.3.0/classmate-1.3.0.pom'.
[11:13:39][Step 2/3]          > Could not GET 'https://repo.maven.apache.org/maven2/com/fasterxml/classmate/1.3.0/classmate-1.3.0.pom'.
[11:13:39][Step 2/3]             > Connect to repo.maven.apache.org:443 [repo.maven.apache.org/199.232.64.215] failed: connect timed out

@rafiss
Copy link
Collaborator

rafiss commented Feb 18, 2021

This could be resolved by the idea here: #59437

@knz
Copy link
Contributor Author

knz commented Feb 18, 2021

Funnily that's what we already do for all the crdb tests in pkg/acceptance, there's an acceptance test image with a bunch of languages/drivers. Maybe we can add to that?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testing Testing tools and infrastructure C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. S-3-productivity Severe issues that impede the productivity of CockroachDB developers. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Projects
No open projects
SQL Sessions - Deprecated
Longer term backlog
Development

No branches or pull requests

4 participants