Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GEOS-9566] Switch from repo.boundlessgeo.com to repo.osgeo.org #4165

Merged
merged 4 commits into from Apr 7, 2020

Conversation

tbarsballe
Copy link
Member

@tbarsballe tbarsballe commented Apr 6, 2020

As part of the ongoing repository migration, the main GeoServer pom needs to be updated with the new repo.

See GeoServer Migrate to OSGeo Infrastructure wiki page for details.

I've tested locally, seems to be working.

Checklist

For all pull requests:

  • Confirm you have read the contribution guidelines
  • You have sent a Contribution Licence Agreement (CLA) as necessary (not required for small changes, e.g., fixing typos in documentation)
  • Make sure the first PR targets the master branch, eventual backports will be managed later. This can be ignored if the PR is fixing an issue that only happens in a specific branch, but not in newer ones.

The following are required only for core and extension modules (they are welcomed, but not required, for community modules):

  • There is a ticket in Jira describing the issue/improvement/feature (a notable exemptions is, changes not visible to end users)
  • PR for bug fixes and small new features are presented as a single commit
  • Commit message must be in the form "[GEOS-XYZW] Title of the Jira ticket" (export to XML in Jira generates the message in this exact form)
  • The pull request contains changes related to a single objective. If multiple focuses cannot be avoided, each one is in its own commit and has a separate ticket describing it.
  • New unit tests have been added covering the changes
  • This PR passes all existing unit tests (test results will be reported by travis-ci after opening this PR)
  • This PR passes the QA checks (QA checks results will be reported by travis-ci after opening this PR)
  • Commits changing the UI, existing user workflows, or adding new functionality, need to include documentation updates (screenshots, text)
  • Commits changing the REST API, or any configuration object, should check if the REST API docs (Swagger YAML files and classic documentation) need to be updated.

Submitting the PR does not require you to check all items, but by the time it gets merged, they should be either satisfied or inapplicable.

@tbarsballe tbarsballe changed the title Switch from repo.boundlessgeo.com to repo.osgeo.org [GEOS-9566] Switch from repo.boundlessgeo.com to repo.osgeo.org Apr 6, 2020
@jodygarnett
Copy link
Member

jodygarnett commented Apr 6, 2020

These builds will not be able to pass until geotools is setup to deploy gt-main to repo.osgeo.org/geotools-snapshots:

[ERROR] Failed to execute goal on project geoserver: Could not resolve dependencies for project org.geoserver:geoserver:pom:2.18-SNAPSHOT: Failed to collect dependencies at org.geotools:gt-main:jar:24-SNAPSHOT: Failed to read artifact descriptor for org.geotools:gt-main:jar:24-SNAPSHOT: Could not transfer artifact org.geotools:gt-main:pom:24-SNAPSHOT from/to nexus (https://repo.osgeo.org/repository/Geoserver-releases/): Transfer failed for https://repo.osgeo.org/repository/Geoserver-releases/org/geotools/gt-main/24-SNAPSHOT/gt-main-24-SNAPSHOT.pom 400 Repository version policy: RELEASE does not allow version: 24-SNAPSHOT -> [Help 1]

@tbarsballe
Copy link
Member Author

These builds will not be able to pass until geotools is setup to deploy gt-main to repo.osgeo.org/geotools-snapshots`:

[ERROR] Failed to execute goal on project geoserver: Could not resolve dependencies for

This should just fail to find and move to the next repo (webdav), I think Nexus is a bit stricter about artifact names (in that it'll give you something other than a 404 if you try and request a snapshot artifact from a release repo) - testing a fix now.

@tbarsballe
Copy link
Member Author

tbarsballe commented Apr 6, 2020

I see webdav doesn't have the snapshots, just the releases. Never mind the above then, you're right, this'll need to wait until the geotools artifact migration is done (technically, we could merge it now to test deploy, since the build server will have all the snapshots cached, but that would likely be a bad idea)

@jodygarnett
Copy link
Member

Should we try updating the geotools build then?

@tbarsballe
Copy link
Member Author

Yeah, seems like we'll need to update the geotools build before merging this.

@aaime
Copy link
Member

aaime commented Apr 7, 2020

Same note as in the geootols PR, I'd try out removing the maven repo cache from the Travis build to make sure it gets a run "from empty repo"

@tbarsballe
Copy link
Member Author

GeoTools builds have completed successfully on Jenkins, and geotools-snapshots is now populated with 24-SNAPSHOT on nexus.

I've restarted the the CI builds here, we'll see if they now pass as-is. Removing the travis cache for GeoTools didn't really have much of an effect, so I'm going to hold off on that for now

@tbarsballe
Copy link
Member Author

New failure, so the GeoTools snapshots are working:

[ERROR] Failed to execute goal on project gs-importer-rest: Could not resolve dependencies for project org.geoserver.importer:gs-importer-rest:jar:2.18-SNAPSHOT: Could not find artifact org.restlet:org.restlet.ext.fileupload:jar:1.0.8 in osgeo-releases (https://repo.osgeo.org/repository/release/)

Looks like we should be mirroring Restlet as well, I'll look into it...

@jodygarnett
Copy link
Member

New failure, so the GeoTools snapshots are working:

[ERROR] Failed to execute goal on project gs-importer-rest: Could not resolve dependencies for project org.geoserver.importer:gs-importer-rest:jar:2.18-SNAPSHOT: Could not find artifact org.restlet:org.restlet.ext.fileupload:jar:1.0.8 in osgeo-releases (https://repo.osgeo.org/repository/release/)

Looks like we should be mirroring Restlet as well, I'll look into it...

What do we use Restlet for? I I can cache it but ... what for?

@tbarsballe
Copy link
Member Author

The answer to your question is literally right in that error message - importer file upload.

I've already added it, and restarted travis

@tbarsballe
Copy link
Member Author

Made it to the community modules:

jython:jython:jar:2.5.2 in osgeo-releases (https://repo.osgeo.org/repository/release/) -> [Help 1]
[ERROR] Failed to execute goal on project gs-dds: Could not resolve dependencies for project org.geoserver.community:gs-dds:jar:2.18-SNAPSHOT: Could not find artifact gov.nasa:worldwind:jar:0.6 in osgeo-releases (https://repo.osgeo.org/repository/release/) -> [Help 1]
[ERROR] Failed to execute goal on project gs-script-bsh: Could not resolve dependencies for project org.geoserver.script:gs-script-bsh:jar:2.18-SNAPSHOT: Could not find artifact org.beanshell:bsh:jar:2.1b5 in osgeo-releases (https://repo.osgeo.org/repository/release/) -> [Help 1]

Missing jython, gov.nasa:worldwind, and org.beanshell:bsh
Assuming these were more manual uploads, going to see if I can find and upload them

@jodygarnett jodygarnett closed this Apr 7, 2020
@jodygarnett jodygarnett reopened this Apr 7, 2020
@jodygarnett
Copy link
Member

jodygarnett commented Apr 7, 2020

May just be able to change dependencies:

@tbarsballe
Copy link
Member Author

I'd rather not change dependencies during the repo migration, even if it is minor versions, tends to make debugging even more annoying when you have to sort out missing artifacts and potential test failures from version updates, and that goes double for community modules.

I've added uploaded the missing artifacts (using the versions that were in repo.boundlessgeo.com), so we'll see if this next build pass works. Since it did get to the community modules last time, I don't expect any more missing artifacts to pop up.

We can always update those modules to a supported / existing version later on.

@jodygarnett
Copy link
Member

jodygarnett commented Apr 7, 2020

Thanks for the uploads, gets a lot further now: For this one org.geoscript:geoscript-py:jar:1.5-SNAPSHOT ... we may be stuck. We could depend on a release version (and exclude transitive dependencies) just to get it to build.

Checking repo.boundelssgeo.com there are no releases of the latest being:

<dependency>
    <groupId>org.geoscript</groupId>
    <artifactId>geoscript-py</artifactId>
    <version>1.5-20190104.230331-1</version>
    <type>pom</type>
</dependency>

@tbarsballe
Copy link
Member Author

tbarsballe commented Apr 7, 2020

Those snapshots are present in repo.boundlessgeo.com, so we could just upload them to geoserver-snapshots: https://repo.boundlessgeo.com/artifactory/main/org/geoscript/

Given that geoscript isn't really being maintained/deployed anymore, we could also just remove the problematic scripting modules from the build, as they are unmaintained community modules

@jodygarnett
Copy link
Member

jodygarnett commented Apr 7, 2020

Those snapshots are present in repo.boundlessgeo.com, so we could just upload them to geoserver-snapshots: #4165

I like that approach best, I was about to try building a release of geoscript or something.

Sigh, the user interface does not allow me to upload to snapshots (perhaps we can do so on the command line).

Update: It it has now been uploaded by hand.

@jodygarnett
Copy link
Member

@tbarsballe this build works for me locally now, if you can confirm we should merge

@tbarsballe
Copy link
Member Author

Alright, sounds good.
Oddly, the last Travis build is still failing on the same three artifacts as before (not the snapshots, but rather the three I uploaded), but I can see them in the repo and the build appears to be running with -U.

Testing out locally, I'm not getting the same failure (while I'm running with -U, I didn't clear my maven repo, so that might be it...)
I guess we can wait and see what it thinks about the current build...

@tbarsballe
Copy link
Member Author

Yep, just tested the build locally for me, I'll go ahead and merge, and keep an eye on Jenkins

@jodygarnett
Copy link
Member

Oddly, the last Travis build is still failing on the same three artifacts as before (not the snapshots, but rather the three I uploaded), but I can see them in the repo and the build appears to be running with -U.

I cleared my local cache locally (just removing the failed snapshot reference) in order to force it to check and download again.

@tbarsballe tbarsballe merged commit cd54dab into geoserver:master Apr 7, 2020
@jodygarnett
Copy link
Member

Thank you for your work on this @tbarsballe is went a lot faster with two of us.

Let's ask the mailing lists to test before backporting.

@tbarsballe
Copy link
Member Author

You are welcome, this turned out to be a rather fraught migration.
Agreed, once Jenkins master is passing we can ask on the lists to test; would definitely be good to get this backported soon enough that it gets into the release.

tbarsballe added a commit to tbarsballe/geoserver that referenced this pull request Apr 8, 2020
…erver#4165)

[GEOS-9566] Switch from repo.boundlessgeo.com to repo.osgeo.org

* Use group repositories from nexus (geotools + geoserver) for downloads

* Remove old osgeo webdav repository
tbarsballe added a commit to tbarsballe/geoserver that referenced this pull request Apr 8, 2020
…erver#4165)

[GEOS-9566] Switch from repo.boundlessgeo.com to repo.osgeo.org

* Use group repositories from nexus (geotools + geoserver) for downloads

* Remove old osgeo webdav repository
tbarsballe added a commit that referenced this pull request Apr 8, 2020
[GEOS-9566] Switch from repo.boundlessgeo.com to repo.osgeo.org

* Use group repositories from nexus (geotools + geoserver) for downloads

* Remove old osgeo webdav repository
tbarsballe added a commit that referenced this pull request Apr 8, 2020
[GEOS-9566] Switch from repo.boundlessgeo.com to repo.osgeo.org

* Use group repositories from nexus (geotools + geoserver) for downloads

* Remove old osgeo webdav repository
aaime pushed a commit to geosolutions-it/geoserver that referenced this pull request Aug 24, 2020
…erver#4165)

[GEOS-9566] Switch from repo.boundlessgeo.com to repo.osgeo.org

* Use group repositories from nexus (geotools + geoserver) for downloads

* Remove old osgeo webdav repository
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants