Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test omego upgrade as part of the Travis build #14

Merged
merged 9 commits into from
Oct 1, 2013

Conversation

sbesson
Copy link
Member

@sbesson sbesson commented Sep 22, 2013

This PR enables the omego upgrade command to be tested by the Travis build:

  • download, set up and start a 4.4.8p1 server in the Travis pre-install step
  • activate the testUpgrade integration test and run it as part of the test suite

Travis status should be 'success' meaning the test was run successfully. Upcoming steps can involve moving away the tests not requiring an OMERO instance into a separate Travis job and using Travis environments to test multiple upgrade conditions (Ice 3.3 -> Ice 3.5)

/cc @spli, @rleigh-dundee

@@ -21,6 +21,12 @@ before_script:
- psql -c "select 1;" -U omero -h localhost omero
- sudo mkdir /OMERO
- sudo chown $USER:$USER /OMERO
- wget http://cvs.openmicroscopy.org.uk/snapshots/omero/4.4.8p1/OMERO.server-4.4.8p1-ice34-b304.zip
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the moment, this makes good sense. I do wonder if we'll be able to come up with a download --latest-stable though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point. Actually, it's ironic that omego upgrade cannot be used for release upgrades at the moment. Worth opening an issue?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If omego upgrade is aimed at sysadmins (rather than devs) then the default should probably be to upgrade to releases only.

@qidane
Copy link

qidane commented Sep 24, 2013

I think this download path is a very bad idea. As travis nodes are provided by lots of different people these requests could come from anywhere (any IP address). I do not see how they could be filtered.

Would it be possible to download from a different domain that we can ignore all stats for?

@qidane
Copy link

qidane commented Sep 24, 2013

Or is there an option you can provide to set the download client to some string containing "ignore" we can then filter all the logs for?

@sbesson
Copy link
Member Author

sbesson commented Sep 24, 2013

Would wget --referer=https://travis-ci.org ... be supported?

@qidane
Copy link

qidane commented Sep 24, 2013

That could work, but with a speed penalty (5 times slower): http://awstats.sourceforge.net/docs/awstats_config.html#SkipReferrersBlackList

@joshmoore
Copy link
Member

I would think once we setup the "latest-downloads" we also just provide a (otherwise unused) travis URL. @qidane: can you imagine an easy-enough scheme for doing something like that?

@qidane
Copy link

qidane commented Sep 25, 2013

I think the right apache rewrite would handle it so you could map cvs.openmicroscopy.org/travis/snapshots to cvs.openmicroscopy.org/snapshots, but the URL could still get public. A custom user agent along with a "SkipUserAgents" directive might be the better option, but as @sbesson pointed out the help for wget says "Use of this option is discouraged, unless you really know what you are doing." :-)

@joshmoore
Copy link
Member

/travis/snapshots seems fine to me. But we'll need the latest login in place for quite a lot of the artifacts, updated on every release (automatically).

@joshmoore
Copy link
Member

@qidane
Copy link

qidane commented Sep 25, 2013

Test using user agent:

Command A:
wget http://cvs.openmicroscopy.org.uk/snapshots/omero/4.4.8p1/OMERO.server-4.4.8p1-ice34-b304.zip
Log entry A:
10.12.1.123 - - [25/Sep/2013:10:25:32 +0100] "GET /snapshots/omero/4.4.8p1/OMERO.server-4.4.8p1-ice34-b304.zip HTTP/1.1" 200 197096680 "-" "Wget/1.14 (darwin11.4.2)"
Command B:
wget --user-agent=Travis http://cvs.openmicroscopy.org.uk/snapshots/omero/4.4.8p1/OMERO.server-4.4.8p1-ice34-b304.zip
Log entry B
10.12.1.123 - - [25/Sep/2013:10:26:29 +0100] "GET /snapshots/omero/4.4.8p1/OMERO.server-4.4.8p1-ice34-b304.zip HTTP/1.1" 200 197096680 "-" "Travis"

@joshmoore
Copy link
Member

@qidane : can we make a choice on what you want to see in the way of URLs and/or agents and get this merged?

@qidane
Copy link

qidane commented Sep 30, 2013

Using wget --user-agent=Travis would be best I think. AWStats can then filter out that agent.

@qidane
Copy link

qidane commented Sep 30, 2013

The request to www.openmicroscopy.org/info/scripts and www.openmicroscopy.org/latest/omero-manual-pdf are generating entries in our log files (302 and 303 response codes respectfully). Both are being ignored by awstats. The subsequent requests to the targets of the redirects are being counted.

This allows an easier customization of the opening request header. This PR
also allows to set the download user-agent using the USER_AGENT environment
variable.
Also remove debugging line in omego/artifacts.py
@joshmoore
Copy link
Member

Any more planned commits, @sbesson?

@sbesson
Copy link
Member Author

sbesson commented Oct 1, 2013

Set of commits above is setting Travis as the User agent for all requests to either cvs.openmicroscopy.org.uk or hudson.openmicroscopy.org.uk. Confirmed by manually parsing the access_log.
Unless @qidane says otherwise, I think this PR is good to go.

@qidane
Copy link

qidane commented Oct 1, 2013

Looks fine to me.

joshmoore added a commit that referenced this pull request Oct 1, 2013
Test `omego upgrade` as part of the Travis build
@joshmoore joshmoore merged commit ca3d71f into ome:master Oct 1, 2013
@sbesson sbesson deleted the upgrade_travis branch October 1, 2013 08:52
@qidane
Copy link

qidane commented Oct 1, 2013

In all awstats config files in /etc/awstats/ I have set SkipUserAgents="Travis Hudson"
Note: this is a space separated list and is NOT case sensitive.

@sbesson sbesson added this to the 0.1.1 milestone Dec 11, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants