RethinkDB addon (WIP) #676

danielmewes · 2016-03-09T20:22:26Z

This adds a RethinkDB addon.

The supplied test is currently failing as follows, and I'm not sure what that means:

Failures:

  1) Travis::Build::Addons::Rethinkdb 
     Failure/Error: raise TaintedOutput if !ignore_taint && script.tainted?

     Travis::Shell::Generator::TaintedOutput:
       Travis::Shell::Generator::TaintedOutput
     # ./lib/travis/shell/generator.rb:16:in `generate'
     # ./lib/travis/shell.rb:10:in `generate'
     # spec/spec_helpers/sexp.rb:55:in `store_example'
     # ./spec/build/addons/rethinkdb_spec.rb:12:in `block (2 levels) in <top (required)>'

  2) Travis::Build::Addons::Rethinkdb behaves like compiled script can be compiled
     Failure/Error: expect { subject }.to_not raise_error

       expected no Exception, got #<Travis::Shell::Generator::TaintedOutput: Travis::Shell::Generator::TaintedOutput> with backtrace:
         # ./lib/travis/shell/generator.rb:16:in `generate'
         # ./lib/travis/shell.rb:10:in `generate'
         # ./lib/travis/build/script.rb:72:in `compile'
         # spec/build/script/shared/script.rb:7:in `block (2 levels) in <top (required)>'
         # spec/build/script/shared/script.rb:14:in `block (3 levels) in <top (required)>'
         # spec/build/script/shared/script.rb:14:in `block (2 levels) in <top (required)>'
     Shared Example Group: "compiled script" called from ./spec/build/addons/rethinkdb_spec.rb:14
     # spec/build/script/shared/script.rb:14:in `block (2 levels) in <top (required)>'

  3) Travis::Build::Addons::Rethinkdb behaves like compiled script includes the expected travis_cmds
     Failure/Error: raise TaintedOutput if !ignore_taint && script.tainted?

     Travis::Shell::Generator::TaintedOutput:
       Travis::Shell::Generator::TaintedOutput
     Shared Example Group: "compiled script" called from ./spec/build/addons/rethinkdb_spec.rb:14
     # ./lib/travis/shell/generator.rb:16:in `generate'
     # ./lib/travis/shell.rb:10:in `generate'
     # ./lib/travis/build/script.rb:72:in `compile'
     # spec/build/script/shared/script.rb:7:in `block (2 levels) in <top (required)>'
     # spec/build/script/shared/script.rb:25:in `block (3 levels) in <top (required)>'
     # spec/build/script/shared/script.rb:24:in `each'
     # spec/build/script/shared/script.rb:24:in `block (2 levels) in <top (required)>'

Finished in 16.45 seconds (files took 1.77 seconds to load)
3090 examples, 3 failures, 11 pending

Failed examples:

rspec ./spec/build/addons/rethinkdb_spec.rb:12 # Travis::Build::Addons::Rethinkdb 
rspec ./spec/build/addons/rethinkdb_spec.rb[1:2:1] # Travis::Build::Addons::Rethinkdb behaves like compiled script can be compiled
rspec ./spec/build/addons/rethinkdb_spec.rb[1:2:3] # Travis::Build::Addons::Rethinkdb behaves like compiled script includes the expected travis_cmds

Mentioning @joshk who suggested opening the pull request despite the remaining failures.

lib/travis/build/addons/rethinkdb.rb

+        def after_prepare
+          sh.fold 'rethinkdb' do
+            sh.echo "Installing RethinkDB version #{rethinkdb_version}", ansi: :yellow
+            sh.cmd "service rethinkdb stop", sudo: true


BanzaiMan · 2016-03-09T23:36:48Z

Not sure where the taintedness is coming into play, but with my fixes, that is not a problem when I run the specs locally.

BanzaiMan · 2016-03-09T23:46:10Z

A possible fix for these issues: https://gist.github.com/BanzaiMan/20cc01553a1948236138

danielmewes · 2016-03-10T21:24:39Z

Thanks for the feedback and proposed fixes @BanzaiMan . I'll update the PR shortly.

Do you have any suggestions or example code for how to check for the operating system?

BanzaiMan · 2016-03-10T21:46:20Z

@danielmewes $TRAVIS_OS_NAME should be set to either linux or osx by the time this addon gets to run.

danielmewes · 2016-03-15T00:21:53Z

I applied your fixes @BanzaiMan and the tests are now passing. However I couldn't get the OS check to work. I tried checking ENV['TRAVIS_OS_NAME'], but it was always empty during testing. Any ideas?

BanzaiMan · 2016-03-15T00:26:57Z

@danielmewes How are you testing it?

danielmewes · 2016-03-15T00:39:01Z

@BanzaiMan I was using the Vagrant file and running bundle exec rspec spec as described in the README https://github.com/travis-ci/travis-build/blob/master/README.md .

lib/travis/build/addons/rethinkdb.rb

+
+        def after_prepare
+          sh.fold 'rethinkdb' do
+            #if ENV['TRAVIS_OS_NAME'] != 'linux'


BanzaiMan · 2016-04-14T01:40:51Z

@danielmewes Do you need help?

danielmewes · 2016-04-20T22:35:41Z

@BanzaiMan Thanks for your comments. Picking this up again now...

danielmewes · 2016-04-20T22:44:43Z

@BanzaiMan Updated as suggested. This seems to be working now.

hueniverse · 2016-04-28T18:12:28Z

What's the issue here? What needs to happen to get this merged? Who is this blocked with?

lib/travis/build/addons/rethinkdb.rb

+
+        def after_prepare
+          sh.fold 'rethinkdb' do
+            sh.if "ENV['TRAVIS_OS_NAME'] != 'linux'" do


hueniverse · 2016-06-02T00:03:30Z

@danielmewes @BanzaiMan Who are we waiting on at this point? I am not sure if more changes are needed from @danielmewes or if @BanzaiMan needs to help fix something. Either way, this was supposed to be a quick solution back in March. It's now June...

Thanks!

danielmewes · 2016-06-02T02:01:48Z

Thanks for the explanation @BanzaiMan .
I've updated the OS check.

RethinkDB works on Mac, but the setup procedure is entirely different and I don't want to deal with that now.

RethinkDB addon (WIP)

BanzaiMan · 2016-06-02T18:20:18Z

@danielmewes Thanks for the update. I ran a quick test to install 2.3, but it is failing: https://staging.travis-ci.org/BanzaiMan/travis_staging_test/builds/493918#L143-L166

Installing RethinkDB version 2.3
rethinkdb: unrecognized service
Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --secret-keyring /tmp/tmp.rvZe4OH5c8 --trustdb-name /etc/apt/trustdb.gpg --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --keyring /etc/apt/trusted.gpg.d//apt.postgresql.org.gpg --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0x3A8F2399
gpg: requesting key 3A8F2399 from hkp server keyserver.ubuntu.com
gpg: key 3A8F2399: public key "RethinkDB Packaging <packaging@rethinkdb.com>" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
W: Failed to fetch http://download.rethinkdb.com/apt/dists/precise/Release  Unable to find expected entry 'main/source/Sources' in Release file (Wrong sources.list entry or malformed file)
E: Some index files failed to download. They have been ignored, or old ones used instead.
$ sudo apt-get install -y -o Dpkg::Options::='--force-confnew' rethinkdb=$package_version
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package rethinkdb
Installing RethinkDB default instance configuration
cp: cannot stat `/etc/rethinkdb/default.conf.sample': No such file or directory
Starting RethinkDB v2.3
$ sudo service rethinkdb start
rethinkdb: unrecognized service
$ rethinkdb --version
/home/travis/build.sh: line 45: rethinkdb: command not found

Could you review these messages?

Also, do you want the build to fail immediately if installation fails for one reason or another?

danielmewes · 2016-06-02T20:28:52Z

Ah, I think this is the same issue as travis-ci/travis-ci#4772 . I'll change the add-apt-repository line...

lib/travis/build/addons/rethinkdb.rb

+  module Build
+    class Addons
+      class Rethinkdb < Base
+        SUPER_USER_SAFE = true


hueniverse · 2016-06-21T22:06:54Z

@BanzaiMan does this mean this will not work with the new infrastructure?

@danielmewes requiring sudo makes this a lot less useful because it blocks it out of the new infrastructure. Goal is to make using rethink as easy as all the other built-in dbs available in Travis.

danielmewes · 2016-06-21T22:14:35Z

@BanzaiMan Thanks for the comments. I'll look into those things.

@hueniverse Both the MariaDB (which I based this on) and PostgreSQL addons also require sudo on the destination system. Do you have an example for a DB addon that works without it that I could look at?

BanzaiMan · 2016-06-21T22:28:56Z

If the only requirement is sudo, you can do stuff in the configure phase instead.

BanzaiMan · 2016-06-21T22:31:12Z

Scratch that. after_prepare also has sudo (and you are using it).

hueniverse · 2016-06-22T03:46:20Z

@BanzaiMan So we are good as is with the sudo requirements?

BanzaiMan · 2016-06-22T04:07:53Z

@hueniverse Yes, I think so. The version specification still needs to be fixed.

BanzaiMan · 2016-06-27T17:28:18Z

Also a PR to docs, please.

hueniverse · 2016-06-27T17:32:39Z

@BanzaiMan Can you point me where those docs are and a similar section to based this on?

BanzaiMan · 2016-06-27T17:50:53Z

@hueniverse It should go into https://github.com/travis-ci/docs-travis-ci-com/blob/gh-pages/user/database-setup.md. (Which is rendered at https://docs.travis-ci.com/user/database-setup/). You can see similar sections (e.g., MariaDB) on that page.

danielmewes · 2016-07-19T21:25:43Z

@BanzaiMan I fixed the SUPER_USER_SAFE line, the line for piping into /etc/apt/sources.list, and re-implemented how the RethinkDB package version is determined. It is now possible to set rethinkdb_version to any prefix of the target version, e.g. "2.3".
The script will then install the highest version with that prefix, e.g. "2.3.4~0precise".

danielmewes · 2016-07-19T21:25:51Z

Documentation is coming.

danielmewes · 2016-07-20T00:07:07Z

Documentation is in travis-ci/docs-travis-ci-com#673

hueniverse · 2016-07-20T06:44:00Z

@danielmewes you're my hero!

RethinkDB addon (WIP)

BanzaiMan · 2016-07-20T16:20:01Z

OK, this has been deployed to staging one more time.

BanzaiMan · 2016-07-20T19:10:26Z

Test: https://staging.travis-ci.org/BanzaiMan/travis_staging_test/builds/506951#L166

lib/travis/build/addons/rethinkdb.rb

+  module Build
+    class Addons
+      class Rethinkdb < Base
+        SUPER_USER_SAFE = false


danielmewes · 2016-07-20T19:21:58Z

Thanks for testing and your comments @BanzaiMan . I turned SUPER_USER_SAFE back on.

BanzaiMan · 2016-07-20T19:32:19Z

Deployed to production now.

danielmewes · 2016-07-20T19:43:41Z

Thanks again for your help in getting this into shape @BanzaiMan !

hueniverse · 2016-07-20T20:09:57Z

Thanks @BanzaiMan @danielmewes! Can't wait to try it out.

RethinkDB addon (WIP)

095836f

BanzaiMan reviewed Mar 9, 2016
View reviewed changes

lib/travis/build/addons/rethinkdb.rb

def after_prepare

sh.fold 'rethinkdb' do

sh.echo "Installing RethinkDB version #{rethinkdb_version}", ansi: :yellow

sh.cmd "service rethinkdb stop", sudo: true

This comment was marked as spam.

Sign in to view

Applied fixes by @BanzaiMan

778f6d8

BanzaiMan reviewed Mar 31, 2016
View reviewed changes

lib/travis/build/addons/rethinkdb.rb

def after_prepare

sh.fold 'rethinkdb' do

#if ENV['TRAVIS_OS_NAME'] != 'linux'

This comment was marked as spam.

Sign in to view

marshallswain mentioned this pull request Apr 1, 2016

Automated tests in Travis feathersjs-ecosystem/feathers-rethinkdb#12

Closed

Working OS check

68d9b2a

BanzaiMan mentioned this pull request Apr 21, 2016

Set status of GitHub merge commits travis-ci/travis-ci#5937

Closed

BanzaiMan reviewed Apr 28, 2016
View reviewed changes

lib/travis/build/addons/rethinkdb.rb

def after_prepare

sh.fold 'rethinkdb' do

sh.if "ENV['TRAVIS_OS_NAME'] != 'linux'" do

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

Fixed OS check

10dfb61

BanzaiMan added a commit that referenced this pull request Jun 2, 2016

Merge pull request #676 from rethinkdb/master

52a6c3d

RethinkDB addon (WIP)

BanzaiMan added a commit that referenced this pull request Jun 2, 2016

Merge pull request #676 from rethinkdb/master

b8aa47a

RethinkDB addon (WIP)

Use echo to add rethinkdb repository

6d1ca4f

BanzaiMan reviewed Jun 21, 2016
View reviewed changes

lib/travis/build/addons/rethinkdb.rb

module Build

class Addons

class Rethinkdb < Base

SUPER_USER_SAFE = true

This comment was marked as spam.

Sign in to view

Improved logic to determine RethinkDB version

9915e9e

chipotle mentioned this pull request Jul 19, 2016

Add RethinkDB addon to docs travis-ci/docs-travis-ci-com#673

Merged

BanzaiMan added a commit that referenced this pull request Jul 20, 2016

Merge pull request #676 from rethinkdb/master

7483110

RethinkDB addon (WIP)

BanzaiMan mentioned this pull request Jul 20, 2016

Allow rethinkdb in PRs travis-ci/travis-scheduler#24

Merged

BanzaiMan reviewed Jul 20, 2016
View reviewed changes

lib/travis/build/addons/rethinkdb.rb

module Build

class Addons

class Rethinkdb < Base

SUPER_USER_SAFE = false

This comment was marked as spam.

Sign in to view

Turn SUPER_USER_SAFE on again.

46af90f

BanzaiMan merged commit ed74f59 into travis-ci:master Jul 20, 2016

BanzaiMan mentioned this pull request Oct 1, 2018

Feature Request: add Consul to available Database Services travis-ci/travis-ci#5463

Closed

RethinkDB addon (WIP) #676

RethinkDB addon (WIP) #676

Conversation

danielmewes commented Mar 9, 2016

This comment was marked as spam.

BanzaiMan commented Mar 9, 2016

BanzaiMan commented Mar 9, 2016

danielmewes commented Mar 10, 2016

BanzaiMan commented Mar 10, 2016

danielmewes commented Mar 15, 2016

BanzaiMan commented Mar 15, 2016

danielmewes commented Mar 15, 2016

This comment was marked as spam.

BanzaiMan commented Apr 14, 2016

danielmewes commented Apr 20, 2016

danielmewes commented Apr 20, 2016

hueniverse commented Apr 28, 2016

This comment was marked as spam.

This comment was marked as spam.

hueniverse commented Jun 2, 2016

danielmewes commented Jun 2, 2016

BanzaiMan commented Jun 2, 2016

danielmewes commented Jun 2, 2016

This comment was marked as spam.

hueniverse commented Jun 21, 2016

danielmewes commented Jun 21, 2016

BanzaiMan commented Jun 21, 2016

BanzaiMan commented Jun 21, 2016

hueniverse commented Jun 22, 2016

BanzaiMan commented Jun 22, 2016

BanzaiMan commented Jun 27, 2016

hueniverse commented Jun 27, 2016

BanzaiMan commented Jun 27, 2016 • edited Loading

danielmewes commented Jul 19, 2016

danielmewes commented Jul 19, 2016

danielmewes commented Jul 20, 2016

hueniverse commented Jul 20, 2016

BanzaiMan commented Jul 20, 2016

BanzaiMan commented Jul 20, 2016

This comment was marked as spam.

danielmewes commented Jul 20, 2016

BanzaiMan commented Jul 20, 2016

danielmewes commented Jul 20, 2016

hueniverse commented Jul 20, 2016

BanzaiMan commented Jun 27, 2016 •

edited

Loading