[Staging] Changes to enable staging instance. #1089
Conversation
|
I've just enabled the Jenkins integration that will deploy to staging in Tycho (on push to stable branch). Results should be visible in https://testing.laboratoriopublico.org/ |
|
Here's the Jenkins log (i've enabled public read-only access). |
|
All right, shall I merge this and then merge it to stable, and do I need to
include a code change along with it?
On Dec 12, 2016 3:30 AM, "Sebastian Silva" <notifications@github.com> wrote:
Here's the Jenkins log (i've enabled public read-only access).
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#1089 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AABfJwmAvqQFCoZo_tjFja5ZMlbfAGmZks5rHQY9gaJpZM4LKR_r>
.
|
|
Just this merge _should_ be enough to trigger the rebuild and deploy.
I've tested it on my fork, and just switched to this repo earlier today.
Hopefully it will work the first time!
…On 12/12/16 08:40, Jeffrey Warren wrote:
All right, shall I merge this and then merge it to stable, and do I
need to
include a code change along with it?
On Dec 12, 2016 3:30 AM, "Sebastian Silva" ***@***.***>
wrote:
Here's the Jenkins log (i've enabled public read-only access).
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#1089 (comment)>,
or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AABfJwmAvqQFCoZo_tjFja5ZMlbfAGmZks5rHQY9gaJpZM4LKR_r>
.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#1089 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAMMSzTfzsDcM8TMlRpd-LES7w506gFnks5rHU7dgaJpZM4LKR_r>.
|
|
On 12/12/16 08:40, Jeffrey Warren wrote:
All right, shall I merge this and then merge it to stable,
The build should trigger upon merging to stable.
|
|
Just wondering about the changes to lib/tasks/test.rake -- they were partially intended to allow people to run solr tests locally without too much configuration change. Will that still be possible, or would it now be container dependent? And if not, might we make a separate"manual" solr rake test task? |
|
Shouldn't they be using the "development" environment?
I wonder otherwise when are Solr tests run?
We can leave out that change if you prefer, the intention was to make
sure they are run.
…On 12/12/16 09:18, Jeffrey Warren wrote:
Just wondering about the changes to lib/tasks/test.rake -- they were
partially intended to allow people to run solr tests locally without
too much configuration change. Will that still be possible, or would
it now be container dependent? And if not, might we make a
separate"manual" solr rake test task?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#1089 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAMMS983p1PFW821e7PxQuHnwoWFVMdVks5rHVfCgaJpZM4LKR_r>.
|
|
Well, the test fixture data for them is in the test database, so that's why
we have that complicated switching on and off of the solr requirements. If
we used development, we'd need to load the fixture data into the
development db, which isn't going to be identical for everyone.
…On Dec 12, 2016 9:23 AM, "Sebastian Silva" ***@***.***> wrote:
Shouldn't they be using the "development" environment?
I wonder otherwise when are Solr tests run?
We can leave out that change if you prefer, the intention was to make
sure they are run.
On 12/12/16 09:18, Jeffrey Warren wrote:
>
> Just wondering about the changes to lib/tasks/test.rake -- they were
> partially intended to allow people to run solr tests locally without
> too much configuration change. Will that still be possible, or would
> it now be container dependent? And if not, might we make a
> separate"manual" solr rake test task?
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <#1089 (comment)>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/
AAMMS983p1PFW821e7PxQuHnwoWFVMdVks5rHVfCgaJpZM4LKR_r>.
>
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1089 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AABfJ22t6UBee7GFnP0fWCCBkcDCvAvhks5rHVj3gaJpZM4LKR_r>
.
|
|
On 12/12/16 09:29, Jeffrey Warren wrote:
Well, the test fixture data for them is in the test database, so
that's why
we have that complicated switching on and off of the solr requirements. If
we used development, we'd need to load the fixture data into the
development db, which isn't going to be identical for everyone.
Okay I didn't understand that last bit. I honestly don't think the
embedded Solr engine should be used for anything.
If it is desired to run tests for Solr, it should be done with the Solr
container.
In the meantime I'm force pushing a new version of the patch omitting
that change. Thanks for the review.
Another aspect of the PR which is a departure from usual practice is
checking in Gemfile.lock. I've read it's a good practice. It was the
only way I got both Travis and Jenkins to build correctly, but there
might be a way around it if you feel strongly about it.
…
On Dec 12, 2016 9:23 AM, "Sebastian Silva" ***@***.***>
wrote:
> Shouldn't they be using the "development" environment?
>
> I wonder otherwise when are Solr tests run?
>
> We can leave out that change if you prefer, the intention was to make
> sure they are run.
>
>
> On 12/12/16 09:18, Jeffrey Warren wrote:
> >
> > Just wondering about the changes to lib/tasks/test.rake -- they were
> > partially intended to allow people to run solr tests locally without
> > too much configuration change. Will that still be possible, or would
> > it now be container dependent? And if not, might we make a
> > separate"manual" solr rake test task?
> >
> > —
> > You are receiving this because you authored the thread.
> > Reply to this email directly, view it on GitHub
> >
<#1089 (comment)>,
> > or mute the thread
> > <https://github.com/notifications/unsubscribe-auth/
> AAMMS983p1PFW821e7PxQuHnwoWFVMdVks5rHVfCgaJpZM4LKR_r>.
> >
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <#1089 (comment)>,
> or mute the thread
>
<https://github.com/notifications/unsubscribe-auth/AABfJ22t6UBee7GFnP0fWCCBkcDCvAvhks5rHVj3gaJpZM4LKR_r>
> .
>
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#1089 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAMMS_gJ5LDr_RJ4t9SlV9wmY1zC6dR4ks5rHVpSgaJpZM4LKR_r>.
|
This patch sets up Solr into its own independent container. A Travis server has been setup to build and deploy automatically upon commit to stable branch.
|
I honestly don't think the embedded Solr engine should be used for anything.
I'm fine with that as a policy - but we should update the README
accordingly. If this pr doesn't depend on this change in policy, perhaps we
can open an issue for it in the project repository so there's a "paper
trail"?
And itll be less of an issue as we shift towards using docker for the
developer install process itself, which is probably a good idea.
Gemfile.lock sounds fine -- does the install/deploy process change as a
result? This essentially locks gem versions, doesn't it?
Thanks!
|
|
Note earlier I pushed a commit amendment withdrawing changes to test.rake. |
|
Merged! |
|
It was triggered!
I'll check the permissions and rebuild, after I serve lunch.
…On 12/12/16 14:51, Jeffrey Warren wrote:
Merged #1089 <#1089>.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#1089 (comment)>, or
mute the thread
<https://github.com/notifications/unsubscribe-auth/AAMMS4yfnI6Mj8fBQqleISftrLV1zstvks5rHaXHgaJpZM4LKR_r>.
|
|
Ack, sorry I didn't notice that this was a PR against the |
|
Yes the reason is that this pull request's purpose is to tweak docker
files for building stable branch, and test automation of this branch's
triggers.
It can be merged into master of course at any time you prefer.
…On 12/12/16 14:53, Jeffrey Warren wrote:
Ack, sorry I didn't notice that this was a PR against the |stable|
branch -- I'd have expected any changes to happen on |master| first,
since our flow is |master| => |stable|. But I don't want to revert
now. Shall we open a PR to merge |stable| into |master|, just this once?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#1089 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAMMS7gbue4WEjHrZDYvVoO7v3TBduenks5rHaZKgaJpZM4LKR_r>.
|
|
I had hoped that this would go green at first try.
I had touched .git with my user (icarito), that's why permissions were
wrong when checking out from github.
I did to Tycho, to /srv/plots_testing/plots and did chown
testing.testing .git -R . After this I manually triggered the build and
it went well all the way to deployment.
'testing' is the user jenkins runs under (Jenkins itself runs under a
container), and /srv/plots_testing/plots is mounted as a volume on it.
So this instance is running from /srv/plots_testing/plots and is
controlled with docker-compose.
I've set it up to depend on having RAILS_ENV set. In the case of
staging, it's using the `*production*` environment.
For instance, I trigger the reindex (which still fails), with:
RAILS_ENV=production docker-compose run web bundle exec rake
sunspot:reindex --trace
…On 12/12/16 14:53, Sebastian Silva wrote:
It was triggered!
I'll check the permissions and rebuild, after I serve lunch.
On 12/12/16 14:51, Jeffrey Warren wrote:
>
> Merged #1089 <#1089>.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <#1089 (comment)>, or
> mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAMMS4yfnI6Mj8fBQqleISftrLV1zstvks5rHaXHgaJpZM4LKR_r>.
>
|
|
By the way, the build results are at http://tycho.media.mit.edu:8080/job/Plots-Staging/ |
|
Great, do we need additional changes or would you mind opening a Pr against
the master branch so we can merge in other contributions to master? Thanks!
…On Dec 12, 2016 3:40 PM, "Sebastian Silva" ***@***.***> wrote:
By the way, the build results are at http://tycho.media.mit.edu:
8080/job/Plots-Staging/
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
<#1089 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AABfJ2AR4gTytc8IUT5tPVAl3otuyRJqks5rHbEmgaJpZM4LKR_r>
.
|
|
@icarito - just checking if this is ready for merge into master, as we have a couple other PRs in the queue. Thanks! |
|
Yes please do.
…On 12/12/16 18:24, Jeffrey Warren wrote:
@icarito <https://github.com/icarito> - just checking if this is ready
for merge into master, as we have a couple other PRs in the queue. Thanks!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1089 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAMMS5ox-kNfZo7lWy_AKCNpJhIXcf8iks5rHdfAgaJpZM4LKR_r>.
|
|
Done! |
|
Going deeper into this:
I wasn't precise - I meant for anything user-facing. Developers should be free to use the embedded engine if it suits them best. I don't think we should require Docker for casual developers (or Java, unless they specifically want to work on Solr). Casual developers probably won't be running tests either, since Travis does this already - We need to get Travis to notice when tests fail though -> https://github.com/jywarren/web/issues/70
The current practice can continue, some care should be taken to avoid committing this file accidentally (it's already in .gitignore, so to commit it you need to -f force it). Every now and then it can be removed and created again by Bundler, when wanting to upgrade Gems. |
This isn't something I'd anticipated when we first got it running, but increasingly I've seen this to be workable... esp. for first-timers, who can later move on to a) a containerized version on Cloud9, and/or b) their own local version. So -- really great! Lower barrier! But yeah, one reason I moved the Solr tests to their own suite that doesn't run on But since running Solr tests locally, without Docker, turned out to be a bit more complex, and had this kind of silly set of requirements, like changing the config file and then changing it back afterwards, we set up the It seems like we may have 2 use cases: local Solr testing (and its eccentricities) and Travis automated Solr testing (which should be container-based). Can we preserve the first as For Gemfile.lock, should we put a note somewhere (like at the top of Gemfile, or even in the PR template) to update it in these scenarios? It sounds like something easy to forget at deploy time, though I guess tests will help us catch it, so maybe it's not needed. |
Make sure these boxes are checked before your pull request is ready to be reviewed and merged. Thanks!
rake test:allschema.rb.examplehas been updated if any database migrations were addedPlease be sure you've reviewed our contribution guidelines at https://publiclab.org/wiki/contributing-to-public-lab-software
We have a loose schedule of reviewing and pulling in changes every Tuesday and Friday, and publishing changes on Fridays. Please alert developers on plots-dev@googlegroups.com when your request is ready or if you need assistance.
Thanks!
This patch sets up Solr into its own independent container.
A Travis server has been setup to build and deploy automatically upon commit to stable branch.