Test using Docker based Solr rather than solr_wrapper

[jcoyne]

No description provided.

[coveralls]

Coverage remained the same at 94.725% when pulling d57cd9f on docker-solr into a0c6247 on master.

[mejackreed]

A question about this approach, do we need to offer a way that Blacklight adopters can easily run Blacklight + Solr locally for their development?

[jcoyne]

@mejackreed I think this still preserves generating the solr_wrapper into the host application gemfile, it just doesn't use solr_wrapper for rake ci

[dkinzer]

I think it will be easier on adopters to use docker compose. And running our application can be done using docker-compose up ... also, we can update rake blacklight:server to use docker-compose up for backwards compatibility.

Here is an example compose file that I've been using successfully for the last year: https://github.com/tulibraries/tul_cob/blob/master/docker-compose.yml

Note that I prefer not to bind the ports so that I can load multiple applications at once. But this is not necessary. Also I've configured to be able to work with traefik with which is not something most users will care about.

[jrochkind]

Where does docker pull solr:7 get things from? Sorry, I'm not too familiar with docker.

This is getting a solr to run from... an image maintained by someone else? How does it have the schema that BL expects, or does BL no longer expect a non-default solr schema?

If new versions of solr are released, will this automatically get them? (Present solution does, theoretically. I am not at all opposed to giving that up though).

[jcoyne]

@jrochkind The image is pulled from the repository here: https://hub.docker.com/_/solr/
The 7 tag is used for the latest packaged version of solr 7.x.x. You could specify a more restrictive match if needed.

The config is sent to solr with the command here:
https://github.com/projectblacklight/blacklight/pull/2012/files#diff-354f30a63fb0907d4ad57269548329e3R32

[jrochkind]

Ah, I see, thanks.

The present solution does that checksum-checking thing, which is an area that has lately been full of bugs/churn. This solution does not do that at all (which I think is fine), which avoids that whole thing being a problem. (Present solution could also have checksum-checking turned off).

[cdmo]

FWIW, as a result of this conversation I decided to test out this docker solr approach for our CI and it worked great. We were having some issues getting solr_wrapper to run, and this worked pretty much right away. So, for CI, this seems like the easier option (speaking as a person fairly new to BL).

[mejackreed]

Great @cdmo ! One more quick question, is there a way that running rake or rake ci from Blacklight can be setup so that this will all work correctly?

I'm trying to understand the CI vs local development modes here and how they might be used.

[dkinzer]

It might be that docker is just available and running in the travis environment. But locally a user would either have to have it running or enable it before running it. Someone in slack pointed this out as a possible hurdle to adoption; and I guess I agree.

[cbeer]

👏

It'd be nice to make it easy to run tests locally that isn't:

In a separate terminal window, run:
$ docker pull solr:7
$ docker run -d -p 8983:8983 -v $PWD/lib/generators/blacklight/templates/solr/conf:/myconfig solr:7 solr-create -c blacklight-core -d /myconfig

I wonder if we ought to wrap that in a rake task? (Actually, saying that, I wonder if we could add a flag to solr_wrapper to have it use docker instead of downloading and running solr from source? Maybe that'd address some of the hurdles to adoption?)

[cdmo]

@cbeer I was thinking the same thing and just made a quick attempt here. But, it actually works pretty well, including the :clean task. It needs to be DRYed out and abstracted out a bit. A full on docker-compose solution like what Temple has done with tul_cob is, for sure, more sophisticated. Oh, and we have a Procfile that calls this along with rails s and a command to run the webpacker server, so it's only one separate terminal window, which we like. Someone at the blacklight summit put us on Foreman, which we've found very useful.

[justinlittman]

Adding a +1 to this ticket. As newcomer to Blacklight, I show up understanding Docker (and Solr in Docker) but having to learn solr_wrapper.

[bess]

Approved by group consensus at blacklight-summit 2019

[cbeer]

It seems like we need to add some documentation about how to run the ci task locally? @jcoyne ?