Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

4762 solr scaling #4924

Merged
merged 4 commits into from
Aug 16, 2018
Merged

4762 solr scaling #4924

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ce3326f
# This is a combination of 5 commits.
thaorell Jul 19, 2018
067ac57
edit code for @danmcp's review
thaorell Aug 13, 2018
f92e14b
fixed some typos
thaorell Aug 13, 2018
d538fac
fix for @pdurbin feedback
thaorell Aug 14, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 4 additions & 2 deletions conf/docker/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,10 @@ echo Images are being built for registry org/username \"$HUBORG\" with the tag \
#
# Build dataverse-solr
#
# Use "conf" directory as context so we can copy schema.xml into Solr image.
docker build -t $HUBORG/dataverse-solr:$TAG -f solr/Dockerfile ../../conf
cp ../solr/7.3.0/schema.xml solr/
# move solr*.tgz to the solr image
cp ../../downloads/solr-7.3.0.tgz solr/
docker build -t $HUBORG/dataverse-solr:$TAG solr/
if [ "$1" == 'internal' ]; then
echo "Skipping docker push because we're using the internal Minishift registry."
else
Expand Down
2 changes: 2 additions & 0 deletions conf/docker/solr/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
solr-7.3.0.tgz
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this intended? I noticed below that we are no longer downloading solr with wget so should it be included?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if somebody wants to build a solr image, they will still need to download solr then paste it in conf/docker/solr

schema.xml
26 changes: 19 additions & 7 deletions conf/docker/solr/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,19 +1,31 @@
FROM centos:7.2.1511
FROM centos:7
MAINTAINER Dataverse (support@dataverse.org)

RUN yum install -y wget unzip perl git java-1.8.0-openjdk-devel postgresql.x86_64
RUN yum install -y unzip java-1.8.0-openjdk-devel lsof

# Install Solr 7.3.0
# The context of the build is the "conf" directory.
COPY solr/7.3.0/schema.xml /tmp
COPY solr/7.3.0/solrconfig.xml /tmp
COPY solr-7.3.0.tgz /tmp
RUN cd /tmp \
&& tar xvfz solr-7.3.0.tgz \
&& rm solr-7.3.0.tgz \
&& mkdir /usr/local/solr \
&& mv solr-7.3.0 /usr/local/solr/

RUN cd /tmp && wget https://archive.apache.org/dist/lucene/solr/7.3.0/solr-7.3.0.tgz
COPY schema.xml /tmp
COPY solrconfig_master.xml /tmp
COPY solrconfig_slave.xml /tmp

RUN chmod g=u /etc/passwd

RUN chgrp -R 0 /usr/local/solr && \
chmod -R g=u /usr/local/solr

EXPOSE 8983

COPY docker/solr/Dockerfile /Dockerfile
COPY docker/solr/entrypoint.sh /
COPY Dockerfile /
COPY entrypoint.sh /

ENTRYPOINT ["/entrypoint.sh"]
USER 1001
CMD ["solr"]
1 change: 1 addition & 0 deletions conf/docker/solr/backup_cron.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
0 */6 * * * curl 'http://localhost:8983/solr/collection1/replication?command=backup&location=/home/share'
29 changes: 23 additions & 6 deletions conf/docker/solr/entrypoint.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,18 +1,35 @@
#!/bin/bash

# FIXME: Don't run Solr out of /tmp!
# Solr is /tmp to avoid AccessDeniedException under Minishift/OpenShift.
SOLR_DIR=/tmp/solr-7.3.0
if ! whoami &> /dev/null; then
if [ -w /etc/passwd ]; then
echo "${USER_NAME:-default}:x:$(id -u):0:${USER_NAME:-default} user:${HOME}:/sbin/nologin" >> /etc/passwd
fi
fi

SOLR_DIR=/usr/local/solr/solr-7.3.0

if [ "$1" = 'solr' ]; then
cd /tmp
tar xvfz solr-7.3.0.tgz

cp -r $SOLR_DIR/server/solr/configsets/_default $SOLR_DIR/server/solr/collection1
cp /tmp/schema.xml $SOLR_DIR/server/solr/collection1/conf
cp /tmp/solrconfig.xml $SOLR_DIR/server/solr/collection1/conf

if [ $HOSTNAME = "dataverse-solr-0" ]; then
echo "I am the master"
mv /tmp/solrconfig_master.xml $SOLR_DIR/server/solr/collection1/conf/solrconfig.xml
cp /tmp/solrconfig_slave.xml $SOLR_DIR/server/solr/collection1/conf

else
echo "I am the slave"
cp /tmp/solrconfig_slave.xml $SOLR_DIR/server/solr/collection1/conf
mv $SOLR_DIR/server/solr/collection1/conf/solrconfig_slave.xml $SOLR_DIR/server/solr/collection1/conf/solrconfig.xml
fi
cd $SOLR_DIR
bin/solr start
bin/solr create_core -c collection1 -d server/solr/collection1/conf
if [ $HOSTNAME = "dataverse-solr-0" ]; then
curl 'http://localhost:8983/solr/collection1/replication?command=restore&location=/home/share'
fi

sleep infinity
elif [ "$1" = 'usage' ]; then
echo 'docker run -d iqss/dataverse-solr solr'
Expand Down