Solr core not automatically created

[hanoii]

I wanted to have, for now, just a solr service on a drupal project, not going full lando yet.

I have the following .yml file:

name: moog-drupal

proxy:
  # Lets get a nice lndo url for the solr web interface
  solrsearch2:
    - moog-solr.lndo.site:8983

# Spin up services to run a basic LAMP stack
services:
  solrsearch2:
    type: solr:6.6
    portforward: true
    core: drupal
    config:
      conf: .platform/solr-conf

For some reason the core is not created or found automatically.

Solr is there without core. I can see the configs on /opt/solr/server/solr/mycores/drupal, but the core is not there.

When I went to try to add the core manually, providing mycores/drupal as the instanceDir, it complained that it didn't have access to /opt/solr/server/solr/mycores/drupal/core.properties.

I ssh to the service and saw that /opt/solr/server/solr/mycores/drupal was owned by root:root. I then chowned it to solr:solr and added the core and that worked, but this surely have been automatically.

Using lando master.

[heshanlk]

I have the same issue, the Solr core would not create automatically. This was working before in beta.16

[heshanlk]

After couple of different config attempts following configs appeared to be working fine for me for search api solr with Solr 5.5.

The Core name should be the same as the name you put in top of your .lando.yml file.

  solr:
    type: solr:5.5
    portforward: true
    core: [LANDOENVNAME]
    config:
      conf: docroot/modules/contrib/search_api_solr/solr-conf/5.x

[heshanlk]

I was wrong, something this is working sometimes not so we need to fix for this.

[hanoii]

@heshanlk you might want to check my pull request/fork and report back.

[dmsmidt]

The PR of @hanoii fixes the problem.
This is currently keeping a complete company (20+ devs) from using Lando.

[jbertoen]

I talked a lot in slack, a lot to myself as well, viva la rubberducking. @Mirnaxvb and @MikeWong1991 know what I mean.

The problem is the following:

• /solrconf is mapped to the conf dir as a volume by setting the conf value in the .lando.yml file
• /opt/solr/server/solr/mycores/CORENAME/conf is also mapped as a volume as a volume by setting the conf value in the .lando.yml file

But the thing is, when creating your solr service from scratch, without preexisting volumes or containers. The core needs to be created. Which would mostly mean that within /opt/solr/server/solr/mycores/CORENAME 2 things need to be added:

• core.properties
• data

But because the conf directory within the core is mapped BEFORE executing this command. /opt/solr/server/solr/mycores/CORENAME is not accessible by solr (owned by root), which would prevent the data folder to be created when solr is started. But also the solr precreate scripts can't be run because it thinks the core is already created, which would add the core.properties.

Allowing this PR to go through would arise the problem that the core schema and such will not be synced without recreating the whole container and it's volumes. Becuase we need the precreate to run to make sure everything is copied and created correctly.

We got a view options to fix this:

1. Allow this PR and accept the fact that we need to rebuild an entire container and it's volumes when the schema changes.
2. Allow this PR and add a script that would copy the /solrconf folder to /opt/solr/server/solr/mycores/CORENAME/conf if the core already exists. And run it before the precreate script. https://github.com/docker-solr/docker-solr#extending-the-image has an explanation about adding scripts to extend solr.
3. Reject this PR and also somehow volume map the core.properties file and set the ownership to solr:solr on the core. Which would mean to map the entire core as a volume instead of only the conf directory within the core. Or individually map core.properties and conf within the core as a 2 volumes and execute a chown command to set the ownership to the solr user.

My preference would be to go for 1 or 2. Because solr schema's do not change that much.

1. would be the quickfix, at least to make solr working again for a lot of people.
2. would be the most desired way, it would make sure that a restart and a rebuild would suffice to sync the configuration to the core. Keeping volumes and containers intact.

[pirog]

@jbertoen this write up is INCREDIBLE! this saves me tons of time.

I need to try this out but there might be a an easy option 4 which is to just make sure the solr core conf directory is owned by solr. Will report back to you all soon. thanks for bearing with us on this on.

[pirog]

ok yeah i can definitely confirm the two major items here:

1. This PR (Do not map to the conf folder within the named volume #553) does create the core successfully
2. The core config can only be changed with a destroy/start

[pirog]

ok, so to handle 2. we are using lando to destroy the core conf directory on every start and then recreate it as a symlink to the /solrconf/conf mounted volume. This should resolve the issue and allow people to change their solr conf ON THE FLY (eg without a restart).

im going to close this and it should be available in the beta.32 i plan on rolling tonight.

[jbertoen]

The solr-precreate calls the precreate-core script. This checks if the directory of the corename exists, if so cancel the precreate.

So this would not work if we already mapped a volume to the corename/conf directory. The script would think the core is already created. Thus not creating the core.properties in the corename directory which is needed to load the core. So fixing file rights would not be enough.

Maybe creating a symlink of the conf folder after running the solr-precreate command?
ln -sf /solrconf /opt/solr/server/solr/mycores/CORENAME/conf

[pirog]

yup! see: c4925bb setting up the symlink is exactly what i ended up doing. in basic testing it seems to satisfy the two big things we wanted to do.

[jbertoen]

Awesome!

[hanoii]

I was away for a few weeks but I just tried master and again the solr core is not precreated. I will follow this up better but just wanted to not. Is there something I need to do? I destroyed/recreated everything.

EDIT: My bad, I updated my own fork's master.

[pirog]

Share the lando file that you have that doesn’t work. I tried this out on a few different versions and it seemed to work as expected

…

On Fri, Jan 12, 2018 at 9:22 AM Ariel Barreiro ***@***.***> wrote: I was away for a few weeks but I just tried master and again the solr core is not precreated. I will follow this up better but just wanted to not. Is there something I need to do? I destroyed/recreated everything. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#551 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAri0OBt65cPaOqX_q08CgVr_d4eewiuks5tJ5ISgaJpZM4Qt_fl> .

[hanoii]

@pirog I was wrongly pulling my fork's master, tried it now with the latest lando and it does work. I edited my post soon after when I found this but you probably didn't receive the email notification of the edit. Thanks!