grandstack with neo4j enterprise

[AdrienLemaire]

Hi,

I'm struggling to setup a local docker environment using a neo4j enterprise image.

A local environment without docker seems to work:

• neo4j-enterprise 4.0.0-1 running with systemctl start neo4j
• neo4j-browser AppImage that connects to neo4j@bolt://localhost:7687
• in /api/, run npm start, then from another pane npm run seedDb

{
  data: {
    u1: { id: 'u1', name: 'Will', __typename: 'User' },
    ...
  }
}

And I can query the data from the graphql playground.

A mixed solution with API in docker connecting to local systemd neo4j fails. That's because my API docker container's localhost is different from my host localhost, and cannot query it. Not sure how to fix it, but I'd rather not end up with this solution.

When I try to use your docker-compose.yml example with neo4j and API containers, and replace the neo4j image with enterprise (also needed to add curl that's removed from the enterprise image), I fail to start the local fleet:

files & errors

docker-compose.yml

# Lint with: docker-compose config
---
version: '3.8'  # 2020/04/28: Docker 19.03 & docker-comose 1.25.5

services:

  neo4j:
    build: ./neo4j_db
    ports:
      # use 49152~65535 range to avoid conflicts with host services
      - 57473:7473  # HTTPS
      - 57474:7474  # HTTP
      - 57687:7687  # BOLT
    environment:
      - NEO4J_dbms_security_procedures_unrestricted=apoc.*
      - NEO4J_apoc_import_file_enabled=true
      - NEO4J_apoc_export_file_enabled=true
      - NEO4J_dbms_shell_enabled=true
      - NEO4J_ACCEPT_LICENSE_AGREEMENT=yes
    #volumes:
      #- neo4j_db:/data
      #- neo4j_logs:/logs

  api:
    build: ./api_server
    ports:
      - 54001:4001
    environment:
      - NEO4J_URI=bolt://neo4j:57687
      - NEO4J_USER=neo4j
      - NEO4J_PASSWORD=custom
      - GRAPHQL_LISTEN_PORT=4001
      - GRAPHQL_URI=http://api:54001/graphql
    links:
      - neo4j
    depends_on:
      - neo4j

networks:
  default:

#volumes:
  #neo4j_db:
  #neo4j_logs:

neo4j/Dockerfile

FROM neo4j:3.5.17-enterprise

# Note: set NEO4J_USER=neo4j NEO4J_PASSWORD=custom in api_server/.env
ENV NEO4J_AUTH=neo4j/custom \
    APOC_VERSION=3.5.0.11 \
    GRAPHQL_VERSION=3.5.0.4 \
    APOC_URI=https://github.com/neo4j-contrib/neo4j-apoc-procedures/releases/download/${APOC_VERSION}/apoc-${APOC_VERSION}-all.jar \
    GRAPHQL_URI=https://github.com/neo4j-graphql/neo4j-graphql/releases/download/${GRAPHQL_VERSION}/neo4j-graphql-${GRAPHQL_VERSION}.jar

RUN apt update \
    && apt install -y curl \
    && sh -c 'cd /var/lib/neo4j/plugins && curl -L -O "${APOC_URI}"' \
    && sh -c 'cd /var/lib/neo4j/plugins && curl -L -O "${GRAPHQL_URI}"' \
    && apt-get -y purge --auto-remove curl

EXPOSE 7474 7473 7687

CMD ["neo4j"]

docker-compose up

Attaching to platform_v2_neo4j_1, platform_v2_api_1
api_1    |
api_1    | > project-api@2.0.1 start /app
api_1    | > npm run build && node build/index.js
api_1    |
neo4j_1  | Changed password for user 'neo4j'.
api_1    |
api_1    | > project-api@2.0.1 build /app
api_1    | > babel src -d build && shx cp .env build && shx cp src/schema.graphql build
api_1    |
neo4j_1  | Active database: graph.db
neo4j_1  | Directories in use:
neo4j_1  |   home:         /var/lib/neo4j
neo4j_1  |   config:       /var/lib/neo4j/conf
neo4j_1  |   logs:         /logs
neo4j_1  |   plugins:      /var/lib/neo4j/plugins
neo4j_1  |   import:       /var/lib/neo4j/import
neo4j_1  |   data:         /var/lib/neo4j/data
neo4j_1  |   certificates: /var/lib/neo4j/certificates
neo4j_1  |   run:          /var/lib/neo4j/run
neo4j_1  | Starting Neo4j.
api_1    | src/graphql-schema.js -> build/graphql-schema.js
api_1    | src/index.js -> build/index.js
api_1    | src/seed/seed-db.js -> build/seed/seed-db.js
api_1    | src/seed/seed-mutations.js -> build/seed/seed-mutations.js
api_1    | GraphQL server ready at http://localhost:4001/graphql
neo4j_1  | 2020-05-01 04:36:53.974+0000 WARN  Unknown config option: dbms.shell.enabled
neo4j_1  | 2020-05-01 04:36:53.994+0000 INFO  ======== Neo4j 3.5.17 ========
neo4j_1  | 2020-05-01 04:36:53.999+0000 INFO  Starting...
neo4j_1  | 2020-05-01 04:36:54.843+0000 INFO  Initiating metrics...
neo4j_1  | 2020-05-01 04:36:55.732+0000 ERROR Failed to start Neo4j: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase@64b7225f' was successfully initialized, but failed to start. Please see the attached cause exception "Some jar procedure files are invalid, see log for details.". Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase@64b7225f' was successfully initialized, but failed to start. Please see the attached cause exception "Some jar procedure files are invalid, see log for details.".
neo4j_1  | org.neo4j.server.ServerStartupException: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase@64b7225f' was successfully initialized, but failed to start. Please see the attached cause exception "Some jar procedure files are invalid, see log for details.".
neo4j_1  |      at org.neo4j.server.exception.ServerStartupErrors.translateToServerStartupError(ServerStartupErrors.java:45)
neo4j_1  |      at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:187)
neo4j_1  |      at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:124)
neo4j_1  |      at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:91)
neo4j_1  |      at com.neo4j.server.enterprise.CommercialEntryPoint.main(CommercialEntryPoint.java:22)
neo4j_1  | Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.server.database.LifecycleManagingDatabase@64b7225f' was successfully initialized, but failed to start. Please see the attached cause exception "Some jar procedure files are invalid, see log for details.".
neo4j_1  |      at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:473)
neo4j_1  |      at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
neo4j_1  |      at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:180)
neo4j_1  |      ... 3 more
neo4j_1  | Caused by: java.lang.RuntimeException: Error starting org.neo4j.graphdb.facade.GraphDatabaseFacadeFactory, /data/databases
neo4j_1  |      at org.neo4j.graphdb.facade.GraphDatabaseFacadeFactory.initFacade(GraphDatabaseFacadeFactory.java:232)
neo4j_1  |      at com.neo4j.commercial.edition.CommercialGraphDatabase.<init>(CommercialGraphDatabase.java:20)
neo4j_1  |      at com.neo4j.server.database.CommercialGraphFactory.newGraphDatabase(CommercialGraphFactory.java:40)
neo4j_1  |      at org.neo4j.server.database.LifecycleManagingDatabase.start(LifecycleManagingDatabase.java:90)
neo4j_1  |      at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:452)
neo4j_1  |      ... 5 more
neo4j_1  | Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.impl.proc.Procedures@af9a89f' was successfully initialized, but failed to start. Please see the attached cause exception "Some jar procedure files are invalid, see log for details.".
neo4j_1  |      at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:473)
neo4j_1  |      at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
neo4j_1  |      at org.neo4j.graphdb.facade.GraphDatabaseFacadeFactory.initFacade(GraphDatabaseFacadeFactory.java:227)
neo4j_1  |      ... 9 more
neo4j_1  | Caused by: java.util.zip.ZipException: Some jar procedure files are invalid, see log for details.
neo4j_1  |      at org.neo4j.kernel.impl.proc.ProcedureJarLoader.loadProceduresFromDir(ProcedureJarLoader.java:76)
neo4j_1  |      at org.neo4j.kernel.impl.proc.Procedures.start(Procedures.java:323)
neo4j_1  |      at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:452)
neo4j_1  |      ... 11 more
neo4j_1  | 2020-05-01 04:36:55.732+0000 INFO  Neo4j Server shutdown initiated by request
platform_v2_neo4j_1 exited with code 1

The java traceback error makes very little sense to me. I first thought it was due to file permissions when using volumes, but commenting that part didn't change the error.
Any idea how should I debug this?

[AdrienLemaire]

@johnymontana it would be great to have the grand starter working with v4 under docker-compose :)

[BrenoMazieiro]

@AdrienLemaire
Like this? https://github.com/BrenoMazieiro/grand-stack-starter
=)

[AdrienLemaire]

@BrenoMazieiro thanks for sharing.
It looks like your fork is using neo4j community, not the enterprise version.

I'd also very much like to understand how to debug these neo4j java traceback errors, and keep my setup as simple as possible for the moment (no need for a proxy).

Might you know what these 'Some jar procedure files are invalid' errors I shared above could mean?
Thanks for your help!

[cellcoresystems]

@AdrienLemaire the errors you have are coming from the wrong version of the apoc procedures.
The apoc library is bond to the neo4j db version, if you use a wrong one, then you get that error.

Always take look -->here howto install apoc.

If you want to run Neo4j v4.0.4 then you can use this here:

FROM neo4j:4.0.4-enterprise

ENV NEO4J_AUTH=neo4j/letmein \
    APOC_VERSION=4.0.0.12 
    ##not needed in v 4.0 because it is integrated now
    #GRAPHQL_VERSION=3.5.0.4

ENV APOC_URI=https://github.com/neo4j-contrib/neo4j-apoc-procedures/releases/download/${APOC_VERSION}/apoc-${APOC_VERSION}-all.jar
##curl is removed during the build of neo4j v4.0, I guess because of its size and dependencies
#RUN sh -c 'cd /var/lib/neo4j/plugins && curl -L -O "${APOC_URI}"'
RUN sh -c 'wget -P /var/lib/neo4j/plugins/ "${APOC_URI}"'

##not needed in v 4.0 because it is integrated now
#ENV GRAPHQL_URI https://github.com/neo4j-graphql/neo4j-graphql/releases/download/${GRAPHQL_VERSION}/neo4j-graphql-${GRAPHQL_VERSION}.jar
#RUN sh -c 'cd /var/lib/neo4j/plugins && curl -L -O "${GRAPHQL_URI}"'

EXPOSE 7474 7473 7687

CMD ["neo4j"]

But be careful with GRANDstack and Neo4j v4.0, not everything is tested, I guess thats why William did not change it until now.

Hope it helps ;o)

[AdrienLemaire]

Sorry, forgot to close the issue. Thanks again for the resolution !