Skip to content

[Feature] support Cassandra(Docker) as backend (compose with HugeGraph) #2274

@aroundabout

Description

@aroundabout

Feature Description (功能描述)

subtask of [Feature] Support build docker image from Dockerfile (Task Summary) #840

Key Problem:

  1. In order to init_store after the cassandra is up, the && ./bin/init-store.sh in Dockfile should be moved to start-hugegraph.sh.
  2. We should wait for the storage (cassandra) being up.
  3. In order to change the 'hugegraph.properties', we should pass environment variable according to docker-compose.yaml. And use an env_var_parser to parse them.

Main process:

  1. Move the init_store.sh from dockerfile to start-hugegraph.sh

  2. Use the env var to change the hugegraph.properties.
    Provide a shell which can process environment variables whose keys start with "hugegraph." and updates a property file accordingly.
    In this way, we can easily pass the env var to the properties, and if we want to support other backend, we need no additional modification in the code.
    Shell is like:

    while IFS=' ' read -r envvar_key envvar_val; do
        if [[ "${envvar_key}" =~ hugegraph\. ]] && [[ ! -z ${envvar_val} ]]; then
            envvar_key=${envvar_key#"hugegraph."}
            if grep -q -E "^\s*${envvar_key}\s*=\.*" ${GRAPH_PROP}; then
                sed -ri "s#^(\s*${envvar_key}\s*=).*#\\1${envvar_val}#" ${GRAPH_PROP}
            else
                echo "${envvar_key}=${envvar_val}" >> ${GRAPH_PROP}
            fi
        else
            continue
        fi
    done < <(env | sort -r | awk -F= '{ st = index($0, "="); print $1 " " substr($0, st+1) }')

    the part of the demo compose file:

    services:
    graph:
      build: /home/dandelion/incubator-hugegraph/
      # image: hugegraph/hugegraph:latest
      container_name: ca-graph
      ports:
        - 18080:8080
      environment:
        hugegraph.backend: cassandra
        hugegraph.serializer: cassandra
        hugegraph.cassandra.host: ca-cassandra
        hugegraph.cassandra.port: 9042
      networks:
        - ca-network
      depends_on:
        - cassandra
  3. Wait for storage backend initialization, then init_store

    demo shell like:

    if ! [ -z "${HUGE_STORAGE_TIMEOUT_S:-}" ]; then
        F="$(mktemp --suffix .groovy)"
        echo "graph = HugeFactory.open('${GRAPH_PROP}')" > $F
        timeout "${HUGE_STORAGE_TIMEOUT_S}s" bash -c \
        "until bin/gremlin-console.sh -e $F > /dev/null 2>&1; do echo \"waiting for storage...\"; sleep 5; done"
        rm -f "$F"
    fi
    "${BIN}/init-store.sh"

    use 'org.apache.tinkerpop.gremlin.groovy.jsr223.ScriptExecutor' in gremlin-console.sh to execute the graph = HugeFactory.open('${GRAPH_PROP}'). If success or overtime, exec init-store.sh

  4. demo docker-compose is like:

version: "3"

services:
  graph:
    build: /home/dandelion/incubator-hugegraph/
    # image: hugegraph/hugegraph:latest
    container_name: ca-graph
    ports:
      - 18080:8080
    environment:
      hugegraph.backend: cassandra
      hugegraph.serializer: cassandra
      hugegraph.cassandra.host: ca-cassandra
      hugegraph.cassandra.port: 9042
    networks:
      - ca-network
    depends_on:
      - cassandra

  cassandra:
    image: cassandra:3.11
    container_name: ca-cassandra
    ports:
      - 7000:7000
      - 9042:9042
    security_opt:
      - seccomp:unconfined
    networks:
      - ca-network
    healthcheck:
      test: ["CMD", "cqlsh", "--execute", "describe keyspaces;"]
      interval: 10s
      timeout: 30s
      retries: 5

networks:
  ca-network:

volumes:
 hugegraph-data:

Risk:

  1. I am not sure if I can move the init_store.sh from dockerfile to start-hugegraph.sh

Others

We can provide a link in dockerhub doc, which link to the docker-compose.yaml in the main repo. I think it is a better way for user to use docker-compose.
File structure is like:

./hugegraph-dist
├── docker
│       └── example
│               └── docker-compose-cassandra.yml
│               └── other-template-docker-compose.yml

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status

    ✅ Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions