bug: Apisix Config File Overwritten in Standalone Mode

[sashokbg]

Current Behavior

Following the tutorial:

https://apisix.apache.org/docs/apisix/stand-alone/ and using https://github.com/apache/apisix-docker

I set mu apisix server in standalone mode

apisix:
image: apache/apisix:latest
restart: always
environment:
APISIX_STAND_ALONE: true
volumes:
- error_vol:/usr/local/apisix/logs
- ./apisix_conf/config.yaml:/usr/local/apisix/conf/config.yaml
depends_on:
- etcd
ports:
- "9180:9180/tcp"
- "9080:9080/tcp"
- "9091:9091/tcp"
- "9443:9443/tcp"
- "9092:9092/tcp"
network_mode: host

And when I start the server the config.yaml file gets overwritten and all my routes / upstreams disappear.

Expected Behavior

When Starting the docker compose with a volume - ./apisix_conf/config.yaml:/usr/local/apisix/conf/config.yaml then the config.yaml should be read into memory as described in the documentation

Error Logs

No response

Steps to Reproduce

1. Use the docker compose example - https://github.com/apache/apisix-docker/tree/master/example
2. Set APISIX_STAND_ALONE: true
3. Put some routes in apisix_conf/config.yaml
4. Run docker-compose up
5. Check the dashboard - no routes imported and file has been overwritten

PS: if you don't remove the :ro flag from the volume mount the server does not start at all

Environment

• APISIX version: 3.1.0
• Operating system: Linux alexander-p14s 6.1.1-arch1-1 change: added doc of how to load plugin. apisix#1 SMP PREEMPT_DYNAMIC Wed, 21 Dec 2022 22:27:55 +0000 x86_64 GNU/Linux

[sashokbg]

It seems there is some log generated at startup:

apache-apisix | 2023/01/05 13:10:40 [error] 38#38: 7 [lua] config_yaml.lua:210: failed to check item data of [routes] err:allOf 1 failed: value should match only one schema, but matches none ,val: {"priority":0,"status":1}, context: init_worker_by_lua

For some reason the lua script thinks the config is bad and ends up overwriting the file.

[juzhiyuan]

Hello, I haven't tried using Standalone mode in Docker, but as far as I know, if using Standalone mode, we should put Routes/Upstreams/... in apisix.yaml file instead of config.yaml. 🤔

Can you please check this doc? https://apisix.apache.org/docs/apisix/stand-alone/

For the overridden issue, other maintainers will reply when available:)

[soulbird]

Can you show your config.yaml ？

[sashokbg]

Hello @soulbird yes:

deployment:
  role: data_plane
  role_data_plane:
    config_provider: yaml

routes:
  -
    uri: /hello
    upstream:
      nodes:
        "127.0.0.1:1980": 1
      type: roundrobin
#END

UPDATE: It seems sometimes the routes are taken into account, but are not visible from the dashboard. Does the dashboard stop working when we are in standalone mode ?

[soulbird]

Yes, when using standalone mode, you will not be able to use the dashboard and admin api

[sashokbg]

@soulbird Thank you for your answer.
So if I am in "normal" mode the only way to import my Routes and Upstreams is to do curl commands ?

[soulbird]

@soulbird Thank you for your answer. So if I am in "normal" mode the only way to import my Routes and Upstreams is to do curl commands ?

dashboard can also

[sashokbg]

No way to do it in "infrastructure as code" fashion ?

[soulbird]

Start apisix in shandalone mode, then write your routing configuration in apisix.yaml, refer to here: https://github.com/apache/apisix-docker#how-to-run-apisix-in-stand-alone- mode

[sashokbg]

@soulbird this is the problem - I don't understand why we have to write to the file after the server is started and why this file gets overwritten at startup.

[juzhiyuan]

I noticed 2 unexpected things:

1. According to Standalone's documentation, it only mentions that in Standalone mode, all configurations should be put in conf/apisix.yaml. But @sashokbg mentioned it's ok to put in config.yaml. @soulbird Can you confirm is it right?

2. When APISIX starts, config.yaml should not be overwritten in my opinion. @soulbird Can you also check this point?

---

As for why should put configurations in apisix.yaml after APISIX starts, the reason is: instead of using etcd to store configurations, Standalone mode uses apisix.yaml file to store them. And APISIX checks this file's changes every 1 second by default, I think that's why @soulbird replied like that :)

[soulbird]

When APISIX_STAND_ALONE is set to true, a minimalist config.yaml will be generated, which will overwrite the config.yaml you are hanging on. This is indeed a bug, and we will fix it later.
As a solution, you can enable standalone mode in your own config.yaml, write routing information into apisix.yaml, and then map these two files to the container.

config.yaml

.....

deployment:
  role: data_plane
  role_data_plane:
    config_provider: yaml

.....

apisix.yaml

routes:
  -
    uri: /hello
    upstream:
        nodes:
            "127.0.0.1:1980": 1
        type: roundrobin
#END

docker-compose

  apisix:
    image: apache/apisix:latest
    restart: always
    volumes:
      - ./apisix_log:/usr/local/apisix/logs
      - ./apisix_conf/config.yaml:/usr/local/apisix/conf/config.yaml:ro
      - ./apisix_conf/apisix.yaml:/usr/local/apisix/conf/apisix.yaml:ro
    depends_on:
      - etcd
    ##network_mode: host
    ports:
      - "9180:9180/tcp"
      - "9080:9080/tcp"
      - "9091:9091/tcp"
      - "9443:9443/tcp"
      - "9092:9092/tcp"
    networks:
      apisix:

[sashokbg]

Hello @soulbird thank you for the clarifications.

Unfortunately the workaround with "ro" mode does not work since it makes the container crash with the bellow error message.

apache-apisix | /docker-entrypoint.sh: line 25: /usr/local/apisix/conf/config.yaml: Read-only file system

Can you point to me the place where the config is overwritten I might find some time to throw a quick pull request if the task is simple enough for me :)

[soulbird]

According to my method above, you should no longer set the APISIX_STAND_ALONE environment variable.

They are covered here, https://github.com/apache/apisix-docker/blob/master/debian/docker-entrypoint.sh#L23-L36
You can think if you have an idea to solve this problem