Nginx Unit --state saving multiple json configs on service restarts #48

centminmod · 2017-09-24T01:19:55Z

I source compiled Nginx Unit on CentOS 7.4 64bit using configure options

./configure --prefix=/opt/unit --pid=/run/unitd.pid --log=/var/log/unitd.log --modules=modules --user=nginx --group=nginx --state=state

I assumed the --state option would save all loaded configurations as when I tested with just a single /root/tools/unitconfigs/php5631start.json loaded, a unitd service restart reloaded the config ok. But when I loaded more php configurations listed below, on unitd service restart, only the last loaded config was restored for /root/tools/unitconfigs/php720start.json ?

5.6.31

/root/tools/unitconfigs/php5631start.json

{
     "listeners": {
         "*:8300": {
             "application": "php56domaincom"
         }
     },
     "applications": {
         "php56domaincom": {
             "type": "php 5.6.31",
              "workers": 20,
              "root": "/home/nginx/domains/domain.com/public",
              "user": "nginx",
              "group": "nginx",
              "index": "index.php"
         }
     }
}

curl -X PUT -d @/root/tools/unitconfigs/php5631start.json --unix-socket /opt/unit/control.unit.sock http://localhost/

curl -X PUT -d @/root/tools/unitconfigs/php5631start.json --unix-socket /opt/unit/control.unit.sock http://localhost/
{
        "success": "Reconfiguration done."
}

7.0.24

/root/tools/unitconfigs/php7024start.json

{
     "listeners": {
         "*:8400": {
             "application": "php70domaincom"
         }
     },
     "applications": {
         "php70domaincom": {
             "type": "php 7.0.24",
              "workers": 20,
              "root": "/home/nginx/domains/domain.com/public",
              "user": "nginx",
              "group": "nginx",
              "index": "index.php"
         }
     }
}

curl -X PUT -d @/root/tools/unitconfigs/php7024start.json --unix-socket /opt/unit/control.unit.sock http://localhost/

curl -X PUT -d @/root/tools/unitconfigs/php7024start.json --unix-socket /opt/unit/control.unit.sock http://localhost/
{
        "success": "Reconfiguration done."
}

7.1.10

/root/tools/unitconfigs/php7110start.json

{
     "listeners": {
         "*:8500": {
             "application": "php71domaincom"
         }
     },
     "applications": {
         "php71domaincom": {
             "type": "php 7.1.10",
              "workers": 20,
              "root": "/home/nginx/domains/domain.com/public",
              "user": "nginx",
              "group": "nginx",
              "index": "index.php"
         }
     }
}

curl -X PUT -d @/root/tools/unitconfigs/php7110start.json --unix-socket /opt/unit/control.unit.sock http://localhost/

{
        "success": "Reconfiguration done."
}

7.2.0

/root/tools/unitconfigs/php720start.json

{
     "listeners": {
         "*:8600": {
             "application": "php72domaincom"
         }
     },
     "applications": {
         "php72domaincom": {
             "type": "php 7.2.0",
              "workers": 20,
              "root": "/home/nginx/domains/domain.com/public",
              "user": "nginx",
              "group": "nginx",
              "index": "index.php"
         }
     }
}

curl -X PUT -d @/root/tools/unitconfigs/php720start.json --unix-socket /opt/unit/control.unit.sock http://localhost/

curl -X PUT -d @/root/tools/unitconfigs/php720start.json --unix-socket /opt/unit/control.unit.sock http://localhost/
{
        "success": "Reconfiguration done."
}

but only the last config was restored on unitd service restart

curl --unix-socket /opt/unit/control.unit.sock http://localhost/
{
        "listeners": {
                "*:8600": {
                        "application": "php72domaincom"
                }
        },

        "applications": {
                "php72domaincom": {
                        "type": "php 7.2.0",
                        "workers": 20,
                        "root": "/home/nginx/domains/domain.com/public",
                        "user": "nginx",
                        "group": "nginx",
                        "index": "index.php"
                }
        }
}

The text was updated successfully, but these errors were encountered:

VBart · 2017-09-24T01:29:43Z

This state directory isn't a configuration storage. It was introduced just for runtime configuration persistence in order to allow Unit to survive server reloads without additional tricks. It's up to you where to store and how to manage your configurations.

centminmod · 2017-09-24T01:32:59Z

i see but even if i do not restart unitd service, i can only list the last added php720start.json configuration ? the previous 3 configs are gone ?

curl --unix-socket /opt/unit/control.unit.sock http://localhost/                                                     
{
        "listeners": {
                "*:8600": {
                        "application": "php72domaincom"
                }
        },

        "applications": {
                "php72domaincom": {
                        "type": "php 7.2.0",
                        "workers": 20,
                        "root": "/home/nginx/domains/domain.com/public",
                        "user": "nginx",
                        "group": "nginx",
                        "index": "index.php"
                }
        }
}

unitd.log

2017/09/24 01:04:42 [info] 18356#18356 discovery started
2017/09/24 01:04:42 [notice] 18356#18356 module: php 5.6.31 "/opt/unit/modules/php5631.unit.so"
2017/09/24 01:04:42 [notice] 18356#18356 module: python 2.7.5 "/opt/unit/modules/python.unit.so"
2017/09/24 01:04:42 [notice] 18356#18356 module: php 5.6.31 "/opt/unit/modules/remiphp5631.unit.so"
2017/09/24 01:04:42 [notice] 18356#18356 ignoring /opt/unit/modules/remiphp5631.unit.so module with the same application language version php 5.6.31 as in /opt/unit/modules/php5631.unit.so
2017/09/24 01:04:42 [notice] 18356#18356 module: php 7.0.24RC1 "/opt/unit/modules/remiphp7023.unit.so"
2017/09/24 01:04:42 [notice] 18356#18356 module: php 7.1.10RC1 "/opt/unit/modules/remiphp719.unit.so"
2017/09/24 01:04:42 [notice] 18356#18356 module: php 7.2.0RC2 "/opt/unit/modules/remiphp720.unit.so"
2017/09/24 01:04:42 [info] 18357#18357 controller started
2017/09/24 01:04:42 [info] 18358#18358 router started
2017/09/24 01:04:42 [crit] 18358#18358 epoll_ctl(3, 3, 9) failed (2: No such file or directory)
2017/09/24 01:04:42 [crit] 18357#18357 epoll_ctl(3, 3, 9) failed (2: No such file or directory)
2017/09/24 01:04:42 [notice] 18355#18355 process 18356 exited with code 0

VBart · 2017-09-24T01:35:10Z

They are gone not because of the reload. When you do PUT command to the root of the configuration / it's replaced by the provided object. The PUT has create or replace semantic.

centminmod · 2017-09-24T01:42:30Z

hmm the docs at isn't exactly clear on this http://unit.nginx.org/docs-configuration.html#creating-configuration-objects we have to separately add the listener and applications from separate json files ?

VBart · 2017-09-24T01:43:03Z

There are 3 methods supported at the moment: GET, PUT, DELETE. You can operate over the configuration root / as well as any sublayers, like /listeners, /listeners/IP:PORT, /listeners/IP:PORT/application, /applications, /applications/NAME, /applications/NAME/workers, etc...

VBart · 2017-09-24T01:50:20Z

You can add an application and then a listener, thus with two requests. For now, there's only one way to add both a listener and an application with just one request: by uploading the full resulting configuration to the root /.

Support for the MERGE/PATCH methods in conformity with RFC 7396 is planned.

VBart · 2017-09-24T02:00:25Z

I also can suggest a nice and very powerful utility for operating with JSON:

% cat conf.json                     
{
        "listeners": {
                "127.0.0.1:8081": {
                        "application": "test"
                }
        },

        "applications": {
                "test": {
                        "type": "python",
                        "module": "wsgi",
                        "path": "/home/vbart/Development/tests"
                }
        }
}
% <conf.json | jq .applications.test
{
  "type": "python",
  "module": "wsgi",
  "path": "/home/vbart/Development/tests"
}
% <conf.json | jq '.listeners[]' 
{
  "application": "test"
}

https://stedolan.github.io/jq/

VBart · 2017-09-24T02:08:32Z

And you can merge JSON files with it:

% cat conf.json
{
        "listeners": {
                "127.0.0.1:8081": {
                        "application": "test"
                }
        },

        "applications": {
                "test": {
                        "type": "python",
                        "module": "wsgi",
                        "path": "/home/vbart/Development/tests"
                }
        }
}
% cat conf_t.json
{
     "listeners": {
         "*:8300": {
             "application": "php56domaincom"
         }
     },
     "applications": {
         "php56domaincom": {
             "type": "php",
              "workers": 10,
              "root": "/home/nginx/domains/domain.com/public",
              "user": "vbart",
              "group": "users",
              "index": "index.php"
         }
     }
}
% jq -s '.[0] * .[1]' conf.json conf_t.json
{
  "listeners": {
    "127.0.0.1:8081": {
      "application": "test"
    },
    "*:8300": {
      "application": "php56domaincom"
    }
  },
  "applications": {
    "test": {
      "type": "python",
      "module": "wsgi",
      "path": "/home/vbart/Development/tests"
    },
    "php56domaincom": {
      "type": "php",
      "workers": 10,
      "root": "/home/nginx/domains/domain.com/public",
      "user": "vbart",
      "group": "users",
      "index": "index.php"
    }
  }
}

centminmod · 2017-09-24T02:10:24Z

cheers but don't we need to define the actual listener *:8300 too ?

cat /root/tools/unitconfigs/php5631start.json 
{
     "listeners": {
         "*:8300": {
             "application": "php56domaincom"
         }
     },
     "applications": {
         "php56domaincom": {
             "type": "php 5.6.31",
              "workers": 20,
              "root": "/home/nginx/domains/domain.com/public",
              "user": "nginx",
              "group": "nginx",
              "index": "index.php"
         }
     }
}

if contents of /root/tools/unitconfigs/php5631start-app.json is

cat /root/tools/unitconfigs/php5631start.json | jq .applications.php56domaincom
{
  "type": "php 5.6.31",
  "workers": 20,
  "root": "/home/nginx/domains/domain.com/public",
  "user": "nginx",
  "group": "nginx",
  "index": "index.php"
}

for

curl -X PUT -d @/root/tools/unitconfigs/php5631start-app.json --unix-socket /opt/unit/control.unit.sock http://localhost/applications/php56domaincom

and for /root/tools/unitconfigs/php5631start-listener.json contents would be

cat /root/tools/unitconfigs/php5631start.json | jq '.listeners[]'
{
  "application": "php56domaincom"
}

for

curl -X PUT -d @/root/tools/unitconfigs/php5631start-listener.json --unix-socket /opt/unit/control.unit.sock http://localhost/listeners

??

VBart · 2017-09-24T02:12:54Z

Yes, you have to define it in URI: /listeners/*:8300

centminmod · 2017-09-24T02:14:08Z

cheers so

curl -X PUT -d @/root/tools/unitconfigs/php5631start-listener.json --unix-socket /opt/unit/control.unit.sock http://localhost/listeners/*:8300

lots to digest.. thanks

Hope this info makes it into official documentation manual too :)

VBart · 2017-09-24T02:18:31Z

Merging with jq also works fine:

% jq -s '.[0] * .[1]' conf.json conf_t.json | curl -X PUT -d@-  http://127.0.0.1:8443
{
        "success": "Reconfiguration done."
}
% curl http://127.0.0.1:8443
{
        "listeners": {
                "127.0.0.1:8081": {
                        "application": "test"
                },

                "*:8300": {
                        "application": "php56domaincom"
                }
        },

        "applications": {
                "test": {
                        "type": "python",
                        "module": "wsgi",
                        "path": "/home/vbart/Development/tests"
                },

                "php56domaincom": {
                        "type": "php",
                        "workers": 10,
                        "root": "/home/nginx/domains/domain.com/public",
                        "user": "vbart",
                        "group": "users",
                        "index": "index.php"
                }
        }
}

centminmod · 2017-09-24T02:45:45Z

@VBart thanks merging seems easier to do as I can script it to read all .json files within a config directory for merging :)

jq -s '.[0] * .[1] * .[2] * .[3] * .[4]' php5631start.json php7024start.json php719start.json php720start.json php7110start.json 
{
  "listeners": {
    "*:8300": {
      "application": "php56domaincom"
    },
    "*:8400": {
      "application": "php71domaincom"
    },
    "*:8600": {
      "application": "php72domaincom"
    },
    "*:8500": {
      "application": "php71domaincom"
    }
  },
  "applications": {
    "php56domaincom": {
      "type": "php 7.1.9",
      "workers": 20,
      "root": "/home/nginx/domains/domain.com/public",
      "user": "nginx",
      "group": "nginx",
      "index": "index.php"
    },
    "php70domaincom": {
      "type": "php 7.0.24",
      "workers": 20,
      "root": "/home/nginx/domains/domain.com/public",
      "user": "nginx",
      "group": "nginx",
      "index": "index.php"
    },
    "php72domaincom": {
      "type": "php 7.2.0",
      "workers": 20,
      "root": "/home/nginx/domains/domain.com/public",
      "user": "nginx",
      "group": "nginx",
      "index": "index.php"
    },
    "php71domaincom": {
      "type": "php 7.1.10",
      "workers": 20,
      "root": "/home/nginx/domains/domain.com/public",
      "user": "nginx",
      "group": "nginx",
      "index": "index.php"
    }
  }
}

though getting 405 not allowed error

 jq -s '.[0] * .[1] * .[2] * .[3] * .[4]' php5631start.json php7024start.json php719start.json php720start.json php7110start.json  | curl -X PUT -d@-  http://localhost
<html>
<head><title>405 Not Allowed</title></head>
<body bgcolor="white">
<center><h1>405 Not Allowed</h1></center>
<hr><center>nginx</center>
</body>
</html>

VBart · 2017-09-24T03:08:57Z

That error comes from nginx, not unit. =)

Note, that I used curl command without unix socket, because I run my Unit with control socket set to 127.0.0.1:8443.

centminmod · 2017-09-24T04:09:31Z

thanks for the hint added the control socket back

jq -s '.[0] * .[1] * .[2] * .[3] * .[4]' php5631start.json php7024start.json php719start.json php720start.json php7110start.json | curl -X PUT -d@- --unix-socket /opt/unit/control.unit.sock http://localhost/ 
{
        "error": "Invalid configuration."
}

jq -s '.[0] * .[1] * .[2] * .[3] * .[4]' php5631start.json php7024start.json php719start.json php720start.json php7110start.json
{
  "listeners": {
    "*:8300": {
      "application": "php56domaincom"
    },
    "*:8400": {
      "application": "php71domaincom"
    },
    "*:8600": {
      "application": "php72domaincom"
    },
    "*:8500": {
      "application": "php71domaincom"
    }
  },
  "applications": {
    "php56domaincom": {
      "type": "php 7.1.9",
      "workers": 20,
      "root": "/home/nginx/domains/domain.com/public",
      "user": "nginx",
      "group": "nginx",
      "index": "index.php"
    },
    "php70domaincom": {
      "type": "php 7.0.24",
      "workers": 20,
      "root": "/home/nginx/domains/domain.com/public",
      "user": "nginx",
      "group": "nginx",
      "index": "index.php"
    },
    "php72domaincom": {
      "type": "php 7.2.0",
      "workers": 20,
      "root": "/home/nginx/domains/domain.com/public",
      "user": "nginx",
      "group": "nginx",
      "index": "index.php"
    },
    "php71domaincom": {
      "type": "php 7.1.10",
      "workers": 20,
      "root": "/home/nginx/domains/domain.com/public",
      "user": "nginx",
      "group": "nginx",
      "index": "index.php"
    }
  }
}

VBart · 2017-09-24T09:04:42Z

If you have the modules collection that you mentioned in #47, then php 7.1.9 is missing.

centminmod · 2017-09-24T09:33:21Z

ah ha thanks @VBart needed that second set of eyes :)

jq -s '.[0] * .[1] * .[2] * .[3]' php5631start.json php7024start.json php720start.json php7110start.json  | curl -X PUT -d@- --unix-socket /opt/unit/control.unit.sock http://localhost
{
        "success": "Reconfiguration done."
}

curl --unix-socket /opt/unit/control.unit.sock http://localhost/
{
        "listeners": {
                "*:8300": {
                        "application": "php56domaincom"
                },

                "*:8400": {
                        "application": "php70domaincom"
                },

                "*:8600": {
                        "application": "php72domaincom"
                },

                "*:8500": {
                        "application": "php71domaincom"
                }
        },

        "applications": {
                "php56domaincom": {
                        "type": "php 5.6.31",
                        "workers": 20,
                        "root": "/home/nginx/domains/domain.com/public",
                        "user": "nginx",
                        "group": "nginx",
                        "index": "index.php"
                },

                "php70domaincom": {
                        "type": "php 7.0.24",
                        "workers": 20,
                        "root": "/home/nginx/domains/domain.com/public",
                        "user": "nginx",
                        "group": "nginx",
                        "index": "index.php"
                },

                "php72domaincom": {
                        "type": "php 7.2.0",
                        "workers": 20,
                        "root": "/home/nginx/domains/domain.com/public",
                        "user": "nginx",
                        "group": "nginx",
                        "index": "index.php"
                },

                "php71domaincom": {
                        "type": "php 7.1.10",
                        "workers": 20,
                        "root": "/home/nginx/domains/domain.com/public",
                        "user": "nginx",
                        "group": "nginx",
                        "index": "index.php"
                }
        }
}

centminmod closed this as completed Sep 24, 2017

centminmod mentioned this issue Sep 27, 2017

Is there a roadmap or project board planned? #51

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Nginx Unit --state saving multiple json configs on service restarts #48

Nginx Unit --state saving multiple json configs on service restarts #48

centminmod commented Sep 24, 2017 •

edited

Loading

VBart commented Sep 24, 2017

centminmod commented Sep 24, 2017 •

edited

Loading

VBart commented Sep 24, 2017

centminmod commented Sep 24, 2017

VBart commented Sep 24, 2017

VBart commented Sep 24, 2017

VBart commented Sep 24, 2017 •

edited

Loading

VBart commented Sep 24, 2017

centminmod commented Sep 24, 2017 •

edited

Loading

VBart commented Sep 24, 2017

centminmod commented Sep 24, 2017 •

edited

Loading

VBart commented Sep 24, 2017

centminmod commented Sep 24, 2017 •

edited

Loading

VBart commented Sep 24, 2017

centminmod commented Sep 24, 2017

VBart commented Sep 24, 2017

centminmod commented Sep 24, 2017

Nginx Unit --state saving multiple json configs on service restarts #48

Nginx Unit --state saving multiple json configs on service restarts #48

Comments

centminmod commented Sep 24, 2017 • edited Loading

5.6.31

7.0.24

7.1.10

7.2.0

VBart commented Sep 24, 2017

centminmod commented Sep 24, 2017 • edited Loading

VBart commented Sep 24, 2017

centminmod commented Sep 24, 2017

VBart commented Sep 24, 2017

VBart commented Sep 24, 2017

VBart commented Sep 24, 2017 • edited Loading

VBart commented Sep 24, 2017

centminmod commented Sep 24, 2017 • edited Loading

VBart commented Sep 24, 2017

centminmod commented Sep 24, 2017 • edited Loading

VBart commented Sep 24, 2017

centminmod commented Sep 24, 2017 • edited Loading

VBart commented Sep 24, 2017

centminmod commented Sep 24, 2017

VBart commented Sep 24, 2017

centminmod commented Sep 24, 2017

centminmod commented Sep 24, 2017 •

edited

Loading

centminmod commented Sep 24, 2017 •

edited

Loading

VBart commented Sep 24, 2017 •

edited

Loading

centminmod commented Sep 24, 2017 •

edited

Loading

centminmod commented Sep 24, 2017 •

edited

Loading

centminmod commented Sep 24, 2017 •

edited

Loading