Port-Authority (PA) is a microservice which does one thing: keep and manage a listing of services to ports. It offers the ability to PUT a service, which will either allocate a new port and return it, or get the existing port for said service and return that instead. You can also pull down information about the current inventory, list of assigned ports, and lis tof available ports. All over HTTP with simple REST-like calls.
One thing it doens't do is accept port assignments. Thus it is not intended to be used to generate an /etc/services file. This service was made for use in Dockerized environments where you may want to dynamically assign ports to services in containers (such as web or database service containers).
By default it will manage ports 30,000 to 39,999. You can set this in the consul config backing store, but keep in mind the end value is not inclusive.
Since we almost assuredly want this data persistent, the service requires access to a Redis instance.
The prefered way to run this is as a Docker container itself, and using a backing store, currently just Consul, to store your config in. With that setup it requires no local config and no command line or environment variables beyond the connection string for your Consul service. It does, however, take some optional ones.
The option -c or --consuladdress requires an argument of the form "ip:port"
and will tell the service what backing store address to connect to. It assumes
localhost:8500 if not provided. Not truly required but highly recommended as
not all configurables are in commandline/env variables yet.
Each instance of PA you run can be named, either via the --name option
or the PA_NAME environment variable. This is useful for situations
where you need different config values for specific servers. Not all
configurables support this mode. Those that do will be called out
specifically below.
The base KV path used is "app/port-authority/config" all paths referenced below are from that base.
The first configurable you'll want to know about is api_port. This
value specifies what port to listen on. If not found it defaults to
8080.
The next pair tell PA what port to start the pool on and where to end
it. They are found in two possible places. The first place is an
instance-specific config. Because you may want to run dedicated
instances of the service to handle different pools for different
services PA will look under NAME/ports_begin for the start of the
pool, and NAME/ports_end, for where it ends.
In this case NAME is the option passed in via the --name commandline
argument or the PA_NAME environment variable. If the key isn't there,
or you did not name the instance PA will look in ports_begin and
ports_end at the base prefix.
To use a port you add a service. The URL is /api/service/ID where ID
is whatever you are calling this particular service. It must be unique,
no exceptions. To use this API call you call the PUT HTTP method on
the full URL.
What you will get back is a JSON mapping structure which contains 'status', 'statusmessage', and 'data' keys. The value of the 'data' key is the port number assigned to this service. Time for an example.
curl http://localhost:8080/api/service/webapp-cars
In this example, say we get back the port "32123" (note: the port number is a random selection of the avialble ports, don't exect a sequential listing). Now we have a mapping between 'webapp-cars' and port '32123'. So however we need to assign that port to our webapp, we do so.
Now, what if you repeated the call? From the client-side, nothing is different. However, PA checks for an existing port for the service ID provided and will return that if found. Thus, it can be considered idempotent as it will do the same thing and get the same results - unless you delete the service mapping in between.
Sometimes you may want to just look for a service, and not assign it a
port. Issue a GET to the above URL for that.
Say you're done with the port, maybe you need to decommission that
service. For that, and keeping the example above, you would issue a
DELETE method HTTP call to
http://localhost:8080/api/service/webapp-cars and it would be removed
from the store in it's entirety.
You can check the current inventory and resrved ports via simple calls as well.
To see how many ports are available:
curl http://localhost:8080/api/ports/inventory/count
To get the list of them:
curl http://localhost:8080/api/ports/inventory/count
Now for listing how many ports have been reserved/assigned:
curl http://localhost:8080/api/ports/assigned/count
And the listing:
curl http://localhost:8080/api/ports/assigned/list
PA uses very few keys in Redis, though not all keys are created at initialization time. As most keys only exists under certain conditions a freshly initialized atabase will have but one key.
When the service is first started it will connect to the configured
Redis instance and attempt to initialize the database. This means it
will check for the existence of the open_ports key first. If not found
it will assume the DB needs initialized.
For initialization a sorted set named open_ports is created with
each and every port number the PA is allowed to manage added to it. When
new ports are requested PA calls SPOP to get a random member.
Once it has it it will then add it to a sorted set named
assigned_ports, then add it to a pair of hashes: i2port (to map IDs
to ports) and port2i to map ports to IDs). As such, once you've
reserved one port all four keys will be created.
When we remove the last assigned port/service, Redis will delete the now empty
hashes and assigned_ports keys. As a result the key count will be very
small.
Depending on how large your port range is this should be quite memory
efficient as the two sets are integer sets, which Redis optimizes for. To
get the maximum benefit the value of set-max-intset-entries in Redis'
configuration should be set to at least the number of ports you will
have PA manage. Eventually this will be configurable in PA as well for
cases where you use a dedicated Redis instance (which you should do in
general).
Since we are storing integers for the id2port hash, you could tune
hash-max-ziplist-entries similarly to the above setting. The
hash-max-ziplist-value may be of some value as well if you have a good
understanding of the size of your IDs. Generally this last setting would
mostly be useful, if at all, in cases of very tight memory limitations
such as if your Redis is running on a RaspberryPi (especially with this
service there as well).
Currently it is hard coded to talk to localhost, but that will be
configured in the config store Real Soon Now (tm). When this is added
the paths will likely be redis/ip, redis/port, and redis/auth to
store the IP address, port, and authentication information for Redis
connectivity respectively. Additional Redis settings, when added, will
go in this space as well.
- Add a call to get the full mapping of assigned ports
- Add configuration support for setting Redis memory settings during initialization
- Write the Web interface portion
- Write the 0MQ based RPC
- Get all configurables in ENV and CLI as well.
- Finish getting Airbrake support added and documented
- Perhaps NewRelic support as well?
- Add in go-metrics stats
- Add in circuitbreaker for talking to Redis.
- Add Dockerfile
- travis-ci.org config - including release using
ghr - ... and write some tests for it to use