Feature: Add a new container Logs API

[pranavs18]

Overview

Logs are always useful to figure out what's going on in the distributed system especially in situations when something goes wrong.Docker currently supports getting logs from a container that logs to stdout/stderr. Everything that the process running in the container writes to stdout or stderr docker will convert to json and store in a file on the host machine's disk which you can then retrieve with the docker logs command. This feature aims to provide a way to monitor logs from the Rancher UI.

What Docker already supports?

The current way of fetching the logs of a container is to run the docker logs command with 5 possible options from the docker remote API as in version 1.16 (https://docs.docker.com/reference/api/docker_remote_api_v1.16/#get-container-logs) -

Query Parameters:

follow – 1/True/true or 0/False/false, return stream. Default false
stdout – 1/True/true or 0/False/false, show stdout log. Default false
stderr – 1/True/true or 0/False/false, show stderr log. Default false
timestamps – 1/True/true or 0/False/false, print timestamps for every log line. Default false
tail – Output specified number of lines at the end of logs: all or . Default all

Need for this feature

Currently, there is no way from the Rancher UI to monitor docker container logs. Hence, this feature aims to obtain docker logs from each container, collect and ship it using this new API to the Rancher UI so that the user can always monitor it through the console on the Rancher UI. Thus, this feature provides an alternative to the end user to monitor the logs as agains the existing ways of either using the Docker Remote API or the Docker CLI.

Tentative Design

The result of this new action(API) should return a url to a websocket on the host and also a jwt token. The websocket should connect to the host-api go process which runs on the host. (I'll update this section as I progress further)

Implementation Details

The implementation of this new API would probably be based on how the stats action and the exec action on instance have been implemented in Rancher.Currently, the host-api go process creates a secure websocket that proxies information. Rancher currently uses it to grab information from "cadvisor". Here in this new action API, I plan to call docker logs command with appropriate input parameters required for fetching the logs of a container.

Query Parameters Format -

Name Format
"follow" - boolean
"lines" - Integer (corresponds to tail in docker's remote API) - default Value - 100
"stdOut" - boolean
"stdErr" - boolean
"timestamps" - boolean

Example API usage -

http://Cattle Server IP:8080/v1/containers/container ID/?action=logs&lines=10&stdOut=true&follow=true

Use Cases Needed to be Explored/Supported

1. Tailing the logs - This functionality will be supported where in the user can input the number of lines of logs to retrieve. The default value will be set to 100 lines.
2. Downloading the logs from the UI - YET TO BE DECIDED?
3. Scrolling through the logs on the UI - Once the logs are being viewed on the UI, the user should be able to scroll through them.
4. Searching through the logs on the UI ?

[pranavs18]

The progress of this feature can be tracked here - rancher/cattle#138

[pranavs18]

Few questions @ibuildthecloud

1. Currently, if I follow the pattern for execHandler, we return the URL to the websocket but when we talk about jwt authentication, how and where do we do its validation? Where can i get the public-key from so that once the jwt token is created by the Rancher server, it could be validated once the request comes in from the user?

2. Once the URL to the websocket with a port, container UUID and other arguments have been returned, the click on that URL should trigger a request and the "host-api" which I believe is listening to these requests. But looking at this GO-code, it seems it is kind of specific to "Stats" command, so do you think i should modify/restructure it and use it to make calls to the docker daemon? Or write a new Go-agent ?

3. When you talk about calling docker twice , one for stdOut and another for stdErr, could you point me to any existing module which handles the multiplexing and how any concurrency issues might have been handled here ?

4. What should actually be returned once the "docker logs...." call has been executed ? This "object" would need to be passed back to the UI.

[ibuildthecloud]

The frame format is

0 1 TS Content

Where 0 is the version number (always 0 for now), 1 is the either 1 or 2 depending on stdout/stderr and the TS is timestamp in the format that Docker returns.

[deniseschannon]

Closing as #40 is closed.