Add Log Rotation for TE components logs

[sanathpholla]

• Currently the Loggers for different components in TE does not do log rotation.
• Due to this, there might be scenario where the disk would be filled up in the long run and with scaled traffic for long period of time.

One such occurrence had occurred where te-postgres.log was taking up 1.7GB of space.

root@Client-te-1:/tmp# ls -lrth
total 1.7G
-rwxr-xr-x 1 root root  502 Feb 25  2020 setup_postgres.sh
-rw-r--r-- 1 root root  398 Feb 25  2020 requirements.txt
-rw-r--r-- 1 root root   80 Nov 30 02:49 wrk.te.log
-rw-r--r-- 1 root root  234 Nov 30 02:49 te-data.json
-rw-r--r-- 1 root root  137 Nov 30 02:49 te-zmq.log
-rw------- 1 root root  280 Nov 30 02:49 TE-stdout---supervisor-l_N4ZA.log
-rw-r--r-- 1 root root 2.2M Dec  1 06:09 te.log
-rw------- 1 root root  13K Dec  1 06:09 TE-stderr---supervisor-dTkxqn.log
-rw-r--r-- 1 root root 1.7G Dec  1 06:09 te-postgres.log

• Ask here is to enable log rotation for the te components logs

Creating this issue to track this enhancement

[sanathpholla]

class Logger:
    def __init__(self, name, logFilePath, level=None, bufferSize=1024*1000*5):
        try:
            ~ trim ~

            fileHandler = logging.FileHandler(logFilePath)
            fileHandler.setLevel(level)
            fileHandler.setFormatter(formatter)

@aravindhank11 We do not have RotatingFilehandler present currently

[aravindhank11]

As per https://github.com/vmware/te-ns/blob/main/te/te_docker/Dockerfile#L66-L78, log rotation must be kicking in for every file sized > 256 MB every 15 minutes.

Could you take a live TE Controller and experiment with what might be going wrong?

Please loop me in if I could be of a helping hand.

Thanks for reporting the issue and offering to help.