csi: add log rotation for csi rbd pod containers

[parth-gr]

1. Make the csi rbd container logs persisted in a file
   (csi plugin, csi provisioner, csi addons sidecar)

2. Use the cephcluster api specs to configure the log rotate

3. Add log rotation to rotate the log file and
   Add a sidecar log collector container

part-of: #12809

Updates:

• --logtostderr=false
• --alsologtostderr=true
• --log_file=/var/lib/cephcsi/log/rbdprov.log

sh-4.4# cat csi
/dev/*.log {
  rotate 3
  daily
  size 1M
  copytruncate
  start 1
}

When you mount a volume to a directory in a container, it effectively overrides the contents of that directory with the contents of the volume. This can cause issues if there are existing configurations or files in that directory that you want to preserve while adding new files or configurations.

Checklist:

• Commit Message Formatting: Commit titles and messages follow guidelines in the developer guide.
• Reviewed the developer guide on Submitting a Pull Request
• Pending release notes updated with breaking and/or notable changes for the next minor release.
• Documentation has been updated, if necessary.
• Unit tests have been added, if necessary.
• Integration tests have been added, if necessary.

[BlaineEXE]

I'd like to get @Madhu-1 's thought's on this as well, but it seems to me that adding these changes to all 4 major CSI deployment templates will be a lot of boilerplate to maintain over time. It seems like it might be in the interest of future devs to define this as a common Golang template that gets applied after template loading.

[travisn]

Just a couple nits, plus as discussed let's see if we can implement purely as a sidecar and no init container.

[parth-gr]

I'd like to get @Madhu-1 's thought's on this as well, but it seems to me that adding these changes to all 4 major CSI deployment templates will be a lot of boilerplate to maintain over time. It seems like it might be in the interest of future devs to define this as a common Golang template that gets applied after template loading.

can we have a common shell script?

@BlaineEXE or I think if one sidecar running the logrotate per node is fine, we can simply add the sidecar with the csi daemon set.

As we run the logrotate for a csi config file, which points to a same csi/log directory.
As all the logs will be present in same folder we can just make use of side car

With CSI as we do not re-load the containers we can use this method