Package Retention Policies on Runbooks are hardcoded to 90 days

[BobJWalker]

Team

• I've assigned a team label to this issue

Severity

Reported by a user on community slack

Version

2022.2 build 244 (customer verified it was happening on their instance as well)

Latest Version

I could reproduce the problem in the latest build

What happened?

I have two steps in my runbook that download and extract a package.

I have set my retention policies on that runbook to 2 runs per environment.

I run the runbook multiple times. I then manually trigger the retention policy process. Runbook runs are cleaned up as expected.

However, the installed packages location is not getting cleaned up. This is because that policy is hardcoded to 90 days. It is not respecting the retention policy value we set in the runbook settings.

Reproduction

See above.

Error and Stacktrace

|   == Success: Apply retention policy on Tentacles ==
16:12:28   Info     |     Apply retention policies...
16:12:31   Verbose  |     Apply Tentacle Retention Policy completed
                    |   
                    |     == Success: dev-server-01 ==
                    |     
                    |       Success: Applying retention policy using 90 Days to set Environments-1/Projects-582/Step-Deploy a Package/Machines-1/<default>
16:12:28   Verbose  |         Acquiring isolation mutex RunningScript with NoIsolation in ServerTasks-96977
16:12:28   Verbose  |         Executable directory is C:\Windows\system32\WindowsPowershell\v1.0
16:12:28   Verbose  |         Executable name or full path: C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe
16:12:28   Verbose  |         No user context provided. Running as current user.
16:12:29   Verbose  |         Starting C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in working directory 'C:\Octopus\Work\20220318211228-96977-499' using 'OEM United States' encoding running as 'NT AUTHORITY\SYSTEM' with the same environment variables as the launching process
16:12:29   Verbose  |         Process C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in C:\Octopus\Work\20220318211228-96977-499 exited with code 0
16:12:29   Verbose  |         Using Calamari.win-x64 21.1.2
16:12:29   Verbose  |         Acquiring isolation mutex RunningScript with NoIsolation in ServerTasks-96977
16:12:29   Verbose  |         Executable directory is C:\Windows\system32\WindowsPowershell\v1.0
16:12:29   Verbose  |         Executable name or full path: C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe
16:12:29   Verbose  |         No user context provided. Running as current user.
16:12:29   Verbose  |         Starting C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in working directory 'C:\Octopus\Work\20220318211229-96977-500' using 'OEM United States' encoding running as 'NT AUTHORITY\SYSTEM' with the same environment variables as the launching process
16:12:29   Verbose  |         Calamari Version: 21.1.2
16:12:29   Verbose  |         Environment Information:
16:12:29   Verbose  |         OperatingSystem: Microsoft Windows NT 10.0.19044.0
16:12:29   Verbose  |         OsBitVersion: x64
16:12:29   Verbose  |         Is64BitProcess: True
16:12:29   Verbose  |         CurrentUser: NT AUTHORITY\SYSTEM
16:12:29   Verbose  |         MachineName: DEMOVM
16:12:29   Verbose  |         ProcessorCount: 4
16:12:29   Verbose  |         CurrentDirectory: C:\Octopus\Work\20220318211229-96977-500
16:12:29   Verbose  |         TempDirectory: C:\Windows\TEMP\
16:12:29   Verbose  |         HostProcess: Calamari (6332)
16:12:29   Info     |         Keeping deployments from the last 90 days
16:12:29   Verbose  |         Keeping C:\Octopus\Applications\Development\hello-world\1.0.4_6 and C:\Octopus\Files\hello-world@S1.0.4@533FA2852AEEAD46A6BCA7557857795A.zip as it was installed 0 days and 0 hours ago
16:12:29   Verbose  |         Keeping C:\Octopus\Applications\Development\hello-world\1.0.4_7 and C:\Octopus\Files\hello-world@S1.0.4@533FA2852AEEAD46A6BCA7557857795A.zip as it was installed 0 days and 0 hours ago
16:12:29   Verbose  |         Keeping C:\Octopus\Applications\Development\hello-world\1.0.4_8 and C:\Octopus\Files\hello-world@S1.0.4@533FA2852AEEAD46A6BCA7557857795A.zip as it was installed 0 days and 0 hours ago
16:12:29   Verbose  |         Keeping C:\Octopus\Applications\Development\hello-world\1.0.4_9 and C:\Octopus\Files\hello-world@S1.0.4@533FA2852AEEAD46A6BCA7557857795A.zip as it was installed 0 days and 0 hours ago
16:12:29   Verbose  |         Keeping C:\Octopus\Applications\Development\hello-world\1.0.4_10 and C:\Octopus\Files\hello-world@S1.0.4@533FA2852AEEAD46A6BCA7557857795A.zip as it was installed 0 days and 0 hours ago
16:12:29   Verbose  |         Keeping C:\Octopus\Applications\Development\hello-world\1.0.4_11 and C:\Octopus\Files\hello-world@S1.0.4@533FA2852AEEAD46A6BCA7557857795A.zip as it was installed 0 days and 0 hours ago
16:12:29   Verbose  |         Keeping C:\Octopus\Applications\Development\hello-world\1.0.4_12 and C:\Octopus\Files\hello-world@S1.0.4@533FA2852AEEAD46A6BCA7557857795A.zip as it was installed 0 days and 0 hours ago
16:12:29   Verbose  |         Keeping C:\Octopus\Applications\Development\hello-world\1.0.4_13 and C:\Octopus\Files\hello-world@S1.0.4@533FA2852AEEAD46A6BCA7557857795A.zip as it was installed 0 days and 0 hours ago
16:12:29   Info     |         Did not find any deployments to clean up
16:12:29   Verbose  |         Process C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in C:\Octopus\Work\20220318211229-96977-500 exited with code 0
16:12:29   Verbose  |         Exit code: 0
                    |       
                    |       Success: Applying retention policy using 90 Days to set Environments-1/Projects-582/Step-Run a Script/Machines-1/<default>
16:12:30   Verbose  |         Acquiring isolation mutex RunningScript with NoIsolation in ServerTasks-96977
16:12:30   Verbose  |         Executable directory is C:\Windows\system32\WindowsPowershell\v1.0
16:12:30   Verbose  |         Executable name or full path: C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe
16:12:30   Verbose  |         No user context provided. Running as current user.
16:12:30   Verbose  |         Starting C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in working directory 'C:\Octopus\Work\20220318211229-96977-501' using 'OEM United States' encoding running as 'NT AUTHORITY\SYSTEM' with the same environment variables as the launching process
16:12:30   Verbose  |         Process C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in C:\Octopus\Work\20220318211229-96977-501 exited with code 0
16:12:30   Verbose  |         Using Calamari.win-x64 21.1.2
16:12:30   Verbose  |         Acquiring isolation mutex RunningScript with NoIsolation in ServerTasks-96977
16:12:30   Verbose  |         Executable directory is C:\Windows\system32\WindowsPowershell\v1.0
16:12:30   Verbose  |         Executable name or full path: C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe
16:12:30   Verbose  |         No user context provided. Running as current user.
16:12:30   Verbose  |         Starting C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in working directory 'C:\Octopus\Work\20220318211230-96977-502' using 'OEM United States' encoding running as 'NT AUTHORITY\SYSTEM' with the same environment variables as the launching process
16:12:30   Verbose  |         Calamari Version: 21.1.2
16:12:30   Verbose  |         Environment Information:
16:12:30   Verbose  |         OperatingSystem: Microsoft Windows NT 10.0.19044.0
16:12:30   Verbose  |         OsBitVersion: x64
16:12:30   Verbose  |         Is64BitProcess: True
16:12:30   Verbose  |         CurrentUser: NT AUTHORITY\SYSTEM
16:12:30   Verbose  |         MachineName: DEMOVM
16:12:30   Verbose  |         ProcessorCount: 4
16:12:30   Verbose  |         CurrentDirectory: C:\Octopus\Work\20220318211230-96977-502
16:12:30   Verbose  |         TempDirectory: C:\Windows\TEMP\
16:12:31   Verbose  |         HostProcess: Calamari (7564)
16:12:31   Info     |         Keeping deployments from the last 90 days
16:12:31   Verbose  |         Keeping C:\Octopus\Files\OctoFX.Database@S2021.9.9.1@0EF6F0EF4EEA5643B1BB5CE9C9B9DB63.nupkg as it was installed 0 days and 0 hours ago
16:12:31   Verbose  |         Keeping C:\Octopus\Files\OctoFX.Database@S2021.9.9.1@0EF6F0EF4EEA5643B1BB5CE9C9B9DB63.nupkg as it was installed 0 days and 0 hours ago
16:12:31   Verbose  |         Keeping C:\Octopus\Files\OctoFX.Database@S2021.9.9.1@0EF6F0EF4EEA5643B1BB5CE9C9B9DB63.nupkg as it was installed 0 days and 0 hours ago
16:12:31   Verbose  |         Keeping C:\Octopus\Files\OctoFX.Database@S2021.9.9.1@0EF6F0EF4EEA5643B1BB5CE9C9B9DB63.nupkg as it was installed 0 days and 0 hours ago
16:12:31   Info     |         Did not find any deployments to clean up
16:12:31   Verbose  |         Process C:\Windows\system32\WindowsPowershell\v1.0\PowerShell.exe in C:\Octopus\Work\20220318211230-96977-502 exited with code 0
16:12:31   Verbose  |         Exit code: 0

### More Information

_No response_

### Workaround

Run a script in the deployment process to clean up that "install location" on each runbook run.

[veochen-octopus]

I can see the 90 days being hardcoded for runbooks. But I don't think runbook retention policy affects tentacles. This is about how many runbook runs to keep. Tentacle retention policy is set in the lifecycle settings. Deployments already correctly use that configuration. All we need to do is making runbooks consistent.

Edit: Runbooks don't have lifecycle or release information. So there's a gap to fill here. Putting it back for some R&D work to be done first.

[Clare-Octopus]

Another report where this issue is affecting the runbooks on a customers instance as each runbook is performing retention on 17800 files as they are not being cleaned up as per the lifecycle policy (internal ticket) - https://octopus.zendesk.com/agent/tickets/131092

[Clare-Octopus]

Another customer affected - https://help.octopus.com/t/work-folder-is-not-being-cleaned/29206/3

[tothegills]

An issue has been addressed to reduce the deployment journal size: #8310