Git operations are slow on instances with a large number of machines

[dylanlerch]

Team

• I've assigned a team label to this issue

Severity

Not blocking, but significant performance degradation for some customers

Version

Impacts creating releases on 2022.2, impacts all Git operations on 2022.3

Latest Version

I could reproduce the problem in the latest build

What happened?

On Octopus 2022.2 versions, creating releases takes a long time on instances with a lot of machines. On Octopus 2022.3, all Git operations are impacted.

Reproduction

• On an Octopus 2022.3 version
• Create ~3000 machines in an instance
• Fetch a deployment process, deployment settings, or project variable set from a Git project
• Request time will be very long

Error and Stacktrace

No response

More Information

When writing to and reading from OCL, we convert between slugs (or names on older version of Octopus) and ids. This allows us to have human readable identifiers in the OCL, rather than something like Channels-102, that wouldn't be particularly helpful to anyone looking through OCL files.

At the moment, the way that we load this data from the database is extremely inefficient. We're often loading a lot more data than we actually need to perform this mapping. For instances with a large number of machines, loading this data can take a long time.

Workaround

No response

[dylanlerch]

This is only in cloud instances for now, we'll roll this performance fix back to 2022.3 in coming days.

[Octobob]

🎉 The fix for this issue has been released in:

Release stream	Release
2022.3	2022.3.10594
2022.4	2022.4.3438
2023.1+	all releases