Support intermediate captured groups in recursive strategy

[sclorng]

What would you like Renovate to be able to do?

I have a file with multiple group sections like

group: 
  ccm:
    baseRepository: foo

  components:
    ccm:
      repository: banana
      version: 1.0.0

group: 
  ccm:
    baseRepository: bar

  components:
   ccm:
      repository: apple
      version: 1.0.0

I would like to be able to first match a group content and capture baseRepository, and then capture each component.
so that I can write a depNameTemplate as {{baseRepository}}/{{repository}}
with something like :

"matchStrings": [
        "group:\\s+ccm:\\s+baseRepository: (?<baseRepository>.*?)\\s.*",
        "components:\\s.*",
        "ccm:\\s+repository: (?<repository>.*?)\\s*branch: (?<currentValue>.*)\\s?"
      ],

It should give out dependencies: foo/banana and bar/apple

If you have any ideas on how this should be implemented, please tell us here.

groups can be pass to handleRecursive calls and mergeGroups be call before createDependency

Is this a feature you are interested in implementing yourself?

No

[rarkins]

@secustor you are probably best suited to evaluate this

[secustor]

There are two topics here:

1. Found fields should be handed over for the recursive strategy.
   This is straight forward and I don't see problem with this.

2. Arbitrary regex capture groups are not allowed
   We not support arbitrary regex capture groups, therefore only the provided names will be captured.
   https://github.com/renovatebot/renovate/tree/main/lib/manager/regex#required-fields
   The reason is that if we would allow that without other precaution we would open up the application to prototype pollution attacks.
   If we confidently can deny this attack, IMO this would definitely improve the usability

[scrocquesel]

I push an update to the PR with a null prototype object as result group. This should avoid the pollution.

[scrocquesel]

@secustor while working on strategy, recursive should allow to narrow the content at each step. Instead of apply the next regex on the complete match of the previous one, we can capture an inner content to work on. This would allow this use case:

{
      matchStrings: [
        '"(?<first>[^"]*)":\\s*(?<innerContent>{[^}]*})',
        '"(?<second>[^"]*)":\\s*\\{[^}]*}',
        '"name":\\s*"(?<depName>.*)"[^"]*"type":\\s*"(?<datasource>.*)"[^"]*"value":\\s*"(?<currentValue>.*)"',
      ],
      matchStringsStrategy: 'recursive',
      depNameTemplate: '{{{ first }}}/{{{ second }}}/{{{ depName }}}',
    };

on example.json

to return depName :

backup/test/grafana/loki
setup/test/python

instead of

group1/group1/prom/prometheus
group2/group2/grafana/grafana
backup/backup/grafana/loki
setup/setup/python

Should this be a new issue ?

[secustor]

Please open a separate issue to discuss this.

[renovate-release]

🎉 This issue has been resolved in version 28.18.0 🎉

The release is available on:

• GitHub release
• 28.18.0

Your semantic-release bot 📦🚀