Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Substitute variables in path names of template repos too #25294

Merged
merged 15 commits into from Jun 20, 2023

Conversation

kdumontnu
Copy link
Contributor

@kdumontnu kdumontnu commented Jun 16, 2023

Summary

Extend the template variable substitution to replace file paths. This can be helpful for setting up log files & directories that should match the repository name.

PR Changes

  • Move files matching glob pattern when setting up repos from template
    • For security, added escaping sanitization for cross-platform support and to prevent directory traversal (thanks @silverwind for the reference)
  • Added unit testing for escaping function
  • Fixed the integration tests for repo template generation by passing the repo_template_id
  • Updated the integration testfiles to add some variable substitution & assert the outputs

I had to fix the existing repo template integration test and extend it to add a check for variable substitutions.

Example:
image

@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Jun 16, 2023
@kdumontnu kdumontnu marked this pull request as ready for review June 16, 2023 02:36
@pull-request-size pull-request-size bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jun 16, 2023
@kdumontnu kdumontnu force-pushed the kd/subst-repo_template_path_var branch from f906781 to 3cdaf8b Compare June 16, 2023 02:47
@delvh delvh changed the title Kd/subst repo template path var Substitute variables in path names of template repos too Jun 17, 2023
@kdumontnu kdumontnu force-pushed the kd/subst-repo_template_path_var branch from 86022f3 to 3b8fe94 Compare June 19, 2023 21:29
@GiteaBot GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Jun 19, 2023
@@ -191,10 +195,24 @@ func generateRepoCommit(ctx context.Context, repo, templateRepo, generateRepo *r
}

if err := os.WriteFile(path,
[]byte(generateExpansion(string(content), templateRepo, generateRepo)),
[]byte(generateExpansion(string(content), templateRepo, generateRepo, false)),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why here use "false" for file naming escaping (actually sanitizing)?

It seems that the rule can be applied to all.

Copy link
Contributor Author

@kdumontnu kdumontnu Jun 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other case expansion is used is in files (eg. in README text). Is there a reason to sanitize in that case?

I planned to leave that unchanged from the current behavior

@GiteaBot GiteaBot removed the lgtm/need 1 This PR needs approval from one additional maintainer to be merged. label Jun 20, 2023
@GiteaBot GiteaBot added the lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. label Jun 20, 2023
@jolheiser jolheiser enabled auto-merge (squash) June 20, 2023 19:25
@jolheiser jolheiser merged commit 8220e50 into go-gitea:main Jun 20, 2023
23 checks passed
@GiteaBot GiteaBot added this to the 1.21.0 milestone Jun 20, 2023
@kdumontnu kdumontnu deleted the kd/subst-repo_template_path_var branch June 20, 2023 21:15
zjjhot added a commit to zjjhot/gitea that referenced this pull request Jun 21, 2023
* giteaofficial/main:
  Refactor path & config system (go-gitea#25330)
  Add actor and status dropdowns to run list (go-gitea#25118)
  Use the new download domain replace the old (go-gitea#25405)
  Avoid polluting config file when "save" (go-gitea#25395)
  Fix dropdown icon layout on diff page (go-gitea#25397)
  Support configuration variables on Gitea Actions (go-gitea#24724)
  Substitute variables in path names of template repos too (go-gitea#25294)
  Navbar styling rework (go-gitea#25343)
  Fix blank dir message when uploading files from web editor (go-gitea#25391)
  Add git-lfs support to devcontainer (go-gitea#25385)
  Use qwtel.sqlite-viewer instead of alexcvzz.vscode-sqlite (go-gitea#25386)
  Use Actions git context instead of dynamically created buildkit one (go-gitea#25381)
  rename tributeValues to mentionValues (go-gitea#25375)
  Fix LDAP sync when Username Attribute is empty (go-gitea#25278)
  Fetch all git data for embedding correct version in docker image (go-gitea#25361)
  Fix sidebar label dropdown divider (go-gitea#25359)
  Fix issue filters on mobile view (go-gitea#25368)
  Refactor: TotalTimest return seconds (go-gitea#25370)
jolheiser added a commit that referenced this pull request Jun 21, 2023
/cc @KN4CK3R
#25294 (comment)

I also searched the codebase and found a few more.

---------

Signed-off-by: jolheiser <john.olheiser@gmail.com>
silverwind added a commit to silverwind/gitea that referenced this pull request Jun 21, 2023
* origin/main: (47 commits)
  Move some regexp out of functions (go-gitea#25430)
  Show outdated comments in files changed tab (go-gitea#24936)
  Remove "CHARSET" config option for MySQL, always use "utf8mb4" (go-gitea#25413)
  Fine tune project board label colors and modal content background (go-gitea#25419)
  Fix missing commit message body when the message has leading newlines (go-gitea#25418)
  add python/poetry to devcontainer (go-gitea#25407)
  Refactor path & config system (go-gitea#25330)
  Add actor and status dropdowns to run list (go-gitea#25118)
  Use the new download domain replace the old (go-gitea#25405)
  Avoid polluting config file when "save" (go-gitea#25395)
  Fix dropdown icon layout on diff page (go-gitea#25397)
  Support configuration variables on Gitea Actions (go-gitea#24724)
  Substitute variables in path names of template repos too (go-gitea#25294)
  Navbar styling rework (go-gitea#25343)
  Fix blank dir message when uploading files from web editor (go-gitea#25391)
  Add git-lfs support to devcontainer (go-gitea#25385)
  Use qwtel.sqlite-viewer instead of alexcvzz.vscode-sqlite (go-gitea#25386)
  Use Actions git context instead of dynamically created buildkit one (go-gitea#25381)
  rename tributeValues to mentionValues (go-gitea#25375)
  Fix LDAP sync when Username Attribute is empty (go-gitea#25278)
  ...
@go-gitea go-gitea locked as resolved and limited conversation to collaborators Sep 18, 2023
@denyskon denyskon added the type/enhancement An improvement of existing functionality label Nov 13, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. type/enhancement An improvement of existing functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants