Multiple Escaping Improvements #17551

zeripath · 2021-11-04T22:58:18Z

There are multiple places where Gitea does not properly escape URLs that it is building and there are multiple places where it builds urls when there is already a simpler function available to use this.

This is an extensive PR attempting to fix these issues.

The first commit in this PR looks through all href, src and links in the Gitea codebase and has attempted to catch all the places where there is potentially incomplete escaping.
Whilst doing this we will prefer to use functions that create URLs over recreating them by hand.
All uses of strings should be directly escaped - even if they are not currently expected to contain escaping characters. The main benefit to doing this will be that we can consider relaxing the constraints on user names and reponames in future.
The next commit looks at escaping in the wiki and re-considers the urls that are used there. Using the improved escaping here wiki files containing '/'. (This implementation will currently still place all of the wiki files the root directory of the repo but this would not be difficult to change.)
The title generation in feeds is now properly escaped.
EscapePound is no longer needed - urls should be PathEscaped / QueryEscaped as necessary but then re-escaped with Escape when creating html with locales

There are multiple places where Gitea does not properly escape URLs that it is building and there are multiple places where it builds urls when there is already a simpler function available to use this. This is an extensive PR attempting to fix these issues. Signed-off-by: Andrew Thornton <art27@cantab.net>

One of the major limitations of the wiki is that it does not currently allow a hierarchy of folders. Here we add all of the escaping to allow this to be instituted. Signed-off-by: Andrew Thornton <art27@cantab.net>

Signed-off-by: Andrew Thornton <art27@cantab.net>

… be used instead Signed-off-by: Andrew Thornton <art27@cantab.net>

codecov-commenter · 2021-11-05T10:34:07Z

Codecov Report

Merging #17551 (cbd5eec) into main (7e1ae38) will decrease coverage by 0.05%.
The diff coverage is 38.64%.

@@            Coverage Diff             @@
##             main   #17551      +/-   ##
==========================================
- Coverage   45.39%   45.33%   -0.06%     
==========================================
  Files         799      799              
  Lines       88946    89043      +97     
==========================================
- Hits        40379    40372       -7     
- Misses      42063    42165     +102     
- Partials     6504     6506       +2

Impacted Files	Coverage Δ
models/notification.go	`64.71% <0.00%> (-0.87%)`	⬇️
models/repo_avatar.go	`35.78% <0.00%> (ø)`
modules/context/context.go	`58.75% <0.00%> (-0.72%)`	⬇️
modules/convert/notification.go	`44.44% <0.00%> (ø)`
modules/templates/helper.go	`49.91% <0.00%> (-1.12%)`	⬇️
routers/api/v1/org/member.go	`19.78% <0.00%> (ø)`
routers/web/admin/repos.go	`0.00% <0.00%> (ø)`
routers/web/feed/convert.go	`0.00% <0.00%> (ø)`
routers/web/org/setting.go	`0.00% <0.00%> (ø)`
routers/web/org/teams.go	`0.00% <0.00%> (ø)`
... and 67 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7e1ae38...cbd5eec. Read the comment docs.

Signed-off-by: Andrew Thornton <art27@cantab.net>

models/action.go

Signed-off-by: Andrew Thornton <art27@cantab.net>

zeripath · 2021-11-05T22:35:11Z

Wiki behaviour is long-standing so I am not changing it here

zeripath · 2021-11-05T22:35:29Z

Ready for review.

cmd/web.go

models/avatars/avatar.go

models/repo.go

models/repo_avatar.go

modules/context/context.go

routers/web/repo/pull.go

templates/repo/branch/list.tmpl

wxiaoguang · 2021-11-06T07:06:20Z

web_src/js/features/repo-branch.js

@@ -1,6 +1,6 @@
 export function initRepoBranchButton() {
  $('.show-create-branch-modal.button').on('click', function () {
-    $('#create-branch-form')[0].action = $('#create-branch-form').data('base-action') + $(this).data('branch-from');
+    $('#create-branch-form')[0].action = $('#create-branch-form').data('base-action') + $(this).data('branch-from-urlcomponent');


Maybe we can do PathEscapeSegments in JS, then we do not need the data-branch-from-urlcomponent

If this is the only place we need it - no.

This question is about the framework design.

Usually encoding/decoding should not cross different modules (otherwise double-encoding or non-encoding bugs may occur). So it's usually better to make every module handle encoding and decoding correctly.

If we do not introduce PathEscapeSegment for JS now, maybe we can leave a comment to tell developers in future: if more JS modules need process paths, we should introduce a PathEscapeSegments for JS then.

I understand, for this PR, I think what I'd like to do is to get people to think a bit more clearly about whether something is going in to an URL or whether it's getting presented.

At the moment what we currently believe is that you can always take the username of a user, reponame etc. and put it into an url directly. Similarly for Branch names etc.

I do not believe that is necessarily true - especially when thinking about how to deal with homoglyph attacks. By separating these two here, when we have a clearer idea of how to make urls and how to homoglyph protect we can merge the two again.

Thus, my intention with data-branch-urlcomponent is to be deliberately ugly. It's not supposed to be the end.

As a note:

Most of this PR is ugly as hell - there are hints at framework improvements, e.g. use url converter functions like .Link() .HTMLURL(). But these converter functions are all at the wrong level they need to be available at the template level/router level.

(For example I think the use of setting.AppSubUrl and setting.AppUrl in models isn't great. If we can move the url generation up into the router level etc we can make this context dependent and make things like requests from [internal] get into internal based urls and [external] get external urls.)

Signed-off-by: Andrew Thornton <art27@cantab.net>

silverwind · 2021-11-17T21:08:26Z

Regular issue view

…and other fixes (#17991) This PR contains multiple fixes. The most important of which is: * Prevent hang in git cat-file if the repository is not a valid repository Unfortunately it appears that if git cat-file is run in an invalid repository it will hang until stdin is closed. This will result in deadlocked /pulls pages and dangling git cat-file calls if a broken repository is tried to be reviewed or pulls exists for a broken repository. Fix #14734 Fix #9271 Fix #16113 Otherwise there are a few small other fixes included which this PR was initially intending to fix: * Fix panic on partial compares due to missing PullRequestWorkInProgressPrefixes * Fix links on pulls pages due to regression from #17551 - by making most /issues routes match /pulls too - Fix #17983 * Fix links on feeds pages due to another regression from #17551 but also fix issue with syncing tags - Fix #17943 * Add missing locale entries for oauth group claims * Prevent NPEs if ColorFormat is called on nil users, repos or teams.

@midnight

Frontport go-gitea#18468 Frontport changelog for 1.16, frontport 1.15.11 changelog and update config.yaml ## [1.16.0](https://github.com/go-gitea/gitea/releases/tag/v1.16.0) - 2022-01-30 * BREAKING * Remove golang vendored directory (go-gitea#18277) * Paginate releases page & set default page size to 10 (go-gitea#16857) * Only allow webhook to send requests to allowed hosts (go-gitea#17482) * SECURITY * Disable content sniffing on `PlainTextBytes` (go-gitea#18359) (go-gitea#18365) * Only view milestones from current repo (go-gitea#18414) (go-gitea#18417) * Sanitize user-input on file name (go-gitea#17666) * Use `hostmatcher` to replace `matchlist` to improve blocking of bad hosts in Webhooks (go-gitea#17605) * FEATURES * Add/update SMTP auth providers via cli (go-gitea#18197) * Support webauthn (go-gitea#17957) * Team permission allow different unit has different permission (go-gitea#17811) * Implement Well-Known URL for password change (go-gitea#17777) * Add support for ssh commit signing (go-gitea#17743) * Allow Loading of Diffs that are too large (go-gitea#17739) * Add copy button to markdown code blocks (go-gitea#17638) * Add .gitattribute assisted language detection to blame, diff and render (go-gitea#17590) * Add `PULL_LIMIT` and `PUSH_LIMIT` to cron.update_mirror task (go-gitea#17568) * Add Reindex buttons to repository settings page (go-gitea#17494) * Make SSL cipher suite configurable (go-gitea#17440) * Add groups scope/claim to OIDC/OAuth2 Provider (go-gitea#17367) * Add simple update checker to Gitea (go-gitea#17212) * Migrated Repository will show modifications when possible (go-gitea#17191) * Create pub/priv keypair for federation (go-gitea#17071) * Make LDAP be able to skip local 2FA (go-gitea#16954) * Add nodeinfo endpoint for federation purposes (go-gitea#16953) * Save and view issue/comment content history (go-gitea#16909) * Use git attributes to determine generated and vendored status for language stats and diffs (go-gitea#16773) * Add migrate from Codebase (go-gitea#16768) * Add migration from GitBucket (go-gitea#16767) * Add OAuth2 introspection endpoint (go-gitea#16752) * Add proxy settings and support for migration and webhook (go-gitea#16704) * Add microsoft oauth2 providers (go-gitea#16544) * Send registration email on user autoregistration (go-gitea#16523) * Defer Last Commit Info (go-gitea#16467) * Support unprotected file patterns (go-gitea#16395) * Add migrate from OneDev (go-gitea#16356) * Add option to update pull request by `rebase` (go-gitea#16125) * Add RSS/Atom feed support for user actions (go-gitea#16002) * Add support for corporate WeChat webhooks (go-gitea#15910) * Add a simple way to rename branch like gh (go-gitea#15870) * Add bundle download for repository (go-gitea#14538) * Add agit flow support in gitea (go-gitea#14295) * API * Add MirrorUpdated field to Repository API type (go-gitea#18267) * Adjust Fork API to allow setting a custom repository name (go-gitea#18066) * Add API to manage repo tranfers (go-gitea#17963) * Add API to get file commit history (go-gitea#17652) * Add API to get issue/pull comments and events (timeline) (go-gitea#17403) * Add API to get/edit wiki (go-gitea#17278) * Add API for get user org permissions (go-gitea#17232) * Add HTML urls to notification API (go-gitea#17178) * Add API to get commit diff/patch (go-gitea#17095) * Respond with updated notifications in API (go-gitea#17064) * Add API to fetch git notes (go-gitea#16649) * Generalize list header for API (go-gitea#16551) * Add API Token Cache (go-gitea#16547) * Allow Token API calls be authorized using the reverse-proxy header (go-gitea#15119) * ENHANCEMENTS * Make the height of the editor in Review Box smaller (4 lines as GitHub) (go-gitea#18319) * Return nicer error if trying to pull from non-existent user (go-gitea#18288) * Show pull link for agit pull request also (go-gitea#18235) * Enable partial clone by default (go-gitea#18195) * Added replay of webhooks (go-gitea#18191) * Show OAuth callback error message (go-gitea#18185) * Increase Salt randomness (go-gitea#18179) * Add MP4 as default allowed attachment type (go-gitea#18170) * Include folders into size cost (go-gitea#18158) * Remove `/email2user` endpoint (go-gitea#18127) * Handle invalid issues (go-gitea#18111) * Load EasyMDE/CodeMirror dynamically, remove RequireEasyMDE (go-gitea#18069) * Support open compare page directly (go-gitea#17975) * Prefer "Hiragino Kaku Gothic ProN" in system-ui-ja (go-gitea#17954) * Clean legacy SimpleMDE code (go-gitea#17926) * Refactor install page (db type) (go-gitea#17919) * Improve interface when comparing a branch which has created a pull request (go-gitea#17911) * Allow default branch to be inferred on compare page (go-gitea#17908) * Display issue/comment role even if repo archived (go-gitea#17907) * Always set a message-id on mails (go-gitea#17900) * Change `<a>` elements to underline on hover (go-gitea#17898) * Render issue references in file table (go-gitea#17897) * Handle relative unix socket paths (go-gitea#17836) * Move accessmode into models/perm (go-gitea#17828) * Fix some org style problems (go-gitea#17807) * Add List-Unsubscribe header (go-gitea#17804) * Create menus for organization pages (go-gitea#17802) * Switch archive URL code back to href attributes (go-gitea#17796) * Refactor "refs/*" string usage by using constants (go-gitea#17784) * Allow forks to org if you can create repos (go-gitea#17783) * Improve install code to avoid low-level mistakes. (go-gitea#17779) * Improve ellipsis buttons (go-gitea#17773) * Add restrict and no-user-rc to authorized_keys (go-gitea#17772) * Add copy Commit ID button in commits list (go-gitea#17759) * Make `bind` error more readable (go-gitea#17750) * Fix navbar on project view (go-gitea#17749) * More pleasantly handle broken or missing git repositories (go-gitea#17747) * Use `*PushUpdateOptions` as receiver (go-gitea#17724) * Remove unused `user` paramater (go-gitea#17723) * Better builtin avatar generator (go-gitea#17707) * Cleanup and use global style on popups (go-gitea#17674) * Move user/org deletion to services (go-gitea#17673) * Added comment for changing issue ref (go-gitea#17672) * Allow admins to change user avatars (go-gitea#17661) * Only set `data-path` once for each file in diff pages (go-gitea#17657) * Add icon to vscode clone link (go-gitea#17641) * Add download button for file viewer (go-gitea#17640) * Add pagination to fork list (go-gitea#17639) * Use a standalone struct name for Organization (go-gitea#17632) * Minor readability patch. (go-gitea#17627) * Add context support for GetUserByID (go-gitea#17602) * Move merge-section to `> .content` (go-gitea#17582) * Remove NewSession method from db.Engine interface (go-gitea#17577) * Move unit into models/unit/ (go-gitea#17576) * Restrict GetDeletedBranchByID to the repositories deleted branches (go-gitea#17570) * Refactor commentTags functionality (go-gitea#17558) * Make Repo Code Indexer an Unique Queue (go-gitea#17515) * Simplify Gothic to use our session store instead of creating a different store (go-gitea#17507) * Add settings to allow different SMTP envelope from address (go-gitea#17479) * Properly determine CSV delimiter (go-gitea#17459) * Hide label comments if labels were added and removed immediately (go-gitea#17455) * Tune UI alignment for nav bar notification icon, avatar image, issue label (go-gitea#17438) * Add appearance section in settings (go-gitea#17433) * Move key forms before list and add cancel button (go-gitea#17432) * When copying executables to the docker chmod them (go-gitea#17423) * Remove deprecated `extendDefaultPlugins` method of svgo (go-gitea#17399) * Fix the click behavior for <tr> and <td> with [data-href] (go-gitea#17388) * Refactor update checker to use AppState (go-gitea#17387) * Improve async/await usage, and sort init calls in `index.js` (go-gitea#17386) * Use a variable but a function for IsProd because of a slight performance increment (go-gitea#17368) * Frontend refactor, PascalCase to camelCase, remove unused code (go-gitea#17365) * Hide command line merge instructions when user can't push (go-gitea#17339) * Move session to models/login (go-gitea#17338) * Sync gitea app path for git hooks and authorized keys when starting (go-gitea#17335) * Make the Mirror Queue a queue (go-gitea#17326) * Add "Copy branch name" button to pull request page (go-gitea#17323) * Fix repository summary on mobile (go-gitea#17322) * Split `index.js` to separate files (go-gitea#17315) * Show direct match on top for user search (go-gitea#17303) * Frontend refactor: move Vue related code from `index.js` to `components` dir, and remove unused codes. (go-gitea#17301) * Upgrade chi to v5 (go-gitea#17298) * Disable form autofill (go-gitea#17291) * Improve behavior of "Fork" button (go-gitea#17288) * Open markdown image links in new window (go-gitea#17287) * Add hints for special Wiki pages (go-gitea#17283) * Move add deploy key form before the list and add a cancel button (go-gitea#17228) * Allow adding multiple issues to a project (go-gitea#17226) * Add metrics to get issues by repository (go-gitea#17225) * Add specific event type to header (go-gitea#17222) * Redirect on project after issue created (go-gitea#17211) * Reference in new issue modal: dont pre-populate issue title (go-gitea#17208) * Always set a unique Message-ID header (go-gitea#17206) * Add projects and project boards in exposed metrics (go-gitea#17202) * Add metrics to get issues by label (go-gitea#17201) * Add protection to disable Gitea when run as root (go-gitea#17168) * Don't return binary file changes in raw PR diffs by default (go-gitea#17158) * Support sorting for project board issuses (go-gitea#17152) * Force color-adjust for markdown checkboxes (go-gitea#17146) * Add option to copy line permalink (go-gitea#17145) * Move twofactor to models/login (go-gitea#17143) * Multiple tokens support for migrating from github (go-gitea#17134) * Unify issue and PR subtitles (go-gitea#17133) * Make Requests Processes and create process hierarchy. Associate OpenRepository with context. (go-gitea#17125) * Fix problem when database id is not increment as expected (go-gitea#17124) * Avatar refactor, move avatar code from `models` to `models.avatars`, remove duplicated code (go-gitea#17123) * Re-allow clipboard copy on non-https sites (go-gitea#17118) * DBContext is just a Context (go-gitea#17100) * Move login related structs and functions to models/login (go-gitea#17093) * Add SkipLocal2FA option to pam and smtp sources (go-gitea#17078) * Move db related basic functions to models/db (go-gitea#17075) * Fixes username tagging in "Reference in new issue" (go-gitea#17074) * Use light/dark theme based on system preference (go-gitea#17051) * Always emit the configuration path (go-gitea#17036) * Add `AbsoluteListOptions` (go-gitea#17028) * Use common sessioner for API and Web (go-gitea#17027) * Fix overflow label in small view (go-gitea#17020) * Report the associated filter if there is an error in LDAP (go-gitea#17014) * Add "new issue" btn on project (go-gitea#17001) * Add doctor dbconsistency check for release and attachment (go-gitea#16978) * Disable Fomantic's CSS tooltips (go-gitea#16974) * Add Cache-Control to avatar redirects (go-gitea#16973) * Make mirror feature more configurable (go-gitea#16957) * Add skip and limit to git.GetTags (go-gitea#16897) * Remove ParseQueueConnStr as it is unused (go-gitea#16878) * Remove unused Fomantic sidebar module (go-gitea#16853) * Allow LDAP Sources to provide Avatars (go-gitea#16851) * Remove Dashboard/Home button from the navbar (go-gitea#16844) * Use conditions but not repo ids as query condition (go-gitea#16839) * Add user settings key/value DB table (go-gitea#16834) * Add buttons to allow loading of incomplete diffs (go-gitea#16829) * Add information for migrate failure (go-gitea#16803) * Add EdDSA JWT signing algorithm (go-gitea#16786) * Add user status filter to admin user management page (go-gitea#16770) * Add Option to synchronize Admin & Restricted states from OIDC/OAuth2 along with Setting Scopes (go-gitea#16766) * Do not use thin scrollbars on Firefox (go-gitea#16738) * Download LFS in git and web workflow from minio/s3 directly (SERVE_DIRECT) (go-gitea#16731) * Compute proper foreground color for labels (go-gitea#16729) * Add edit button to wiki sidebar and footer (go-gitea#16719) * Fix migration svg color (go-gitea#16715) * Add link to vscode to repo header (go-gitea#16664) * Add filter by owner and team to issue/pulls search endpoint (go-gitea#16662) * Kanban colored boards (go-gitea#16647) * Allow setting X-FRAME-OPTIONS (go-gitea#16643) * Separate open and closed issue in metrics (go-gitea#16637) * Support direct comparison (git diff a..b) as well merge comparison (a…b) (go-gitea#16635) * Add setting to OAuth handlers to skip local 2FA authentication (go-gitea#16594) * Make PR merge options more intuitive (go-gitea#16582) * Show correct text when comparing commits on empty pull request (go-gitea#16569) * Pre-fill suggested New File 'name' and 'content' with Query Params (go-gitea#16556) * Add an abstract json layout to make it's easier to change json library (go-gitea#16528) * Make Mermaid.js limit configurable (go-gitea#16519) * Improve 2FA autofill (go-gitea#16473) * Add modals to Organization and Team remove/leave (go-gitea#16471) * Show tag name on dashboard items list (go-gitea#16466) * Change default cron schedules from @every 24h to @midnight (go-gitea#16431) * Prevent double sanitize (go-gitea#16386) * Replace `list.List` with slices (go-gitea#16311) * Add configuration option to restrict users by default (go-gitea#16256) * Move login out of models (go-gitea#16199) * Support pagination of organizations on user settings pages (go-gitea#16083) * Switch migration icon to svg (go-gitea#15954) * Add left padding for chunk header of split diff view (go-gitea#13397) * Allow U2F 2FA without TOTP (go-gitea#11573) * BUGFIXES * GitLab reviews may not have the updated_at field set (go-gitea#18450) (go-gitea#18461) * Fix detection of no commits when the default branch is not master (go-gitea#18422) (go-gitea#18423) * Fix broken oauth2 authentication source edit page (go-gitea#18412) (go-gitea#18419) * Place inline diff comment dialogs on split diff in 4th and 8th columns (go-gitea#18403) (go-gitea#18404) * Fix restore without topic failure (go-gitea#18387) (go-gitea#18400) * Fix commit's time (go-gitea#18375) (go-gitea#18392) * Fix partial cloning a repo (go-gitea#18373) (go-gitea#18377) * Stop trimming preceding and suffixing spaces from editor filenames (go-gitea#18334) * Prevent showing webauthn error for every time visiting `/user/settings/security` (go-gitea#18386) * Fix mime-type detection for HTTP server (go-gitea#18370) (go-gitea#18371) * Stop trimming preceding and suffixing spaces from editor filenames (go-gitea#18334) * Restore propagation of ErrDependenciesLeft (go-gitea#18325) * Fix PR comments UI (go-gitea#18323) * Use indirect comparison when showing pull requests (go-gitea#18313) * Replace satori/go.uuid with gofrs/uuid (go-gitea#18311) * Fix commit links on compare page (go-gitea#18310) * Don't show double error response in git hook (go-gitea#18292) * Handle missing default branch better in owner/repo/branches page (go-gitea#18290) * Fix CheckRepoStats and reuse it during migration (go-gitea#18264) * Prevent underline hover on cards (go-gitea#18259) * Don't delete branch if other PRs with this branch are open (go-gitea#18164) * Require codereview to have content (go-gitea#18156) * Allow admin to associate missing LFS objects for repositories (go-gitea#18143) * When attempting to subscribe other user to issue report why access denied (go-gitea#18091) * Add option to convert CRLF to LF line endings for sendmail (go-gitea#18075) * Only create pprof files for gitea serv if explicitly asked for (go-gitea#18068) * Abort merge if head has been updated before pressing merge (go-gitea#18032) * Improve TestPatch to use git read-tree -m and implement git-merge-one-file functionality (go-gitea#18004) * Use JSON module instead of stdlib json (go-gitea#18003) * Fixed issue merged/closed wording (go-gitea#17973) * Return nicer error for ForcePrivate (go-gitea#17971) * Fix overflow in commit graph (go-gitea#17947) * Prevent services/mailer/mailer_test.go tests from deleteing data directory (go-gitea#17941) * Use disable_form_autofill on Codebase and Gitbucket (go-gitea#17936) * Fix a panic in NotifyCreateIssueComment (caused by string truncation) (go-gitea#17928) * Fix markdown URL parsing (go-gitea#17924) * Apply CSS Variables to all message elements (go-gitea#17920) * Improve checkBranchName (go-gitea#17901) * Update chi/middleware to chi/v5/middleware (go-gitea#17888) * Fix position of label color picker colors (go-gitea#17866) * Fix ListUnadoptedRepositories incorrect total count (go-gitea#17865) * Remove whitespace inside rendered code `<td>` (go-gitea#17859) * Make Co-committed-by and co-authored-by trailers optional (go-gitea#17848) * Fix value of User.IsRestricted when oauth2 user registration (go-gitea#17839) * Use new OneDev /milestones endpoint (go-gitea#17782) * Prevent deadlock in TestPersistableChannelQueue (go-gitea#17717) * Simplify code for writing SHA to name-rev (go-gitea#17696) * Fix database deadlock when update issue labels (go-gitea#17649) * Add warning for BIDI characters in page renders and in diffs (go-gitea#17562) * Fix ipv6 parsing for builtin ssh server (go-gitea#17561) * Multiple Escaping Improvements (go-gitea#17551) * Fixes go-gitea#16559 - Do not trim leading spaces for tab delimited (go-gitea#17442) * Show client-side error if wiki page is empty (go-gitea#17415) * Fix context popup error (go-gitea#17398) * Stop sanitizing full name in API (go-gitea#17396) * Fix issue close/comment buttons on mobile (go-gitea#17317) * Fix navbar UI (go-gitea#17235) * Fix problem when database id is not increment as expected (go-gitea#17229) * Open the DingTalk link in browser (go-gitea#17084) * Remove heads pointing to missing old refs (go-gitea#17076) * Fix commit status index problem (go-gitea#17061) * Handle broken references in mirror sync (go-gitea#17013) * Fix for create repo page layout (go-gitea#17012) * Improve LDAP synchronization efficiency (go-gitea#16994) * Add repo_id for attachment (go-gitea#16958) * Clean-up HookPreReceive and restore functionality for pushing non-standard refs (go-gitea#16705) * Remove duplicate csv import in modules/csv/csv.go (go-gitea#16631) * Improve SMTP authentication and Fix user creation bugs (go-gitea#16612) * Fixed emoji alias not parsed in links (go-gitea#16221) * Calculate label URL on API (go-gitea#16186) * TRANSLATION * Fix mispelling of starred as stared (go-gitea#17465) * Re-separate the color translation strings (go-gitea#17390) * Enable Malayalam, Greek, Persian, Hungarian & Indonesian by default (go-gitea#16998) * BUILD * Add lockfile-check (go-gitea#18285) * Don't store assets modified time into generated files (go-gitea#18193) * Use shadowing script for docker (go-gitea#17846) * MISC * Update JS dependencies (go-gitea#17611) Signed-off-by: Andrew Thornton <art27@cantab.net>

@midnight

Frontport #18468 Frontport changelog for 1.16, frontport 1.15.11 changelog and update config.yaml ## [1.16.0](https://github.com/go-gitea/gitea/releases/tag/v1.16.0) - 2022-01-30 * BREAKING * Remove golang vendored directory (#18277) * Paginate releases page & set default page size to 10 (#16857) * Only allow webhook to send requests to allowed hosts (#17482) * SECURITY * Disable content sniffing on `PlainTextBytes` (#18359) (#18365) * Only view milestones from current repo (#18414) (#18417) * Sanitize user-input on file name (#17666) * Use `hostmatcher` to replace `matchlist` to improve blocking of bad hosts in Webhooks (#17605) * FEATURES * Add/update SMTP auth providers via cli (#18197) * Support webauthn (#17957) * Team permission allow different unit has different permission (#17811) * Implement Well-Known URL for password change (#17777) * Add support for ssh commit signing (#17743) * Allow Loading of Diffs that are too large (#17739) * Add copy button to markdown code blocks (#17638) * Add .gitattribute assisted language detection to blame, diff and render (#17590) * Add `PULL_LIMIT` and `PUSH_LIMIT` to cron.update_mirror task (#17568) * Add Reindex buttons to repository settings page (#17494) * Make SSL cipher suite configurable (#17440) * Add groups scope/claim to OIDC/OAuth2 Provider (#17367) * Add simple update checker to Gitea (#17212) * Migrated Repository will show modifications when possible (#17191) * Create pub/priv keypair for federation (#17071) * Make LDAP be able to skip local 2FA (#16954) * Add nodeinfo endpoint for federation purposes (#16953) * Save and view issue/comment content history (#16909) * Use git attributes to determine generated and vendored status for language stats and diffs (#16773) * Add migrate from Codebase (#16768) * Add migration from GitBucket (#16767) * Add OAuth2 introspection endpoint (#16752) * Add proxy settings and support for migration and webhook (#16704) * Add microsoft oauth2 providers (#16544) * Send registration email on user autoregistration (#16523) * Defer Last Commit Info (#16467) * Support unprotected file patterns (#16395) * Add migrate from OneDev (#16356) * Add option to update pull request by `rebase` (#16125) * Add RSS/Atom feed support for user actions (#16002) * Add support for corporate WeChat webhooks (#15910) * Add a simple way to rename branch like gh (#15870) * Add bundle download for repository (#14538) * Add agit flow support in gitea (#14295) * API * Add MirrorUpdated field to Repository API type (#18267) * Adjust Fork API to allow setting a custom repository name (#18066) * Add API to manage repo tranfers (#17963) * Add API to get file commit history (#17652) * Add API to get issue/pull comments and events (timeline) (#17403) * Add API to get/edit wiki (#17278) * Add API for get user org permissions (#17232) * Add HTML urls to notification API (#17178) * Add API to get commit diff/patch (#17095) * Respond with updated notifications in API (#17064) * Add API to fetch git notes (#16649) * Generalize list header for API (#16551) * Add API Token Cache (#16547) * Allow Token API calls be authorized using the reverse-proxy header (#15119) * ENHANCEMENTS * Make the height of the editor in Review Box smaller (4 lines as GitHub) (#18319) * Return nicer error if trying to pull from non-existent user (#18288) * Show pull link for agit pull request also (#18235) * Enable partial clone by default (#18195) * Added replay of webhooks (#18191) * Show OAuth callback error message (#18185) * Increase Salt randomness (#18179) * Add MP4 as default allowed attachment type (#18170) * Include folders into size cost (#18158) * Remove `/email2user` endpoint (#18127) * Handle invalid issues (#18111) * Load EasyMDE/CodeMirror dynamically, remove RequireEasyMDE (#18069) * Support open compare page directly (#17975) * Prefer "Hiragino Kaku Gothic ProN" in system-ui-ja (#17954) * Clean legacy SimpleMDE code (#17926) * Refactor install page (db type) (#17919) * Improve interface when comparing a branch which has created a pull request (#17911) * Allow default branch to be inferred on compare page (#17908) * Display issue/comment role even if repo archived (#17907) * Always set a message-id on mails (#17900) * Change `<a>` elements to underline on hover (#17898) * Render issue references in file table (#17897) * Handle relative unix socket paths (#17836) * Move accessmode into models/perm (#17828) * Fix some org style problems (#17807) * Add List-Unsubscribe header (#17804) * Create menus for organization pages (#17802) * Switch archive URL code back to href attributes (#17796) * Refactor "refs/*" string usage by using constants (#17784) * Allow forks to org if you can create repos (#17783) * Improve install code to avoid low-level mistakes. (#17779) * Improve ellipsis buttons (#17773) * Add restrict and no-user-rc to authorized_keys (#17772) * Add copy Commit ID button in commits list (#17759) * Make `bind` error more readable (#17750) * Fix navbar on project view (#17749) * More pleasantly handle broken or missing git repositories (#17747) * Use `*PushUpdateOptions` as receiver (#17724) * Remove unused `user` paramater (#17723) * Better builtin avatar generator (#17707) * Cleanup and use global style on popups (#17674) * Move user/org deletion to services (#17673) * Added comment for changing issue ref (#17672) * Allow admins to change user avatars (#17661) * Only set `data-path` once for each file in diff pages (#17657) * Add icon to vscode clone link (#17641) * Add download button for file viewer (#17640) * Add pagination to fork list (#17639) * Use a standalone struct name for Organization (#17632) * Minor readability patch. (#17627) * Add context support for GetUserByID (#17602) * Move merge-section to `> .content` (#17582) * Remove NewSession method from db.Engine interface (#17577) * Move unit into models/unit/ (#17576) * Restrict GetDeletedBranchByID to the repositories deleted branches (#17570) * Refactor commentTags functionality (#17558) * Make Repo Code Indexer an Unique Queue (#17515) * Simplify Gothic to use our session store instead of creating a different store (#17507) * Add settings to allow different SMTP envelope from address (#17479) * Properly determine CSV delimiter (#17459) * Hide label comments if labels were added and removed immediately (#17455) * Tune UI alignment for nav bar notification icon, avatar image, issue label (#17438) * Add appearance section in settings (#17433) * Move key forms before list and add cancel button (#17432) * When copying executables to the docker chmod them (#17423) * Remove deprecated `extendDefaultPlugins` method of svgo (#17399) * Fix the click behavior for <tr> and <td> with [data-href] (#17388) * Refactor update checker to use AppState (#17387) * Improve async/await usage, and sort init calls in `index.js` (#17386) * Use a variable but a function for IsProd because of a slight performance increment (#17368) * Frontend refactor, PascalCase to camelCase, remove unused code (#17365) * Hide command line merge instructions when user can't push (#17339) * Move session to models/login (#17338) * Sync gitea app path for git hooks and authorized keys when starting (#17335) * Make the Mirror Queue a queue (#17326) * Add "Copy branch name" button to pull request page (#17323) * Fix repository summary on mobile (#17322) * Split `index.js` to separate files (#17315) * Show direct match on top for user search (#17303) * Frontend refactor: move Vue related code from `index.js` to `components` dir, and remove unused codes. (#17301) * Upgrade chi to v5 (#17298) * Disable form autofill (#17291) * Improve behavior of "Fork" button (#17288) * Open markdown image links in new window (#17287) * Add hints for special Wiki pages (#17283) * Move add deploy key form before the list and add a cancel button (#17228) * Allow adding multiple issues to a project (#17226) * Add metrics to get issues by repository (#17225) * Add specific event type to header (#17222) * Redirect on project after issue created (#17211) * Reference in new issue modal: dont pre-populate issue title (#17208) * Always set a unique Message-ID header (#17206) * Add projects and project boards in exposed metrics (#17202) * Add metrics to get issues by label (#17201) * Add protection to disable Gitea when run as root (#17168) * Don't return binary file changes in raw PR diffs by default (#17158) * Support sorting for project board issuses (#17152) * Force color-adjust for markdown checkboxes (#17146) * Add option to copy line permalink (#17145) * Move twofactor to models/login (#17143) * Multiple tokens support for migrating from github (#17134) * Unify issue and PR subtitles (#17133) * Make Requests Processes and create process hierarchy. Associate OpenRepository with context. (#17125) * Fix problem when database id is not increment as expected (#17124) * Avatar refactor, move avatar code from `models` to `models.avatars`, remove duplicated code (#17123) * Re-allow clipboard copy on non-https sites (#17118) * DBContext is just a Context (#17100) * Move login related structs and functions to models/login (#17093) * Add SkipLocal2FA option to pam and smtp sources (#17078) * Move db related basic functions to models/db (#17075) * Fixes username tagging in "Reference in new issue" (#17074) * Use light/dark theme based on system preference (#17051) * Always emit the configuration path (#17036) * Add `AbsoluteListOptions` (#17028) * Use common sessioner for API and Web (#17027) * Fix overflow label in small view (#17020) * Report the associated filter if there is an error in LDAP (#17014) * Add "new issue" btn on project (#17001) * Add doctor dbconsistency check for release and attachment (#16978) * Disable Fomantic's CSS tooltips (#16974) * Add Cache-Control to avatar redirects (#16973) * Make mirror feature more configurable (#16957) * Add skip and limit to git.GetTags (#16897) * Remove ParseQueueConnStr as it is unused (#16878) * Remove unused Fomantic sidebar module (#16853) * Allow LDAP Sources to provide Avatars (#16851) * Remove Dashboard/Home button from the navbar (#16844) * Use conditions but not repo ids as query condition (#16839) * Add user settings key/value DB table (#16834) * Add buttons to allow loading of incomplete diffs (#16829) * Add information for migrate failure (#16803) * Add EdDSA JWT signing algorithm (#16786) * Add user status filter to admin user management page (#16770) * Add Option to synchronize Admin & Restricted states from OIDC/OAuth2 along with Setting Scopes (#16766) * Do not use thin scrollbars on Firefox (#16738) * Download LFS in git and web workflow from minio/s3 directly (SERVE_DIRECT) (#16731) * Compute proper foreground color for labels (#16729) * Add edit button to wiki sidebar and footer (#16719) * Fix migration svg color (#16715) * Add link to vscode to repo header (#16664) * Add filter by owner and team to issue/pulls search endpoint (#16662) * Kanban colored boards (#16647) * Allow setting X-FRAME-OPTIONS (#16643) * Separate open and closed issue in metrics (#16637) * Support direct comparison (git diff a..b) as well merge comparison (a…b) (#16635) * Add setting to OAuth handlers to skip local 2FA authentication (#16594) * Make PR merge options more intuitive (#16582) * Show correct text when comparing commits on empty pull request (#16569) * Pre-fill suggested New File 'name' and 'content' with Query Params (#16556) * Add an abstract json layout to make it's easier to change json library (#16528) * Make Mermaid.js limit configurable (#16519) * Improve 2FA autofill (#16473) * Add modals to Organization and Team remove/leave (#16471) * Show tag name on dashboard items list (#16466) * Change default cron schedules from @every 24h to @midnight (#16431) * Prevent double sanitize (#16386) * Replace `list.List` with slices (#16311) * Add configuration option to restrict users by default (#16256) * Move login out of models (#16199) * Support pagination of organizations on user settings pages (#16083) * Switch migration icon to svg (#15954) * Add left padding for chunk header of split diff view (#13397) * Allow U2F 2FA without TOTP (#11573) * BUGFIXES * GitLab reviews may not have the updated_at field set (#18450) (#18461) * Fix detection of no commits when the default branch is not master (#18422) (#18423) * Fix broken oauth2 authentication source edit page (#18412) (#18419) * Place inline diff comment dialogs on split diff in 4th and 8th columns (#18403) (#18404) * Fix restore without topic failure (#18387) (#18400) * Fix commit's time (#18375) (#18392) * Fix partial cloning a repo (#18373) (#18377) * Stop trimming preceding and suffixing spaces from editor filenames (#18334) * Prevent showing webauthn error for every time visiting `/user/settings/security` (#18386) * Fix mime-type detection for HTTP server (#18370) (#18371) * Stop trimming preceding and suffixing spaces from editor filenames (#18334) * Restore propagation of ErrDependenciesLeft (#18325) * Fix PR comments UI (#18323) * Use indirect comparison when showing pull requests (#18313) * Replace satori/go.uuid with gofrs/uuid (#18311) * Fix commit links on compare page (#18310) * Don't show double error response in git hook (#18292) * Handle missing default branch better in owner/repo/branches page (#18290) * Fix CheckRepoStats and reuse it during migration (#18264) * Prevent underline hover on cards (#18259) * Don't delete branch if other PRs with this branch are open (#18164) * Require codereview to have content (#18156) * Allow admin to associate missing LFS objects for repositories (#18143) * When attempting to subscribe other user to issue report why access denied (#18091) * Add option to convert CRLF to LF line endings for sendmail (#18075) * Only create pprof files for gitea serv if explicitly asked for (#18068) * Abort merge if head has been updated before pressing merge (#18032) * Improve TestPatch to use git read-tree -m and implement git-merge-one-file functionality (#18004) * Use JSON module instead of stdlib json (#18003) * Fixed issue merged/closed wording (#17973) * Return nicer error for ForcePrivate (#17971) * Fix overflow in commit graph (#17947) * Prevent services/mailer/mailer_test.go tests from deleteing data directory (#17941) * Use disable_form_autofill on Codebase and Gitbucket (#17936) * Fix a panic in NotifyCreateIssueComment (caused by string truncation) (#17928) * Fix markdown URL parsing (#17924) * Apply CSS Variables to all message elements (#17920) * Improve checkBranchName (#17901) * Update chi/middleware to chi/v5/middleware (#17888) * Fix position of label color picker colors (#17866) * Fix ListUnadoptedRepositories incorrect total count (#17865) * Remove whitespace inside rendered code `<td>` (#17859) * Make Co-committed-by and co-authored-by trailers optional (#17848) * Fix value of User.IsRestricted when oauth2 user registration (#17839) * Use new OneDev /milestones endpoint (#17782) * Prevent deadlock in TestPersistableChannelQueue (#17717) * Simplify code for writing SHA to name-rev (#17696) * Fix database deadlock when update issue labels (#17649) * Add warning for BIDI characters in page renders and in diffs (#17562) * Fix ipv6 parsing for builtin ssh server (#17561) * Multiple Escaping Improvements (#17551) * Fixes #16559 - Do not trim leading spaces for tab delimited (#17442) * Show client-side error if wiki page is empty (#17415) * Fix context popup error (#17398) * Stop sanitizing full name in API (#17396) * Fix issue close/comment buttons on mobile (#17317) * Fix navbar UI (#17235) * Fix problem when database id is not increment as expected (#17229) * Open the DingTalk link in browser (#17084) * Remove heads pointing to missing old refs (#17076) * Fix commit status index problem (#17061) * Handle broken references in mirror sync (#17013) * Fix for create repo page layout (#17012) * Improve LDAP synchronization efficiency (#16994) * Add repo_id for attachment (#16958) * Clean-up HookPreReceive and restore functionality for pushing non-standard refs (#16705) * Remove duplicate csv import in modules/csv/csv.go (#16631) * Improve SMTP authentication and Fix user creation bugs (#16612) * Fixed emoji alias not parsed in links (#16221) * Calculate label URL on API (#16186) * TRANSLATION * Fix mispelling of starred as stared (#17465) * Re-separate the color translation strings (#17390) * Enable Malayalam, Greek, Persian, Hungarian & Indonesian by default (#16998) * BUILD * Add lockfile-check (#18285) * Don't store assets modified time into generated files (#18193) * Use shadowing script for docker (#17846) * MISC * Update JS dependencies (#17611) Signed-off-by: Andrew Thornton <art27@cantab.net>

* Previously, `GET {username}/{reponame}/raw///file-path` (the middle two slashes are blank to get the default branch) when the repo name has uppercase letters, e.g., https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware, using a lowercase version of the name redirected to the correct URL * In other words both * `GET https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware/raw///images/back.png` * `GET https://try.gitea.io/AbdulrhmnGhanem/ch330_hardware/raw///images/back.png` were redirecting to ` GET https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware/raw/branch/master/images/back.png` This isn't the case after go-gitea#17551. Specifically because of this [line](https://github.com/zeripath/gitea/blob/cbd5eecd148dfca5fcb1a3da469e491a84f6b32b/modules/context/repo.go#L860).

* Previously, `GET {username}/{reponame}/raw///file-path` (the middle two slashes are blank to get the default branch) when the repo name has uppercase letters, e.g., https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware, using a lowercase version of the name redirected to the correct URL * In other words both * `GET https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware/raw///images/back.png` * `GET https://try.gitea.io/AbdulrhmnGhanem/ch330_hardware/raw///images/back.png` were redirecting to ` GET https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware/raw/branch/master/images/back.png` This isn't the case after #17551. Specifically because of this [line](https://github.com/zeripath/gitea/blob/cbd5eecd148dfca5fcb1a3da469e491a84f6b32b/modules/context/repo.go#L860).

* Previously, `GET {username}/{reponame}/raw///file-path` (the middle two slashes are blank to get the default branch) when the repo name has uppercase letters, e.g., https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware, using a lowercase version of the name redirected to the correct URL * In other words both * `GET https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware/raw///images/back.png` * `GET https://try.gitea.io/AbdulrhmnGhanem/ch330_hardware/raw///images/back.png` were redirecting to ` GET https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware/raw/branch/master/images/back.png` This isn't the case after go-gitea#17551. Specifically because of this [line](https://github.com/zeripath/gitea/blob/cbd5eecd148dfca5fcb1a3da469e491a84f6b32b/modules/context/repo.go#L860).

* Previously, `GET {username}/{reponame}/raw///file-path` (the middle two slashes are blank to get the default branch) when the repo name has uppercase letters, e.g., https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware, using a lowercase version of the name redirected to the correct URL * In other words both * `GET https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware/raw///images/back.png` * `GET https://try.gitea.io/AbdulrhmnGhanem/ch330_hardware/raw///images/back.png` were redirecting to ` GET https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware/raw/branch/master/images/back.png` This isn't the case after #17551. Specifically because of this [line](https://github.com/zeripath/gitea/blob/cbd5eecd148dfca5fcb1a3da469e491a84f6b32b/modules/context/repo.go#L860). Co-authored-by: Ghanem <37152329+AbdulrhmnGhanem@users.noreply.github.com>

There are multiple places where Gitea does not properly escape URLs that it is building and there are multiple places where it builds urls when there is already a simpler function available to use this. This is an extensive PR attempting to fix these issues. 1. The first commit in this PR looks through all href, src and links in the Gitea codebase and has attempted to catch all the places where there is potentially incomplete escaping. 2. Whilst doing this we will prefer to use functions that create URLs over recreating them by hand. 3. All uses of strings should be directly escaped - even if they are not currently expected to contain escaping characters. The main benefit to doing this will be that we can consider relaxing the constraints on user names and reponames in future. 4. The next commit looks at escaping in the wiki and re-considers the urls that are used there. Using the improved escaping here wiki files containing '/'. (This implementation will currently still place all of the wiki files the root directory of the repo but this would not be difficult to change.) 5. The title generation in feeds is now properly escaped. 6. EscapePound is no longer needed - urls should be PathEscaped / QueryEscaped as necessary but then re-escaped with Escape when creating html with locales Signed-off-by: Andrew Thornton <art27@cantab.net> Signed-off-by: Andrew Thornton <art27@cantab.net>

…itea#17697)

…and other fixes (go-gitea#17991) This PR contains multiple fixes. The most important of which is: * Prevent hang in git cat-file if the repository is not a valid repository Unfortunately it appears that if git cat-file is run in an invalid repository it will hang until stdin is closed. This will result in deadlocked /pulls pages and dangling git cat-file calls if a broken repository is tried to be reviewed or pulls exists for a broken repository. Fix go-gitea#14734 Fix go-gitea#9271 Fix go-gitea#16113 Otherwise there are a few small other fixes included which this PR was initially intending to fix: * Fix panic on partial compares due to missing PullRequestWorkInProgressPrefixes * Fix links on pulls pages due to regression from go-gitea#17551 - by making most /issues routes match /pulls too - Fix go-gitea#17983 * Fix links on feeds pages due to another regression from go-gitea#17551 but also fix issue with syncing tags - Fix go-gitea#17943 * Add missing locale entries for oauth group claims * Prevent NPEs if ColorFormat is called on nil users, repos or teams.

@midnight

Frontport go-gitea#18468 Frontport changelog for 1.16, frontport 1.15.11 changelog and update config.yaml ## [1.16.0](https://github.com/go-gitea/gitea/releases/tag/v1.16.0) - 2022-01-30 * BREAKING * Remove golang vendored directory (go-gitea#18277) * Paginate releases page & set default page size to 10 (go-gitea#16857) * Only allow webhook to send requests to allowed hosts (go-gitea#17482) * SECURITY * Disable content sniffing on `PlainTextBytes` (go-gitea#18359) (go-gitea#18365) * Only view milestones from current repo (go-gitea#18414) (go-gitea#18417) * Sanitize user-input on file name (go-gitea#17666) * Use `hostmatcher` to replace `matchlist` to improve blocking of bad hosts in Webhooks (go-gitea#17605) * FEATURES * Add/update SMTP auth providers via cli (go-gitea#18197) * Support webauthn (go-gitea#17957) * Team permission allow different unit has different permission (go-gitea#17811) * Implement Well-Known URL for password change (go-gitea#17777) * Add support for ssh commit signing (go-gitea#17743) * Allow Loading of Diffs that are too large (go-gitea#17739) * Add copy button to markdown code blocks (go-gitea#17638) * Add .gitattribute assisted language detection to blame, diff and render (go-gitea#17590) * Add `PULL_LIMIT` and `PUSH_LIMIT` to cron.update_mirror task (go-gitea#17568) * Add Reindex buttons to repository settings page (go-gitea#17494) * Make SSL cipher suite configurable (go-gitea#17440) * Add groups scope/claim to OIDC/OAuth2 Provider (go-gitea#17367) * Add simple update checker to Gitea (go-gitea#17212) * Migrated Repository will show modifications when possible (go-gitea#17191) * Create pub/priv keypair for federation (go-gitea#17071) * Make LDAP be able to skip local 2FA (go-gitea#16954) * Add nodeinfo endpoint for federation purposes (go-gitea#16953) * Save and view issue/comment content history (go-gitea#16909) * Use git attributes to determine generated and vendored status for language stats and diffs (go-gitea#16773) * Add migrate from Codebase (go-gitea#16768) * Add migration from GitBucket (go-gitea#16767) * Add OAuth2 introspection endpoint (go-gitea#16752) * Add proxy settings and support for migration and webhook (go-gitea#16704) * Add microsoft oauth2 providers (go-gitea#16544) * Send registration email on user autoregistration (go-gitea#16523) * Defer Last Commit Info (go-gitea#16467) * Support unprotected file patterns (go-gitea#16395) * Add migrate from OneDev (go-gitea#16356) * Add option to update pull request by `rebase` (go-gitea#16125) * Add RSS/Atom feed support for user actions (go-gitea#16002) * Add support for corporate WeChat webhooks (go-gitea#15910) * Add a simple way to rename branch like gh (go-gitea#15870) * Add bundle download for repository (go-gitea#14538) * Add agit flow support in gitea (go-gitea#14295) * API * Add MirrorUpdated field to Repository API type (go-gitea#18267) * Adjust Fork API to allow setting a custom repository name (go-gitea#18066) * Add API to manage repo tranfers (go-gitea#17963) * Add API to get file commit history (go-gitea#17652) * Add API to get issue/pull comments and events (timeline) (go-gitea#17403) * Add API to get/edit wiki (go-gitea#17278) * Add API for get user org permissions (go-gitea#17232) * Add HTML urls to notification API (go-gitea#17178) * Add API to get commit diff/patch (go-gitea#17095) * Respond with updated notifications in API (go-gitea#17064) * Add API to fetch git notes (go-gitea#16649) * Generalize list header for API (go-gitea#16551) * Add API Token Cache (go-gitea#16547) * Allow Token API calls be authorized using the reverse-proxy header (go-gitea#15119) * ENHANCEMENTS * Make the height of the editor in Review Box smaller (4 lines as GitHub) (go-gitea#18319) * Return nicer error if trying to pull from non-existent user (go-gitea#18288) * Show pull link for agit pull request also (go-gitea#18235) * Enable partial clone by default (go-gitea#18195) * Added replay of webhooks (go-gitea#18191) * Show OAuth callback error message (go-gitea#18185) * Increase Salt randomness (go-gitea#18179) * Add MP4 as default allowed attachment type (go-gitea#18170) * Include folders into size cost (go-gitea#18158) * Remove `/email2user` endpoint (go-gitea#18127) * Handle invalid issues (go-gitea#18111) * Load EasyMDE/CodeMirror dynamically, remove RequireEasyMDE (go-gitea#18069) * Support open compare page directly (go-gitea#17975) * Prefer "Hiragino Kaku Gothic ProN" in system-ui-ja (go-gitea#17954) * Clean legacy SimpleMDE code (go-gitea#17926) * Refactor install page (db type) (go-gitea#17919) * Improve interface when comparing a branch which has created a pull request (go-gitea#17911) * Allow default branch to be inferred on compare page (go-gitea#17908) * Display issue/comment role even if repo archived (go-gitea#17907) * Always set a message-id on mails (go-gitea#17900) * Change `<a>` elements to underline on hover (go-gitea#17898) * Render issue references in file table (go-gitea#17897) * Handle relative unix socket paths (go-gitea#17836) * Move accessmode into models/perm (go-gitea#17828) * Fix some org style problems (go-gitea#17807) * Add List-Unsubscribe header (go-gitea#17804) * Create menus for organization pages (go-gitea#17802) * Switch archive URL code back to href attributes (go-gitea#17796) * Refactor "refs/*" string usage by using constants (go-gitea#17784) * Allow forks to org if you can create repos (go-gitea#17783) * Improve install code to avoid low-level mistakes. (go-gitea#17779) * Improve ellipsis buttons (go-gitea#17773) * Add restrict and no-user-rc to authorized_keys (go-gitea#17772) * Add copy Commit ID button in commits list (go-gitea#17759) * Make `bind` error more readable (go-gitea#17750) * Fix navbar on project view (go-gitea#17749) * More pleasantly handle broken or missing git repositories (go-gitea#17747) * Use `*PushUpdateOptions` as receiver (go-gitea#17724) * Remove unused `user` paramater (go-gitea#17723) * Better builtin avatar generator (go-gitea#17707) * Cleanup and use global style on popups (go-gitea#17674) * Move user/org deletion to services (go-gitea#17673) * Added comment for changing issue ref (go-gitea#17672) * Allow admins to change user avatars (go-gitea#17661) * Only set `data-path` once for each file in diff pages (go-gitea#17657) * Add icon to vscode clone link (go-gitea#17641) * Add download button for file viewer (go-gitea#17640) * Add pagination to fork list (go-gitea#17639) * Use a standalone struct name for Organization (go-gitea#17632) * Minor readability patch. (go-gitea#17627) * Add context support for GetUserByID (go-gitea#17602) * Move merge-section to `> .content` (go-gitea#17582) * Remove NewSession method from db.Engine interface (go-gitea#17577) * Move unit into models/unit/ (go-gitea#17576) * Restrict GetDeletedBranchByID to the repositories deleted branches (go-gitea#17570) * Refactor commentTags functionality (go-gitea#17558) * Make Repo Code Indexer an Unique Queue (go-gitea#17515) * Simplify Gothic to use our session store instead of creating a different store (go-gitea#17507) * Add settings to allow different SMTP envelope from address (go-gitea#17479) * Properly determine CSV delimiter (go-gitea#17459) * Hide label comments if labels were added and removed immediately (go-gitea#17455) * Tune UI alignment for nav bar notification icon, avatar image, issue label (go-gitea#17438) * Add appearance section in settings (go-gitea#17433) * Move key forms before list and add cancel button (go-gitea#17432) * When copying executables to the docker chmod them (go-gitea#17423) * Remove deprecated `extendDefaultPlugins` method of svgo (go-gitea#17399) * Fix the click behavior for <tr> and <td> with [data-href] (go-gitea#17388) * Refactor update checker to use AppState (go-gitea#17387) * Improve async/await usage, and sort init calls in `index.js` (go-gitea#17386) * Use a variable but a function for IsProd because of a slight performance increment (go-gitea#17368) * Frontend refactor, PascalCase to camelCase, remove unused code (go-gitea#17365) * Hide command line merge instructions when user can't push (go-gitea#17339) * Move session to models/login (go-gitea#17338) * Sync gitea app path for git hooks and authorized keys when starting (go-gitea#17335) * Make the Mirror Queue a queue (go-gitea#17326) * Add "Copy branch name" button to pull request page (go-gitea#17323) * Fix repository summary on mobile (go-gitea#17322) * Split `index.js` to separate files (go-gitea#17315) * Show direct match on top for user search (go-gitea#17303) * Frontend refactor: move Vue related code from `index.js` to `components` dir, and remove unused codes. (go-gitea#17301) * Upgrade chi to v5 (go-gitea#17298) * Disable form autofill (go-gitea#17291) * Improve behavior of "Fork" button (go-gitea#17288) * Open markdown image links in new window (go-gitea#17287) * Add hints for special Wiki pages (go-gitea#17283) * Move add deploy key form before the list and add a cancel button (go-gitea#17228) * Allow adding multiple issues to a project (go-gitea#17226) * Add metrics to get issues by repository (go-gitea#17225) * Add specific event type to header (go-gitea#17222) * Redirect on project after issue created (go-gitea#17211) * Reference in new issue modal: dont pre-populate issue title (go-gitea#17208) * Always set a unique Message-ID header (go-gitea#17206) * Add projects and project boards in exposed metrics (go-gitea#17202) * Add metrics to get issues by label (go-gitea#17201) * Add protection to disable Gitea when run as root (go-gitea#17168) * Don't return binary file changes in raw PR diffs by default (go-gitea#17158) * Support sorting for project board issuses (go-gitea#17152) * Force color-adjust for markdown checkboxes (go-gitea#17146) * Add option to copy line permalink (go-gitea#17145) * Move twofactor to models/login (go-gitea#17143) * Multiple tokens support for migrating from github (go-gitea#17134) * Unify issue and PR subtitles (go-gitea#17133) * Make Requests Processes and create process hierarchy. Associate OpenRepository with context. (go-gitea#17125) * Fix problem when database id is not increment as expected (go-gitea#17124) * Avatar refactor, move avatar code from `models` to `models.avatars`, remove duplicated code (go-gitea#17123) * Re-allow clipboard copy on non-https sites (go-gitea#17118) * DBContext is just a Context (go-gitea#17100) * Move login related structs and functions to models/login (go-gitea#17093) * Add SkipLocal2FA option to pam and smtp sources (go-gitea#17078) * Move db related basic functions to models/db (go-gitea#17075) * Fixes username tagging in "Reference in new issue" (go-gitea#17074) * Use light/dark theme based on system preference (go-gitea#17051) * Always emit the configuration path (go-gitea#17036) * Add `AbsoluteListOptions` (go-gitea#17028) * Use common sessioner for API and Web (go-gitea#17027) * Fix overflow label in small view (go-gitea#17020) * Report the associated filter if there is an error in LDAP (go-gitea#17014) * Add "new issue" btn on project (go-gitea#17001) * Add doctor dbconsistency check for release and attachment (go-gitea#16978) * Disable Fomantic's CSS tooltips (go-gitea#16974) * Add Cache-Control to avatar redirects (go-gitea#16973) * Make mirror feature more configurable (go-gitea#16957) * Add skip and limit to git.GetTags (go-gitea#16897) * Remove ParseQueueConnStr as it is unused (go-gitea#16878) * Remove unused Fomantic sidebar module (go-gitea#16853) * Allow LDAP Sources to provide Avatars (go-gitea#16851) * Remove Dashboard/Home button from the navbar (go-gitea#16844) * Use conditions but not repo ids as query condition (go-gitea#16839) * Add user settings key/value DB table (go-gitea#16834) * Add buttons to allow loading of incomplete diffs (go-gitea#16829) * Add information for migrate failure (go-gitea#16803) * Add EdDSA JWT signing algorithm (go-gitea#16786) * Add user status filter to admin user management page (go-gitea#16770) * Add Option to synchronize Admin & Restricted states from OIDC/OAuth2 along with Setting Scopes (go-gitea#16766) * Do not use thin scrollbars on Firefox (go-gitea#16738) * Download LFS in git and web workflow from minio/s3 directly (SERVE_DIRECT) (go-gitea#16731) * Compute proper foreground color for labels (go-gitea#16729) * Add edit button to wiki sidebar and footer (go-gitea#16719) * Fix migration svg color (go-gitea#16715) * Add link to vscode to repo header (go-gitea#16664) * Add filter by owner and team to issue/pulls search endpoint (go-gitea#16662) * Kanban colored boards (go-gitea#16647) * Allow setting X-FRAME-OPTIONS (go-gitea#16643) * Separate open and closed issue in metrics (go-gitea#16637) * Support direct comparison (git diff a..b) as well merge comparison (a…b) (go-gitea#16635) * Add setting to OAuth handlers to skip local 2FA authentication (go-gitea#16594) * Make PR merge options more intuitive (go-gitea#16582) * Show correct text when comparing commits on empty pull request (go-gitea#16569) * Pre-fill suggested New File 'name' and 'content' with Query Params (go-gitea#16556) * Add an abstract json layout to make it's easier to change json library (go-gitea#16528) * Make Mermaid.js limit configurable (go-gitea#16519) * Improve 2FA autofill (go-gitea#16473) * Add modals to Organization and Team remove/leave (go-gitea#16471) * Show tag name on dashboard items list (go-gitea#16466) * Change default cron schedules from @every 24h to @midnight (go-gitea#16431) * Prevent double sanitize (go-gitea#16386) * Replace `list.List` with slices (go-gitea#16311) * Add configuration option to restrict users by default (go-gitea#16256) * Move login out of models (go-gitea#16199) * Support pagination of organizations on user settings pages (go-gitea#16083) * Switch migration icon to svg (go-gitea#15954) * Add left padding for chunk header of split diff view (go-gitea#13397) * Allow U2F 2FA without TOTP (go-gitea#11573) * BUGFIXES * GitLab reviews may not have the updated_at field set (go-gitea#18450) (go-gitea#18461) * Fix detection of no commits when the default branch is not master (go-gitea#18422) (go-gitea#18423) * Fix broken oauth2 authentication source edit page (go-gitea#18412) (go-gitea#18419) * Place inline diff comment dialogs on split diff in 4th and 8th columns (go-gitea#18403) (go-gitea#18404) * Fix restore without topic failure (go-gitea#18387) (go-gitea#18400) * Fix commit's time (go-gitea#18375) (go-gitea#18392) * Fix partial cloning a repo (go-gitea#18373) (go-gitea#18377) * Stop trimming preceding and suffixing spaces from editor filenames (go-gitea#18334) * Prevent showing webauthn error for every time visiting `/user/settings/security` (go-gitea#18386) * Fix mime-type detection for HTTP server (go-gitea#18370) (go-gitea#18371) * Stop trimming preceding and suffixing spaces from editor filenames (go-gitea#18334) * Restore propagation of ErrDependenciesLeft (go-gitea#18325) * Fix PR comments UI (go-gitea#18323) * Use indirect comparison when showing pull requests (go-gitea#18313) * Replace satori/go.uuid with gofrs/uuid (go-gitea#18311) * Fix commit links on compare page (go-gitea#18310) * Don't show double error response in git hook (go-gitea#18292) * Handle missing default branch better in owner/repo/branches page (go-gitea#18290) * Fix CheckRepoStats and reuse it during migration (go-gitea#18264) * Prevent underline hover on cards (go-gitea#18259) * Don't delete branch if other PRs with this branch are open (go-gitea#18164) * Require codereview to have content (go-gitea#18156) * Allow admin to associate missing LFS objects for repositories (go-gitea#18143) * When attempting to subscribe other user to issue report why access denied (go-gitea#18091) * Add option to convert CRLF to LF line endings for sendmail (go-gitea#18075) * Only create pprof files for gitea serv if explicitly asked for (go-gitea#18068) * Abort merge if head has been updated before pressing merge (go-gitea#18032) * Improve TestPatch to use git read-tree -m and implement git-merge-one-file functionality (go-gitea#18004) * Use JSON module instead of stdlib json (go-gitea#18003) * Fixed issue merged/closed wording (go-gitea#17973) * Return nicer error for ForcePrivate (go-gitea#17971) * Fix overflow in commit graph (go-gitea#17947) * Prevent services/mailer/mailer_test.go tests from deleteing data directory (go-gitea#17941) * Use disable_form_autofill on Codebase and Gitbucket (go-gitea#17936) * Fix a panic in NotifyCreateIssueComment (caused by string truncation) (go-gitea#17928) * Fix markdown URL parsing (go-gitea#17924) * Apply CSS Variables to all message elements (go-gitea#17920) * Improve checkBranchName (go-gitea#17901) * Update chi/middleware to chi/v5/middleware (go-gitea#17888) * Fix position of label color picker colors (go-gitea#17866) * Fix ListUnadoptedRepositories incorrect total count (go-gitea#17865) * Remove whitespace inside rendered code `<td>` (go-gitea#17859) * Make Co-committed-by and co-authored-by trailers optional (go-gitea#17848) * Fix value of User.IsRestricted when oauth2 user registration (go-gitea#17839) * Use new OneDev /milestones endpoint (go-gitea#17782) * Prevent deadlock in TestPersistableChannelQueue (go-gitea#17717) * Simplify code for writing SHA to name-rev (go-gitea#17696) * Fix database deadlock when update issue labels (go-gitea#17649) * Add warning for BIDI characters in page renders and in diffs (go-gitea#17562) * Fix ipv6 parsing for builtin ssh server (go-gitea#17561) * Multiple Escaping Improvements (go-gitea#17551) * Fixes go-gitea#16559 - Do not trim leading spaces for tab delimited (go-gitea#17442) * Show client-side error if wiki page is empty (go-gitea#17415) * Fix context popup error (go-gitea#17398) * Stop sanitizing full name in API (go-gitea#17396) * Fix issue close/comment buttons on mobile (go-gitea#17317) * Fix navbar UI (go-gitea#17235) * Fix problem when database id is not increment as expected (go-gitea#17229) * Open the DingTalk link in browser (go-gitea#17084) * Remove heads pointing to missing old refs (go-gitea#17076) * Fix commit status index problem (go-gitea#17061) * Handle broken references in mirror sync (go-gitea#17013) * Fix for create repo page layout (go-gitea#17012) * Improve LDAP synchronization efficiency (go-gitea#16994) * Add repo_id for attachment (go-gitea#16958) * Clean-up HookPreReceive and restore functionality for pushing non-standard refs (go-gitea#16705) * Remove duplicate csv import in modules/csv/csv.go (go-gitea#16631) * Improve SMTP authentication and Fix user creation bugs (go-gitea#16612) * Fixed emoji alias not parsed in links (go-gitea#16221) * Calculate label URL on API (go-gitea#16186) * TRANSLATION * Fix mispelling of starred as stared (go-gitea#17465) * Re-separate the color translation strings (go-gitea#17390) * Enable Malayalam, Greek, Persian, Hungarian & Indonesian by default (go-gitea#16998) * BUILD * Add lockfile-check (go-gitea#18285) * Don't store assets modified time into generated files (go-gitea#18193) * Use shadowing script for docker (go-gitea#17846) * MISC * Update JS dependencies (go-gitea#17611) Signed-off-by: Andrew Thornton <art27@cantab.net>

* Previously, `GET {username}/{reponame}/raw///file-path` (the middle two slashes are blank to get the default branch) when the repo name has uppercase letters, e.g., https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware, using a lowercase version of the name redirected to the correct URL * In other words both * `GET https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware/raw///images/back.png` * `GET https://try.gitea.io/AbdulrhmnGhanem/ch330_hardware/raw///images/back.png` were redirecting to ` GET https://try.gitea.io/AbdulrhmnGhanem/CH330_Hardware/raw/branch/master/images/back.png` This isn't the case after go-gitea#17551. Specifically because of this [line](https://github.com/zeripath/gitea/blob/cbd5eecd148dfca5fcb1a3da469e491a84f6b32b/modules/context/repo.go#L860).

zeripath added 2 commits November 4, 2021 22:45

Add escaping and deeper URLs into wiki

03ffce5

One of the major limitations of the wiki is that it does not currently allow a hierarchy of folders. Here we add all of the escaping to allow this to be instituted. Signed-off-by: Andrew Thornton <art27@cantab.net>

zeripath added type/bug type/enhancement An improvement of existing functionality pr/wip This PR is not ready for review labels Nov 4, 2021

zeripath changed the title ~~Multiple Escaping Improvements~~ WIP: Multiple Escaping Improvements Nov 4, 2021

zeripath added 4 commits November 5, 2021 09:58

Ensure that links and titles in feeds are correctly escaped

d563cd2

Signed-off-by: Andrew Thornton <art27@cantab.net>

Merge remote-tracking branch 'origin/main' into really-escape

684851b

placate lint

9313c4c

Signed-off-by: Andrew Thornton <art27@cantab.net>

Remove EscapePound as Escape and PathEscape/PathEscapeSegments should…

59100b8

… be used instead Signed-off-by: Andrew Thornton <art27@cantab.net>

GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Nov 5, 2021

zeripath added 2 commits November 5, 2021 10:34

Remove last example of url building in locale

b96b85e

Signed-off-by: Andrew Thornton <art27@cantab.net>

More escaping

b266534

Signed-off-by: Andrew Thornton <art27@cantab.net>

lunny reviewed Nov 5, 2021

View reviewed changes

models/action.go Show resolved Hide resolved

zeripath added 2 commits November 5, 2021 12:11

fix bug on parentpath

d4c18fb

Signed-off-by: Andrew Thornton <art27@cantab.net>

Merge remote-tracking branch 'origin/main' into really-escape

c519b12

zeripath changed the title ~~WIP: Multiple Escaping Improvements~~ Multiple Escaping Improvements Nov 5, 2021

zeripath removed the pr/wip This PR is not ready for review label Nov 5, 2021