Add LDAP group sync to Org teams #1

svenseeberg · 2021-05-30T11:51:59Z

This is work in progress to solve go-gitea#1395

What does work:

Input fields in login source creation and edit template
Reading configuration from database
Saving settings to database

What does not yet work (see fad23d7629b4e080eeb055c354a06098e2659713):

hook functions into existing code (continuous sync & login)
function to retrieve groups from LDAP (Get all entries of memberOf attribute for a user. The attribute set in "Group Attribute Containing List Of Users" should/could be used here.)
function to map LDAP groups to teams (parse JSON in settings and return org+team for each LDAP group membership)
function to add user to teams (add user to team in org, see https://github.com/netzbegruenung/gitea/blob/main/models/org_team.go#L190)
function to remove user from teams (see 615f70583571deb64ba7cafaca16326e765661cb)

Ideas for improvement

Move configuration into expandable LDAP group filter section (see 7cf4982fe8a556591c51e124100cbef7a556c54d)

Additional hint

In the moment LDAP groups sync only to organizations teams, that already exists.
LDAP group sync does not create new organizations or teams

Fix go-gitea#15932 Signed-off-by: Andrew Thornton <art27@cantab.net>

Use async clipboard api [1] over this dependency, saving around 10kB bundle size before minify while delivering the same functionality. The issue comment button works but does not have a popup indication. We could add some toast-style notifications in the future to fix that but I think it's out of scope of this PR. [1] https://developer.mozilla.org/en-US/docs/Web/API/Clipboard/writeText

* Make modules/context.Context a context.Context Signed-off-by: Andrew Thornton <art27@cantab.net> * Simplify context calls Signed-off-by: Andrew Thornton <art27@cantab.net> * Set the base context for requests to the HammerContext Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>

Unfortunately go panics you try to cast a nil interface{} as another primitive therefore you need to check interfaces are not nil before casting. Fix go-gitea#16025 Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: techknowlogick <techknowlogick@gitea.io>

* encrypt migration credentials in task persistence Not sure this is the best approach, we could encrypt the entire `PayloadContent` instead. Also instead of clearing individual fields in payload content, we could just delete the task once it has (successfully) finished..? * remove credentials of past migrations * only run DB migration for completed tasks * fix binding * add omitempty * never serialize unencrypted credentials * fix import order Co-authored-by: techknowlogick <techknowlogick@gitea.io> Co-authored-by: zeripath <art27@cantab.net> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>

This PR simply hides mirror passwords from being displayed on the repo settings page. Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: techknowlogick <techknowlogick@gitea.io>

…6037) The i_like_gitea cookie appears to be missing the SameSite settings. I think they were present at some point but may have been removed in a merge. This PR ensures that they are set. Fix go-gitea#15972 Signed-off-by: Andrew Thornton <art27@cantab.net>

* Make sshd_config more flexible regarding MaxStartups and MaxSessions. See https://man.openbsd.org/sshd_config for more information. * make property prefix equals other existing Gitea SSH properties. Co-authored-by: dlouzado <dlouzado@senado.leg.br>

* update ``.raw-content`` when edit issue/comment content fix go-gitea#16000 Signed-off-by: a1012112796 <1012112796@qq.com> * handle empty content Co-authored-by: techknowlogick <techknowlogick@gitea.io>

Signed-off-by: a1012112796 <1012112796@qq.com>

Unforunately go-gitea#16009 makes these settings mandatory. This PR uses the same technique as used for the certificates to make these settings non-mandatory. Fix go-gitea#16044 Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: 6543 <6543@obermui.de>

If you change the case of a username the change needs to be propagated to their repositories. Signed-off-by: Andrew Thornton <art27@cantab.net>

* Added type sniffer. * Switched content detection from base to typesniffer. * Added GuessContentType to Blob. * Moved image info logic to client. Added support for SVG images in diff. * Restore old blocked svg behaviour. * Added missing image formats. * Execute image diff only when container is visible. * add margin to spinner * improve BIN tag on image diffs * Default to render view. * Show image diff on incomplete diff. Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Lauris BH <lauris@nix.lv>

Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv>

There is an incorrect casting in the wrapped queue. Fix go-gitea#16071 Signed-off-by: Andrew Thornton <art27@cantab.net>

Signed-off-by: Steven Kriegler <61625851+justusbunsi@users.noreply.github.com>

* Restructured code. Moved static checks out of loop. * Restructured batch api. Add support for individual errors. * Let router decide if LFS is enabled. * Renamed methods. * Return correct status from verify handler. * Unified media type check in router. * Changed error code according to spec. * Moved checks into router. * Removed invalid v1 api methods. * Unified methods. * Display better error messages. * Added size parameter. Create meta object on upload. * Use object error on invalid size. * Skip upload if object exists. * Moved methods. * Suppress fields in response. * Changed error on accept. * Added tests. * Use ErrorResponse object. * Test against message property. * Add support for the old invalid lfs client. * Fixed the check because MinIO wraps the error. * Use individual repositories. Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv>

* Fix regression of renderer * Fix render setting load twice bug

* Make modules/context.Context a context.Context Signed-off-by: Andrew Thornton <art27@cantab.net> * Simplify context calls Signed-off-by: Andrew Thornton <art27@cantab.net> * Set the base context for requests to the HammerContext Signed-off-by: Andrew Thornton <art27@cantab.net> * pass context into get-last-commit Signed-off-by: Andrew Thornton <art27@cantab.net> * Make commit_info cancellable Signed-off-by: Andrew Thornton <art27@cantab.net> * use context as context Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: 6543 <6543@obermui.de>

…ranch don't close related PRs (go-gitea#16067) * Fix bug when delete branch don't close related PRs * Merge all deletebranch as one method * Add missed branch.go * fix comment Co-authored-by: Lauris BH <lauris@nix.lv>

…6069) You can limit or hide organisations. This pull make it also posible for users - new strings to translte - add checkbox to user profile form - add checkbox to admin user.edit form - filter explore page user search - filter api admin and public user searches - allow admins view "hidden" users - add app option DEFAULT_USER_VISIBILITY - rewrite many files to use Visibility field - check for teams intersection - fix context output - right fake 404 if not visible Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Andrew Thornton <art27@cantab.net>

…-gitea#16260) One of the repeatedly reported issues has been that gitea produces too much console logging during set up even if the console logger is turned off. Fundamentally this is due to some otherwise very helpful logging that has to occur before logging is set up. This has come to a head with the merging of go-gitea#16243 where otherwise potentially helpful Trace logging in the git module now appears on the console. This PR proposes three things: 1. Change the initial default logger to Info not Trace. 2. Change the logging for the AppPath things to Info in recompense. 3. Add two new command line options to gitea web: --quiet and --verbose `gitea web -q` or `gitea web --quiet` will only log Fatal level initially. `gitea web -verbose` will log at Trace. Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: techknowlogick <techknowlogick@gitea.io>

Signed-off-by: Steven Kriegler <61625851+justusbunsi@users.noreply.github.com>

Fixes go-gitea#16263 Co-authored-by: zeripath <art27@cantab.net>

* review comments: break-word for long file names fixes go-gitea#16248 Co-authored-by: zeripath <art27@cantab.net>

Now that go-gitea#16069 is merged, some sites may wish to enforce that users are all public, limited or private, and/or disallow users from becoming private. This PR adds functionality and settings to constrain a user's ability to change their visibility. Co-authored-by: zeripath <art27@cantab.net>

This PR removes multiple unneeded fields from the `HookTask` struct and adds the two headers `X-Hub-Signature` and `X-Hub-Signature-256`. ## ⚠️ BREAKING ⚠️ * The `Secret` field is no longer passed as part of the payload. * "Breaking" change (or fix?): The webhook history shows the real called url and not the url registered in the webhook (`deliver.go`@129). Close go-gitea#16115 Fixes go-gitea#7788 Fixes go-gitea#11755 Co-authored-by: zeripath <art27@cantab.net>

Co-authored-by: Norwin <noerw@users.noreply.github.com> Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: zeripath <art27@cantab.net>

* Handle misencoding of login_source cfg in mssql Unfortunately due a bug in xorm (see https://gitea.com/xorm/xorm/pulls/1957) updating loginsources on MSSQL causes them to become corrupted. (go-gitea#16252) Whilst waiting for the referenced PR to be merged and to handle the corrupted loginsources correctly we need to add a wrapper to the `FromDB()` methods to look for and ignore the misplaced BOMs that have been added. Fix go-gitea#16252 Signed-off-by: Andrew Thornton <art27@cantab.net> * Update models/login_source.go

Adds a link to each blame hunk, to view the blame of an earlier version of the file, similar to GitHub. Also refactors the blame render from fmtstring based to template based. * Fix blame bottom line and add blame prior button * Jump to previous parent commit from the commit. * Fix previous commit link * Fix previous blame link * Fix the given file not exist in the previous commit. * Fix blameRow struct not export * fix theming issues, rename template var * remove unused LastCommit fetch * fix location of blame-hunk divider * rewrite previous commit checks * remove duplicate commit lookup its already resolved and stored in ctx.Repo.Commit! * split out blamePart processing into function Co-authored-by: rogerluo410 <rogerluo410@gmail.com>

As title, the change counter-works the effect from go-gitea#14926 that links seem unclickable (especially in the default gitea theme), while maintaining some sort of visual harmony. Co-authored-by: Andrew Thornton <art27@cantab.net>

svenseeberg · 2021-06-28T09:38:05Z

Hey, nice work. Do you need help implementing anything?

Thanks. @melegiul already finished most of the work. @localleon If you like, you can review our changes.

svenseeberg · 2021-06-28T11:29:45Z

modules/auth/ldap/README.md

+* Team group map (optional)
+  * Automatically add users to Organization teams, depending on LDAP group memberships.
+  * Note: this function only adds users to teams, it never removes users.
+  * Example: {'MyOrg': [{'MyTeam': 'cn=MyGroup,cn=groups,dc=example,dc=org'}, ...], ...}


@melegiul this example needs an update as well.

One other idea just came to mind: can we make the Org: Team dict part of a list? Otherwise it would not be possible to sync one LDAP group into multiple teams. That means it the best format probably is the following:

{'cn=MyGroup,cn=groups,dc=example,dc=org': [{'MyOrg1': 'MyTeam1'}, ...], ...}

You're right, multiple teams is not quite supported in the moment, I will address this in a commit 👍

This is done in c61005a with a slighty differnt json format:

{'cn=MyGroup,cn=groups,dc=example,dc=org': {'MyOrg1': ['MyTeam1', 'MyTeam2', ...]}, ...}

…gitea#16278) * Upgrade Gliderlabs SSH to 0.3.3 and add FailedConnectionCallback Following the merging of gliderlabs/ssh#143 we can now report connections to the ssh server that have failed before public key exchange has completed using the standard fail2ban message. This PR updates Gliderlabs SSH and adds a callback that will provide this logging. Signed-off-by: Andrew Thornton <art27@cantab.net> * move the callback to its own function to make the logging appear little nicer Signed-off-by: Andrew Thornton <art27@cantab.net>

Following the merge of go-gitea#16278 we need to update the fail2ban documentation to take account of the availability of the new sshConnectionFailed failed authentication attempt log message. Also add a deprecation notice regarding the previous publicKeyHandler messages, as these may be a source of false positives. Signed-off-by: Andrew Thornton <art27@cantab.net>

* Removed Len field. * Added head_commit webhook field. * Added comment for returns.

* Add setting for a JSON that maps LDAP groups to Org Teams. * Sync is being run on login and periodically. * Existing group filter settings are reused. Co-authored-by: Giuliano Mele <mele@integreat-app.de> Co-authored-by: Sven Seeberg <mail@sven-seeberg.de>

svenseeberg mentioned this pull request May 30, 2021

Support for LDAP and AD Group sync go-gitea/gitea#1395

Closed

zeripath and others added 29 commits May 30, 2021 17:38

Close the dataRC reader sooner (go-gitea#16023)

36dce0e

Fix go-gitea#15932 Signed-off-by: Andrew Thornton <art27@cantab.net>

[skip ci] Updated translations via Crowdin

518ed50

Hide mirror passwords on repo settings page (go-gitea#16022)

7a484c0

This PR simply hides mirror passwords from being displayed on the repo settings page. Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: techknowlogick <techknowlogick@gitea.io>

[skip ci] Updated translations via Crowdin

c63d1d8

update .raw-content when edit issue/comment content (go-gitea#16021)

072df3f

* update ``.raw-content`` when edit issue/comment content fix go-gitea#16000 Signed-off-by: a1012112796 <1012112796@qq.com> * handle empty content Co-authored-by: techknowlogick <techknowlogick@gitea.io>

Fix language switch for install page (go-gitea#16043)

7081046

Signed-off-by: a1012112796 <1012112796@qq.com>

Fix case change in ownernames (go-gitea#16045)

9fdda90

If you change the case of a username the change needs to be propagated to their repositories. Signed-off-by: Andrew Thornton <art27@cantab.net>

use "true" for RACE_ENABLED (go-gitea#15261)

c1ff59c

[skip ci] Updated translations via Crowdin

6a7bf97

Fixed unknown translation key. (go-gitea#16057)

8e5a37c

[skip ci] Updated translations via Crowdin

a38f62a

Copy git data from disk when restore repository (go-gitea#16066)

7979c36

Fix typo and add TODO notice (go-gitea#16064)

b77c62d

Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv>

Fix panic (go-gitea#16072)

fe18a85

There is an incorrect casting in the wrapped queue. Fix go-gitea#16071 Signed-off-by: Andrew Thornton <art27@cantab.net>

Change default TMPDIR path in rootless containers (go-gitea#16077)

683cfe3

Signed-off-by: Steven Kriegler <61625851+justusbunsi@users.noreply.github.com>

[skip ci] Updated translations via Crowdin

b3ef6a6

Fix regression of renderer (go-gitea#16091)

b6762e2

* Fix regression of renderer * Fix render setting load twice bug

Fixed assert statements. (go-gitea#16089)

3607f79

sergey-dryabzhinsky and others added 15 commits June 26, 2021 20:53

[skip ci] Updated licenses and gitignores

5402155

Make app.ini more restrictive on new installations (go-gitea#16266)

f533b5d

Signed-off-by: Steven Kriegler <61625851+justusbunsi@users.noreply.github.com>

Add scroll-margin-top to account for sticky header (go-gitea#16269)

eee03ae

Fixes go-gitea#16263 Co-authored-by: zeripath <art27@cantab.net>

review comments: break-word for long file names (go-gitea#16272)

615001d

* review comments: break-word for long file names fixes go-gitea#16248 Co-authored-by: zeripath <art27@cantab.net>

Add jpraet to MAINTAINERS (go-gitea#16274)

2a98ec1

Add previous/next buttons to review comments (go-gitea#16273)

fe66b61

Co-authored-by: Norwin <noerw@users.noreply.github.com> Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: zeripath <art27@cantab.net>

Fix API to return author for author on commits(go-gitea#16276)

59c5855

[skip ci] Updated translations via Crowdin

6ca667d

melegiul force-pushed the feature/ldap-group-sync branch from fad23d7 to 8e839fe Compare June 28, 2021 11:01

svenseeberg commented Jun 28, 2021

View reviewed changes

zeripath and others added 6 commits June 28, 2021 13:05

Fix previous/next button size (go-gitea#16284)

f424862

[skip ci] Updated translations via Crowdin

579fcad

Implemented head_commit for webhooks (go-gitea#16282)

aac663e

* Removed Len field. * Added head_commit webhook field. * Added comment for returns.

Add custom emoji support (go-gitea#16004)

6554835

svenseeberg force-pushed the feature/ldap-group-sync branch 3 times, most recently from b6a6605 to d7c98f0 Compare June 29, 2021 18:33

svenseeberg force-pushed the feature/ldap-group-sync branch from d7c98f0 to 50ae1e1 Compare June 29, 2021 19:07

svenseeberg closed this Jun 30, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add LDAP group sync to Org teams #1

Add LDAP group sync to Org teams #1

svenseeberg commented May 30, 2021 •

edited by melegiul

Loading

svenseeberg commented Jun 28, 2021 •

edited

Loading

svenseeberg Jun 28, 2021 •

edited

Loading

svenseeberg Jun 28, 2021

melegiul Jun 28, 2021

melegiul Jun 29, 2021

Add LDAP group sync to Org teams #1

Add LDAP group sync to Org teams #1

Conversation

svenseeberg commented May 30, 2021 • edited by melegiul Loading

Additional hint

svenseeberg commented Jun 28, 2021 • edited Loading

svenseeberg Jun 28, 2021 • edited Loading

Choose a reason for hiding this comment

svenseeberg Jun 28, 2021

Choose a reason for hiding this comment

melegiul Jun 28, 2021

Choose a reason for hiding this comment

melegiul Jun 29, 2021

Choose a reason for hiding this comment

svenseeberg commented May 30, 2021 •

edited by melegiul

Loading

svenseeberg commented Jun 28, 2021 •

edited

Loading

svenseeberg Jun 28, 2021 •

edited

Loading