SOLR-15771: bin/solr auth enable should model best practices for security.json by epugh · Pull Request #1851 · apache/solr

epugh · 2023-08-18T11:40:10Z

https://issues.apache.org/jira/browse/SOLR-15771

Description

Set up a properly configured security.json with multiple levels of authentication demonstrating a good place to start.

Solution

have a barebones security.json in Solr that is then modified by the bin/solr auth tool

Tests

bats

Checklist

Please review the following and check all that apply:

I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
I have created a Jira issue and added the issue ID to my pull request title.
I have given Solr maintainers access to contribute to my PR branch. (optional but recommended)
I have developed this patch against the main branch.
I have run ./gradlew check.
I have added tests for my changes.
I have added documentation for the Reference Guide

Have a stock security.json that is shipped with Solr and used by the auth tool

epugh · 2023-08-18T12:23:18Z

I’m struggling to get pathing to work to include solr/core/src/resources/security.json to load in the ref guide, maybe someone has some ideas?

…ension...

…ector to work..

epugh · 2023-08-18T16:19:23Z

Okay, I thought that the Antora "Collector Extension" would somehow give me a reference to solr/core/src/resources/security.json so I could include it in the docs, but I never quite got it to work. So at this point I just checked in a copy of security.json into the /examples directory. One nice thing, I was able to update Antora to the latest with no issues... Maybe someone could look at this and have a better idea?

epugh · 2023-08-18T16:19:54Z

Aside from the Antora stuff, this is ready for review and potential commit. I think this COULD be backported to 9x.

janhoy

Some comments inline

solr/core/src/resources/security.json

janhoy · 2023-08-22T11:53:58Z

Also, there should be some agreement between security-dashboard screen in Admin UI and this default security.json. I.e. if you bootstrap a cluster with this default, it should not trigger any of the "best practice" warnings in the security dashboard. Have not checked to see if that is the case. But the UI has some nice recommendations built-in.

epugh · 2023-08-22T13:58:40Z

@janhoy could I talk you into pushing up a commit to the security.json that implements that changes you mentioned? For all that I try to work on security stuff, I don't actually grok it too well, (part of why I like this model best practices so I never have to think about it again!).... I;d love those changes you suggested!

Signed-off-by: Jan Høydahl <janhoy@users.noreply.github.com>

janhoy · 2023-08-22T16:43:19Z

I compiled my suggested changes as a PR to your branch: epugh#5 with these changes:

Renames role user as search
Adds role index which can do /update
Require role superadmin for security-edit
Set blockUnknown: false
Open up /solr/admin/info/health, /api/node/health, /solr/admin/metrics to the world
User search can only search (and do clusterstatus), user index can do update and search, user admin can do all admin commands, update and search, and user superadmin has all roles.
Changed password for each user to same as username
Removed unnecessary "index" json keys from sample

See screenshot for how it looks in security screen - no warnings there.

Worth to note that the search and index users now lacks permissions that would be required to use the Admin UI (config-read, collection-admin-read etc), so the UI is useless for other users than admin. Perhaps we should change that and let both search and index users get access to read those configs so they can use the Admin UI?

As an alternative to predefining all users, we could predefine only the "superadmin" user and generate a strong random password for it that would be printed on the CLI. Then document how this user now can enter the "security" screen of admin UI to create more users, and assign them one or more of the predefined four roles.

Suggested update to default security json

epugh · 2023-08-22T17:52:32Z

My assumption is that "search" and "index" belong primarily to machines, i.e your ETL pipeline and your search API, so wouldn't use the admin UI anyway.... And might even have passwords defined that aren't really reachable by humans!

epugh · 2023-08-22T17:54:01Z

also, I think if we want to have really strong security, I think at that point, we are sort of suggesting that it happens at start up... (Someday I hope to follow the Rails pattern of having environment variables that are global, but also overridden for dev, test, and production so that you might have strong security in prod, but minimal or none in dev).

janhoy · 2023-08-22T18:02:37Z

Re. syncing - could we link to the file on github instead of displaying it verbatim in ref-guide?

This is a tiny step towards secure-by-default, making it easy to get a production grade security config up and running.

PS: Crave is not happy, perhaps need to squash, rebase and force-push.

epugh · 2023-08-22T18:04:42Z

Do we have other examples of linking to Github in teh ref guide? I could definitly link to just the main branch of it.

And yeah, generally I find Crave hates me after a couple of commits, and I just ignore it. Since "squash, rebase and force-push" feels very risky and when I do things like that I get urgent IM's from Houston from mucking up the source tree... ;-)

epugh · 2023-08-22T18:06:12Z

In Solr it is common to configure settings in https://github.com/apache/solr/blob/main/solr/bin/solr.in.sh[solr.in.sh]

From solr-in-docker.adoc...

HoustonPutman · 2023-08-22T18:09:11Z

Do we have other examples of linking to Github in teh ref guide? I could definitly link to just the main branch of it.

We should link to the correct branch. If the capability isn't there already, I'll add it to your PR.

epugh · 2023-08-22T18:09:50Z

Thanks @HoustonPutman !

epugh · 2023-08-22T18:10:25Z

Now I am getting excited about this PR ;-). Better Auth defaults and Better Ref Guide ;-)

janhoy · 2023-08-22T19:13:09Z

solr/core/src/java/org/apache/solr/cli/AuthTool.java

+        ((ObjectNode) credentialsNode)
+            .put(username, Sha256AuthenticationProvider.getSaltedHashedValue(password));
+        JsonNode userRoleNode = securityJson1.get("authorization").get("user-role");
+        ((ObjectNode) userRoleNode).put(username, "admin");


Is it documented somewhere that the --credentials myUser:myPass arg of the CLI will be an admin role user, and what that means? Should it instead define the "superadmin" user?

Good question. I think, that if you are able to run bin/solr auth, then you probably are a "superadmin" and we should set you to that role, and document it in the Ref Guide. I wonder if a feedback message "User X,Y has been granted superadmin permissions" would be useful.....

What if we could do stuff like:

bin/solr auth enable --credentials root:xxxx --type basic # superuser creds bin/solr auth adduser --credentials jane:xxxx --role admin # admin human user bin/solr auth adduser --credentials frontend:xxxx --role search # search machine user bin/solr auth adduser --credentials indexer:xxxx --role index # indexer machine user bin/solr auth deluser jane

Then the first "enable" command simply adds the root user. And the adduser commands will add new users with predefined roles. And the script can make sure that an admin user also has index and search roles etc.

It's not crucial to have since we have a nice AdminUI for adding users, but sounds like a win for those who want to script a simple setup.

I like this idea.... So, at some point I am hoping that as the v2 API's come online, we move the CLI to using them. And that at that point adding a command like "adduser" or "deleteuser" becomes super easy because we have nice clean SolrJ client objects (methods?) that map to the v2 apis with all the docs, and that we don't add more custom logic on the auth tool....

janhoy · 2023-08-22T19:13:24Z

solr/core/src/resources/security.json

+{
+  "authentication": {
+    "blockUnknown": false,
+    "class": "solr.BasicAuthPlugin",


What about type=kerberos?

Humm.. My somewhat snarky response is "What is this Kerberos thing you speak of"? So, the auth tool didn't support that previously, so I never thought about it. Maybe for a seperate ticket? I wonder if we need to support "point at a existing security.json"? Or, do we add another param to the command?

Never used it. Looks like you'll configure your entire KRB environment in other files and sysprops, and then at the end you create security.json with little more than the "class" property. When creating user:role mapping, the username needs to match the kerberos principal, so I suppose that a user could be mapped to role here too, although it would not be adduser, but perhaps mapuser?

Probably best to document that for kerberos we only support initial bootstrap of security.json, no user:role mapping, which can be done in the UI (?)

Since we know so little about KRB, we already have the text "Currently, this script only enables Basic Authentication, and is only available when using SolrCloud mode" in the ref guide, which I think covers the concern. I don't know anything about KRB, and don't think I ever will!

wait, we have a -type command... so yes, we do need to update the ref guide...

epugh · 2023-08-29T09:41:27Z

@HoustonPutman this PR is getting close to be ready to merge, wanted to bug you on the ref guide "branch" magic links!

epugh · 2023-08-29T12:23:44Z

Ugh... test_ssl.bats failing.. thought it was ssl, but running not in ssl, I can create a collection with my user:password, but I can't query it!!!

 curl -u name:password 'http://localhost:8983/solr/test/select?q=*:*' 
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 403 Unauthorized request, Response code: 403</title>
</head>
<body><h2>HTTP ERROR 403 Unauthorized request, Response code: 403</h2>
<table>
<tr><th>URI:</th><td>/solr/test/select</td></tr>
<tr><th>STATUS:</th><td>403</td></tr>
<tr><th>MESSAGE:</th><td>Unauthorized request, Response code: 403</td></tr>
<tr><th>SERVLET:</th><td>default</td></tr>
</table>

… doesn't cover permission of other roles..

epugh · 2023-08-30T18:39:36Z

Okay, I didn't realize the superadmin doesn't extend admin extend search/index, instead you have to assign all of them...

I wonder if we should map alll roles to superadmin, some roles to admin, just a few to index and search? so that if you have just "superadmin" role, but not admin etc, then it works the way you would expect? @janhoy ???

janhoy · 2023-08-30T18:41:41Z

I wonder if we should map alll roles to superadmin

That is absolutely an option. Not as flexible but fairly logical and easier to grok.

epugh · 2023-09-09T11:03:07Z

Ugh..... so I went to do the mapping of explicit individual permissions per Role, and wihtout the abilityt o comment in JSON, it started feeling VERY error prone.... I mulled a bit more on it, but no joy. So.. I think I'm going to call this done.

I know right now the ref guide link is to the main repo of Github and I tried to chagne that a bit but no luck... @HoustonPutman if you are going to get to that, let me know, but I'm thinking maybe that could be it's own ticket and we could merge this? I want to start looking at basic auth for the CLI, and having this in there would help with testing...

…rity.json (#1851) Introduce a security.json that defines four roles: `search`, `index`, `admin` and `superadmin`, and assigns the `superadmin` to the user created by running bin/solr auth.

epugh added 2 commits August 18, 2023 07:37

model good security design

19d62af

Have a stock security.json that is shipped with Solr and used by the auth tool

Point folks to richer set of auth related tests

46700dc

epugh requested a review from janhoy August 18, 2023 11:40

epugh added 3 commits August 18, 2023 07:48

tabs to spaces

23ad980

remove debugging

b7d9cdf

attempt to integrate security.json into ref guide

67b1871

epugh added 3 commits August 18, 2023 11:21

bumping antora version to support installing the antora collector ext…

df968ba

…ension...

humm, can't quite figure out what tag it's under

f6b5c12

Committing a copy of security.json in docs since I can't get the coll…

c6b20b6

…ector to work..

janhoy reviewed Aug 22, 2023

View reviewed changes

solr/core/src/resources/security.json Outdated Show resolved Hide resolved

solr/core/src/resources/security.json Outdated Show resolved Hide resolved

Suggested update to default security json

29e080b

Signed-off-by: Jan Høydahl <janhoy@users.noreply.github.com>

Merge pull request #5 from janhoy/default-security-json

fe0588a

Suggested update to default security json

syncing these will be a pain

0d3dd83

wordsmithing, and removing example security.json to avoid duplication

5bd89b6

janhoy reviewed Aug 22, 2023

View reviewed changes

epugh added 3 commits August 29, 2023 06:03

Merge remote-tracking branch 'upstream/main' into SOLR-15771

86d5d45

nicer messaging about krb and basic auth

7457c9a

fix formatting

852f84b

epugh added 7 commits August 29, 2023 13:40

back out failed attempt

3d990f8

more backing out of changes...

fec58e4

remove tab

73c209f

last bit of roll back...

3fcee2a

associate user with all the roles... superadmin isn't enough, as that…

b429d5f

… doesn't cover permission of other roles..

lint

4b94639

update assert check

808c6ed

epugh added 3 commits September 12, 2023 07:47

Merge remote-tracking branch 'upstream/main' into SOLR-15771

f44cb34

track change

c302561

Merge remote-tracking branch 'upstream/main' into SOLR-15771

bfa8039

epugh merged commit b163a0e into apache:main Sep 12, 2023

Conversation

epugh commented Aug 18, 2023

Description

Solution

Tests

Checklist

Uh oh!

epugh commented Aug 18, 2023

Uh oh!

epugh commented Aug 18, 2023

Uh oh!

epugh commented Aug 18, 2023

Uh oh!

janhoy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

janhoy commented Aug 22, 2023

Uh oh!

epugh commented Aug 22, 2023

Uh oh!

janhoy commented Aug 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

epugh commented Aug 22, 2023

Uh oh!

epugh commented Aug 22, 2023

Uh oh!

janhoy commented Aug 22, 2023

Uh oh!

epugh commented Aug 22, 2023

Uh oh!

epugh commented Aug 22, 2023

Uh oh!

HoustonPutman commented Aug 22, 2023

Uh oh!

epugh commented Aug 22, 2023

Uh oh!

epugh commented Aug 22, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

epugh commented Aug 29, 2023

Uh oh!

epugh commented Aug 29, 2023

Uh oh!

epugh commented Aug 30, 2023

Uh oh!

janhoy commented Aug 30, 2023

Uh oh!

epugh commented Sep 9, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

janhoy commented Aug 22, 2023 •

edited

Loading