Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deactivate users #7629

Merged
merged 52 commits into from
Mar 29, 2021
Merged

deactivate users #7629

merged 52 commits into from
Mar 29, 2021

Conversation

pdurbin
Copy link
Member

@pdurbin pdurbin commented Feb 19, 2021

Note: This feature was originally called "disable users" and was later renamed to "deactivate users" so you will likely see mention of "disable" instead of "deactivate" below.

What this PR does / why we need it:

The people who run a Dataverse installation want the ability to deal with users that should be deleted (#4475), disabled (#2419), or otherwise reduced from a normal, active account.

New API endpoints:

  • This pull request delivers "disable user account" functionality (via API) as well as documentation in the guides that provides details about the consequences this action. Please note that there is no way to "undisable" an account.
  • This pull request also provides a new "Show User Traces" API endpoint. The thinking behind showing the traces is that the "delete" functionality only gives vague messages about why the delete cannot proceed ("the user has created Dataverse object(s)", for example) but now the details are exposed.
  • Finally, this pull request includes an API for "remove all roles" that does the equivalent of the button in the superuser dashboard. It was useful in testing (and is included in new tests).

Changes to existing API endpoints:

  • The "merge user" API has a new rule: "User accounts can only be merged if they are either both enabled or both disabled." In addition, Unable to merge a user when they have initiated a password reset #7575 ("Unable to merge a user when they have initiated a password reset") was fixed.
  • The "delete" API is largely unchanged. An untracked bug was fixed whereby an account can now be deleted even if it has data in the OAuth2TokenData database table. In addition, the delete command now warns that an account cannot be delete if it has been used to create saved searches.
  • The "show user as JSON" API is the same except that two new fields have been added: disabled (a boolean), and disabledTime (a timestamp).

Which issue(s) this PR closes:

Note that #4475 was the issue we estimated during sprint planning (large) but as discussion went on we took several turns. We considered the idea that deleting users could mean the database tables would point toward some sort of anonymous or deleted user. We considered promoting merging as a solution for getting rid of users. In the end, we decided to offer the ability to disable a user.

Special notes for your reviewer:

We don't have a way to write browser-based tests (#4202) so I didn't include any tests for testing that accounts are disabled via browser/session.

I'm somewhat concerned that we don't include the ability to "undisable" an account but didn't work on this because it doesn't seem strictly necessary and we haven't discussed what that account would be like (permissions, for example, are blown away by the "disable" command). Perhaps this could be added in a future pull request, especially if there's demand for it. In practice it means that if you disable the account of someone using HarvardKey, for example, that person can no longer use HarvardKey to log into a Dataverse installation.

I added the ability to disable a user by both "identifier" (username) and database id because I was looking at the "delete" user code but I'm not sure if this needed or not. We could probably get away with just delete by username.

If you're interested in more discussion on the guides (which served as a spec), see pull request #7585. The branch behind that pull request was eventually merged into this one.

Suggestions on how to test this:

First, I tested what I could locally by mocking Shib, and OAuth, but I recommend testing these with real systems. I did not test OIDC accounts at all (I don't know how to) but from talking to Oliver it seems like it should take the same code path as OAuth.

Generally speaking, the guides should be helpful in understanding the expected behavior. See especially bullets under "Disabling a user with this endpoint will:..." One note on "Disabling a user with this endpoint will keep:..." is that disabled users should still show up as contributors under the versions tab of a dataset. I imagine the bulk of the testing involves investigating each of those bullets.

As mentioned above, there's a new rule with regard to merging accounts: "User accounts can only be merged if they are either both enabled or both disabled." The pull request includes automated tests for this but it wouldn't hurt to test it.

As mentioned above, there are two specific bugs that were fixed:

Does this PR introduce a user interface change? If mockups are available, please link/include them here:

Yes, when accounts are disabled, users will see "Sorry, your account has been disabled. If you believe this is an error, please contact support for assistance." There is a screenshot below. The message should be identical for Builtin, Shib, OAuth, and OIDC accounts.

account-disabled

If an OAuth account attempts the "convert" process on a disabled account, it should fail with "Your account can only be converted if you provide the correct username and password for your existing account. If your existing account has been disabled by an administrator, you cannot convert your account." The screenshot below shows the old message before it was updated.

oauth

If a Shib/Institutional account attempts the "convert" process on a disabled account, it should fail with "Validation Error - Your account can only be converted if you provide the correct password for your existing account. If your existing account has been disabled by an administrator, you cannot convert your account." The screenshot below shows the old message before it was updated.

shib

Is there a release notes update needed for this change?:

Yes, included.

Additional documentation:

None.

djbrooke and others added 19 commits February 4, 2021 23:53
Primarily this has been an investigation into existing code for
deleting and merging users. Additionally, a "get user traces"
command was added to get a sense of why a user can't be deleted
or what would be merged. The top of DeleteUsersIT has a lengthy
comment about which database tables are in play and a variety
of scenarios involving users to be deleted, merged or (in the
future) disabled (#2419).

The following bugs where fixed:

- Unable to merge (or delete) a user when they have initiated
a password reset. #7575
- Delete OAuth2 tokens on delete.
This commit adds an API endpoint for disabling users.
Once users are disabled:

- They lose all their roles.
- They lose pending file access requests.
- They lose group memberships.
- They lose banner messages.
- They lose all notifications.
- They get an error on login. (Builtin/Shib/OAuth tested.)
- They get an error if they try to use the API.
- They cannot be assigned roles.
- They cannot be added to groups.
- They cannot initiate password reset.
- They cannot confirm their email (from a previous link).
"User accounts can only be merged if they are either both enabled or both disabled."
@@ -1686,7 +1714,7 @@ public Response submitDatasetVersionToArchive(@PathParam("id") String dsid, @Pat
// DataverseRequest and is sent to the back-end command where it is used to get
// the API Token which is then used to retrieve files (e.g. via S3 direct
// downloads) to create the Bag
session.setUser(au);
session.setUser(au); // TODO: Stop using session. Use createDataverseRequest instead.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still a TODO?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a future TODO that I discussed with @qqmyers . I didn't want to mess with this SubmitToArchive (and ask for QA to re-test it) but it's somewhat of session.setUser, I feel. As I indicated, createDataverseRequest is what we usually do in this case.

export SERVER_URL=http://localhost:8080
export USERNAME=jdoe

curl -H "X-Dataverse-key:$API_TOKEN" -X POST $SERVER_URL/api/admin/authenticatedUsers/$USERNAME/disable
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would it be necessary to use a token here? IIRC all other endpoints in admin API are used without these tokens. For security, you can either use it on localhost only or add the ?unblock-key=... feature. This might cause confusion.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's because DisableUserCommand calls RevokeAllRolesCommand, which requires superuser permissions.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should think of a strategy here to not require a token for an admin api, as I agree it's confusing. Generally (I think this is true), things in the admin API do not call commands, as commands require users. I think that has been a good model (or we need to decide on having commands that don't require users, but that feels dangerous).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in d6e1173 (using authSvc.getAdminUser as suggested elsewhere).

@scolapasta scolapasta self-assigned this Feb 23, 2021
@djbrooke djbrooke added this to the 5.4 milestone Feb 23, 2021
@pdurbin
Copy link
Member Author

pdurbin commented Mar 22, 2021

The post at https://groups.google.com/g/dataverse-dev/c/wh3m55EGOPU/m/Mij-l0SlAQAJ just reminded me about #4356 about oauth2tokendata which has also been fixed by this pull request (mostly recently touched in 8afbf5e). So I added it to the list of issues above that this pull request closes.

Copy link
Contributor

@scolapasta scolapasta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One suggested change related to getUser (which can simplify the code) and a few other minor questions / notes that may or may not need to be acted on. @pdurbin can you review and let me know what you think?

@@ -3055,6 +3055,8 @@ Example: ``curl -H "X-Dataverse-key: $API_TOKEN" -X POST http://demo.dataverse.o

This action moves account data from jsmith2 into the account jsmith and deletes the account of jsmith2.

Note: User accounts can only be merged if they are either both non-deactivated or both deactivated. See :ref:`deactivate a user<deactivate-a-user>`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we use the term "active" instead of non-deactivated, which is just a little weird?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Active" is the opposite of "inactive". Is "active" also the opposite of "deactivated"? Maybe. I felt that "non-deactivated" is more accurate.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that "Active" is the opposite of "inactive"; I think of deactivated accounts as inactive, it's just more specific in that it states that they were "deactivated" (how they got to inactive). We probably could have called them "inactive" as well. But changing that now is overkill.

Non-deactivated is wordy and not a good way to describe the the normal state of a user, e.g when you just create one. "I just created an account on Dataverse; it is non-deactivated"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still prefer the precision but I made the change in df32ec0.

@@ -695,7 +695,7 @@ public String save() {

} catch (CommandException ex) {
logger.log(Level.SEVERE, "Unexpected Exception calling dataverse command", ex);
String errMsg = create ? BundleUtil.getStringFromBundle("dataverse.create.failure") : BundleUtil.getStringFromBundle("dataverse.update.failure");
String errMsg = create ? ex.getLocalizedMessage() : BundleUtil.getStringFromBundle("dataverse.update.failure");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we get the ex.localized message in one, but give a specific message in the other?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see d0191c3 for how ex.localized message comes from EjbDataverseEngine with (for example) command.exception.user.deactivated={0} failed: User account has been deactivated.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see, but a) I also requested simplifying that code and changing it - and it still isn't clear to me why this is only on "create".

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because "create" is the action that all authenticated users can take in the root (for prod and various dev environments).

@@ -663,6 +695,10 @@ public Response builtin2shib(String content) {
password);
if (authenticatedUser != null) {
knowsExistingPassword = true;
if (builtInUserToConvert.isDeactivated()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not have this check earlier (other logic could then be skipped?)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right. It could be. I'm testing moving it up.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved up in 530dab3.

}

if (consumedAU.isDeactivated() && !ongoingAU.isDeactivated() || !consumedAU.isDeactivated() && ongoingAU.isDeactivated()) {
throw new IllegalCommandException("User accounts can only be merged if they are either both non-deactivated or both deactivated.", this);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another use of non-deactivated

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see above.

Comment on lines +221 to +235

AuthenticatedUser authenticatedUser = dvReq.getAuthenticatedUser();
if (authenticatedUser != null) {
AuthenticatedUser auFreshLookup = authentication.findByID(authenticatedUser.getId());
if (auFreshLookup == null) {
logger.fine("submit method found user no longer exists (was deleted).");
throw new CommandException(BundleUtil.getStringFromBundle("command.exception.user.deleted", Arrays.asList(aCommand.getClass().getSimpleName())), aCommand);
} else {
if (auFreshLookup.isDeactivated()) {
logger.fine("submit method found user is deactivated.");
throw new CommandException(BundleUtil.getStringFromBundle("command.exception.user.deactivated", Arrays.asList(aCommand.getClass().getSimpleName())), aCommand);
}
}
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can simplify and remove this - there is already a DataverseRequest associated with this call to the command and that should be enough, as long as we also change the one place where it gets the user from the session in DataverseRequestService to call the new getUser(true) verson of that method.

   @PostConstruct
    protected void setup() {
        dataverseRequest = new DataverseRequest(dataverseSessionSvc.getUser(), request);
    }

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried putting it there in the postconstruct and couldn't get it to work. So I put it in the engine instead.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, in what way did it not work? Did it throw a specific error?

Copy link
Member Author

@pdurbin pdurbin Mar 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't remember an error. I think it simply didn't work. It had no effect, as I recall.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's weird - since it should set the request to point to a GuestUser in that case and therefore fail due to permission. I would prefer this simpler mechanism if we can get it to work, though (and at least understand why it doesn't if it doesn't). Let me go ahead and try it and see what happens.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. Good luck.

@kcondon kcondon self-assigned this Mar 24, 2021
@kcondon
Copy link
Contributor

kcondon commented Mar 25, 2021

Test results:

  1. Disable user endpoint doesn't care what permissions api key has but requires a key, otherwise http 405. Thought, based on discussion in pr and that it is an admin api, that it would not require a key.

I can't reproduce this and the automated tests do not use a token. However, the docs were incorrect and showed the use of an API token so I corrected them in abb8be0.

[Kevin] Resolved but could reproduce. I see the doc was updated to remove the key. However, this is reproducible. After discussing with Gustavo, it is expected behavior for admin endpoints that assume console log on perms. So, not a bug but doc change removes confusion.

  1. After disabling a user, some existing users cannot be found in assign roles ui, autocomplete.
    As far as I can tell, autocomplete for assigning roles was completely broken in the branch, due to me botching the query. Fixed in 0efee28.

[Kevin] This is fixed.

  1. Attempting to remove roles using api endpoint with a key that does not have sufficient permissions gives incomplete/confusing error message: {"status":"ERROR","message":"Unable to revoke all roles: null"}
    The error wasn't being bubbled up from the command. Fixed in 3713104.

[Kevin] This is fixed

  1. On manage users page, disabled user still appears, can make a superuser. Suggestion, make authentication type for disabled user, disabled.
    As discussed in Slack, we're putting the word "disabled" in parentheses. In addition the JSON output from /api/admin/list-users now contained "deactivated" and "deactivatedTime". This code and the dashboard code are related. Fixed in c2b7b0a.

[Kevin] Mostly resolved. User is listed as deactivated, though the superuser checkbox is still active.

  1. Disable user if logged in, remains logged in and can edit account info. Update: this happens with delete user too. For disabled case, reenables it but without some stuff, for delete it adds a new account on save!
    Fixed in 3e59e70. Now the getUser code checks if the user in the session still exists and is not disabled.

Concern has been expressed that modifying getUser like this could result in a performance problem. A quick benchmark indicates that if you navigate to the homepage, sign up for an account, and then create a dataverse, the getUser method is called 264 times.

  1. Can log in using builtin disabled user account, if that disabled user edited their account info in step 5
    Fixed in 3e59e70, the same getUser fix mentioned above.

[Kevin] This still happens.
[SEK] Put an additional test on the save button in 36f25bb

  1. Can merge the same account into itself with result being user is deleted. In 5.3, throws constraint error, though could use better error msg.
    Indeed, the user was deleted. This bug is probably preexisting in released versions. Fixed in 4f81fbd.

[Kevin] This is fixed.

  1. Convert oauth (github) account link/page with same email as a newly created, enabled builtin account results in 500 error:
    I can't reproduce this. I'm on 4f81fbd.

[Kevin] Resolved. This now just says This email address is already taken. Phil says works as expected.

  1. Convert oauth (github) account with same email as disabled builtin account, sends user to convert account form, then when enter account to convert, provides both a success and failure message and oauth is logged in. Checking account status after shows merge didn't happen.
    Fixed in fbded77 to prevent deactivated accounts from being converted to OAuth accounts.

[Kevin] Fixed. Now says cannot convert disabled.

  1. Convert shib account works with enabled builtin but on disabled builtin, gives both success and failure though it actually fails. Should check and prevent submitting account for conversion as ui above.
    Fix in fdc9cbe to prevent deactivated accounts from being converted to Shib accounts.

[Kevin] This is fixed.

The createAuthenticatedUserForView method can't handle a null
for the deactivated boolean so we'll set the boolean to false
for existing users. For new users, this is set to false already.
Conflicts (just imports):
src/main/java/edu/harvard/iq/dataverse/api/Admin.java
@pdurbin
Copy link
Member Author

pdurbin commented Mar 25, 2021

the superuser checkbox is still active

Fixed in c326af4. As shown in the screenshot below, the checkbox is disabled for deactivated users. I also enforced this in the underlying command as well as via API.

Screen Shot 2021-03-25 at 4 47 07 PM

@sekmiller
Copy link
Contributor

Kevin, I put in an additional test if a user is trying to edit his account info while he is being deleted/deactivated. Hitting save will boot them to the root dv page, and if they try to log in again they will get the bad news.

@@ -1,4 +1,6 @@
-- Users can be deactivated.
ALTER TABLE authenticateduser ADD COLUMN IF NOT EXISTS deactivated BOOLEAN;
-- Prevent old users from having null for deactivated.
UPDATE authenticateduser SET deactivated = false;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pdurbin should there be any concern at all if this is run on a db that already has deactivated users (if someone removes this from their flyway table, for example)? Maybe not, but might be safer to add a where clause OR as a default on the Alter table, i.e.

ALTER TABLE authenticateduser ADD COLUMN IF NOT EXISTS deactivated BOOLEAN DEFAULT false

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

either works for me. I think "WHERE deactivated IS NULL;" stands out more as intent.


//First reget user to make sure they weren't deactivated or deleted
if (session.getUser().isAuthenticated() && !session.getUser(true).isAuthenticated()) {
return "dataverse.xhtml?alias=" + dataverseService.findRootDataverse().getAlias() + "&faces-redirect=true";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not simplify and just go to homepage, i.e. "/"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can make that change. i was just using the pattern from the bottom of the save method.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to leave it as is because when I tried return "/"; it left me in edit mode on the user page which I think would be confusing to the deleted/deactivated user.

@@ -284,6 +284,12 @@ public void validateNewPassword(FacesContext context, UIComponent toValidate, Ob

public String save() {
boolean passwordChanged = false;

//First reget user to make sure they weren't deactivated or deleted
if (session.getUser().isAuthenticated() && !session.getUser(true).isAuthenticated()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the first part of this needed?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess not. it's left over from when I did the re-get user within the save method.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no. we need it because if we're creating an account the session user is a guest before they save the new account.

@kcondon kcondon merged commit 36f235b into develop Mar 29, 2021
@kcondon kcondon deleted the 2419-4475-7575-disable-users branch March 29, 2021 15:58
pdurbin added a commit that referenced this pull request Mar 30, 2021
Use the id from authenticateduser rather than builtinuser.

Also remove superuser, no longer needed.
kcondon added a commit that referenced this pull request Mar 31, 2021
fix failing test for deactivate by id #7629
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
7 participants