Unable to add a new user when logged using LDAP auth

[pawsok]

Apache Airflow version

2.1.3 (latest released)

Operating System

Amazon Linux AMI 2018.03

Versions of Apache Airflow Providers

No response

Deployment

Other Docker-based deployment

Deployment details

• AWS ECS EC2 mode
• RDS PostgreSQL for DB
• LDAP authentication enabled

What happened

We upgraded Airflow from 2.0.1 to 2.1.3 and now when i log into Airflow (Admin role) using LDAP authentication and go to Security --> List Users i cannot see add button ("plus").

Airflow 2.0.1 (our current version):

Airflow 2.1.3:

What you expected to happen

Option to add a new user (using LDAP auth).

How to reproduce

1. Upgrade to Airflow 2.1.3
2. Log in to Airflow as LDAP user type
3. Go to Security --> List Users

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template!

[BasPH]

Not sure what changed this behaviour, but do you consider this a bug? I'd argue if you have LDAP enabled, you would want all your users to live in your LDAP directory and not be able to create any local users.

[potiuk]

Yeah. @pawsok - were you atually ABLE to add new users in case of LDAP authentication before (and they Landed in LDAP)? Unless you have custom authentication module, I believe there is no code for that functionality in Airflow so I'd also be quite surprised. Maybe that was a bug and when you added the users they landed in Airflow DB instead?

Can you please explain what was the behaviour you observed before and where the users landed (converting that into discussion)

[BasPH]

@potiuk FAB will sync the LDAP users to the Airflow DB (and there's a flag AUTH_ROLES_SYNC_AT_LOGIN to re-sync at every login). I suspect that button previously allowed creating additional "local" users. With AUTH_ROLES_SYNC_AT_LOGIN=False, I suspect that might have worked as a fallback mechanism. But @pawsok can probably explain best what he was using this button for?

[pawsok]

@potiuk Yes, I'm still able to add LDAP users from Airflow 2.0.1. Here is an exemplary add user view:

I have added many users in this way and they can log in Airflow, as:
username: domain/user_domain_name
password: domain_password

In Airflow webserver config file there are three variables set as the below:

from flask_appbuilder.security.manager import AUTH_LDAP
...

AUTH_TYPE = AUTH_LDAP
AUTH_LDAP_SERVER = "ldap://xxxx.xxxx.xxxxx"
AUTH_LDAP_USE_TLS = False

EDIT:

Maybe it's something related to a newer version of Flask AppBuilder? In Airflow 2.0.1 we have Flask-AppBuilder==3.1.1, now it's Flask-AppBuilder==3.3.2.

[BasPH]

@pawsok Those users aren't stored in your LDAP directory are they (only in the Airflow DB)?

[potiuk]

That's also my thinking - that they are just in the DB and you won't find them in your LDAP - can you please double-check that?

[pawsok]

Users exist in Airlow DB and also in LDAP directory, otherwise I think that users are not able to log in to Airflow.

[potiuk]

But did you actually check if they are in LDAP directory after you add them in Airflow manually (and whethere they were not there before)? Just add a new user in airflow and see if it appears in LDAP.

As @BasPH mentioned - the LDAP syncrhronisation works in the way that LDAP users are synchronized from LDAP to Airflow DB (and this should be refreshed either periodically or at login). I highly doubt there is a way Airflow could create an LDAP user - it must have been in LDAP already when you added it. And in this case you should not "add" the users manually but you should run synchronisation to bring the LDAP users to Airflow.

Eventually those users will end-up in Airflow DB but what @BasPH and myself are trying to say is that you should not need to add them manually - they should be automatically synchronized by Airflow.

[pawsok]

I do not have access to check LDAP directory, but so far it has worked this way - if the user added doest not exist in LDAP it was just impossible to log in to Airflow. So I don't expect that when adding a user in Airflow, user will also be added to LDAP automatically. That was the way it worked flawlessly.
What I did add additionally (I will remind you that I have version 2.0.1 now):

1. install 2.0.2 (button "add" is still visible)
2. install 2.1.0, as the next version after "working" one (button "add" is not visible)
3. create a fresh installation on my local laptop 2.1.3 (button "add" is not visible)
4. compare LDAP permissions between 2.0.1 and 2.1.0:

2.0.1:
can_add UserLDAPModelView
can_delete UserLDAPModelView
can_download UserLDAPModelView
can_edit UserLDAPModelView
can_list UserLDAPModelView
can_show UserLDAPModelView
can_userinfo UserLDAPModelView
userinfoedit UserLDAPModelView

2.1.0:

can_add UserLDAPModelView
can_delete CustomUserLDAPModelView
can_delete UserLDAPModelView
can_download UserLDAPModelView
can_edit CustomUserLDAPModelView
can_edit UserLDAPModelView
can_list UserLDAPModelView
can_read CustomUserLDAPModelView
can_show UserLDAPModelView
can_userinfo UserLDAPModelView
userinfoedit UserLDAPModelView

As we can see here (and also in the source code) something has changed about LDAP settings.

[BasPH]

Did some simple testing but haven't found the cause. Granted all possible permissions to the Admin role:

-- Create all possible permissions for all views (added 389 rows)
insert into ab_permission_view (permission_id, view_menu_id)
(
	select a.id, b.id
	from ab_permission a
	cross join
	(select id from ab_view_menu) b
)
on conflict do nothing;

-- Add all permissions to Admin role (added 391 rows)
insert into ab_permission_view_role (permission_view_id, role_id)
(
	select id, 1
	from ab_permission_view
)
on conflict do nothing;

But no button when logged in with Admin role.

Between Airflow 2.0.1 and 2.1.3 the Flask-AppBuilder version changed from 3.1.1 to 3.3.2. There were some extensive changes to the LDAP authentication mechanism (PR), but I wasn't able to quickly pinpoint any potential changes with regards to falling back to local users.

---

Regardless, I'd argue that this is the intended way it should work. If there is only one authentication mechanism (LDAP), then there shouldn't be a second place for users to live in. From a security perspective, having two places for user accounts to live in, increases the attack surface.

You could ask your LDAP admin if it's possible for yourself/your team to create new accounts in your LDAP directory. Assuming you were using the local users for debugging purposes only, you might also want to look at account or password expiration options in your LDAP.

[pawsok]

Hi, it turned out that this is a problem related to missing permissions in Airflow. Class CustomUserLDAPModelView inherits from MultiResourceUserMixin and I noticed there isn't can_create action, so what I did:

1. Edit /www/views.py and class MultiResourceUserMixin

from:

method_permission_name = {
        'userinfo': 'read',
        'download': 'read',
        'show': 'read',
        'list': 'read',
        'edit': 'edit',
        'userinfoedit': 'edit',
        'delete': 'delete',
    }

    base_permissions = [
        permissions.ACTION_CAN_READ,
        permissions.ACTION_CAN_EDIT,
        permissions.ACTION_CAN_DELETE,
    ]

to:

method_permission_name = {
        'add': 'create',
        'userinfo': 'read',
        'download': 'read',
        'show': 'read',
        'list': 'read',
        'edit': 'edit',
        'userinfoedit': 'edit',
        'delete': 'delete',
    }

    base_permissions = [
        permissions.ACTION_CAN_CREATE,
        permissions.ACTION_CAN_READ,
        permissions.ACTION_CAN_EDIT,
        permissions.ACTION_CAN_DELETE
    ]

2. Run airflow sync-perm
3. Button (to add user) is visible now in Users View.
4. My colleague is able to log in to Airflow once I added him (he is of course in LDAP directory)

Everything was done on Airflow 2.1.3 version.

[potiuk]

But will it work with sync without adding the user ?

[pawsok]

@potiuk No, This works only if I add a new user directly from Airflow who is already in LDAP. Users existing in LDAP do not appear automatically in Airflow. This approach is right for me because it gives me an extra layer of authentication that I can manage myself.
So I only specify one parameter, the path to LDAP server: AUTH_LDAP_SERVER = "ldap://xxxx.xxxx.xxxxx" in webserver_config.py.

[bparhy]

We do exactly the same approach. @pawsok the solution you suggested is it already implemented in any airflow version ? Also is there a way to achieve this from the UI only ?

Right now to add a new user what I am doing is asking the user to login to our airflow which results in a failure but I can see that user in the list user tab with [public] role and since user is present in LDAP I am simply changing their role.

But is there a better solution that this ?. Thanks in advance.

[potiuk]

I see - I created the issue back then #18545 and I understand where it came from - you simply do not want automated user registration with LDAP. That's fine and there are others with similar workflow.

[potiuk]

This has been fixed in #19963 and I asked Jed (Release Manager of Airflow for 2.2.4) to include it in the upcoming release.

[pawsok]

Thanks a lot @potiuk @lwyszomi.

[andrewkuraksa]

Is there a similar fix for OAUTH? The register user button is not visible.

[potiuk]

Feel free to open one it is not. Different users contributed fixes for Remote (#19963) and LDAP (#22619). We are just about to make first release to 2.3.0 so if you want to add it for Oauth you should be quick to contribute it.

Airflow is created by more than 2000 users and a lot of the contributions came from the users who wanted to improve things. You can easily become one by contributing such a fix.