TTL issue for pam_password with v2.0.0

[mstfdkmn]

Hi, we have an iinit snippet (iinit.exe too) that is used by our windows users to setup the necessary files to authenticate against irods. It writes the irods environment file and the .irodsA file. So that the obfuscated password file helps users grant access for 60 hours. Basically it mimics iinit of iCommands. Before the v2.0.0 all was working normal. However it seems with the v2.0.0 there is something broken in the flow. I explain it below.

iinit snippet to be executed in an interactive shell/interpreter:

config = """
{
    "irods_host": "irods.host",
    "irods_port": 1247,
    "irods_zone_name": "tempZone",
    "irods_authentication_scheme": "pam_password",
    "irods_encryption_algorithm": "AES-256-CBC",
    "irods_encryption_salt_size": 8,
    "irods_encryption_key_size": 32,
    "irods_encryption_num_hash_rounds": 8,
    "irods_user_name": "u0137480",
    "irods_ssl_ca_certificate_file": "",
    "irods_ssl_verify_server": "cert",
    "irods_client_server_negotiation": "request_server_negotiation",
    "irods_client_server_policy": "CS_NEG_REQUIRE",
    "irods_default_resource": "default",
    "irods_cwd": "/tempZone/home",
    "irods_authentication_uid": 1000
}
"""
password = "some-password...."

import os, os.path
from irods.session import iRODSSession
from irods.password_obfuscation import encode

def iinit(config, password):
    def put(file, contents):
        os.makedirs(os.path.dirname(file), exist_ok=True)
        with open(file, "w") as f:
            f.write(contents)
    env_file = os.getenv('IRODS_ENVIRONMENT_FILE', os.path.expanduser('~/.irods/irods_environment.json'))
    put(env_file, config)
    with iRODSSession(irods_env_file=env_file, password=password) as session:
        put(iRODSSession.get_irods_password_file(), encode(session.pam_pw_negotiated[0], uid=1000))
        print("You have successfully authenticated. Use iRODSSession(irods_env_file=" + repr(env_file) +") to start interacting with iRODS.")

iinit(config, password)

any script that contains session connection to be executed:

import os, os.path
from irods.session import iRODSSession

env_file = os.getenv('IRODS_ENVIRONMENT_FILE', os.path.expanduser('~/.irods/irods_environment.json'))
with iRODSSession(irods_env_file=env_file) as session:
    pass

coll = session.collections.get("/tempZone/home/u0137480")
coll.id

If the script that contains a session connection is executed immediately just after the iinit script, the flows work normal. But if it is executed later (my impression is 120 sec), we are getting the error here:

>>> env_file = os.getenv('IRODS_ENVIRONMENT_FILE', os.path.expanduser('~/.irods/irods_environment.json'))
>>> with iRODSSession(irods_env_file=env_file) as session:
...     pass
>>> coll = session.collections.get("/gbiomed/home/u0137480")
>>> coll.id
10075

>>> # Some time later (120 seconds)

>>> env_file = os.getenv('IRODS_ENVIRONMENT_FILE', os.path.expanduser('~/.irods/irods_environment.json'))
>>> with iRODSSession(irods_env_file=env_file) as session:
...           pass
>>> coll = session.collections.get("/gbiomed/home/u0137480")

Traceback (most recent call last): 
File "C:\Workdir\MyApps\Anaconda\envs\pam\Lib\site-packages\irods\session.py", line 290, in server_version
return tuple(ast.literal_eval(reported_vsn))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Workdir\MyApps\Anaconda\envs\pam\Lib\ast.py", line 66, in literal_eval
node_or_string = parse(node_or_string.lstrip(" \t"), mode='eval')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                          File "C:\Workdir\MyApps\Anaconda\envs\pam\Lib\ast.py", line 52, in parse
return compile(source, filename, mode, flags,
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<unknown>", line 0
SyntaxError: invalid syntax
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Workdir\MyApps\Anaconda\envs\pam\Lib\site-packages\irods\session.py", line 296, in __server_version
conn = next(iter(self.pool.active))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
StopIteration
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Workdir\MyApps\Anaconda\envs\pam\Lib\site-packages\irods\pool.py", line 62, in get_connection
conn = self.idle.pop()
^^^^^^^^^^^^^^^
KeyError: 'pop from an empty set'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Workdir\MyApps\Anaconda\envs\pam\Lib\site-packages\irods\connection.py", line 468, in _login_pam
self._login_native()
File "C:\Workdir\MyApps\Anaconda\envs\pam\Lib\site-packages\irods\connection.py", line 615, in _login_native
self.recv()
File "C:\Workdir\MyApps\Anaconda\envs\pam\Lib\site-packages\irods\connection.py", line 151, in recv
raise get_exception_by_code(msg.int_info, err_msg)
irods.exception.CAT_PASSWORD_EXPIRED: None
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Workdir\MyApps\Anaconda\envs\pam\Lib\site-packages\irods\manager\collection_manager.py", line 21, in get
query = self.sess.query(Collection).filter(*filters)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Workdir\MyApps\Anaconda\envs\pam\Lib\site-packages\irods\session.py", line 268, in query
return Query(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Workdir\MyApps\Anaconda\envs\pam\Lib\site-packages\irods\query.py", line 45, in __init__
if self.sess.server_version >= col.min_version:
^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Workdir\MyApps\Anaconda\envs\pam\Lib\site-packages\irods\session.py", line 292, in server_version
return self.__server_version()
^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Workdir\MyApps\Anaconda\envs\pam\Lib\site-packages\irods\session.py", line 299, in __server_version
conn = self.pool.get_connection()
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Workdir\MyApps\Anaconda\envs\pam\Lib\site-packages\irods\pool.py", line 17, in method_
ret = method(self,*s,**kw)
^^^^^^^^^^^^^^^^^^^^
File "C:\Workdir\MyApps\Anaconda\envs\pam\Lib\site-packages\irods\pool.py", line 78, in get_connection
conn = Connection(self, self.account)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Workdir\MyApps\Anaconda\envs\pam\Lib\site-packages\irods\connection.py", line 75, in __init__
auth_module.login(self)
File "C:\Workdir\MyApps\Anaconda\envs\pam\Lib\site-packages\irods\auth\pam_password.py", line 2, in login
conn._login_pam()
File "C:\Workdir\MyApps\Anaconda\envs\pam\Lib\site-packages\irods\connection.py", line 477, in _login_pam
raise RuntimeError(message)
RuntimeError: Time To Live has expired for the PAM password, and no new password is given in legacy_auth.pam.password_for_auto_renew.  Please run iinit.

Confused with this. Btw, we don't use the native authentication in our flow.

Could you look into this issue? Meanwhile, please let us know if there is any workaround or if we are doing something missing. We tried several thing but didn't work. Thanks.

[korydraughn]

What version of the PRC are you using?
What version of iRODS are you experiencing this against?

[mstfdkmn]

What version of the PRC are you using?

>>> import irods
>>> irods.__version__
'2.0.0'

What version of iRODS are you experiencing this against?

>>> session.server_version
(4, 3, 1)

[korydraughn]

Oh right, the PRC version is in the title.

Anyway, have you adjusted your PAM TTL settings according to the following section?

• https://docs.irods.org/4.3.1/system_overview/configuration/#authentication-configuration

If yes, can you share what you have for each option?

[mstfdkmn]

these are:

[irods@gbiomed ~]$ iadmin get_grid_configuration authentication password_max_time
1209600
[irods@gbiomed ~]$ iadmin get_grid_configuration authentication password_min_time
121
[irods@gbiomed ~]$ iadmin get_grid_configuration authentication password_extend_lifetime
1

[peterverraedt]

The real issue here is that in the v1 client the default password lifetime is 60 hours
https://github.com/irods/python-irodsclient/blob/v1.1.9/irods/connection.py#L440
(although the naming of that variable is confusing).

In v2 the default is that the server should decide on the lifetime

python-irodsclient/irods/connection.py

Lines 461 to 462 in ed2e73c

	# By default, let server determine the TTL.
	time_to_live_in_hours = 0

apparently without possibility for the client to overwrite it and ask another value. And the default lifetime of the server is the minimal one, 121 seconds.

[trel]

Everything is more configurable with 'seconds', so that was the new standard - so I think that part is expected/desired.

[trel]

you're sure it was being treated as 'hours' in 1.1.9? that seems... surprising.

[korydraughn]

One thing that sticks out is the message at the end of stacktrace(?).

RuntimeError: Time To Live has expired for the PAM password, and no new password is given in legacy_auth.pam.password_for_auto_renew.  Please run iinit.

That is generated here:

python-irodsclient/irods/connection.py

Lines 475 to 477 in 1d8433e

	message = ('Time To Live has expired for the PAM password, and no new password is given in ' +
	'legacy_auth.pam.password_for_auto_renew. Please run iinit.')
	raise RuntimeError(message)

Seems you may want to review this section. There are several PAM related option described there and they are referenced in the code leading to that exception.

• https://github.com/irods/python-irodsclient/tree/v2.0.0?tab=readme-ov-file#python-irods-client-settings-file

[korydraughn]

See the following for the full function impl. Notice the lines starting from line 470.

python-irodsclient/irods/connection.py

Lines 457 to 480 in 1d8433e

	def _login_pam(self):
	import irods.client_configuration as cfg
	inline_password = (self.account.authentication_scheme == self.account._original_authentication_scheme)
	# By default, let server determine the TTL.
	time_to_live_in_hours = 0
	# For certain characters in the pam password, if they need escaping with '\' then do so.
	new_pam_password = PAM_PW_ESC_PATTERN.sub(lambda m: '\\'+m.group(1), self.account.password)
	if not inline_password:
	# Login using PAM password from .irodsA
	try:
	self._login_native()
	except (ex.CAT_PASSWORD_EXPIRED, ex.CAT_INVALID_USER, ex.CAT_INVALID_AUTHENTICATION):
	time_to_live_in_hours = cfg.legacy_auth.pam.time_to_live_in_hours
	if cfg.legacy_auth.pam.password_for_auto_renew:
	new_pam_password = cfg.legacy_auth.pam.password_for_auto_renew
	# Fall through and retry the native login later, after creating a new PAM password
	else:
	message = ('Time To Live has expired for the PAM password, and no new password is given in ' +
	'legacy_auth.pam.password_for_auto_renew. Please run iinit.')
	raise RuntimeError(message)
	else:
	# Login succeeded, so we're within the time-to-live and can return without error.
	return

[peterverraedt]

you're sure it was being treated as 'hours' in 1.1.9? that seems... surprising.

https://github.com/irods/irods/blob/main/plugins/database/src/db_plugin.cpp#L7102

The number passed over the wire is multiplied by 3600, so it always has been hours. So this is the regression for the python client: by default v1 attempted to generate a native password with validity of 60 hours, and v2 takes the shorter 121 seconds from server side.

[peterverraedt]

Actually correct code snippet is

https://github.com/irods/irods/blob/main/plugins/database/src/db_plugin.cpp#L7241-L7252

[korydraughn]

I think adding a settings file will allow you to make progress. The option you want to set in that file appears to be legacy_auth.pam.time_to_live_in_hours.

[korydraughn]

You may also need legacy_auth.pam.password_for_auto_renew.

[d-w-moore]

Confused with this. Btw, we don't use the native authentication in our flow.

Note this line too, showing that eventually _login_pam routes through _login_native anyway, with a transformed value it receives from the server, as part of its own internal workings. Yes , it's been that way for a while! : )

[mstfdkmn]

I am closing this - because we decided to use the native scheme. And apparently we are touching on an issue that existed in older versions.