Structure of FM parameters is cached

[Pentusha]

Hello.
I've got functional module ZHHDEMO_STRUCT_MOD with EX_ZHHT_COL2 param.

FUNCTION ZHHDEMO_STRUCT_MOD.
*"----------------------------------------------------------------------
*"*"Local Interface:
*"  EXPORTING
*"     VALUE(EX_ZHHT_COL2) TYPE  ZHHTT_COL2
*"----------------------------------------------------------------------

  DATA: ls_col2 TYPE zhhs_col2.

  ls_col2-COL1 = 'col1_1'.
  ls_col2-COL2 = 'col2_1'.

  APPEND ls_col2 TO EX_ZHHT_COL2.

  ls_col2-COL1 = 'col1_2'.
  ls_col2-COL2 = 'col2_2'.

  APPEND ls_col2 TO EX_ZHHT_COL2.

ENDFUNCTION.

And I've got python script that performs the following sequence of actions:

1. Prints structure of EX_ZHHT_COL2 parameter;
2. Prints function result ZHHDEMO_STRUCT_MOD;
3. Gives me the time to modify structure of EX_ZHHT_COL2 parameter;
4. Prints structure of EX_ZHHT_COL2 param again. This is proof that the structure has been changed;
5. Prints function result ZHHDEMO_STRUCT_MOD which exactly the same as pt 2.

import pyrfc

connection_info = {
    # some connection data here
}

function_name = u'ZHHDEMO_STRUCT_MOD'
table_name = u'ZHHTT_COL2'


def get_structure():
    with pyrfc.Connection(**connection_info) as con:
        interface_response = con.call(
            'RFC_GET_FUNCTION_INTERFACE',
            **{'FUNCNAME': function_name}
        )
        assert any(p[u'TABNAME'] == table_name for p in interface_response[u'PARAMS'])
        structure_response = con.call(
            'RFC_GET_STRUCTURE_DEFINITION',
            **{'TABNAME': table_name}
        )
        fields = structure_response[u'FIELDS']
        return [f[u'FIELDNAME'] for f in fields]


def function_call():
    with pyrfc.Connection(**connection_info) as con:
        return con.call(function_name)


if __name__ == '__main__':
    print('STRUCTURE 1', get_structure())
    print('RESULT 1', function_call())
    raw_input('Structure changed now. Press Enter to continue...')
    print('STRUCTURE 2', get_structure())
    print('RESULT 2', function_call())

I change the structure as follows:

Python script gives me output:

('STRUCTURE 1', [u'ZHHS_COL2', u'COL1', u'COL2'])
('RESULT 1', {u'EX_ZHHT_COL2': [{u'COL2': u'col2_1', u'COL1': u'col1_1'}, {u'COL2': u'col2_2', u'COL1': u'col1_2'}]})
Structure changed now. Press Enter to continue...
('STRUCTURE 2', [u'ZHHS_COL2', u'COL1', u'COL2', u'COL3'])
('RESULT 2', {u'EX_ZHHT_COL2': [{u'COL2': u'col2_1', u'COL1': u'col1_1'}, {u'COL2': u'col2_2', u'COL1': u'col1_2'}]})

The next launch gives me the right output:

('STRUCTURE 1', [u'ZHHS_COL2', u'COL1', u'COL2', u'COL3'])
('RESULT 1', {u'EX_ZHHT_COL2': [{u'COL2': u'col2_1', u'COL3': u'', u'COL1': u'col1_1'}, {u'COL2': u'col2_2', u'COL3': u'', u'COL1': u'col1_2'}]})
Structure changed now. Press Enter to continue...
('STRUCTURE 2', [u'ZHHS_COL2', u'COL1', u'COL2', u'COL3'])
('RESULT 2', {u'EX_ZHHT_COL2': [{u'COL2': u'col2_1', u'COL3': u'', u'COL1': u'col1_1'}, {u'COL2': u'col2_2', u'COL3': u'', u'COL1': u'col1_2'}]})

[bsrdjan]

That is actually correct behaviour. The pyrfc caches the RFC metadata to gain performance and does not support changing the RFC signature during runtime.
Is there a real problem/requirement and scenario in which RFC parameters' structures change during runtime ?

[Pentusha]

I've got a daemon/webserver which periodically call function. So I need to restart it when structure is changed. Is there a way to invalidate cache?

[bsrdjan]

There is a nwrfc lib api available to invalidate the cache, for a given sysid and function module name. Would it be enough to expose it as pyrfc method? It should be called before calling the RFC. Would that be ok?

[Pentusha]

Yes, it will be ok.

[bsrdjan]

The fix is available in dev branch now, also the ubuntu wheel. Could you please check if works for you? Method names and examples how to use are in respective unit test.

Please consider that if incorrect function module name supplied, like 'XXXXXX_STRUCT_MOD' for example, no error is returned because not finding it in cache is not an error. The removal from cache does not check if the function module exists. If such check needed, the application must take care.

[Pentusha]

I added a deletion of the cache before function call and recompile pyrfc with your commit (in new virtual environment), but unfortunately I still get same result.

def function_call():
    with pyrfc.Connection(**connection_info) as con:
        con.func_desc_remove(connection_info['sysid'], function_name)
        response = con.call(function_name)
        return response

('STRUCTURE 1', [u'ZHHS_COL2', u'COL1', u'COL2'])
('RESULT 1', {u'EX_ZHHT_COL2': [{u'COL2': u'col2_1', u'COL1': u'col1_1'}, {u'COL2': u'col2_2', u'COL1': u'col1_2'}]})
Structure changed now. Press Enter to continue...
('STRUCTURE 2', [u'ZHHS_COL2', u'COL1', u'COL2', u'COL3'])
('RESULT 2', {u'EX_ZHHT_COL2': [{u'COL2': u'col2_1', u'COL1': u'col1_1'}, {u'COL2': u'col2_2', u'COL1': u'col1_2'}]})

[bsrdjan]

after more investigation I am sorry to confirm the feature can't be supported and cache invalidation can't help. The DDIC structure changes are visible only in the same session. If the change is done by another session in backend, the pyrfc session can't see it, only after logout/login. That is application server restriction, not of nwrfclib or pyrfc.

[Pentusha]

Thanks for the explanation. I reproduced it in the SAP GUI. Can you explain me what do the terms "session", "login/logout" stands for. I thought that the opening of the connection is equal to the login and the start of the session, and closing of connection is equal to logout and drop the session. Why session drops when I terminate process that uses sapnwrfc? I might be wrong.

[bsrdjan]

"login/logout" terms are used in relation to sapgui client, used to run transactions like SE11, SE80 ...
Login to sapgui client creates a user session, just like opening a pyrfc connection. On SAP Help more info are available.

As your test environment is already configured, could you please try one more thing, to call the reset_server_context() method, instead of removing from cache?

[Pentusha]

reset_server_context() does not help.

[bsrdjan]

ok, that was the last hope. sorry that I have to confirm that the feature is not supported. The structure changes can be visible only after closing and re-opening the connection. The question will not be forgotten and if any other solution comes in the meantime, I will update this thread.

[Pentusha]

I see, but still cannot understood which connection I should close and re-open. SAP GUI or pyrfc? Each pyrfc request is made in a new connection. SAP GUI restarting has no matter too. Even reimport of pyrfc has no matter, just process termination.

[bsrdjan]

The pyrfc connection. To recap, the test case is following:

1. Open the pyrfc connection and read the RFC structure
2. Change the RFC structure in backend (SAP GUI), while pyrfc connection still open
3. Re-read the RFC structure by pyrfc, without re-starting python script or re-opening the connection.

The change is here not visible, this is the current behaviour and the problem, correct?

Reading the initial description, the next launch of the python script (process termination) shows the change, correct ?

My expectation was that the change shall be visible after closing and opening the connection, without re-starting the script. If that not the case, need to test locally.

[Pentusha]

Changes are not visible in different connections until restarting the script.

Pseudocode:

connction.open()
function_call()
# structure changed with sap gui
function_call()  # changes are not visible here
connection.close()

connection.open()
function_call()  # changes are not visible even here
connection.close()