IDArray injection with local IDArray referencing external keystore fails

[PackeTsar]

When a local IDArray references an external keystore entry and the external keystore is merged, the IDArray info is not injected into the merge

Reproduction

ZTP Config

ztp set idarray LOCALKEYSTORE TESTSERIAL
ztp set idarray STACK1 SERIAL1 SERIAL2 SERIAL3
#
ztp set external-keystore EXTERNAL type csv
ztp set external-keystore EXTERNAL file '/root/test.csv'

File:/root/test.csv

keystore_id,association,idarray_1,idarray_2,idarray_3,vl1_ip_address,hostname,vl1_netmask
SERIAL100,SHORT_TEMPLATE,,,,10.0.0.201,SOMEDEVICE,
STACK1,LONG_TEMPLATE,,,,10.0.0.200,CORESWITCH,255.255.255.0

File:/root/test.csv Equivalent Local Config

[root@ZTPSERVER ~]# ztp request external-keystore-test EXTERNAL                 
#
#
#
ztp set keystore SERIAL100 keystore_id SERIAL100
ztp set keystore SERIAL100 vl1_ip_address 10.0.0.201
ztp set keystore SERIAL100 hostname SOMEDEVICE
#
ztp set keystore STACK1 keystore_id STACK1
ztp set keystore STACK1 vl1_ip_address 10.0.0.200
ztp set keystore STACK1 hostname CORESWITCH
ztp set keystore STACK1 vl1_netmask 255.255.255.0
#
#
#
#
#
#
ztp set association id SERIAL100 template SHORT_TEMPLATE
ztp set association id STACK1 template LONG_TEMPLATE
#
#
#

Run the successful Merge-Test

• IDarray info is injected into the merge

[root@ZTPSERVER ~]# ztp request merge-test TESTSERIAL
2020-06-17 17:48:06:   cfact.get_keystore_id: Checking Keystores and IDArrays for (TESTSERIAL)

2020-06-17 17:48:06:   cfact.get_keystore_id: Checking Keystore names for (TESTSERIAL)

2020-06-17 17:48:06:   cfact.get_keystore_id: ID (TESTSERIAL) not found in keystore names, checking local IDArrays

2020-06-17 17:48:06:   cfact.get_keystore_id: ID 'TESTSERIAL' resolved to arrayname 'LOCALKEYSTORE'

2020-06-17 17:48:06:   cfact.get_keystore_id: Keystore with name 'LOCALKEYSTORE' has been found

2020-06-17 17:48:06:   cfact.get_template: Looking up association for identity (LOCALKEYSTORE)

2020-06-17 17:48:06:   cfact.get_template: Default-template is pointing to (LONG_TEMPLATE)

2020-06-17 17:48:06:   cfact.get_template: Template (LONG_TEMPLATE) exists in local config. Returning


Some variables in jinja template do not exist in keystore:
        -vl1_netmask
        -range


2020-06-17 17:48:06:   cfact.pull_keystore_values: Inserting IDArray keys

2020-06-17 17:48:06:   cfact._global_lookup: Checking if a global-keystore is configured and ready...

2020-06-17 17:48:06:   cfact._global_lookup: Global-keystore configured as none. Discarding

2020-06-17 17:48:06:   cfact.merge_test: Merging with values:
{
    "vl1_ip_address": "10.0.0.201", 
    "idarray_1": "TESTSERIAL", 
    "snmpinfo": {
        "WS_C3850_SERIAL_NUMBER": "WS_C3850_SERIAL_NUMBER_FAKESERIAL", 
        "WS_C2960_SERIAL_NUMBER": "WS_C2960_SERIAL_NUMBER_FAKESERIAL", 
        "matched": "FAKEMATCHEDSERIAL"
    }, 
    "hostname": "SOMEDEVICE", 
    "idarray": [
        "TESTSERIAL"
    ]
}

##############################

Run the failed Merge-Test

• IDArray info is not injected into the merge

[root@ZTPSERVER ~]# ztp request merge-test SERIAL1
2020-06-17 17:49:28:   cfact.get_keystore_id: Checking Keystores and IDArrays for (SERIAL1)

2020-06-17 17:49:28:   cfact.get_keystore_id: Checking Keystore names for (SERIAL1)

2020-06-17 17:49:28:   cfact.get_keystore_id: ID (SERIAL1) not found in keystore names, checking local IDArrays

2020-06-17 17:49:28:   cfact.get_keystore_id: ID 'SERIAL1' resolved to arrayname 'STACK1'

2020-06-17 17:49:28:   cfact.get_keystore_id: Keystore with name 'STACK1' has been found in an external-keystore

2020-06-17 17:49:28:   cfact.get_template: Looking up association for identity (STACK1)

2020-06-17 17:49:28:   cfact.get_template: Found associated template (LONG_TEMPLATE) in an external keystore

2020-06-17 17:49:28:   cfact.get_template: Template (LONG_TEMPLATE) exists in local config. Returning


Some variables in jinja template do not exist in keystore:
        -range


2020-06-17 17:49:28:   cfact._global_lookup: Checking if a global-keystore is configured and ready...

2020-06-17 17:49:28:   cfact._global_lookup: Global-keystore configured as none. Discarding

2020-06-17 17:49:28:   cfact.merge_test: Merging with values:
{
    "vl1_netmask": "255.255.255.0", 
    "hostname": "CORESWITCH", 
    "snmpinfo": {
        "WS_C3850_SERIAL_NUMBER": "WS_C3850_SERIAL_NUMBER_FAKESERIAL", 
        "WS_C2960_SERIAL_NUMBER": "WS_C2960_SERIAL_NUMBER_FAKESERIAL", 
        "matched": "FAKEMATCHEDSERIAL"
    }, 
    "vl1_ip_address": "10.0.0.200", 
    "keystore_id": "STACK1", 
    "idarray": [], 
    "association": "LONG_TEMPLATE"
}