In the HAPI JDBC mode, when resources are created with PUT, resource Id's mismatch between the original FHIR resource and the corresponding resource in the parquet file.

[vbothe23]

During the process of transforming FHIR resources into parquet files, an issue arises where the ID of the resource in the transformed parquet files does not match the original FHIR resource id.

patient resource:

{
    "resourceType": "Patient",
    "id": "6f866a1c-c1af-40a0-94c6-cab20b7f33c2",
    "meta": {
        "versionId": "1",
        "lastUpdated": "2023-07-04T04:54:55.663+00:00",
        "source": "#7xtul0v7pPE2VN0v"
    },
    "text": {
        "status": "generated",
        "div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><div class=\"hapiHeaderText\">PATIENT <b>TEST </b></div><table class=\"hapiPropertyTable\"><tbody><tr><td>Identifier</td><td>P8654</td></tr><tr><td>Address</td><td><span>Abc </span><br/></td></tr><tr><td>Date of birth</td><td><span>15 July 1997</span></td></tr></tbody></table></div>"
    },
    "identifier": [
        {
            "system": "http://example.com/identifiers/patient-card",
            "value": "P8654"
        }
    ],
    "name": [
        {
            "family": "TEST",
            "given": [
                "PATIENT"
            ]
        }
    ],
    "telecom": [
        {
            "system": "phone",
            "value": "+91-0070707070070"
        }
    ],
    "gender": "female",
    "birthDate": "1997-07-15",
    "address": [
        {
            "line": [
                "Abc"
            ]
        }
    ],
    "contact": [
        {
            "telecom": [
                {
                    "system": "phone",
                    "value": "+91-000707070777"
                }
            ]
        }
    ]
}

Content in the parquet file:

{
    "id": "82656",
    "meta": {
        "versionId": "1",
        "lastUpdated": "2023-07-04T04:54:55.000+00:00"
    },
    "text": {
        "status": "generated"
    },
    "identifier": {
        "array": [
            {
                "system": "http://example.com/identifiers/patient-card",
                "value": "P8654"
            }
        ]
    },
    "name": {
        "array": [
            {
                "family": "TEST",
                "given": {
                    "array": [
                        "PATIENT"
                    ]
                }
            }
        ]
    },
    "telecom": {
        "array": [
            {
                "system": "phone",
                "value": "+91-0070707070070"
            }
        ]
    },
    "gender": "female",
    "birthDate": "1997-07-15",
    "address": {
        "array": [
            {
                "line": {
                    "array": [
                        "Abc"
                    ]
                }
            }
        ]
    },
    "contact": {
        "array": [
            {
                "telecom": {
                    "array": [
                        {
                            "system": "phone",
                            "value": "+91-000707070777"
                        }
                    ]
                }
            }
        ]
    }
}

[bashir2]

Thanks @vbothe23 for filing this issue; are you using the FHIR-search mode or direct DB access when fetching resources? Is it possible that in the source FHIR server, there is also a Patient resource with id 82656? This behavior is unexpected and the IDs should be preserved.

[vbothe23]

Hi @bashir2,
I am using JDBC mode to fetch the resources.

In the HAPI FHIR server, resource IDs are automatically generated by the system, even if we explicitly specify an ID during resource creation. These auto-generated IDs are stored in the hfj_resource table, while explicitly specified IDs are stored in the hfj_forced_id table. However, the pipeline relies on the IDs from the hfj_resource table, leading to a discrepancy between the actual resource ID and the ID in the parquet file.

[bashir2]

Thanks @vbothe23 for the clarification; yes this is a bug that we should fix. In the meantime you can use the FHIR search mode (i.e., avoid JDBC) and I believe that should return the original resource IDs.

[bashir2]

This was fixed by #754 thanks @vbothe23.