Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Requesting help about the FHIR Loader project. #70

Closed
Kedar782 opened this issue Jul 22, 2024 · 49 comments · Fixed by #72
Closed

Requesting help about the FHIR Loader project. #70

Kedar782 opened this issue Jul 22, 2024 · 49 comments · Fixed by #72
Labels
pending-user-input Waiting on input from the customer

Comments

@Kedar782
Copy link

I want to add a FHIR Loader project on my new FHIR Data service.
https://github.com/microsoft/fhir-loader

For that, can you please guide me with URLs.

WHere do I get the FHIR service service URL, FHIR audience URL?
Also in the installation process it is asking me for the MSI or SP?

If I decide to go with the SP then how can I create a SP on my fhir service for the bulk loader. I'm using the FHIR service for ONC g10 and We will be adding new data every day through the bulk data in our FHIR service and clients can access this data using the standalone patient apps and EHR apps.
@evachen96
Copy link
Collaborator

Hi @Kedar782 , thanks for your question. We'll take a look and get back to you.

@Kedar782
Copy link
Author

Kedar782 commented Jul 22, 2024 via email

@Kedar782
Copy link
Author

Hi @evachen96,

Do you have any update on this?

@evachen96
Copy link
Collaborator

Hi @Kedar782,

image

  • Where do I get the FHIR service service URL, FHIR audience URL? You can find these in Azure Portal in the info for your FHIR server. They should be the same URL, and you can find this in your Azure Portal under Settings > Authentication > Audience
  • Also in the installation process it is asking me for the MSI or SP? If using the "Deploy to Azure" button, you can choose between MSI or SP.

I would recommend testing FHIR Loader on a FHIR server that does not have SMART on FHIR first, so that we can see if there are any issues there and remove other factors. Then, once we validate that works, we can try with a FHIR server that does have SMART on FHIR. Please let me know how that goes!

@Kedar782
Copy link
Author

Kedar782 commented Jul 31, 2024 via email

@Kedar782
Copy link
Author

Kedar782 commented Jul 31, 2024 via email

@evachen96
Copy link
Collaborator

Hi @Kedar782 , I'm not seeing the image, can you upload it again?
image

Also, please submit a separate issue at https://github.com/Azure-Samples/azure-health-data-and-ai-samples/issues regarding Postman and include details on there.

@Kedar782
Copy link
Author

image

@Kedar782
Copy link
Author

Kedar782 commented Aug 6, 2024

Hello @evachen96
Do you have any update on this?

@evachen96 evachen96 linked a pull request Aug 16, 2024 that will close this issue
Fixes for FHIR Loader ARM Template #72
Merged
@evachen96
Copy link
Collaborator

Hi @Kedar782 , we have fixed the issue that occurs when using service principal for both the bash script and the Deploy to Azure button. Let me know if that works!

@evachen96 evachen96 added the pending-user-input Waiting on input from the customer label Aug 16, 2024
@evachen96
Copy link
Collaborator

Closing this issue, let me know if there are still any questions!

@Kedar782

This comment was marked as duplicate.

Sign in to view
@erikhoward
Copy link

Thanks for the new information. We will take a look.

@erikhoward erikhoward reopened this Sep 9, 2024
@evachen96
Copy link
Collaborator

Hi @Kedar782 - team has reviewed and would like to provide the following information -

  1. Deploy to Azure button: The error message that you shared, "A valid principal ID must be provided for role assignment," typically occurs when the "existing service principal" option is selected, but no principal ID is provided. Currently, there is no validation to ensure that this field is filled. Please ensure that the correct service principal is specified, along with the appropriate secret value.
  2. Deployment using Bash script: We assume the Bash script is being executed in Azure Cloud Shell – Bash Shell after logging in with valid user credentials. The "assignee" parameter in the az role assignment command refers to the user account executing the script. Could you verify if the script is being run using the same user account that was used to log into Azure Cloud Shell – Bash Shell? Additionally, ensure the script is being executed within the same tenant as that user.

Attaching some screenshots as well to show how the deployment typically looks like using the Deploy to Azure button. We followed the same sequence (first accessed the FHIR service using Postman and used same app registration while deploying the FHIR Loader using Service Principal):

btndeploy1
btndeploy2
btndeploy3
btndeploy4
btndeploy5
btndeploy6

@Kedar782
Copy link
Author

Kedar782 commented Sep 9, 2024 via email

@Kedar782
Copy link
Author

@evachen96

Do you have any update on this?

Please let me know if you need any more information from me.

@evachen96
Copy link
Collaborator

Hi Kedar - the team is still working on trying to reproduce your issue. In the meantime, we have added hopefully more clear instructions on how to deploy the FHIR loader using Azure Portal and more clear prompts in the Bash script in the latest PR (#73). Please try https://github.com/microsoft/fhir-loader/blob/main/docs/portaldeployment.md and https://github.com/microsoft/fhir-loader/blob/main/scripts/deployFhirBulk.bash and let us know if you're able to resolve your issue.

@Kedar782
Copy link
Author

Kedar782 commented Sep 16, 2024 •
edited
Loading

@evachen96

I tried again with the detail description documents you shared but still getting the same errors.

  1. Using the button I get an error

image

2.Using the script I'm getting error which is

Cannot find user or service principal in graph database for 'live.com#XXXXXi@XXXXXXX.com'. If the assignee is an appId, make sure the corresponding service principal is created with 'az ad sp create --id live.com#XXXXXi@XXXXXXX.com'.

@evachen96
Copy link
Collaborator

@Kedar782 , could we get the following details?

  1. A complete screenshot and additional details regarding which resource failed during deployment when using the "Deploy to Azure" button.
  2. Regarding the Bash script error: Is the user (assignee) shown in the error is an invited user created via an email invitation, or if the user was created directly within Entra ID?
    Thanks!

@Kedar782
Copy link
Author

@evachen96,
1.I tried again using button to send you the error screenshot adn it worked this time. I did not get any error.
When I try to add a bundle file in bundles container on a storage container that file is not going in bundlesprocessed or bundleserr container. So data is not going in the fhir data service. Please note that I manually created bundlesprocessed and bundleserr container. Installation process only created the bundles container.
2. I do not have any guest user on this azure health data service resource. But this user is gues on a different azure account.
Can you please let me know how can I confirm I'm using the correct user. I have only one user for this account and is the owner for azure health data service resource. I'm using the same user for the bash script. Please let me know any extra steps to confirm any other thing related to this user.

Please let me knoe if you need any more information from us.

@Kedar782
Copy link
Author

@evachen96

I added a bundle file in the bundles container but still it is in bundles container.
image

@Kedar782
Copy link
Author

Kedar782 commented Sep 23, 2024 via email

@evachen96
Copy link
Collaborator

Hi @Kedar782 -

Based on the information provided in the GitHub issue, we recommend the following steps:

Deploy to Azure Button:

It appears that the deployment was completed, but the setup is corrupted and wasn’t properly configured during the process.
Can you perform a fresh setup using a new resource group?

Bash Script:

To determine if the user is a native or invited user in the directory, you can navigate to Entra ID > Users and check the User Principal ID of the account used for deployment:
A native user in the tenant is typically formatted like: XXXXXX@XXXXXX.com
An invited user is formatted like: XXXX#EXT#@XXXXXX.XXXX.com
For now, we suggest that you create a new user account directly in Entra ID, assign the necessary roles, and attempt the deployment again.

Thanks!

@Kedar782
Copy link
Author

@evachen96

I tried to deploy it on brand new resource and now I'm getting the same error again.(Using the deploy to Azure button)

image

Please let me know if you need any more information from me.

I will try it with bash script and let you know.

@Kedar782
Copy link
Author

@evachen96

I have only one user in my azure ad account and i can pass all inferno test using this user.
Also, I can insert data in fhir data service using the postman with this user.
It does has a EXT in it but it's User Type is a Member.
I added FHIR Loader on my old fhir server (Azure API for FHIR) using the same user.
I have deadline for this project which is approaching very fast. Can you please help me with this?
I have tried everything.

@Kedar782
Copy link
Author

@evachen96

I created a new user in azure AD and assign a role of owner for this user in the subscription where my fhir data service is located.
I also assigned a role of owner in my fhir data service resource for this user. I also assigned a role of fhir data contributor for this user in my fhir data service.
Then using the bash script I successfully deployed the fhir Loader project.
In the deployment process I selected SP in MSI/SP option and I used the client id and clinet secret of the app where I can successfully insert and retirve the data using the postman.
When I upload a bundle in the bundles container I get error response in the bundleserr container stating Authentication failed.
If I use the same client in POSTMAN I can insert data.

Please advised.

@evachen96
Copy link
Collaborator

Hi @Kedar782 , comments from the team:

Deploy to Azure Button:
It appears that the deployment failed during role assignment to the storage account, as the storage account was not found. We couldn’t reproduce the issue on our end. However, to assist us in troubleshooting, could you provide the following details?

  • Verify if the storage account was created in the resource group.
  • Let us know the roles assigned to the user (in Entra ID, subscription level, etc.) being used for the deployment. We will assign similar roles to a test user and try to reproduce the issue.

Bash Script:
Could you provide additional details on how the user was created or invited? We will replicate the user creation process and investigate further.
In the meantime, we will continue to investigate this issue.

@Kedar782
Copy link
Author

Hello @evachen96 ,

Right now I have two users in my azure account.
Originally I had only one user in my account which was of the type Guest.
In Entra ID this user has a role of Global Administrator.
In subscription level this user has a role of Owner.

After you asked me to create a brand new member user, I created a new Member user in Entra ID(without inviting) and it has a role of Application Administrator and Application Developer role in Entra ID. It has a Owner role at the subscription level.

Both of this users has owner role on the Fhir data service resource group and FHIR Data contributor role on the FHIR Data Service.

If I use both above users on the deploy to azure button I get the same error that I posted.(role assignment to the storage account).
Yes using both users the storage account was created in the resource group.

Bash Script:

If I use bash script using the new user I created (Member) I can able to deploy the FHIR Loader but when I add a bundle in bundles container I get error for that bundle stating authentication failed.
I'm using Service Principle for the deployment process and I'm using the same application that I use for POSTMAN clinet credential flow to insert data into my fhir server and it works fine.

If I use GUEST user I get error in the deployment process stating role assignement failed. I posted this error in the issue.

Please let me know if you need any more information from me.

@evachen96
Copy link
Collaborator

evachen96 commented Sep 30, 2024 •
edited
Loading

We tried to reproduce the issue in-house on multiple tenants but couldn’t replicate it, nor have we encountered this specific problem before. We successfully deployed the FHIR Loader sample using both a bash script and the "Deploy to Azure" button with a new Member user who has the same roles mentioned by you.

You may experience certain deployment issues that could be specific to Azure infrastructure and may not be persistent. We recommend redeploying the sample using the redeploy button if you encounter any problems. Also, please check if the following role assignments are granted to the new Member user in your environment:

image

Note: Ensuring all prerequisites are met and following the documented deployment steps correctly should make it work.

@Kedar782
Copy link
Author

Kedar782 commented Oct 1, 2024

@evachen96
Can you please let me know what storage account I need to assign a role oif owner?
As I said before, I already have owner role on the FHIR data service resource.
So every storage account in this resource group has a role of owner for the user.

@Kedar782
Copy link
Author

Kedar782 commented Oct 1, 2024

@evachen96 @erikhoward
I confirmed that I have above roles assigned to the user.
Still I'm getting the same error again and again.

image

@Kedar782
Copy link
Author

Kedar782 commented Oct 3, 2024 •
edited
Loading

@erikhoward @evachen96

Do you have any update on this issue?

I tried to deploy it again and I'm getting the same error.

image
image
{
"code": "DeploymentFailed",
"target": "/subscriptions/c4cbd8a0-XXXX-XXXX-XXXX-XXXXc852f414/resourceGroups/calmedfhirproduction12-rg/providers/Microsoft.Resources/deployments/role-assign-storage",
"message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.",
"details": [
{
"code": "ResourceNotFound",
"message": "The Resource 'Microsoft.Storage/storageAccounts/bulkvw67stor' under resource group 'calmedfhirproduction12-rg' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"
},
{
"code": "ResourceNotFound",
"message": "The Resource 'Microsoft.Storage/storageAccounts/bulkvw67stor' under resource group 'calmedfhirproduction12-rg' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"
},
{
"code": "ResourceNotFound",
"message": "The Resource 'Microsoft.Storage/storageAccounts/bulkvw67stor' under resource group 'calmedfhirproduction12-rg' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"
}
]
}

Error is saying that storage account bulkvw67stor is not found under the resource calmedfhirproduction12-rg which is the resource for fhir data service. This storage account bulkvw67stor is created under new resource I create in the custom deployment process for the fhir loader app.

@evachen96
Copy link
Collaborator

Hi @Kedar782 -

From the screenshot, it appears that the resource group used for deployment differs from the resource group where the error occurred. The FHIR Loader template deploys the Storage Account and assigns roles within the resource group specified during deployment. This suggests that an incorrect configuration might be causing the error you're encountering. We recommend reviewing the deployment configuration you're using. Please refer to the deployment guide for detailed instructions. Additionally, if possible, please share the exact steps and screenshots of your deployment process so we can attempt to reproduce the issue on our side.

Additionally, can you please try with completely new setup including creating new FHIR service also? Create new resource group > Create FHIR service in it > Use this resource group for FHIR loader resources deployment. So finally, FHIR service and all other resource would be in same resource group. The FHIR Loader should be deployed in the same resource group as the FHIR server.

image

@Kedar782
Copy link
Author

Kedar782 commented Oct 3, 2024 •
edited
Loading

@evachen96 @erikhoward

I will try to explain.

I'm creating a new resource group in the deployment process called bulk121212.
calmedfhirproduction12-rg is the resource group where my fhir service is at.
I'm deploying fhir bulk loader in the brand new resource bulk121212.

Is it wrong? I alos tried to deploy it in the fhir service resource and i'm still experiencing issues.

image

image

image

image
image
image
image
image

Please let me know if anything is wrong in the deployment process.

Got the same error again.

image

@evachen96
Copy link
Collaborator

Hi @Kedar782 , you should deploy the FHIR Loader in the same resource group that your FHIR service is located in.

@Kedar782
Copy link
Author

Kedar782 commented Oct 3, 2024

@evachen96

Is it mandatory?

@Kedar782
Copy link
Author

Kedar782 commented Oct 3, 2024

@evachen96 @erikhoward

If I try to deploy it in the same resource group which calmedfhirproduction12-rg(Resource group for the fhir service).
I get different error now which is below
image

{
"code": "Endpoint validation",
"message": "Destination endpoint not found. Resource details: resourceId: /subscriptions/c4cbd8a0-XXXX-XXXX-XXXX-694ac852f414/resourceGroups/calmedfhirproduction12-rg/providers/Microsoft.Storage/storageAccounts/bulki4x4stor. Resource should pre-exist before attempting this operation. Activity id:1731b373-f64f-4966-844f-e1af4639f103, timestamp: 10/3/2024 7:14:33 PM (UTC)."
}

image

{
"code": "Endpoint validation",
"message": "Destination endpoint not found. Resource details: resourceId: /subscriptions/c4cbd8a0-XXXX-XXXX-XXXX-694ac852f414/resourceGroups/calmedfhirproduction12-rg/providers/Microsoft.Storage/storageAccounts/bulki4x4stor. Resource should pre-exist before attempting this operation. Activity id:fa0834c9-7a0c-462d-acd9-0292be4f5968, timestamp: 10/3/2024 7:14:33 PM (UTC)."
}
image
image

Please advice.

@evachen96
Copy link
Collaborator

evachen96 commented Oct 7, 2024 •
edited
Loading

Hi @Kedar782, we are investigating the error message that you shared to see why that error may be coming up when you use Deploy to Azure button in the existing resource group where the FHIR service is located. In the meantime, can you try the new setup process from scratching following the steps below and ensuring both the FHIR service and FHIR Loader are deployed in the same resource group?

  1. Use a newly created user with the following roles:

  • Entra ID role: Global Administrator

  • Subscription level: Owner

  1. Create a new resource group and deploy a new AHDS workspace and FHIR service within the same resource group.
  2. Create an app registration (SP) and assign the FHIR Data Contributor role to this app on the FHIR service.
  3. Confirm that you can access the FHIR service via POSTMAN using this SP.
  4. Once confirmed, remove the FHIR Data contributor role assigned to above app on FHIR service. (This will be assigned again by script during deployment)
  5. Follow the 'Deploy to Azure' button wizard to deploy the FHIR Loader.
  6. Ensure that the newly created resource group and FHIR service is selected, the correct existing SP is used, and valid credentials (secret) are provided.
  7. Proceed with the deployment and verify.

Thanks!

@Kedar782
Copy link
Author

Kedar782 commented Oct 8, 2024

@evachen96

Can you please give me link to the documentation to publish HDS workspace and FHIR service?
Last time Ideployed it using the ONC g10 instructions from the azure ai samples repo.

Kedarnath

@evachen96
Copy link
Collaborator

Hi @Kedar782 - you can deploy AHDS workspace and FHIR service directly from Azure Portal. Please note that you will need to deploy a AHDS workspace first, and then inside of the AHDS workspace, deploy a FHIR service. Here is the info: https://learn.microsoft.com/en-us/azure/healthcare-apis/healthcare-apis-quickstart (Creating AHDS workspace in Azure Portal)
https://learn.microsoft.com/en-us/azure/healthcare-apis/fhir/deploy-azure-portal (Deploy FHIR service in the new AHDS workspace that you just created).

Hope that helps!

@Kedar782
Copy link
Author

Kedar782 commented Oct 8, 2024

@evachen96

It worked.

Why it is not working on my old fhir data service? My instinct is saying because I do not use SMART on FHIR for this new FHIR data service.

Can you please let me know what I'm doing wrong?

Kedarnath

@evachen96
Copy link
Collaborator

Hi @Kedar782 - we are trying to investigate why that might be the case and will get back to you if we find anything.

@Kedar782
Copy link
Author

@erikhoward @evachen96

Did you guys find anything?
Have you guys tried this with the server which has ONC smart on fhir?

Please guide me. I have a deadline and I'm working on this from long time.

@evachen96
Copy link
Collaborator

Hi @Kedar782 , we are still working on it and will let you know when we have an update.

@Kedar782
Copy link
Author

Kedar782 commented Oct 18, 2024 via email

@evachen96
Copy link
Collaborator

@Kedar782
We do not support SMART on FHIR and FHIR Loader running on the same FHIR server, as SMART on FHIR only supports reads, not writes (see note here).

However, if you choose to still use SMART on FHIR and FHIR Loader together on the same FHIR server, you would need to set up FHIR Loader to interact with the FHIR server directly, not through SMART on FHIR. So, when you set up FHIR Loader, you would need to use the normal audience URL of the FHIR server, not the APIM endpoint that is set up with SMART on FHIR. We have provided screenshot on how to get the FHIR audience on the other Github issue.

We have also fixed the issue you brought up earlier where deploying FHIR Loader in a different resource group than the FHIR server using Service Principal was giving problems in this PR #75

@Kedar782
Copy link
Author

@erikhoward @evachen96

I tried to create a brand new fhir server and then I added fhir loader project on it.
It is working. However, after that I need to add smart on fhir on my server and pass the inferno test.
I followed every step from this.
https://github.com/Azure-Samples/azure-health-data-and-ai-samples/blob/main/samples/patientandpopulationservices-smartonfhir-oncg10/docs/deployment.md

But my first test is failing after it opens a context app stating 502 bad gateway.

So I have two scenarios with two fhir server(Please note that I tried above after long time because I have not acheived anything)

and the second scenario for which I created this issue.

I have a smart on fhir server on second fhir data service and when I try to add fhir loader without the smart url I get error that I shared in this issue.

@Kedar782
Copy link
Author

@evachen96 @erikhoward

Hello Eva,

I tried to create a new fhir data service and then I' trying to add smart on fhir it for ONC.
But I'm getting error for it.
I created a new issue for it. Please look through it. If it works i can add a new fhir loader sample on it.

Azure-Samples/azure-health-data-and-ai-samples#247

It is long time I'm working on this. I'm very afraid for our deadline. Please advise.

@evachen96
Copy link
Collaborator

Adding answer here and closing this issue as we have confirmed over email that the following resolves the issue:

Team has investigated and resolved the issue. Ask from you is to deploy FHIR loader with latest update
Also with this new deployment, if you encounter failure of ndjsoncreated or bundlecreated deployments, please follow the steps below:

Go to the resource group where you are trying to deploy the FHIR Loader.
Navigate to Settings > Deployments.
Select the most recent failed deployment.
Click on the Redeploy button as shown below.

Image

The Custom Deployment Wizard will open with pre-populated values.
Fill in the required fields correctly, as indicated in the screenshot below.

Image
Image

Click on Review + Create.

Please check if the redeployment process resolves the deployment issue.

Post Deployment actions:

After deployment completes successfully, please verify below configurations before using FHIR Loader functionality.

Go to resource group where FHIR Loader is deployed.
Go to Function app resource
Navigate to Settings > Environment Variables, check values of below variables are correct
FS-CLIENT-ID = Service principal Client ID
FS-SECRET = Service principal secret
FS-RESOURCE = FHIR Service Audience URL. (FHIR Service > Settings > Authentication > Audience)
FS-URL = FHIR Server URL ( FHIR Service > Overview > FHIR metadata endpoint without ‘/metadata’)

Image

Once above steps are completed and validated try uploading the bundle in required container.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
pending-user-input Waiting on input from the customer
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fixes for FHIR Loader ARM Template microsoft/fhir-loader
3 participants
@erikhoward @evachen96 @Kedar782