BulkData: Delete Job #2713

dabimahesh · 2021-08-24T15:02:17Z

We are getting "An unexpected error has occurred while deleting the job", when a DELETE is initiated. Is it that we are missing any configuration?

prb112 · 2021-08-24T15:05:56Z

We are getting "An unexpected error has occurred while deleting the job", when a DELETE is initiated. Is it that we are missing any configuration?

@dabimahesh Thank you for opening

A few questions:
1 - What is the topology of your environment? Do you have more than one node?
2 - can you share your logs? messages.log and trace.log
3 - Can you share the rest call you executed?

With these questions asked, I can more easily replicate your issue locally.

dabimahesh · 2021-08-25T11:00:24Z

Hello Paul,
Please find below some points to your questions:

1 - We have one server on ubuntu and we are using Postgre as DB. We are in the process of testing IBM FHIR server on Inferno testing tool

 2 - We are using SmartOnFHIR and the token used is as below
	{
	  "iss": "https://identityserverforfhir.azurewebsites.net",
	  "nbf": 1629885567,
	  "iat": 1629885567,
	  "exp": 1629889167,
	  "aud": [
		"https://fhirserverurl:9443/fhir-server/api/v4",
		"Inferno"
	  ],
	  "scope": "system/AllergyIntolerance.read system/CarePlan.read system/CareTeam.read system/Condition.read system/Device.read system/DiagnosticReport.read system/DocumentReference.read system/Encounter.read system/Goal.read system/Immunization.read system/Location.read system/Medication.read system/MedicationRequest.read system/Observation.read system/Organization.read system/Patient.read system/Practitioner.read system/Procedure.read system/Provenance.read",
	  "group": [
		"FHIRUsers",
		"FHIRAdmins"
	  ],
	  "client_id": "Inferno-Bulk",
	  "patient": "17b5dc09184-54ce1834-2501-4e85-a3fe-e04223da69cf",
	  "patient_id": "17b5dc09184-54ce1834-2501-4e85-a3fe-e04223da69cf",
	  "upn": "fhiruser",
	  "need_patient_banner": "false",
	  "smart_style_url": "https://identityserverforfhir.azurewebsites.net/SmartOnFHIR/v1.json",
	  "jti": "1BA86FE46C70CFA13522743DFAB4EACA"
	}

3 - Operation GET (worked fine)

		GET   https://fhirserverurl:9443/fhir-server/api/v4/$bulkdata-status?job=qqe8NaKf76Eh0aFyQJCfeg
         
         Response Body:
         {
    "transactionTime": "2021-08-25T10:01:50.175Z",
    "request": "https://fhirserverurl:9443/fhir-server/api/v4/Group/17b5e38b010-74858f83-7570-45cb-979b-8d00b694a293/$export",
    "requiresAccessToken": false,
    "output": [
        {
            "type": "Provenance",
            "url": "PaUz4moaf-DMx9VeIMWhNARJFlRct4UGTotPw9iTRQg/Provenance_1.ndjson",
            "count": 1
        },
        {
            "type": "AllergyIntolerance",
            "url": "PaUz4moaf-DMx9VeIMWhNARJFlRct4UGTotPw9iTRQg/AllergyIntolerance_1.ndjson",
            "count": 1
        },
        {
            "type": "CarePlan",
            "url": "PaUz4moaf-DMx9VeIMWhNARJFlRct4UGTotPw9iTRQg/CarePlan_1.ndjson",
            "count": 1
        },
        {
            "type": "Goal",
            "url": "PaUz4moaf-DMx9VeIMWhNARJFlRct4UGTotPw9iTRQg/Goal_1.ndjson",
            "count": 1
        },
        {
            "type": "Procedure",
            "url": "PaUz4moaf-DMx9VeIMWhNARJFlRct4UGTotPw9iTRQg/Procedure_1.ndjson",
            "count": 1
        },
        {
            "type": "DocumentReference",
            "url": "PaUz4moaf-DMx9VeIMWhNARJFlRct4UGTotPw9iTRQg/DocumentReference_1.ndjson",
            "count": 5
        },
        {
            "type": "Immunization",
            "url": "PaUz4moaf-DMx9VeIMWhNARJFlRct4UGTotPw9iTRQg/Immunization_1.ndjson",
            "count": 1
        },
        {
            "type": "MedicationRequest",
            "url": "PaUz4moaf-DMx9VeIMWhNARJFlRct4UGTotPw9iTRQg/MedicationRequest_1.ndjson",
            "count": 5
        },
        {
            "type": "Observation",
            "url": "PaUz4moaf-DMx9VeIMWhNARJFlRct4UGTotPw9iTRQg/Observation_1.ndjson",
            "count": 14
        },
        {
            "type": "Group",
            "url": "PaUz4moaf-DMx9VeIMWhNARJFlRct4UGTotPw9iTRQg/Group_1.ndjson",
            "count": 1
        },
        {
            "type": "CareTeam",
            "url": "PaUz4moaf-DMx9VeIMWhNARJFlRct4UGTotPw9iTRQg/CareTeam_1.ndjson",
            "count": 2
        },
        {
            "type": "DiagnosticReport",
            "url": "PaUz4moaf-DMx9VeIMWhNARJFlRct4UGTotPw9iTRQg/DiagnosticReport_1.ndjson",
            "count": 4
        },
        {
            "type": "Encounter",
            "url": "PaUz4moaf-DMx9VeIMWhNARJFlRct4UGTotPw9iTRQg/Encounter_1.ndjson",
            "count": 1
        },
        {
            "type": "Condition",
            "url": "PaUz4moaf-DMx9VeIMWhNARJFlRct4UGTotPw9iTRQg/Condition_1.ndjson",
            "count": 1
        },
        {
            "type": "Patient",
            "url": "PaUz4moaf-DMx9VeIMWhNARJFlRct4UGTotPw9iTRQg/Patient_1.ndjson",
            "count": 1
        }
    ]
}


    Message.log: 

     [8/25/21, 10:05:35:480 UTC] 0000004e com.ibm.fhir.server.filter.rest.FHIRRestServletFilter        I Received request: tenantId:[default] dsId:[default] user:[fhiruser] method:[GET] uri:[https://fhirserverurl:9443/fhir-server/api/v4/$bulkdata-status?job=qqe8NaKf76Eh0aFyQJCfeg]
[8/25/21, 10:05:35:628 UTC] 0000004e com.ibm.fhir.operation.bulkdata.client.BulkDataClient        W Logging the JobExecutionResponse Details -> 
{
    "jobName": "bulkgroupexportchunkjob",
    "instanceId": 78,
    "batchStatus": "COMPLETED",
    "exitStatus": "Provenance[1]:AllergyIntolerance[1]:CarePlan[1]:Goal[1]:Procedure[1]:DocumentReference[5]:Immunization[1]:MedicationRequest[5]:Observation[14]:Group[1]:CareTeam[2]:DiagnosticReport[4]:Encounter[1]:Condition[1]:Patient[1]",
    "lastUpdatedTime": "2021/08/25 10:01:50.175 +0000",
    "_links": [
        {
            "href": "https://fhirserverurl:9443/ibm/api/batch/jobexecutions/78",
            "rel": "self"
        },
        {
            "href": "https://fhirserverurl:9443/ibm/api/batch/jobinstances/78",
            "rel": "job instance"
        },
        {
            "href": "https://fhirserverurl:9443/ibm/api/batch/jobexecutions/78/stepexecutions",
            "rel": "step executions"
        },
        {
            "href": "https://127.0.0.1:9443/ibm/api/batch/jobexecutions/78/joblogs",
            "rel": "job logs"
        },
        {
            "href": "https://127.0.0.1:9443/ibm/api/batch/jobexecutions/78?action=stop",
            "rel": "stop url"
        }
    ],
    "jobParameters": {
        "fhir.resourcetype": "Appointment,Account,Provenance,Invoice,ExplanationOfBenefit,DocumentManifest,Specimen,AllergyIntolerance,CarePlan,Goal,EnrollmentRequest,EpisodeOfCare,Consent,Procedure,List,ResearchSubject,DocumentReference,Immunization,MedicationRequest,CoverageEligibilityResponse,VisionPrescription,Media,MolecularSequence,MeasureReport,Flag,RelatedPerson,ServiceRequest,SupplyRequest,AppointmentResponse,AdverseEvent,BodyStructure,Observation,MedicationAdministration,MedicationStatement,Person,CommunicationRequest,RiskAssessment,Basic,Group,CareTeam,ClaimResponse,MedicationDispense,DiagnosticReport,ImagingStudy,FamilyMemberHistory,ChargeItem,NutritionOrder,Encounter,AuditEvent,Communication,Condition,Composition,DetectedIssue,Patient,Coverage,ImmunizationEvaluation,QuestionnaireResponse,DeviceUseStatement,CoverageEligibilityRequest,RequestGroup,Schedule,SupplyDelivery,ClinicalImpression,DeviceRequest,Claim,ImmunizationRecommendation",
        "fhir.tenant": "default",
        "fhir.search.patientgroupid": "17b5e38b010-74858f83-7570-45cb-979b-8d00b694a293",
        "fhir.datastoreid": "default",
        "fhir.bulkdata.source": "default",
        "fhir.bulkdata.outcome": "default",
        "import.fhir.storagetype": "https",
        "fhir.incomingUrl": "https://fhirserverurl:9443/fhir-server/api/v4/Group/17b5e38b010-74858f83-7570-45cb-979b-8d00b694a293/$export"
    }
}
[8/25/21, 10:05:35:639 UTC] 0000004e com.ibm.fhir.server.filter.rest.FHIRRestServletFilter        I Completed request[0.16 secs]: tenantId:[default] dsId:[default] user:[fhiruser] method:[GET] uri:[https://fhirserverurl:9443/fhir-server/api/v4/$bulkdata-status?job=qqe8NaKf76Eh0aFyQJCfeg] status:[200]



4 - Operation DELETE    (Error)

	DELETE https://fhirserverurl:9443/fhir-server/api/v4/$bulkdata-status?job=qqe8NaKf76Eh0aFyQJCfeg
		
	    Response Body:      
	  
	    {
		"resourceType": "OperationOutcome",
		"id": "7f-0-0-1-d8d3418e-b457-46a8-b302-cfbdaebf8547",
		"issue": [
			{
				"severity": "error",
				"code": "transient",
				"diagnostics": "An unexpected error has ocurred while deleting the job"
			}
		]
		}
		
		Messages.log
		
		[8/25/21, 10:06:33:754 UTC] 00000053 com.ibm.fhir.server.filter.rest.FHIRRestServletFilter        I Received request: tenantId:[default] dsId:[default] user:[fhiruser] method:[DELETE] uri:[https://fhirserverurl:9443/fhir-server/api/v4/$bulkdata-status?job=qqe8NaKf76Eh0aFyQJCfeg]
	[8/25/21, 10:06:33:912 UTC] 00000053 com.ibm.fhir.server.resources.FHIRResource                   E An unexpected error has ocurred while deleting the job
	com.ibm.fhir.exception.FHIROperationException: An unexpected error has ocurred while deleting the job  [probeId=7f-0-0-1-a47e76a0-d64b-477b-be28-c028e0d547fb]
		at com.ibm.fhir.operation.bulkdata.util.BulkDataExportUtil.buildOperationException(BulkDataExportUtil.java:143)
		at com.ibm.fhir.operation.bulkdata.util.BulkDataExportUtil.buildOperationException(BulkDataExportUtil.java:139)
		at com.ibm.fhir.operation.bulkdata.client.BulkDataClient.runDeleteJobForTenant(BulkDataClient.java:598)
		at com.ibm.fhir.operation.bulkdata.client.BulkDataClient.delete(BulkDataClient.java:444)
		at com.ibm.fhir.operation.bulkdata.processor.impl.ExportImportImpl.delete(ExportImportImpl.java:154)
		at com.ibm.fhir.operation.bulkdata.StatusOperation.doInvoke(StatusOperation.java:61)
		at com.ibm.fhir.server.operation.spi.AbstractOperation.invoke(AbstractOperation.java:66)
		at com.ibm.fhir.server.util.FHIRRestHelper.doInvoke(FHIRRestHelper.java:1169)
		at com.ibm.fhir.server.resources.Operation.invokeDelete(Operation.java:171)
		at com.ibm.fhir.server.resources.Operation$Proxy$_$$_WeldClientProxy.invokeDelete(Unknown Source)
		at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
		at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
		at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
		at java.base/java.lang.reflect.Method.invoke(Method.java:566)
		at com.ibm.ws.jaxrs20.cdi.component.JaxRsFactoryImplicitBeanCDICustomizer.serviceInvoke(JaxRsFactoryImplicitBeanCDICustomizer.java:342)
		at com.ibm.ws.jaxrs20.server.LibertyJaxRsServerFactoryBean.performInvocation(LibertyJaxRsServerFactoryBean.java:641)
		at com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.performInvocation(LibertyJaxRsInvoker.java:160)
		at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:101)
		at com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.invoke(LibertyJaxRsInvoker.java:273)
		at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:213)
		at com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.invoke(LibertyJaxRsInvoker.java:444)
		at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:112)
		at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
		at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96)
		at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
		at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:123)
		at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:274)
		at com.ibm.ws.jaxrs20.endpoint.AbstractJaxRsWebEndpoint.invoke(AbstractJaxRsWebEndpoint.java:137)
		at com.ibm.websphere.jaxrs.server.IBMRestServlet.handleRequest(IBMRestServlet.java:146)
		at com.ibm.websphere.jaxrs.server.IBMRestServlet.doDelete(IBMRestServlet.java:131)
		at javax.servlet.http.HttpServlet.service(HttpServlet.java:712)
		at com.ibm.websphere.jaxrs.server.IBMRestServlet.service(IBMRestServlet.java:96)
		at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1258)
		at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:746)
		at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:443)
		at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:183)
		at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:94)
		at com.ibm.fhir.server.filter.rest.FHIRRestServletFilter.doFilter(FHIRRestServletFilter.java:155)
		at javax.servlet.http.HttpFilter.doFilter(HttpFilter.java:127)
		at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:197)
		at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
		at com.ibm.ws.security.jaspi.JaspiServletFilter.doFilter(JaspiServletFilter.java:56)
		at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:197)
		at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
		at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:1002)
		at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1140)
		at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1011)
		at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:75)
		at com.ibm.ws.webcontainer40.servlet.CacheServletWrapper40.handleRequest(CacheServletWrapper40.java:85)
		at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:938)
		at com.ibm.ws.webcontainer.osgi.DynamicVirtualHost$2.run(DynamicVirtualHost.java:279)
		at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink$TaskWrapper.run(HttpDispatcherLink.java:1159)
		at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.wrapHandlerAndExecute(HttpDispatcherLink.java:428)
		at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.ready(HttpDispatcherLink.java:387)
		at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:566)
		at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleNewRequest(HttpInboundLink.java:500)
		at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.processRequest(HttpInboundLink.java:360)
		at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.ready(HttpInboundLink.java:327)
		at com.ibm.ws.channel.ssl.internal.SSLConnectionLink.determineNextChannel(SSLConnectionLink.java:1100)
		at com.ibm.ws.channel.ssl.internal.SSLConnectionLink$MyReadCompletedCallback.complete(SSLConnectionLink.java:675)
		at com.ibm.ws.channel.ssl.internal.SSLReadServiceContext$SSLReadCompletedCallback.complete(SSLReadServiceContext.java:1824)
		at com.ibm.ws.tcpchannel.internal.WorkQueueManager.requestComplete(WorkQueueManager.java:504)
		at com.ibm.ws.tcpchannel.internal.WorkQueueManager.attemptIO(WorkQueueManager.java:574)
		at com.ibm.ws.tcpchannel.internal.WorkQueueManager.workerRun(WorkQueueManager.java:958)
		at com.ibm.ws.tcpchannel.internal.WorkQueueManager$Worker.run(WorkQueueManager.java:1047)
		at com.ibm.ws.threading.internal.ExecutorServiceImpl$RunnableWrapper.run(ExecutorServiceImpl.java:238)
		at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
		at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
		at java.base/java.lang.Thread.run(Thread.java:829)

	[8/25/21, 10:06:33:919 UTC] 00000053 com.ibm.fhir.server.filter.rest.FHIRRestServletFilter        I Completed request[0.166 secs]: tenantId:[default] dsId:[default] user:[fhiruser] method:[DELETE] uri:[https://fhirserverurl:9443/fhir-server/api/v4/$bulkdata-status?job=qqe8NaKf76Eh0aFyQJCfeg] status:[500]

Please let us know if you need any other information.

dabimahesh · 2021-08-25T12:18:29Z

Hi Paul

I think before looking into above logs, we reviewed the code and found out that the delete job URL is not getting formed correctly.

https://fhir.secureit.co.in:9443/ibm/api/batch/**jobexecutions**/85

There is nothing called jobexecutions so it we change to jobinstances it works, so URL should be

https://fhir.secureit.co.in:9443/ibm/api/batch/**jobinstances**/85

I think if we change the following code in BulkDataClient.Java it will work. This is just our code review suggestion. Let me know your thoughts.
fhir-operation-bulkdata/src/main/java/com/ibm/fhir/operation/bulkdata/client/BulkDataClient.java#L571

prb112 · 2021-08-25T12:20:53Z

Thanks Dabimahesh... I'll look at this... there are a few reasons why it calls jobexecutions as it may have multiple running executions.

it's a little bit more complicated than just changing it. I'll definitely consider all the data you have provided and comments. Many thanks

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

prb112 · 2021-08-26T00:00:14Z

Thanks @dabimahesh great find - I confirmed your fix and raised a PR. I'll add an IntegrationTest to confirm.

dabimahesh · 2021-08-26T03:13:37Z

Great. How we can get this fix...rather the one fixed in this sprint?

Mahesh

BulkData: Delete Job #2713

lmsurpre · 2021-09-07T18:10:43Z

I've been working with the updated implementation and I cannot recreate the An unexpected error has occurred while deleting the job error any longer.

There is still some very "interesting" behavior when it comes to stopping (DELETE /$bulkdata-status) and resuming, (GET /$bulkdata-status), but I will open separate issues for tracking those.

A DELETE to a completed job is consistently producing the proper result for me (202, followed by a 404--as if the job had never existed).

dabimahesh · 2021-09-21T14:04:26Z

Following are different log information when trying to DELETE a job. (4.9.1 version)

Message.log
[INFO ] Completed request[0.372 secs]: tenantId:[default] dsId:[default] user:[fhiruser] method:[DELETE] uri:[https://fhir.secureit.co.in:9443/fhir-server/api/v4/$bulkdata-status?job=zViDXoINlCCzE9mXfdDFmw] status:[404]

trace.log
[9/21/21, 10:55:40:073 UTC] 00000042 FHIRResource 1 The Job is stopped intentionally with an in memory database.
com.ibm.fhir.exception.FHIROperationException: The Job is stopped intentionally with an in memory database. [probeId=7f-0-0-1-10adbe1f-4351-46d8-9891-15275dfee94b]
at com.ibm.fhir.operation.bulkdata.util.BulkDataExportUtil.buildOperationException(BulkDataExportUtil.java:143)
at com.ibm.fhir.operation.bulkdata.util.BulkDataExportUtil.buildOperationException(BulkDataExportUtil.java:139)
at com.ibm.fhir.operation.bulkdata.client.action.batch.BatchCancelRequestAction.run(BatchCancelRequestAction.java:122)
at com.ibm.fhir.operation.bulkdata.client.BulkDataClient.delete(BulkDataClient.java:436)
at com.ibm.fhir.operation.bulkdata.processor.impl.ExportImportImpl.delete(ExportImportImpl.java:154)
at com.ibm.fhir.operation.bulkdata.StatusOperation.doInvoke(StatusOperation.java:61)
at com.ibm.fhir.server.operation.spi.AbstractOperation.invoke(AbstractOperation.java:56)
at com.ibm.fhir.server.util.FHIRRestHelper.doInvoke(FHIRRestHelper.java:1171)
at com.ibm.fhir.server.resources.Operation.invokeDelete(Operation.java:171)
at com.ibm.fhir.server.resources.Operation$Proxy$_$$_WeldClientProxy.invokeDelete(Unknown Source)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.ibm.ws.jaxrs20.cdi.component.JaxRsFactoryImplicitBeanCDICustomizer.serviceInvoke(JaxRsFactoryImplicitBeanCDICustomizer.java:342)
at com.ibm.ws.jaxrs20.server.LibertyJaxRsServerFactoryBean.performInvocation(LibertyJaxRsServerFactoryBean.java:641)
at com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.performInvocation(LibertyJaxRsInvoker.java:160)
at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:101)
at com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.invoke(LibertyJaxRsInvoker.java:273)
at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:213)
at com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.invoke(LibertyJaxRsInvoker.java:444)
at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:112)
at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:123)
at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:274)
at com.ibm.ws.jaxrs20.endpoint.AbstractJaxRsWebEndpoint.invoke(AbstractJaxRsWebEndpoint.java:137)
at com.ibm.websphere.jaxrs.server.IBMRestServlet.handleRequest(IBMRestServlet.java:146)
at com.ibm.websphere.jaxrs.server.IBMRestServlet.doDelete(IBMRestServlet.java:131)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:712)
at com.ibm.websphere.jaxrs.server.IBMRestServlet.service(IBMRestServlet.java:96)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1258)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:746)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:443)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:183)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:94)
at com.ibm.fhir.server.filter.rest.FHIRRestServletFilter.doFilter(FHIRRestServletFilter.java:155)
at javax.servlet.http.HttpFilter.doFilter(HttpFilter.java:127)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:197)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
at com.ibm.ws.security.jaspi.JaspiServletFilter.doFilter(JaspiServletFilter.java:56)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:197)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:1002)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1140)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1011)
at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:75)
at com.ibm.ws.webcontainer40.servlet.CacheServletWrapper40.handleRequest(CacheServletWrapper40.java:85)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:938)
at com.ibm.ws.webcontainer.osgi.DynamicVirtualHost$2.run(DynamicVirtualHost.java:279)
at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink$TaskWrapper.run(HttpDispatcherLink.java:1159)
at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.wrapHandlerAndExecute(HttpDispatcherLink.java:428)
at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.ready(HttpDispatcherLink.java:387)
at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:566)
at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleNewRequest(HttpInboundLink.java:500)
at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.processRequest(HttpInboundLink.java:360)
at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.ready(HttpInboundLink.java:327)
at com.ibm.ws.channel.ssl.internal.SSLConnectionLink.determineNextChannel(SSLConnectionLink.java:1100)
at com.ibm.ws.channel.ssl.internal.SSLConnectionLink$MyReadCompletedCallback.complete(SSLConnectionLink.java:675)
at com.ibm.ws.channel.ssl.internal.SSLReadServiceContext$SSLReadCompletedCallback.complete(SSLReadServiceContext.java:1824)
at com.ibm.ws.tcpchannel.internal.WorkQueueManager.requestComplete(WorkQueueManager.java:504)
at com.ibm.ws.tcpchannel.internal.WorkQueueManager.attemptIO(WorkQueueManager.java:574)
at com.ibm.ws.tcpchannel.internal.WorkQueueManager.workerRun(WorkQueueManager.java:958)
at com.ibm.ws.tcpchannel.internal.WorkQueueManager$Worker.run(WorkQueueManager.java:1047)
at com.ibm.ws.threading.internal.ExecutorServiceImpl$RunnableWrapper.run(ExecutorServiceImpl.java:238)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)

[9/21/21, 10:55:40:073 UTC] 00000042 FHIRResource 1
OperationOutcome:
{"resourceType":"OperationOutcome","id":"7f-0-0-1-10adbe1f-4351-46d8-9891-15275dfee94b","issue":[{"severity":"error","code":"not-found","diagnostics":"The Job is stopped intentionally with an in memory database."}]}

prb112 · 2021-09-21T14:32:00Z

That's right. Your job is not configured with a real operational datastore. There is a default in memory. You can configure using this dropin https://github.com/IBM/FHIR/blob/main/fhir-server/liberty-config/configDropins/disabled/postgres/bulkdata.xml

dabimahesh added the bug Something isn't working label Aug 24, 2021

prb112 added a commit that referenced this issue Aug 25, 2021

BulkData: Delete Job #2713

1e07a66

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

prb112 self-assigned this Aug 26, 2021

prb112 added this to the Sprint 2021-12 milestone Aug 26, 2021

prb112 added a commit that referenced this issue Sep 2, 2021

Merge pull request #2727 from IBM/issue-2713

0e0f5cd

BulkData: Delete Job #2713

lmsurpre closed this as completed Sep 8, 2021

prb112 mentioned this issue Dec 16, 2021

Delete of Bulk Export Job fails #3120

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BulkData: Delete Job #2713

BulkData: Delete Job #2713

dabimahesh commented Aug 24, 2021

prb112 commented Aug 24, 2021

dabimahesh commented Aug 25, 2021

dabimahesh commented Aug 25, 2021

prb112 commented Aug 25, 2021 •

edited

Loading

prb112 commented Aug 26, 2021

dabimahesh commented Aug 26, 2021 •

edited

Loading

lmsurpre commented Sep 7, 2021 •

edited

Loading

dabimahesh commented Sep 21, 2021

prb112 commented Sep 21, 2021

BulkData: Delete Job #2713

BulkData: Delete Job #2713

Comments

dabimahesh commented Aug 24, 2021

prb112 commented Aug 24, 2021

dabimahesh commented Aug 25, 2021

dabimahesh commented Aug 25, 2021

prb112 commented Aug 25, 2021 • edited Loading

prb112 commented Aug 26, 2021

dabimahesh commented Aug 26, 2021 • edited Loading

lmsurpre commented Sep 7, 2021 • edited Loading

dabimahesh commented Sep 21, 2021

prb112 commented Sep 21, 2021

prb112 commented Aug 25, 2021 •

edited

Loading

dabimahesh commented Aug 26, 2021 •

edited

Loading

lmsurpre commented Sep 7, 2021 •

edited

Loading