How to configure external DICOM Archive for long term storage

vrindanayak edited this page Oct 18, 2016 · 6 revisions
Clone this wiki locally

Overview

This feature enables configuration of an external DICOM archive for long term storage of studies. The feature can be divided in two parts :

  1. Forward the studies to external archive, do Storage Commit and constrain the deletion of studies on original archive (whose objects are retrievable from external archive) if usable space fell below configurable threshold.

    forward-stgcmt.puml

  2. Forward the C-MOVE requests to external archive (for no Study found with Study Instance UID on DCM4CHEE)

    retrieve-cache.puml

Configuration

For the first part of the feature, you may

  • create a LDIF file (e.g.):

    version: 1
    dn: cn=Forward to STORESCP,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    objectClass: dcmExportRule
    cn: Forward to STORESCP
    dcmEntity: Series
    dcmExporterID: STORESCP
    dcmDuration: PT1M
    dcmProperty: SendingApplicationEntityTitle!=DCMQRSCP
    -
    
    dn: dcmExporterID=STORESCP,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    objectClass: dcmExporter
    dcmExporterID: STORESCP
    dcmQueueName: Export1
    dcmURI: dicom:DCMQRSCP
    dcmStgCmtSCP: DCMQRSCP
    dicomAETitle: DCM4CHEE
    dicomDescription: Export to STORESCP
    -
    
    dn: dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    changetype: modify
    add: dcmPurgeStorageFetchSize
    dcmPurgeStorageFetchSize: 100
    -
    
    dn: dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    changetype: modify
    add: dcmDeleteStudyBatchSize
    dcmDeleteStudyBatchSize: 10
    -
    
    dn: dcmStorageID=fs1,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    objectClass: dcmStorage
    dcmStorageID: fs1
    dcmURI: file:///Z:/
    dcmDeleterThreshold: 400MB
    dcmDigestAlgorithm: MD5
    dcmExternalRetrieveAET: DCMQRSCP
    dcmInstanceAvailability: ONLINE
    dcmProperty: pathFormat={now,date,yyyy/MM/dd}/{0020000D,hash}/{0020000E,hash}/{00080018,hash}
    dcmProperty: checkMountFile=NO_MOUNT
    -
    

    and import it to the LDAP Server by using the ldapmodify command line utility.

For the second part of the feature, you may

  • create a LDIF file (e.g.):

    version: 1
    dn: dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    changetype: modify
    add: dcmExternalRetrieveAEDestination
    dcmExternalRetrieveAEDestination: DCM4CHEE
    -
    

    and import it to the LDAP Server by using the ldapmodify command line utility.

One can refer to Exporter Descriptor, Storage Descriptor, Export Rule and Archive Device to understand the description of attributes.

Refer ISO-8601 Duration format for more know-how on Durations.

Testing

For the first part of the feature :

  1. Export the studies and do storage commit. For testing purpose we use the dcmqrscp tool which supports the storage commitment.

    a. Start the dcmqrscp tool

      dcmqrscp -b DCMQRSCP:11113 --dicomdir /tmp/DICOMDIR
    

    Log snippet :

      13:03:15,356 INFO  - Start TCP Listener on 0.0.0.0/0.0.0.0:11113
    

    b. Store the objects of studies to DCM4CHEE archive

      storescu -c DCM4CHEE@localhost:11112 /path_to_study.dcm
    

    Log snippet :

      13:04:05,124 INFO  - STORESCU->DCM4CHEE(1) >> 1:C-STORE-RSP[pcid=7, status=0H
       cuid=1.2.840.10008.5.1.4.1.1.1 - Computed Radiography Image Storage
       iuid=1.3.51.0.7.1265535119.17104.33870.32953.8838.26602.20894 - ?
       tsuid=1.2.840.10008.1.2.4.80 - JPEG-LS Lossless Image Compression
    

    c. After the duration configured in export rule, you can see in the dcmqrscp tool window that the objects exported have been stored and also the storage commitment has been done. Log snippet :

      13:05:08,719 INFO  - DCMQRSCP<-DCM4CHEE(1) << 1:C-STORE-RSP[pcid=5, status=0H
        cuid=1.2.840.10008.5.1.4.1.1.1 - Computed Radiography Image Storage
        iuid=1.3.51.0.7.1265535119.17104.33870.32953.8838.26602.20894 - ?
        tsuid=1.2.840.10008.1.2.4.80 - JPEG-LS Lossless Image Compression  
      .....
      13:05:09,398 INFO  - DCMQRSCP<-DCM4CHEE(2) << 1:N-ACTION-RSP[pcid=1, status=0H
        cuid=1.2.840.10008.1.20.1 - Storage Commitment Push Model SOP Class
        iuid=1.2.840.10008.1.20.1.1 - Storage Commitment Push Model SOP Instance
        tsuid=1.2.840.10008.1.2 - Implicit VR Little Endian
      ......
      13:05:09,772 INFO  - DCMQRSCP->DCM4CHEE(3) >> 1:N-EVENT-REPORT-RSP[pcid=1, status=0H
        cuid=1.2.840.10008.1.20.1 - Storage Commitment Push Model SOP Class
        iuid=1.2.840.10008.1.20.1.1 - Storage Commitment Push Model SOP Instance
        tsuid=1.2.840.10008.1.2 - Implicit VR Little Endian
    
  2. Objects of least recently accessed studies are deleted from storage system as usable space falls below threshold. One can see this in the server logs, once the value configured in dcmPurgeStoragePollingInterval has been reached. Log snippet :

    14:05:26,048 INFO  [org.dcm4chee.arc.delete.impl.PurgeStorageScheduler] (EE-ManagedScheduledExecutorService-default-Thread-3) Usable Space on file:///Z:/ below 400MB - start deleting 40.28928MB
    14:05:26,136 INFO  [org.dcm4chee.arc.delete.impl.PurgeStorageScheduler] (EE-ManagedScheduledExecutorService-default-Thread-3) Successfully delete objects of Study[pk=1918, uid=1.2.4.0.13.1.432252867.1552647.1, id=1552647.1] on file:///Z:/
    ....
    

For the second part of the feature :

  1. Trigger C-Move request to store the studies (whose objects have been deleted on DCM4CHEE archive) to destination.

    a. Start the storescp tool which acts as a destination

       storescp -b STORESCP:11115
    

    Log snippet :

       14:10:37,970 INFO  - Start TCP Listener on 0.0.0.0/0.0.0.0:11115
    

    b. Trigger C-Move for the study which is no longer available on DCM4CHEE archive

       movescu -c DCM4CHEE@localhost:11112 -m StudyInstanceUID=1.2.4.0.13.1.432252867.1552647.1 --dest STORESCP
    

    Log snippet :

       14:13:01,695 INFO  - MOVESCU->DCM4CHEE(1) >> 1:C-MOVE-RSP[pcid=1, completed=1, failed=0, warning=0, status=0H
         cuid=1.2.840.10008.5.1.4.1.2.2.2 - Study Root Query/Retrieve Information Model - MOVE
         tsuid=1.2.840.10008.1.2 - Implicit VR Little Endian
       14:13:01,697 DEBUG - Command:
       (0000,0002) UI [1.2.840.10008.5.1.4.1.2.2.2] AffectedSOPClassUID
       (0000,0100) US [32801] CommandField
       (0000,0120) US [1] MessageIDBeingRespondedTo
       (0000,0800) US [257] CommandDataSetType
       (0000,0900) US [0] Status
       (0000,1021) US [1] NumberOfCompletedSuboperations
       (0000,1022) US [0] NumberOfFailedSuboperations
       (0000,1023) US [0] NumberOfWarningSuboperations
    

    c. One can simultaneously also see in the storescp and dcmqrscp tool windows

    Storescp tool window Log snippet :

       14:13:01,577 INFO  - STORESCP<-DCM4CHEE(1): M-WRITE .\1.3.51.0.7.1265535119.17104.33870.32953.8838.26602.20894.part
       14:13:01,656 INFO  - STORESCP<-DCM4CHEE(1): M-RENAME .\1.3.51.0.7.1265535119.17104.33870.32953.8838.26602.20894.part to .\1.3.51.0.7.1265535119.17104.33870.32953.8838.26602.20894
       14:13:01,660 INFO  - STORESCP<-DCM4CHEE(1) << 1:C-STORE-RSP[pcid=1, status=0H
         cuid=1.2.840.10008.5.1.4.1.1.1 - Computed Radiography Image Storage
         iuid=1.3.51.0.7.1265535119.17104.33870.32953.8838.26602.20894 - ?
         tsuid=1.2.840.10008.1.2.4.80 - JPEG-LS Lossless Image Compression
       14:13:01,662 DEBUG - Command:
       (0000,0002) UI [1.2.840.10008.5.1.4.1.1.1] AffectedSOPClassUID
       (0000,0100) US [32769] CommandField
       (0000,0120) US [1] MessageIDBeingRespondedTo
       (0000,0800) US [257] CommandDataSetType
       (0000,0900) US [0] Status
       (0000,1000) UI [1.3.51.0.7.1265535119.17104.33870.32953.8838.26602.20894] Affe
    

    Dcmqrscp tool window Log snippet :

       14:13:01,377 INFO  - DCMQRSCP->DCM4CHEE(425) >> 1:C-STORE-RSP[pcid=1, status=0H
         cuid=1.2.840.10008.5.1.4.1.1.1 - Computed Radiography Image Storage
         iuid=1.3.51.0.7.1265535119.17104.33870.32953.8838.26602.20894 - ?
         tsuid=1.2.840.10008.1.2.4.80 - JPEG-LS Lossless Image Compression
       14:13:01,378 DEBUG - Command:
       (0000,0002) UI [1.2.840.10008.5.1.4.1.1.1] AffectedSOPClassUID
       (0000,0100) US [32769] CommandField
       (0000,0120) US [1] MessageIDBeingRespondedTo
       (0000,0800) US [257] CommandDataSetType
       (0000,0900) US [0] Status
       (0000,1000) UI [1.3.51.0.7.1265535119.17104.33870.32953.8838.26602.20894] Affe
    
       14:13:01,380 INFO  - DCMQRSCP->DCM4CHEE(425) << A-RELEASE-RQ
       14:13:01,381 DEBUG - DCMQRSCP->DCM4CHEE(425): enter state: Sta7 - Awaiting A-RELEASE-RP PDU
       14:13:01,382 INFO  - DCMQRSCP<-MOVESCU(424) << 1:C-MOVE-RSP[pcid=1, completed=1, failed=0, warning=0, status=0H
         cuid=1.2.840.10008.5.1.4.1.2.2.2 - Study Root Query/Retrieve Information Model - MOVE
         tsuid=1.2.840.10008.1.2 - Implicit VR Little Endian
       14:13:01,384 INFO  - DCMQRSCP->DCM4CHEE(425) >> A-RELEASE-RP
       14:13:01,384 DEBUG - Command:
       (0000,0002) UI [1.2.840.10008.5.1.4.1.2.2.2] AffectedSOPClassUID
       (0000,0100) US [32801] CommandField
       (0000,0120) US [1] MessageIDBeingRespondedTo
       (0000,0800) US [257] CommandDataSetType
       (0000,0900) US [0] Status
       (0000,1021) US [1] NumberOfCompletedSuboperations
       (0000,1022) US [0] NumberOfFailedSuboperations
       (0000,1023) US [0] NumberOfWarningSuboperations