Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add custom partition interceptor examples #2145

Merged
merged 5 commits into from Oct 25, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Expand Up @@ -26,13 +26,17 @@
import ca.uhn.fhir.interceptor.model.RequestPartitionId;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import ca.uhn.fhir.rest.server.RestfulServer;
import ca.uhn.fhir.rest.server.exceptions.InvalidRequestException;
import ca.uhn.fhir.rest.server.servlet.ServletRequestDetails;
import ca.uhn.fhir.rest.server.tenant.UrlBaseTenantIdentificationStrategy;
import org.hl7.fhir.instance.model.api.IBaseResource;
import org.hl7.fhir.r4.model.Observation;
import org.hl7.fhir.r4.model.Patient;
import org.springframework.beans.factory.annotation.Autowired;

import javax.servlet.http.HttpServletRequest;
import java.util.Set;

@SuppressWarnings("InnerClassMayBeStatic")
public class PartitionExamples {

Expand Down Expand Up @@ -104,6 +108,43 @@ public RequestPartitionId PartitionIdentifyCreate(IBaseResource theResource) {
// END SNIPPET: partitionInterceptorResourceContents


// START SNIPPET: partitionInterceptorReadAllPartitions
@Interceptor
public class PartitionInterceptorReadAllPartitions {

@Hook(Pointcut.STORAGE_PARTITION_IDENTIFY_READ)
public RequestPartitionId readPartition() {
return RequestPartitionId.allPartitions();
}

}
// END SNIPPET: partitionInterceptorReadAllPartitions


// START SNIPPET: partitionInterceptorReadBasedOnScopes
@Interceptor
public class PartitionInterceptorReadPartitionsBasedOnScopes {

@Hook(Pointcut.STORAGE_PARTITION_IDENTIFY_READ)
public RequestPartitionId readPartition(ServletRequestDetails theRequest) {

HttpServletRequest servletRequest = theRequest.getServletRequest();
Set<String> approvedScopes = (Set<String>) servletRequest.getAttribute("ca.cdr.servletattribute.session.oidc.approved_scopes");

String partition = approvedScopes
.stream()
.filter(t->t.startsWith("partition-"))
.map(t->t.substring("partition-".length()))
.findFirst()
.orElseThrow(()->new InvalidRequestException("No partition scopes found in request"));
return RequestPartitionId.fromPartitionName(partition);

}

}
// END SNIPPET: partitionInterceptorReadBasedOnScopes


// START SNIPPET: multitenantServer
public class MultitenantServer extends RestfulServer {

Expand Down
@@ -0,0 +1,6 @@
---
type: change
issue: 2145
title: "In HAPI FHIR 5.1.0 JPA Server, if partitioning was enabled, but no interceptor was registered to the
`STORAGE_PARTITION_IDENTIFY_READ` pointcut, the system would simply default to 'App Partitions'. This was not the
intended behaviour, so this will now result in an error."
@@ -0,0 +1,49 @@
# Partition Interceptor Examples

This page shows examples of partition interceptors.

# Example: Partitioning based on Tenant ID

The [RequestTenantPartitionInterceptor](/docs/interceptors/built_in_server_interceptors.html#request-tenant-partition-interceptor) uses the request tenant ID to determine the partition name. A simplified version of its source is shown below:

```java
{{snippet:classpath:/ca/uhn/hapi/fhir/docs/PartitionExamples.java|partitionInterceptorRequestPartition}}
```

# Example: Partitioning based on headers

If requests are coming from a trusted system, that system might be relied on to determine the partition for reads and writes.

The following example shows a simple partition interceptor that determines the partition name by looking at a custom HTTP header:

```java
{{snippet:classpath:/ca/uhn/hapi/fhir/docs/PartitionExamples.java|partitionInterceptorHeaders}}
```

# Example: Using Resource Contents

When creating resources, the contents of the resource can also be factored into the decision on which tenant to use. The following example shows a very simple algorithm, placing resources into one of three partitions based on the resource type. Other contents in the resource could also be used instead.

```java
{{snippet:classpath:/ca/uhn/hapi/fhir/docs/PartitionExamples.java|partitionInterceptorResourceContents}}
```

# Example: Always Read All Partitions

This is an example of a simple interceptor that causes read/search/history requests to always use all partitions. This would be useful if partitioning is being used for use cases that do not involve data segregation for end users.

This interceptor only provides the [`STORAGE_PARTITION_IDENTIFY_READ`](/hapi-fhir/apidocs/hapi-fhir-base/ca/uhn/fhir/interceptor/api/Pointcut.html#STORAGE_PARTITION_IDENTIFY_READ) pointcut, so a separate interceptor would have to be added to provide the [`STORAGE_PARTITION_IDENTIFY_WRITE`](/hapi-fhir/apidocs/hapi-fhir-base/ca/uhn/fhir/interceptor/api/Pointcut.html#STORAGE_PARTITION_IDENTIFY_WRITE) pointcut in order to be able to write data to the server.

```java
{{snippet:classpath:/ca/uhn/hapi/fhir/docs/PartitionExamples.java|partitionInterceptorReadAllPartitions}}
```

# Example: Smile CDR SMART Scopes

When deploying a partitioned server in Smile CDR using SMART on FHIR security, it may be desirable to use OAuth2 scope approval as a mechanism for determining which partitions a user should have access to.

This interceptor looks for approved scopes named `partition-ABC` where "ABC" represents the tenant name that the user should have access to.

```java
{{snippet:classpath:/ca/uhn/hapi/fhir/docs/PartitionExamples.java|partitionInterceptorReadBasedOnScopes}}
```
Expand Up @@ -67,32 +67,9 @@ The criteria for determining the partition will depend on your use case. For exa

A hook against the [`Pointcut.STORAGE_PARTITION_IDENTIFY_READ`](/hapi-fhir/apidocs/hapi-fhir-base/ca/uhn/fhir/interceptor/api/Pointcut.html#STORAGE_PARTITION_IDENTIFY_READ) pointcut must be registered, and this hook method will be invoked every time a resource is created in order to determine the partition to assign the resource to.

## Example: Partitioning based on Tenant ID

The [RequestTenantPartitionInterceptor](/docs/interceptors/built_in_server_interceptors.html#request-tenant-partition-interceptor) uses the request tenant ID to determine the partition name. A simplified version of its source is shown below:

```java
{{snippet:classpath:/ca/uhn/hapi/fhir/docs/PartitionExamples.java|partitionInterceptorRequestPartition}}
```

## Example: Partitioning based on headers

If requests are coming from a trusted system, that system might be relied on to determine the partition for reads and writes.

The following example shows a simple partition interceptor that determines the partition name by looking at a custom HTTP header:

```java
{{snippet:classpath:/ca/uhn/hapi/fhir/docs/PartitionExamples.java|partitionInterceptorHeaders}}
```

## Example: Using Resource Contents

When creating resources, the contents of the resource can also be factored into the decision on which tenant to use. The following example shows a very simple algorithm, placing resources into one of three partitions based on the resource type. Other contents in the resource could also be used instead.

```java
{{snippet:classpath:/ca/uhn/hapi/fhir/docs/PartitionExamples.java|partitionInterceptorResourceContents}}
```
## Examples

See [Partition Interceptor Examples](./partition_interceptor_examples.html) for various samples of how partitioning interceptors can be set up.

# Complete Example: Using Request Tenants

Expand Down
Expand Up @@ -28,18 +28,17 @@
import ca.uhn.fhir.jpa.entity.PartitionEntity;
import ca.uhn.fhir.jpa.model.config.PartitionSettings;
import ca.uhn.fhir.rest.api.server.RequestDetails;
import ca.uhn.fhir.rest.server.exceptions.InternalErrorException;
import ca.uhn.fhir.rest.server.exceptions.InvalidRequestException;
import ca.uhn.fhir.rest.server.exceptions.ResourceNotFoundException;
import ca.uhn.fhir.rest.server.exceptions.UnprocessableEntityException;
import ca.uhn.fhir.rest.server.servlet.ServletRequestDetails;
import org.apache.commons.lang3.Validate;
import org.hl7.fhir.instance.model.api.IBaseResource;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.transaction.support.TransactionSynchronizationManager;

import javax.annotation.Nonnull;
import javax.annotation.Nullable;
import javax.transaction.Transactional;
import java.util.HashSet;

import static ca.uhn.fhir.jpa.util.JpaInterceptorBroadcaster.doCallHooks;
Expand Down Expand Up @@ -78,7 +77,7 @@ public RequestPartitionHelperSvc() {

/**
* Invoke the {@link Pointcut#STORAGE_PARTITION_IDENTIFY_READ} interceptor pointcut to determine the tenant for a read request.
*
* <p>
* If no interceptors are registered with a hook for {@link Pointcut#STORAGE_PARTITION_IDENTIFY_READ}, return
* {@link RequestPartitionId#allPartitions()} instead.
*/
Expand All @@ -100,7 +99,7 @@ public RequestPartitionId determineReadPartitionForRequest(@Nullable RequestDeta
.addIfMatchesType(ServletRequestDetails.class, theRequest);
requestPartitionId = (RequestPartitionId) doCallHooksAndReturnObject(myInterceptorBroadcaster, theRequest, Pointcut.STORAGE_PARTITION_IDENTIFY_READ, params);
} else {
requestPartitionId = RequestPartitionId.allPartitions();
requestPartitionId = null;
}

validatePartition(requestPartitionId, theResourceType, Pointcut.STORAGE_PARTITION_IDENTIFY_READ);
Expand Down Expand Up @@ -189,8 +188,10 @@ private RequestPartitionId normalizeAndNotifyHooks(@Nonnull RequestPartitionId t

}

private void validatePartition(@Nonnull RequestPartitionId theRequestPartitionId, @Nonnull String theResourceName, Pointcut thePointcut) {
Validate.notNull(theRequestPartitionId, "Interceptor did not provide a value for pointcut: %s", thePointcut);
private void validatePartition(RequestPartitionId theRequestPartitionId, @Nonnull String theResourceName, Pointcut thePointcut) {
if (theRequestPartitionId == null) {
throw new InternalErrorException("No interceptor provided a value for pointcut: " + thePointcut);
}

if (theRequestPartitionId.getPartitionId() != null) {

Expand Down
Expand Up @@ -44,6 +44,7 @@
import org.hl7.fhir.r4.model.Meta;
import org.hl7.fhir.r4.model.UriType;

import javax.annotation.Nonnull;
import javax.servlet.http.HttpServletRequest;
import java.util.Collection;
import java.util.Collections;
Expand Down Expand Up @@ -75,7 +76,7 @@ public JpaConformanceProviderR4() {
/**
* Constructor
*/
public JpaConformanceProviderR4(RestfulServer theRestfulServer, IFhirSystemDao<Bundle, Meta> theSystemDao, DaoConfig theDaoConfig, ISearchParamRegistry theSearchParamRegistry) {
public JpaConformanceProviderR4(@Nonnull RestfulServer theRestfulServer, @Nonnull IFhirSystemDao<Bundle, Meta> theSystemDao, @Nonnull DaoConfig theDaoConfig, @Nonnull ISearchParamRegistry theSearchParamRegistry) {
super(theRestfulServer);

Validate.notNull(theRestfulServer);
Expand Down
Expand Up @@ -83,9 +83,9 @@
import static org.mockito.Mockito.when;

@SuppressWarnings("unchecked")
public class PartitioningR4Test extends BaseJpaR4SystemTest {
public class PartitioningSqlR4Test extends BaseJpaR4SystemTest {

private static final org.slf4j.Logger ourLog = org.slf4j.LoggerFactory.getLogger(PartitioningR4Test.class);
private static final org.slf4j.Logger ourLog = org.slf4j.LoggerFactory.getLogger(PartitioningSqlR4Test.class);

private MyReadWriteInterceptor myPartitionInterceptor;
private LocalDate myPartitionDate;
Expand Down Expand Up @@ -879,44 +879,6 @@ public void testRead_PidId_SpecificPartition() {
}
}

@Test
public void testRead_PidId_AllPartitionsBecauseNoReadInterceptor() {
IIdType patientId1 = createPatient(withPartition(1), withActiveTrue());
IIdType patientId2 = createPatient(withPartition(2), withActiveTrue());

myInterceptorRegistry.unregisterInterceptor(myPartitionInterceptor);
MyWriteInterceptor writeInterceptor = new MyWriteInterceptor();
myInterceptorRegistry.registerInterceptor(writeInterceptor);
try {
{
myCaptureQueriesListener.clear();
IdType gotId1 = myPatientDao.read(patientId1, mySrd).getIdElement().toUnqualifiedVersionless();
assertEquals(patientId1, gotId1);

String searchSql = myCaptureQueriesListener.getSelectQueriesForCurrentThread().get(0).getSql(true, true);
ourLog.info("Search SQL:\n{}", searchSql);

// Only the read columns should be used, no criteria use partition
assertEquals(2, StringUtils.countMatches(searchSql, "PARTITION_ID as "));
assertEquals(2, StringUtils.countMatches(searchSql, "PARTITION_ID"));
}
{
IdType gotId2 = myPatientDao.read(patientId2, mySrd).getIdElement().toUnqualifiedVersionless();
assertEquals(patientId2, gotId2);

String searchSql = myCaptureQueriesListener.getSelectQueriesForCurrentThread().get(0).getSql(true, true);
ourLog.info("Search SQL:\n{}", searchSql);

// Only the read columns should be used, no criteria use partition
assertEquals(2, StringUtils.countMatches(searchSql, "PARTITION_ID as "));
assertEquals(2, StringUtils.countMatches(searchSql, "PARTITION_ID"));
}
} finally {
myInterceptorRegistry.unregisterInterceptor(writeInterceptor);
}
}


@Test
public void testRead_PidId_DefaultPartition() {
IIdType patientIdNull = createPatient(withPartition(null), withActiveTrue());
Expand Down