Api Policy Component

An HTTP service provides a facade over the reader endpoint for use by licenced partners.

• adds calculated fields for use by B2B partners
• blocks or hides content that is not permitted to the partner
• rewrites queries according to account configuration

This component is generally deployed with a proxy (Varnish) between it and the actual reader endpoints. Therefore, for clarity, the reader endpoint configuration options are called the proxy configuration options.

Interface

This facade deliberately does not define its own set of endpoints or interface contracts. Instead it makes specific modifications to the interface of the Reader API and has minimal knowledge of them.

Filters and Policies

Note that one policy might be used by many filters and filters might work with multiple policies.

Api filter	Description	Applied endpoints
identifiersFilter	Removes the identifiers field from the response unless the INCLUDE_IDENTIFIERS policy is present	/content, /content-preview, /internalcontent-preview, /enrichedcontent, /internalcontent
webUrlAdder	Adds the webUrl field to the response for specific content	/content, /content-preview, /internalcontent-preview, /enrichedcontent, /internalcontent
AddCanonicalWebUrl	Adds the canonicalWebUrl field to the response for specific content	/content, /content-preview, /internalcontent-preview, /enrichedcontent, /internalcontent
addSyndication	Adds the canBeSyndicated field to the response if not present	/content, /content-preview, /internalcontent-preview, /enrichedcontent, /internalcontent
linkValidationFilter	Adds validateLinkedResources=true to the request query if the INCLUDE_RICH_CONTENT policy is present	/content, /enrichedcontent, /internalcontent
suppressMarkup	Removes rich content related markup from the bodyXML and openingXML JSON fields from the response unless the INCLUDE_RICH_CONTENT policy is present	/content, /content-preview, /internalcontent-preview, /enrichedcontent, /internalcontent
mainImageFilter	Removes the mainImage field from the response unless the INCLUDE_RICH_CONTENT policy is present	/content, /content-preview, /internalcontent-preview, /enrichedcontent, /internalcontent
alternativeTitlesFilter	Removes the alternativeTitles field from the response unless the INTERNAL_UNSTABLE policy is present	/content, /content-preview, /internalcontent-preview, /enrichedcontent, /internalcontent
alternativeImagesFilter	Removes the alternativeImages field from the response unless the INTERNAL_UNSTABLE policy is present	/content, /content-preview, /internalcontent-preview, /enrichedcontent, /internalcontent
alternativeStandfirstsFilter	Removes the alternativeStandfirsts field from the response unless the INTERNAL_UNSTABLE policy is present	/content, /content-preview, /internalcontent-preview, /enrichedcontent, /internalcontent
removeCommentsFieldRegardlessOfPolicy	Removes the comments field from the response	/content
stripProvenance	Removes the publishReference and masterSource fields from the response unless the INCLUDE_PROVENANCE policy is present	/content, /content-preview, /internalcontent-preview, /enrichedcontent, /internalcontent, /lists
stripLastModifiedDate	Removes the lastModified field from the response unless the INCLUDE_LAST_MODIFIED_DATE policy is present	/content, /content-preview, /internalcontent-preview, /enrichedcontent, /internalcontent, /lists
stripLite	Removes the lite field from the response unless the INCLUDE_LITE policy is present	/internalcontent
stripOpeningXml	Removes the openingXML field from the response unless the INTERNAL_UNSTABLE policy is present	/content, /content-preview, /internalcontent-preview, /enrichedcontent, /internalcontent
removeAccessFieldRegardlessOfPolicy	Removes the accessLevel field from the response	/content, /content-preview, /internalcontent-preview
canBeDistributedAccessFilter	Returns HTTP 403 "Access denied" response for content without canBeDistributed=yes field unless the INTERNAL_UNSTABLE policy is present	/content, /enrichedcontent, /internalcontent
canBeSyndicatedAccessFilter	Returns HTTP 403 "Access denied" response for content without canBeSyndicated=yes field when the RESTRICT_NON_SYNDICATABLE_CONTENT policy is present	/content, /content-preview, /internalcontent-preview, /enrichedcontent, /internalcontent
unrolledContentFilter	Adds unrollContent=true to the request query if the INCLUDE_RICH_CONTENT and EXPAND_RICH_CONTENT policies are present	/content-preview, /internalcontent-preview, /enrichedcontent, /internalcontent
stripCommentsFields	Removes the comments field from the response unless the INCLUDE_COMMENTS policy is present	/content-preview, /internalcontent-preview, /enrichedcontent, /internalcontent
brandFilter	Adds forBrand=XXX to the request query if FASTFT_CONTENT_ONLY policy is present or adds notForBrand=XXX to the request query if EXCLUDE_FASTFT_CONTENT policy is present, where XXX is the brand id for FastFT	/content/notifications
contentNotificationsFilter	Adds type=all and monitor=true to the request query if the INTERNAL_UNSTABLE policy is present; adds type=article&type=liveblogpackage&type=liveblogpost and monitor=false to the request query if the APPEND_LIVE_BLOG_NOTIFICATIONS policy is present, otherwise adds type=article and monitor=false	/content/notifications
accessLevelPropertyFilter	Removes the accessLevel field from the response unless the INTERNAL_UNSTABLE policy is present	/enrichedcontent, /internalcontent
accessLevelHeaderFilter	Removes the X-FT-Access-Level header from the response unless the INTERNAL_UNSTABLE policy is present	/enrichedcontent, /internalcontent
contentPackageFilter	Removes the contains and containedIn fields from the response unless the INTERNAL_UNSTABLE or the INCLUDE_RELATIONS policy is present	/enrichedcontent, /internalcontent
editorialDeskFilter	Removes the editorialDesk field from the response unless the INTERNAL_ANALYTICS policy is present	/content, /enrichedcontent, /internalcontent
internalAnalyticsTagsFilter	Removes the internalAnalyticsTags field from the response unless the INTERNAL_ANALYTICS policy is present	/content, /enrichedcontent, /internalcontent

Policy	Description	Affected fields
INCLUDE_RICH_CONTENT	Allows rich content (images) related fields/content to be returned in response	mainImage, bodyXML, openingXML
INCLUDE_IDENTIFIERS	Allows including the identifiers field in the response	identifiers
INCLUDE_COMMENTS	Allows including the comments field in the response	comments
INCLUDE_PROVENANCE	Allows including information about the provenance of the content in the response	publishReference, masterSource
INCLUDE_LAST_MODIFIED_DATE	Allows including the lastModified field in the response	lastModified
INCLUDE_LITE	Allows including the lite field in the response	lite
INCLUDE_RELATIONS	Allows including the contains and containedIn fields in the response	contains, containedIn
FASTFT_CONTENT_ONLY	Includes events only for FastFT branded content into notification response	*
EXCLUDE_FASTFT_CONTENT	Excludes events for content with FastFT brand from notification response	*
INTERNAL_UNSTABLE	Allows including fields considered as "unstable" for internal usage	alternativeTitles, alternativeImages, alternativeStandfirsts, openingXML, accessLevel, contains, containedIn
INTERNAL_ANALYTICS	Allows fields for internal analytics usage	editorialDesk
EXPAND_RICH_CONTENT	If present along with INCLUDE_RICH_CONTENT it allows expanding rich content related fields in the response	mainImage, embeds, alternativeImages, promotionalImage, members, leadImages, image
RESTRICT_NON_SYNDICATABLE_CONTENT	If present non-syndicatable content will throw a 403 Forbidden HTTP error as a response	canBeSyndicated
APPEND_LIVE_BLOG_NOTIFICATIONS	Allows including live blog package & live blog post content in the notifications response	*
ADVANCED_NOTIFICATIONS	Allows distinguishing between CREATE and UPDATE notifications in the notifications-rw response	*

Header Handling

In general, headers are passed from the gateway through the facade to the Varnish layer. Varnish is expected to perform a similar forwarding of headers with the result that headers seen here are seen at the reader API.

The following headers are exceptions, they all related directly to the underlying TCP connection and the encoding of data over it. Since each leg of the request workflow is a separate connection to a new host, these header are not forwarded from one TCP connection to the next but will likely be regenerated by local libraries (e.g. Jersey Client, Jetty).

• "Host" - Seen in requests this names the host you intend to connect to.

• "Connection" - Seen in responses this signals to the client that the TCP connection will be kept alive or closed.

• "Accept-Encoding" - Seen in requests this signals that GZip is or is not supported by the client.

• "Content-Length" - Seen in responses. Gives the length of the entity. We strip this and let the platform regenerate it because it is subject to change.

• "Transfer-Encoding" the opposite number to Accept-Encoding.

• Date - We remove this and regenerate it as a hacky way to avoid having two in our response - Jetty was adding a second value. Since we are modifying responses it is not inaccurate to bump the date by a few milliseconds.

Running locally

To compile, run tests and build jar

mvn clean install

To run locally, run:

java -jar api-policy-component-service/target/api-policy-component-service-1.0-SNAPSHOT.jar server api-policy-component-service/config-local.yml

Building with docker:

docker build -t coco/api-policy-component:your-version .

Running as a docker container:

docker run --rm -p 8080 -p 8081 --env "JAVA_OPTS=-Xms384m -Xmx384m -XX:+UseG1GC -server" --env "READ_ENDPOINT=localhost:8080:8080" --env "JERSEY_TIMEOUT_DURATION=10000ms" coco/api-policy-component:your-version

Running all tests using Docker Compose

• Set the following environment variables (get the values from LastPass) so that Maven will be able to fetch all needed dependencies:

  export SONATYPE_USER="xxx"
  export SONATYPE_PASSWORD="xxx"
  

• Run the standard triplet of Docker/Compose commands:

  docker-compose -f docker-compose-tests.yml up -d --build && \
  docker logs -f test-runner && \
  docker-compose -f docker-compose-tests.yml down -v
  

Note: The docker-compose-tests.yml file is set to mount the standard directory used for the local Maven repository (~/.m2/repository) into the test-runner container, but if you have not used Maven before you can set it to any other directory on your system.