First steps to add encoding capability to outbound traffic #73

Merged
merged 37 commits into from Feb 28, 2014

Conversation

Projects
None yet
2 participants
@nicolas-rempulski
Member

nicolas-rempulski commented Jan 13, 2014

Need Review
Merge Candidate

Here are first steps of adding encoding to Wisdom as stated in #10

Done :

  • Add content encoding constants
  • Add content encoders in the content manager service (GZIP, DEFLATE, IDENTITY)
  • Add content encoder retrieval in content engine (with encoding)
  • Add Accepting_Encoding parsing (partially done) + encoding sorting based on order and qValue (lots of TODO in code)
  • Add content encoding to result processing
  • Add a configuration key to enable / disable encoding globally
  • Add annotations to enable / disable encoding for a route / controller
  • Add size limit to @AllowEncoding and global configuration (code is still rough)
  • Review Accepting_Encoding parsing as I'm not sure of some behaviors + "*;qValue" is not handled
  • Don't work on /assets/xxx routes, they have a length of -1 => Not RenderableFile but RenderableStream. Added a configuration key to allow encoding of ALL stream. Still disable with the @DenyEncoding annotation.
  • Documentation for annotations, default behaviour
  • Filter on mime types
  • Stream are not encoded (no size no mime type). RenderableUrl are encoded if encoding.url is set and mime type not in exclude list
  • Filter on already encoded files (Content_Encoding set and different from identitity)
  • Unit tests (done for encoders)
  • Documentation
  • Encoding is now asynchrone

ToDo :

  • Check encoding behavior with cache (Compressed each time or the result is cached ?)

Future Work :

To discuss :

I also made a change to RenderableString, as the Charset was not used to convert the string to bytes. Need reviews. (I think it is linked to #42)

nicolas-rempulski added some commits Jan 13, 2014

Force Charset specification for strings
String.getBytes() encode a string using the platform default encoding.

In RenderableString, the string was encoded without using the Charset specified or didn't specify the default Charset if no Charset was set.

Now, when using RenderableString, it will always specify the charset.
Add ContentEncoder interface
A ContentEncoder encodes an InputStream to a given compression format
Add content encoders for wisdom
- GZIP
- DEFLATE
- IDENTITY
Adding EncodingHelper
EncodingHelp allows to parse the Accept_Encoding header and sort the encoding methods according (mostly) to HTTP/1.1
RenderableString.render was not passing UT
Result parameter as null was not handled
Add @AllowEncoding and @DenyEncoding
These annotations allow to override the application configuration regarding content encoding.

They can be set both on class & methods
add size fields on @AllowEncoding annotations
* maxSize is the upper boundary for encoding
* minSize is the lower boundary for encoding
Adding size computation to shouldEncode
* if renderable.length is not in encoding boundaries, abort encoding

* shouldEncode now take a renderable as parameter, preparing for filter on extensions
Finalizing encoders to codecs refactoring
Changing package names to org.wisdom.content.codecs
Codecs refactoring
* Adding AbstractDefInfCodec for Gzip and Deflate

* Adding in file documentation
Refactoring shouldEncode
Moving shouldEncode from WisdomHandler to EncodingHelper
Refactoring shouldEncode and parseAcceptEncodingHeader
* shouldEncode is now in EncodingHelper

* parseAcceptEncodingHeader now correctly handle wildcard "*"
Add configuration values for encoding
* Add default value for encoding.global : true

* Add encoding.stream configuration key to allow stream encoding (no size check !)

* Add default value for encoding.stream : true
Refactor shouldEncode do handle stream
Should encode will not check stream size in encoding.stream is set to true in the configuration
Add functions to handle Long properties
* Adding get functions

* Update Configuration Unit Test
ContentEncodingHelper refactoring
* Added a ContentEncodingHelper interface
* Provide a ContentEncodingHelperImpl in content-manager as a service
* Retrieval through content-engine
* Updated UT (was throwing null pointer)
Add ContentEncodingHelperImpl UT
* Add UT and FakeControllers
* Add mockito dependency
* Modifications to ContentEncodingHelperImpl to satisfy tests
Update documentation to match latest changes
* Configuration keys
* URL encoding
* MimeType filtering
* Cosmetic changes
@nicolas-rempulski

This comment has been minimized.

Show comment
Hide comment
@nicolas-rempulski

nicolas-rempulski Jan 23, 2014

Member

This PR is now ready for review & merge.

Interaction between cache and encoding should be examined.

Member

nicolas-rempulski commented Jan 23, 2014

This PR is now ready for review & merge.

Interaction between cache and encoding should be examined.

nicolas-rempulski added some commits Jan 23, 2014

Refactor WisdomHandler to handle asynchronous content encoding
* add fromAsync parameter to write / finalizeWrite to know is their call is from an async result and need cleanup at the end
* Remove cleanup after AsyncResult handling as there could be an async content encoding running
* Extract content encoding from process result to handle it in writeResponse
* Add a proceedAsyncEncoding function to encode content InputStream asynchronously
* FinalizeWrite now check if the call is from an Async function and if keepAlive is not set, cleanup this handler
Merge branch 'refs/heads/master' into encoding-implem
Conflicts:
	wisdom-api/src/main/java/org/wisdom/api/bodies/RenderableString.java
	wisdom-api/src/main/java/org/wisdom/api/utils/KnownMimeTypes.java
	wisdom-engine/src/main/java/org/wisdom/engine/server/WisdomHandler.java
Merge branch 'refs/heads/master' into encoding-implem
Conflicts:
	application-configuration/src/main/java/org/wisdom/configuration/ApplicationConfigurationImpl.java
	application-configuration/src/test/java/org/wisdom/configuration/ApplicationConfigurationTest.java
	wisdom-api/src/main/java/org/wisdom/api/configuration/ApplicationConfiguration.java
@nicolas-rempulski

This comment has been minimized.

Show comment
Hide comment
Member

nicolas-rempulski commented Feb 27, 2014

Conflicts too high

cescoffier added a commit that referenced this pull request Feb 28, 2014

Merge pull request #73 from wisdom-framework/encoding-implem
First steps to add encoding capability to outbound traffic

@cescoffier cescoffier merged commit d28e7c0 into master Feb 28, 2014

@cescoffier cescoffier added this to the 0.4.0 milestone Apr 4, 2014

@cescoffier cescoffier deleted the encoding-implem branch Aug 25, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment