chore(storage-image-processing-api): initial extension build

[Salakar]

Introducing a Storage image processing extension which provides an extensive image processing API with 30+ image mutation operations from resizing and colour manipulation to overlaying text and compositing images.

The core image processing logic in the implementation is portable and therefore can be used in a variety of ways, e.g.,

• reacting to a storage event to automatically process an image (operations could be defined via a metadata key)
• adding image operations to Firestore and listening to a Firestore collection that would trigger these operations to be applied and output
• etc

but for this implementation, initially, interfacing with the image processing is provided as a URL safe HTTP GET API via a Cloud Function (and could be used for embedding e.g., <img src="<extensionFunctionUrl/process/...ops />).

Accessing GCS bucket objects is done via tokenated download urls, e.g., getDownloadUrl() on Firebase SDKs or providing a path to a public object on the bucket, e.g., /foo/bar.png. URLs & paths must be URL encoded.

Process Image URL Schema

The URL schema to apply operations is built into url parts separated by /, each segment is then in the following format;

/<operationName>~<optionName>:<optionValue>/

• A single operation and it's options are defined per url segment. Multiple options are separated by ~.
• Option values are automatically coerced into their appropriate types, e.g. '5' as a string would become 5 as a number, or '[1,2,3,4]' as a string would result in a JS array of numbers.
• Truthy boolean options can just specify the <optionName> only to have the value default to true (think flags).
• Operations that have no options or only have optional options can just specify the operation without options, e.g./negate/.
• An input operation must be the first operation specified and is required.
• An output operation must be the last operation specified and is required.

Having the URL schema as url segments vs query parameters allows us to easily have operation ordering and support using multiple operations of the same type in a chain of operations. Additionally makes options clearer that they are for a specific operation.

Example

URL:

/process/input~type:gcs~source:%2Fassets%2Fhexagons.jpeg/resize~width:500~height:500~fit:cover~position:left%20bottom/extend~all:20~background:tomato/extend~all:20~background:lightgreen/composite~top:20~left:26~tile~blend:screen~input:%2Fassets%2Finvertase.png/text~strokeWidth:4~top:75~left:75~font:95px%20tahoma~strokeColor:SeaGreen~textAlign:center~value:Hello%0DWorld~gravity:northwest~backgroundColor:pink~textColor:MediumSpringGreen~padding:15~borderWidth:5~borderColor:black/output~format:png

Output:

Supported operations

• input
• output
• affine
• blur
• clahe
• composite
  • experimental, needs some TLC around retrieving input sources
• extend
• extract
  • a.k.a crop
• flatten
  • e.g., can be used to remove transparency
• flip
• flop
• gamma
• median
• negate
• normalize
• resize
• rotate
• sharpen
• text
  • custom implementation since Sharp doesn't have a direct api for this - uses the composition API to overlay a generated SVG. This same kind of logic with overlaying SVGs could be used to do things like gradients & shapes
• trim
• convolve
• threshold
• boolean
• linear
• recomb
• modulate
• tint
• grayscale
• colorspace

Operations pending implementation

Not blocked or difficult by any means, just needs time invested to implement. These will be added in a future PR.

• removeAlpha
• ensureAlpha
• extractChannel
• joinChannel
• bandbool

---

TODO

• Finish up composite - currently sourcing from local files for testing purposes, should source from same sources that the input operation supports.
• Add remaining output operation format specific options, e.g., https://github.com/invertase/firebase-extensions/blob/storage-image-processing-api/extensions/storage-image-processing-api/functions/src/operations/output.ts#L32
• Add all available operation types (Currently Prs)
• Tests
• Documentation
• Input from GCS bucket path
• Investigate possible alternatives to output to a bucket rather than returning to the requesting client (maybe a POST api)

[Salakar]

Example of me mutating our Melos logo;

Original:

Mutated:

/process/input~type:url~url:https%3A%2F%2Fraw.githubusercontent.com%2Finvertase%2Fmelos%2Fmaster%2Fdocs%2Fassets%2Flogo-dark.png/flip/flop/median~size:55/extend~right:300~top:100~left:100/text~value:elos~top:350~left:475~font:110px%20arial/output~format:jpeg