Add regex blob rename middleware

[rsrdesarrollo]

Introduction

This is probably a crazy idea, but I needed s3proxy to be able to rename the names of the blobs before they were written, so I have developed this middleware, and I am sharing it in case it might be useful to someone else.

Use case

In our case, we have a product that can only talk to S3, uploading blobs in a very limited way, with almost no configuration. Originally, blobs were uploaded to /prefix/<source>/<YYYYYY>/<MM>/<DD> (e.g. /prefix/typea/2023/01/31). But due to other integrations that are not relevant, we needed the blobs to be uploaded in a path with the following format:

/prefix/date=<YYYY>-<MM>-<DD>/type=<source>

Configuration of middleware

The regex is configured as:

s3proxy.regex-blobstore.match.<regex name> = <regex match expression>
s3proxy.regex-blobstore.replace.<regex name> = <regex replace expression>

You can use multiple regex, and they are evaluated in order. The first regex that matches breaks the evaluation.

For example, for the use case previously mentioned, we have the following rule:

s3proxy.regex-blobstore.match.type_partition=^prefix/(\\w+)/(\\d{4})/(\\d{2})/(\\d{2})/(.*)$
s3proxy.regex-blobstore.replace.type_partition=prefix/date=$2-$3-$4/type=$1/$5

I would love to have your feedback regarding this middleware, as well as possible things I might have overlooked when implementing it.

Thank you very much for your project.

[gaul]

Seems like a good idea! Before I review this, can you add some tests? Also add a blurb to the README and wiki explaining how to use this.

[rsrdesarrollo]

Yes, no problem there is the wiki page
https://github.com/gaul/s3proxy/wiki/Middleware-regex

I will update README and test ASAP

[rsrdesarrollo]

@gaul done

[gaul]

Thank you for your contribution @rsrdesarrollo!