Include metadata on options for projections

[StevenBlair123]

Is your feature request related to a problem? Please describe.
When you create a new projection, it is currently not possible to set metadata for result stream.
One example when this could be very useful is streams where you want to set maxAge / maxCount for streams where holding the last few events is desirable.

An example we have in production is stock level values for Products.
We have 1000's of events per product, which are no longer needed after a certain time.
These streams are created via a projection, so we have no way of easily setting the maxCount / maxAge on each stream.

Describe the solution you'd like

When using options on a Projections, be able to include metadata fields for maxAge and maxCount like this:

options({
resultStreamName: "my_demo_projection_result",
metadata : "{ "maxAge": 2592000, "maxCount" : 2 }"
})

Describe alternatives you've considered
We can build some software to listen for streams being created, then set the metadata that way, or alternatively, write out events into a "bucket" stream, then just manually set maxAge.

Additional context
Add any other context or screenshots about the feature request here.

_DB-455

[StevenBlair123]

I actually have a proof of concept working for this, but a littler nervous on some of the changes for this.

Projections.js

I included a metadata property on options;

ProjectionNamesBuilder.cs

_sources.Metadata added

ProjectionSourceDefination.cs

		string IQuerySources.Metadata {
			get { return Options != null ? Options.Metadata : null; }
		}

and set Metadata when constructing QuerySourceOptions

ResultEventEmitter.cs

_resultStreamMetadata is no longer readonly

Inside CreateResultUpdatedEvents I am doing this:

string metadata = _namesBuilder.GetMetaData();

if (!String.IsNullOrEmpty(metadata)) {
	//TODO: verify whetehr this is called on each new event
	//What happens if the option is removed / updated

	//Parse
	var md = metadata.ParseJson<Metadata>(); //new record I created

	_resultStreamMetadata = new EmittedStream.WriterConfiguration.StreamMetadata(md.maxCount, null); //TODO maxAge
}

SourceDefinitionBuilder.cs

		public void SetMetadata(string metadata) {
			_options.Metadata = metadata;
		}

[StevenBlair123]

It seems to be working, and I have tried it on projections with partitions.
It's really about someone looking at the bigger picture of this change to make sure it makes sense.

If there are no major problems with this, I am happy enough to implement this.

[hayley-jean]

Thanks for the suggestion @StevenBlair123, we're discussing this internally

Just to clarify, this issue is only for setting metadata on result streams created by the projections, not for emitted stream. Is that correct?

[StevenBlair123]

@hayley-jean - yeah exactly.
The code change seems fairly straight forward, but just wanted someone else to verify it wont break anything.
We have it running here on a development version of the Eventstore and seems to be behaving itself.

[hayley-jean]

Thanks! We're discussing this feature internally and will let you know what we decide

[StevenBlair123]

Did you make any decision on this?
We could really use this feature just now, but if it's not going ahead, we can look for another permanent solution.

[hayley-jean]

Hi @StevenBlair123, we haven't made a decision on this yet. It's also worth noting that if we do decide to add it, it'll likely only be added to the interpreted runtime rather than the V8 one

[kodeine]

I think this is really important otherwise theres a huge gap missing. I was shocked to see this feature was missing as it is a common use case.

[alexeyzimarev]

I know it might sound a bit harsh, but in reality we haven't heard people complaining about it before. Apparently, it's useful, but it's not shocking that we don't have it, and it maybe is not as common use case as one might think. We will triage the issue shortly and post an update here.
Certainly, I agree that allowing to set the stream meta for new projected streams is very useful, especially for state streams. I think it wasn't called out before is that state projections aren't very common.