supporting multi-tenancy

[zxwing]

Is there any plan to support multi-tenancy in Prometheus? Making users in a public cloud only see monitoring data belonging to them. Frankly speaking, I don't know how a multi-tenancy monitoring system looks like or if there is anything existing. Now we retrieve all data users ask and do a post-process filtering out data users not supposed to see, it works but inefficient.

[fabxc]

Different companies are interested in that and there's some ongoing explorative work: https://docs.google.com/document/d/1C7yhMnb1x2sfeoe45f4mnnKConvroWhJ8KQZwIHJOuw/edit#

The current working approach seems to be to attach a user label to the metrics and pre-process incoming queries to always have a user="x" selection in each retrieved metric. Some companies are doing that but there's no open source version of it.
It shouldn't be too complex to re-implement though using the promql package in this repository.

[zxwing]

Good to see there is something on-going! However, labeling user="x" is not working in a sophisticated system where resources can be shared to other tenancies. This is why we do filtering instead of labeling metrics.

[fabxc]

I see that it can get more complex but cannot immediately see for which cases filtering the result provides something that cannot be implemented with a similar rule set on pre-processing queries.
Calculating results and throwing them away is always unideal.

[zxwing]

Let's say the user A has a service with metrics labeled with user='A'; after a while, he transfers the service to the user B, how should I re-label the service's metric from user='A' to user='B'?

[fabxc]

Ah, got it. I suppose you could check the query selections beforehand anyway. But filtering the result might be easier, especially as regex queries along the __name__ label get involved.

[zxwing]

I planned to check query selections originally but found it's not easy with Prometheus's query DSL.
The DSL is great for operating but not friendly for API integration. A DSL parser is required so I can know what labels users select. Another way is to create my APIs that wrap Promeheus's DSL, but then I have to write tons of documents to explain how my APIs map to Prometheus's DSL, and users will definitely get confused. So for now the filtering is an easier way.

[zxwing]

Is there any DSL spec for Prometheus?

[fabxc]

Not really. It was changing a fair bit in the past but now we could
actually have one.
But the promql package in the prometheus source tree provides a parser
that gives you an AST, which is easy to work with. It's used by other
organizations to solve similar problems.

On Thu, Jul 14, 2016 at 5:12 PM zxwing notifications@github.com wrote:

Is there any DSL spec for Prometheus?

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
#1813 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AEuA8j5su5MlXwxZnNAN8KFBgM7VMV55ks5qVe9xgaJpZM4JMHi2
.

[zxwing]

Got it! Thanks a lot!

[zxwing]

it's answered, close it

[yasongxu]

some project is open cortex