Support multiple databases for regulatory compliance

[florianmonfort]

For regulatory reasons, we are not able to store all users' tracking data in the same database / same server.

We have one application/website serving different parts of the world, but different parts of the world require us to store data in specific locations.

We are able to figure out where users are or establish rules to determine where data should be stored, however we cannot seem to find a way in Matomo to set up multiple databases for one app.

Please note that for us this is a complete blocker to being able to use Matomo for now, until we have found a solution for this.

Summary

We need a way to be able to set up / specify different Matomo databases under one app, then have a way to "route" requests/data transfers to either of these databases.

Your Environment

This seems to affect any environment, we use the standard installation for now.

[tsteur]

Hi @florianmonfort
could you let us know exactly what these "regulatory reasons" are?

We need a way to be able to set up / specify different Matomo databases under one app, then have a way to "route" requests/data transfers to either of these databases.

I'm not really sure what you mean by that? Like what do you mean by "one app"? can you use different endpoints aka different Matomo installations and route the people to the right Matomo installation?

There's also a multi tenant setup but it's effectively similar to having multiple installations but one shared code base https://developer.matomo.org/guides/multi-tenants

[florianmonfort]

Hi @florianmonfort could you let us know exactly what these "regulatory reasons" are?

We need a way to be able to set up / specify different Matomo databases under one app, then have a way to "route" requests/data transfers to either of these databases.

I'm not really sure what you mean by that? Like what do you mean by "one app"? can you use different endpoints aka different Matomo installations and route the people to the right Matomo installation?

There's also a multi tenant setup but it's effectively similar to having multiple installations but one shared code base https://developer.matomo.org/guides/multi-tenants

Hello @tsteur ,

We have a platform available in markets such as Mainland China, Singapore, Hong Kong etc... where for example how long we can keep the record, in what format once deleted, "where" meaning on which of our servers and "for which users" this data can be stored etc. is all quite different.

So for us splitting the databases would be great because we can make sure to be in full compliance with this, while retaining only one single dashboard to "access" or "monitor" this data.

Hope this clarifies the use case?

[tsteur]

Thanks. I understand a bit better now.

The simplest workaround would be for now to install different Matomo's in each location. There will be different dashboards though which is not what you are after. If you were to access them using one dashboard you would still transfer the data to the different regions in the end and process them in a central region. Not sure how much this be actually allowed.

There could be few workarounds potentially to make logging into these dashboards easier. Like have one dashboard to log users into the different dashboards using https://matomo.org/faq/how-to/faq_30/

It won't be possible to view aggregate of all sites though.

There's also the possibility to embed widgets using https://matomo.org/docs/embed-matomo-reports/#embed-a-matomo-report-in-a-html-page but I think this might not help there.

[florianmonfort]

Thanks. I understand a bit better now.

The simplest workaround would be for now to install different Matomo's in each location. There will be different dashboards though which is not what you are after. If you were to access them using one dashboard you would still transfer the data to the different regions in the end and process them in a central region. Not sure how much this be actually allowed.

There could be few workarounds potentially to make logging into these dashboards easier. Like have one dashboard to log users into the different dashboards using https://matomo.org/faq/how-to/faq_30/

It won't be possible to view aggregate of all sites though.

There's also the possibility to embed widgets using https://matomo.org/docs/embed-matomo-reports/#embed-a-matomo-report-in-a-html-page but I think this might not help there.

Thank you for your advice, it does look like to be compliant we have to have multiple installs then.

Thanks for at least checking it out.

[jszczypk]

Hi @florianmonfort,

I am not MySQL expert, but you could check partitioning of MySQL tables holding users' data. - https://dev.mysql.com/doc/refman/8.0/en/partitioning.html
It may allow you to have one "virtual" table combining multiple partitions that may be stored in different files. By mounting remote directories (located on servers in specific parts of the world to fullfill regulatory requirements) on main server running Matomo, you will have central dashboard.
There are probably tons of other problems regarding Matomo's database structure in this awkward setup, but it might be something to look into anyway.

[florianmonfort]

Hi @florianmonfort,

I am not MySQL expert, but you could check partitioning of MySQL tables holding users' data. - https://dev.mysql.com/doc/refman/8.0/en/partitioning.html It may allow you to have one "virtual" table combining multiple partitions that may be stored in different files. By mounting remote directories (located on servers in specific parts of the world to fullfill regulatory requirements) on main server running Matomo, you will have central dashboard. There are probably tons of other problems regarding Matomo's database structure in this awkward setup, but it might be something to look into anyway.

Thanks, we will look into this solution and feedback here later