M1 compatibility

[kevindew]

This PR tries to pull together all the fixes we know are needed for using GOV.UK docker on an M1 mac using information from: #561

This is marked as a draft as I'm not sure if we'll merge these things in directly, I expect we'd probably do some more testing and then consider adding some explanatory comments to code.

I've done some testing now and other M1 mac users are making use of this branch. I've also confirmed it allows database restores for MySQL, MongoDB and Elasticsearch.

[Tetrino]

For anyone who uses this branch, you'll need to make sure you use Docker's debug feature to purge all the images before you attempt using MySQL. Without doing so, you will have odd behaviour in regards to Whitehall connectivity and replicating data.

[DilwoarH]

Just tested Whitehall with this, ran fine 👍

[KludgeKML]

So, in short (because this is something I've been on a deep dive into recently): Router currently uses a feature called OpTime to find out whether it has to refresh its internal representation of the routing tree. The way Router is written it requires a replicaset (the optime calls basically open up the replicaset, dig down into it to find the primary, then use that as the check). The replicaset can be only 1 service (the primary), but it has to be there - router doesn't have any fallback. So if you're using router in a way on your local machine that needs you never to update the routes, it would work (although it would be chucking out warning messages about not finding the replica primary). If you wanted to update routes, or you wanted to run the router test suite, it would fail without the replicaset. At the moment I'm working on moving router in production to a DocumentDB database, which will require me to get rid of the replicaset requirement. But it's not quite done yet (and has just had a week+ delay due to me getting covid).

…

On Tue, 5 Apr 2022 at 13:48, Dilwoar Hussain ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In docker-compose.yml <#581 (comment)>: > @@ -39,6 +39,7 @@ services: ports: - "27017:27017" - "28017:28017" + command: ["--replSet", "mongo-replica-set"] Why do we need it to replicate locally? — Reply to this email directly, view it on GitHub <#581 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABAHVSM2LANM32O2YD5NBY3VDQY77ANCNFSM5Q7MU2BA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[kevindew]

So, in short (because this is something I've been on a deep dive into recently): Router currently uses a feature called OpTime to find out whether it has to refresh its internal representation of the routing tree. The way Router is written it requires a replicaset (the optime calls basically open up the replicaset, dig down into it to find the primary, then use that as the check). The replicaset can be only 1 service (the primary), but it has to be there - router doesn't have any fallback. So if you're using router in a way on your local machine that needs you never to update the routes, it would work (although it would be chucking out warning messages about not finding the replica primary). If you wanted to update routes, or you wanted to run the router test suite, it would fail without the replicaset. At the moment I'm working on moving router in production to a DocumentDB database, which will require me to get rid of the replicaset requirement. But it's not quite done yet (and has just had a week+ delay due to me getting covid).

Do hope you're feeling better now Keith

Hmm I see, so for that aspect of router to work it needs to be running as a replica set. Whereas for all other apps they don't care if replicaset or not?

From what I can tell just starting mongo with --replSet <replica-set-name> doesn't seem to be enough anymore. You need to also run rs.initiate() in MongoDB directly to start the replicaset (I imagine there's perhaps other ways to do this, such as config files) - so it seems quite the pain.

Do you think we need to wait until(/if) we resolve this before we merge this PR for M1 compatibility?

I feel like, aside from this, all the other bits work.

[KludgeKML]

Short story: I think you should merge this for the moment and we keep a separate branch for working on router until I can update router enough that that's unnecessary.

Long story: I would say that this affects:

• people actively working on router (AFAIK only me + occasionally some people doing dependabot version bumps)
• people who want to test things locally that require route creation going all the way through to router

People in group 1 won't be able to run the test suite locally. But it'll still run on Jenkins, and it can be made to run locally with the small change to their local govuk-docker. So I think they will live with that.

People in group 2 won't see routes get updated locally. But - that might not be a thing people care too much about in local dev? I'm also not 100% sure whether this is true, either - is the endpoint on router that triggers an update completely unused now? If some things still use it, maybe that is fine. You have a better idea of how frequent this need comes up, but I guess maybe not a lot?

Either way, the impact of merging this as is seems minimal. It might only ever affect me, and I already know how to work around it. So I'm happy for this to get merged.

[KludgeKML]

Looks good to me.