[RRFC] Exclude `node_modules` from Time Machine backups (macOS)

[probablykasper]

Motivation

node_modules often contains a lot of files that change frequently, which can significantly slow down backup solutions like Time Machine, so it would make sense to exclude them from backups.

How

Current Behaviour

node_modules folders are included in Time Machine backups

Desired Behaviour

Exclude node_modules from Time Machine backups using a "sticky exclusion". This can be done by adding the xattr com.apple.metadata:com_apple_backup_excludeItem to the node_modules folders with the value com.apple.backupd encoded as a binary plist:

00000000  62 70 6C 69 73 74 30 30 5F 10 11 63 6F 6D 2E 61  |bplist00_..com.a|
00000010  70 70 6C 65 2E 62 61 63 6B 75 70 64 08 00 00 00  |pple.backupd....|
00000020  00 00 00 01 01 00 00 00 00 00 00 00 01 00 00 00  |................|
00000030  00 00 00 00 00 00 00 00 00 00 00 00 1C           |.............|
0000003d

References

Rust already does this for their target folders: rust-lang/cargo#4386
My PR for implementing this for the poetry Python package manager: python-poetry/poetry#4599

[ljharb]

Ooh, this is interesting, i wonder if this would also exclude them from Dropbox syncing?

[probablykasper]

@ljharb I don't think so. You can try run tmutil addexclusion <path> to see

[isaacs]

Does node provide a way to set xattrs? Shipping a binary addon in npm would be tricky.

[ljharb]

Presumably the binary exists only on Macs, so you could shell out to it when present?

[targos]

I spoke about this with a colleague and he said that he would be very surprised if it was the default behavior.
Maybe the proposal should be about an option that could be set in .npmrc

[ljharb]

@targos can you get some elaboration on why they’d want their node_modules to be backed up?

[targos]

@ljharb They'd like their setup to work without an internet connection after restoring from a backup.

[ljharb]

On the rfc call, we discussed some of the tradeoffs of an opt-in vs an opt-out; my take is that the risk caused by using opt-out is that your colleague would have to discover it, be unable to avoid needing internet after a restore, and then Google for the option, and then set it; the risk caused by it being opt-in is that someone’s backup disk prematurely fills up with node_modules and there’s no backup available to restore at all.

[probablykasper]

With opt-in, you might not realize that node_modules is clogging up and slowing down backups, and once you do you may assume there's no easy way around it.
With opt-out, you might not realize it's not backed up, and once you do you're likely to search for a solution if you want different behavior.

I think the vast majority of people would prefer node_modules to not be backed up, which would mean opt-out results in less issues/confusion. As far as I'm concerned, node_modules just acts as a cache based on package-lock.json, so I would not expect it to be in backups in the first place.

[adlawan]

Need the machine to rolling