Inventory: Restrict access from too far away

[sofar]

Backport from master

[rubenwardy]

Wrong branch, put into backport-0.4

[rubenwardy]

Also, it's probably better to backport and fixes before this, so we don't miss them

[sfan5]

That's not how I intended the backports branch to work.
Instead someone should go through all commits from the point at which it was last updated to now, and cherry pick each commit in the correct order. (list is here)

Not doing so might make merging messy and complicates keeping track of what was backported and what wasn't. It gets even worse when you decide that you want to remove a backported commit (and honor the no forcepush rule).

[sfan5]

added to branch