Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add options to require an access_token to GET /profile and /publicRooms on CS API #5083

Merged
merged 31 commits into from May 8, 2019

Conversation

Projects
None yet
4 participants
@ara4n
Copy link
Member

commented Apr 21, 2019

This deliberately doesn't try to restrict access to profile or publicRooms data over SS API, and as such is only useful on trusted private federations.

A more generic solution would be MSC1301 or matrix-org/matrix-doc#612, but this focuses on restricting the CS API as an immediate fix.

@ara4n ara4n requested a review from matrix-org/synapse-core Apr 21, 2019

@codecov

This comment has been minimized.

Copy link

commented Apr 21, 2019

Codecov Report

Merging #5083 into dinsic will increase coverage by 15.53%.
The diff coverage is 42.85%.

@@             Coverage Diff             @@
##           dinsic    #5083       +/-   ##
===========================================
+ Coverage   45.19%   60.72%   +15.53%     
===========================================
  Files         351      335       -16     
  Lines       35845    34695     -1150     
  Branches        0     5733     +5733     
===========================================
+ Hits        16199    21069     +4870     
+ Misses      19646    12063     -7583     
- Partials        0     1563     +1563
@codecov

This comment has been minimized.

Copy link

commented Apr 21, 2019

Codecov Report

Merging #5083 into develop will increase coverage by 0.06%.
The diff coverage is 78.04%.

@@             Coverage Diff             @@
##           develop    #5083      +/-   ##
===========================================
+ Coverage    61.67%   61.73%   +0.06%     
===========================================
  Files          334      334              
  Lines        34512    34549      +37     
  Branches      5680     5688       +8     
===========================================
+ Hits         21284    21328      +44     
+ Misses       11699    11692       -7     
  Partials      1529     1529
@babolivier
Copy link
Member

left a comment

lgtm

Show resolved Hide resolved docs/sample_config.yaml Outdated

@ara4n ara4n changed the title add option to require an access_token to GET /profile on CS API add options to require an access_token to GET /profile and /publicRooms on CS API Apr 21, 2019

@richvdh
Copy link
Member

left a comment

Why would you want to lock down these APIs in the CS API whilst still leaving the federation API wide-open? It sounds like a massive footgun to me.

Show resolved Hide resolved synapse/config/server.py Outdated
Show resolved Hide resolved synapse/config/server.py Outdated
Show resolved Hide resolved synapse/config/server.py Outdated
Show resolved Hide resolved synapse/rest/client/v1/room.py Outdated
Show resolved Hide resolved synapse/rest/client/v1/room.py Outdated
@richvdh

This comment has been minimized.

Copy link
Member

commented Apr 23, 2019

also, the CI is failing

Show resolved Hide resolved synapse/config/server.py Outdated
@richvdh

This comment has been minimized.

Copy link
Member

commented Apr 23, 2019

ok I now see

as such is only useful on trusted private federations.

Which is fine, but don't really understand why you are taking this approach. Is the idea that we follow-up with another PR to lock down the SS API, or is this PR meant to stand alone? If the former, I'm concerned that by only considering the CS API here, we will end up with two options which are hard to use correctly and easy to screw up. If the latter, the config setting needs big health warnings.

@ara4n

This comment has been minimized.

Copy link
Member Author

commented Apr 23, 2019

but don't really understand why you are taking this approach.

as an immediate fix for private federations (i.e. DINSIC, hence this PR being against the DINSIC branch).

Is the idea that we follow-up with another PR to lock down the SS API,

yes, as per the original comment.

also, the CI is failing

yes, looks like CI being broken, given the failures relate to ASes.

@ara4n

This comment has been minimized.

Copy link
Member Author

commented Apr 23, 2019

so, i wrote this PR as an immediate fix to an operational problem. I think there is value in locking down the CS API independently to the SS API for people running private federations, but it's a bit of an edge case so would be better to fix the SS API too.

please can someone in @matrix-org/synapse-core take it and handle @richvdh's feedback and make it work sensibly with SS API.

My suggestion would be:

  • Add a config option to make the room directory only visible to locally registered users (i.e. restrict access via SS API)
  • Implement MSC1301 to lock down visibility of profiles.
@babolivier

This comment has been minimized.

Copy link
Member

commented Apr 23, 2019

please can someone in @matrix-org/synapse-core take it and handle @richvdh's feedback and make it work sensibly with SS API.

/me does

Implement MSC1301 to lock down visibility of profiles.

Shouldn't we wait for MSC1301 to be approved before implementing it? The config option that's already in the PR (i.e. requesting valid access tokens on the CS API for profile queries) sounds good enough to me in the meantime. Though I'm unsure about what to do with the matching federation endpoints in this case.

useless now

@richvdh

This comment has been minimized.

Copy link
Member

commented Apr 23, 2019

hence this PR being against the DINSIC branch

I'm afraid I missed that. It would be helpful to call that sort of thing out explicitly, though given I evidently failed to read the PR description anyway, I'm not entirely sure what more you could have done :/. Generally I think it's the sort of problem that happens when you have special branches.

@ara4n

This comment has been minimized.

Copy link
Member Author

commented Apr 23, 2019

Shouldn't we wait for MSC1301 to be approved before implementing it?

I wouldn't (as per #synapse-dev). MSC1301 is a suggestion for an implementation behaviour (i.e. additional validation of profile endpoints) to be standardised into the spec. There's nothing stopping an implementation to independently provide an impl-specific option to apply additional validation to its endpoints if it chooses - just like this PR already does. MSC1301 just says "hey, why don't we all do it like that".

@ara4n

This comment has been minimized.

Copy link
Member Author

commented Apr 23, 2019

The config option that's already in the PR (i.e. requesting valid access tokens on the CS API for profile queries) sounds good enough to me in the meantime. Though I'm unsure about what to do with the matching federation endpoints in this case.

AIUI, the idea is to expand this PR to cover both CS & SS API behaviour to mitigate @richvdh's concern about it only covering CS API being a footgun.

@babolivier

This comment has been minimized.

Copy link
Member

commented Apr 25, 2019

So here are the config options this PR adds:

  • restrict_public_rooms_to_local_users

Requires auth to fetch the public rooms directory through the CS API and disables fetching it through the federation API.

  • require_auth_for_profile_requests

When set to true, requires that requests to /profile over the CS API are authenticated, and only returns the user's profile if the requester shares a room with the profile's owner, as per MSC1301.

MSC1301 also specifies a behaviour for federation (only returning the profile if the server asking for it shares a room with the profile's owner), but that's currently really non-trivial to do in a not too expensive way. When discussing that with @erikjohnston, we concluded that a good way to solve the issue would be to propose a MSC that adds the user ID of the profile's requester to profile queries over federation, which I plan on doing. In the current state of this PR, Synapse won't send a profile query over federation if it doesn't believe it already shares a room with the profile's owner, which should be good enough in the meantime?

I've also omitted groups from this PR (for context, a group's server sends profiles of members when previewing it), because I don't think we can realistically expect the server owning a group to be aware of whether every server asking for info on the group share a room with each member (and afaik the groups implementation isn't in its final form yet).

@babolivier

This comment has been minimized.

Copy link
Member

commented Apr 25, 2019

Also, no idea why CI is failing, despite spending some time looking into it. Possibly a race (given it only happens to one sytest on unrelated tests), but can't find the source. If someone could have another look at it to see if they can find something obviously wrong with my PR that would explain it, that'd be amazing.

Edit: Well, now that I rebased the PR against develop, all of the sytest runs succeed...

@babolivier babolivier requested a review from matrix-org/synapse-core Apr 25, 2019

@babolivier babolivier changed the base branch from dinsic to develop Apr 25, 2019

@babolivier babolivier changed the base branch from develop to dinsic Apr 25, 2019

babolivier and others added some commits Apr 25, 2019

fix
Merge pull request #5098 from matrix-org/rav/fix_pep_517
Workarounds for pep-517 errors
fix
fix
@richvdh

This comment has been minimized.

Copy link
Member

commented Apr 30, 2019

I'm going to merge develop into this branch in an attempt to get github to give me a sensible diff

@richvdh
Copy link
Member

left a comment

Generally looks good! A few suggestions.

Show resolved Hide resolved synapse/config/server.py
Show resolved Hide resolved synapse/federation/transport/server.py Outdated
Show resolved Hide resolved synapse/handlers/profile.py Outdated
Show resolved Hide resolved synapse/handlers/profile.py Outdated
Show resolved Hide resolved synapse/handlers/profile.py Outdated

@babolivier babolivier requested a review from matrix-org/synapse-core May 1, 2019

babolivier added some commits May 1, 2019

Show resolved Hide resolved synapse/federation/transport/server.py Outdated
Show resolved Hide resolved synapse/handlers/profile.py Outdated

richvdh and others added some commits May 8, 2019

Update synapse/handlers/profile.py
Co-Authored-By: babolivier <contact@brendanabolivier.com>

@babolivier babolivier requested a review from matrix-org/synapse-core May 8, 2019

@richvdh

richvdh approved these changes May 8, 2019

Copy link
Member

left a comment

lgtm!

@defer.inlineCallbacks
def on_GET(self, origin, content, query):
if self.deny_access:

This comment has been minimized.

Copy link
@richvdh

richvdh May 8, 2019

Member

feels like maybe we should have a completely different Servlet class, but <shrug>. Let's go with this!

@babolivier babolivier merged commit c0e0740 into develop May 8, 2019

24 checks passed

buildkite/synapse Build #1269 passed (14 minutes, 4 seconds)
Details
buildkite/synapse/check-sample-config Passed (1 minute, 9 seconds)
Details
buildkite/synapse/isort Passed (21 seconds)
Details
buildkite/synapse/newspaper-newsfile Passed (15 seconds)
Details
buildkite/synapse/packaging Passed (18 seconds)
Details
buildkite/synapse/pep-8 Passed (54 seconds)
Details
buildkite/synapse/pipeline Passed (2 seconds)
Details
buildkite/synapse/python-2-dot-7-slash-postgres-9-dot-4 Passed (11 minutes, 26 seconds)
Details
buildkite/synapse/python-2-dot-7-slash-postgres-9-dot-5 Passed (11 minutes, 14 seconds)
Details
buildkite/synapse/python-2-dot-7-slash-sqlite Passed (6 minutes, 22 seconds)
Details
buildkite/synapse/python-2-dot-7-slash-sqlite-slash-old-deps Passed (7 minutes, 43 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-postgres-9-dot-4 Passed (12 minutes, 31 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-postgres-9-dot-5 Passed (12 minutes, 55 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-sqlite Passed (7 minutes, 36 seconds)
Details
buildkite/synapse/python-3-dot-6-slash-sqlite Passed (7 minutes, 20 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-postgres-11 Passed (12 minutes, 22 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-postgres-9-dot-5 Passed (12 minutes, 30 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-sqlite Passed (7 minutes, 33 seconds)
Details
ci/circleci: sytestpy2merged Your tests passed on CircleCI!
Details
ci/circleci: sytestpy2postgresmerged Your tests passed on CircleCI!
Details
ci/circleci: sytestpy3merged Your tests passed on CircleCI!
Details
ci/circleci: sytestpy3postgresmerged Your tests passed on CircleCI!
Details
codecov/patch 78.04% of diff hit (target 0%)
Details
codecov/project 61.73% (target 0%)
Details

anoadragon453 added a commit that referenced this pull request May 10, 2019

Merge branch 'develop' into anoa/blacklist_ip_ranges
* develop: (45 commits)
  URL preview blacklisting fixes (#5155)
  Revert 085ae34
  Add a DUMMY stage to captcha-only registration flow
  Make Prometheus snippet less confusing on the metrics collection doc (#4288)
  Set syslog identifiers in systemd units (#5023)
  Run Black on the tests again (#5170)
  Add AllowEncodedSlashes to apache (#5068)
  remove instructions for jessie installation (#5164)
  Run `black` on per_destination_queue
  Limit the number of EDUs in transactions to 100 as expected by receiver (#5138)
  Fix bogus imports in tests (#5154)
  add options to require an access_token to GET /profile and /publicRooms on CS API (#5083)
  Do checks on aliases for incoming m.room.aliases events (#5128)
  Remove the requirement to authenticate for /admin/server_version. (#5122)
  Fix spelling in server notices admin API docs (#5142)
  Fix sample config
  0.99.3.2
  include disco in deb build target list
  changelog
  Debian: we now need libpq-dev.
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.