Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Make AS's faster #2076

Merged
merged 4 commits into from Mar 31, 2017

Conversation

Projects
None yet
3 participants
Owner

erikjohnston commented Mar 28, 2017

No description provided.

erikjohnston added some commits Mar 28, 2017

synapse/appservice/__init__.py
- if not isinstance(regex_obj.get("regex"), basestring):
+ regex = regex_obj.get("regex")
+ if isinstance(regex, basestring):
+ regex_obj["regex"] = re.compile(regex)
@Kegsay

Kegsay Mar 28, 2017

Contributor

Can you please comment that you're pre-compiling the regexps.

@erikjohnston

erikjohnston Mar 28, 2017

Owner

Err, like this?

regex_obj["regex"] = re.compile(regex)  # Pre-compile regex

Isn't that commenting on what a line does? I don't see how it helps understanding?

@Kegsay

Kegsay Mar 28, 2017

Contributor

You're clobbering directly over a regex string with a regex object, but fail to say why you're doing this - which is to pre-compile it so you don't need to incur repeated compilation costs. A one-liner inline comment as you have there looks good to me.

@erikjohnston

erikjohnston Mar 28, 2017

Owner

Hmm, in which case I think I'll change it to not clobber the existing value, or comment in the function name/docstring.

A comment which is literally describing what a code line is doing always feels wrong to me, and from your comment it sounds like you more dislike the clobbering here than not understanding what's going on. The question is: who is this comment going to help? People who are trying to figure out what the type is will either a) not find the line and guess or b) see from the code its a regex object.

@Kegsay

Kegsay Mar 28, 2017

Contributor

A comment which is literally describing what a code line is doing

It isn't. This is a comment which literally describes what a code line is doing:

# replace regex string with regex object
regex_obj["regex"] = re.compile(regex)

What I want to know is why you're doing this. Why don't we keep them as strings and then do it on the fly? Because that's needlessly slow compared to pre-compiling the regex. I don't care about the fact you're clobbering, I care that you fail to describe your intent.

raise ValueError(
"Expected string for 'regex' in ns '%s'" % ns
)
return namespaces
- def _matches_regex(self, test_string, namespace_key, return_obj=False):
- if not isinstance(test_string, basestring):
@Kegsay

Kegsay Mar 28, 2017

Contributor

We've lost this type check entirely - is this check expensive enough to warrant that (eg in JS it actually kinda is, introspection of types is painful, python idk).

@erikjohnston

erikjohnston Mar 28, 2017

Owner

This is basically against the code style, we never check parameters (except in exceptional circumstances). I think it hurts readability if we do type checking of all parameters, even worse when we only check one parameter.

Besides, the call to re.match will implode if its not a string.

@Kegsay

Kegsay Mar 28, 2017

Contributor

Okay.

raise ValueError(
"Expected string for 'regex' in ns '%s'" % ns
)
return namespaces
- def _matches_regex(self, test_string, namespace_key, return_obj=False):
@Kegsay

Kegsay Mar 28, 2017

Contributor

We've lost the return_obj arg. Was it ever used?

@erikjohnston

erikjohnston Mar 28, 2017

Owner

Yes, hence why I changed it at the call sites as per commit message

@Kegsay

Kegsay Mar 28, 2017

Contributor

Okay.

@Kegsay Kegsay assigned Kegsay and unassigned richvdh Mar 28, 2017

Contributor

Kegsay commented Mar 28, 2017

Otherwise LGTM!

Contributor

Kegsay commented Mar 30, 2017

Pending comment, but then LGTM

@erikjohnston erikjohnston merged commit 350333a into develop Mar 31, 2017

0 of 5 checks passed

Sytest Dendron (Merged PR) Build triggered. sha1 is merged.
Details
Sytest Postgres (Merged PR) Build triggered. sha1 is merged.
Details
Sytest SQLite (Merged PR) Build triggered. sha1 is merged.
Details
continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
continuous-integration/travis-ci/push The Travis CI build is in progress
Details

psaavedra added a commit to psaavedra/synapse that referenced this pull request May 19, 2017

Merge tag 'v0.21.0' into v0.21.0_no_federate_by_default
Changes in synapse v0.21.0 (2017-05-18)
=======================================

No changes since v0.21.0-rc3

Changes in synapse v0.21.0-rc3 (2017-05-17)
===========================================

Features:

* Add per user rate-limiting overrides (PR #2208)
* Add config option to limit maximum number of events requested by ``/sync``
  and ``/messages`` (PR #2221) Thanks to @psaavedra!

Changes:

* Various small performance fixes (PR #2201, #2202, #2224, #2226, #2227, #2228,
  #2229)
* Update username availability checker API (PR #2209, #2213)
* When purging, don't de-delta state groups we're about to delete (PR #2214)
* Documentation to check synapse version (PR #2215) Thanks to @hamber-dick!
* Add an index to event_search to speed up purge history API (PR #2218)

Bug fixes:

* Fix API to allow clients to upload one-time-keys with new sigs (PR #2206)

Changes in synapse v0.21.0-rc2 (2017-05-08)
===========================================

Changes:

* Always mark remotes as up if we receive a signed request from them (PR #2190)

Bug fixes:

* Fix bug where users got pushed for rooms they had muted (PR #2200)

Changes in synapse v0.21.0-rc1 (2017-05-08)
===========================================

Features:

* Add username availability checker API (PR #2183)
* Add read marker API (PR #2120)

Changes:

* Enable guest access for the 3pl/3pid APIs (PR #1986)
* Add setting to support TURN for guests (PR #2011)
* Various performance improvements (PR #2075, #2076, #2080, #2083, #2108,
  #2158, #2176, #2185)
* Make synctl a bit more user friendly (PR #2078, #2127) Thanks @APwhitehat!
* Replace HTTP replication with TCP replication (PR #2082, #2097, #2098,
  #2099, #2103, #2014, #2016, #2115, #2116, #2117)
* Support authenticated SMTP (PR #2102) Thanks @DanielDent!
* Add a counter metric for successfully-sent transactions (PR #2121)
* Propagate errors sensibly from proxied IS requests (PR #2147)
* Add more granular event send metrics (PR #2178)

Bug fixes:

* Fix nuke-room script to work with current schema (PR #1927) Thanks
  @zuckschwerdt!
* Fix db port script to not assume postgres tables are in the public schema
  (PR #2024) Thanks @jerrykan!
* Fix getting latest device IP for user with no devices (PR #2118)
* Fix rejection of invites to unreachable servers (PR #2145)
* Fix code for reporting old verify keys in synapse (PR #2156)
* Fix invite state to always include all events (PR #2163)
* Fix bug where synapse would always fetch state for any missing event (PR #2170)
* Fix a leak with timed out HTTP connections (PR #2180)
* Fix bug where we didn't time out HTTP requests to ASes  (PR #2192)

Docs:

* Clarify doc for SQLite to PostgreSQL port (PR #1961) Thanks @benhylau!
* Fix typo in synctl help (PR #2107) Thanks @HarHarLinks!
* ``web_client_location`` documentation fix (PR #2131) Thanks @matthewjwolff!
* Update README.rst with FreeBSD changes (PR #2132) Thanks @feld!
* Clarify setting up metrics (PR #2149) Thanks @encks!

@erikjohnston erikjohnston deleted the erikj/as_perf branch Oct 26, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment