Add query parameter ts to allow appservices set the origin_server_ts for state events

[lukasdenk]

Closes #10634.

Allow appservices to set the origin_server_ts of a state event by providing the query parameter ts in PUT /_matrix/client/r0/rooms/{roomId}/state/{eventType}/{stateKey}.

Signed-off by: Lukas Denk, lukasdenk@web.de

Pull Request Checklist

• Pull request is based on the develop branch
• Pull request includes a changelog file. The entry should:
  • Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
  • Use markdown where necessary, mostly for code blocks.
  • End with either a period (.) or an exclamation mark (!).
  • Start with a capital letter.
  • Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
• Pull request includes a sign off
• Code style is correct
  (run the linters)

[anoadragon453]

Thanks for this @lukasdenk! As noted in #10634 (comment), we'll need to use an unstable prefix (something like org.matrix.msc3316.ts) in place of ts before the MSC can be merged. That unstable prefix first should be decided on in matrix-org/matrix-spec-proposals#3316, then used here.

I'd also like to see this PR introduce the unstable prefix as an acceptable alternative parameter alongside ts for PUT /_matrix/client/r0/rooms/{roomId}/send/{eventType}/{txnId}. Otherwise we'll end up in a state where one endpoint supports the stable parameter, while the other endpoint only supports the unstable one - which could complicate client code.

Also note that linting isn't passing currently. Note that you can run the linting tools locally (and even have many issues auto-corrected), as described in the contributing guide.

[anoadragon453]

Heads up - when this is ready for review again just click "request review" next to my avatar to let me know :)

The little 🔁 icon.

[lukasdenk]

@anoadragon453 Do you have any idea why trial-olddeps fails with tests.rest.client.test_rooms does not exist?

[squahtx]

@anoadragon453 Do you have any idea why trial-olddeps fails with tests.rest.client.test_rooms does not exist?

@lukasdenk MemoryReactor's moved around between twisted versions. You'll have to import it like so:
from twisted.test.proto_helpers import MemoryReactor

[lukasdenk]

@richvdh I got a weird problem. My local tests do not work, even for this commit, which has passed the pipeline.
Running trial tests.rest.client.test_rooms.RoomAppserviceTsParamTestCase.test_send_event_ts gives:

tests.rest.client.test_rooms
  RoomAppserviceTsParamTestCase
    test_send_event_ts ...                                              [ERROR]

===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/home/lukas/PycharmProjects/synapse/env/lib/python3.8/site-packages/twisted/trial/runner.py", line 693, in loadByName
    return self.suiteFactory([self.findByName(name, recurse=recurse)])
  File "/home/lukas/PycharmProjects/synapse/env/lib/python3.8/site-packages/twisted/trial/runner.py", line 453, in findByName
    obj = reflect.namedModule(searchName)
  File "/home/lukas/PycharmProjects/synapse/env/lib/python3.8/site-packages/twisted/python/reflect.py", line 157, in namedModule
    topLevel = __import__(name)
  File "/home/lukas/PycharmProjects/synapse/tests/rest/client/test_rooms.py", line 29, in <module>
    import synapse.rest.admin
  File "/home/lukas/PycharmProjects/synapse/synapse/rest/__init__.py", line 17, in <module>
    from synapse.http.server import HttpServer, JsonResource
  File "/home/lukas/PycharmProjects/synapse/synapse/http/server.py", line 59, in <module>
    from synapse.http.site import SynapseRequest
  File "/home/lukas/PycharmProjects/synapse/synapse/http/site.py", line 30, in <module>
    from synapse.http.request_metrics import RequestMetrics, requests_counter
  File "/home/lukas/PycharmProjects/synapse/synapse/http/request_metrics.py", line 24, in <module>
    from synapse.metrics import LaterGauge
  File "/home/lukas/PycharmProjects/synapse/synapse/metrics/__init__.py", line 37, in <module>
    from matrix_common.versionstring import get_distribution_version_string
builtins.ModuleNotFoundError: No module named 'matrix_common.versionstring'

tests.rest.client.test_rooms.RoomAppserviceTsParamTestCase.test_send_event_ts
-------------------------------------------------------------------------------
Ran 1 tests in 0.020s

FAILED (errors=1)

Do you have any idea how that can be? As far as I remember, a month ago, the test worked locally.

[DMRobertson]

Do you have any idea how that can be? As far as I remember, a month ago, the test worked locally.

Synapse now depends on a newer version of matrix-common than you have installed. Run pip install -U matrix-common and try running the tests again.

[anoadragon453]

Hi @lukasdenk. This PR looks like it was pretty close to being completed. Do you have any time to get it over the line?

The first step that I would suggest doing is to pull in the latest changes from the develop branch with:

git pull origin develop

and then to resolve the git conflicts.

[tulir]

All the unstable prefixes can also be removed now since the MSC is stable and fully released as of Matrix v1.3

[lukasdenk]

Hi @lukasdenk. This PR looks like it was pretty close to being completed. Do you have any time to get it over the line?

The first step that I would suggest doing is to pull in the latest changes from the develop branch with:

git pull origin develop

and then to resolve the git conflicts.

It is quite close, I know. But I don't think I have the time to finish it. Sorry.

[clokep]

@lukasdenk Hope you don't mind, but I merged in develop and handled some of the review comments. Hopefully we can get this over the line! Seems like you did most of the work already! 🎉

[richvdh]

lgtm!