Commit graph

148 commits

Author SHA1 Message Date
Neil Alexander
bcbe6512a4
Merge branch 'master' into matthew/peeking 2020-09-08 14:07:06 +01:00
Kegsay
7913759921
Remove QueryBulkStateContent from current state server (#1404)
* Remove QueryBulkStateContent from current state server

Expected fail due to db impl not existing

* Implement query bulk state content

* Fix up rejecting invites over federation

* Fix bulk content marshalling
2020-09-07 12:38:09 +01:00
Neil Alexander
b9caccbce8
Update GetStateEvent behaviour (#1399) 2020-09-04 19:40:21 +01:00
Neil Alexander
843b7a7d04
Fix PerformPeekPath 2020-09-04 16:32:29 +01:00
Neil Alexander
2ccd4fa069
Clear a whole bunch of lint issues 2020-09-04 16:17:02 +01:00
Neil Alexander
5c04c52571
Add PerformPeek full-HTTP serverside 2020-09-04 16:09:22 +01:00
Neil Alexander
9ad05e708d
Merge branch 'master' into matthew/peeking 2020-09-04 14:39:44 +01:00
Kegsay
ca8dcf46b7
Remove QuerySharedUsers from current state server (#1396)
* Remove QuerySharedUsers from current state server

* Bugfixes
2020-09-04 14:25:01 +01:00
Neil Alexander
2b8f0b8f59
Merge branch 'master' into matthew/peeking 2020-09-04 14:15:51 +01:00
Kegsay
67cc8619d3
Remove QueryKnownUsers from current state server (#1393)
* Remove QueryKnownUsers from current state server

* Fix HTTP mode
2020-09-04 11:46:01 +01:00
Kegsay
2570418f42
Remove ServerACLs from the current state server (#1390)
* Remove ServerACLs from the current state server

Functionality moved to roomserver

* Nothing to see here, move along
2020-09-04 10:40:58 +01:00
Neil Alexander
f1a98e1193
Fix nil txn bug 2020-09-04 10:22:32 +01:00
Matthew Hodgson
c775643c50 correctly refactor perform_peek 2020-09-03 23:19:17 +01:00
Matthew Hodgson
3c5e079b87 fix package 2020-09-03 23:06:14 +01:00
Matthew Hodgson
da3742c8e2 move perform_peek 2020-09-03 22:39:12 +01:00
Matthew Hodgson
eda84cd915 reject peeks for non-worldreadable rooms 2020-09-03 22:15:30 +01:00
Matthew Hodgson
8712ea337a Merge branch 'master' into matthew/peeking 2020-09-03 21:01:30 +01:00
Kegsay
33b8143a95
Implement more CSS storage functions in roomserver (#1388) 2020-09-03 18:27:02 +01:00
Kegsay
b20386123e
Move currentstateserver API to roomserver (#1387)
* Move currentstateserver API to roomserver

Stub out DB functions for now, nothing uses the roomserver version yet.

* Allow it to startup

* Implement some current-state-server storage interface functions

* Add missing package
2020-09-03 17:20:54 +01:00
Neil Alexander
6150de6cb3
FIFO ordering of input events (#1386)
* Initial FIFOing of roomserver inputs

* Remove EventID response from api.InputRoomEventsResponse

* Don't send back event ID unnecessarily

* Fix ordering hopefully

* Reduce copies, use buffered task channel to reduce contention on other rooms

* Fix error handling
2020-09-03 15:22:16 +01:00
Kegsay
9d9e854fe0
Add Queryer and Inputer and factor out more RSAPI stuff (#1382)
* Add Queryer and use embedded structs

* Add Inputer and factor out more RS API stuff

This neatly splits up the RS API based on the functionality it provides,
whilst providing a useful place for code sharing via the `helpers` package.
2020-09-02 17:13:15 +01:00
Neil Alexander
096191ca24
Use federation sender for backfill/getting missing events (#1379)
* Use federation sender for backfill and getting missing events

* Fix internal URL paths

* Update go.mod/go.sum for matrix-org/gomatrixserverlib#218

* Add missing server implementations in HTTP interface
2020-09-02 15:26:30 +01:00
Kegsay
e473320e73
Refactor roomserver/internal - split perform stuff out (#1380)
- New package `perform` which contains all `Perform` functions
- New package `helpers` which contains helper functions used by both
  perform and query/input functions.
- Perform invite/leave have no idea how to `WriteOutputEvents` and this
  is now returned from `PerformInvite` or `PerformLeave` respectively.

Still to do:
 - RSAPI is fed into the inviter/joiner/leaver - this introduces circular
   logic so will need to be removed.
 - Put query operations in a `query` package.
 - Put input operations (and output) in an `input` package.
 - Factor out helper functions as much as possible, possibly rejigging the
   storage layer in the process.
2020-09-02 13:47:31 +01:00
Kegsay
02a73f29f8
Expand RoomInfo to cover more DB storage functions (#1377)
* Factor more things to RoomInfo

* Factor out remaining bits for RoomInfo

* Linting for now
2020-09-02 10:02:48 +01:00
Matthew Hodgson
d0d5f70105 Merge branch 'master' into matthew/peeking 2020-09-01 19:11:51 +03:00
Neil Alexander
a9f4d83d30
Fix duplicate writers (#1376)
* Fix writers

* Don't use writers in both shared and sqlite3
2020-09-01 16:58:21 +01:00
Matthew Hodgson
28219c66f5 Merge branch 'master' into matthew/peeking 2020-09-01 18:03:06 +03:00
Kegsay
6d79f04354
Add RoomInfo metadata struct (#1367)
* Add RoomInfo struct

* Remove RoomNID and replace with RoomInfo

* Bugfix and remove another needless query

* nil guard
2020-09-01 12:40:49 +01:00
Matthew Hodgson
c4e5f60d71 make it build 2020-08-31 16:12:09 +03:00
Matthew Hodgson
cfa0be544d merge master 2020-08-31 12:26:27 +03:00
Matthew Hodgson
b9342d9ee2 a very very WIP first cut of peeking via MSC2753.
doesn't yet compile or work.
needs to actually add the peeking block into the sync response.
checking in now before it gets any bigger, and to gather any initial feedback on the vague shape of it.
2020-08-30 17:46:15 +03:00
Neil Alexander
034e5d5a55
Speed up accepting invites from remote users if we're already in the room (#1349) 2020-08-27 11:47:13 +01:00
oliverpool
3205b9212d
[readability] use event.StateKeyEquals where relevant and minor for-loop refactoring (#1339)
* use event.StateKeyEquals where relevant

Signed-off-by: Olivier Charvin <git@olivier.pfad.fr>

* use userID
2020-08-25 21:04:35 +01:00
Neil Alexander
819b926820
Make redactions permanent (#1337)
* Make redactions permanent

* Update storage.go
2020-08-25 15:44:19 +01:00
oliverpool
a4db43e096
Don't overwrite global err before return (#1293)
Signed-off-by: Olivier Charvin <git@olivier.pfad.fr>
2020-08-25 13:11:52 +01:00
Neil Alexander
c8b873abc8
Roomserver NID caches (#1335)
* Initial work on roomserver NID caches

* Give caches to roomserver storage

* Populate caches

* Fix bugs

* Fix WASM build

* Don't hit cache twice in RoomNIDExcludingStubs

* Store reverse room ID-room NID mapping, consult caches when assigning NIDs
2020-08-25 12:32:29 +01:00
Neil Alexander
9d53351dc2
Component-wide TransactionWriters (#1290)
* Offset updates take place using TransactionWriter

* Refactor TransactionWriter in current state server

* Refactor TransactionWriter in federation sender

* Refactor TransactionWriter in key server

* Refactor TransactionWriter in media API

* Refactor TransactionWriter in server key API

* Refactor TransactionWriter in sync API

* Refactor TransactionWriter in user API

* Fix deadlocking Sync API tests

* Un-deadlock device database

* Fix appservice API

* Rename TransactionWriters to Writers

* Move writers up a layer in sync API

* Document sqlutil.Writer interface

* Add note to Writer documentation
2020-08-21 10:42:08 +01:00
Neil Alexander
068a3d3c9f
Roomserver per-room input parallelisation (Postgres) (#1289)
* Per-room input mutex

* GetMembership should use transaction when assigning state key NID

* Actually use writer transactions rather than ignoring them

* Limit per-room mutexes to Postgres

* Flip the check in InputRoomEvents
2020-08-20 16:24:33 +01:00
Neil Alexander
5ad47d3b3d
Fix more roomserver transactions/locks (#1287)
* Fix transaction to InsertTransaction

* Remove unnecessary txn, add txns around setting up updaters
2020-08-20 09:24:52 +01:00
Neil Alexander
b24747b305
Transaction writer changes, move roomserver writers (#1285)
* Updated TransactionWriters, moved locks in roomserver, various other tweaks

* Fix redaction deadlocks

* Fix lint issue

* Rename SQLiteTransactionWriter to ExclusiveTransactionWriter

* Fix us not sending transactions through in latest events updater
2020-08-19 15:38:27 +01:00
Neil Alexander
775b04d776
Roomserver updater changes (#1283)
* Take input transaction when setting up updaters

* Fix nil pointer exceptions

* Rename room recent events updater to latest events updater

* Contd rename room recent events updater to latest events updater

* Remove unnecessary interfaces for latest events and membership updaters
2020-08-19 13:24:54 +01:00
Neil Alexander
3d58417555
Roomserver database-wide TransactionWriters (#1282)
* Database-wide TransactionWriter

* Fix deadlocking Sync API tests

* Undo non-roomserver changes for now
2020-08-19 10:57:29 +01:00
Neil Alexander
a5a85c6a11
Make PerformJoin responsible for sending invite to RS input (#1277)
* Make PerformJoin send input membership event

* Invite input room events in separate goroutine

* Don't limit roomserver input events using request context

* Synchronous input room events

* Nope, that didn't work

* oops send state key to GetMembership

* Don't generate stripped state in client API more times than necessary, generate output events on receiving end of federated invite

* Commit membership updater changes

* Tweaks
2020-08-17 17:33:19 +01:00
Neil Alexander
6cb1a65809
Synchronous invites (#1273)
* Refactor invites to be synchronous

* Fix synchronous invites

* Fix client API return type for send invite error

* Linter

* Restore PerformError on rsAPI.PerformInvite

* Update sytest-whitelist

* Don't override PerformError with normal errors

* Fix error passing

* Un-whitelist a couple of tests

* Update sytest-whitelist

* Try to handle multiple invite rejections better

* nolint

* Update gomatrixserverlib

* Fix /v1/invite test

* Remove replace from go.mod
2020-08-17 11:40:49 +01:00
Neil Alexander
52eeeb1627
Prefix-defined Kafka topics (#1254)
* Prefix-defined Kafka topics

* Fix current state server test
2020-08-10 15:18:37 +01:00
Neil Alexander
4b09f445c9
Configuration format v1 (#1230)
* Initial pass at refactoring config (not finished)

* Don't forget current state and EDU servers

* More shifting around

* Update server key API tests

* Fix roomserver test

* Fix more tests

* Further tweaks

* Fix current state server test (sort of)

* Maybe fix appservices

* Fix client API test

* Include database connection string in database options

* Fix sync API build

* Update config test

* Fix unit tests

* Fix federation sender build

* Fix gobind build

* Set Listen address for all services in HTTP monolith mode

* Validate config, reinstate appservice derived in directory, tweaks

* Tweak federation API test

* Set MaxOpenConnections/MaxIdleConnections to previous values

* Update generate-config
2020-08-10 14:18:04 +01:00
Kegan Dougal
8ee4c0f71e Always set invite_room_state else synapse rejects invites 2020-08-04 12:14:18 +01:00
Kegsay
c632867135
Modify /state/{eventType}/{stateKey} to return the event at the time the user left (#1222)
* Modify /state/{eventType}/{stateKey} to return the event at the time the user left

Or live, depending on their current state. Hopefully fixes some sytests!

* Linting

* Set HasBeenInRoom

* Fix cases for world-readable history visibility

* Fix bug in finding the requested state event

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-07-28 10:09:10 +01:00
Neil Alexander
f54689f862
Fix lint error 2020-07-21 17:21:48 +01:00
Neil Alexander
b6bc132485
Use TransactionWriter in other component SQLite (#1209)
* Use TransactionWriter on other component SQLites

* Fix sync API tests

* Fix panic in media API

* Fix a couple of transactions

* Fix wrong query, add some logging output

* Add debug logging into StoreEvent

* Adjust InsertRoomNID

* Update logging
2020-07-21 15:48:21 +01:00