Commit graph

1152 commits

Author SHA1 Message Date
Neil Alexander
ca0406e6c7
Fix unit test 2020-09-04 14:40:01 +01:00
Neil Alexander
9ad05e708d
Merge branch 'master' into matthew/peeking 2020-09-04 14:39:44 +01:00
Kegsay
ca8dcf46b7
Remove QuerySharedUsers from current state server (#1396)
* Remove QuerySharedUsers from current state server

* Bugfixes
2020-09-04 14:25:01 +01:00
Neil Alexander
56d772fafc
Cancel peeks on join again 2020-09-04 14:20:00 +01:00
Neil Alexander
2b8f0b8f59
Merge branch 'master' into matthew/peeking 2020-09-04 14:15:51 +01:00
Kegsay
81688d6bde
Remove QueryCurrentState from current-state-server (#1395) 2020-09-04 12:30:56 +01:00
Kegsay
67cc8619d3
Remove QueryKnownUsers from current state server (#1393)
* Remove QueryKnownUsers from current state server

* Fix HTTP mode
2020-09-04 11:46:01 +01:00
Neil Alexander
c63abc8660
Sync bug fixes (#1394)
* Sync bug fixes

* Remove logging
2020-09-04 11:20:47 +01:00
Kegsay
2570418f42
Remove ServerACLs from the current state server (#1390)
* Remove ServerACLs from the current state server

Functionality moved to roomserver

* Nothing to see here, move along
2020-09-04 10:40:58 +01:00
Neil Alexander
f1a98e1193
Fix nil txn bug 2020-09-04 10:22:32 +01:00
Matthew Hodgson
4f5c8cac33
Merge branch 'master' into matthew/peeking 2020-09-04 00:06:58 +01:00
Matthew Hodgson
c775643c50 correctly refactor perform_peek 2020-09-03 23:19:17 +01:00
Matthew Hodgson
3c5e079b87 fix package 2020-09-03 23:06:14 +01:00
Matthew Hodgson
da3742c8e2 move perform_peek 2020-09-03 22:39:12 +01:00
Matthew Hodgson
eda84cd915 reject peeks for non-worldreadable rooms 2020-09-03 22:15:30 +01:00
Neil Alexander
04bc09f591
Defer keyserver and federationsender wakeups to give HTTP listeners time to start (#1389) 2020-09-03 21:17:55 +01:00
Matthew Hodgson
8712ea337a Merge branch 'master' into matthew/peeking 2020-09-03 21:01:30 +01:00
Kegsay
33b8143a95
Implement more CSS storage functions in roomserver (#1388) 2020-09-03 18:27:02 +01:00
Kegsay
b20386123e
Move currentstateserver API to roomserver (#1387)
* Move currentstateserver API to roomserver

Stub out DB functions for now, nothing uses the roomserver version yet.

* Allow it to startup

* Implement some current-state-server storage interface functions

* Add missing package
2020-09-03 17:20:54 +01:00
Neil Alexander
6150de6cb3
FIFO ordering of input events (#1386)
* Initial FIFOing of roomserver inputs

* Remove EventID response from api.InputRoomEventsResponse

* Don't send back event ID unnecessarily

* Fix ordering hopefully

* Reduce copies, use buffered task channel to reduce contention on other rooms

* Fix error handling
2020-09-03 15:22:16 +01:00
Neil Alexander
74743ac8ae
Rate limiting (#1385)
* Initial rate limiting

* Move rate limiting to client API

* Update rate limits to hopefully be self-cleaning

* Use X-Forwarded-For, add comments

* Reduce rate limit threshold

* Tweak interval

* Configurable backoff

* Review comments, set cleanup interval to 30 seconds

* Allow generate-config to produce sane CI config

* Fix Complement dockerfile
2020-09-03 10:12:11 +01:00
Kegsay
d64d0c4be2
Update complement.sh 2020-09-03 10:07:14 +01:00
Kegsay
9d9e854fe0
Add Queryer and Inputer and factor out more RSAPI stuff (#1382)
* Add Queryer and use embedded structs

* Add Inputer and factor out more RS API stuff

This neatly splits up the RS API based on the functionality it provides,
whilst providing a useful place for code sharing via the `helpers` package.
2020-09-02 17:13:15 +01:00
Neil Alexander
f06637435b
Fix #1381 (#1384) 2020-09-02 16:52:06 +01:00
Neil Alexander
3b0774805c
Version imprint (#1383)
* Versions

* Update build.sh
2020-09-02 16:18:08 +01:00
Neil Alexander
096191ca24
Use federation sender for backfill/getting missing events (#1379)
* Use federation sender for backfill and getting missing events

* Fix internal URL paths

* Update go.mod/go.sum for matrix-org/gomatrixserverlib#218

* Add missing server implementations in HTTP interface
2020-09-02 15:26:30 +01:00
Kegsay
e473320e73
Refactor roomserver/internal - split perform stuff out (#1380)
- New package `perform` which contains all `Perform` functions
- New package `helpers` which contains helper functions used by both
  perform and query/input functions.
- Perform invite/leave have no idea how to `WriteOutputEvents` and this
  is now returned from `PerformInvite` or `PerformLeave` respectively.

Still to do:
 - RSAPI is fed into the inviter/joiner/leaver - this introduces circular
   logic so will need to be removed.
 - Put query operations in a `query` package.
 - Put input operations (and output) in an `input` package.
 - Factor out helper functions as much as possible, possibly rejigging the
   storage layer in the process.
2020-09-02 13:47:31 +01:00
Matthew Hodgson
f6af656945 re-add accidentally deleted field 2020-09-02 10:08:10 +01:00
Matthew Hodgson
b6cc4417cc re-add txn to SelectPeeks 2020-09-02 10:07:49 +01:00
Kegsay
02a73f29f8
Expand RoomInfo to cover more DB storage functions (#1377)
* Factor more things to RoomInfo

* Factor out remaining bits for RoomInfo

* Linting for now
2020-09-02 10:02:48 +01:00
Matthew Hodgson
75b91ac9e5 strip out empty roomd deltas 2020-09-01 21:59:35 +01:00
Matthew Hodgson
85bce11964 don't log ascii in binary at sql trace... 2020-09-01 21:59:11 +01:00
Matthew Hodgson
6424117852 use exclusive writer, and MarkPeeksAsOld more efficiently 2020-09-01 21:31:04 +01:00
Matthew Hodgson
5d7f688fa7 wrap peek storage in goid hack 2020-09-01 21:30:03 +01:00
Kegsay
82a9617659
Put redactions/filters in the writer goroutine (#1378)
* Put redactions in the writer goroutine

* Update filters on writer goroutine
2020-09-01 20:35:38 +01:00
Matthew Hodgson
3cebd8dbfb Merge branch 'kegan/HACK-goid-sqlite-db-is-locked' into matthew/peeking 2020-09-01 21:37:22 +03:00
Matthew Hodgson
ed4b3a58a7 Merge branch 'kegan/redact-txn' into matthew/peeking 2020-09-01 21:34:01 +03:00
Kegan Dougal
6410b702ce Update filters on writer goroutine 2020-09-01 19:27:21 +01:00
Kegan Dougal
fcdb90c91b Put redactions in the writer goroutine 2020-09-01 19:21:33 +01:00
Kegan Dougal
7bf2a27319 Track partition offsets and only log unsafe for non-selects 2020-09-01 19:17:01 +01:00
Kegan Dougal
bfecc8e0e9 HACK: Track goroutine IDs to determine when we write by the wrong thread
To use: set `DENDRITE_TRACE_SQL=1` then grep for `unsafe`
2020-09-01 19:10:57 +01:00
Matthew Hodgson
d0d5f70105 Merge branch 'master' into matthew/peeking 2020-09-01 19:11:51 +03:00
Neil Alexander
a9f4d83d30
Fix duplicate writers (#1376)
* Fix writers

* Don't use writers in both shared and sqlite3
2020-09-01 16:58:21 +01:00
Neil Alexander
89c772fb78
Report which component failed to consume (#1375) 2020-09-01 16:53:38 +01:00
Matthew Hodgson
86e9736ca3 fix s/join/peek/ cargocult fail 2020-09-01 18:04:55 +03:00
Matthew Hodgson
28219c66f5 Merge branch 'master' into matthew/peeking 2020-09-01 18:03:06 +03:00
Kegsay
6d79f04354
Add RoomInfo metadata struct (#1367)
* Add RoomInfo struct

* Remove RoomNID and replace with RoomInfo

* Bugfix and remove another needless query

* nil guard
2020-09-01 12:40:49 +01:00
Neil Alexander
0ab5bccd11
Storage tweaks (#1373)
* Sync API tweaks

* User API tweaks
2020-09-01 11:28:35 +01:00
Rohit Mohan
3f9b829bc5
Public room client API changes (#1368)
Signed-off-by: Rohit Mohan <rohitmohan96@gmail.com>
2020-09-01 10:26:34 +01:00
Neil Alexander
b0d2b39739
Remove unused SyncStreamPosition 2020-09-01 10:26:02 +01:00