Implement Push Notifications (#1842)

* Add Pushserver component with Pushers API

Co-authored-by: Tommie Gannert <tommie@gannert.se>
Co-authored-by: Dan Peleg <dan@globekeeper.com>

* Wire Pushserver component

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>

* Add PushGatewayClient.

The full event format is required for Sytest.

* Add a pushrules module.

* Change user API account creation to use the new pushrules module's defaults.

Introduces "scope" as required by client API, and some small field
tweaks to make some 61push Sytests pass.

* Add push rules query/put API in Pushserver.

This manipulates account data over User API, and fires sync messages
for changes. Those sync messages should, according to an existing TODO
in clientapi, be moved to userapi.

Forks clientapi/producers/syncapi.go to pushserver/ for later extension.

* Add clientapi routes for push rules to Pushserver.

A cleanup would be to move more of the name-splitting logic into
pushrules.go, to depollute routing.go.

* Output rooms.join.unread_notifications in /sync.

This is the read-side. Pushserver will be the write-side.

* Implement pushserver/storage for notifications.

* Use PushGatewayClient and the pushrules module in Pushserver's room consumer.

* Use one goroutine per user to avoid locking up the entire server for
  one bad push gateway.
* Split pushing by format.
* Send one device per push. Sytest does not support coalescing
  multiple devices into one push. Matches Synapse. Either we change
  Sytest, or remove the group-by-url-and-format logic.
* Write OutputNotificationData from push server. Sync API is already
  the consumer.

* Implement read receipt consumers in Pushserver.

Supports m.read and m.fully_read receipts.

* Add clientapi route for /unstable/notifications.

* Rename to UpsertPusher for clarity and handle pusher update

* Fix linter errors

* Ignore body.Close() error check

* Fix push server internal http wiring

* Add 40 newly passing 61push tests to whitelist

* Add next 12 newly passing 61push tests to whitelist

* Send notification data before notifying users in EDU server consumer

* NATS JetStream

* Goodbye sarama

* Fix `NewStreamTokenFromString`

* Consume on the correct topic for the roomserver

* Don't panic, NAK instead

* Move push notifications into the User API

* Don't set null values since that apparently causes Element upsetti

* Also set omitempty on conditions

* Fix bug so that we don't override the push rules unnecessarily

* Tweak defaults

* Update defaults

* More tweaks

* Move `/notifications` onto `r0`/`v3` mux

* User API will consume events and read/fully read markers from the sync API with stream positions, instead of consuming directly

Co-authored-by: Piotr Kozimor <p1996k@gmail.com>
Co-authored-by: Tommie Gannert <tommie@gannert.se>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
This commit is contained in:
Dan 2022-03-03 13:40:53 +02:00 committed by GitHub
parent 111f01ddc8
commit f05ce478f0
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
92 changed files with 5840 additions and 194 deletions

View file

@ -339,17 +339,17 @@ Existing members see new members' join events
Inbound federation can receive events
Inbound federation can receive redacted events
Can logout current device
Can send a message directly to a device using PUT /sendToDevice
Can recv a device message using /sync
Can recv device messages until they are acknowledged
Device messages with the same txn_id are deduplicated
Device messages wake up /sync
Can recv device messages over federation
Device messages over federation wake up /sync
Can send messages with a wildcard device id
Can send messages with a wildcard device id to two devices
Wildcard device messages wake up /sync
Wildcard device messages over federation wake up /sync
Can send a message directly to a device using PUT /sendToDevice
Can recv a device message using /sync
Can recv device messages until they are acknowledged
Device messages with the same txn_id are deduplicated
Device messages wake up /sync
Can recv device messages over federation
Device messages over federation wake up /sync
Can send messages with a wildcard device id
Can send messages with a wildcard device id to two devices
Wildcard device messages wake up /sync
Wildcard device messages over federation wake up /sync
Can send a to-device message to two users which both receive it using /sync
User can create and send/receive messages in a room with version 6
local user can join room with version 6
@ -477,7 +477,7 @@ Federation key API can act as a notary server via a GET request
Inbound /make_join rejects attempts to join rooms where all users have left
Inbound federation rejects invites which include invalid JSON for room version 6
Inbound federation rejects invite rejections which include invalid JSON for room version 6
GET /capabilities is present and well formed for registered user
GET /capabilities is present and well formed for registered user
m.room.history_visibility == "joined" allows/forbids appropriately for Guest users
m.room.history_visibility == "joined" allows/forbids appropriately for Real users
POST rejects invalid utf-8 in JSON
@ -588,6 +588,59 @@ User can invite remote user to room with version 9
Remote user can backfill in a room with version 9
Can reject invites over federation for rooms with version 9
Can receive redactions from regular users over federation in room version 9
Pushers created with a different access token are deleted on password change
Pushers created with a the same access token are not deleted on password change
Can fetch a user's pushers
Can add global push rule for room
Can add global push rule for sender
Can add global push rule for content
Can add global push rule for override
Can add global push rule for underride
Can add global push rule for content
New rules appear before old rules by default
Can add global push rule before an existing rule
Can add global push rule after an existing rule
Can delete a push rule
Can disable a push rule
Adding the same push rule twice is idempotent
Can change the actions of default rules
Can change the actions of a user specified rule
Adding a push rule wakes up an incremental /sync
Disabling a push rule wakes up an incremental /sync
Enabling a push rule wakes up an incremental /sync
Setting actions for a push rule wakes up an incremental /sync
Can enable/disable default rules
Trying to add push rule with missing template fails with 400
Trying to add push rule with missing rule_id fails with 400
Trying to add push rule with empty rule_id fails with 400
Trying to add push rule with invalid template fails with 400
Trying to add push rule with rule_id with slashes fails with 400
Trying to add push rule with override rule without conditions fails with 400
Trying to add push rule with underride rule without conditions fails with 400
Trying to add push rule with condition without kind fails with 400
Trying to add push rule with content rule without pattern fails with 400
Trying to add push rule with no actions fails with 400
Trying to add push rule with invalid action fails with 400
Trying to add push rule with invalid attr fails with 400
Trying to add push rule with invalid value for enabled fails with 400
Trying to get push rules with no trailing slash fails with 400
Trying to get push rules with scope without trailing slash fails with 400
Trying to get push rules with template without tailing slash fails with 400
Trying to get push rules with unknown scope fails with 400
Trying to get push rules with unknown template fails with 400
Trying to get push rules with unknown attribute fails with 400
Getting push rules doesn't corrupt the cache SYN-390
Test that a message is pushed
Invites are pushed
Rooms with names are correctly named in pushes
Rooms with canonical alias are correctly named in pushed
Rooms with many users are correctly pushed
Don't get pushed for rooms you've muted
Rejected events are not pushed
Test that rejected pushers are removed.
Notifications can be viewed with GET /notifications
Trying to add push rule with no scope fails with 400
Trying to add push rule with invalid scope fails with 400
Forward extremities remain so even after the next events are populated as outliers
If a device list update goes missing, the server resyncs on the next one
uploading self-signing key notifies over federation
@ -607,4 +660,4 @@ registration accepts non-ascii passwords
registration with inhibit_login inhibits login
The operation must be consistent through an interactive authentication session
Multiple calls to /sync should not cause 500 errors
/context/ with lazy_load_members filter works