Update README/CONTRIBUTING/sytest for AWSY issues (#1333)

* Update README/CONTRIBUTING/sytest for AWSY issues

* More docs

README.md

@@ -1,8 +1,7 @@
 # Dendrite [![Build Status](https://badge.buildkite.com/4be40938ab19f2bbc4a6c6724517353ee3ec1422e279faf374.svg?branch=master)](https://buildkite.com/matrix-dot-org/dendrite) [![Dendrite Dev on Matrix](https://img.shields.io/matrix/dendrite-dev:matrix.org.svg?label=%23dendrite-dev%3Amatrix.org&logo=matrix&server_fqdn=matrix.org)](https://matrix.to/#/#dendrite-dev:matrix.org) [![Dendrite on Matrix](https://img.shields.io/matrix/dendrite:matrix.org.svg?label=%23dendrite%3Amatrix.org&logo=matrix&server_fqdn=matrix.org)](https://matrix.to/#/#dendrite:matrix.org)
 
 Dendrite is a second-generation Matrix homeserver written in Go. It is not recommended to use Dendrite as
-a production homeserver at this time as there is no stable release. An overview of the design can be found
+a production homeserver at this time as there is no stable release.
-in [DESIGN.md](docs/DESIGN.md).
 
 # Quick start
 
@@ -60,8 +59,33 @@ This means Dendrite supports amongst others:
 
 # Contributing
 
-Everyone is welcome to help out and contribute! See
+We would be grateful for any help on issues marked as
-[CONTRIBUTING.md](docs/CONTRIBUTING.md) to get started!
+[Are We Synapse Yet](https://github.com/matrix-org/dendrite/labels/are-we-synapse-yet). These issues
+all have related Sytests which need to pass in order for the issue to be closed. Once you've written your
+code, you can quickly run Sytest to ensure that the test names are now passing.
+
+For example, if the test `Local device key changes get to remote servers` was marked as failing, find the
+test file (e.g via `grep` or via the
+[CI log output](https://buildkite.com/matrix-dot-org/dendrite/builds/2826#39cff5de-e032-4ad0-ad26-f819e6919c42)
+it's `tests/50federation/40devicelists.pl` ) then to run Sytest:
+```
+docker run --rm --name sytest
+-v "/Users/kegan/github/sytest:/sytest"
+-v "/Users/kegan/github/dendrite:/src"
+-v "/Users/kegan/logs:/logs"
+-v "/Users/kegan/go/:/gopath"
+-e "POSTGRES=1" -e "DENDRITE_TRACE_HTTP=1"
+matrixdotorg/sytest-dendrite:latest tests/50federation/40devicelists.pl
+```
+See [sytest.md](docs/sytest.md) for the full description of these flags.
+
+Sometimes Sytest is testing the wrong thing or is flakey, so it will need to be patched.
+Ask on `#dendrite-dev:matrix.org` if you think this is the case for you and we'll be happy to help.
+
+If you're new to the project, see [CONTRIBUTING.md](docs/CONTRIBUTING.md) to get up to speed then
+look for [Good First Issues](https://github.com/matrix-org/dendrite/labels/good%20first%20issue). If you're
+familiar with the project, look for [Help Wanted](https://github.com/matrix-org/dendrite/labels/help-wanted)
+issues.
 
 # Discussion
 

docs/CONTRIBUTING.md

@@ -12,18 +12,18 @@ See [INSTALL.md](INSTALL.md) for instructions on setting up a running dev
 instance of dendrite, and [CODE_STYLE.md](CODE_STYLE.md) for the code style
 guide.
 
-As of May 2019, we're not using `gb` anymore, which is the tool we had been
+We use [golangci-lint](https://github.com/golangci/golangci-lint) to lint
-using for managing our dependencies. We're now using Go modules. To build
+Dendrite which can be executed via:
-Dendrite, run the `build.sh` script at the root of this repository (which runs
-`go install` under the hood), and to run unit tests, run `go test ./...` (which
-should pick up any unit test and run it). There are also [build/scripts](/build/scripts) for
-[linting](/build/scripts/find-lint.sh) and doing a [build/test/lint
-run](/build/scripts/build-test-lint.sh).
 
-As of February 2020, we are deprecating support for Go 1.11 and Go 1.12 and are
+```
-now targeting Go 1.13 or later. Please ensure that you are using at least Go
+$ golangci-lint run
-1.13 when developing for Dendrite - our CI will lint and run tests against this
+```
-version.
+
+We also have unit tests which we run via:
+
+```
+$ go test ./...
+```
 
 ## Continuous Integration
 
@@ -73,7 +73,7 @@ These should be well-contained, small pieces of work that can be picked up to
 help you get familiar with the code base.
 
 Once you're comfortable with hacking on Dendrite there are issues lablled as
-[help wanted](https://github.com/matrix-org/dendrite/labels/help%20wanted),
+[help wanted](https://github.com/matrix-org/dendrite/labels/help-wanted),
 these are often slightly larger or more complicated pieces of work but are
 hopefully nonetheless fairly well-contained.
 

docs/sytest.md

@@ -10,9 +10,9 @@ passes.
 
 ## Finding out which tests to add
 
-We recommend you run the tests locally by using the SyTest docker image or
+We recommend you run the tests locally by using the SyTest docker image.
-manually setting up SyTest. After running the tests, a script will print the
+After running the tests, a script will print the tests you need to add to
-tests you need to add to `sytest-whitelist`.
+`sytest-whitelist`.
 
 You should proceed after you see no build problems for dendrite after running:
 
@@ -20,8 +20,16 @@ You should proceed after you see no build problems for dendrite after running:
 ./build.sh
 ```
 
+If you are fixing an issue marked with
+[Are We Synapse Yet](https://github.com/matrix-org/dendrite/labels/are-we-synapse-yet)
+then there will be a list of Sytests that you should add to the whitelist when you
+have fixed that issue. This MUST be included in your PR to ensure that the issue
+is fully resolved.
+
 ### Using the SyTest Docker image
 
+**We strongly recommend using the Docker image to run Sytest.**
+
 Use the following commands to pull the latest SyTest image and run the tests:
 
 ```sh
@@ -38,12 +46,29 @@ When debugging, the following Docker `run` options may also be useful:
 * `-v /path/to/sytest/:/sytest/`: Use your local SyTest repository at
   `/path/to/sytest` instead of pulling from GitHub. This is useful when you want
   to speed things up or make modifications to SyTest.
+* `-v "/path/to/gopath/:/gopath"`: Use your local `GOPATH` so you don't need to
+  re-download packages on every run.
 * `--entrypoint bash`: Prevent the container from automatically starting the
   tests.  When used, you need to manually run `/bootstrap.sh dendrite` inside
   the container to start them.
+* `-e "DENDRITE_TRACE_HTTP=1"`: Adds HTTP tracing to server logs.
+* `-e "DENDRITE_TRACE_INTERNAL=1"`: Adds roomserver internal API tracing to
+  server logs.
+* `-e "DENDRITE_TRACE_SQL=1"`: Adds tracing to all SQL statements to server logs.
+
+The docker command also supports a single positional argument for the test file to
+run, so you can run a single `.pl` file rather than the whole test suite. For example:
+```
+docker run --rm --name sytest -v "/Users/kegan/github/sytest:/sytest"
+-v "/Users/kegan/github/dendrite:/src" -v "/Users/kegan/logs:/logs"
+-v "/Users/kegan/go/:/gopath" -e "POSTGRES=1" -e "DENDRITE_TRACE_HTTP=1"
+matrixdotorg/sytest-dendrite:latest tests/50federation/40devicelists.pl
+```
 
 ### Manually Setting up SyTest
 
+**We advise AGAINST using manual SyTest setups.**
+
 If you don't want to use the Docker image, you can also run SyTest by hand. Make
 sure you have Perl 5 or above, and get SyTest with: