How to Setup your Matrix Synapse Messaging Homeserver (Apr 2020/English)

Feel free to link to this page if you like this blog post, but do not copy/paste the text somewhere else without asking. If you want to post the text in full or partial on your site or blog you must add in bold inside a h1 tag:

This is the server side part, find client intro here:

This is the server side part. If you need a tutorial for the client side using Riot messenger I wrote an very detailed english version for my https://EURAFRI.com Business Networking Project! If you need more details than shown here please visit https://eurafri.com/info-whatsinside/ and choose the Riot area. In German you can find infos here: https://kmj.at/riot-im-messenger-kurzanleitung-fuer-benutze-updated/

FreeBSD PostgresQL Update 2019-10-30:

Be careful with updating, see updating section below. FreeBSD pkg wants PostgreSQL 11.5 now for py36-psycopg2 and turnserver. You can end up with deinstalled PostgreSQL 9.5 Server and create a problem. Read section below carefully!

Security Fix 2020-04-03:

The Twisted fix was already mentioned in the 1.12.0 release note, but due to a sqlite module load error, even one does not use it, updates was delayed till 1.12.1 comes out. Please upgrade Twisted to >=20.3.0 while env is activated using:

pip install 'Twisted>=20.3.0'

Cross Signing Update 2020-05-06


Just to be clear

This is how I have done it. Whatever you do, you do it for yourself. I am in no way responsible for anything. Everything you do is on your own risk!

Public Matrix Room

There is a public room #kmj:matrix.ctseuro.com where I answer questions from time to time in English or German. Please join if you have questions.

A guide to setup a Matrix Synapse Homeserver

End-to-end encrypted messenger and collaboration server using Python 3.7. We will add Turnserver for voice based communicatioa, add a firewall and more. If we are done you run your own homeserver which is able to communicate with all other matrix servers on the Internet via federation. Prefered Client on Linux, Mac, Windows, Android and IOS is https://riot.im .

At the time of writing this age we use FreeBSD 12.1. Newer FreeBSD Versions are very similar to setup.

prepare everything

For bare bone setup write the ISO to some USB stick, for virtual setup using Proxmox or VMWare, place the ISO on your host and create a virtual machine booting with the ISO file.

The required resources are:

  • RAM 4-6 GB for up to 100 users
  • 2-3 Cores
  • 80GB Harddisk
  • network card
  • static IP address with DNS entry pointing to it

Assume we create the homeserver matrix.ourdomain.com on IP 1.2.3.4. Each Matrix Synapse server is called homeserver. We have pointed an DNS A record matrix.ourdomain.com to IP 1.2.3.4. Check with nslookup matrix.ourdomain.com, or ping matrix.ourdomain.com that the name resolves.

NOTE: you must change ourdomain.com to your domain name and 1.2.3.4 to your IP!

Setup FreeBSD

Boot the ISO file and do a minimum setup of FreeBSD as described on FreeBSD.org. Enable sshd to be able to login. This sets the hostname, creates your user account, formats the harddisk and creates the partitions. Do not install ports or other stuff, except sshd. Finally reboot and login with the user you created. Detailed FreeBSD setup instructions is out of the scope of this document.

Setup additional software

use su to become root and enter y if asked to fetch and install pkg system.

  • pkg install ca_root_nss gcc mc bash curl wget
  • pkg install libffi libxslt jpeg
  • pkg install py37-virtualenv
  • pkg install py37-psycopg2
  • pkg clean -a

Create Environment

  • mkdir -p /home/synapse/env
  • virtualenv-3.7 -p python3.7 /home/synapse/env

Always use sh for working with virtualenv.

sh
. /home/synapse/env/bin/activate
(env) #

The env # must be shown and do not forget to add the . at the beginning of the line. If it is not shown you have made some error! Go back and recheck if this happens!

Setup basic Synapse inside virtualenv

While still showing (env) # do:

- pip install --upgrade pip
- pip install --upgrade setuptools

This should show:

 . /home/synapse/env/bin/activate
(env) # pip install --upgrade pip
Requirement already up-to-date: pip in /usr/home/synapse/env/lib/python3.7/site-packages (19.1.1)
(env) # pip install --upgrade setuptools
Requirement already up-to-date: setuptools in /usr/home/synapse/env/lib/python3.7/site-packages (41.0.1)

If this is fine, continue with:

  • pip install matrix-synapse[all]

This will take some time, download different things and should finally end without error.

Setup PostgreSQL Database

changed from 9.5 to 11.5 on 20191030

  • leave virtualenv with typing: deactivate
  • you should only see # without env now
  • pkg install postgresql11-server
  • sysrc postgresql_enable=yes
  • service postgresql initdb
  • service postgresql start

now we have a running PostgreSQL database engine.

Setup the Firewall, DNS, NTP and Mail.

Before we continue we should setup our firewall to avoid attacks while runnign our setup.

Create /etc/rc.firewall.KMJ with the following content:

#!/bin/sh -
#
# Setup system for ipfw(4) firewall service.
#

# Suck in the configuration variables.
if [ -z "${source_rc_confs_defined}" ]; then
        if [ -r /etc/defaults/rc.conf ]; then
                . /etc/defaults/rc.conf
                source_rc_confs
        elif [ -r /etc/rc.conf ]; then
                . /etc/rc.conf
        fi
fi

############

setup_loopback() {
        ############
        # Only in rare cases do you want to change these rules
        #
        ${fwcmd} add 100 pass all from any to any via lo0
        ${fwcmd} add 200 deny all from any to 127.0.0.0/8
        ${fwcmd} add 300 deny ip from 127.0.0.0/8 to any
        ${fwcmd} add 400 deny all from any to ::1
        ${fwcmd} add 500 deny all from ::1 to any
}

. /etc/rc.subr
. /etc/network.subr

############
# Set quiet mode if requested
#
case ${firewall_quiet} in
[Yy][Ee][Ss])
        fwcmd="/sbin/ipfw -q"
        ;;
*)
        fwcmd="/sbin/ipfw"
        ;;
esac

############
# Flush out the list before we begin.
#
${fwcmd} -f flush
setup_loopback

############
#
# We do not allow IPv6 Traffic
${fwcmd} add deny  ip6 from any to any

# Allow TCP through if setup succeeded
${fwcmd} add pass tcp from any to any established

# Allow IP fragments to pass through
${fwcmd} add pass all from any to any frag

# Allow setup of incoming connections
         
${fwcmd} add pass tcp from ${MGMTIP} to me 22 setup     # SSH

${fwcmd} add pass tcp from any to me 8448 setup         # MATRIX
${fwcmd} add pass tcp from any to me 3478 setup         # TURN
${fwcmd} add pass udp from any to me 3478 keep-state    # TURN
${fwcmd} add pass tcp from any to me 5349 setup         # TURN TLS
${fwcmd} add pass udp from any to me 5349 keep-state    # TURN TLS

${fwcmd} add pass tcp from any to me 80 setup           # letsencrypt only
${fwcmd} add pass tcp from any to me 443 setup          # https rev proxy fuer matrix

# Allow setup of outgoing TCP connections only
${fwcmd} add pass tcp from me to any setup

# Disallow setup of all other TCP connections
${fwcmd} add deny tcp from any to any setup

# Allow DNS queries out in the world
${fwcmd} add pass udp from me to any 53 keep-state

# Allow NTP queries out in the world
${fwcmd} add pass udp from me to any 123 keep-state

# Deny the Rest
${fwcmd} add deny all from any to any

Assuming you have an IPv4 connectivity only we block all IPv6 traffic. Furthermore it is not recommended to open SSH to the public. I assume you are a good administrator and restrict ssh access to the static IP of your notebook or desktop system. Otherwise you should setup OpenVPN to access your server. Do not use any as MGMTIP. If you open critial services to the public you will regret it later. They will brute force you and one day somebody will break into your system. Be smart and work like a professional!

Now add or change the settings in /etc/rc.conf. We already add settings we need later. You network configuration is not shown here and should not be touched. Do not change

  • hostname=
  • ifconfig_
  • defaultrouter=

parameters.

keymap="de.kbd" # do not change your keymap. we use german
clear_tmp_enable="YES"
syslogd_flags="-ss"
dumpdev="NO"

MGMTIP="5.6.8.9" # change to static IP of your notebook or desktop system
 
firewall_enable="YES"           # Set to YES to enable firewall functionality
firewall_script="/etc/rc.firewall.KMJ" # Which script to run to set up the firewall
firewall_type="KMJ"         # Firewall type (see /etc/rc.firewall)
firewall_quiet="NO"             # Set to YES to suppress rule display
firewall_logging="YES"          # Set to YES to enable events logging
firewall_logif="NO"             # Set to YES to create logging-pseudo interface

#######################
 
local_unbound_enable="YES"
sshd_enable="YES"
ntpd_enable="YES"

apache24_enable="YES"
postgresql_enable="YES"

synapse_enable="YES"
turnserver_enable="YES"

###############################

now change /etc/resolv.conf so it reads:

search ourdomain.com
nameserver 127.0.0.1
options edns0

The final setting requires that our server is able to send out e-mails.

  • cd /etc/mail
  • make
  • make install
  • edit /etc/mail/aliases and uncomment the root: entry.
  • change it to root: you@yourmaildomain.com
  • run newaliases

Using this settings we run a local cache DNS and resolve locally. Furthermore we run a time syncing via NTP daemon. To make sure everything is done OK, we should reboot now.

  • shutdown -r now

Check our Setup

  • login after reboot
  • do a ps -xaj and check that local-ubound, syslogd, ntpd and postgresql deamons are running
  • do a mail -s “test” root, enter some text+ and press CTRL+d to send. You should receive your root mail. Check /var/log/maillog if not,

Setup Turnserver

To handle voice calls behind NAT you should setup Turnserver as kind of proxy. Setup is pretty easy.

  • pkg install turnserver
  • create /usr/local/etc/turnserver.conf like below, but change your IP, domain and secret!
realm=matrix.ourdomain.com
listening-ip=1.2.3.4
#lt-cred-mech
use-auth-secret
static-auth-secret=1cd6c96c1f8d4f1008042d76d646691e3b0512e830d7f486d6ad6b702bfc234e
no-tcp-relay
user-quota=12 # 4 streams per video call, so 12 streams = 3 simultaneous relayed calls per user.
total-quota=1200

cert=/usr/local/etc/matrix-synapse/certs/matrix.ourdomain.com/cert.pem
pkey=/usr/local/etc/matrix-synapse/certs/matrix.ourdomain.com/privkey.pem

do not start turnserver now!

Setup Apache Reversed-Proxy and SSL Cert

We want Apache to handle the connections to https Port 443 from the clients. Federation is done via port 8448. So we need to setup an Apache Reversed-Proxy and get a free SSL Cert from Letsencrypt.

  • pkg install apache24 cronolog
  • we already added apache24_enable=“YES” to /etc/rc.conf earlier
  • save /usr/local/etc/apache24/httpd.conf and replace its content with
  • replace ourdomain.com with your domain
ServerRoot "/usr/local"

ServerSignature Off

Listen 80

LoadModule authn_file_module libexec/apache24/mod_authn_file.so
LoadModule authn_core_module libexec/apache24/mod_authn_core.so
LoadModule authz_host_module libexec/apache24/mod_authz_host.so
LoadModule authz_groupfile_module libexec/apache24/mod_authz_groupfile.so
LoadModule authz_user_module libexec/apache24/mod_authz_user.so
LoadModule authz_core_module libexec/apache24/mod_authz_core.so
LoadModule access_compat_module libexec/apache24/mod_access_compat.so
LoadModule auth_basic_module libexec/apache24/mod_auth_basic.so
LoadModule socache_shmcb_module libexec/apache24/mod_socache_shmcb.so
LoadModule reqtimeout_module libexec/apache24/mod_reqtimeout.so
LoadModule filter_module libexec/apache24/mod_filter.so
LoadModule mime_module libexec/apache24/mod_mime.so
LoadModule log_config_module libexec/apache24/mod_log_config.so
LoadModule env_module libexec/apache24/mod_env.so
LoadModule headers_module libexec/apache24/mod_headers.so
LoadModule setenvif_module libexec/apache24/mod_setenvif.so
LoadModule version_module libexec/apache24/mod_version.so
LoadModule ssl_module libexec/apache24/mod_ssl.so
LoadModule mpm_prefork_module libexec/apache24/mod_mpm_prefork.so
LoadModule unixd_module libexec/apache24/mod_unixd.so
LoadModule autoindex_module libexec/apache24/mod_autoindex.so
<IfModule !mpm_prefork_module>
        #LoadModule cgid_module libexec/apache24/mod_cgid.so
</IfModule>
<IfModule mpm_prefork_module>
        #LoadModule cgi_module libexec/apache24/mod_cgi.so
</IfModule>
LoadModule dir_module libexec/apache24/mod_dir.so
LoadModule userdir_module libexec/apache24/mod_userdir.so
LoadModule alias_module libexec/apache24/mod_alias.so
LoadModule rewrite_module libexec/apache24/mod_rewrite.so

LoadModule proxy_module libexec/apache24/mod_proxy.so
LoadModule proxy_connect_module libexec/apache24/mod_proxy_connect.so
LoadModule proxy_http_module libexec/apache24/mod_proxy_http.so
IncludeOptional etc/apache24/modules.d/[0-9][0-9][0-9]_*.conf
<IfModule unixd_module>
    User www
    Group www
</IfModule>

ServerAdmin hostmaster@ourdomain.com
ServerName matrix.ourdomain.com:80

DirectoryIndex index.html

<Directory />
    AllowOverride none
    Require all denied
</Directory>

DocumentRoot "/usr/local/www/apache24/data"
<Directory "/usr/local/www/apache24/data">
    Options -Indexes +FollowSymLinks -ExecCGI -MultiViews
    AllowOverride None
    Require all granted
</Directory>

<Files ".ht*">
    Require all denied
</Files>

ErrorLog  "|/usr/local/sbin/cronolog /var/log/www/%Y/%m/%d/errors.log"
LogLevel warn

LogFormat "%v %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i port:%p\"" combined
CustomLog "|/usr/local/sbin/cronolog /var/log/www/%Y/%m/%d/access.log" combined

<IfModule mime_module>
    #
    TypesConfig etc/apache24/mime.types
    #AddType application/x-gzip .tgz
    #AddEncoding x-compress .Z
    #AddEncoding x-gzip .gz .tgz
    AddType application/x-compress .Z
    AddType application/x-gzip .gz .tgz
</IfModule>

Include etc/apache24/extra/httpd-ssl.conf

SSLRandomSeed startup builtin
SSLRandomSeed connect builtin

Include etc/apache24/Includes/*.conf

then save /usr/local/etc/apache24/extra/httpd-ssl.conf and replace it with the following content. Again replace IP and ourdomain.com with yours.

lISTEN 443

SSLCipherSuite HIGH:MEDIUM:!SSLv3:!kRSA
SSLProxyCipherSuite HIGH:MEDIUM:!SSLv3:!kRSA

SSLHonorCipherOrder on

SSLProtocol all -SSLv3
SSLProxyProtocol all -SSLv3

SSLPassPhraseDialog  builtin

SSLSessionCache        "shmcb:/var/run/ssl_scache(512000)"
SSLSessionCacheTimeout  300

<VirtualHost _default_:443>

ServerName matrix.ourdomain.com:443
ServerAdmin hostmaster@ourdomain.com

DocumentRoot "/usr/local/www/apache24/data-ssl"
SSLEngine on

SSLCertificateFile /usr/home/letsencrypt/cts/conf-ip1/certs/matrix.ourdomain.com/cert.pem
SSLCertificateKeyFile /usr/home/letsencrypt/cts/conf-ip1/certs/matrix.ourdomain.com/privkey.pem
SSLCertificateChainFile //usr/home/letsencrypt/cts/conf-ip1/certs/matrix.ourdomain.com/fullchain.pem

UserDir disabled
DirectoryIndex index.html index.htm


<Directory /usr/local/www/apache24/data-ssl>
    Options -Indexes -FollowSymLinks -ExecCGI -MultiViews
    AllowOverride None
    require all granted
</Directory>

################################################################################
# Matrix Synapse
################################################################################
# do NOT change 127.0.0.1 to your IP !!!!!!
ProxyPass /_matrix http://127.0.0.1:8008/_matrix
ProxyPassReverse /_matrix http://127.0.0.1:8008/_matrix

################################################################################

BrowserMatch "MSIE [2-5]" \
         nokeepalive ssl-unclean-shutdown \
         downgrade-1.0 force-response-1.0

LogFormat "%v %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i port:%p\"" combined
CustomLog "|/usr/local/sbin/cronolog /var/log/www/%Y/%m/%d/access.log" combined

</VirtualHost>

Now create the SSL directory:

  • mkdir /usr/local/www/apache24/data-ssl
  • touch /usr/local/www/apache24/data-ssl/index.html
  • echo “Server is offline” > /usr/local/www/apache24/data-ssl/index.html

NOTE

To be able to receive our cert from Letsenrypt, temporary change

  • Include etc/apache24/extra/httpd-ssl.conf

to

  • #Include etc/apache24/extra/httpd-ssl.conf

in httpd.conf. This must be changed back after we received the initial cert.

Now create the Letsencrypt stuff

  • mkdir -p /home/letsencrypt/cts
  • mkdir -p /usr/local/etc/matrix-synapse/certs
  • create /home/letsencrypt/cts/Run_Certs.sh with the following content
#!/bin/sh

cd /home/letsencrypt/cts
chmod 750 dehydrated
/home/letsencrypt/cts/dehydrated  -c  --config /home/letsencrypt/cts/conf-ip1/config.sh \
                                      --hook /home/letsencrypt/cts/conf-ip1/hook.sh

/usr/sbin/service apache24 restart

########################################################################

HOST=`hostname`
echo "copy cert for synapse / matrix server on ${HOST}"

/bin/cp -r /usr/home/letsencrypt/cts/conf-ip1/certs/${HOST} /usr/local/etc/matrix-synapse/certs
/usr/sbin/chown -R root:nobody /usr/local/etc/matrix-synapse/certs
/bin/chmod -R 755 /usr/local/etc/matrix-synapse/certs

/usr/sbin/service synapse restart

/bin/ps -xaj | /usr/bin/grep synapse
########################################################################
  • chown root:wheel /home/letsencrypt/cts/Run_Certs.sh
  • chmod 750 /home/letsencrypt/cts/Run_Certs.sh

  • create /home/letsencrypt/cts/dehydrated with the following commands:

  • cd /home/letsencrypt/cts/

  • wget https://github.com/lukas2511/dehydrated/raw/master/dehydrated

  • chown root:wheel /home/letsencrypt/cts/dehydrated

  • chmod 750 /home/letsencrypt/cts/dehydrated

We now need to create some additional stuff for our Letsencrypt script. Remember to change IP and ourdomain.com to yours!

  • mkdir -p /home/letsencrypt/cts/conf-ip1
  • mkdir -p /usr/home/letsencrypt/cts/www/wellknown/acme-challenge
  • touch /usr/home/letsencrypt/cts/www/wellknown/acme-challenge/index.html
  • ln -s /usr/home/letsencrypt/cts/www/wellknown /usr/local/www/apache24/data/.well-known
  • echo “matrix.ourdomain.com” > /home/letsencrypt/cts/conf-ip1/domains.txt

  • create /home/letsencrypt/cts/conf-ip1/config.sh with the following content:

#!/usr/bin/env /usr/local/bin/bash
# FreeBSD special:
alias openssl='/usr/bin/openssl'
# Path to license agreement (default: https://letsencrypt.org/documents/LE-SA-v1.0.1-July-27-2015.pdf)
#LICENSE="https://letsencrypt.org/documents/LE-SA-v1.0.1-July-27-2015.pdf"
# Which challenge should be used? Currently http-01 and dns-01 are supported
CHALLENGETYPE="http-01"
#WELLKNOWN="${BASEDIR}/.acme-challenges"
WELLKNOWN="/home/letsencrypt/cts/www/wellknown/acme-challenge"
  • chown root:wheel /home/letsencrypt/cts/conf-ip1/config.sh
  • chmod 750 /home/letsencrypt/cts/conf-ip1/config.sh

  • create /home/letsencrypt/cts/conf-ip1/hook.sh with the following command:

cd /home/letsencrypt/cts/conf-ip1/
wget https://github.com/lukas2511/dehydrated/raw/master/docs/examples/hook.sh
  • chown root:wheel /home/letsencrypt/cts/conf-ip1/hook.sh
  • chmod 750 /home/letsencrypt/cts/conf-ip1/hook.sh

to receive our first cert we should start Apache and accept the terms in advance.

  • service apache24 start

  • cd /home/letsencrypt/cts

  • chmod 750 dehydrated

  • /home/letsencrypt/cts/dehydrated –register –accept-terms –config /home/letsencrypt/cts/conf-ip1/config.sh

  • ./Run_Certs.sh

if all goes well you should have placed your certs in

  • /home/letsencrypt/cts/conf-ip1/certs

and

  • /usr/local/etc/matrix-synapse/certs

Add the following to /etc/crontab:

#######################################################################
# Run letsencrypt renewals every thuesday @ 9:01
#######################################################################

5       9       *       *       2       root    /home/letsencrypt/cts/Run_Certs.sh

Now where we have the correct cert, change back

  • #Include etc/apache24/extra/httpd-ssl.conf

to

  • Include etc/apache24/extra/httpd-ssl.conf

in /usr/local/etc/apache24/httpd.conf and

  • service apache24 restart

Point your browser to https://matrix.ourdomain.com and check if you get a secure connection. Logs are in /var/log/www.

We have done most of our way. Lets finalize everything.

Finish PostgreSQL Setup

su to user postgres and create synape user and database.su - postgres

su - postgres
createdb postgres
createuser --pwprompt synapse_user
# check with
psql
\l
\du
\?
CREATE DATABASE synapse ENCODING 'UTF8' LC_COLLATE='C' LC_CTYPE='C'  template=template0  OWNER synapse_user;
\l
\q
exit

Finish Matrix Synapse setup

Add a synapse user, so we do not need to run the homeserver as root.

  • adduser synapse
adduser synapse
Username: synapse
Full name: synapse
Uid (Leave empty for default): 
Login group [synapse]: 
Login group is synapse. Invite synapse into other groups? []: 
Login class [default]: 
Shell (sh csh tcsh bash rbash nologin) [sh]: nologin
Home directory [/home/synapse]: 
Home directory permissions (Leave empty for default): 
Use password-based authentication? [yes]: 
Use an empty password? (yes/no) [no]: 
Use a random password? (yes/no) [no]: yes
Lock out the account after creation? [no]: 
Username   : synapse
Password   : <random>
Full Name  : synapse
Uid        : 1002
Class      : 
Groups     : synapse 
Home       : /nonexistent
Home Mode  : 
Shell      : /usr/sbin/nologin
Locked     : no
OK? (yes/no): yes

Now lets create a base config for our homeserver. Again, do not forget to change ourdomain.com to yours!

- sh
- . /home/synapse/env/bin/activate
- cd /usr/local/etc/matrix-synapse
- python -m synapse.app.homeserver --server-name matrix.ourdomain.com --config-path homeserver.yaml --generate-config --report-stats=no

We need to load a template which is not in the standard setup:

tls_certificate_path: "/usr/local/etc/matrix-synapse/certs/matrix.ourdomain.com/cert.pem"
tls_certificate_path: "/usr/local/etc/matrix-synapse/certs/matrix.ourdomain.com/fullchain.pem"
tls_private_key_path: "/usr/local/etc/matrix-synapse/certs/matrix.ourdomain.com/privkey.pem"

# PEM dh parameters for ephemeral keys
tls_dh_params_path: "/usr/local/etc/matrix-synapse/tls.dh"

# Don't bind to the https port
no_tls: False

tls_fingerprints: []

## Server ##
server_name: "matrix.ourdomain.com"
pid_file: /var/run/matrix-synapse/homeserver.pid
# We don't want the web client
web_client: False

public_baseurl: "https://matrix.ourdomain.com:8448/"

soft_file_limit: 0

listeners:
  -
    port: 8448

    bind_addresses:
      - '1.2.3.4'

    type: http

    tls: true
    x_forwarded: false

    resources:
      -
        names:
          - client     # The client-server APIs, both v1 and v2

        compress: true

      - names: [federation]  # Federation APIs
        compress: false

  - port: 8008
    tls: false
    bind_addresses: ['127.0.0.1']
    type: http

    x_forwarded: false

    resources:
      - names: [client]
        compress: true
      - names: [federation]
        compress: false

# Database configuration
database:
  name: "psycopg2"
  args:
    database: "synapse"
    user: "synapse_user"
    password: "yourpassword"
    cp_min: 5
    cp_max: 10

event_cache_size: "10K"

verbose: 0

#log_file: "/var/log/matrix-synapse/homeserver.log"
log_config: "/usr/local/etc/matrix-synapse/matrix.ourdomain.com.log.config"

rc_messages_per_second: 0.2
rc_message_burst_count: 10.0
federation_rc_window_size: 1000
federation_rc_sleep_limit: 10
federation_rc_sleep_delay: 500
federation_rc_reject_limit: 50
federation_rc_concurrent: 3

media_store_path: "/var/db/matrix-synapse/media_store"
uploads_path: "/var/db/matrix-synapse/uploads"
max_upload_size: "1M"
max_image_pixels: "32M"
dynamic_thumbnails: false

# List of thumbnail to precalculate when an image is uploaded.
thumbnail_sizes:
- width: 32
  height: 32
  method: crop
- width: 96
  height: 96
  method: crop
- width: 320
  height: 240
  method: scale
- width: 640
  height: 480
  method: scale
- width: 800
  height: 600
  method: scale

url_preview_enabled: False

max_spider_size: "1M"
recaptcha_public_key: "DISABLED"
recaptcha_private_key: "DISABLED"
enable_registration_captcha: False

recaptcha_siteverify_api: "https://www.google.com/recaptcha/api/siteverify"

## Turn ##
## CHANGE AS USED above in turnserver.conf!!!!!!!!!

# The public URIs of the TURN server to give to clients
turn_uris: ["turn:matrix.ourdomain.com:3478?transport=udp", "turn:matrix.ourdomain.com:3478?transport=tcp"]
turn_shared_secret: "1cd6c96c1f8d4f1008042d76d646691e3b0512e830d7f486d6ad6b702bfc234e"
turn_user_lifetime: "1h"
turn_allow_guests: False


## Registration ##
enable_registration: False
registration_shared_secret: "jkdshkdjshdfzv9d87v89df77v897df987v89df7v87df89vu89df7v89df7v7df897vsdfuvhfdjkhvdf89"
bcrypt_rounds: 12
allow_guest_access: False

trusted_third_party_id_servers:
    - matrix.org
    - vector.im


enable_metrics: False
report_stats: False

room_invite_state_types:
    - "m.room.join_rules"
    - "m.room.canonical_alias"
    - "m.room.avatar"
    - "m.room.name"


app_service_config_files: []
macaroon_secret_key: "hfdjkfhdjkshjbncnsuivhsdf8sduv8duv8dfuvufd8uv8fdu89"
expire_access_token: False

## Signing Keys ##

# Path to the signing key to sign messages with
signing_key_path: "/usr/local/etc/matrix-synapse/matrix.ourdomain.com.signing.key"

old_signing_keys: {}
key_refresh_interval: "1d" # 1 Day.

# The trusted servers to download signing keys from.
perspectives:
  servers:
    "matrix.org":
      verify_keys:
        "ed25519:auto":
          key: "Noi6WqcDj0QmPxCNQqgezwTlBKrfqehY1u2FyWP9uYw"

# Enable password for login.
password_config:
   enabled: true
   # Uncomment and change to a secret random string for extra security.
   # DO NOT CHANGE THIS AFTER INITIAL SETUP!
   #pepper: ""

email:
   enable_notifs: false
   smtp_host: "localhost"
   smtp_port: 25
   require_transport_security: False
   notif_from: "Ourdomain MATRIX <noreply@ourdomain.com>"
   app_name: Matrix
   template_dir:  /usr/local/etc/matrix-synapse/res/templates
   riot_base_url: "https://matrix.ourdomain.com/riot"


#server_notices:
#   system_mxid_localpart: serveralarm
#   system_mxid_display_name: "Server Infos and Alerts"
#   system_mxid_avatar_url: "https://matrix.ourdomain.com/_matrix/media/v1/thumbnail/matrix.ourdomain.com/fcpNAbnjbfjjHIegqEMoPFew"
#   room_name: "Server Infos and Alerts"

  • mkdir -p /var/run/matrix-synapse
  • chown synapse /var/run/matrix-synapse
  • mkdir -p /var/log/matrix-synapse/
  • chown synapse /var/log/matrix-synapse/
  • change log file in matrix.ourdomain.com.log.config to filename: /var/log/matrix-synapse/homeserver.log
  • mkdir -p /var/db/matrix-synapse/media_store
  • mkdir -p /var/db/matrix-synapse/uploads
  • chown -R synapse:synapse /var/db/matrix-synapse

  • create tls.h file in /usr/local/etc/matrix-synapse

  • cd /usr/local/etc/matrix-synapse/

  • openssl dhparam -out tls.dh 2048

Finally we need a startup script for Matrix Synapse. Lets create /usr/local/etc/rc.d/synapse:

#!/bin/sh
#
# Created by: Karl M. Joch <k.joch@cts.at>
#

# PROVIDE: synapse
# REQUIRE: LOGIN postgresql
# KEYWORD: shutdown

# synapse_enable="YES"

. /etc/rc.subr

#----------------------
# we run in virtual env
#----------------------
VIRTUAL_ENV="/home/synapse/env"
export VIRTUAL_ENV
PATH="$VIRTUAL_ENV/bin:$PATH"
export PATH
# to check run here: env

name=synapse

rcvar=synapse_enable
load_rc_config ${name}

: ${synapse_enable:=NO}
: ${synapse_user:=synapse}
: ${synapse_conf:=/usr/local/etc/matrix-synapse/homeserver.yaml}
: ${synapse_dbdir:=/var/db/matrix-synapse}
: ${synapse_logdir:=/var/log/matrix-synapse}
: ${synapse_pidfile:=/var/run/matrix-synapse/homeserver.pid}

pidfile="${synapse_pidfile}"
procname=/usr/home/synapse/env/bin/python3.7
command=/usr/home/synapse/env/bin/python3.7
command_args="-m synapse.app.homeserver --daemonize -c ${synapse_conf}"
start_precmd=start_precmd

start_precmd()
{
        if [ ! -d ${synapse_pidfile%/*} ] ; then
                install -d -o synapse -g wheel ${synapse_pidfile%/*};
        fi

        if [ ! -d ${synapse_dbdir} ] ; then
                install -d -o synapse -g wheel ${synapse_dbdir};
        fi

        if [ ! -d ${synapse_logdir} ] ; then
                install -d -o synapse -g wheel ${synapse_logdir};
        fi

        if $(grep -q CHANGEME ${synapse_conf}) ; then
                echo "Error: Default secret values in config."
                echo "Please replace the CHANGEME values in ${synapse_conf}"
                exit 1
        fi
}

run_rc_command "$1"
  • chown root:wheel /usr/local/etc/rc.d/synapse
  • chmod 750 /usr/local/etc/rc.d/synapse

Start Matrix Synapse the first time

  • service synapse start

will hopefully end in something like this:

2019-05-11 18:02:13,846 - synapse.server - 222 - INFO - None - Setting up.
2019-05-11 18:02:13,884 - synapse.storage.event_push_actions - 471 - INFO - None - Searching for stream ordering 1 month ago
2019-05-11 18:02:13,885 - synapse.storage.event_push_actions - 477 - INFO - None - Found stream ordering 1 month ago: it's 0
2019-05-11 18:02:13,885 - synapse.storage.event_push_actions - 479 - INFO - None - Searching for stream ordering 1 day ago
2019-05-11 18:02:13,885 - synapse.storage.event_push_actions - 485 - INFO - None - Found stream ordering
2019-05-11 18:02:13,887 - synapse.server - 226 - INFO - None - Finished setting up.

You can check the running synapse with ps -xaj and netstat -an should show you listeners on 8008 and 8448. If yes, you can open the first beer. Your log is in /var/log/matrix-synapse.

Adding the admin user

Again, change ourdomain.com to your domain!!!!!

  • sh
  • . /home/synapse/env/bin/activate
  • cd /usr/local/etc/matrix-synapse
  • register_new_matrix_user -c homeserver.yaml https://matrix.ourdomain.com:8448
  • make this user the admin for you!

  • reboot your server, to restart everything

  • shutdown -r now

  • download the https://riot.im messenger for your platform and login

  • test everything

if everything works as expected:

  • create all other users the same way, but without being admin!

UPDATING

Updating FreeBSD and Packages

  • freebsd-update fetch
  • freebsd-update install
  • pkg update
  • pkg upgrade

Updating Matrix Synapse

  • service synapse stop
  • sh
  • . /home/synapse/env/bin/activate
  • pip install -U matrix-synapse[all]
  • service synapse start
  • if massive update, simply reboot the server

Upgrade notes PostgreSQL 9.5 to 11.5

READ BELOW FOR POSTGRES UPDATE 9.5 to 11.5 if you are below 11.5!!

While upgrading Matrix/Synapse to 1.5.0 a problem with the pkg system occured. The depencies for py37-psycopg2 and turnserver changed to postgresql11-client-11.5 and it ended up with deinstalled postgresql95-server, which was not nice. Thanks to the Proxmox virtualization infrastructure I was able to move back to the snapshot before the update, skipped the pkg upgrade section and put the server online again. So far so good, but now there is need to upgrade to postgresql11-server-11.5 to be on track again.

As written in the documentation (https://www.postgresql.org/docs/11/upgrading.html): “The traditional method for moving data to a new major version is to dump and reload the database, though this can be slow. A faster method is pg_upgrade.” I already had the dump/restore idea because the FreeBSD 11.5 port also changes the location of the servers data.

The steps done to make it work (read 18.6.1 of PostgreSQL docu in advance):

  • deactivate some services on boot in /etc/rc.conf

    apache24_enable=“NO” synapse_enable=“NO” turnserver_enable=“NO”

  • full server shutdown and snapshot creation of Proxmox VM

  • as alternate if bare installed do a full server backup

  • now we need to create a full database dump of the 9.5 server

su
# create temp migration dir
mkdir /var/pgupdate
chmod 777 /var/pgupdate
# change to user pgsql
su - pgsql
cd /var/pgupdate
# backup config changes
cp /usr/local/pgsql/data/pg_hba.conf .
cp /usr/local/pgsql/data/postgresql.conf .
# dump database
pg_dumpall > database_dump_95.sql
# be patient, depending on server this can take time 
exit
# we are root again
service postgresql stop
# rename old data dir
mv /usr/local/pgsql/data /usr/local/pgsql/data-9.5
# delete your postgresql packages / use pkg info first
pkg delete postgresql95-client-9.5.19 postgresql95-server-9.5.19
# this will delete "Installed packages to be REMOVED:"
# postgresql95-client-9.5.19 postgresql95-server-9.5.19
# py36-psycopg2-2.8.3 turnserver-4.5.1.1
# which is ok for us. we reinstall last 2 later
# install server 11 which includes install of client
pkg install postgresql11-server
# initialize the database, run
/usr/local/etc/rc.d/postgresql initdb
# check for changes in pg_hba.conf and postgresql.conf
# best using 
diff -u  /var/pgupdate/pg_hba.conf /var/db/postgres/data11/pg_hba.conf | more
diff -u  /var/pgupdate/postgresql.conf /var/db/postgres/data11/postgresql.conf | more
# reinstall py36-psycopg2 turnserver
pkg install py36-psycopg2 turnserver
# start the postgres server
service postgresql start
# if start runs well, reload the data
su - postgres
psql -d postgres -f /var/pgupdate/database_dump_95.sql
# be patient again!
# if all goes well exit to be root again
exit
# finally upgrade all packages and clean out stuff
pkg update
pkg upgrade
pkg clean -a
# delete old psql data dir and user if all is finished
rm -r /usr/local/pgsql/data
rmuser pgsql
rm -r /var/pgupdate
  • reactivate some services on boot in /etc/rc.conf

    apache24_enable=“YES” synapse_enable=“YES” turnserver_enable=“YES”

# as root start the services and check if all is fine
service apache24 start
service turnserver start
service synapse start

have fun with Riot/Matrix/Synapse again.

NOTE Older versions used user pgsql for PostgreSQL, now user is named postgres and /var/db/postgres is owned in that way!

NOTE from Package The port is set up to use autovacuum for new databases, but you might also want to vacuum and perhaps backup your database regularly. There is a periodic script, /usr/local/etc/periodic/daily/502.pgsql, that you may find useful. You can use it to backup and perform vacuum on all databases nightly. Per default, it performs `vacuum analyze’. See the script for instructions. For autovacuum settings, please review ~pgsql/data/postgresql.conf.

Join my public room

If you like this blog entry feel free to join my public room by entering

/join #kmj:matrix.ctseuro.com

anywhere in the box to send a message! Feedback welcome!

Enterprise Support