Reference Deployment

21 Travel Rule is a software solution for FATF’s Travel Rule by 21 Analytics.

These instructions only work if you have a valid 21 Travel Rule license, which comes with a username and password for OCI registry access.

If you get stuck with the deployment instructions then please refer to the Troubleshooting section where common usage errors are clarified.

Required Tools

To run 21 Travel Rule the following tools needs to be installed:

  • Podman (version 5.6.0 or higher)
  • helm (version 3.8.0 or higher)
  • yq (The Golang version is preferred over the Python one)
  • git
  • systemd (installed by default on all popular Linux distributions)

Any operating system which can run Linux OCI containers will work.

Best-Practise Deployment Strategy

  • Follow the reference deployment closely!
  • Avoid changes to the architecture.
  • Avoid pre-mature optimisations. E.g. splitting the service tables into multiple databases instead of multiple schemas won't work.

Deployment Checklist

The deployment checklist is a guideline for DevOps teams to keep track of their deployment and configuration progress:

  • Successfully logged in into the 21 Analytics OCI registry
  • Successfully deployed the container services as described below
  • Verified that the container services don't crash but run without restarts
  • Successfully logged in as admin into Keycloak
  • Successfully created first user in Keycloak
  • Successfully logged in with created user into Compliance Dashboard
  • Successfully verified that the REST interfaces of trpd and trustd are publicly accessible (required for VASP to VASP messaging)
  • Successfully verified that the GraphQL interface of aopd is publicly accessible (required for AOPP Portal)
  • Successfully verified that the static files served by the aopp-portal container are publicly accessible (required for AOPP Portal)
  • Successfully configured all required modules (TRUST, TRP, EMAIL, AOPP) using the Compliance Dashboard user interface
  • Successfully opened the AOPP Portal user interface and observed the "Invalid AOPP Portal Link" error message. Only links created using the Compliance Dashboard are valid.

After the steps above have been fully completed, continue with the checklist for functional testing:

  • Successfully created a test customer with a Bitcoin account in the Travel Address Book
  • Successfully sent an outgoing TRP notification to Henry Foster using Bitcoin to Travel Address ta24asGLrECMNQPtNt6SoMsgBHkg6zD7pFkkMHp9TRFYEuTs9GXTn1xDDZ65hQXbk3hoMQm6HKK7UozDvTkeWZeT1NtzoXTb42mS7PB4CyNYRsAg
  • Successfully requested an AOPP Proof and opened the returned Portal link displaying different Wallets
  • Successfully sent an outgoing Encrypted Email notification to a "Test VASP" configured with an email address whose inbox can be accessed by you
  • Successfully received the Encrypted Email
  • Successfully replied to the Encrypted Email (i.e. the "FROM" email address was configured without typos)

Deployment with Podman Quadlet

Clone Git Repository

First, clone the deployment git repository with

git clone https://gitlab.com/21analytics/21-travel-deployment.git

Log into Container Registry

Second, log in to the container registry using the username / password that you have obtained from us by executing the following two commands:

podman login registry.21analytics.ch -u YourUsername
helm registry login registry.21analytics.ch -u YourUsername

Edit Minimum Configuration

Then edit values.yaml which is the minimum configuration required for deployment.

You probably want to commit your changes to values.yaml to simplify upgrades later:

git add values.yaml
git commit -m "values.yaml: save configuration"

Generate Deployment YAML Files

The YAML files declaring the entire deployment can be generated with the following command.

helm template prod oci://registry.21analytics.ch/21analytics/twenty-one-travel-rule -f values.yaml > quadlet/travel.yaml

Extract Secrets

Run the interactive extract_secrets.sh script to obtain and backup auto-generated secrets. NOTE: losing the secrets will cut-off your access to all productive data.

./extract_secrets.sh

After that, you can spin up the services as shown in the next subsection.

Start 21 Travel Platform

The following commands will register the 21 Travel platform to be managed by systemd and instruct it to start the platform.

podman quadlet install ./quadlet
systemctl daemon-reload
systemctl start travel.service

Allow approximately three minutes for all services to start up.

User Management

The reference deployment uses a Keycloak service to manage user accounts. Please refer to the Keycloak page for more information.

Logging

All our services emit log messages. The log level can be adjusted by setting the corresponding settings in values.yaml. Starting with the least verbose level the available log levels are:

  1. error
  2. warn
  3. info
  4. debug
  5. trace

where info is the default log level.

Furthermore, it is possible to selectively adjust the logging level for certain modules only (the module names can be obtained from existing logging output), e.g. to increase the logging level for HTTP traffic tower_http::trace=debug should be set.

Compliance Dashboard and AOPP Portal

The Compliance Dashboard and AOPP Portal can be accessed on the domains configured in values.yaml. Both need to be served from the root path /.

Upgrade

For an upgrade from version 8.2.0 to version 8.3.0 please refer to the upgrade guide.

APIs Exposed by Reverse Proxy

Here, we document the API endpoints that require to be publicly accessible. This is meant as a reference for firewall and WAF (Web Application Firewall) configuration.

TRUST:
443 (HTTPS) unterminated TLS Incoming as well as TCP Outgoing. This has to be accessible for your counterparty VASPs.

Travel Rule Protocol (TRP):
443 (HTTPS) at /transfers and /transfers/
TCP Incoming and Outgoing. This has to be accessible for your counterparty VASPs.

AOPP
443 (HTTPS) at /proofs/ TCP, incoming only. This has to be accessible by your customers.

Working with OpenShift/Kubernetes

The helm charts found at oci://registry.21analytics.ch/21analytics/twenty-one-travel-rule offer a good starting point for various Kubernetes deployments. Consult the next sections to get hints on customisation.

Configuring Your Ingress Proxy

You very likely have an ingress proxy running in your cluster already. HAProxy, which is used as an ingress proxy in the reference deployment, can be disabled in values.yaml. You can extract its configuration from the ConfigMap and easily port it to any ingress proxy, e.g. by using AI tools.

Using an Existing Postgres Database

The init_db.sh script, found in the ConfigMap, can be inspected to extract the required configuration for Postgres (users, passwords, schemas, permissions). As a consequence the database connection URLs that are passed to the 21 Travel Rule services need to be changed.

TLS Certificates

TLS certificates are automatically set up using the ACME protocol. The only pre-requisite is to configure your domains in the values.yaml after you have registered them with your domain name registrar.