Configuration

Changelog (last updated v2.1)

v2.1: multi-database support

databases was introduced in v2.1.

Prior to that, the log and storage keys were at the top-level of the configuration:

log: !Local
  path: /path/to/log-file

storage: !Local
  path: /path/to/storage-dir

# became

databases:
  xtdb:
    log: !Local
      path: /path/to/log-file

    storage: !Local
      path: /path/to/storage-dir

For more details on the changes to the log and storage configurations, see the Transaction Logs and Object Storage documentation.

XTDB nodes are configured using YAML files.

The two main pluggable components of XTDB are transaction logs and object storage - these can be configured in the databases section of the configuration file.

databases:
  # Currently only one database is supported, named `xtdb`.
  xtdb:
    # -- optional

    # transaction log configuration
    # defaults to an in-memory transaction log
    log: !Local
      path: /path/to/log-file

    # object store configuration
    # defaults to an in-memory object store
    storage: !Local
      path: /path/to/storage-dir

If no databases section is specified, XTDB will use the default configuration - an in-memory transaction log and an in-memory object store.

For more details on the log and storage configurations, including the available components, see the Transaction Logs and Object Storage documentation.

Using !Env

For certain keys, we allow the use of environment variables - typically, the keys where we allow this are things that may change location across environments. Generally, they are either "paths" or "strings".

When specifying a key, you can use the !Env tag to reference an environment variable. As an example:

databases:
  xtdb:
    storage: !Local
      path: !Env XTDB_STORAGE_PATH

Any key that we allow the use of !Env will be documented as such.

Monitoring & Observability

XTDB provides a suite of tools & templates to facilitate monitoring and observability. See Monitoring & Observability.

Authentication

The pg-wire server and the http-server both support authentication which can be configured via authentication rules. See Authentication.

Other configuration:

XTDB nodes accept other optional configuration, as follows:

server:
  # Host on which to start a read-write Postgres wire-compatible server.
  #
  # Default is "localhost", which means the server will only accept connections on the loopback interface.
  # Set to '*' to accept connections on all interfaces.
  host: localhost

  # Port on which to start a read-write Postgres wire-compatible server.
  #
  # Default is 0, to have the server choose an available port.
  # (In the XTDB Docker images, this is defaulted to 5432.)
  # Set to -1 to not start a read-write server.
  port: 0

  # Port on which to start a read-only Postgres wire-compatible server.
  #
  # The server on this port will reject any attempted DML/DDL,
  # regardless of whether the user would otherwise have the permission to do so.
  #
  # Default is -1, to not start a read-only server.
  # Set to 0 to have the server choose an available port.
  readOnlyPort: -1

compactor:
  # Number of threads to use for compaction.

  # Defaults to min(availableProcessors / 2, 1).
  # Set to 0 to disable the compactor.
  threads: 4

modules:
  # You can enable optional modules by specifying them under the `modules` key.
  # For example:
  - !HttpServer
    # The HTTP server will listen on the specified port.
    # Default is 3000.
    port: 3000

For details on the available modules, see the modules documentation.

CLI flags

The following flags can be passed, either directly to the CLI or via Docker’s arguments:

• -f <file>, --file <file>: specifies the configuration file to use.

• --compactor-only: runs a compactor-only node.