Set up multitenancy EARLY ACCESS

Setting up multitenancy has two parts:

1. Prepare a writable, dedicated cgroup for the YB-TServer process (a one-time, environment-specific, operating system step that typically requires root).
2. Enable the resource governor using YB-TServer and YB-Master flags.

Setup assumes YugabyteDB is running as a systemd service.

Prepare cgroups

When multitenancy is enabled, the YB-TServer assumes it is started in a dedicated cgroup (the root cgroup) that it fully manages. The YB-TServer creates and maintains a cgroup hierarchy under this root cgroup, and assigns CPU limits and threads to descendant cgroups.

You must therefore start the YB-TServer inside a cgroup that:

• has the CPU controller available;
• is writable by the user running the YB-TServer process; and
• is a leaf cgroup not managed by any other process (no other process moves processes in or out of it, or creates cgroups underneath it).

Creating this cgroup and granting write access generally requires root privileges (running the YB-TServer afterwards does not). The exact steps depend on your operating system and how you run the YB-TServer.

cgroup v2 with systemd

On most newer Linux distributions, there is a single unified cgroup hierarchy managed by systemd, so you configure delegation through systemd.

If YB-TServer is running as a systemd service under the user slice, make the CPU controller available to the user slice by creating an override for the user@.service template (requires root). Create the file /etc/systemd/system/user@.service.d/override.conf with the following contents:

[Service]
Delegate=pids memory cpu

The YB-TServer service unit must also delegate the CPU controller:

[Service]
Delegate=cpu

If the YB-TServer service unit uses ExecStartPost=, ExecReload=, ExecStop=, or ExecStopPost=, it must also delegate a subgroup so that these commands don't interfere with the YB-TServer's own cgroup:

[Service]
DelegateSubgroup=ybtserver

DelegateSubgroup= requires systemd 254 or later. On earlier systemd versions (for example, AlmaLinux 9), create the subgroup and delegate the CPU controller manually in the service before starting the YB-TServer:

SERVICE_CGROUP="/sys/fs/cgroup$(cut -d: -f3 /proc/self/cgroup)"
SUBGROUP_NAME="$SERVICE_CGROUP/ybtserver"
# Create the subgroup.
mkdir -p "$SUBGROUP_NAME"
# Move this process into the subgroup.
echo $$ > "$SUBGROUP_NAME/cgroup.procs"
# Delegate the CPU controller (only works now that no processes are left in $SERVICE_CGROUP).
echo +cpu > "$SERVICE_CGROUP/cgroup.subtree_control"
# Start yb-tserver in the subgroup.
./yb-tserver ...

If YB-TServer is running as a systemd service under the system slice, the CPU controller is available by default, so the user@.service override is not needed; delegate the CPU controller on the YB-TServer service unit as shown above.

Other deployments

For deployments that don't rely on systemd (such as cron-based or container-based deployments) or for deployments on systems that use cgroups v1

, place the YB-TServer process in a cgroup that meets the requirements. For cgroups v2 systems, make the CPU controller available by writing +cpu to the cgroup.subtree_control file of every ancestor cgroup, top down.

Under cgroup v2, even with write permissions to a cgroup, a non-root user can move a process into it only if the user has write access to the cgroup.procs file of the common ancestor of the target cgroup and the cgroup the user is currently in. Depending on the cgroup chosen, root may be required to start the YB-TServer.

Container deployments (Docker and Kubernetes) typically mount the cgroups filesystem read-only. To use multitenancy, the container must be privileged with the root cgroup hierarchy mounted read/write so that child cgroups can be created. Some platforms provide workarounds; for example, GKE supports writable cgroups in pods.

Enable and configure the resource governor

The resource governor is controlled entirely through flags. To enable multitenancy, set the enable_qos flag to true on Masters and TServers.

Set the following flags to configure multitenancy. Unless noted otherwise, set flags on both YB-Masters and YB-TServers.

Because settings are applied as flags, you can also change the flags after the cluster is created. Note that enable_qos requires a restart to take effect.

yugabyted

With yugabyted, you are responsible for setting up the root cgroup manually (see Prepare cgroups) because the required setup is highly dependent on your environment. After the cgroup is ready, pass the flags using --tserver_flags and --master_flags when starting the node.

For example, to reserve 5% of CPU for system work, cap each database at 25% of the remaining CPU, and allow at most 20 databases (a 5% per-database minimum):

./bin/yugabyted start \
    --tserver_flags="enable_qos=true,qos_max_db_cpu_percent=25,qos_system_high_cpu_reserved_percent=5" \
    --master_flags="enable_qos=true,qos_max_db_cpu_percent=25,qos_system_high_cpu_reserved_percent=5,qos_max_db_count=20"

Disable the resource governor

To turn off per-database CPU limits, set enable_qos to false and restart the YB-Master and YB-TServer processes. The cgroup setup performed on the operating system does not need to be undone.

Learn more

• Quality of service for other admission control and resource management controls.