Scale out a universe

Horizontal scale-out and scale-in in YugabyteDB

YugabyteDB can scale seamlessly while running a read-write workload. You can see this by using the YB Workload Simulator application against a three-node universe with a replication factor of 3 and add a node while the workload is running. Using the built-in metrics, you can observe how the universe scales out by verifying that the number of read and write IOPS are evenly distributed across all nodes at all times.

Set up a universe

Follow the setup instructions to start a three-node universe in YugabyteDB Anywhere, connect the YB Workload Simulator application, and run a read-write workload. To verify that the application is running correctly, navigate to the application UI at http://localhost:8080/ to view the universe network diagram, as well as Latency and Throughput charts for the running workload.

Observe IOPS per node

You can use YugabyteDB Anywhere to view per-node statistics for the universe, as follows:

  1. Navigate to Universes and select your universe.

  2. Select Nodes to view the total read and write IOPS per node and other statistics, as shown in the following illustration:

    Read and write IOPS with 3 nodes

    Note that both the reads and the writes are approximately the same across all the nodes, indicating uniform load.

  3. Select Metrics to view charts such as YSQL operations per second and latency, as shown in the following illustration:

    Performance charts for 3 nodes

  4. Navigate to the YB Workload Simulator application UI to view the latency and throughput on the universe while the workload is running, as per the following illustration:

    Latency and throughput with 3 nodes

Add a node

You can add a node to the universe in YugabyteDB Anywhere as follows:

  1. Navigate to Universes, select your universe, and click Actions > Edit Universe.

  2. In the Nodes field, change the number to 4 to add a new node.

  3. Click Save. Note that the scaling operation can take several minutes.

Observe linear scale-out

Verify that the node has been added by selecting Nodes, as per the following illustration:

Read and write IOPS with 4 nodes

Shortly, you should see the new node performing a comparable number of reads and writes as the other nodes. The tablets are also distributed evenly across all four nodes.

The universe automatically lets the client know to use the newly added node for serving queries. This scaling out of client queries is completely transparent to the application logic, allowing the application to scale linearly for both reads and writes.

Navigate to Metrics to observe a slight spike and drop in the latency and YSQL Ops / Sec charts when the node is added, and then both return to normal, as shown in the following illustration:

Latency and throughput graph with 4 nodes

Alternatively, you can navigate to the YB Workload Simulator application UI to see the new node being added to the network diagram. You can also notice a slight spike and drop in the latency and throughput when the node is added, and then both return to normal, as shown in the following illustration:

Latency and throughput graph with 4 nodes

Remove a node

You can remove a node from the universe as follows:

  1. Navigate to Universes and select your universe.

  2. Select Nodes, find the node to be removed, and then click its corresponding Actions > Remove Node.

Observe linear scale-in

Verify that the details by selecting Nodes. The scale-in operation can take several minutes and expect to see that the load has been moved off the removed node and redistributed to the remaining nodes.

Navigate to Metrics to observe a slight spike and drop in the latency and YSQL Ops / Sec charts when the node is removed, and then both return to normal, as shown in the following illustration:

Performance metrics with 4th node dead

Alternatively, you can navigate to the YB Workload Simulator application UI to see the node being removed from the network diagram when it is stopped. Also notice a slight spike and drop in the latency and throughput, both of which resume immediately, as shown in the following illustration:

Latency and throughput graph after stopping node 4

Clean up

You can delete your universe by following instructions provided in Delete a universe.