System statistics

Host resource usage

To check the CPU, memory, and disk usage on a Linux machine, you can run the following command.

$ sudo echo -n "CPUs: ";cat /proc/cpuinfo | grep processor | wc -l; echo -n "Mem: ";free -h | grep Mem | tr -s " " | cut -d" " -f 2; echo -n "Disk: "; df -h / | grep -v Filesystem; 

CPUs: 72
Mem: 251G
Disk: /dev/sda2       160G   13G  148G   8% /
10.1.12.104
CPUs: 88
Mem: 251G
Disk: /dev/sda2       208G   22G  187G  11% /
10.1.12.105
CPUs: 88
Mem: 251G
Disk: /dev/sda2       208G  5.1G  203G   3% /

Generally, common tools like top or iostat may be useful.

Auditd

If top reports high CPU usage for the auditd process, it may have some rules auditing some system calls frequently used Yugabyte which can significantly affect performance. You can try temporarily disabling audit by running (on each Yugabyte node).

$ auditctl -e 0

and check if this improves peformance.

To re-enable it afterwards, run:

$ auditctl -e 1

Yugabyte processes state

YugabyteDB provides web endpoints where the current state of each process is aggregated. This includes logs, gflags as well as memory, disk, and network usage metrics. Additionally, it provides dedicated metrics endpoints for CQL and, respectively, Redis requests.

Note that, when running yb-ctl locally with default settings, it will create three local ips 127.0.0.1, 127.0.0.2, and 127.0.0.3, one for each YugabyteDB node.