--- title: "Monitor with Active Session History" url: "https://docs.yugabyte.com/stable/launch-and-manage/monitor-and-alert/active-session-history-monitor/" --- # Monitor with Active Session History Use Active Session History to monitor and troubleshoot performance issues in YugabyteDB. Use Active Session History to monitor and troubleshoot performance issues in YugabyteDB [Active Session History](/stable/explore/observability/active-session-history/ "Active Session History") (ASH) provides a powerful way to troubleshoot performance by giving you a real-time and historical view of your database's activity. ASH captures samples of active sessions and exposes them through a set of SQL views. By querying these views, you can analyze wait events, identify performance bottlenecks, and understand where your database is spending its time. ASH is currently available for YSQL, YCQL, and YB-TServer, and records wait events like CPU, WaitOnCondition, RPCWait, and Disk IO. By analyzing this data, you can troubleshoot performance by answering questions like: - Why is a specific query or application slow? - Which queries are causing the most database load? ## Configure ASH To run ASH queries, regardless of whether you are using YSQL or YCQL, the YSQL API must be enabled on your universe (the default). If YSQL is not enabled on your universe, you can enable it by setting the [enable\_ysql](/stable/reference/configuration/yb-tserver/#enable-ysql "enable_ysql") flag to true on your Master and TServer servers. If you are using YugabyteDB Anywhere, to enable YSQL refer to [Enable database endpoints and authorization](/stable/yugabyte-platform/security/authorization-platform/#enable-database-endpoints-and-authorization "Enable database endpoints and authorization"). To configure ASH, you can set the following YB-TServer flags for each node of your cluster. Flag Description ysql\_yb\_enable\_ash Enables ASH. Changing this flag requires a TServer restart. Default: true ysql\_yb\_ash\_circular\_buffer\_size Size (in KiB) of circular buffer where the samples are stored. Defaults: - 32768 (32 MiB) for 1-2 cores - 65536 (64 MiB) for 3-4 cores - 131072 (128 MiB) for 5-8 cores - 262144 (256 MiB) for 9-16 cores - 524288 (512 MiB) for 17-32 cores - 1048576 (1024 MiB) for more than 32 cores Changing this flag requires a TServer restart. ysql\_yb\_ash\_sampling\_interval\_ms Sampling interval (in milliseconds). Changing this flag doesn't require a TServer restart. Default: 1000 ysql\_yb\_ash\_sample\_size Maximum number of events captured per sampling interval. Changing this flag doesn't require a TServer restart. Default: 500 ## YSQL views ASH exposes the following views in each node to analyze and troubleshoot performance issues. ### yb\_active\_session\_history This view provides a list of wait events and their metadata. The columns of the view are described in the following table. Column Type Description sample\_time timestamp Timestamp when the sample was captured root\_request\_id UUID A 16-byte UUID that is generated per request. Generated for queries at YSQL/YCQL layer. rpc\_request\_id integer ID for internal requests, it is a monotonically increasing number for the lifetime of a YB-TServer. wait\_event\_component text Component of the wait event, which can be YSQL, YCQL, or TServer. wait\_event\_class text Class of the wait event, such as TabletWait, RocksDB, and so on. wait\_event text Name of the wait event. wait\_event\_type text Type of the wait event such as CPU, WaitOnCondition, RPCWait, Disk IO, and so on. wait\_event\_aux text Additional information for the wait event, when available. For TServer events tied to a tablet, the first 15 characters of the tablet ID; for YCQL read/write events with table context, the first 15 characters of the table ID; for YSQL `WaitingOnTServer` events, the RPC name. Null for most other events. wait\_event\_code bigint Code of the wait event. top\_level\_node\_id UUID 16-byte TServer UUID of the YSQL/YCQL node where the query is being executed. query\_id bigint Query ID as seen on the `/statements` endpoint. This can be used to join with [pg\_stat\_statements](/stable/launch-and-manage/monitor-and-alert/query-tuning/pg-stat-statements/ "pg_stat_statements")/[ycql\_stat\_statements](/stable/launch-and-manage/monitor-and-alert/query-tuning/ycql-stat-statements/ "ycql_stat_statements"). See [Constant query identifiers](#constant-query-identifiers "Constant query identifiers"). pid bigint PID of the process that is executing the query. For YCQL and background activities, this will be the YB-TServer PID. client\_node\_ip text IP address of the client which sent the query to YSQL/YCQL. Null for background activities. sample\_weight float If in any sampling interval there are too many events, YugabyteDB only collects `ysql_yb_ash_sample_size` samples/events. Based on how many were sampled, weights are assigned to the collected events. For example, if there are 200 events, but only 100 events are collected, each of the collected samples will have a weight of (200 / 100) = 2.0 ysql\_dbid oid Database OID of the YSQL database. This is 0 for YCQL databases. ysql\_userid oid User OID of the YSQL user. This is 0 for YCQL users. pss\_mem\_bytes bigint Proportional Set Size (PSS) memory of the YSQL process in Linux builds. Resident Set Size (RSS) memory of the YSQL process in macOS builds. Both values are in bytes. ### yb\_wait\_event\_desc This view displays the class, type, name, and description of each wait event. The columns of the view are described in the following table. | Column | Type | Description | |-------------------------------|--------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | wait\_event\_class | text | Class of the wait event, such as TabletWait, RocksDB, and so on. | | wait\_event\_type | text | Type of the wait event such as CPU, WaitOnCondition, RPCWait, Disk IO, and so on. | | wait\_event | text | Name of the wait event. | | wait\_event\_code | bigint | Code of the wait event. | | wait\_event\_description | text | Description of the wait event. | | wait\_event\_aux\_description | text | Information about what type of auxiliary data the wait event contains. For example, if the wait event contains a tablet ID (first 15 characters), table ID (first 15 characters), or RPC name. | ## Constant query identifiers These fixed constants are used to identify various YugabyteDB background activities. The query IDs of the fixed constants are described in the following table. | Query ID | Wait Event Component | Description | |----------|----------------------|-------------------------------------------------------------------------------------------------------------| | 1 | TServer | Query ID for write ahead log (WAL) appender thread. | | 2 | TServer | Query ID for background flush tasks. | | 3 | TServer | Query ID for background compaction tasks. | | 4 | TServer | Query ID for Raft update consensus. | | 5 | YSQL/TServer | Default query ID, assigned in the interim before pg\_stat\_statements calculates a proper ID for the query. | | 6 | TServer | Query ID for write ahead log (WAL) background sync. | | 7 | YSQL | Query ID for YSQL background workers. | | 8 | TServer | Query ID for remote bootstraps. | | 9 | TServer | Query ID for snapshot operations. | | 10 | YCQL | Query ID for YCQL authentication request. | | 11 | YSQL | Query ID for YSQL walsender process. | | 12 | TServer | QueryID for xCluster poller thread. | To obtain the IP address and location of a node where a query is being executed, use the `top_level_node_id` from the active session history view in the following command: ```sql SELECT * FROM pg_catalog.yb_servers() WHERE uuid = ; ``` ```output host | port | num_connections | node_type | cloud | region | zone | public_ip | uuid --------------+------+-----------------+-----------+-------+-----------+------------+-----------+---------------------------------- 10.9.111.111 | 5433 | 0 | primary | aws | us-west-2 | us-west-2a | | 5cac7c86ba4e4f0e838bf180d75bcad5 ``` ## Wait events The following describes the wait events available in the [active session history](#yb-active-session-history "active session history"), along with their type and, where applicable, the auxiliary information (`wait_event_aux`) provided with that type. The events are categorized by wait event class. In the **Aux** column, **Tablet ID** and **Table ID** refer to the first 15 characters of the respective ID. ### YSQL These are the wait events introduced by YugabyteDB. Some of the following [wait events](https://www.postgresql.org/docs/15/monitoring-stats.html "wait events") inherited from PostgreSQL might also show up in the [yb\_active\_session\_history](#yb-active-session-history "yb_active_session_history") view. #### TServerWait class | Wait Event | Type | Aux | Description | |--------------------------------|---------|----------|--------------------------------------------------------------------------------------------------------| | TableRead | RPCWait | | A YSQL backend is waiting for a table read from DocDB. | | CatalogRead | RPCWait | | A YSQL backend is waiting for a catalog read from master. | | IndexRead | RPCWait | | A YSQL backend is waiting for a secondary index read from DocDB. | | StorageFlush | RPCWait | | A YSQL backend is waiting for a table/index read/write from DocDB. | | TableWrite | RPCWait | | A YSQL backend is waiting for a table write from DocDB. | | CatalogWrite | RPCWait | | A YSQL backend is waiting for a catalog write from master. | | IndexWrite | RPCWait | | A YSQL backend is waiting for a secondary index write from DocDB. | | WaitingOnTServer | RPCWait | RPC name | A YSQL backend is waiting on TServer for an RPC. The RPC name is present on the wait event aux column. | | TransactionCommit | RPCWait | | A YSQL backend is committing a transaction. | | TransactionTerminate | RPCWait | | A YSQL backend is terminating/rolling back a transaction. | | TransactionRollbackToSavepoint | RPCWait | | A YSQL backend is rolling back to a savepoint. | | TransactionCancel | RPCWait | | A YSQL backend is canceling a transaction. | #### YSQLQuery class | Wait Event | Type | Description | |----------------------------------|---------|--------------------------------------------------------------------------------------------| | OnCpu\_Active | CPU | A YSQL backend is doing CPU work. | | yb\_ash\_metadata | LWLock | A YSQL backend is waiting to update ASH metadata for a query. | | YBParallelScanEmpty | IPC | A YSQL backend is waiting on an empty queue while fetching parallel range keys. | | CopyCommandStreamRead | IO | A YSQL backend is waiting for a read from a file or program during COPY. | | CopyCommandStreamWrite | IO | A YSQL backend is waiting for a write to a file or program during COPY. | | YbAshCircularBuffer | LWLock | A YSQL backend is waiting for YugabyteDB ASH circular buffer memory access. | | YbQueryDiagnostics | LWLock | A YSQL backend is waiting for YugabyteDB query diagnostics hash table memory access. | | YbQueryDiagnosticsCircularBuffer | LWLock | A YSQL backend is waiting for YugabyteDB query diagnostics circular buffer memory access. | | YBTxnConflictBackoff | Timeout | A YSQL backend is waiting for transaction conflict resolution with an exponential backoff. | ### YB-TServer #### Common class | Wait Event | Type | Description | |-------------------------------|-----------------|-----------------------------------------------------------------------------------------| | OnCpu\_Passive | CPU | An RPC or task is waiting for a thread to pick it up. | | OnCpu\_Active | CPU | An RPC or task is being actively processed on a thread. | | Idle | WaitOnCondition | The Raft log appender/sync thread is idle. | | Rpc\_Done | WaitOnCondition | An RPC is done and waiting for the reactor to send the response to a YSQL/YCQL backend. | | RetryableRequests\_SaveToDisk | DiskIO | The in-memory state of the retryable requests is being saved to the disk. | #### TabletWait class Wait Event Type Aux Description MVCC\_WaitForSafeTime WaitOnCondition Tablet ID A read/write RPC is waiting for the safe time to be at least the desired read-time. LockedBatchEntry\_Lock WaitOnCondition Tablet ID A read/write RPC is waiting for a DocDB row-level lock. BackfillIndex\_WaitForAFreeSlot WaitOnCondition Tablet ID A backfill index RPC is waiting for a slot to open if there are too many backfill requests at the same time. BackfillIndex\_WaitToBackfillTablet WaitOnCondition Tablet ID Waiting for an index backfill chunk to be processed. Supported in v2026.1.2 or later. CreatingNewTablet DiskIO Tablet ID The CreateTablet RPC is creating a new tablet, this may involve writing metadata files, causing I/O wait. SaveRaftGroupMetadataToDisk DiskIO Tablet ID The Raft/tablet metadata is being written to disk, generally during snapshot or restore operations. TransactionStatusCache\_DoGetCommitData RPCWait Tablet ID An RPC needs to look up the commit status of a particular transaction. WaitForYSQLBackendsCatalogVersion WaitOnCondition Tablet ID CREATE INDEX is waiting for YSQL backends to have up-to-date pg\_catalog. WriteSysCatalogSnapshotToDisk DiskIO Tablet ID Writing initial system catalog snapshot during initdb. DumpRunningRpc\_WaitOnReactor WaitOnCondition Tablet ID DumpRunningRpcs is waiting on reactor threads. ConflictResolution\_ResolveConficts RPCWait Tablet ID A read/write RPC is waiting to identify conflicting transactions. ConflictResolution\_WaitOnConflictingTxns WaitOnCondition Tablet ID A read/write RPC is waiting for conflicting transactions to complete. WaitForReadTime WaitOnCondition Tablet ID A read/write RPC is waiting for the current time to catch up to [read time](/stable/architecture/transactions/single-row-transactions/#safe-timestamp-assignment-for-a-read-request "read time"). XCluster\_WaitingForGetChanges RPCWait Target Tablet ID An xCluster poller on the target universe is waiting for changes from the source universe. RemoteBootstrap\_StartRemoteSession RPCWait Tablet ID A remote bootstrap client is waiting for a remote session to be started on the remote bootstrap server. RemoteBootstrap\_FetchData RPCWait Tablet ID A remote bootstrap client is fetching data from a remote bootstrap server. RemoteBootstrap\_ReadDataFromFile DiskIO Tablet ID A remote bootstrap server is reading data from a file. RemoteBootstrap\_RateLimiter WaitOnCondition Tablet ID A remote bootstrap client is slowing down due to rate limiter throttling network access to remote bootstrap server. RocksDB\_CreateCheckpoint DiskIO Tablet ID RocksDB is creating a database checkpoint. Snapshot\_CleanupSnapshotDir DiskIO Tablet ID A snapshot operation is cleaning a snapshot directory. Snapshot\_RestoreCheckpoint DiskIO Tablet ID A snapshot operation is restoring a database checkpoint. Snapshot\_WaitingForFlush DiskIO Tablet ID A snapshot operation is waiting for flush. #### Consensus class Wait Event Type Aux Description WAL\_Append DiskIO Tablet ID A write RPC is persisting WAL edits. WAL\_Sync DiskIO Tablet ID A write RPC is synchronizing WAL edits. Raft\_WaitingForReplication RPCWait Tablet ID A write RPC is waiting for Raft replication. Raft\_WaitingForPipelinedReplication RPCWait Tablet ID A WaitForAsyncWrite RPC is waiting for a pipelined write to be Raft replicated. Supported in v2026.1.2 or later. Raft\_ApplyingEdits WaitOnCondition/CPU Tablet ID A write RPC is applying Raft edits locally. ConsensusMeta\_Flush DiskIO Tablet ID ConsensusMetadata is flushed, for example, during Raft term, configuration change, remote bootstrap, and so on. ReplicaState\_TakeUpdateLock WaitOnCondition Tablet ID A write/alter RPC needs to wait for the ReplicaState lock to replicate a batch of writes through Raft. WAL\_Read DiskIO Tablet ID An RPC is reading WAL. #### RocksDB class | Wait Event | Type | Aux | Description | |---------------------------------------|-----------------|-----------|--------------------------------------------------------------------------------------------------------| | RocksDB\_ReadBlockFromFile | DiskIO | Tablet ID | RocksDB is reading a block from a file. | | RocksDB\_OpenFile | DiskIO | Tablet ID | RocksDB is opening a file. | | RocksDB\_WriteToFile | DiskIO | Tablet ID | RocksDB is writing to a file. | | RocksDB\_Flush | CPU | Tablet ID | RocksDB is doing a flush. | | RocksDB\_Compaction | CPU | Tablet ID | RocksDB is doing a compaction. | | RocksDB\_PriorityThreadPoolTaskPaused | WaitOnCondition | Tablet ID | RocksDB pauses a flush or compaction task for another flush or compaction task with a higher priority. | | RocksDB\_CloseFile | DiskIO | Tablet ID | RocksDB is closing a file. | | RocksDB\_RateLimiter | WaitOnCondition | Tablet ID | RocksDB flush/compaction is slowing down due to rate limiter throttling access to disk. | | RocksDB\_WaitForSubcompaction | WaitOnCondition | Tablet ID | RocksDB is waiting for a compaction to complete. | | RocksDB\_NewIterator | DiskIO | Tablet ID | RocksDB is waiting for a new iterator to be created. | #### VectorIndex class Wait Event Type Aux Description VectorIndex\_Search CPU Tablet ID The vector index is performing an approximate nearest neighbor search. Supported in v2026.1.1.0 or later. ### YCQL #### YCQLQuery class | Wait Event | Type | Aux | Description | |---------------|------|----------|-----------------------------------| | YCQL\_Parse | CPU | | YCQL is parsing a query. | | YCQL\_Read | CPU | Table ID | YCQL is processing a read query. | | YCQL\_Write | CPU | Table ID | YCQL is processing a write query. | | YCQL\_Analyze | CPU | | YCQL is analyzing a query. | | YCQL\_Execute | CPU | | YCQL is executing a query. | #### Client class Wait Event Type Description YBClient\_WaitingOnDocDB RPCWait YB Client is waiting on DocDB to return a response. YBClient\_LookingUpTablet RPCWait YB Client is looking up tablet information from the master. YBClient\_WaitingForPipelinedWrites RPCWait YB Client is waiting for its pipelined writes to be Raft replicated before it can commit the transaction. Supported in v2026.1.2 or later. ## Limitations Note that the following limitations are subject to change. - ASH is available per node and is not aggregated across the cluster. To query ASH samples across every live YB-TServer, see [Cluster-wide database views](/stable/explore/observability/cluster-wide-db-views/ "Cluster-wide database views") (`gv$yb_active_session_history`). - ASH is not available for [YB-Master](/stable/architecture/yb-master/ "YB-Master") processes. - ASH is available for queries and a few background activities like compaction and flushes. ASH support for other background activities will be added in future releases. ## Learn more - Active Session History [examples](/stable/explore/observability/active-session-history/#examples "examples")