What's new in the YugabyteDB v2.18 STS release series

Release announcements

Introducing YugabyteDB 2.18: Unlock Kubernetes and Simplify Operations

Release notes

YugabyteDB Anywhere release notes have moved

Starting with v2.16, the release notes for YugabyteDB Anywhere have moved to their own page.

Upgrade to release 2.18.7.0 or later

To avoid a rare race condition that can be encountered in v2.18.5-2.18.6, upgrade to release v2.18.7.0 or later.

What follows are the release notes for the YugabyteDB v2.18 release series. Content will be added as new notable features and changes are available in the patch releases of the YugabyteDB v2.18 release series.

For an RSS feed of all release series to track the latest product updates, point your feed reader to the RSS feed for releases.

v2.18.7.0 - April 1, 2024

Build: 2.18.7.0-b30

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.18.7.0-b30

Improvements

YSQL

Treats REFRESH MATERIALIZED VIEW as a non-disruptive change, preventing unnecessary transaction terminations. The default option, REFRESH MATERIALIZED VIEW NONCONCURRENTLY, modifies metadata but without making a disruptive alteration. #20420

Bug fixes

YSQL

Ensures the Linux PDEATH_SIG mechanism signals child processes of their parent process's exit, by correctly configuring all PG backends immediately after their fork from the postmaster process. #20396
Fixes table rewrite issue on non-colocated tables/matviews in colocated DB, ensuring the new table uses the original table's colocation setting. Includes a workaround for GH issue 20914. #20856
Prevents simultaneous send of read and write operations in the same RPC request that could lead to inconsistent read results, by ensuring that, in case of multiple operations, all buffered ones are flushed first. #20864
Increases the schema version of the default partition whenever you create a new partition, preventing erroneous data insertion into the default partition due to cache refresh issues. #17942
Adjusts heartbeat mechanism to shut down when an "Unknown Session" error occurs, reducing log alerts. This benefits idle connections with expired sessions. #21264
Allows BNL's on outer and inner tables, even if the inner table has "unbatchable" join restrictions that can't accept batches of inputs, enhancing queries with complex join conditions. #21366
Corrects error in creating an index using both SPLIT AT and INCLUDE by properly considering included columns in computing split rows. #11880

DocDB

Corrects the RPATH setting for certain OpenLDAP libraries, ensuring correct versions are used and eliminating potential system conflicts. #21236
Corrects block cache metrics discrepancy by ensuring Statistics object passes into LRUCache from TableCache for accurate updates. #21407
Resolves issues in the under-replicated endpoint algorithm, ensuring correct counting of replicas only when the block's minimum number of replicas has not been fulfilled yet, hence offering accurate replica tally for placement blocks. #20657
Validates the use of two arguments for disable_tablet_splitting, addressing a previous condition where only one was required, thereby enhancing backup process reliability. #8744
Updates the condition for HT lease reporting to ensure accurate leaderless tablet detection in RF-1 setup, preventing false alarms. #20919

CDC

Reduces tablet split data loss risk by updating cdc_sdk_safe_time only when a GetChanges explicit checkpoint acknowledgment is received, improving data reliability. #15718
Fixed the decoding of NUMERIC value in CDC records to prevent precision loss by ensuring that the decoded string is not converted to scientific notation if its length is more than 20 characters. Additionally, the fix involves using the string representation with no limit on length and employing the Postgres numeric_out method for decoding, which is identical to the decoding of numerics in a PG query. #20414

v2.18.6.0 - February 6, 2024

Use 2.18.7.0 or later.

Improvements

YSQL

Issues a notice for unsafe ALTER TABLE operations, including for ADD COLUMN...DEFAULT, to indicate existing rows won't be backfilled with the default value, enhancing user awareness. Suppression possible by setting ysql_suppress_unsafe_alter_notice gflag to true. #19360
Added sorting capabilities to BatchNestedLoopJoin to return the rows in the same order as NestedLoopJoin #19589
Replaces the ysql_max_read_restart_attempts and ysql_max_write_restart_attempts gflags with yb_max_query_layer_retries, applies limit to Read Committed isolation statement retries, and adjusts retry_backoff_multiplier and retry_min_backoff defaults. #20359

DocDB

Allows tracing of UpdateConsensus API by enabling the collect_update_consensus_traces GFlag, providing trace logs for UpdateConsensus from remote followers while also ignoring request.trace_requested if both leader and follower doesn't support this change. #19417
Allows customizing retryable request timeouts to respect client-side YCQL and YSQL timings, optimizing log replay and preventing the tserver from rejecting requests that exceed durations. Adjusts default retryable request timeout to 660 seconds and offers a configuration to eliminate server-side retention of retryable requests with FLAGS_retryable_request_timeout_secs =0. #18736
Logs all instances of tablet metadata creation/updation, enabling additional insights for troubleshooting in cases of multiple meta records for the same tablet. #20042
Automates recovery of index tables impacted by a bug, preventing performance degradation and disk size leak, by ensuring schema.table_properties.retain_delete_markers is reset to false when index backfilling is done. #19731
Offers redesigned server level aggregation for metrics, thus introducing more metrics for enhanced debugging. Removes several unused URL parameters and makes the new output compatible with YBA and YBM, preventing double-counting issues in charts. Drops unused JSON and Prometheus callbacks from MetricEntity for a cleaner design. #18078
Enhances debugging by adding verbose logging for load balancer global and per-table state changes and including global state information when printing sorted load. #20289
Reduces server initialization time by eliminating the accumulation of deleted tablet superblocks during startup, through a modification in the DeleteTablet operation. #19840
Enables automatic recovery of index tables affected by a bug, verifying their backfilling status and correcting the retain_delete_markers property to enhance performance. #20247
Integrates hard and soft server memory limit metrics into Prometheus, enabling better visualization of memory allocation and usage data, specifically in non-default value universes. #20578
Enables control over the batching of non-deferred indexes during backfill via a new flag, improving index management. #20213

Bug fixes

YSQL

Eliminates the issue where killing a background worker consumed a Proc struct without returning it, effectively limiting webserver restarts to eight times. #20154
Introduces enhanced logging in YSQL webserver for better debugging of failures, by adding two new flags that log endpoint access and print basic tcmalloc stats respectively. The flags can be updated by sending a SIGHUP signal. Additionally, offers more visibility into the webserver's memory usage through the newly created :13000/memz and :13000/webserver-heap-prof. #20157
Rectifies a segmentation fault issue in the Postmaster process in rare lock acquisition cases. #20166
Enables postmaster restart when a backend process is unexpectedly terminated while in a critical section, preventing potential data corruption and system hangs. #20255
Establishes :13000/memz and :13000/webserver-heap-prof endpoints to improve visibility into the webserver memory usage while managing tcmalloc stats and current allocations. Integrates a new runtime variable yb_pg_metrics.webserver_profiler_sample_freq_bytes to control tcmalloc sampling frequency. #20157
Introduces the pg_stat_statements.yb_qtext_size_limit flag to control the maximum file size read into memory, preventing system crashes due to oversized or corrupt qtext files. #20211
Limits the number of attempts to read inconsistent backend entries to 1000 and logs every hundredth attempt to prevent potentially infinite waiting loop. #20274
Fixes a segmentation fault during cleanup phase in the webserver by ensuring MyLatch is used consistently throughout the process lifecycle. #20309
Reduces excessive memory consumption during secondary index scans #20275
Refines the computation of required batched/un-batched path parameters, eliminating the need to manually track non-batchable relations and simplifying logic. Fixes a bug affecting Nested Loop joins. #19642, #19946
Upgrades "Unknown session" error to FATAL permitting drivers to instantly terminate stale connections, minimizing manual user intervention. #16445
Rectifies correctness issue when nested correlated subqueries uses sequential scan #19694
Rectifies correctness issues when BatchNestedLoop join is used and the join condition contained a mix of equality and non-equality filters. #20531
Rectifies correctness issue when join on inequality condition and join columns contains NULL values #20642
Rectifies correctness issue when queries involving outer joins and aggregate uses BNL. #20660
Corrects the Batch Nested Loop's optimization logic for proper handling of cases where the given limit matches the outer table's exact size, ensuring accurate query results. #20707
Addresses a bug that caused backup failure due to the absence of yb_catalog_version, by ensuring the function's existence post-normal migration. #18507

DocDB

Mitigates the issue of uneven tablet partitions and multiple pollers writing to the same consumer tablet by only applying intents on the consumer that match the producer tablet's key range. If some keys/values are filtered out from a batch, it will not delete the consumer's intents, as they may be needed by subsequent applications. Also guarantees idempotency, even if some apply records stutter and fetch older changes. #19728
Reduces chances of transaction conflicts upon promotion by delaying the sending of UpdateTransactionStatusLocation RPCs until after the first PROMOTED heartbeat response is received, enhancing transaction consistency and accuracy. #17319
Resolves potential WriteQuery leak issue in CQL workloads, ensuring proper execution and destruction of queries, while preventing possible tablet shutdown blockages during conflict resolution failure. #19919
Unblocks single shard waiters once a blocking sub-transaction rolls back, by applying identical conflict check logic for both distributed transactions and single shard transactions. #20113
Eliminates a race condition that can occur when simultaneous calls to SendAbortToOldStatusTabletIfNeeded try to send the abort RPC, thus preventing avoidable FATALs for failed geo promotions. #17113
Deprecates the enable_process_lifetime_heap_sampling flag and allows to control tcmalloc sampling via the profiler_sample_freq_bytes flag alone, enhancing user control over database sampling settings. #20236
Resolves instances of the leaderless tablet endpoint incorrectly reporting a tablet as leaderless post-leader change, by tweaking the detection logic to depend on the last occurrence of a valid leader, ensuring more accurate tablet reporting. #20124
Allows early termination of ReadCommitted transactions with a kConflict error, enhancing overall system throughput by eliminating unnecessary blockages without waiting for the next RPC restart. #20329
Modifies SysCatalog tablet's retryable request retention duration to consider both YQL and YSQL client timeouts, reducing the likelihood of request is too old errors during YSql DDLs. #20330
Fixes FATAL errors occurring during tablet participant shutdown due to in-progress RPCs by ensuring rpcs_.Shutdown is invoked after all status resolvers have been shut down. #19823
Handles backfill responses gracefully even when they overlap across multiple operations, reducing risks of crashes and errors due to network delays or slow masters. #20510

CDC

Corrects the computation of the cdcsdk_sent_lag metric to prevent steep, disproportionate increases by updating the last_sent_record_time when a SafePoint record is spotted, in addition to DMLs and READ ops. #15415
Adjusts table drop process to shift CDCSDK streams from ACTIVE to DELETING_METADATA state, enhancing the ability to add new tables to streams even after a table drop. #20428

Other

Adjusts tserver start and tserver stop scripts to successfully terminate all running PG processes, irrespective of their PID digit count. #19817

v2.18.5.2 - January 17, 2024

Use 2.18.7.0 or later.

Bug fixes

YSQL

Fix BNL local join lookup equality function. #20531

CDC

Fix addition of new tables to stream metadata after drop table. #20428

v2.18.5.1 - January 10, 2024

Use 2.18.7.0 or later.

Improvements

YSQL

In Read Committed Isolation, limit the number of retry attempts when the aborted query is retried. #20359

Bug fixes

YSQL

Return correct results when Batch Nested Loop join is used for queries involving Nested LEFT JOINs on LATERAL views. #19642, #19946

DocDB

In Read Committed Isolation, immediately abort transactions when conflict is detected. #20329

v2.18.5.0 - December 21, 2023

Use 2.18.7.0 or later.

New features

CDC

Introduces a flag: enable_tablet_split_of_cdcsdk_streamed_tables (default:false) to toggle automatic tablet splitting for tables in a CDCSDK stream, enhancing user control over replication processes. #19482

Improvements

YSQL

Enables the display of unique DIST statistics through the auto_explain module, available with a new option auto_explain.log_dist and can be turned off as desired. #17508
Enables the modification of is_single_row_txn for finer control over non-transactional writes required by COPY, index backfill, or when yb_disable_transactional_writes is set, preventing issues during non-bufferable operations for single row transactions. #4906
Modifies YSQLDump to generate create index NONCONCURRENTLY, preventing automated index backfilling during the backup-restore process, making it quicker and more effective. #19457
Verifies error return of px_cipher_decrypt in pgcrypto, a crucial step towards supporting OpenSSL 3.0+. #19732
Enables a prerequisite import of upstream PG commit Disable OpenSSL EVP digest padding in pgcrypto, paving the way for OpenSSL 3.0+ upgrade. #19733
Allows for the import of upstream PG commit Add alternative output for OpenSSL 3 without legacy loaded, aiding OpenSSL 3.0+ upgrade. #19734
Prevents potential SQL injections in extension scripts by rejecting substitutions of extension schemas or owners, mitigating vulnerability CVE-2023-39417. #14419
Mitigates potential security issues by importing upstream Postgres commit from REL_11_STABLE for future support of ALTER <object> DEPENDS ON EXTENSION, although it's not currently available in YugabyteDB. #14419
Refines the planner to recognize YB LSM indexes' order sustaining properties, even with IN conditions on non-leading columns, thus supporting ordered distinct index scans effectively. #19576
Increase the oom_score_adj (Out of Memory score adjustment) of the YSQL webserver to 900, the same as PG backends, prioritizing its termination when it significantly consumes memory. #20028
Enhances debugging capabilities in YSQL Webserver process by introducing two new flags for endpoint access logging and tcmalloc stats printing. #20157
Enhances YSQL webserver with new /memz and /webserver-heap-prof endpoints for greater visibility into webserver memory usage. Also introduces a new runtime variable yb_pg_metrics.webserver_profiler_sample_freq_bytes for controlling tcmalloc sampling frequency. #20157

DocDB

Prevents tserver heartbeats to a master leader in a different universe, preventing incorrect registrations and data loss risks, gated by a new autoflag master_enable_universe_uuid_heartbeat_check = true, requiring user to manually update post-upgrade. #17904
Allows customization of MemTracker's metric names for better distinguishability, and enables aggregation of metrics at table-level for more comprehensive analysis. #18627
Adopts the trace outside the block for ensuring correct execution of per-session tracing with standalone traces, and fixes callbacks to adopt the appropriate trace. #19099
Prevents potential lags and errors in the WaitOnConflict workload testing by ensuring in-memory locks get properly resettled, even in the event of a failed conflict resolution sequence. This fix helps avoid lock-related issues that might occur with any leaked or stuck WriteQuery objects. #18770
Introduces support for rocksdb_check_sst_file_tail_for_zeros flag, enabling verification of SST data file trailing zeros and reporting errors for anomalies detected. #19691
Includes the version info in the error message for easier identification when the Yugabyte process incorrectly starts on an older version after AutoFlags enabled. #16181
Adjusts the verbose level for VLOG macros to help differentiate between INFO and VERBOSE logs, fostering ease in debugging and analysis with better log filtration. #15273
Adds a metric for the number of running tablet peers on a tserver, making it easier to compute the ratio of tablet peers to cores and memory on YBM clusters. #9647
Allows for additional debug logs that offer relevant information for investigating instances of read path failure due to the expectation of a hybrid time error. #19876
Rejects ConfigChange requests for system catalog while another server is transitioning, preventing potential data loss from mistaken quorum formation by new peers. #18335
Enables SST write retries on detected corruption after write with the rocksdb_max_sst_write_retries flag, enhancing data reliability. #19730
Replaces glog includes with yb/util, introducing yb VLOG macros for clearer differentiation between INFO and VERBOSE logs, while addressing issues of duplicate includes. #15273
Balances tablet distribution across drives to prevent bottlenecks and better utilizes disk bandwidth during remote-bootstrapping of multiple tablets. #19846
Upgrades OpenSSL to version 3.0.8 as the previous version reached End of Life, pulling in changes to disable Linuxbrew builds and update glog for stack unwinding based on the backtrace function. #19736
Ensures the master_join_existing_universe flag value check before restoring the initial sys catalog snapshot to avoid incorrect operations. #19357
Enables automatic recovery of index tables for bugs detected in the context of all new index tables created from 2.14; prevents retention of tombstones in SST files, mitigates performance degradation and disk size leak. #19731

yugabyted

Corrects the CPU usage reporting in the sankey diagram by filtering nodes based on region selection on the performance page. #19991
Adjusts the yugabyted start command to interpret 0.0.0.0 as 127.0.0.1 in the advertise_address, aligning with the IP use in master, tserver, and yugabyted-UI. #18580

Bug fixes

YSQL

Allows ALTER TABLE DROP CONSTRAINT IF EXISTS command to execute without throwing an error, even when the table lacks a constraint, enhancing the user experience. #17659
Trims down the probability of inaccurate behaviour involving conflicts between single shard INSERT operations by ensuring read times are chosen after conflict resolution, enhancing data consistency. #19407
Addresses mismatches that occurred due to non-execution of the postprocess script on alternate expected files and default_expectfile in pg_regress. #19737
Resolves a bug that hindered consecutive lock acquisition due to not releasing LWLocks during AbortTransaction, preventing PostgreSQL processes from getting stuck. #19509
Introduces a per-database Postgres new object identifier (OID) allocation option, enabling unique OID in each database and supporting increased scalability for multi-node or multi-tenant environments. New config flag ysql_enable_pg_per_database_oid_allocator added to toggle between old and new allocator behavior. #16130
Prevents potential postmaster crashes during cleanup of killed connections by using the killed process's ProcStruct to wait on an unavailable LWLock. #18000
Prevents missed rows during distinct queries to tables with dead tuples by allowing AdvanceToNextRow to return false without seeking further if it encounters a dead tuple. #19911
Corrects a bug affecting type-checking for bound tuple IN conditions involving binary columns like UUID, appearing in releases 2.17.1 and up. #19753
Stabilizes the PgCatalogVersionTest.RecyleManyDatabases unit test by adding a delay in the ExternalMiniCluster::Restart routine to ensure full readiness of master/tserver, preventing potential deadlock situations. #17849
Reduces spinlock deadlock detection time by 75% for prompt handling of potential freezes and restarts Postmaster when a process holding a spinlock is killed, ensuring successful initiation of new connections. #18272, #18265
Restarts postmaster for easier and more effective cleanup when a process is accidentally killed during its own initialization or cleanup, avoiding potential issues with shared memory items handling. #19945
Adjusts the batched IN condition derivation in Batched Nested Loop queries to ensure accurate array type matching, preventing unexpected database crashes during query with aggregation and join operations. #20003
Refines background worker processes structure to prevent loss of Proc struct after killing a background worker, ensuring continued webserver functionality. #20154
Rectifies gcc11 compilation failure on CentOS by modifying iterator to non-const. #20087
Resolves a segmentation fault issue in postmaster by enabling it to use the Proc struct of a killed process, thereby enhancing process cleanup during unexpected terminations. #20166
Restart the postmaster whenever a process is killed while writing in a critical section, preventing corrupted state and CPU consumption due to infinite loops. #20255
Caps the number of attempts to read inconsistent back-end entries at 1000 to prevent indefinite waiting, proceeding with potentially incomplete yet safe results if consistently fails. #20274
Resolves segmentation faults in the webserver SIGHUP handler during cleanup. #20309
Corrects an error during table rewrite processes that previously omitted the relforcerowsecurity, preserving the Row-Level Security (RLS) behavior. #19815

DocDB

Overcomes issues with YCQL, snapshots, and metrics by migrating the TableInfo namespace_name field through the master migration framework, first introduced in version 2.3, with the namespace loader now preceding the table loader. #17713
Allows setting all columns of a row to NULL, resulting in deletion instead of creating a row consisting of NULLs, rectifying an issue during compaction. #18157
Allows tracing of outgoing calls only if the current RPC is being traced, reducing excessive memory consumption and logging. #19497
Allows loading transactions data when tablets are opened after a split or restart without relying on the assumption of seeing a strong intent and avoiding process failures in DEBUG builds. #18615
Restores the original behavior that doesn't count tablets on dead tservers towards the replica count in the under-replicated endpoint after a master leader failover. #19514
Facilitates replay of alter schema during local bootstrap and resolves inconsistencies between checkpoint and metadata, enhancing reliability of the database operations. #19546
Streamlines transaction processing by updating TabletState only for tablets engaged in writes and ignoring old statuses during transaction promotion, reducing failure errors and boosting consistency. #18081, #19535
Prevents premature garbage collection of schema packings on the target cluster in xCluster configuration, ensuring that schema versions referenced will not be removed before any related rows are received, enhancing the stability of your databases during schema changes or network partitions. #17229
Reduces index size growth and enhances query speed by properly updating the retain_delete_markers field on index backfilling, addressing an issue where delete markers persisted in SST files. #19544
Refines meta cache updates to avoid overwriting child tablets and consequently causing stale data, ensuring more accurate partition map refreshes. #18732
Increases tablet_creation_timeout_ms from 30 seconds to 600 seconds to avoid tablet replacement during table creation, enhancing restore operations' success rate under high network latency conditions. #13030
Resolves a use-after-free issue in RefinedStream's failure path to enhance system stability and prevent memory corruption. #19727
Synchronizes Master and TServer's MemTable metric names to improve monitoring consistency, aiding in the visibility of total memory consumption across all tablets in the Master's memory breakdown UI page. #19904
Fixes an intermittent index creation failure by checking the result of is_running, ensuring stability for empty YCQL tables, and preventing retain_delete_markers from being incorrectly set. #19933
Resolves a concurrency issue in MasterChangeConfigTest by operating on a copy of ExternalMaster* to prevent timing out when removing a follower with a transitioning server. #19927
Allows a single heap profile run at a time, preventing concurrent running that causes inconsistent sampling frequencies, enhancing the accuracy of these profiles. #19841
Resolves data inconsistency between source and target instances during xCluster +Transactions +Aggressive tablet Splitting operations. Now, ensures accurate application of intents matching the producer tablet's key range on the consumer, facilitating correct data replication in diverse transaction scenarios. #19728

CDC

Allows continuation of tablet fetching, even if certain tables face errors, by logging a warning instead of sending unnecessary errors to the client. #19434
Returns error to the client if reading the cdc_state fails, ensuring consistent and correct list of tablets, enabling your client to take suitable action. #19428
Eliminates deadlocks during database drops by properly handling namespace-level CDC stream deletion and setting up the database deletion process to finish in a finite time. #19879
Adjusts CDCSDK's Catalog Manager to delete entries from cdc_state on new tablets, resolving race conditions due to tablet split and ensuring accurate metadata cleanup in cdc_state table for dropped tables. #19746

Other

Avoids a segmentation fault in the stats collector following a Postmaster reset, solving potential crashes experienced when queries are terminated by SIGSEGV or SIGKILL. #19672
Eliminates potential crashes by adding nullity checks to CDCServiceImpl::GetChanges that previously could return a null pointer. #19524
Allows for the dynamic adjustment of previously hardcoded ports such as master_rpc_port, tserver_webserver_port, and master_webserver_port based on user's custom configurations, thereby facilitating successful creation of multi-region/zone clusters even with non-default port settings. #15334

v2.18.4.2 - November 8, 2023

Build: 2.18.4.2-b2

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.18.4.2-b2

This is a YugabyteDB Anywhere-only release, with no changes to the database.

v2.18.4.1 - November 6, 2023

Build: 2.18.4.1-b3

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.18.4.1-b3

Bug fix

[18081] [19535] [DocDB] Update TabletState only for tablets that are involved in write, Ignore statuses from old status tablet on promotion

v2.18.4.0 - October 25, 2023

Build: 2.18.4.0-b52

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.18.4.0-b52

Highlights

The yugabyted utility for deploying and managing clusters is now GA for production deployments.

New features

[18748] [CDCSDK] Change default checkpointing type to EXPLICIT while stream creating

Improvements

[12751] [CDCSDK] Made maxAttempts and sleepTime for retrying RPCs configurable in AsyncClient
[16780] [YCQLSH] Updating yugabyte-db-thirdparty release version
[17270] [DST] Fast PITR: Add ability to set cotables filter map
[17360] [DocDB] Dynamically set the db_block_cache_num_shard_bits based on the number of cores
[17464] [DST] Fast PITR - Use cotables filter for master restore
[17716] [DocDB] Move tcmalloc utility code out of MemTracker
[17758] [18561] [DocDB] Enable allocation sampling endpoint for gperftools tcmalloc.
[18207] [DST] Fast PITR - Custom history retention for DocDB metadata table from other tables of sys catalog tablet
[18482] [yugabyted] Improve logging of yugabyted-ui, and changes to make it more robust against node failures
[18668] [DocDB] Block stack collection during thread creation
[18698] [xCluster] Check for existence of data in table for IsBootstrapRequired check of a new stream
[18818] [yugabyted] Cloud based fault tolerance is supported with the yugabyted start command.
[18998] [DocDB] New flag to force shell mode for empty masters regardless of master_addresses.
[19071] [DocDB] Add estimated bytes / count to pprof memory pages
[19097] [DocDB] Escape output of master and tserver UI endpoints
[19176] [DocDB] Add URL param to disable Prometheus metrics help and type metadata
[19196] [CDCSDK] Add test case for tablet split followed by Drop table
[19203] [xCluster] Add xCluster tserver UI page
[19221] [DocDB] Improved timeout handling for YCQL index scan
[19244] Update logo in yugabyted UI
[19454] [Build] Move macOS support off of mac11
[19482] [CDCSDK] Add GFlag to disable tablet split on tables part of CDCSDK stream
[19524] [xCluster] Check for nullptr of CDCTabletMetrics

Bug fixes

[13128] [YSQL] Fix ClearSeparateDdlTxnMode error handling
[13999] [DocDB] Implement code and tests for /api/v1/tables API, /api/v1/table?id=X JSON API endpoints
[14658] [yugabyted] Deleting the conf directory when yugabyted destroy command is used.
[17025] [xCluster] Calculate replication lag metrics for split tablet children
[17727] [DocDB] Use allocated_size instead of size / count in tcmalloc sampling report.
[17959] [yugabyted] Use new /table API on the tablets page, and new /api/v1/tablets tserver API in go apiserver. Migrate the yugabyted API server to use /api/v1/tables endpoint instead of HTML parsing
[18379] [DocDB] Retain schema packing while creating snapshot
[18388] [yugabyted] Fixing yugayted start --secure command
[18408] [yugabyted] Fixing /api/tables to divide tables and indexes
[18482] [yugabyted] Fix total disk usage on overview page
[18658] [xCluster] Skip local peer during bootstrap if not ready/valid
[18738] [yugabyted] Corrected the spelling error in the cert command description when displaying the help output.
[18744] [DocDB] Add ability to recover from follower lag caused by stuck OutboundCall
[18744] [DocDB] Fixed issue with last_rpc_start_time_ reset which resulted in tsan race in connection
[18770] [DocDB] Stricter memory order in PreparerImpl::Run to prevent re-ordering of instructions
[19017] [yugabyted] yugabyted UI redirects should work with page refresh
[19292] [CDCSDK] Publish snapshot_key in GetCheckpointResponse only if it is present
[19323] [yugabyted] Fixes on the yugabyted UI
[19348] [CDCSDK] Only delete removed tablets from cdc_state in CleanUpCDCStreamsMetadata
[19384] [YSQL] Fix handling of case where a given RowCompareExpression cannot be bound
[19385] [CDCSDK] Fix WAL GC issue for tables added after stream creation
[19394] [CDCSDK] Only populate key to GetChangesRequest when it is not null
[19394] [CDCSDK] Set snapshot key from correct parameter in explicit check pointing
[19414] [yugabyted] UI bugs
[19428] Return error to client if scan of cdc_state fails
[19434] [CDCSDK] Do not completely fail while fetching tablets if one table hits error
[19440] [DocDB] Fix bug where invalid filter key was passed to Iterator initialization in backwards scans

v2.18.3.1 - October 19, 2023

Build: 2.18.3.1-b1

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.18.3.1-b1

Bug fixes

[19440] [DocDB] Fix bug where invalid filter key was passed to Iterator initialization in backward scans

v2.18.3.0 - September 20, 2023

Build: 2.18.3.0-b75

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.18.3.0-b75

New features

[18069] [YSQL] Add support for OIDC (JWT) based authentication

Improvements

[12853] [PITR] Turn consistent_restore flag on by default
[16963] [#18838] [DOCDB] Add verbose logging to the load balancer
[17570] [DOCDB] Improve leaderless tablet endpoint
[18069] [YSQL] Introduce jwtcpp_util as a wrapper over JWT-CPP
[18321] [YSQL] Enable preloading additional tables by specifying catalog list in flags; restore --ysql_catalog_preload_additional_tables
[18465] [DOCDB] Dump PeerManager to tserver debug ui consensus page
[18555] [CDCSDK] Refactored cdcsdk_ysql_tests to add waiting mechanism for GetChanges() call
[18826] Build - thirdparty_tool --list-compilers option enhancement
[18854] [YSQL] SELECT INTO and CREATE TABLE AS cause catalog version increase

Bug fixes

[12574] [xCluster] Hold reference count of YBTable in XClusterWriteRpc
[13171] [DOCDB] reject expired retryable requests
[14210] [DOCDB] fixed YSQL tablet routing
[14369] [DOCDB] Add a flag to ignore NULL sys catalog entries during enumeration
[15925] Disabling pg_tablet_split-test.cc tests for tsan
[16432] [17080] [xCluster] Disallow N:1 replication topology
[16446] [YSQL] Make yb_pg_batch_detection_mechanism configurable at runtime for session, Peek message following Execute from JDBC driver to identify batch execution
[16556] [CDCSDK] Only ignore unpolled tablets depending on context
[17146] [xCluster] Fix missing pgschema_name during SetupUniverseReplication
[17148] [DOCDB] Fixed wrong metric_type of prometheus metric
[17468] Retry ysqlsh in the Docker-based package check
[17682] [DOCDB] fix hybrid time too low due to race between TransactionParticipant::Poll and RaftConsensus::Start in rf-1 setup
[17751] [CDCSDK] Turn flag enable_update_local_peer_min_index to true by default
[17795] [CDCSDK] Don't stream same txn twice in same GetChanges call
[18164] [DOCDB] Ignore deleted tablets in leaderless endpoint
[18281] [YSQL] Fix query layer's read restart retries in Read Committed isolation
[18344] [YCQL] Fix TOJSON() to also unpack nested FROZENs
[18387] [DOCDB] Select statement fails with TABLET_DATA_SPLIT_COMPLETED
[18518] [DOCDB] removed unused MemTracker::rand_ field
[18519] [CDCSDK] Stop sending checkpoint details for unpolled child tablets in UpdatePeersAndMetrics
[18586] [YCQL] Return reserved keywords in double quotes
[18592] [DOCDB] Fix log message format in WritePostgresConfig
[18635] [xCluster] Fix ordering in GetLatestEntryOpId during bootstrap
[18642] [DOCDB] Code refactoring to match usage of has_been_fully_compacted to a proper name
[18652] [YSQL] YB Sequential Scans don't take locks in SERIALIZABLE isolation
[18879] [YSQL] Disable pushdown for TextSearch functions that are not thread safe

Known issues

N/A

v2.18.2.1 - August 16, 2023

Build: 2.18.2.1-b1

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.18.2.1-b1

New features

[16912] [DocDB] Expose GetOldTransactions RPC to fetch long-running transaction metadata
[17312] [CDCSDK] Add begin/commit records for single shard txns
[17608] [DocDB] Add HTTP security headers
[17696] [YSQL] Add yb_latency_histogram to server statements
[17700] [YSQL] Add yb_get_percentile YSQL function
[17795] [CDCSDK] Add wal_segment_index to GetChanges Response and Request
[17868] [DocDB] Export histogram quantile p50/95/99 as gauge to Prometheus #TYPE
[18068] [YSQL] add a new tserver (preview) convenience GFlag ysql_ident_conf_csv for user name maps

Improvements

[14308] [DocDB] update flag description now that savepoints work with xCluster
[15060] [DocDB] Makes backoff loop aggressive in colocated table creation path
[16116] [DocDB] Lazily flush superblock on colocated table creation
[16466] [DocDB] Flush superblock before reading in RBS with lazy superblock flush
[16604] [DocDB] Renames last_change_metadata_op_id to last_flushed_change_metadata_op_id in superblock
[16778] [DocDB] Background flush task when lazy superblock flush is enabled
[17104] [DocDB] Shuts downs superblock_flush_bg_task_ on tserver shutdown
[17356] [YSQL] Reduced memory_tracking overhead in high use situations from 10% to 3%
[17683] [DocDB] Invalidate YCQL system.partitions cache on table creation
[17730] [YSQL] Import Improve pruning of a default partition
[17801] [YSQL] Option to toggle distinct pushdown
[17805] [17965] [YSQL] Enhance TServer PG daemon robustness
[17826] [17823] [YSQL] Use a more granular clock for RPC timing on Linux builds
[17855] [CDCSDK] Force add snapshot bootstrap operation checkpoint to cdc_state table
[17867] [DocDB] Account for table placement and read replicas in underreplicated endpoint
[17884] [CDCSDK] Add checkpoint details to snapshot records and change check of copying checkpoint from colocated snapshot to streaming
[17966] [DocDB] Unify ModifyPlacementInfo string parsing
[17972] [CDCSDK] Add guards around updating invalid checkpoints during snapshot phase
[18064] [DocDB] enhance tserver consensus state page
[18067] [YSQL] Import 'Remove arbitrary restrictions on password length.'
[18067] [YSQL] Import "Make StringInfo available to frontend code."
[18069] [YSQL] Update third-party dependencies & add CMake configs for JWT-CPP
[18116] [CDCSDK] Persist the safe_time from explicit checkpoint in cdc_state table
[18391] [17864] [YSQL] RPC timers in write path default to granular clocks in Linux
[18418] [CDCSDK] Set cdc_populate_safepoint_record default to true
[DB-2613] [YSQL] Remove unused log4j reference from the java modules

Bug fixes

[13395] [YSQL] Return error for unset bind PK value
[14762] [DocDB] Prevent crashing the master if trying to restore a deleted table
[16267] [17755] [DocDB] Fix issues in RBS from closest peer codepath
[16441] [YSQL] Retry CREATE DATABASE when oid collision happens
[16508] [YSQL] Fix incorrect tablet count assignment on user tables created within geo-partitoned tablespace
[16597] [DocDB] Fixed PartitionSchema::CreateHashPartitions to provide even hash codes distribution
[16665] [14308] [DocDB] fix logging of RocksDB write batches when packing is used, part I and II
[16680] [CDCSDK] Fix colocated before image test with packed row
[16711] [DocDB] Only display AutoFlags in UI that are relevent to the process
[17038] [DocDB] Async open table while updating indexes
[17043] [YSQL] Fix check to see when to attempt binding hash key options in scanspec
[17225] [YSQL] Use consistent read snapshot in DEFERRABLE READ ONLY mode
[17290] [DocDB] Fix race condition in deadlock_detector causing invalid probe_num
[17351] [yugabyted] Enable EAR command is failing on Mac builds
[17400] [17488] [16665] [14308] [DocDB] fix crash due to speculatively decoding a slice ending in a Typeset
[17512] [yugabyted] Code changes to provide the master and tserver webserver port information to the API server
[17558] [DocDB] Issue with picking local_limit during paging read
[17574] [DocDB] Add EncryptedWritableFile::Size() to fix incorrect WAL footer start offset
[17592] [17760] [DocDB] Rename WAL file after writing the file header on the disk
[17648] [YSQL] Unexpected SELECT DISTINCT behaviour with non index conditions
[17663] [DocDB] Clear drive load / leader load maps before appending to them
[17718] [xCluster] Check table leader term in xCluster Poller
[17795] [CDCSDK] Emit CDC records in order of commit time instead of op ID and fix response safe_time calculation in corner cases
[17848] [DocDB] Fix race on RemoteTablet::replicas_ between RemoteTablet::GetRemoteTabletServers and RemoteTablet::Refresh
[17869] [YSQL] fix incorrect optimization on NULL scans
[17887] [DST] PITR restores should ignore Deleted namespaces
[17890] [xCluster] create intent iterator outside of RegularDB batch writer to avoid deadlock
[17925] [CDCSDK] Refetch correct schema in cases of schema versioning errors
[17935] [CDCSDK] Reduce the number of txns in TestCDCSDKConsistentStreamWithManyTransactions
[17961] [18008] [YSQL] Fix max connection limit errors with additional locking
[17969] [CDCSDK] Fix heap-use-after-free failure for TestFromOpIdInGetChangesResponse
[17981] [yugabyted] yugabyted start should work when custom certs_dir is provided
[17997] [xCluster] IsBootstrapRequired should return False if any data exists in the WAL
[18001] [DocDB] Skip key registry decryption in EncryptionManager::GetFullUniverseKeyRegistry() when EAR is disabled
[18023] [DocDB] Log the result of chronyd when tserver crashes with a high clock skew
[18071] [yugabyted] Changing the status display to not show the actual password when it's generated in secure mode
[18133] [CDCSDK] GetTabletListToPollForCDC should return safe_time as well
[18184] [DocDB] Fix crash in case of IN handling
[18227] [DocDB] Fix log reader missing max_replicate_index
[18322] [YSQL] fix index ScanKey bugs
[18328] [CDCSDK] Fix bug in GetTabletIdsToPoll when only 1 child tablet has been polled
[18393] [DocDB] Fix unknown int64 flags incorrectly getting tagged as runtime
[18445] [CDCSDK] Call ResolveIntents if consistent_safe_time is stuck
[18467] [CDCSDK] Do not send safepoint records in snapshot phase
[18635] [xCluster] Fix ordering in GetLatestEntryOpId during bootstrap
[18756] [xCluster] Only Throttle xCluster Traffic from source side

Known issues

N/A

v2.18.1.0 - June 27, 2023

Build: 2.18.1.0-b84

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.18.1.0-b84

Highlights

Improvements to xCluster, including:
- Support for transactional atomicity
- Faster bootstrapping
- Automatic transfer of the source universe certificate when new nodes are added to the target universe
- Ability to set up replication even if matching tables do not exist on the target universe
- Improved performance and reliability when gathering replication stream statuses
- Disallowing replication between two YugabyteDB versions that are incompatible
View latency histograms for queries using the yb_latency_histogram column in pg_stat_statements
Faster nested loops with batching. Requests to YB-TServers are batched during joins using a new join execution method referenced as Batched Nested Loop, and derived from classic nested loop joins.

New features

[14445] [YSQL] Add query latency histogram column in pg_stat_statements
[15036] [xCluster] Add transactional flag to setup_universe_replication
[16294] [14067] [YSQL] Implementation Enhancements for EXPLAIN ANALYZE
[16757] [xCluster] Add transactional flag on Producer stream

Improvements

[13786] [17395] [17494] [DOCDB] Auto full compactions based on deletes in SST files
[14284] [13041] [YSQL] Avoid setting read time when using serializable isolation level
[14473] [DOCDB] Accept registrations from tservers on the same node with greater sequence numbers
[16240] [YSQL] Use per-tuple memory context in relcache preloading
[16343] [DOCDB] Include TableType in the table and cdc metrics
[16750] [xCluster] Disallow setting up transactional replication if any YCQL tables are included in replication
[16826] [DOCDB] [xCluster] Use RocksDB DirectWriter for applying external intents
[17013] [DOCDB] Add colocation info to ListTables output
[17016] [DOCDB] Add #HELP and #TYPE lines for prometheus metric
[17150] [YSQL] Allow BNL on expressions with multiple relations on one side
[17318] [YSQL] Import 'Replace last PushOverrideSearchPath() call with set_config_option().' from PG
[17358] [DOCDB] Add tracking for memory used by YBMetadataCache
[17455] [xCluster] Log when UpdateCdcMetrics is delayed
[17463] [xCluster] Producer and consumer side network throttling
[17633] [YCQL] Introduce flag to control local calls on RPC thread

Bug fixes

[14308] [xCluster] fix xCluster replication so it correctly handles SAVEPOINTs
[14926] [DOCDB] Prevent intent clean-up while reading YSQL system tables
[15769] [YSQL] Fix functional indexes returning wrong results
[16157] [17433] [YSQL] Set PgClient session timeout value
[16367] [DOCDB] Delete Snapshot keeps on retrying indefinitely after master failover
[16513] [DOCDB] Correcting statistics for DOCDB keys found
[16590] [DOCDB] Add test which ensures deadlocks are detected across partitions
[16612] [DOCDB] Fix transaction promotion race condition resulting in unnecessary abort
[16631] [DOCDB] Fix delete_snapshot sending delete RPCs indefinitely for deleted tablets
[16697] [DOCDB] Disallow multiple OpenTable calls in YBMetaDataCache::GetTable
[16698] [DOCDB] Add NewWritableFile Encryption Wrapper to fix WAL reuse issue
[16794] [DOCDB] Make full compactions non-exclusive by default
[16805] [DOCDB] Split PgMiniTest into smaller ones
[16863] [DOCDB] Ignore missing YSQL indices when importing a snapshot
[16911] [DOCDB] Augment GetLockStatus tserver endpoint to handle transaction_id list
[16943] [xCluster] switch xcluster_ysql-test to use consistent WAL mode
[16999] [DOCDB] Retry read if read buffer was full on previous step
[17026] [xCluster] Fix metrics jump when regaining leadership
[17041] [YCQL] Add cache in FindMemberForIndex for jsonb
[17074] [DOCDB] Root memory consumption should not include pageheap_free_bytes
[17107] [YCQL] Prevent DROP TYPE for the used YCQL type
[17163] [YCQL] Use Metadata Cache by default in PREPARE
[17194] [CQL] Add memtracker to track CQL requests memory usage
[17203] Fix post_install.sh failure related to pg_stat_statements.so
[17219] [YCQL] Fixing tests AlterDropAdd*
[17253] [DOCDB] Fix RPC error handling in pg client
[17261] [DST] [PITR] Single file compaction does not reset Hybrid Time filter
[17276] [xCluster] cleanup the transaction to write_id map whenever transaction is applied
[17293] [xCluster] Avoid creating multiple copies of GetChangesPB and cleanup PBs as soon as apply finishes
[17296] [xCluster] Fix StreamMetadata TSAN race
[17308] [DOCDB] [xCluster] track schema packings in frontier for external records
[17317] [YSQL] Import Handle RLS dependencies in inlined set-returning functions properly
[17362] Avoid double memory tracking of request data and fix tracking for plain response buffer size
[17380] [YSQL] Make sure ybctid_column_value is still sent in index scans for upgrade compatibility
[17398] [CQL] Account for user query and keyspace name for prepared statements
[17404] Use LLVM URL from the third-party dir for ASAN/TSAN
[17431] [DOCDB] Fix partial hash key columns projection in DocRowwiseIterator
[17437] [DOCDB] Rename WAL reuse flag to include unit
[17480] [DOCDB] Update to NULL could be lost after compaction when packed row is enabled
[17502] [DOCDB] Exclude yugabyted-ui from sed binary manipulation
[17502] Fix the shared library interpreter used for yugabyted-ui
[17531] [DocDB] Disable wait-on-conflict behavior for YCQL traffic
[17559] [DST] Adding loop call during tablet server init to wait on master leader to get encryption keys at rest
[17632] [DOCDB] Fix GC of schema packings of colocated table
[17712] [DOCDB] Fix system partitions query to not return empty namespace name when its not set at table level
[17714] [YSQL] Resolve version_metadata.json missing issue
[17726] [DOCDB] A workaround for memory leak from InboundCall weak pointers
[17762] Bound the number of cloned ops by the number of partitions for IN conditions sent to hash sharded tables

Known issues

N/A

v2.18.0.1 - May 30, 2023

Build: 2.18.0.1-b4

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.18.0.1-b4

Bug fixes

[17431] [DocDB] Fix partial hash key columns projection in DocRowwiseIterator
[17480] [DocDB] Update to NULL could be lost after compaction when packed row is enabled
[17502] [DocDB] Exclude yugabyted-ui from sed binary manipulation, Fix the shared library interpreter used for yugabyted-ui.

Known issues

N/A

v2.18.0.0 - May 16, 2023

Build: 2.18.0.0-b65

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.18.0.0-b65

New features

[12417] [YSQL] Use relcache init files in per-database catalog version mode
[13336] [YSQL] push down the avg() aggregate
[13488] [xCluster] Create index in transactionally consistent xCluster
[14234] [14674] [DOCDB] Support Colocated+Packed+XCluster & auto pause/resume on schema changes
[14665] [DOCDB] enable packed row for co-located tables by default
[16068] [DOCDB] Support storing wait-for probes of distributed waiter txns spanning across tservers
[16137] [DST] PITR + Sequences - Ensure generated sequence values are monotonic after restore
[16165] [16358] [yugabyted] [UI] New design implementation for Overview and Database pages (#16167)
[16165] [yugabyted] UI Adding replication factor info in the /cluster API
[16195] [DOCDB] Add metrics support for ScanForward API
[16218] [DOCDB] Adding xcluster_read_only mode to standby cluster databases
[16351] [DocDB] Add Cql Tests to Simulate Update Index Callback Leak
[16370] [DOCDB] enable packed row in locking unit tests
[16380] Use thread safety analysis annotations to enforce the reactor thread requirement
[16447] [DOCDB] Use shared memory for Perform queries between postgres and tserver
[16503] [CDCSDK] Support colocated table with before image
[16513] Add statistics to track DOCDB keys found (total, obsolete, and obsolete past history cutoff)
[16515] [xCluster] Enable using metacache in cdcservice
[16571] [DOCDB] Protect InboundCall::trace_ with the mutex
[16600] Odyssey connection pooler CMake integration
[16601] [DOCDB] Introduce simplified version for RocksDB iterator Next
[16648] [DOCDB] Decode directly to QLValuePB
[16663] [YSQL] Extend Batched Nested Loop Join functionality to Appended base relations for partitioned tables
[16700] [DOCDB] Read directly to QLTableRow
[16719] [YSQL] Make ysql_yb_bnl_batch_size a runtime flag
[16742] Adding additional logging for API server
[16746] [xCluster] Compute and propagate apply_safe_time from producer
[16892] [DOCDB] AutoFlags should provide the capability to enable flags on Install scenarios only, but not upgrade

Improvements

[3090] [YCQL] Limit topology change events to subscribed connections
[13701] Build: Use Google TCMalloc for fastdebug build
[14961] [14962] [YSQL] Further reduce first-query metadata RPCs
[15014] [14333] [YSQL] Refactor some code to honour coding best pratices
[15746] [DOCDB] harden master leaderless tablet endpoint
[15749] [DOCDB] Delay rocksdb gauge metrics update to end of request
[15794] [DOCDB] Reduced Number of Tablets in AreLeadersOnPreferredOnly test
[15884] [YSQL] Skip indexed table's stats update after non-concurrent create index
[15909] [DOCDB] Move sys catalog writes out of catalog loader
[16058] [CDCSDK] Use safe time from request rather than response
[16091] [DOCDB] Changing post-split compaction to run non-exclusively
[16123] [DOCDB] Remove co-partitioning logic
[16237] [xCluster] deprecate flags cdc_max_apply_batch_num_records and cdc_max_apply_batch_size_bytes
[16255] [YSQL] Reduce memory use on insert, update, and delete by using per-tuple contexts and freeing bitmapsets
[16317] [13718] Move common functionality to base iterator class
[16374] [DOCDB] Improve error message when SAVEPOINT feature is attempted when disabled by --enable_pg_savepoint
[16397] [DOCDB] Set table to RUNNING state only after all tablets are ready
[16402] [yugabyted] Set cluster config as soon as the 3rd node joins
[16442] [DST] PITR + Per DB catalog version: Increment only the catalog version of the relevant database
[16454] [YSQL] Log on unexpected backend termination
[16460] [YSQL] Improve error messages for transaction conflicts
[16461] [YSQL] Allow hash batching to apply to an unbounded number of permutations
[16467] compact_table Now Updates last_full_compaction_time For Tables without Writes
[16472] [DOCDB] Replace std::thread with yb::Thread
[16480] Reduce the amount of recompilation due to Postgres header copying
[16494] [DOCDB] Create VerifyPgClientServiceCleanupQueue unit test
[16504] [CDCSDK] [Colocation] YBTable object should contain metadata about colocation
[16510] [xCluster] Return nullopt from GetSafeTime if namespace does not require xCluster safe time
[16513] [DOCDB] Improve collecting statistics for DOCDB keys found
[16545] Build: build/package on mac11
[16551] [xCluster] Account for xcluster safe time for history retention on consumer
[16555] [DOCDB] Avoid decoding doc hybrid time during read when possible
[16562] [DOCDB] Avoid unnecessary calls to SeekForward and FindLatestValue in doc reader prepare phase
[16581] [DOCDB] Remove outstanding tablet split limit by default
[16587] [DOCDB] Move common ScanSpec functionality to base class and remove specific handling from iterator
[16588] build-infra: Don't use pipes python module
[16617] [DB-6011] [YCQLSH] Updating yugabyte-db-thirdparty release version
[16650] [DOCDB] Remove intent_doc_ht control field from packed column value
[16651] [DOCDB] Sanity check for RocksDB iterator status validation
[16666] [YSQL] Switch to new query context for get_tablespace_distance() operations
[16683] Support build with Clang 16
[16715] [DOCDB] Optimize handling of add_table change_metadata_op
[16756] [xCluster] Move yb-admin get_xcluster_estimated_data_loss to get_xcluster_safe_time
[16789] [YSQL] Assume that CAPABILITY_PickReadTimeAtTabletServer is enabled all the time
[16849] [DOCDB] Tablet splitting: Lower the thresholds for splits on low core machines
[16868] [DOCDB] Sanitizes debug messages
[16875] [DST] PITR - Extend master's sys catalog retention based on snapshot schedule
[16900] [CDCSDK] Remove cdcsdk_unprocessed_tables from CDCStreamInfo
[16969] yugabyted UI updates
[16991] [YSQL] Update pruning quals when nested loop batching is on
[16993] [YSQL] Restructure type checking when forming batched expressions
[17075] [17078] Send full universe key registry to Tablet Server on Startup
[17115] [YSQL] Restructure batched indexqual creation
[CLOUDGA-12738] Make the package updates work on AlmaLinux 9
[CLOUDGA-12738] Update OS pacakges during AMI build process
[DB-5975] [YCQLSH] Updating yugabyte-db-thirdparty release version

Bug fixes

[506] [YCQL] Disable alter table rename on columns with index
[13862] [YSQL] Fix metric for total number of connections made
[14167] [YSQL] Fix PgPackedRowTest.AddDropColumn flakiness
[14270] [DOCDB] Fixes test SnapshotTxnTest.BankAccountsWithTimeStrobe
[15090] [DOCDB] Tablet splitting: Some queries fail with "The tablet is shutting down" during the workload
[15166] [DOCDB] Fix crashes due to tablet shutdown timeout
[15449] [CDCSDK] Send snapshot end marker when snapshot has no data
[15612] [YSQL] Fix a memory leak in pg_isolation_regress
[15930] [DOCDB] Fixes data race issue with YBSubTransaction in TableSizeTest_PartitionedTableSize
[16009] [YSQL] Fix the system catalog versioning behaviour on CREATE TABLE with FK reference command
[16010] [DOCDB] Fixes tsan data race in CompactionTestWithFileExpiration.FileExpirationAfterExpiry
[16131] [DST] B/R, PITR Fix invalid enum value of restoration state object
[16156] [DOCDB] Limit duplicate address check to used host-port combination in Tserver registration
[16195] [DOCDB] Handle duplicate user keys in ScanForward API
[16262] [DOCDB] deflake failover pitr tests on tsan
[16329] [YSQL] Fix metric for rejections due to connections limit
[16401] [yugabyted] configure data_placement not working with constraint value with uppercases
[16444] [DOCDB] Fix tsan and fastdebug test, Ignore heartbeats from peers not in reported config
[16451] [YSQL] Fix imports in D22790 and D22956
[16481] [CDCSDK] [Colocation] YBClient fails to map multiple colocated tables to same tablet
[16518] [DOCDB] Prevent resuming conflict resolution at special timestamps in case transaction is not found
[16526] [DOCDB] Enable remote bootstrap to fix missing tablet replica problem in CreateTableStressTest::TestConcurrentCreateTableAndReloadMetadata.
[16540] [YSQL] Address HLL ASAN failures
[16541] [yugabyted] Show EAR in status if enabled even when --secure flag is false
[16552] [YSQL] Prevent prefix_length_ from taking non-key columns into account
[16565] [DOCDB] Fixed missing data in case of RocksDB iterator error
[16570] [YSQL] Fix build errors when building with --no-tcmalloc
[16571] [DOCDB] Fix InboundCall::trace() regression
[16583] [YSQL] Fix IndexOnlyScan with no targets returning empty rows
[16615] Dev tools: Fix script compatibility with newer arcanist
[16626] [DOCDB] Fix TSAN failure:XClusterConsumer and XClusterOutputClient deadlock
[16632] [DOCDB] Fix TestIndex.testDropDuringWrite
[16637] [CDCSDK] Fix safe_time issues around large transactions
[16644] [DST] PITR Fix YbAdminSnapshotScheduleTest.PgsqlAddColumnCompactWithPackedRow test
[16669] [DOCDB] Fix tablet splitting / PITR deadlock
[16679] [CDCSDK] Correcting safe_time bug for batches with only commit record
[16713] [YSQL] Fix heap-buffer-overflow in ysql_upgrade.cc
[16745] [YSQL] Fix TestPgRegressAuthorization failure
[16748] [DOCDB] Fix colocation parent table's metrics update when a colocated table is altered
[16763] [xCluster] Reduce logging noise during xcluster errors
[16773] [xCluster] Fix TSAN race on StreamTabletMetadata
[16790] [CDCSDK] GetCheckpoint will return invalid checkpoint for unpolled tablets
[16807] [DOCDB] Enable automatic_tablet_splitting for new clusters using AutoFlags
[16851] Fix DB UT for learner-TS due to broadcast address changes in yb_backup.py
[16944] [16715] [DOCDB] Fix 2 bugs with xcluster + packed, Fixing the demo connect creating tables to default database.
[16964] [DOCDB] Compilation failure for Apple clang 14.0.3
[16976] [YSQL] Fix postgres crash when pg_stat_progress_create_index has an index from a different database
[16992] [YSQL] Fix scan upperbound calculation during batched IN query on hash columns
[16995] [CDCSDK] Removed duplicated access to CDC Stream table
[17067] [YSQL] Disallow hash batching if there is only one batch element
[17118] [CDCSDK] Remove tables with no active streams from namespace_to_cdcsdk_unprocessed_table_map_
[17150] [YSQL] Disable batching on join clauses that involve more than one base relation on a side
[17165] [CDCSDK] Added serve_as_proxy member to GetCheckpointRequestPB
[17253] [DocDB] Fix RPC error handling in pg_client
[CLOUDGA-12305] Missing getters for provider fields - Part 2
[CLOUDGA-13117] Remove the OS update playbook components
[YSQL] style: fix/improve most of D21043 style
Fix xCluster utest build error

Known issues

N/A