What's new in the v2.25 preview release series
What follows are the release notes for the YugabyteDB v2.25 release series. Content will be added as new notable features and changes are available in the patch releases of the YugabyteDB v2.25 release series.
For an RSS feed of all release series, point your feed reader to the RSS feed for releases.
v2.25.0.0 - January 17, 2025
Build: 2.25.0.0-b489
Downloads
Docker:
docker pull yugabytedb/yugabyte:2.25.0.0-b489
Highlights
We're excited to announce the Technical Preview of two foundational features as part of this release.
PostgreSQL 15 Support TP
As part of this release, we have upgraded our PostgreSQL fork from version 11.2 to 15.0, enabling you to leverage the many key capabilities introduced in PostgreSQL between these two versions. This upgrade brings YSQL API support for numerous features, including stored generated columns, foreign keys on partitioned tables, and non-distinct NULLs in unique indexes. It also introduces query execution optimizations like incremental sort and memoization, along with various observability and security enhancements. For more information, refer to PostgreSQL 15 features.
Query Diagnostics TP
This feature significantly simplifies tuning poorly performing SQL queries by allowing you to capture and export detailed diagnostic information, including bind variables and constants, pg_stat_statements statistics, schema details, active session history, and execution plans. Refer to Query diagnostics.
Active session history
In addition, the Active Session History, which provides real-time and historical views of system activity, is now enabled by default. For more information, refer to Active Session History.
Change log
View the detailed changelog
Improvements
YSQL
- Adds pg_hint_plan syntax and functionality to control batched nested loop joins, allows setting hints
YbBatchedNL(t1 t2)
andNoYbBatchedNL
, and modifiesyb_prefer_bnl
handling. Also, it removes BNL's dependency onenable_nestloop
and adjusts cost model. #19494 - Enables control over in-place index updates with
yb_enable_inplace_index_update
. #20908 - Added yb_make_next_ddl_statement_nonincrementing to YbDbAdminVariables for yb_db_admin role. #23786
- Allows use of
UNLOGGED
tables syntax, handling it asLOGGED
with a warning. #23895, #23889 - Adds
pg_yb_logical_client_version
table to manage session configurations. #23871 - Introduces a dedicated auth-backend for YSQL to handle connection authentication, enhancing security and streamlining the process. #24147
- Ensures correct error handling by adding a missing InvalidOid check in YSQL. #24147
- Enables authentication via a specialized backend, enhancing security and reliability. #24147, #24427
- Enables toggling authentication method in YSQL using the
ysql_conn_mgr_use_auth_backend
flag. #24147 - Sets
yb_bnl_batch_size
to 1024 andyb_prefer_bnl
to true by default, ensuring BNL's replace nested loop joins without altering non-NL join plans. #19273 - Allows preloading of foreign key lists in relcache, avoiding on-demand master fetches, controlled by
yb_enable_fkey_catcache
. #23686 - Adds notes on pinned objects and script backporting to the README. #24334
- Prevents catalog version bumps on no-op
ALTER ROLE
commands, saving system resources. #24390 - Enhances object creation efficiency by using static factory methods in
PgDML
classes. #24412 - Disables JIT in YB PG15 by default for stability. #24479
- Enables successful major version upgrades of functions containing semicolons. #24571
- Introduces
GetStatementAs<...>
to simplifyPgStatement*
casts. #24754 - Fixes style issues and lint errors in
ybcModifyTable.c
. #24882 - Simplifies
ybcSetupTargets
by using BMS only, enhancing target column handling. #25007 - Excludes third-party extensions from PostgreSQL linter checks. #25054
- Enables creation of indexes that are covered by the main table. #24123
- Enables ASH by default for better monitoring and diagnostics. #24686
- Restores performance hints in prepared statements to ensure consistent execution. #24055
- Updates YSQL upgrade tests for compatibility with pg15. #24289
- Simplifies descriptions for inherited PostgreSQL wait events. #24972
- Enables users to control YB query execution plans using
pg_hint_plan
. #6720 - Adds Oracle-like functions to YSQL with the Orafce extension. #3460
YCQL
- Resolves timestamp precision mismatches in YCQL, aligning inserts and index scans to millisecond precision. Enables users to optionally revert to microsecond precision using the flag
cql_revert_to_partial_microsecond_support=false
. #11052, #23476 - Tracks microsecond precision timestamps in YCQL with new metric
cql_microseconds_timestamps_used
. This tracking occurs if thecql_revert_to_partial_microsecond_support
flag is true. #24776
DocDB
- Allows reduction of thread stack size to avoid Linux hugepage backing. #23927
- Reduces unnecessary attribute map creations during Prometheus metric aggregation. #24405
- Enhances load balancer to prioritize under-replicated tablets first. #20263
- Limits concurrent remote bootstraps per TServer using
load_balancer_max_inbound_remote_bootstraps_per_tserver
flag. #2426 - Prevents overloading by correctly categorizing bootstrapping tablets in load balancing. #23487
- Upgrades non-FIPS OpenSSL to 3.0.15 and removes CentOS 7 builds. #24436
- Enhances RBS throttling by focusing on active fetch sessions and adjusts expiration logic. #21563, #24031
- Enables persisting TServer registry entries to the sys catalog, enhancing failover recovery. #22925
- Initial implementation of Vector LSM supports multi-threaded batch inserts. #24020
- Updates and organizes third-party dependencies like
usearch
for better maintainability. #23998 - Simplifies HNSW library structure by using a factory-based approach. #24085
- Allows explicit addition of flags to
gflag_allowlist.txt
for secure callhome data collection, plusversion_info
in TServer data. #24103 - Enables xCluster to handle non-colocated ALTER TABLE commands without pausing replication. #23951
- Passes
automatic_ddl_mode
to xCluster pollers for enhanced replication handling. #24091 - Adds a metric to alert users when the master follower heartbeat delay is too high. #21178
- Removes the need to set
set_cdc_min_replicated_index
during xCluster bootstrap for improved log retention. #24105 - Enables SimSIMD in Usearch for enhanced search operations. #23998
- Reduces thread usage by removing CDC-specific
yb_client
fromPgClientServiceImpl
. #24145 - Adds a simple test to verify Vector LSM functionality during and post-insertions. #24242
- Adds a
yb-admin
command to remove non-active, blacklisted TServers. #24044 - Skips schema validation for newly added tables in xCluster replication to handle schema changes. #23078
- Standardizes log prefixes for retryable requests with tablet and peer IDs. #19516
- Adds a
yb-admin
command to remove tablet servers from the registry. #24044 - Enables persistent TServer registry updates via autoflag configuration. #24045
- Enables persistent storage for vector LSM via separate index chunk files and meta files. #24371
- Enhances Vector LSM handling to maintain data integrity after unexpected restarts. #24466
- Issues NOTICE when creating indexes in bi-directional xCluster to ensure concurrent execution. #24362
- Reduces Python virtual environment changes using
yugabyte-bash-common
for top-level management. #19478 - Renames the vector library to vector_index and updates the namespace accordingly. #24636
- Enables selecting compaction files at task start with
rocksdb_determine_compaction_input_at_start
flag. #24541 - Creates vector index tablets colocated with the indexed table for effective data retrieval. #24696
- Enables direct replication of specific DDL commands in xCluster. #23953
- Reverts a change that issued NOTICE on index creation in bi-directional xCluster setups. #24362
- Enables creation of Vector LSM for tablets with vector indices. #24892
- Reduces argument redundancy in PgsqlReadOperation executions. #24855
- Enables safe index creation on bidirectional xCluster tables without halting operations. #24884
- Replaces
ProducerSchemaPB
withSchemaVersionsPB
for xCluster configurations. #24901 - Supports DDL replication for additional PostgreSQL objects including foreign tables and text search configurations. #25051
- Enables ALTER operations on colocated tables with active replication. #24910
- Adds packed row support for vector indexes during transaction apply. #24912
- Fetches intents DB for vector index queries to account for unapplied committed transactions. #24947
- Enables building with third party PR artifacts using specific GitHub IDs. #25089
- Enables vector indexes on colocated tables. #24994
- Allows vector index inserts to use a dedicated thread pool, enhancing overall system performance. #25029
- Replaces
vector_index::VertexId
withStronglyTypedUuid
asVectorId
. #25038 - Limits concurrent reads on usearch vector index to avoid exceptions. #25069
- Enables file determination at compaction start by default. #25113
- Enhances stability by fixing a missing null check in tablet lookup processes. #25145
- Removes deprecated vector index storage in RocksDB. #24854
- Removes outdated code to streamline heartbeat processing. #24872
- Simplifies load balancer's placement info management. #25065
- Updates lz4 library to fix a specific compression test failure. #24965
- Increases
RollbackYsqlMajorVersionUpgrade
RPC timeout to 3 minutes. #24361 - Adds race conditions in the odyssey library to the TSAN suppressions list. #24735
- Reduces data race when changing RocksDB options during auto-compactions. #24238
- Fixes deadlock issue by adjusting lock acquisition order in HierarchicalNSW. #25017
- Ensures tasks don't hang during shutdown by removing unnecessary checks. #17898
- Rejects alter table requests during table preparation to prevent state inconsistencies. #25130
CDC
- Enables asynchronous removal of user tables from CDC streams by adjusting how background threads process and persist stream metadata. #23700
- Adds a tag for the slot name attribute in the CDC metrics. #24307
- Enables tablet splitting on tables under CDCSDK stream by default using the GFLAG
enable_tablet_split_of_cdcsdk_streamed_tables
. #24190 - Avoids caching IntentDB block reads for CDC to prevent redundant data access. #24909
- Automatically removes metrics when a CDC stream is deleted. #24876
- Speeds up tablet bootstrap under CDC by skipping initial WAL segments, controlled by
skip_wal_replay_from_beginning_with_cdc
flag. #24516 - Enables
kNoCacheQueryId
to prevent crashes during RocksDB operations. #24516 - Speeds up tablet bootstrap under CDC by fixing WAL replay. #24516
yugabyted
- Enables better handling of complex YSQL configuration parameters using sticky connections. #21637
- Stops sending gflags details in the callhome diagnostics to eliminate redundant data. #24029
- Ensures the user interface accurately displays disk size, even when multiple data directories are used. #23810
- Ensures
join
flag in config files is properly validated to prevent node misconfiguration. #23007 - Updates column names in
pg_stat_statements
to fix slow queries page on yugabyted UI. #25094 - Enables
tserver_master_addresses
to update immediately in multi-node deployments usingYb-ts-cli
. #24659 - Adds fully qualified names for non-table SQL objects in the UI. #24330, #24331
- Lists manually refactorable objects by type in the YugabyteD UI. #24329
Bug fixes
YSQL
- Introduces a per-database PG new OID allocator, ensuring OID uniqueness within the database and enhancing horizontal scalability in multi-node and multi-tenancy environments. This new mechanism mitigates OID collisions and allows OID consistency in backup-restore scenarios across clusters. A new GFlag
ysql_enable_pg_per_database_oid_allocator
is provided to return to old OID allocator behavior if necessary. #16130 - Resets
current_hint_retrieved
after query execution to ensure consistent hint parsing. #12741 - Modifies backup/restore process to skip column name checks for indexes, allowing for successful restoration even with renamed columns. #24207
- Enables
ALTER SEQUENCE
commands to change sequence name, schema, and owner without errors. #17271 - Enhances PostgreSQL restart stability by refining shutdown handling and lock file management. Includes killing only Postgres-associated PIDs, using
SIGQUIT
for immediate shutdowns to reduce hang-ups, and monitoring for stuck processes on macOS #24396 - Fixes crashes caused by using a RowComparisonExpression on a reordered primary key index. #23824
- Disables in-place index updates by default to address index inconsistency issues. #24672
- Increases YSQL connection manager's thread stack size to 512 KB to prevent crashes on Alma 9 machines. #24979
- Enhances read performance on colocated tables by skipping redundant tombstone checks. #21561
- Fixes flag overlap to differentiate
sticky
settings fromexplain
in configurations. #24954 - Renames
pipe
type toorafce_pipe
to resolve naming conflicts. #12990 - Eliminates unnecessary file creation for views on temporary tables by checking if storage is actually needed. #19522
- Reduces maintenance time by switching to a less complex implementation of SideBySideDiff.java, thereby eliminating errors from
SideBySideDiff.sanityCheckLinesMatch
. #19690 - Prevents segmentation faults in parallel index scans with aggregate pushdown. #21427
- Increases performance by batching index reads during
INSERT ON CONFLICT
, with customizable batch sizes usingyb_insert_on_conflict_read_batch_size
. #24179 - Improves cache re-invalidation for
ALTER TABLE
commands to avoid schema version mismatch errors in the same session. #23882 - Fixes UPDATE errors on partitioned tables by correctly mapping column numbers. #23857
- Ensures restart read time does not exceed the global limit to avoid inconsistent reads. #24017
- Fixes the flaw in the current DDL atomicity workflow where only the first table's schema was compared, even if it doesn't change after a schema version increment. #23988
- Streamlines
ysql_dump
process by eliminating faulty statements for colocated table unique constraint, enhancing database reliability. #24057 - Introduces batching for the
INSERT ON CONFLICT
command to enhance performance by minimizing alternating read and write operations. Control the batch size with the new YSQL configuration parameteryb_insert_on_conflict_read_batch_size
, which, when default at1
, disables batching. #24179 - Enhances index relation checks with
YBIsCoveredByMainTable
method. #24239 - Alters encoding setup order to ensure
UTF8
use instead ofSQL_ASCII
and addspg_collation
to pre-loaded tables to prevent specific errors. #24149 - Disables LZ4 build option to prevent PostgreSQL build failures. #24286
- Fixes crashes in creating tables with over 585 columns. #24304
- Adds support for
ALTER TABLE VALIDATE CONSTRAINT
in YugabyteDB. #24291 - Prevent incorrect loading of pg_collation catalog caches, eliminating associated performance issues. #24149
- Fixes out-of-bound memory access during index scan cost estimation ensuring stable query execution on system catalogs and expression-based indexes when using the
yb_enable_base_scans_cost_model
configuration #24246 - Blocks
REINDEX
andTRUNCATE
on system tables using table rewrite. #24356 - Blocks the "NULLS NOT DISTINCT" command until supported. #24452
- Enhances consistency and correctness of INSERT...ON CONFLICT queries via YBCTID infrastructure integration and index management. #24692, #24784
- Assumes indexes contain 1000 rows when statistics are unavailable, ensuring better query planning. #24496
- Enhances performance during nested loop joins by ensuring correct limit resets in colocated index scans. #24560
- Removes time sync service requirement from the prerequisites. #24758
- Enhances UPDATE query execution on generated columns by including them in
updatedCols
calculations. #24641 - Fixes foreign key checks to avoid errors with unique indexes. #24663
- Ensures accurate catalog version tracking during concurrent index creation. #24775
- Ensures YB master reads from valid catalog tables during upgrades. #24647
- Re-enables in-place index updates by default after fixing related inconsistencies. #24849
- Prevents DML operations on sequence data during upgrades. #24943
- Fixes
ysql_dump
crashes related topg_catalog
schema dumps. #24907 - Accurately estimates the cost of partial index scans by including index conditions. #24916
- Reduces memory usage spikes by using a new memory context for
RelationBuildTriggers
. #24951 - Adds debugging logs to track new OIDs for resolving table OID collisions. #24320
- Ensures successful cache invalidation after a failed ALTER TABLE to prevent "schema mismatch" errors. #24981
- Disables index update optimization by default to fix
ON CONFLICT UPDATE
issues. #25075 - Reverts changes related to
yb_amiscoveredbymaintable
for updated index co-partitioning design. #25119
YCQL
- Allows users to control error suppression for
GROUP BY
in YCQL usingycql_ignore_group_by_error
flag. #25030 - Eliminates unexpected error during shell type definition update across multiple sessions. #24217
DocDB
- Allows the packed row size to exceed its limit during repacking, preventing TServer crash after
alter table add column
commands with a default value. #24050 - Prevents server crashes during table truncation by safely shutting down iterators before releasing locks. #23243
- Prevents resource waste by checking disk space before remote bootstrap operations. #23987
- TServers now reject remote bootstrap requests for previously deleted tablets. #24574
- Sets a default 10MB max size for RocksDB MANIFEST files, now configurable via flags. #11117
- Adds mtime and checksum to block cache key prefix to prevent conflicts. #20852
- Enhances visibility for unexpected rate limiter sleep times in RBS threads. #24342
- Eliminates inconsistencies by not including inactive tablets in
OpenTable
operations. #24860 - Adds a null check to prevent a system crash when cleaning up a recently applied transaction state after a flush operation. #24026
- Prevents crashes during compaction when
ysql_enable_packed_row
is off and existing packed rows are present. #24545 - Updates YSQL migration script to align with PostgreSQL 15 catalog changes. #24593
- Switches pg_cron back to using standard
SIGTERM
handling for cleaner shutdowns. #24658 - Ensures tablets remain effectively managed during transition from RF1 to higher replication factors. #24575
- Ensures consistent tablegroup assignments for colocated table backups without using explicit tablespaces. #24809
- Fixes error for select queries in large tables by adjusting backward scan bounds. #24824
- Eliminates segmentation faults by ensuring
CleanupHiddenTables
holds a write lock duringUpsert
. #24929 - Prevents fatal errors during multiple splits and PITR restores. #25103
- Ensures "create table" requests for colocated tables don't fail due to mistaken tablet limit checks. #23922
- Reduces unnecessary checks when adding tables in alter replication requests. #24164
- Enhances stability by replacing raw pointers with
std::weak_ptr
in tablet server handling. #24182 - Enhances index parameter consistency between Usearch and Hnswlib for balanced performance. #24465
- Ensures correct metadata sequencing for vector index chunks, preventing data inconsistencies. #24511
- Speeds up vector LSM loading by using a thread pool. #24588
- Adds a flag to limit multiple pending compactions for priority pools. #24540
- Enables merging of small vector index chunks to enhance search efficiency. #24069
- Resolves scheduling of contentious waiters to the correct, larger RPC thread pool. #24478
- Fixes shutdown path issues to prevent fatal errors during RPC completion. #24524
- Ensures
pg_txn_start_us
is set for restarted transactions, fixing priority issues in waiter resumption. #24778 - Fixes mini cluster restarts to ensure master leaders track all TServer re-connections. #24772
- Reduces latency by enhancing early resume of RPCs blocked on aborted transactions. #24799
- Fixes memory leak in MasterPathHandlers by properly managing shared pointers. #24823
- Ensures callback triggers after loading tablet server data if the count meets requirements. #23744
- Ensures consistent reads by verifying restart conditions before reporting false corruption. #25072
CDC
- Allows sending a null "before image" for CDC if
cdc_send_null_before_image_if_not_exists
flag is true. #18381 - Enhances accuracy of transaction start time logging for better consistency in CDC. #24521
- Fixes memory leaks in the walsender's handling of transactions and file nodes. #24788
- Fixes memory leaks in the walsender's handling of transactions and file nodes. #24788
- Ensures null values are now correctly captured in CDC before image records. #22983
- Enables removal of dropped tables from the replica identity map to prevent replication errors. #24308
- Excludes expired or irrelevant entries from metrics calculations for more accurate reporting. #24975
yugabyted
- Nodes now start using private IP as
advertise_address
instead of hostname. #25037