Key concepts EARLY ACCESS

PostgreSQL logical replication concepts

The YugabyteDB logical replication feature uses PostgreSQL Logical Replication, which operates using a publish-subscribe model. Understanding the following key concepts will help you set up and manage a logical replication environment effectively.

Concepts

Replication slot

A replication slot represents a stream of changes that can be replayed to a client in the order they were made on the origin server. Each slot streams a sequence of changes from a single database.

In logical replication, the fundamental unit of data transmission is a transaction. A logical slot emits each change just once in normal operation. The current position of each slot is persisted only at checkpoint, so if a replication process is interrupted and restarts, even if the checkpoint or the starting Log Sequence Number (LSN) falls in the middle of a transaction, the entire transaction is retransmitted. This behavior guarantees that clients receive complete transactions without missing any intermediate changes, maintaining data integrity across the replication stream​. Logical decoding clients are responsible for avoiding ill effects from handling the same message more than once. Clients may wish to record the last LSN they saw when decoding and skip over any repeated data or (when using the replication protocol) request that decoding start from that LSN rather than letting the server determine the start point.

For more information, refer to Replication slots in the PostgreSQL documentation.

LSN type

A Log Sequence Number (LSN) in YugabyteDB differs from what you may be accustomed to in PostgreSQL. In PostgreSQL, an LSN represents a specific 'location' in the WAL, and has significance that spans databases and replication slots. In YugabyteDB, an LSN uniquely identifies a change event, and the LSN is valid only in the context of a specific replication slot. Due to these differences, there are inherent limitations in how LSNs can be used.

You can specify the type of LSN to use when you create a replication slot. YugabyteDB currently supports the following types:

  • SEQUENCE - (Default) PostgreSQL-style LSN that is valid in the context of a slot. It is a monotonic increasing number that determines the record in global order in the context of a slot. It can't be compared across two different slots.
  • HYBRID_TIME - A hybrid time value which can be used natively with YugabyteDB. HYBRID_TIME is denoted by the HybridTime of the transaction commit record. All the records of the transaction that is streamed will have the same LSN as that of the commit record. You need to ensure that the changes of a transaction are applied in totality and the acknowledgement is sent only if the commit record of a transaction is processed.

Slot scope

A replication slot belongs to one database and is not tied to a publication. The client names the publication when it starts streaming (the publication_names option of START_REPLICATION), so the same slot can be started with different publications over time, as in PostgreSQL.

When you create a slot, YugabyteDB takes a consistent snapshot of the database and starts retaining changes for every table in the database, not only for the tables in a publication. Tables created after the slot are added to the slot automatically. The publication only selects which of those tables are streamed to the client.

If the client starts streaming without naming a publication, YugabyteDB streams the changes of all tables in the database, as if a FOR ALL TABLES publication had been used. This applies to test_decoding and wal2json. pgoutput and yboutput require the publication_names option and fail without it, as in PostgreSQL.

Tables of interest

Retaining changes for a table has a cost. Every tablet of the table keeps its WAL and transaction intents until the slot consumes them, and compaction is held back if the slot needs before images. To avoid paying that cost for tables a slot never reads, YugabyteDB tracks whether each tablet is of interest to the slot:

  • A tablet becomes of interest the first time the slot polls it. The slot polls every tablet of every table in the publication, whether or not the table has changes.
  • A tablet that has not been polled within cdcsdk_tablet_not_of_interest_timeout_secs (default 4 hours) of slot creation is marked as not of interest. Its retention barriers are released, and the slot stops tracking it.
  • Tablets of tables created after the slot are always of interest.

A table whose tablets are not of interest, or have expired, is unqualified for the slot. This has the following implications:

  • Every table the slot will ever stream must be polled within 4 hours of slot creation, so it must be in a publication the client is streaming during that window. A table not polled in that window becomes unqualified: its changes are no longer retained for the slot, and adding it to the publication later does not bring it back. To stream such a table, create a new slot.

  • Start streaming within 4 hours of creating a slot. If no client connects in that window, every table becomes unqualified and the slot can't be used.

  • If the publication contains an unqualified table, at start or after an ALTER PUBLICATION ... ADD TABLE while streaming, streaming fails with the following error:

    Cannot add tablet: <id> to the polling list as it has been unqualified for stream: <id>
    

    Either remove the table from the publication, or set the cdc_skip_unqualified_tables_for_polling flag (v2025.2.7.0, v2026.1.2.0, and later) so that the slot skips unqualified tables and streams the rest. If this is not resolved within cdc_intent_retention_ms, the slot expires.

Restart time

The restart time of a slot is the commit time from which streaming resumes when the client reconnects. It is also the point up to which the database must retain changes for the slot. pg_replication_slots reports it as restart_lsn and as yb_restart_commit_ht (a hybrid time). From v2025.1, yb_restart_time reports the same value as a timestamp. confirmed_flush_lsn is the last LSN the client acknowledged.

The restart time advances in the following cases:

  • When the client acknowledges an LSN by reporting it as flushed in its feedback message. The restart time moves to the commit of the latest transaction whose COMMIT record is at or before the acknowledged LSN. Acknowledging an LSN in the middle of a transaction does not move the restart time past the previous commit. With the HYBRID_TIME LSN type, acknowledge only the LSN of a COMMIT record, after applying the whole transaction.
  • When the slot is idle. If the client has acknowledged everything it received and there is nothing new to stream, the restart time advances to the slot's safe time every 60 seconds, so that an idle slot does not accumulate retained changes. This is controlled by the cdcsdk_update_restart_time_when_nothing_to_stream and cdcsdk_update_restart_time_interval_secs flags.

The restart time does not advance while the client is disconnected, or while a connected client receives changes without acknowledging them. In both cases the retained WAL and intents keep growing on every tablet the slot polls.

The slot state is stored in the cdc_state system table together with a checkpoint for each tablet, so it survives client reconnections, walsender restarts, and node failures.

Disconnection and reconnection

When the client disconnects, the walsender exits, and the in-memory state of the slot on the YB-TServer is discarded. The slot itself, its restart time, and the per-tablet checkpoints remain. After 60 seconds without consumption (the ysql_cdc_active_replication_slot_window_ms time), pg_replication_slots shows the slot as not active.

While the client is disconnected, every tablet the slot polls keeps retaining WAL, transaction intents, and, if the slot needs before images, row history from the slot's restart time. Nothing is released before the client reconnects and acknowledges changes, unless the slot expires first. The retention lasts up to cdc_intent_retention_ms (default 8 hours), measured from the last successful poll of each tablet. After that the slot expires; see When a slot becomes unusable. The matching WAL retention is set by cdc_wal_retention_time_secs (default 8 hours), which must be at least as long.

When the client reconnects, streaming resumes from the restart time. Every transaction committed after the restart time is sent again, in full, including transactions the client had received, and possibly acknowledged, if the server did not process the acknowledgment before the disconnection. The records keep the LSNs they had before (see Determinism), so the client can detect and skip duplicates.

When a slot becomes unusable

A slot stops working in the following cases:

  • Expired. A tablet of the slot was not polled for cdc_intent_retention_ms (default 8 hours), for example because the client was disconnected. The retention barriers are released, intents are garbage collected, and the next poll fails with Stream ID <id> is expired for Tablet ID <id>. From v2025.1, pg_replication_slots shows wal_status as lost when no tablet of the slot has been polled for that long. The column does not reflect the expiry of individual tablets or tables that are not of interest, so a slot can show reserved and still fail to start.
  • Unqualified table in the publication. A table that is not of interest is in the publication, at start or added while streaming. Streaming fails unless the table is removed from the publication or cdc_skip_unqualified_tables_for_polling is set.
  • Database dropped. Dropping the database drops its slots.

An expired slot can't be recovered. Drop the slot and create a new one. The new slot takes a fresh snapshot of the database, so the consumer must take a new initial snapshot as well.

For how this behavior has changed across releases and how it compares with PostgreSQL, see Slot behavior by version.

Publication

A publication is a set of changes generated from a table or a group of tables, and might also be described as a change set or replication set. Each publication exists in only one database.

Publications are different from schemas and do not affect how the table is accessed. Each table can be added to multiple publications if needed. Publications may currently only contain tables. Objects must be added explicitly, except when a publication is created for ALL TABLES.

Changing a publication while a slot is streaming takes effect at commit time with implicit publication (the default in v2026.1 and later), or at the next refresh with periodic publication (the only mode in earlier releases). The mode is recorded in the slot when it is created, based on the ysql_yb_enable_implicit_dynamic_tables_logical_replication flag, and doesn't change afterward. Slots created before upgrading to v2026.1 keep using periodic publication; to use implicit publication, create a new slot. See Adding tables to publication for details, and Slot behavior by version for the versions.

For more information, refer to Publication in the PostgreSQL documentation.

Output plugin

Output plugins transform the data from the write-ahead log's internal representation into the format that can be consumed by replication clients. These plugins are notified about the change events that need to be processed and sent via various callbacks. These callbacks are only invoked when the transaction actually commits.

YugabyteDB supports the following four output plugins:

  • yboutput
  • pgoutput
  • test_decoding
  • wal2json

All these plugins are pre-packaged with YugabyteDB and do not require any external installation.

Note

The plugin yboutput is YugabyteDB specific. It is similar to pgoutput in most aspects. The only difference being that replica identity CHANGE is not supported in pgoutput. All other plugins support replica identity CHANGE.

For more information, refer to Logical Decoding Output Plugins in the PostgreSQL documentation.

LSN

LSN (Log Sequence Number) in YugabyteDB is an unsigned 64-bit integer that uniquely identifies a change record or a transaction boundary record that is consumed from a given replication slot.

In YugabyteDB, LSN values from different slots are considered unrelated and should not be compared. In YugabyteDB, LSN no longer represents the byte offset of a WAL record.

LSN values for a single replication slot satisfy the following properties:

  • Uniqueness

    LSN values for the change and COMMIT records for a given replication slot are unique. In particular, changes from different tablets of the same or different tables will have unique LSN values for a replication slot.

  • Ordering

    LSN values can be compared ( <, >, = ).

    The LSN of the change records in a transaction will be strictly lower than the LSN of the COMMIT record of the same transaction.

    The LSNs of change records in a transaction will be in increasing order and will correspond to the order in which those changes were made in that transaction. That is, the LSN of an earlier change will have a strictly lower value than the LSN of a later change in the same transaction. This is the case even if the changes correspond to rows in different tablets of the same or different tables.

    For a given replication slot, the LSN of a COMMIT record of an earlier transaction will be strictly lower than the LSN of the COMMIT record of a later transaction.

  • Determinism

    For a given replication slot, the LSN value of a change record (or a transaction boundary record) remains the same for the lifetime of that replication slot. In particular, this is true across server and client restarts and client re-connections. Thus, LSN values for a single replication slot may be used to uniquely identify records that are consumed from that replication slot. The values can be compared for determining duplicates at the client side.

Replica identity

Replica identity is a table-level parameter that controls the amount of information being written to the change records. YugabyteDB supports the following four replica identities:

  • CHANGE (default)
  • DEFAULT
  • FULL
  • NOTHING

The PostgreSQL replica identity INDEX is not supported in YugabyteDB.

Replica identity CHANGE is the best performant and the default replica identity. The replica identity of a table can be changed by performing an ALTER TABLE. However, for a given slot, any ALTER TABLE performed to change the replica identity after the creation of the slot will have no effect. This means that the effective replica identity for any table for a slot, is the replica identity of the table that existed at the time of slot creation. A dynamically created table (a table created after slot creation) will have the default replica identity. For a replica identity modified after slot creation to take effect, a new slot will have to be created after performing the ALTER TABLE.

The ysql_yb_default_replica_identity flag determines the default replica identity for user tables at the time of table creation. The default value is CHANGE. The purpose of this flag is to set the replica identities for dynamically created tables. In order to create a dynamic table with desired replica identity, the flag must be set accordingly and then the table must be created.

Advisory

You should refrain from altering the replica identity of a dynamically created table for at least 5 minutes after its creation.

For more information, refer to Replica Identity.

For information on replica identity in PostgreSQL, refer to REPLICA IDENTITY in the PostgreSQL documentation.

Replication protocols

PostgreSQL has defined protocols for replication that need to be followed by clients to establish replication connection as well as message structures for streaming data. This includes the Streaming Replication protocol and the Logical Streaming Replication protocol.

The logical streaming replication protocol sends individual transactions one-by-one. This means that all messages between a pair of BEGIN and COMMIT messages belong to the same transaction.

YugabyteDB supports both the streaming replication protocols used in PostgreSQL to support logical replication, maintaining the same semantics described in PostgreSQL:

  • Streaming Replication Protocol - This protocol is followed by all output plugins.

  • Logical Streaming Replication Protocol - This protocol is followed by pgoutput and yboutput, in addition to the Streaming replication protocol.

Note

YugabyteDB does not support Physical Replication.

Learn more

CDC using Logical Replication architecture