yb-voyager CLI reference

yb-voyager is a command line executable for migrating databases from PostgreSQL, Oracle, and MySQL to a YugabyteDB database.

Syntax

yb-voyager [ <migration-step>... ] [ <arguments> ... ]

• migration-step: See Commands
• arguments: See Arguments

Command line help

To display the available online help, run:

yb-voyager --help

To display the available online help for any migration step, run:

yb-voyager [ <migration-step>... ] --help

Version check

To verify the version of yb-voyager installed on your machine, run:

yb-voyager version

Commands

The following command line options specify the migration steps.

export schema

Export the schema from the source database.

Syntax

yb-voyager export schema [ <arguments> ... ]

The valid arguments for export schema are described in the following table:

Example

yb-voyager export schema --export-dir /path/to/yb/export/dir \
        --source-db-type sourceDB \
        --source-db-host localhost \
        --source-db-port port \
        --source-db-user username \
        --source-db-password password \ # Enclose the password in single quotes if it contains special characters.
        --source-db-name dbname \
        --source-db-schema schemaName \ # Not applicable for MySQL
        --start-clean

analyze-schema

Analyse the PostgreSQL schema dumped in the export schema step.

Syntax

yb-voyager analyze-schema [ <arguments> ... ]

The valid arguments for analyze schema are described in the following table:

Example

yb-voyager analyze-schema --export-dir /path/to/yb/export/dir --output-format txt

export data

For offline migration, export data dumps the source database to the machine where yb-voyager is installed.

For live migration (and fall-forward), export data dumps the snapshot in the data directory and starts capturing the new changes made to the source database.

Syntax

yb-voyager export data [ <arguments> ... ]

The valid arguments for export data are described in the following table:

Example

yb-voyager export data --export-dir /path/to/yb/export/dir \
        --source-db-type sourceDB \
        --source-db-host hostname \
        --source-db-port port \
        --source-db-user username \
        --source-db-password password \ # Enclose the password in single quotes if it contains special characters.
        --source-db-name dbname \
        --source-db-schema schemaName # Not applicable for MySQL

export data status

For offline migration, get the status report of an ongoing or completed data export operation.

For live migration (and fall-forward), get the report of the ongoing export phase which includes metrics such as the number of rows exported in the snapshot phase, the total number of change events exported from the source, the number of INSERT/UPDATE/DELETE events, and the final row count exported.

Syntax

yb-voyager export data status [ <arguments> ... ]

The valid arguments for export data status are described in the following table:

Example

yb-voyager export data status --export-dir /path/to/yb/export/dir

import schema

Import the schema to the YugabyteDB database.

During migration, run the import schema command twice, first without the --post-import-data argument and then with the argument. The second invocation creates indexes and triggers in the target schema, and must be done after import data is complete.

Syntax

yb-voyager import schema [ <arguments> ... ]

The valid arguments for import schema are described in the following table:

Example

yb-voyager import schema --export-dir /path/to/yb/export/dir \
        --target-db-host hostname \
        --target-db-user username \
        --target-db-password password \ # Enclose the password in single quotes if it contains special characters.
        --target-db-name dbname \
        --target-db-schema schemaName # MySQL and Oracle only

import data

For offline migration, Import the data to the YugabyteDB database.

For live migration (and fall-forward), the command imports the data from the data directory to the target database, and starts ingesting the new changes captured by export data to the target database.

Syntax

yb-voyager import data [ <arguments> ... ]

The valid arguments for import data are described in the following table:

Example

yb-voyager import data --export-dir /path/to/yb/export/dir \
        --target-db-host hostname \
        --target-db-user username \
        --target-db-password password \ # Enclose the password in single quotes if it contains special characters.
        --target-db-name dbname \
        --target-db-schema schemaName \ # MySQL and Oracle only
        --parallel-jobs connectionCount

import data file

Load data from files in CSV or text format directly to the YugabyteDB database. These data files can be located either on a local filesystem, an AWS S3 bucket, GCS bucket, or an Azure blob. For more details, see Bulk data load from files.

Syntax

yb-voyager import data file [ <arguments> ... ]

The valid arguments for import data file are described in the following table:

Example

yb-voyager import data file --export-dir /path/to/yb/export/dir \
        --target-db-host hostname \
        --target-db-port port \
        --target-db-user username \
        --target-db-password password \ # Enclose the password in single quotes if it contains special characters.
        --target-db-name dbname \
        --target-db-schema schemaName \ # MySQL and Oracle only
        --data-dir "/path/to/files/dir/" \
        --file-table-map "filename1:table1,filename2:table2" \
        --delimiter "|" \
        --has-header \
        --file-opts "escape_char=\",quote_char=\"" \
        --format format

import data status

For offline migration, get the status report of an ongoing or completed data import operation. The report contains migration status of tables, number of rows or bytes imported, and percentage completion.

For live migration, get the status report of import data. For live migration with fall forward, the report also includes the status of fall forward setup. The report includes the status of tables, the number of rows imported, the total number of changes imported, the number of INSERT, UPDATE, and DELETE events, and the final row count of the target or fall-forward database.

Syntax

yb-voyager import data status [ <arguments> ... ]

The valid arguments for import data status are described in the following table:

Example

yb-voyager import data status --export-dir /path/to/yb/export/dir

fall-forward setup

Imports data to the fall-forward database, and streams new changes from the YugabyteDB database to the fall-forward database.

Syntax

yb-voyager fall-forward setup [ <arguments> ... ]

The valid arguments for fall-forward setup are described in the following table:

Example

yb-voyager fall-forward setup --export-dir /path/to/yb/export/dir \
        --ff-db-host hostname \
        --ff-db-user username \
        --ff-db-password password \ # Enclose the password in single quotes if it contains special characters.
        --ff-db-name dbname \
        --ff-db-schema schemaName \
        --parallel-jobs connectionCount

fall-forward synchronize

Exports new changes from the YugabyteDB database to import to the fall-forward database so that the fall-forward database can be in sync with the YugabyteDB database after cutover.

Syntax

yb-voyager fall-forward synchronize [ <arguments> ... ]

The valid arguments for fall-forward synchronize are described in the following table:

Example

yb-voyager fall-forward synchronize --export-dir /path/to/yb/export/dir \
        --target-db-host hostname \
        --target-db-port port \
        --target-db-user username \
        --target-db-password password \ # Enclose the password in single quotes if it contains special characters.
        --target-db-name dbname \
        --target-db-schema schemaName

cutover initiate

Initiate cutover to the YugabyteDB database.

Syntax

yb-voyager cutover initiate [ <arguments> ... ]

The valid arguments for cutover initiate are described in the following table:

Example

yb-voyager cutover initiate --export-dir /path/to/yb/export/dir

cutover status

Shows the status of the cutover to the YugabyteDB database. Status can be "INITIATED", "NOT INITIATED", or "COMPLETED".

Syntax

yb-voyager cutover status [ <arguments> ... ]

The valid arguments for cutover status are described in the following table:

Example

yb-voyager cutover status --export-dir /path/to/yb/export/dir

fall-forward switchover

Initiate a switchover to the fall-forward database.

Syntax

yb-voyager fall-forward switchover [ <arguments> ... ]

The valid arguments for fall-forward switchover are described in the following table:

Example

yb-voyager fall-forward switchover --export-dir /path/to/yb/export/dir

fall-forward status

Shows the status of the fall-forward switchover to the fall-forward database. Status can be "INITIATED", "NOT INITIATED", or "COMPLETED".

Syntax

yb-voyager fall-forward status [ <arguments> ... ]

The valid arguments for fall-forward switchover status are described in the following table:

Example

yb-voyager fall-forward status --export-dir /path/to/yb/export/dir \

archive changes

Archives the streaming data from the source database.

Syntax

yb-voyager archive changes [ <arguments> ... ]

The valid arguments for archive changes status are described in the following table:

Arguments

--export-dir

Specifies the path to the directory containing the data files to export.

An export directory is a workspace used by yb-voyager to store the following:

• exported schema DDL files
• export data files
• migration state
• log file

--source-db-type

Specifies the source database type (postgresql, mysql, or oracle).

--source-db-host

Specifies the domain name or IP address of the machine on which the source database server is running.

--source-db-port

Specifies the port number of the machine on which the source database server is running.

Default: 5432 (PostgreSQL), 3306 (MySQL), and 1521 (Oracle)

--source-db-user

Specifies the username of the source database.

--source-db-password

Specifies the password of the source database.

If you don't provide a password via the CLI during any migration phase, yb-voyager will prompt you at runtime for a password.

Alternatively, you can also specify the password by setting the environment variable SOURCE_DB_PASSWORD.

If the password contains special characters that are interpreted by the shell (for example, # and $), enclose it in single quotes.

--source-db-name

Specifies the name of the source database.

--source-db-schema

Specifies the schema name of the source database. Not applicable for MySQL.

For Oracle, you can specify only one schema name using this option.

For PostgreSQL, you can specify a list of comma-separated schema names.

Case-sensitive schema names are not yet supported. Refer to Importing with case-sensitive schema names for more details.

--output-format

Specifies the format in which the report file is generated. It can be in html, txt, json, or xml.

--target-db-host

Specifies the domain name or IP address of the machine on which target database server is running.

--target-db-port

Specifies the port number of the machine on which the target database server is running.

Default: 5433

--target-db-user

Specifies the username in the target database to be used for the migration.

--target-db-password

Specifies the password for the target database user to be used for the migration.

Alternatively, you can also specify the password by setting the environment variable TARGET_DB_PASSWORD.

If you don't provide a password via the CLI during any migration phase, yb-voyager will prompt you at runtime for a password.

If the password contains special characters that are interpreted by the shell (for example, # and $), enclose the password in single quotes.

--target-db-name

Specifies the name of the target database.

Default: yugabyte

--target-db-schema

Specifies the schema name of the target database. MySQL and Oracle migrations only.

--parallel-jobs

For export data

Specifies the number of tables to be exported in parallel from the source database at a time.

Default: 4; exports 4 tables at a time by default.

If you use BETA_FAST_DATA_EXPORT to accelerate data export, yb-voyager exports only one table at a time and the --parallel-jobs argument is ignored.

For import data

Specifies the number of parallel COPY commands issued to the target database.

Depending on the YugabyteDB database configuration, the value of --parallel-jobs should be tweaked such that at most 50% of target cores are utilised.

Default: If yb-voyager can determine the total number of cores N in the YugabyteDB database cluster, it uses N/2 as the default. Otherwise, it defaults to twice the number of nodes in the cluster.

--batch-size

Specifies the number of rows in each batch generated for ingestion during import data.

Default: 20000; rows.

--data-dir

Path to the directory containing the data files to import. You can also provide the URI of an AWS S3 bucket, GCS bucket, or an Azure blob. For more details, see Bulk data load from files.

--file-table-map

Comma-separated mapping between the files in data-dir to the corresponding table in the database.

Example: filename1:tablename1,filename2:tablename2[,...]

You can import multiple files in one table either by providing one <fileName>:<tableName> entry for each file OR by passing a glob expression in place of the file name. For example, fileName1:tableName,fileName2:tableName OR fileName*:tableName.

--delimiter

Character used as delimiter in rows of the table(s).

Default: comma (,) for CSV file format and tab (\t) for TEXT file format.

--escape-char

Escape character; only applicable to CSV file format.

Default: double quotes (")

--quote-char

Quote character; only applicable to CSV file format.

Default: double quotes (")

--has-header

This argument is to be specified only for CSV file type.

Default: false; change to true if the CSV file contains column names as a header.

Note: Boolean flags take arguments in the format --flag-name=[true|false], and not --flag-name [true|false].

--file-opts

[Deprecated] Comma-separated string options for CSV file format. The options can include the following:

• escape_char: escape character

• quote_char: character used to quote the values

Default: double quotes (") for both escape and quote characters

Note that escape_char and quote_char are only valid and required for CSV file format.

Example: --file-opts "escape_char=\",quote_char=\"" or --file-opts 'escape_char=",quote_char="'

--null-string

String that represents null value in the data file.

Default: ""(empty string) for CSV, and '\N' for text.

--format

Specifies the format of your data file with CSV or text as the supported formats.

Default: CSV

--post-import-data

Run this argument with import schema command to import indexes and triggers in the YugabyteDB database after data import is complete. The --post-import-data argument assumes that data import is already done and imports only indexes and triggers in the YugabyteDB database .

--oracle-db-sid

Oracle System Identifier (SID) you can use while exporting data from Oracle instances. Oracle migrations only.

--oracle-home

Path to set $ORACLE_HOME environment variable. tnsnames.ora is found in $ORACLE_HOME/network/admin. Not applicable during import phases or analyze schema. Oracle migrations only.

--use-orafce

Enable using orafce extension in export schema. Oracle migrations only.

Default: true

--yes

By default, answer yes to all questions during migration.

--start-clean

For the export phase, starts a fresh schema export during export schema after clearing the schema directory, and a fresh data export during export data after clearing data directory.

For the import phase, starts a fresh import of schema during import schema on the target for the schema present in the schema directory.

For import data or import data file, the flag starts a fresh import with data files present in the data directory and if any table on the target database is non-empty, it prompts whether you want to continue the import without the truncating those tables. If yes, then yb-voyager starts ingesting the data present in the data files with upsert mode.

For tables with no primary key, you should exclude them using --exlcude-table-list to avoid duplicate data, if any, or truncate those tables manually before using the start-clean flag.

--table-list

Comma-separated list of the tables for which data needs to be migrated. Do not use in conjunction with --exclude-table-list.

--exclude-table-list

Comma-separated list of tables to exclude while migrating data.

--send-diagnostics

Controls whether to send diagnostics information to Yugabyte.

Default: true

--verbose

Displays extra information in the output.

Default: false

--disable-pb

Use this argument to not display progress bars. For live migration, --disable-pb can also be used to hide metrics for export data or import data.

Default: false

--ff-db-host

Specifies the domain name or IP address of the machine on which Fall-forward database server is running.

--ff-db-name

Specifies the name of the fall-forward database.

--ff-db-password

Specifies the password for the fall-forward database user to be used for the migration. Alternatively, you can also specify the password by setting the environment variable FF_DB_PASSWORD.

If you don't provide a password via the CLI or environment variable during any migration phase, yb-voyager will prompt you at runtime for a password.

If the password contains special characters that are interpreted by the shell (for example, # and $), enclose the password in single quotes.

--ff-db-port

Specifies the port number of the machine on which the fall-forward database server is running.

Default: For Oracle databases, 1521

--ff-db-schema

Specifies the schema name of the fall-forward database.

--ff-db-user

Specifies the username in the Fall-forward database to be used for the migration.

--export-type

Specifies the type of migration [snapshot-only (offline), snapshot-and-changes(live, and optionally fall-forward) ]

Default: snapshot-only

--delete

Deletes the exported data in the CDC phase after moving it to another location.

Default: false

--fs-utilization-threshold

Specifies disk utilization percentage after which you can archive changes.

--move-to

Specifies the destination path to which archive changes should move the files to.

SSL connectivity

You can instruct yb-voyager to connect to the source or target database over an SSL connection. Connecting securely to PostgreSQL, MySQL, and YugabyteDB requires you to pass a similar set of arguments to yb-voyager. Oracle requires a different set of arguments.

The following table summarizes the arguments and options you can pass to yb-voyager to establish an SSL connection.