import data reference

yb-voyager import data

The following page describes the usage of the following import commands:

import data

For offline migration, Import the data to the YugabyteDB database.

For live migration (with fall-forward and fall-back), the import data command is an alias of import data to target which imports the data from the data directory to the target database, and starts ingesting the new changes captured by export data to the target database.

Syntax

Syntax for import data is as follows:

Usage: yb-voyager import data [ <arguments> ... ]

Syntax for import data to target is as follows:

Usage: yb-voyager import data to target [ <arguments> ... ]

Arguments

The valid arguments for import data are described in the following table:

Argument Description/valid options
--batch-size Size of batches in the number of rows generated for ingestion during import data.
Default: 20000 rows
Example: yb-voyager import data ... --batch-size 20000
--disable-pb Use this argument to disable progress bar or statistics during data import.
Default: false
Accepted parameters: true, false, yes, no, 0, 1
--enable‑upsert Enable UPSERT mode on target tables while importing data.
Note: Ensure that tables on the target YugabyteDB database do not have secondary indexes. If a table has secondary indexes, setting this flag to true may lead to corruption of the indexes.
Default: false
Usage for disabling the mode: yb-voyager import data ... --enable-upsert false
Accepted parameters: true, false, yes, no, 0, 1
--table-list Comma-separated list of names of source database tables whose data is to be imported. Table names can also be glob patterns containing wildcard characters, such as an asterisk (*) (matches zero or more characters) or question mark (?) (matches one character). To use a glob pattern for table names, enclose the list in single quotes ('').
For example, --table-list '"Products", order*'.
For example, --table-list '"Products", order*'.
This argument is unsupported for live migration.
--exclude‑table‑list Comma-separated list of names of source database tables for which data needs to be excluded during import. Table names follow the same convention as --table-list.
This argument is unsupported for live migration.
--table-list-file‑path Path of the file containing the list of names of source database tables (comma separated or line separated) to import. Table names use the same convention as --table-list.
‑‑exclude‑table‑list‑file‑path Path of the file containing the list of names of source database tables (comma separated or line separated) to exclude while importing data of those exported tables. Table names follow the same convention as --table-list.
‑‑enable‑adaptive‑parallelism Adapt parallelism based on the resource usage (CPU, memory) of the target YugabyteDB cluster.
Default: true
Accepted parameters: true, false, yes, no, 0, 1
--adaptive-parallelism-max Number of maximum parallel jobs to use while importing data when adaptive parallelism is enabled. By default, voyager tries to determine the total number of cores N and use N/2 as the maximum parallel jobs.
-e, --export-dir Path to the export directory. This directory is a workspace used to store exported schema DDL files, export data files, migration state, and a log file.
-h, --help Command line help.
--parallel-jobs Number of parallel jobs to use while importing data. Depending on the YugabyteDB database configuration, the value of --parallel-jobs should be tweaked such that at most 50% of target cores are utilised.
Default: If yb-voyager can determine the total number of cores N in the YugabyteDB database cluster, it uses N/2 as the default for import data and N/4 for import data to target. Otherwise, it defaults to twice the number of nodes in the cluster.
--send-diagnostics Enable or disable sending diagnostics information to Yugabyte.
Default: true
Accepted parameters: true, false, yes, no, 0, 1
--start-clean Starts a fresh import with data files present in the data directory.
If the target YugabyteDB database has non-empty tables, you are prompted to continue the import without truncating those tables; if you go ahead without truncating, then yb-voyager starts ingesting the data present in the data files in upsert mode.
Note that for cases where a table doesn't have a primary key, duplicate data may be inserted. You can avoid duplication by excluding the table using --exclude-table-list, or by truncating those tables manually before using the start-clean flag.
Default: false
Accepted parameters: true, false, yes, no, 0, 1
--target-db-host Domain name or IP address of the machine on which target database server is running.
Default: "127.0.0.1"
--target-db-name Target database name.
Default: yugabyte
‑‑target‑db‑password Password to connect to the target YugabyteDB database. Alternatively, you can also specify the password by setting the environment variable TARGET_DB_PASSWORD. If you don't provide a password via the CLI during any migration phase, yb-voyager will prompt you at runtime for a password. If the password contains special characters that are interpreted by the shell (for example, # and $), enclose the password in single quotes.
--target-db-port Port number of the target database server.
Default: 5433
--target-db-schema Schema name of the target database. MySQL and Oracle migrations only.
--target-db-user Username of the target database.
--target-endpoints Comma-separated list of node's endpoint to use for parallel import of data
Default: Use all the nodes in the cluster. For example: "host1:port1,host2:port2" or "host1,host2". Note: use-public-ip flag will be ignored if this is used.
--use-public-ip Use the node public IP addresses to distribute --parallel-jobs uniformly on data import.
Default: false
Note that you may need to configure the database with public IP addresses by setting server-broadcast-addresses.
Example: yb-voyager import data ... --use-public-ip true
Accepted parameters: true, false, yes, no, 0, 1
--target-ssl-cert Path to a file containing the certificate which is part of the SSL <cert,key> pair.
--target-ssl-key Path to a file containing the key which is part of the SSL <cert,key> pair.
--target-ssl-crl Path to a file containing the SSL certificate revocation list (CRL).
--target-ssl-mode Specify the SSL mode for the target database as one of disable, allow, prefer (default), require, verify-ca, or verify-full.
--target-ssl-root-cert Path to a file containing SSL certificate authority (CA) certificate(s).
--truncate-tables Truncate tables on target YugabyteDB database before importing data. This option is only valid if --start-clean is set to true.
Default: false
-y, --yes Answer yes to all prompts during the export schema operation.
Default: false

Example

Offline migration

An example for offline migration is as follows:

yb-voyager import data --export-dir /dir/export-dir \
        --target-db-host 127.0.0.1 \
        --target-db-user ybvoyager \
        --target-db-password 'password' \
        --target-db-name target_db \
        --target-db-schema target_schema \
        --parallel-jobs 12

Live migration

An example for all live migration scenarios is as follows:

yb-voyager import data to target --export-dir /dir/export-dir \
        --target-db-host 127.0.0.1 \
        --target-db-user ybvoyager \
        --target-db-password 'password' \
        --target-db-name target_db \
        --target-db-schema target_schema \
        --parallel-jobs 12

import data status

For offline migration, get the status report of an ongoing or completed data import operation. The report contains migration status of tables, number of rows or bytes imported, and percentage completion.

Syntax

Usage: yb-voyager import data status [ <arguments> ... ]

Arguments

The valid arguments for import data status are described in the following table:

Argument Description/valid options
-e, ‑‑export‑dir Path to the export directory. This directory is a workspace used to keep the exported schema, data, state, and logs.
-h, --help Command line help.
-y, --yes Answer yes to all prompts during the import data operation.
Default: false

Example

yb-voyager import data status --export-dir /dir/export-dir

get data-migration-report

Note that this command is applicable for Live migrations only.

Provides a consolidated report of data migration per table among all the databases involved in the live migration. The report includes the number of rows exported, the number of rows imported, change events exported and imported (INSERTS, UPDATES, and DELETES), and the final row count on the database.

Syntax

Usage: yb-voyager get data-migration-report [ <arguments> ... ]

Arguments

The valid arguments for get data-migration-report are described in the following table:

Argument Description/valid options
-e, ‑‑export‑dir Path to the export directory. This directory is a workspace used to store exported schema DDL files, export data files, migration state, and a log file.
-h, --help Command line help.
‑‑source‑db‑password Password to connect to the source database. If you don't provide a password via the CLI during any migration phase, yb-voyager will prompt you at runtime for a password. Alternatively, you can also specify the password by setting the environment variable SOURCE_DB_PASSWORD. If the password contains special characters that are interpreted by the shell (for example, # and $), enclose it in single quotes.
‑‑source‑replica‑db‑password Password to connect to the source-replica database. Alternatively, you can also specify the password by setting the environment variable SOURCE_REPLICA_DB_PASSWORD. If the password contains special characters that are interpreted by the shell (for example, # and $), enclose it in single quotes.
--target-db-password Password to connect to the target YugabyteDB database. Alternatively, you can also specify the password by setting the environment variable TARGET_DB_PASSWORD. If you don't provide a password via the CLI during any migration phase, yb-voyager will prompt you at runtime for a password. If the password contains special characters that are interpreted by the shell (for example, # and $), enclose the password in single quotes.

Example

yb-voyager get data-migration-report --export-dir /dir/export-dir

import data to source

Note that this command is applicable for Live migrations only.

Imports data to the source database, and streams new changes from the YugabyteDB database to the source database.

Syntax

Usage: yb-voyager import data to source [ <arguments> ... ]

Arguments

The valid arguments for import data to source are described in the following table:

Argument Description/valid options
-e, --export-dir Path to the export directory. This directory is a workspace used to store exported schema DDL files, export data files, migration state, and a log file.
-h, --help Command line help for import data to source.
--parallel-jobs Number of parallel jobs to use while importing data.
Default: 16(Oracle)
--send-diagnostics Enable or disable sending diagnostics information to Yugabyte.
Default: true
Accepted parameters: true, false, yes, no, 0, 1
‑‑source‑db‑password Password to connect to the source database.
--start-clean Starts a fresh import with exported data files present in the export-dir/data directory.
If any table on YugabyteDB database is non-empty, it prompts whether you want to continue the import without truncating those tables; if you go ahead without truncating, then yb-voyager starts ingesting the data present in the data files with upsert mode.
Note that for the cases where a table doesn't have a primary key, this may lead to insertion of duplicate data. To avoid this, exclude the table using the --exclude-file-list or truncate those tables manually before using the start-clean flag.
Default: false
Accepted parameters: true, false, yes, no, 0, 1
-y, --yes Answer yes to all prompts during the migration.
Default: false

Example

yb-voyager import data to source --export-dir /dir/export-dir \
        --source-db-password 'password'

import data to source-replica

Note that this command is applicable for Live migrations only.

Imports data to the source-replica database, and streams new changes from the YugabyteDB database to the source-replica database.

Syntax

Usage: yb-voyager import data to source-replica [ <arguments> ... ]

Arguments

The valid arguments for import data to source-replica are described in the following table:

Argument Description/valid options
--batch-size Size of batches in the number of rows generated for ingestion when you import data to source-replica database.
Default: 10000000 (Oracle) or 100000 (PostgreSQL)
--disable-pb Use this argument to disable progress bar or statistics during data import.
Default: false
Accepted parameters: true, false, yes, no, 0, 1
-e, ‑‑export‑dir Path to the export directory. This directory is a workspace used to store exported schema DDL files, export data files, migration state, and a log file.
‑‑source‑replica‑db‑host Domain name or IP address of the machine on which source-replica database server is running.
Default: 127.0.0.1
‑‑source‑replica‑db‑name Name of the database in the source-replica database on which import needs to be done.
‑‑source‑replica‑db‑password Password to connect to the source-replica database. Alternatively, you can also specify the password by setting the environment variable SOURCE_REPLICA_DB_PASSWORD. If you don't provide a password via the CLI or environment variable during any migration phase, yb-voyager will prompt you at runtime for a password. If the password contains special characters that are interpreted by the shell (for example, # and $), enclose the password in single quotes.
--source-replica-db-port Port number of the source-replica database server.
Default: 1521 (Oracle)
--source-replica-db-schema Schema name of the source-replica database.
--source-replica-db-sid Oracle System Identifier (SID) that you wish to use while importing data to Oracle instances. Oracle migrations only.
--source-replica-db-user Username to connect to the source-replica database.
--source-replica-ssl-cert Source-replica database SSL Certificate path.
--source-replica-ssl-crl Source-replica database SSL Root Certificate Revocation List (CRL).
--source-replica-ssl-key Source-replica database SSL key path.
--source-replica-ssl-mode One of disable, allow, prefer(default), require, verify-ca, or verify-full.
--source-replica-ssl-root-cert Source-replica database SSL Root Certificate path.
-h, --help Command line help for import data to source-replica.
--oracle-home Path to set $ORACLE_HOME environment variable. tnsnames.ora is found in $ORACLE_HOME/network/admin. Oracle migrations only.
--oracle-tns-alias TNS (Transparent Network Substrate) alias configured to establish a secure connection with the server. Oracle migrations only.
--parallel-jobs The number of parallel batches issued to the source-replica database.
Default: 16 (Oracle); or, if yb-voyager can determine the total number of cores N, then N/2, otherwise 8 (PostgreSQL)
--start-clean Starts a fresh import with data files present in the data directory.
If the target YugabyteDB database has any non-empty tables, you are prompted to continue the import without truncating those tables; if you proceed without truncating, then yb-voyager starts ingesting the data present in the data files in non-upsert mode.
Note that for cases where a table doesn't have a primary key, duplicate data may be inserted. You can avoid duplication by excluding the table using --exclude-table-list, or by truncating those tables manually before using the start-clean flag.
Default: false
Accepted parameters: true, false, yes, no, 0, 1
--send-diagnostics Enable or disable sending diagnostics information to Yugabyte.
Default: true
Accepted parameters: true, false, yes, no, 0, 1
-y, --yes Answer yes to all prompts during the migration.
Default: false

Example

yb-voyager import data to source-replica --export-dir /dir/export-dir \
        --source-replica-db-host 127.0.0.1 \
        --source-replica-db-user ybvoyager \
        --source-replica-db-password 'password' \
        --source-replica-db-name source_replica_db \
        --source-replica-db-schema source_replica_schema \
        --parallel-jobs 12