Move Hive Table from One Cluster to Another
This tutorial uses examples to describe how to move Hive table from one cluster to another. The basic idea is to use the EXPORT and IMPORT commands.
The EXPORT command exports the data of a table or partition, along with the metadata, into a specified output location. This output location can then be moved over to a different Hadoop or Hive instance and imported from there with the IMPORT command.
Export Syntax
EXPORT TABLE tablename [PARTITION (part_column="value"[, ...])] TO 'export_target_path' [ FOR replication('eventid') ] |
Import Syntax
IMPORT [[EXTERNAL] TABLE new_or_original_tablename [PARTITION (part_column="value"[, ...])]] FROM 'source_path' [LOCATION 'import_target_path'] |
Examples to Move Hive Table from one cluster (grid) to another
Suppose you have two clusters : cluster A and cluster B.
- On Cluster A, use
EXPORTcommand to exports the data of a table or a partition, along with the metadata to a specified output location named hdfs_path_a; - Use
discpto copy the data in cluster A to cluster B. e.g.,discp hdfs_path_a hdfs_path_b - On cluster B, use
IMPORTcommand to import the data inhdfs_path_bto a new_table.
The following code shows the process:
On Cluster A:
|
1 |
export table student to 'hdfs://cluster_a_host:/path_a/student'; |
Copy the file to cluster B using discp:
|
1 |
discp hdfs://cluster_a_host:/path_a/student hdfs://cluster_b_host:/path_b/student |
On Cluster B:
|
1 |
import from 'hdfs://cluster_b_host:/path_b/student/student'; |
Or rename the table:
|
1 2 |
import table new_student from 'hdfs_exports_location/student'; |
Rename table on import:
|
1 2 3 |
export table department to 'hdfs_exports_location/department'; import table imported_dept from 'hdfs_exports_location/department'; |
Export partition and import:
|
1 2 3 |
export table employee partition (emp_country="in", emp_state="ka") to 'hdfs_exports_location/employee'; import from 'hdfs_exports_location/employee'; |
Export table and import partition:
|
1 2 3 |
export table employee to 'hdfs_exports_location/employee'; import table employee partition (emp_country="us", emp_state="tn") from 'hdfs_exports_location/employee'; |
Specify the import location:
|
1 2 3 4 |
export table department to 'hdfs_exports_location/department'; import table department from 'hdfs_exports_location/department' location 'import_target_location/department'; |
Import as an external table:
|
1 2 |
export table department to 'hdfs_exports_location/department'; import external table department from 'hdfs_exports_location/department'; |
