[user00@master ~]$ sqoop version 16/07/08 06:34:13 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.3.1 Sqoop 1.4.5-cdh5.3.1 git commit id Compiled by jenkins on Tue Jan 27 16:34:48 PST 2015
[user00@master ~]$ sqoop Try 'sqoop help' for usage. [user00@master ~]$ sqoop help 16/07/08 06:37:53 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.3.1 usage: sqoop COMMAND [ARGS] . Available commands: codegen Generate code to interact with database records create-hive-table Import a table definition into Hive eval Evaluate a SQL statement and display the results export Export an HDFS directory to a database table help List available commands import Import a table from a database to HDFS import-all-tables Import tables from a database to HDFS import-mainframe Import datasets from a mainframe server to HDFS job Work with saved jobs list-databases List available databases on a server list-tables List available tables in a database merge Merge results of incremental imports metastore Run a standalone Sqoop metastore version Display version information . See 'sqoop help COMMAND' for information on a specific command.
user00@master ~ $ export DBID=帳號 ## 或者使用環境變數 $USER 來代表您的登入帳號 user00@master ~ $ export DBID=$USER
[user00@master ~]$ sqoop list-databases --connect "jdbc:mysql://localhost/" --username $DBID -P Enter password: 輸入密碼
information_schema user00
[user00@master ~]$ sqoop list-tables --connect "jdbc:mysql://localhost/$DBID" --username $DBID -P Enter password: 輸入密碼
16/07/08 06:57:36 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. mysql_data
[user00@master ~]$ sqoop import --connect "jdbc:mysql://localhost/$DBID" --table mysql_data --username $DBID -P Enter password: 輸入密碼
16/07/08 07:02:30 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 16/07/08 07:02:30 INFO tool.CodeGenTool: Beginning code generation 16/07/08 07:02:31 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `mysql_data` AS t LIMIT 1 16/07/08 07:02:31 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `mysql_data` AS t LIMIT 1 16/07/08 07:02:31 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce Note: /tmp/sqoop-user00/compile/223418ffeba6b499e5a2444342aca53b/mysql_data.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. 16/07/08 07:02:33 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-user00/compile/223418ffeba6b499e5a2444342aca53b/mysql_data.jar 16/07/08 07:02:33 WARN manager.MySQLManager: It looks like you are importing from mysql. 16/07/08 07:02:33 WARN manager.MySQLManager: This transfer can be faster! Use the --direct 16/07/08 07:02:33 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path. 16/07/08 07:02:33 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql) 16/07/08 07:02:33 INFO mapreduce.ImportJobBase: Beginning import of mysql_data . ... 略 ... . File Input Format Counters Bytes Read=0 File Output Format Counters Bytes Written=16 16/07/08 07:03:01 INFO mapreduce.ImportJobBase: Transferred 16 bytes in 26.9385 seconds (0.5939 bytes/sec) 16/07/08 07:03:01 INFO mapreduce.ImportJobBase: Retrieved 2 records.
[user00@master ~]$ hadoop fs -ls mysql_data Found 4 items -rw-r--r-- 1 user00 user00 0 2015-08-05 10:06 mysql_data/_SUCCESS drwxr-xr-x - user00 user00 0 2015-08-05 10:05 mysql_data/_logs -rw-r--r-- 1 user00 user00 8 2015-08-05 10:06 mysql_data/part-m-00000 -rw-r--r-- 1 user00 user00 8 2015-08-05 10:06 mysql_data/part-m-00001
[user00@master ~]$ hadoop fs -cat mysql_data/part-* 1,Hello 2,World
rm mysql_data.java
user00@master ~ $ hadoop fs -rmr mysql_data
user00@master ~ $ sqoop import --connect "jdbc:mysql://localhost/$DBID" --table mysql_data --username $DBID -P --target-dir MYSQL_TMP