Sqoop command and list all databases

 1. How to connect & list all databases in Sqoop:

Sqoop list-databases tool parses and executes the show databases query against the database server. Syntax: $ sqoop list-databases connection-args Example: $ sqoop list-databases --connect jdbc:mysql//localhost/ --username username --password password It will display the list of databases. 2. Common Arguments in Sqoop: 2.1. --connect jdbc-uri : JDBC connect string. 2.2. --connect-manager class-name: If you write your java program and tell sqoop how to connect database. Suppose you have a java program class like myConnect.java then you can use:- --connect-manager myConnect 2.3. --driver class-name: We can manually specify the jdbc driver. Jdbc:mysql is the jdbc driver. 2.4. --hadoop-home dir: Override the $HADOOP_HOME 2.5. -P: Read the password from console. 2.6. --username username: Set Authentic username. 2.7. --password password: Set Authentic password. 2.8. --verbose: Print more information while working. 2.10. --connection-param-file : Rather than writing username and password to (with the) import command, you can pass the filename which having the username and password. When connecting to a database using JDBC, you can optionally specify extra JDBC parameters via property file using the option –connection-param-file. 2.11. --target-dir: Name of directory instead of table name 2.12. --warehouse-dir: Directory name to store all the table directory. 2.13. --options-file: we can put import command and connection in a file and run it. 2.14. --help: Print the usage instructions

Comments

Post a Comment

Popular posts from this blog

SQOOP file format

  How many file  formats  in Sqoop?   Sqoop supports the following 4 file  formats  for import operation.    1. Text file format.    2. Avro file format.    3. Sequence file format.    4. Parquet file format.    1st is  text file format  and  other three are binary file format .    Binary formats are Avro, Sequence and Parquet file.     Text file format in Sqoop:     1. Text file format: Apache Sqoop uses text file format as the default file format for importing the data from SQL to HDFS.    If we want to import data into text file then we need to specify --as- textfile  in  sqoop  command.  
Read more

Why do we use $CONDITIONS in Apache Sqoop?

Usage of Sqoop:  By default 4 mapper running in sqoop. When we are going to import data and if a table has a primary key then by default 4 mapper otherwise 1 mapper runs. If more than 1 mapper will run then sqoop distributes the data equally among the mappers to get high performance. Number of mappers = Number of part files on the HDFS.
Read more

Sqoop where condition , Sqoop join two tables

  1. Import data from MySQL using JOIN in Sqoop Example: Sqoop import --connect jdbc:mysql://localhost/sales --username root --password password --target-dir /user/YT/joinedData --query `select s.name, d.name from students s join department d on d.dept_id = s.dept_id where s.country_name = ‘India’ and $CONDITIONS order by d.name` -m 2 Above SQL will display all those student who belongs to India with the student name and department name. 2. Where Clause without query in Sqoop: Here we are not using the query, using only table name and where clause. It will return all the matching rows. Example: Sqoop import --connect jdbc:mysql://localhost/sales --username root --password password --table students --target-dir /user/YT/studentsCountryWiseData --where country_name = ‘India’ -m 1
Read more