How to use password-file in Apache Sqoop?

 There are 3 ways to pass password in sqoop import command:

1. --password mypassword --> not secured way 2. –P: Read the password from the console. 3. Reading password from a file. Reading the password from a file: Syntax: --password-file password-filename Example: Sqoop import --connect jdbc:mysql://localhost/sales --username root --password-file my_sqoop_password --table student We can store and use the password-file from both locations (LFS or HDFS). 1. Store and use password file from the local file system: Create a file in the local file system : ~]$ echo -n “mypassword” greater-than symbol /home/YT/pass.txt Here we saved mypassword text into pass.txt file on LFS. ~]$ cat /home/YT/pass.txt mypassword In the below example, we can see the --password-file command with the local file system. Example: Sqoop import --connect jdbc:mysql://localhost/sales --username root --password-file /home/YT/pass.txt --table student

2. Store and use password file from Hadoop distributed file system(HDFS): Copy pass.txt file from local file system to HDFS : ~]$ hadoop fs -put /home/YT/pass.txt /user/YT Here we copied the pass.txt file on HDFS. ~]$ hadoop fs -cat /user/YT/pass.txt mypassword Now, let's use the HDFS password file path with –password-file command. Example: Sqoop import --connect jdbc:mysql://localhost/sales --username root --password-file /user/YT/pass.txt --table student

Comments

Post a Comment

Popular posts from this blog

SQOOP file format

  How many file  formats  in Sqoop?   Sqoop supports the following 4 file  formats  for import operation.    1. Text file format.    2. Avro file format.    3. Sequence file format.    4. Parquet file format.    1st is  text file format  and  other three are binary file format .    Binary formats are Avro, Sequence and Parquet file.     Text file format in Sqoop:     1. Text file format: Apache Sqoop uses text file format as the default file format for importing the data from SQL to HDFS.    If we want to import data into text file then we need to specify --as- textfile  in  sqoop  command.  
Read more

Why do we use $CONDITIONS in Apache Sqoop?

Usage of Sqoop:  By default 4 mapper running in sqoop. When we are going to import data and if a table has a primary key then by default 4 mapper otherwise 1 mapper runs. If more than 1 mapper will run then sqoop distributes the data equally among the mappers to get high performance. Number of mappers = Number of part files on the HDFS.
Read more

Sqoop where condition , Sqoop join two tables

  1. Import data from MySQL using JOIN in Sqoop Example: Sqoop import --connect jdbc:mysql://localhost/sales --username root --password password --target-dir /user/YT/joinedData --query `select s.name, d.name from students s join department d on d.dept_id = s.dept_id where s.country_name = ‘India’ and $CONDITIONS order by d.name` -m 2 Above SQL will display all those student who belongs to India with the student name and department name. 2. Where Clause without query in Sqoop: Here we are not using the query, using only table name and where clause. It will return all the matching rows. Example: Sqoop import --connect jdbc:mysql://localhost/sales --username root --password password --table students --target-dir /user/YT/studentsCountryWiseData --where country_name = ‘India’ -m 1
Read more