Hadoop : Shell Commands

Note : In Hadoop File system each command starts with the : – hadoop fs where fs stands for file system 

1.  hadoop fs -ls /
This command is similar to the Linux 'ls' command to list the files in the hadoop system.
The parameter '/' it means list the files from the root of the hadoop

2.  -mkdir
This command is used to created the dirctories in the HDFS(Hadoop File system)
Example : -
hadoop fs -mkdir /user/hadoop/test  (This command will create the 'test' folder inside the /user/hadoop) 

3.  -count
This command used to cound the number of dirctories
Example : -
hadoop fs -count /user 

4.   -touchz
This command is used to create a file of 0 length. This is similar to the Unix ‘touch’ command
Example : -
hadoop fs -touchz /user/hadoop/test/test.txt  (This command will create a file 'test.txt' inside the directory '/user/hadoop/test') 

5.  -cp and -mv
These commands operate like regular Unix commands to copy and rename a file. 

6.  -put and -copyFromLocal
As the name suggest these command will be used to copy the files from Local Hard Disk(Storage Device) to HDFS (Hadoop File System)
The only difference between -put and -copyFromLocal is 'In put we have the option of stdin but not in the copyFromLocal' 

Example : -
$ hadoop fs -copyFromLocal /home/rahul/Hadoop-Script/rahul.txt /user/hadoop/test
$ hadoop fs -ls /user/hadoop/test
Found 2 items
-rw-r–r–   1 rahul supergroup          0 2014-07-27 18:36 /user/hadoop/test/rahul.txt
-rw-r–r–   1 rahul supergroup          0 2014-07-27 18:27 /user/hadoop/test/test.txt 

7.   -rm
It is used to delete or remove file
Example : -
$ hadoop fs -rm /user/hadoop/test/test.txt
Deleted hdfs://localhost:9000/user/hadoop/test/test.txt 

8.   -get and -copyToLocal –
This commands are used to copy the file from HDFS to Local System.
Example : -
$ hadoop fs -copyToLocal /user/hadoop/test /home/rahul/Hadoop-Script/rahul.txt 

9.hadoop fs -rmr /user
To remove the directory from HDFS