Table of Contents

In this article we will go over some frequently used HDFS commands

Hadoop Commands

The Hadoop Distributed File System (HDFS) is a distributed file system designed to run on commodity hardware. It has many similarities with existing distributed file systems. However, the differences from other distributed file systems are significant. HDFS is highly fault-tolerant and is designed to be deployed on low-cost hardware. HDFS provides high throughput access to application data and is suitable for applications that have large data sets. HDFS commands helps to perform Hadoop HDFS file operations.

Hadoop version

Command : hadoop version

List the contents of the root directory in HDFS

Command: hadoop fs -ls

Report the amount of space used and available on currently mounted file system

Command: hadoop fs -df hdfs
Command: hadoop fs -df -h hdfs

Create a new directory named “hadoop”

Command: hadoop fs -mkdir /user/entechlog/hadoop

Add a sample text file from the local directory to HDFS

Command:hadoop fs -put /data/home/entechlog/hadoop/input/HadoopFile1.txt /user/entechlog/hadoop/input
Command: hadoop fs -copyFromLocal /data/home/entechlog/hadoop/input/HadoopFile1.txt /user/entechlog/hadoop/input/

List the contents of a directory in HDFS

Command: hadoop fs -ls /user/entechlog/hadoop/input/

Add the entire local directory to HDFS

Command:hadoop fs -put /data/home/entechlog/hadoop/input/copydir/ /user/entechlog/hadoop/input/

Space used by a directory in HDFS

Command: hadoop fs -du -s -h /user/entechlog/hadoop/input/copydir/

Delete a file

Command: hadoop fs -rm /user/entechlog/hadoop/input/copydir/HadoopFile1.txt

Ensure this file is no longer in HDFS

Command: hadoop fs -ls /user/entechlog/hadoop/input/copydir/HadoopFile1.txt

Delete all files from the folder

Command: hadoop fs -rm /user/entechlog/hadoop/input/copydir/\*

Empty the trash

Command: hadoop fs –expunge

Remove all files from a directory and the directory itself

Command: hadoop fs -rm -r /user/entechlog/hadoop/input/copydir/

Copy files between HDFS

Command: hadoop fs -cp /user/entechlog/hadoop/input/HadoopFile3.txt /user/entechlog/hadoop/input/HadoopFile4.txt

Copy file from HDFS to Local

Command: hadoop fs -copyToLocal /user/entechlog/hadoop/input/HadoopFile4.txt /data/home/entechlog/hadoop/input/HadoopFile4.txt
Command: hadoop fs -get /user/entechlog/hadoop/input/HadoopFile4.txt /data/home/entechlog/hadoop/input/HadoopFile5.txt

Display last kilobyte of the file

Command: hadoop fs -tail /user/entechlog/hadoop/input/HadoopFile4.txt

Changing permission of a file

Command: hadoop fs -chmod 600 /user/entechlog/hadoop/input/HadoopFile4.txt

Changing the owner

Command: hadoop fs -chown newUser:newGroup /user/entechlog/hadoop/input/HadoopFile4.txt

Changing the group

Command: hadoop fs -chgrp newGroup /user/entechlog/hadoop/input/HadoopFile4.txt

Move a directory from one location to other

Command: hadoop fs -mv source target

List all the hadoop file system commands

Command: hadoop fs

Hadoop file system commands help

Command: hadoop fs –help

To know the system is running under safemode or not

Command: hadoop dfsadmin -safemode get

Turn on safemode on when it is off

Command: hadoop  dfsadmin  -safemode enter

Turn off safemode off when it is on

Command: hadoop  dfsadmin  -safemode leave