Run hadoop command in Python

Hadoop is the most widely used big data platform for big data analysis. It is easy to run Hadoop command in Shell or a shell script. However, there is often a need to run manipulate hdfs file directly from python. We use examples to describe how to run hadoop command in python to list, save hdfs files.

We already know how to call an extern shell command from python. We can simply call Hadoop command using the run_cmd method.

Run Hadoop ls command in Python

 

Run Hadoop get command in Python

Run Hadoop put command in Python