Abstract:

As an open software framework developed by Apache, Hadoop provides the distributed processing of large data
sets that across clusters of computer by simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. And now a wide variety of companies and organizations use Hadoop for both research and production. The core modules of Hadoop are HDFS (Hadoop distributed file system)and MapReduce distributed programming model. An HDFS cluster primarily consists of a NameNode that manages the file system metadata and DataNodes that store the actual data. However, the management issues of HDFS cluster resources has became urgent. There is a web page to browse HDFS provided by Hadoop, but this interface of HDFS is not friendly for users because command line is the main method to achieve operation of the document now. In order to use HDFS conveniently, using the struts2 develop framework and JFreeChart, the architecture of HDFS is researched and then an HDFS visualization system is designed based on web interface. And the development efficiency based on Hadoop for QAR data is improved.

Key words: HDFS, visualization, Hadoop, QAR data file, cluster management