An Optimized Model for MapReduce Based on Hadoop
Abstract: Aiming at the waste
of computing resources resulting from sequential control of running mechanism
of MapReduce model on Hadoop platform，Fork/Join framework has been introduced
into this model to make full use of CPU resource of each node. From the
perspective of fine-grained parallel data processing, combined with Fork/Join
framework，a parallel and multi-thread model，this paper optimizes MapReduce
model and puts forward a MapReduce+Fork/Join programming model which is a
distributed and parallel architecture combined with coarse-grained and
fine-grained on Hadoop platform to Support two-tier levels of parallelism
architecture both in shared and distributed memory machines. A test is made under
the environment of Hadoop cluster composed of four nodes. And the experimental
results prove that this model really can improve performance and efficiency of
the whole system and it is not only suitable for handling tasks with data
intensive but also tasks with computing intensive. it is an effective
optimization and improvement to the MapReduce model of big data processing.
Author: Zhang Hong
Journal Code: jptkomputergg160305
