English | 简体中文
Hadoop is an open - source distributed computing platform used for processing large - scale data storage and computing. This product is based on the Huawei Cloud EulerOS 2.0 64 - bit system of Kunpeng servers, providing an out - of - the - box Hadoop computing platform.
- HDFS (Hadoop Distributed File System)
- Storage Architecture: It adopts a master - slave structure. The NameNode manages metadata, and the DataNode stores actual data blocks (default 128MB/block), supporting multi - replica redundancy (default 3 copies) to ensure fault tolerance.
- Applicable Scenarios: It is suitable for large - file batch processing (such as video storage and log analysis), but has low efficiency for low - latency access and small - file storage.
- MapReduce
- Computing Model: It realizes distributed computing through two stages of Map (data sharding processing) and Reduce (result aggregation), simplifying the complexity of parallel programming.
- YARN (Resource Scheduling System)
- Function: It dynamically allocates cluster resources (CPU, memory), supports concurrent execution of multiple tasks (such as MapReduce and Spark), and improves resource utilization.
The open - source image product Hadoop Distributed Computing Platform provided by this project has pre - installed Hadoop version 3.3.6 and its related operating environment, and provides deployment templates. Come and refer to the usage guide to easily start the "out - of - the - box" efficient experience!
System requirements are as follows:
- CPU: 2vCPUs or higher
- RAM: 4GB or larger
- Disk: At least 40GB
Register a Huawei account and activate Huawei Cloud
| Image Specification | Feature Description | Remarks |
|---|---|---|
| hadoop - 3.3.6 - kunpeng | Installed and deployed based on Kunpeng servers + Huawei Cloud EulerOS 2.0 64 - bit |
- For more questions, you can contact us via issue or the service support of the specified product in the Huawei Cloud Marketplace.
- For other open - source images, see [open - source - image - repos](https://github.com/HuaweiCloudDeveloper/open - source - image - repos)
- Fork this repository and submit a merge request.
- Synchronously update README.md based on your open - source image information.