建议使用以下浏览器,以获得最佳体验。 IE 9.0+以上版本 Chrome 31+ 谷歌浏览器 Firefox 30+ 火狐浏览器
请选择 进入手机版 | 继续访问电脑版
设置昵称

在此一键设置昵称,即可参与社区互动!

确定
我再想想
选择版块
汇聚鲲鹏领域技术专家大咖

真我Realme

发帖: 69粉丝: 12

级别 : 版主

发消息 + 关注

发表于2019-10-30 14:57:50 163990 7
直达本楼层的链接
楼主
显示全部楼层
[大数据] 鲲鹏生态 |【大数据精华帖 | 精品内容汇总~推荐收藏~持续更新~】

## TaiShan大数据解决方案 移植、部署、调优使用流程和一站式资源获取 ![10-TaiShan-大数据解决方案_05.png](https://bbs-img-cbc-cn.obs.cn-north-1.myhuaweicloud.com/data/attachment/forum/202004/02/1916138aihc7qpi1iwfqcl.png) #### [旧版精华帖链接](https://bbs.huaweicloud.com/forum/thread-38802-1-1.html) #### [鲲鹏社区大数据解决方案文档链接](https://support.huaweicloud.com/productdesc-kunpengbds/kunpengbds_01_0002.html) # 1. Apache版本 ## 1.1 移植篇 **Apache平台移植** | 组件 | Apache源码包 | 文档 | ---- | ---- | ---- | Hadoop |[hadoop-3.1.1-src.tar.gz](https://archive.apache.org/dist/hadoop/core/hadoop-3.1.1/hadoop-3.1.1-src.tar.gz)|[Hadoop 3.1.1 移植指南](https://bbs.huaweicloud.com/forum/thread-22653-1-1.html) | Hbase | [hbase-2.1.0-src.tar.gz](http://archive.apache.org/dist/hbase/2.1.0/hbase-2.1.0-src.tar.gz) |[HBase 2.1.0 移植指南](https://bbs.huaweicloud.com/forum/thread-22663-1-1.html) | Hive | [hive-rel-release-3.0.0.tar.gz](https://github.com/apache/hive/archive/rel/release-3.0.0.tar.gz) |[Hive 3.0.0移植指南](https://bbs.huaweicloud.com/forum/thread-24733-1-1.html) | Spark | [spark-2.3.0.tar.gz](https://github.com/apache/spark/archive/v2.3.0.tar.gz) |[Spark 2.3.0 移植指南](https://bbs.huaweicloud.com/forum/thread-22661-1-1.html) | Druid | [druid-0.12.1.tar.gz](https://github.com/apache/druid/archive/druid-0.12.1.tar.gz) |[Druid 0.12.1 移植指南](https://bbs.huaweicloud.com/forum/thread-38176-1-1.html) | Flink | [flink-release-1.8.1.tar.gz](https://github.com/apache/flink/archive/release-1.8.1.tar.gz) |[Flink 1.8.1 移植指南](https://bbs.huaweicloud.com/forum/thread-26263-1-1.html) | Kafka | [kafka-2.1.0-src.tgz](https://archive.apache.org/dist/kafka/2.1.0/kafka-2.1.0-src.tgz) |[Kafka 2.1.0 移植指南](https://bbs.huaweicloud.com/forum/thread-26274-1-1.html) | Zookeeper | [zookeeper-release-3.4.6.tar.gz](https://github.com/apache/zookeeper/archive/release-3.4.6.tar.gz) |[Zookeeper 移植指南](https://bbs.huaweicloud.com/forum/thread-26229-1-1.html) ## 1.2 部署篇 **Apache平台部署** | 组件 | 文档 | ---- | ---- | Hadoop | [开源Hadoop集群部署指南](https://bbs.huaweicloud.com/forum/forum.php?mod=viewthread&tid=22718&page=1&extra=#pid72294) | Hbase | [开源Hbase集群部署指南](https://bbs.huaweicloud.com/forum/thread-22719-1-1.html) | Hive | [开源Hive集群部署指南](https://bbs.huaweicloud.com/forum/thread-26239-1-1.html) | Flink | [开源Flink集群部署指南](https://bbs.huaweicloud.com/forum/thread-26213-1-1.html) | Kafka | [开源Kafka集群部署指南](https://bbs.huaweicloud.com/forum/thread-26214-1-1.html) | Redis | [开源Redis集群部署指南](https://bbs.huaweicloud.com/forum/thread-26215-1-1.html) | Solr | [开源Solr集群部署指南](https://bbs.huaweicloud.com/forum/thread-26217-1-1.html) | Spark | [开源Spark集群部署指南](https://bbs.huaweicloud.com/forum/thread-22720-1-1.html) | Storm | [开源Storm集群部署指南](https://bbs.huaweicloud.com/forum/thread-26219-1-1.html) |ElasticSearch | [开源Elasticsearch集群部署指南](https://bbs.huaweicloud.com/forum/thread-26210-1-1.html) | Zookeeper | [开源Zookeeper集群部署指南](https://bbs.huaweicloud.com/forum/thread-26220-1-1.html) # 2. HDP版本 ## 2.1 移植篇 #### [Ambari移植指南](https://bbs.huaweicloud.com/forum/thread-28829-1-1.html) **HDP平台移植** | 组件 | HDP源码包 | 文档 | ---- | ---- | ---- | Accumulo | [Accumulo-release-HDP-3.1.0.0-78](https://github.com/hortonworks/accumulo-release/archive/HDP-3.1.0.0-78-tag.tar.gz) |[HDP3.1 Accumulo移植指南](https://bbs.huaweicloud.com/forum/thread-38228-1-1.html) | Atlas | [atlas-release-HDP-3.1.0.0-78](https://github.com/hortonworks/atlas-release/archive/HDP-3.1.0.0-78-tag.tar.gz) |[HDP 3.1 Atlas 移植指南](https://bbs.huaweicloud.com/forum/thread-38205-1-1.html) | Hadoop | [hadoop-release-HDP-3.1.0.0-78](https://github.com/hortonworks/hadoop-release/archive/HDP-3.1.0.0-78-tag.tar.gz) |[Hadoop 3.1 移植指南](https://bbs.huaweicloud.com/forum/thread-38329-1-1.html) | Hbase | [hbase-release-HDP-3.1.0.0-78](https://github.com/hortonworks/hbase-release/archive/HDP-3.1.0.0-78-tag.tar.gz) |[HDP 3.1 HBase 移植指南](https://bbs.huaweicloud.com/forum/thread-38207-1-1.html) | Hive | [hive-release-HDP-3.1.0.0-78](https://github.com/hortonworks/hive-release/archive/HDP-3.1.0.0-78-tag.tar.gz) |[HDP 3.1 Hive 移植指南](https://bbs.huaweicloud.com/forum/thread-38208-1-1.html) | Kafka | [kafka-release-HDP-3.1.0.0-78](https://github.com/hortonworks/kafka-release/archive/HDP-3.1.0.0-78-tag.tar.gz) |[HDP 3.1 Kafka 移植指南](https://bbs.huaweicloud.com/forum/thread-38209-1-1.html) | Knox | [knox-release-HDP-3.1.0.0-78](https://github.com/hortonworks/knox-release/archive/HDP-3.1.0.0-78-tag.tar.gz) |[HDP 3.1 Knox 移植指南](https://bbs.huaweicloud.com/forum/thread-38210-1-1.html) | Livy | [livy-release-HDP-3.1.0.0-78](https://github.com/hortonworks/livy-release/archive/HDP-3.1.0.0-78-tag.tar.gz) |[HDP 3.1 Livy 移植指南](https://bbs.huaweicloud.com/forum/thread-38211-1-1.html) | Oozie | [oozie-release-HDP-3.1.0.0-78](https://github.com/hortonworks/oozie-release/archive/HDP-3.1.0.0-78-tag.tar.gz) |[HDP 3.1 Oozie 移植指南](https://bbs.huaweicloud.com/forum/thread-38212-1-1.html) | Phoenix | [phoenix-release-HDP-3.1.0.0-78](https://github.com/hortonworks/phoenix-release/archive/HDP-3.1.0.0-78-tag.tar.gz) |[HDP 3.1 Phoenix 移植指南](https://bbs.huaweicloud.com/forum/thread-38213-1-1.html) | Pig | [pig-release-HDP-3.1.0.0-78](https://github.com/hortonworks/pig-release/archive/HDP-3.1.0.0-78-tag.tar.gz) |[HDP 3.1 Pig 移植指南](https://bbs.huaweicloud.com/forum/thread-38215-1-1.html) | Ranger | [ranger-release-HDP-3.1.0.0-78](https://github.com/hortonworks/ranger-release/archive/HDP-3.1.0.0-78-tag.tar.gz) |[HDP 3.1 Ranger 移植指南](https://bbs.huaweicloud.com/forum/thread-38216-1-1.html) | Spark | [spark-release-HDP-3.1.0.0-78](https://github.com/hortonworks/spark2-release/archive/HDP-3.1.0.0-78-tag.tar.gz) |[HDP 3.1 Spark 移植指南](https://bbs.huaweicloud.com/forum/thread-38217-1-1.html) | Sqoop | [sqoop-release-HDP-3.1.0.0-78](http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.0.0/HDP-3.1.0.0-centos7-rpm.tar.gz) |[HDP 3.1 Sqoop 移植指南](https://bbs.huaweicloud.com/forum/thread-38218-1-1.html) | Storm | [storm-release-HDP-3.1.0.0-78](https://github.com/hortonworks/storm-release/archive/HDP-3.1.0.0-78-tag.tar.gz) |[HDP 3.1 Storm 移植指南](https://bbs.huaweicloud.com/forum/thread-38219-1-1.html) | Tez | [tez-release-HDP-3.1.0.0-78](https://github.com/hortonworks/tez-release/archive/HDP-3.1.0.0-78-tag.tar.gz) |[HDP 3.1 Tez 移植指南](https://bbs.huaweicloud.com/forum/thread-38220-1-1.html) | Zeppelin | [zeppelin-release-HDP-3.1.0.0-78](http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.0.0/HDP-3.1.0.0-centos7-rpm.tar.gz) |[HDP 3.1 Zeppelin 移植指南](https://bbs.huaweicloud.com/forum/thread-38224-1-1.html) | Zookeeper | 不需要移植 |[HDP 3.1 Zookeeper 移植指南](https://bbs.huaweicloud.com/forum/thread-38224-1-1.html) ## 2.2 部署篇 **HDP平台部署** #### [Ambari部署rpm源码包](https://mirrors.huaweicloud.com/kunpeng/yum/el/7/aarch64/Packages/bigdata/) | 组件 | 文档 | ---- | ---- | Ambari | [Ambari 部署指南(CentOS 7.6)](https://bbs.huaweicloud.com/forum/thread-27845-1-1.html) | Accumulo | [HDP3.1 Accumulo部署指南](https://bbs.huaweicloud.com/forum/thread-38228-1-1.html) | Atlas&Kafka&solr | [HDP3.1 Atlas&Kafka&solr部署指南](https://bbs.huaweicloud.com/forum/thread-38628-1-1.html) | Hbase&Phoenix | [HDP3.1 Hbase&Phoenix部署指南](https://bbs.huaweicloud.com/forum/thread-38629-1-1.html) | Kafka | [HDP3.1 Kafka部署指南](https://bbs.huaweicloud.com/forum/thread-38630-1-1.html) | Knox | [HDP3.1 Knox部署指南](https://bbs.huaweicloud.com/forum/thread-38631-1-1.html) | Oozie | [HDP3.1 Oozie部署指南](https://bbs.huaweicloud.com/forum/thread-38632-1-1.html) | Pig | [HDP3.1 Pig部署指南](https://bbs.huaweicloud.com/forum/thread-38633-1-1.html) | Ranger | [HDP3.1 Ranger部署指南](https://bbs.huaweicloud.com/forum/thread-38639-1-1.html) | spark&livy | [HDP3.1 spark&livy部署指南](https://bbs.huaweicloud.com/forum/thread-38641-1-1.html) | Sqoop | [HDP3.1 Sqoop部署指南](https://bbs.huaweicloud.com/forum/thread-38642-1-1.html) | Storm | [HDP3.1 Storm部署指南](https://bbs.huaweicloud.com/forum/thread-38643-1-1.html) | Zepplin | [HDP3.1 Zeppelin部署指南](https://bbs.huaweicloud.com/forum/thread-38644-1-1.html) ## 2.3 调优篇 **HDP调优** | 组件 | 文档 | ---- | ---- | ElasticSearch | [HDP 3.1.0 ElasticSearch组件性能调优指南](https://bbs.huaweicloud.com/forum/thread-26302-1-1.html) | Flink | [HDP 3.1.0 Flink组件性能调优指南](https://bbs.huaweicloud.com/forum/thread-26522-1-1.html) | Hbase | [HDP 3.1.0 HBase组件性能调优指南](https://bbs.huaweicloud.com/forum/thread-26526-1-1.html) | Hive | [HDP 3.1.0 Hive组件性能调优指南](https://bbs.huaweicloud.com/forum/thread-26527-1-1.html) | Kafka | [HDP 3.1.0 Kafka组件性能调优指南](https://bbs.huaweicloud.com/forum/thread-26528-1-1.html) | Redis | [HDP 3.1.0 Redis组件性能调优指南](https://bbs.huaweicloud.com/forum/thread-26530-1-1.html) | Spark | [HDP 3.1.0 Spark组件性能调优指南](https://bbs.huaweicloud.com/forum/thread-26532-1-1.html) | Storm | [HDP 3.1.0 Storm组件性能调优指南](https://bbs.huaweicloud.com/forum/thread-26533-1-1.html) # 3. CDH版本 ## 3.1 移植篇 **CDH平台移植** | 组件 | CDH源码包 | 文档 | | -------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | | Solr | [cdh6.3.0-release.tar.gz](https://github.com/cloudera/lucene-solr/archive/cdh6.3.0-release.tar.gz) | [Solr-7.4.0-cdh6.3.0 移植指南 (CentOS 7.6) ](https://bbs.huaweicloud.com/forum/thread-38204-1-1.html) | | Sentry | [cdh6.3.0-release.tar.gz](https://github.com/cloudera/sentry/archive/cdh6.3.0-release.tar.gz) | [Sentry-2.1.0-cdh6.3.0 移植指南 (CentOS 7.6)](https://bbs.huaweicloud.com/forum/thread-38203-1-1.html) | | Parquet-mr | [cdh6.3.0-release.tar.gz](https://github.com/cloudera/parquet-mr/archive/cdh6.3.0-release.tar.gz) | [Parquet-mr-cdh6.3.0 移植指南 (CentOS7.6)](https://bbs.huaweicloud.com/forum/thread-38201-1-1.html) | | Parquet-format | [cdh6.3.0-release.tar.gz](https://github.com/cloudera/parquet-format/archive/cdh6.3.0-release.tar.gz) | [Parquet-format-2.4.0-cdh6.3.0 移植指南(CentOS 7.6)](https://bbs.huaweicloud.com/forum/thread-38200-1-1.html) | | Kudu | [cdh5-1.7.0_5.16.1.zip](https://github.com/cloudera/kudu/archive/cdh5-1.7.0_5.16.1.zip) | [Kudu-1.7.0-cdh-5.16.1软件移植指导书](https://bbs.huaweicloud.com/forum/thread-38198-1-1.html) | | Impala | [cdh5-2.12.0_5.16.1]( https://codeload.github.com/cloudera/Impala/zip/cdh5-2.12.0_5.16.1) | [Impala-2.12.0-cdh-5.16.1软件移植(手动)指导书](https://bbs.huaweicloud.com/forum/thread-38195-1-1.html) | | Flume-ng | [cdh6.3.0-release.tar.gz](https://github.com/cloudera/flume-ng/archive/cdh6.3.0-release.tar.gz) | [Flume-ng-cdh6.3.0 移植指南 (CentOS 7.6)](https://bbs.huaweicloud.com/forum/thread-38187-1-1.html) | | Avro | [cdh6.3.0-release.tar.gz](https://github.com/cloudera/avro/archive/cdh6.3.0-release.tar.gz) | [Avro-1.8.2-cdh6.3.0 移植指南 (CentOS 7.6)](https://bbs.huaweicloud.com/forum/thread-38178-1-1.html) | # 4. 第三方依赖库 | 组件 | 源码包 | 文档 | ---- | ---- | ---- | Level DB | [leveldbjni-all-1.8.jar](https://repo.huaweicloud.com/repository/maven/kunpeng/org/fusesource/leveldbjni/leveldbjni-all/1.8/leveldbjni-all-1.8.jar) | [leveldbjni-all 移植指南 ](https://bbs.huaweicloud.com/forum/thread-22684-1-1.html) | Commons Crypto | [commons-crypto-1.0.0.jar](https://repo.huaweicloud.com/repository/maven/kunpeng/org/apache/commons/commons-crypto/1.0.0/commons-crypto-1.0.0.jar) | [commons-crypto 移植指南 ](https://bbs.huaweicloud.com/forum/thread-22748-1-1.html) | Hbase-shaded-netty | [Hbase-shaded-netty-2.1.0.tar.gz](https://github.com/apache/hbase-thirdparty/archive/rel/2.1.0.tar.gz) | [Hbase-shaded-netty 移植指南](https://bbs.huaweicloud.com/forum/thread-22746-1-1.html) | Snappy | [snappy-java-1.1.1.7](https://github.com/xerial/snappy-java/archive/1.1.1.7.zip) | [snappy-java 移植指南](https://bbs.huaweicloud.com/forum/thread-22744-1-1.html) | Jline-2.11 | [jline-2.11.jar](https://repo.huaweicloud.com/repository/maven/kunpeng/jline/jline/2.11/jline-2.11.jar) | [Jline 移植指南](https://bbs.huaweicloud.com/forum/thread-22742-1-1.html) | Jline-2.12 | [jline-2.12.jar](https://repo.huaweicloud.com/repository/maven/kunpeng/jline/jline/2.12/jline-2.12.jar) | [Jline 移植指南](https://bbs.huaweicloud.com/forum/thread-22742-1-1.html) | Jline-2.12.1 | [jline-2.12.1.jar](https://repo.huaweicloud.com/repository/maven/kunpeng/jline/jline/2.12.1/jline-2.12.1.jar) | [Jline 移植指南](https://bbs.huaweicloud.com/forum/thread-22742-1-1.html) | Jansi | [Jansi-1.4](https://github.com/fusesource/jansi/archive/jansi-1.4.tar.gz) | [Jansi-1.4 移植指南](https://bbs.huaweicloud.com/forum/thread-22740-1-1.html) | Scala | [scala-compiler-2.11.12](https://github.com/scala/scala.git) | [scala-compiler-2.11.12 移植指南](https://bbs.huaweicloud.com/forum/thread-22738-1-1.html) | Netty-4.0.23 | [netty-all-4.0.23](https://github.com/netty/netty/archive/netty-4.0.23.Final.tar.gz) | [netty-all-4.0.23 移植指南](https://bbs.huaweicloud.com/forum/thread-22724-1-1.html) | Netty-4.0.52 | [netty-all-4.0.52](https://github.com/netty/netty/archive/netty-4.0.52.Final.tar.gz) | [ netty-all-4.0.52 移植指南](https://bbs.huaweicloud.com/forum/thread-22707-1-1.html) | Netty-4.1.17 | [netty-all-4.1.17](https://github.com/netty/netty/archive/netty-4.1.17.Final.tar.gz) | [ netty-all-4.1.17 移植指南](https://bbs.huaweicloud.com/forum/thread-22723-1-1.html) | Lz4-java-1.2.0 | [lz4-1.2.0.jar](https://repo.huaweicloud.com/repository/maven/kunpeng/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.jar) | [lz4-java-1.2.0 移植指南](https://bbs.huaweicloud.com/forum/thread-22704-1-1.html) | Lz4-java-1.3.0 | [lz4-1.3.0.jar](https://repo.huaweicloud.com/repository/maven/kunpeng/net/jpountz/lz4/lz4/1.3.0/lz4-1.3.0.jar) | [lz4-java-1.3.0 移植指南](https://bbs.huaweicloud.com/forum/thread-22704-1-1.html) | Lz4-java-1.4.0 | [lz4-java-1.4.0.jar](https://repo.huaweicloud.com/repository/maven/kunpeng/org/lz4/lz4-java/1.4.0/lz4-java-1.4.0.jar) | [lz4-java-1.4.0 移植指南](https://bbs.huaweicloud.com/forum/thread-22704-1-1.html) # 5.鲲鹏解决方案篇 ### [Hadoop 2.x NUMA Aware 用户指南](https://bbs.huaweicloud.com/forum/thread-39277-1-1.html) ### [Hadoop EC 用户指南](https://bbs.huaweicloud.com/forum/thread-39279-1-1.html) ### [HBase 读取锁优化 用户指南](https://bbs.huaweicloud.com/forum/thread-39282-1-1.html) # 6. 工具/脚本 ### [Check So用户指南](https://bbs.huaweicloud.com/forum/thread-22679-1-1.html)
举报
分享

分享文章到朋友圈

分享文章到微博

寻水的鱼

发帖: 131粉丝: 35

级别 : 管理员

发消息 + 关注

发表于2019-11-4 10:03:09
直达本楼层的链接
沙发
显示全部楼层

点赞 评论 引用 举报

一只笨熊喵

发帖: 2粉丝: 1

级别 : 新手上路

发消息 + 关注

发表于2019-11-15 15:54:42
直达本楼层的链接
板凳
显示全部楼层

ambari相关的rpm包能用在哪些os版本上?是否有os版本限制?

评论
真我Realme 2019-11-18 10:48 评论

可以用在Centos操作系统上,已验证OS:CentOS 7.6

... 查看全部
一只笨熊喵 2019-11-18 15:30 评论

评论 真我Realme:多谢,后续是否有centos 7.7的验证计划?

... 查看全部
真我Realme 2019-11-19 14:45 评论

理论上centos 7.7如果内核与gcc版本与centos 7.6一样的话,rpm包还是可用的。 centos 7.7暂时没有验证计划。

... 查看全部
点赞 评论 引用 举报

KnightKong

发帖: 1粉丝: 0

级别 : 注册会员

发消息 + 关注

发表于2020-1-11 12:18:47
直达本楼层的链接
地板
显示全部楼层

看了这篇精华帖,大数据上鲲鹏云没啥好担心的了!

点赞 评论 引用 举报

聆聽心情

发帖: 7粉丝: 2

级别 : 版主

发消息 + 关注

发表于2020-1-17 09:27:48
直达本楼层的链接
5#
显示全部楼层
**很好**
点赞 评论 引用 举报

活力新征程

发帖: 37粉丝: 16

级别 : 管理员

发消息 + 关注

发表于2020-1-17 12:52:56
直达本楼层的链接
6#
显示全部楼层
**棒棒的呢

**
点赞 评论 引用 举报

五湖四海@银河护卫队

发帖: 81粉丝: 24

级别 : 版主

发消息 + 关注

发表于2020-2-11 14:15:40
直达本楼层的链接
7#
显示全部楼层

收藏~

点赞 评论 引用 举报

开心的小朋友

发帖: 6粉丝: 0

级别 : 新手上路

发消息 + 关注

发表于2020-4-18 09:38:55
直达本楼层的链接
8#
显示全部楼层

centos 7.7兼容吗?

评论
真我Realme 昨天 16:12 评论

这个没有验证过,理论上也是兼容的

... 查看全部
点赞 评论 引用 举报

游客

富文本
Markdown
您需要登录后才可以回帖 登录 | 立即注册