Hadoop-2.6.0-cdh5.7.0安装详解

举报
TT-千叶 发表于 2022/12/28 11:12:05 2022/12/28
【摘要】 下载Hadoop和JDK下载Hadoop地址:http://archive.cloudera.com/cdh5/cdh/5/hadoop-2.6.0-cdh5.7.0.tar.gz下载jdk:注册甲骨文账号下载,推荐1.7版本 安装JDK解压jdk压缩包tar -zxvf /home/hadoop/software/jdk-7u80-linux-x64.tar.gz -C /usr/jav...

下载Hadoop和JDK

安装JDK

  • 解压jdk压缩包

    tar -zxvf /home/hadoop/software/jdk-7u80-linux-x64.tar.gz -C /usr/java/

  • 配置jdk环境变量

    hadoop:root:/usr/java:>vi /etc/profile

    /etc/profile

    System wide environment and startup programs, for login setup

    Functions and aliases go in /etc/bashrc

    It’s NOT a good idea to change this file unless you know what you

    are doing. It’s much better to create a custom.sh shell script in

    /etc/profile.d/ to make custom changes to your environment, as this

    will prevent the need for merging in future updates.

    #add path
    export JAVA_HOME=/usr/java/jdk1.7.0_80
    export PATH= J A V A H O M E / b i n : JAVA_HOME/bin: PATH
    #show path

    hadoop:root:/usr/java:>source /etc/profile
    hadoop:root:/usr/java:>java -version
    java version “1.7.0_80”
    Java™ SE Runtime Environment (build 1.7.0_80-b15)
    Java HotSpot™ 64-Bit Server VM (build 24.80-b11, mixed mode)

配置shh

hadoop:hadoop:/home/hadoop:>ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/hadoop/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/hadoop/.ssh/id_rsa.
Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.
The key fingerprint is:
ca:20:e2:68:64:46:e0:f2:62:63:b9:60:71:a5:75:4a hadoop@hadoop
The key's randomart image is:
+--[ RSA 2048]----+
|.    E .         |
|o   = o          |
|.+ o .           |
|o.+              |
|+Xo .   S        |
|@oo. o .         |
|.+    o          |
|.                |
|                 |
+-----------------+
hadoop:hadoop:/home/hadoop:>
hadoop:hadoop:/home/hadoop:>cp .ssh/id_rsa.pub ~/.ssh/authorized_keys
hadoop:hadoop:/home/hadoop:>cd .ssh/
hadoop:hadoop:/home/hadoop/.ssh:>ll
total 12
-rw-r--r-- 1 hadoop hadoop  395 Jan  2 02:16 authorized_keys
-rw------- 1 hadoop hadoop 1675 Jan  2 02:16 id_rsa
-rw-r--r-- 1 hadoop hadoop  395 Jan  2 02:16 id_rsa.pub

安装Hadoop

  • 解压Hadoop
    hadoop:hadoop:/home/hadoop/app:>tar -zxvf /home/hadoop/software/hadoop-2.6.0-cdh5.7.0.tar.gz -C /home/hadoop/app/

  • 配置环境

    hadoop:hadoop:/home/hadoop:>vi .bash_profile

    .bash_profile

    Get the aliases and functions

    if [ -f ~/.bashrc ]; then
    . ~/.bashrc
    fi

    User specific environment and startup programs

    export HADOOP_HOME=/home/hadoop/app/hadoop-2.6.0-cdh5.7.0
    export PATH= H A D O O P H O M E / b i n : HADOOP_HOME/bin: HADOOP_HOME/sbin:$PATH

    hadoop:hadoop:/home/hadoop:>source .bash_profile
    hadoop:hadoop:/home/hadoop:>echo $HADOOP_HOME
    /home/hadoop/app/hadoop-2.6.0-cdh5.7.0

修改配置文件

  • hadoop-env.sh

    export JAVA_HOME=/usr/java/jdk1.7.0_80

  • core-site.xml

    <configuration> <property> <name>fs.defaultFS</name> <value>hdfs://hadoop:8020</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/home/hadoop/app/tmp</value> </property> </configuration>
  • hdfs-site.xml

    <configuration>
        <property>
                <name>dfs.replication</name>
                <value>1</value>
        </property>
    
    </configuration>
  • slaves

    hadoop

  • mapred-site.xml(正常情况下没有这个文件,可由 mapred-queues.xml.template
    复制而来)

    <configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> </configuration>
  • yarn-site.xml

    <configuration> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> </configuration>

格式化Hadoop

   hdfs namenode -format

启动Hadoop

hadoop:hadoop:/home/hadoop/app/hadoop-2.6.0-cdh5.7.0/etc/hadoop:>start-all.sh 
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
18/01/02 02:49:33 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [hadoop]
hadoop: starting namenode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-namenode-hadoop.out
hadoop: starting datanode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-datanode-hadoop.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-hadoop-secondarynamenode-hadoop.out
18/01/02 02:50:15 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
starting yarn daemons
starting resourcemanager, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/yarn-hadoop-resourcemanager-hadoop.out
hadoop: starting nodemanager, logging to /home/hadoop/app/hadoop-2.6.0-cdh5.7.0/logs/yarn-hadoop-nodemanager-hadoop.out
hadoop:hadoop:/home/hadoop/app/hadoop-2.6.0-cdh5.7.0/etc/hadoop:>jps
8345 NodeManager
8066 SecondaryNameNode
7820 NameNode
7914 DataNode
8249 ResourceManager
8613 Jps
【版权声明】本文为华为云社区用户原创内容,转载时必须标注文章的来源(华为云社区)、文章链接、文章作者等基本信息, 否则作者和本社区有权追究责任。如果您发现本社区中有涉嫌抄袭的内容,欢迎发送邮件进行举报,并提供相关证据,一经查实,本社区将立刻删除涉嫌侵权内容,举报邮箱: cloudbbs@huaweicloud.com
  • 点赞
  • 收藏
  • 关注作者

评论(0

0/1000
抱歉,系统识别当前为高风险访问,暂不支持该操作

全部回复

上滑加载中

设置昵称

在此一键设置昵称,即可参与社区互动!

*长度不超过10个汉字或20个英文字符,设置后3个月内不可修改。

*长度不超过10个汉字或20个英文字符,设置后3个月内不可修改。