hadoop安装和启动指导

1、hadoop的官网下载和各版本下载方法:（推荐下载hadoop-3.2.4版本）
https://blog.csdn.net/leileihhxx/article/details/122181215

2、在/etc/profile文件添加如下环境变量：
JAVA_HOME=/usr/java/jdk1.8.0_171
CLASSPATH=$JAVA_HOME/lib/tools.jar
PATH=$JAVA_HOME/bin:$PATH
HADOOP_HOME=/usr/hadoop-3.2.4
PATH=$HADOOP_HOME/bin:$PATH
export JAVA_HOME CLASSPATH PATH USER LOGNAME MAIL HOSTNAME HISTSIZE HISTCONTROL
3、重载配置文件：source /etc/profile
4、hadoop version 查看是否安装成功。
5、配置和安装jdk1.8，网上有大量文档指导，此处不再详解。 
6、修改/etc/hadoop/core-site.xml、/etc/hadoop/ hdfs-site.xml两个文件
/etc/hadoop/core-site.xml添加如下代码，设置hdfs访问的路径和hdfs保存文件的路径。
<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://localhost:9000</value>
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>/usr/hadoop-3.2.4/tmp</value>
    </property>
</configuration>
/etc/hadoop/ hdfs-site.xml添加如下代码，设置副本数量
<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>
7、 Hadoop namenode 格式化
    cd /usr/hadoop-3.2.4/sbin
    hdfs namenode -format 
成功则会看到”successfully formatted”和”Exitting with status 0”的提示，若为 “Exitting with status 1” 则是出错。
8、启动hadoop
    ./start-all.sh

第一次启动报错信息如下：
[root@flink1 sbin]# ./stop-all.sh
Stopping namenodes on [0.0.0.0]
ERROR: Attempting to operate on hdfs namenode as root
ERROR: but there is no HDFS_NAMENODE_USER defined. Aborting operation.
Stopping datanodes
ERROR: Attempting to operate on hdfs datanode as root
ERROR: but there is no HDFS_DATANODE_USER defined. Aborting operation.
Stopping secondary namenodes [flink1]
ERROR: Attempting to operate on hdfs secondarynamenode as root
ERROR: but there is no HDFS_SECONDARYNAMENODE_USER defined. Aborting operation.
Stopping nodemanagers
ERROR: Attempting to operate on yarn nodemanager as root
ERROR: but there is no YARN_NODEMANAGER_USER defined. Aborting operation.
Stopping resourcemanager
ERROR: Attempting to operate on yarn resourcemanager as root
ERROR: but there is no YARN_RESOURCEMANAGER_USER defined. Aborting operation.
解决办法：
在/etc/profile文件添加如下环境变量：
export HDFS_NAMENODE_USER=root
export HDFS_DATANODE_USER=root
export HDFS_SECONDARYNAMENODE_USER=root
export YARN_RESOURCEMANAGER_USER=root
export YARN_NODEMANAGER_USER=root

第二次启动报错信息如下：
[root@flink1 sbin]# ./start-all.sh
Starting namenodes on [localhost]
Last login: Mon Feb 20 07:06:48 UTC 2023 from localhost on pts/7
localhost: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
Starting datanodes
Last login: Mon Feb 20 07:36:14 UTC 2023 on pts/6
localhost: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
Starting secondary namenodes [flink1]
Last login: Mon Feb 20 07:36:15 UTC 2023 on pts/6
flink1: Warning: Permanently added 'flink1,192.168.200.16' (ECDSA) to the list of known hosts.
flink1: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
Starting resourcemanager
Last login: Mon Feb 20 07:36:16 UTC 2023 on pts/6
Starting nodemanagers
Last login: Mon Feb 20 07:36:22 UTC 2023 on pts/6
localhost: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
解决办法：
1、输入命令：ssh-keygen -t rsa
备注：① 输入生成密码的命令，ssh-keygen命令是生成秘钥，-t是加密类型，rsa是一种加密方式 ② 回车继续 ③ 回车继续 ④ 回车完成，
2、查看文件是否生成：
输入命令：ls -l ~/.ssh  或   cat ~/.ssh/id_rsa.pub
备注：配置完成后，会在当前用户的家目录的.ssh目录生成两个文件，一个是id_rsa即私钥，另一个是id_rsa.pub即公钥。
3、复制公钥到本地用户，ssh-copy-id -i ~/.ssh/id_rsa.pub root@localhost
如果遇到输入密码，通常默认密码是root

启动成功：
[root@flink1 sbin]# ./start-all.sh
Starting namenodes on [localhost]
Last login: Mon Feb 20 07:43:09 UTC 2023 on pts/6
Starting datanodes
Last login: Mon Feb 20 07:43:24 UTC 2023 on pts/6
Starting secondary namenodes [flink1]
Last login: Mon Feb 20 07:43:27 UTC 2023 on pts/6
Starting resourcemanager
Last login: Mon Feb 20 07:43:31 UTC 2023 on pts/6
Starting nodemanagers
Last login: Mon Feb 20 07:43:38 UTC 2023 on pts/6
[root@flink1 sbin]#