Hadoop单节点配置
【摘要】 系统版本：[root@db01 opt]# cat /etc/redhat-releaseCentOS Linux release 7.8.2003 (Core)软件版本：hadoop-3.3.0.tar.gz01，解压jdk软件包：[root@db01 opt]# ll /opt/jdk1.8.0_40/total 25960drwxr-xr-x 2 10 143 4096 Feb...
系统版本：
[root@db01 opt]# cat /etc/redhat-release
CentOS Linux release 7.8.2003 (Core)
软件版本：
hadoop-3.3.0.tar.gz
01，解压jdk软件包：
[root@db01 opt]# ll /opt/jdk1.8.0_40/
total 25960
drwxr-xr-x 2 10 143 4096 Feb 11 2015 bin
-r--r--r-- 1 10 143 3244 Feb 11 2015 COPYRIGHT
drwxr-xr-x 4 10 143 122 Feb 11 2015 db
drwxr-xr-x 3 10 143 132 Feb 11 2015 include
-rw-r--r-- 1 10 143 5172315 Feb 10 2015 javafx-src.zip
drwxr-xr-x 5 10 143 185 Feb 11 2015 jre
drwxr-xr-x 5 10 143 245 Feb 11 2015 lib
-r--r--r-- 1 10 143 40 Feb 11 2015 LICENSE
drwxr-xr-x 4 10 143 47 Feb 11 2015 man
-r--r--r-- 1 10 143 159 Feb 11 2015 README.html
-rw-r--r-- 1 10 143 525 Feb 11 2015 release
-rw-r--r-- 1 10 143 21098483 Feb 11 2015 src.zip
-rw-r--r-- 1 10 143 110114 Feb 10 2015 THIRDPARTYLICENSEREADME-JAVAFX.txt
-r--r--r-- 1 10 143 178392 Feb 11 2015 THIRDPARTYLICENSEREADME.txt
02，新建用户hadoop
useradd hadoop
03,使用hadoop用户 解压hadoop软件
[hadoop@db01 ~]$ tar xvf hadoop-3.3.0.tar.gz
04，修改配置文件
etc/hadoop/core-site.xml:
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
etc/hadoop/hdfs-site.xml:
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>05,ssh配置
[hadoop@db01 hadoop-3.3.0]$ ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
Generating public/private rsa key pair.
Created directory '/home/hadoop/.ssh'.
Your identification has been saved in /home/hadoop/.ssh/id_rsa.
Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:DaJPr6X+19CpNWqzNjnyvJ8haedoGEMXxClu/wBZ/Qo hadoop@db01
The key's randomart image is:
+---[RSA 2048]----+
| o.o |
| . = . |
| ...+ . . |
| . .*oE . |
| . .oS+.o o |
| o .o +.* |
| . o++Xo. |
| +oo%=oo |
| .+..B*B+ |
+----[SHA256]-----+
[hadoop@db01 hadoop-3.3.0]$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
[hadoop@db01 hadoop-3.3.0]$ chmod 0600 ~/.ssh/authorized_keys
[hadoop@db01 hadoop-3.3.0]$ ssh localhost
The authenticity of host 'localhost (::1)' can't be established.
ECDSA key fingerprint is SHA256:ZaV0x9/JzLozY2oMF5Mt4fBwp1IvzEzKdy66JCOWWUs.
ECDSA key fingerprint is MD5:36:20:56:4e:09:49:e7:ac:9b:c4:29:16:b6:0c:08:0c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
Last login: Mon Dec 13 21:50:37 2021
06，格式化并启动hdfs
[hadoop@db01 hadoop-3.3.0]$ export JAVA_HOME=/opt/jdk1.8.0_40
[hadoop@db01 hadoop-3.3.0]$ bin/hdfs namenode -format
WARNING: /home/hadoop/hadoop-3.3.0/logs does not exist. Creating.
2021-12-13 21:55:25,115 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = db01/192.168.5.200
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 3.3.0
STARTUP_MSG: classpath = /home/hadoop/hadoop-3.3.0/etc/hadoop:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/zookeeper-jute-3.5.6.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/avro-1.7.7.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/javax.activation-api-1.2.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/hadoop-auth-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/commons-math3-3.1.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/log4j-1.2.17.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jetty-webapp-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jersey-servlet-1.19.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jetty-server-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/accessors-smart-1.2.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/kerb-server-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jackson-annotations-2.10.3.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/commons-logging-1.1.3.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/gson-2.2.4.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jettison-1.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/commons-collections-3.2.2.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/commons-beanutils-1.9.4.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/httpclient-4.5.6.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/hadoop-annotations-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/asm-5.0.4.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jetty-util-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/kerb-core-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/kerby-util-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jetty-security-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jsch-0.1.55.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/commons-compress-1.19.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/curator-framework-4.2.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jersey-core-1.19.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/commons-net-3.6.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/metrics-core-3.2.4.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jersey-json-1.19.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/dnsjava-2.1.7.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/curator-recipes-4.2.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jetty-io-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/guava-27.0-jre.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/paranamer-2.3.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/curator-client-4.2.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/commons-codec-1.11.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/kerb-client-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jetty-xml-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/stax2-api-3.1.4.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/token-provider-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/hadoop-shaded-protobuf_3_7-1.0.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/httpcore-4.4.10.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/kerb-util-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/commons-lang3-3.7.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/commons-daemon-1.0.13.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jackson-databind-2.10.3.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/checker-qual-2.5.2.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/j2objc-annotations-1.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/kerb-common-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jetty-http-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jersey-server-1.19.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/json-smart-2.3.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/animal-sniffer-annotations-1.17.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jackson-core-2.10.3.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/commons-cli-1.2.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/commons-text-1.4.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/kerby-config-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/netty-3.10.6.Final.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jsr305-3.0.2.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/commons-io-2.5.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/re2j-1.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/zookeeper-3.5.6.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/failureaccess-1.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/nimbus-jose-jwt-7.9.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jsp-api-2.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/snappy-java-1.0.5.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jul-to-slf4j-1.7.25.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/jetty-servlet-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/hadoop-registry-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/hadoop-nfs-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/hadoop-common-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/hadoop-kms-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/common/hadoop-common-3.3.0-tests.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/zookeeper-jute-3.5.6.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/avro-1.7.7.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/javax.activation-api-1.2.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/hadoop-auth-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jetty-webapp-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jetty-server-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jackson-annotations-2.10.3.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/gson-2.2.4.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jettison-1.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/commons-beanutils-1.9.4.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/httpclient-4.5.6.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/hadoop-annotations-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/asm-5.0.4.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jetty-util-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jetty-util-ajax-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jetty-security-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jsch-0.1.55.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/commons-compress-1.19.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/curator-framework-4.2.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/commons-net-3.6.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/dnsjava-2.1.7.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/curator-recipes-4.2.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jetty-io-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/guava-27.0-jre.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/paranamer-2.3.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/curator-client-4.2.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/commons-codec-1.11.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/netty-all-4.1.50.Final.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jetty-xml-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/stax2-api-3.1.4.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/hadoop-shaded-protobuf_3_7-1.0.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/httpcore-4.4.10.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/woodstox-core-5.0.3.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/commons-lang3-3.7.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/okio-1.6.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jackson-databind-2.10.3.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/checker-qual-2.5.2.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/j2objc-annotations-1.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jetty-http-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/json-smart-2.3.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/animal-sniffer-annotations-1.17.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jackson-core-2.10.3.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/commons-text-1.4.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/netty-3.10.6.Final.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jsr305-3.0.2.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/commons-io-2.5.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/re2j-1.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/zookeeper-3.5.6.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/failureaccess-1.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/nimbus-jose-jwt-7.9.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/snappy-java-1.0.5.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/jetty-servlet-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/hadoop-hdfs-nfs-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/hadoop-hdfs-native-client-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/hadoop-hdfs-client-3.3.0-tests.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/hadoop-hdfs-rbf-3.3.0-tests.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/hadoop-hdfs-3.3.0-tests.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/hadoop-hdfs-rbf-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/hadoop-hdfs-native-client-3.3.0-tests.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/hadoop-hdfs-client-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/hdfs/hadoop-hdfs-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.3.0-tests.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/bcprov-jdk15on-1.60.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.10.3.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/asm-tree-7.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/asm-analysis-7.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/jetty-jndi-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/jetty-plus-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/websocket-server-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/guice-4.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/jna-5.2.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/asm-commons-7.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/jline-3.9.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/jackson-jaxrs-base-2.10.3.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/snakeyaml-1.16.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/jersey-client-1.19.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/javax.websocket-api-1.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/websocket-servlet-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/websocket-common-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/jakarta.xml.bind-api-2.3.2.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/jetty-client-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/json-io-2.5.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/java-util-1.9.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/javax.inject-1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/jakarta.activation-api-1.2.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/javax-websocket-server-impl-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/websocket-api-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/bcpkix-jdk15on-1.60.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.10.3.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/javax-websocket-client-impl-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/websocket-client-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/objenesis-2.6.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/fst-2.50.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/jetty-annotations-9.4.20.v20190813.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/lib/javax.websocket-client-api-1.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/hadoop-yarn-server-tests-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/hadoop-yarn-services-api-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/hadoop-yarn-api-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/hadoop-yarn-client-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/hadoop-yarn-services-core-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/hadoop-yarn-applications-mawo-core-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/hadoop-yarn-server-common-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/hadoop-yarn-common-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/hadoop-yarn-registry-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/hadoop-yarn-server-router-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.3.0.jar:/home/hadoop/hadoop-3.3.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.3.0.jar
STARTUP_MSG: build = https://gitbox.apache.org/repos/asf/hadoop.git -r aa96f1871bfd858f9bac59cf2a81ec470da649af; compiled by 'brahma' on 2020-07-06T18:44Z
STARTUP_MSG: java = 1.8.0_40
************************************************************/
2021-12-13 21:55:25,137 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
2021-12-13 21:55:25,382 INFO namenode.NameNode: createNameNode [-format]
2021-12-13 21:55:26,557 INFO namenode.NameNode: Formatting using clusterid: CID-0ab580a9-8484-4a2a-8e92-ed77d2de6e29
2021-12-13 21:55:26,647 INFO namenode.FSEditLog: Edit logging is async:true
2021-12-13 21:55:26,728 INFO namenode.FSNamesystem: KeyProvider: null
2021-12-13 21:55:26,733 INFO namenode.FSNamesystem: fsLock is fair: true
2021-12-13 21:55:26,734 INFO namenode.FSNamesystem: Detailed lock hold time metrics enabled: false
2021-12-13 21:55:26,748 INFO namenode.FSNamesystem: fsOwner = hadoop (auth:SIMPLE)
2021-12-13 21:55:26,748 INFO namenode.FSNamesystem: supergroup = supergroup
2021-12-13 21:55:26,748 INFO namenode.FSNamesystem: isPermissionEnabled = true
2021-12-13 21:55:26,748 INFO namenode.FSNamesystem: isStoragePolicyEnabled = true
2021-12-13 21:55:26,749 INFO namenode.FSNamesystem: HA Enabled: false
2021-12-13 21:55:26,872 INFO common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling
2021-12-13 21:55:26,903 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit: configured=1000, counted=60, effected=1000
2021-12-13 21:55:26,903 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true
2021-12-13 21:55:26,913 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
2021-12-13 21:55:26,913 INFO blockmanagement.BlockManager: The block deletion will start around 2021 Dec 13 21:55:26
2021-12-13 21:55:26,917 INFO util.GSet: Computing capacity for map BlocksMap
2021-12-13 21:55:26,917 INFO util.GSet: VM type = 64-bit
2021-12-13 21:55:26,920 INFO util.GSet: 2.0% max memory 6.5 GB = 133.3 MB
2021-12-13 21:55:26,921 INFO util.GSet: capacity = 2^24 = 16777216 entries
2021-12-13 21:55:26,979 INFO blockmanagement.BlockManager: Storage policy satisfier is disabled
2021-12-13 21:55:26,980 INFO blockmanagement.BlockManager: dfs.block.access.token.enable = false
2021-12-13 21:55:26,997 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.threshold-pct = 0.999
2021-12-13 21:55:26,997 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.min.datanodes = 0
2021-12-13 21:55:26,997 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.extension = 30000
2021-12-13 21:55:26,999 INFO blockmanagement.BlockManager: defaultReplication = 1
2021-12-13 21:55:26,999 INFO blockmanagement.BlockManager: maxReplication = 512
2021-12-13 21:55:26,999 INFO blockmanagement.BlockManager: minReplication = 1
2021-12-13 21:55:26,999 INFO blockmanagement.BlockManager: maxReplicationStreams = 2
2021-12-13 21:55:26,999 INFO blockmanagement.BlockManager: redundancyRecheckInterval = 3000ms
2021-12-13 21:55:27,000 INFO blockmanagement.BlockManager: encryptDataTransfer = false
2021-12-13 21:55:27,000 INFO blockmanagement.BlockManager: maxNumBlocksToLog = 1000
2021-12-13 21:55:27,066 INFO namenode.FSDirectory: GLOBAL serial map: bits=29 maxEntries=536870911
2021-12-13 21:55:27,066 INFO namenode.FSDirectory: USER serial map: bits=24 maxEntries=16777215
2021-12-13 21:55:27,066 INFO namenode.FSDirectory: GROUP serial map: bits=24 maxEntries=16777215
2021-12-13 21:55:27,066 INFO namenode.FSDirectory: XATTR serial map: bits=24 maxEntries=16777215
2021-12-13 21:55:27,132 INFO util.GSet: Computing capacity for map INodeMap
2021-12-13 21:55:27,132 INFO util.GSet: VM type = 64-bit
2021-12-13 21:55:27,133 INFO util.GSet: 1.0% max memory 6.5 GB = 66.7 MB
2021-12-13 21:55:27,133 INFO util.GSet: capacity = 2^23 = 8388608 entries
2021-12-13 21:55:27,141 INFO namenode.FSDirectory: ACLs enabled? true
2021-12-13 21:55:27,141 INFO namenode.FSDirectory: POSIX ACL inheritance enabled? true
2021-12-13 21:55:27,141 INFO namenode.FSDirectory: XAttrs enabled? true
2021-12-13 21:55:27,142 INFO namenode.NameNode: Caching file names occurring more than 10 times
2021-12-13 21:55:27,154 INFO snapshot.SnapshotManager: Loaded config captureOpenFiles: false, skipCaptureAccessTimeOnlyChange: false, snapshotDiffAllowSnapRootDescendant: true, maxSnapshotLimit: 65536
2021-12-13 21:55:27,159 INFO snapshot.SnapshotManager: SkipList is disabled
2021-12-13 21:55:27,170 INFO util.GSet: Computing capacity for map cachedBlocks
2021-12-13 21:55:27,170 INFO util.GSet: VM type = 64-bit
2021-12-13 21:55:27,170 INFO util.GSet: 0.25% max memory 6.5 GB = 16.7 MB
2021-12-13 21:55:27,170 INFO util.GSet: capacity = 2^21 = 2097152 entries
2021-12-13 21:55:27,192 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10
2021-12-13 21:55:27,192 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10
2021-12-13 21:55:27,192 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25
2021-12-13 21:55:27,201 INFO namenode.FSNamesystem: Retry cache on namenode is enabled
2021-12-13 21:55:27,201 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
2021-12-13 21:55:27,206 INFO util.GSet: Computing capacity for map NameNodeRetryCache
2021-12-13 21:55:27,206 INFO util.GSet: VM type = 64-bit
2021-12-13 21:55:27,206 INFO util.GSet: 0.029999999329447746% max memory 6.5 GB = 2.0 MB
2021-12-13 21:55:27,206 INFO util.GSet: capacity = 2^18 = 262144 entries
2021-12-13 21:55:27,271 INFO namenode.FSImage: Allocated new BlockPoolId: BP-740711949-192.168.5.200-1639450527249
2021-12-13 21:55:27,399 INFO common.Storage: Storage directory /tmp/hadoop-hadoop/dfs/name has been successfully formatted.
2021-12-13 21:55:27,470 INFO namenode.FSImageFormatProtobuf: Saving image file /tmp/hadoop-hadoop/dfs/name/current/fsimage.ckpt_0000000000000000000 using no compression
2021-12-13 21:55:27,773 INFO namenode.FSImageFormatProtobuf: Image file /tmp/hadoop-hadoop/dfs/name/current/fsimage.ckpt_0000000000000000000 of size 401 bytes saved in 0 seconds .
2021-12-13 21:55:27,834 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
2021-12-13 21:55:27,845 INFO namenode.FSImage: FSImageSaver clean checkpoint: txid=0 when meet shutdown.
2021-12-13 21:55:27,845 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at db01/192.168.5.200
************************************************************/
#启动hdfs:
启动报错信息：
[hadoop@db01 hadoop-3.3.0]$ sbin/start-dfs.sh
Starting namenodes on [db01]
db01: Warning: Permanently added 'db01,fe80::97b1:582a:8aa5:efc2%ens192' (ECDSA) to the list of known hosts.
db01: ERROR: JAVA_HOME is not set and could not be found.
Starting datanodes
localhost: ERROR: JAVA_HOME is not set and could not be found.
Starting secondary namenodes [db01]
db01: ERROR: JAVA_HOME is not set and could not be found.
#解决办法修改，配置文件，添加JAVA_HOME
etc/hadoop/hadoop-env.sh
export JAVA_HOME=/opt/jdk1.8.0_40#启动HDFS:
[hadoop@db01 hadoop-3.3.0]$ sbin/start-dfs.sh
Starting namenodes on [db01]
Starting datanodes
Starting secondary namenodes [db01]
[hadoop@db01 hadoop-3.3.0]$ /opt/jdk1.8.0_40/bin/jps
9092 NameNode
9445 SecondaryNameNode
9734 Jps
9213 DataNode
07,检查hdfs，并建立目录
[hadoop@db01 hadoop-3.3.0]$ ./bin/hdfs dfs -mkdir /test
[hadoop@db01 hadoop-3.3.0]$ ./bin/hdfs dfs -ls /
Found 1 items
drwxr-xr-x - hadoop supergroup 0 2021-12-13 22:02 /test
【版权声明】本文为华为云社区用户原创内容，转载时必须标注文章的来源（华为云社区）、文章链接、文章作者等基本信息， 否则作者和本社区有权追究责任。如果您发现本社区中有涉嫌抄袭的内容，欢迎发送邮件进行举报，并提供相关证据，一经查实，本社区将立刻删除涉嫌侵权内容，举报邮箱：
cloudbbs@huaweicloud.com
- 点赞
- 收藏
- 关注作者
评论（0）