- 微信
- 微博
  
  分享文章到微博
- 复制链接
  
  复制链接到剪贴板

大数据技术HDFS之Shell指令

tea_year 发表于 2024/02/15 08:49:23 2024/02/15

【摘要】 hdfs dfs -ls和linux的命令差不多hdfs dfs -查看相关帮助命令hadoop dfs -ls /hadoop fs -ls /版本不同创建目录hdfs dfs -mkdir -p /aaa/01/02hdfs dfs -lsr /aaa会看到01 02二．使用dfs创建文件查看文件结果echo "hello,zhangChenguang">>localfilehdfs d...

hdfs dfs -ls和linux的命令差不多

hdfs dfs -查看相关帮助命令

hadoop dfs -ls /

hadoop fs -ls /

版本不同

创建目录

hdfs dfs -mkdir -p /aaa/01/02

hdfs dfs -lsr /aaa

会看到01 02

二．使用dfs创建文件

查看文件

结果

echo "hello,zhangChenguang">>localfile

hdfs dfs -appendToFile localfile output3/file2

一直报错：

Failed to replace a bad datanode on the existing pipeline due to no more good datanodes being available to try. (Nodes: current=[DatanodeInfoWithStorage[192.168.80.131:50010,DS-ae4d2aae-49b9-4d3f-bbf1-a255217746b8,DISK], DatanodeInfoWithStorage[192.168.80.132:50010,DS-2e26c66c-f173-489a-af5c-671377ec4eae,DISK]], original=[DatanodeInfoWithStorage[192.168.80.131:50010,DS-ae4d2aae-49b9-4d3f-bbf1-a255217746b8,DISK], DatanodeInfoWithStorage[192.168.80.132:50010,DS-2e26c66c-f173-489a-af5c-671377ec4eae,DISK]]). The current failed datanode replacement policy is DEFAULT, and a client may configure this via 'dfs.client.block.write.replace-datanode-on-failure.policy' in its configuration.

at org.apache.hadoop.hdfs.DataStreamer.findNewDatanode(DataStreamer.java:1281)

at org.apache.hadoop.hdfs.DataStreamer.addDatanode2ExistingPipeline(DataStreamer.java:1351)

at org.apache.hadoop.hdfs.DataStreamer.handleDatanodeReplacement(DataStreamer.java:1539)

at org.apache.hadoop.hdfs.DataStreamer.setupPipelineForAppendOrRecovery(DataStreamer.java:1440)

at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:708)

一个是节点原因，新建不报错，但是追加报错：

最后发现解决办法，在hdfs-site.xml增加内容

<name>

dfs.client.block.write.replace-datanode-on-failure.enable

</name>

</property>

<name>