双层flume搭建过程中遇到的坑
【摘要】 INFO - Creating /user/hive/warehouse/ods.db/origin_user_behavior/19-04-19/events.1555671386480.tmp
ERROR - Avro source r2: Unable to process event batch. Exception follows.
org.apache.flume....
INFO - Creating /user/hive/warehouse/ods.db/origin_user_behavior/19-04-19/events.1555671386480.tmp
ERROR - Avro source r2: Unable to process event batch. Exception follows.
org.apache.flume.ChannelFullException: Space for commit to queue couldn't be acquired. Sinks are likely not keeping up with sources, or the buffer size is
解决方法：
先将flume的JVM内存改为512M
capacity = 1000
transactionCapacity = 1000
keep-alive = 30
ERROR - Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: Failed to send events at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:389) at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67) at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145) at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: hadoop104, port: 4545 }: Failed to send batch at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:314) at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:373) ... 3 more
Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: hadoop104, port: 4545 }: Avro RPC call returned Status: FAILED at org.apache.flume.api.NettyAvroRpcClient.waitForStatusOK(NettyAvroRpcClient.java:390) at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:373) at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:302) ... 4 more
image.png
请求超时,导致发送event失败
解决：
设置request-timeout长一点，默认20秒
文章来源: www.jianshu.com，作者：百忍成金的虚竹，版权归原作者所有，如需转载，请联系作者。
原文链接：www.jianshu.com/p/219a7bd27238
推荐
华为开发者空间发布
让每位开发者拥有一台云主机
【版权声明】本文为华为云社区用户转载文章，如果您发现本社区中有涉嫌抄袭的内容，欢迎发送邮件进行举报，并提供相关证据，一经查实，本社区将立刻删除涉嫌侵权内容，举报邮箱：
cloudbbs@huaweicloud.com
- 点赞
- 收藏
- 关注作者
评论（0）