升级python3.7后无法连接kafka问题
【摘要】 python3.7, kafka升级失败
python从3.6升级到3.7后,kafka出现无法连接的问题,具体问题如下:
kafka-python==1.4.7以下:
Traceback (most recent call last):
File "/home/yujialiang/PycharmProjects/kafka-learn/sender.py", line 1, in <module>
from kafka import KafkaProducer
File "/home/yujialiang/anaconda3/lib/python3.7/site-packages/kafka/__init__.py", line 23, in <module>
from kafka.producer import KafkaProducer
File "/home/yujialiang/anaconda3/lib/python3.7/site-packages/kafka/producer/__init__.py", line 4, in <module>
from kafka.producer.simple import SimpleProducer
File "/home/yujialiang/anaconda3/lib/python3.7/site-packages/kafka/producer/simple.py", line 54
return '<SimpleProducer batch=%s>' % self.async
^
SyntaxError: invalid syntax
错误原因:
kafka-python1.4.7以下版本中.async关键词和python3.7以后版本冲突,python3.7重新定义了.async关键词,故导致不能用
kafka-python==1.4.7及以上版本:
Traceback (most recent call last):
File "/home/yujialiang/anaconda3/lib/python3.7/site-packages/kafka/producer/sender.py", line 60, in run
self.run_once()
File "/home/yujialiang/anaconda3/lib/python3.7/site-packages/kafka/producer/sender.py", line 160, in run_once
self._client.poll(timeout_ms=poll_timeout_ms)
File "/home/yujialiang/anaconda3/lib/python3.7/site-packages/kafka/client_async.py", line 578, in poll
self._maybe_connect(node_id)
File "/home/yujialiang/anaconda3/lib/python3.7/site-packages/kafka/client_async.py", line 388, in _maybe_connect
conn.connect()
File "/home/yujialiang/anaconda3/lib/python3.7/site-packages/kafka/conn.py", line 422, in connect
if self._try_handshake():
File "/home/yujialiang/anaconda3/lib/python3.7/site-packages/kafka/conn.py", line 501, in _try_handshake
self._sock.do_handshake()
File "/home/yujialiang/anaconda3/lib/python3.7/ssl.py", line 1108, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: UNKNOWN_STATE] unknown state (_ssl.c:1045)
ERROR:kafka.producer.sender:Uncaught error in kafka producer I/O thread
或者在接收数据时发生如下错误:
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid for '100.95.146.129'. (_ssl.c:1045)
原因:
1.版本问题导致,服务端版本合客户端版本不一致,如服务端版本kafka_2.11-0.9.0.1,客户端使用kafka自带 shell,版本 kafka_2.11-0.10.0,改为kafka_2.11-0.9.0.1 版本即可解决问题。
2.如果使用的是安全kafka(dms或者经过ssl认证),在python3.7及以上版本对匹配服务端ip地址的校验及cn认证,解决办法在初始化kafkaProducer或kafkaConsumer中设置ssl_check_hostname=False即可
【声明】本内容来自华为云开发者社区博主,不代表华为云及华为云开发者社区的观点和立场。转载时必须标注文章的来源(华为云社区)、文章链接、文章作者等基本信息,否则作者和本社区有权追究责任。如果您发现本社区中有涉嫌抄袭的内容,欢迎发送邮件进行举报,并提供相关证据,一经查实,本社区将立刻删除涉嫌侵权内容,举报邮箱:
cloudbbs@huaweicloud.com
- 点赞
- 收藏
- 关注作者
评论(0)