升级python3.7后无法连接kafka问题
【摘要】 python3.7, kafka升级失败
python从3.6升级到3.7后,kafka出现无法连接的问题,具体问题如下:
kafka-python==1.4.7以下:
Traceback (most recent call last): File "/home/yujialiang/PycharmProjects/kafka-learn/sender.py", line 1, in <module> from kafka import KafkaProducer File "/home/yujialiang/anaconda3/lib/python3.7/site-packages/kafka/__init__.py", line 23, in <module> from kafka.producer import KafkaProducer File "/home/yujialiang/anaconda3/lib/python3.7/site-packages/kafka/producer/__init__.py", line 4, in <module> from kafka.producer.simple import SimpleProducer File "/home/yujialiang/anaconda3/lib/python3.7/site-packages/kafka/producer/simple.py", line 54 return '<SimpleProducer batch=%s>' % self.async ^ SyntaxError: invalid syntax
错误原因:
kafka-python1.4.7以下版本中.async关键词和python3.7以后版本冲突,python3.7重新定义了.async关键词,故导致不能用
kafka-python==1.4.7及以上版本:
Traceback (most recent call last): File "/home/yujialiang/anaconda3/lib/python3.7/site-packages/kafka/producer/sender.py", line 60, in run self.run_once() File "/home/yujialiang/anaconda3/lib/python3.7/site-packages/kafka/producer/sender.py", line 160, in run_once self._client.poll(timeout_ms=poll_timeout_ms) File "/home/yujialiang/anaconda3/lib/python3.7/site-packages/kafka/client_async.py", line 578, in poll self._maybe_connect(node_id) File "/home/yujialiang/anaconda3/lib/python3.7/site-packages/kafka/client_async.py", line 388, in _maybe_connect conn.connect() File "/home/yujialiang/anaconda3/lib/python3.7/site-packages/kafka/conn.py", line 422, in connect if self._try_handshake(): File "/home/yujialiang/anaconda3/lib/python3.7/site-packages/kafka/conn.py", line 501, in _try_handshake self._sock.do_handshake() File "/home/yujialiang/anaconda3/lib/python3.7/ssl.py", line 1108, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: UNKNOWN_STATE] unknown state (_ssl.c:1045) ERROR:kafka.producer.sender:Uncaught error in kafka producer I/O thread
或者在接收数据时发生如下错误:
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid for '100.95.146.129'. (_ssl.c:1045)
原因:
1.版本问题导致,服务端版本合客户端版本不一致,如服务端版本kafka_2.11-0.9.0.1,客户端使用kafka自带 shell,版本 kafka_2.11-0.10.0,改为kafka_2.11-0.9.0.1 版本即可解决问题。
2.如果使用的是安全kafka(dms或者经过ssl认证),在python3.7及以上版本对匹配服务端ip地址的校验及cn认证,解决办法在初始化kafkaProducer或kafkaConsumer中设置ssl_check_hostname=False即可
【版权声明】本文为华为云社区用户原创内容,转载时必须标注文章的来源(华为云社区)、文章链接、文章作者等基本信息, 否则作者和本社区有权追究责任。如果您发现本社区中有涉嫌抄袭的内容,欢迎发送邮件进行举报,并提供相关证据,一经查实,本社区将立刻删除涉嫌侵权内容,举报邮箱:
cloudbbs@huaweicloud.com
- 点赞
- 收藏
- 关注作者
评论(0)