error: scrapy TypeError: 'float' object is not iterable

举报
彭世瑜 发表于 2021/08/13 23:35:08 2021/08/13
【摘要】 正常运行爬虫,在公司电脑上没问题,回到家自己电脑上就出现了下面的报错,百思不得解 Traceback (most recent call last): File "D:\.virtualenvs\spider\lib\site-packages\twisted\internet\defer.py", line 1384, in _inlineCallbacks r...

正常运行爬虫,在公司电脑上没问题,回到家自己电脑上就出现了下面的报错,百思不得解

Traceback (most recent call last):
  File "D:\.virtualenvs\spider\lib\site-packages\twisted\internet\defer.py", line 1384, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g)
  File "D:\.virtualenvs\spider\lib\site-packages\twisted\python\failure.py", line 422, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb)
  File "D:\.virtualenvs\spider\lib\site-packages\scrapy\core\downloader\middleware.py", line 43, in process_request defer.returnValue((yield download_func(request=request,spider=spider)))
  File "D:\.virtualenvs\spider\lib\site-packages\scrapy\utils\defer.py", line 45, in mustbe_deferred result = f(*args, **kw)
  File "D:\.virtualenvs\spider\lib\site-packages\scrapy\core\downloader\handlers\__init__.py", line 65, in download_request return handler.download_request(request, spider)
  File "D:\.virtualenvs\spider\lib\site-packages\scrapy\core\downloader\handlers\http11.py", line 60, in download_request return agent.download_request(request)
  File "D:\.virtualenvs\spider\lib\site-packages\scrapy\core\downloader\handlers\http11.py", line 285, in download_request method, to_bytes(url, encoding='ascii'), headers, bodyproducer)
  File "D:\.virtualenvs\spider\lib\site-packages\twisted\web\client.py", line 1657, in request parsedURI.originForm)
  File "D:\.virtualenvs\spider\lib\site-packages\twisted\web\client.py", line 1435, in _requestWithEndpoint d = self._pool.getConnection(key, endpoint)
  File "D:\.virtualenvs\spider\lib\site-packages\twisted\web\client.py", line 1320, in getConnection return self._newConnection(key, endpoint)
  File "D:\.virtualenvs\spider\lib\site-packages\twisted\web\client.py", line 1332, in _newConnection return endpoint.connect(factory)
  File "D:\.virtualenvs\spider\lib\site-packages\twisted\internet\endpoints.py", line 924, in connect EndpointReceiver, self._hostText, portNumber=self._port
  File "D:\.virtualenvs\spider\lib\site-packages\twisted\internet\_resolver.py", line 189, in resolveHostName onAddress = self._simpleResolver.getHostByName(hostName)
  File "D:\.virtualenvs\spider\lib\site-packages\scrapy\resolver.py", line 21, in getHostByName d = super(CachingThreadedResolver, self).getHostByName(name, timeout)
  File "D:\.virtualenvs\spider\lib\site-packages\twisted\internet\base.py", line 276, in getHostByName timeoutDelay = sum(timeout)
TypeError: 'float' object is not iterable
  
 
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32

网上查到是scrapy与twisted版本不兼容的问题
当前版本:

Scrapy==1.1.2
Twisted==18.4.0
  
 
  • 1
  • 2

卸载twisted重装

pip uninstall twisted

pip install twisted==16.6.0
  
 
  • 1
  • 2
  • 3

参考

  1. scrapy domz报错TypeError: ‘float’ object is not iterable
  2. Scapry does not work with Twisted version 17.1.0

文章来源: pengshiyu.blog.csdn.net,作者:彭世瑜,版权归原作者所有,如需转载,请联系作者。

原文链接:pengshiyu.blog.csdn.net/article/details/82079084

【版权声明】本文为华为云社区用户转载文章,如果您发现本社区中有涉嫌抄袭的内容,欢迎发送邮件进行举报,并提供相关证据,一经查实,本社区将立刻删除涉嫌侵权内容,举报邮箱: cloudbbs@huaweicloud.com
  • 点赞
  • 收藏
  • 关注作者

评论(0

0/1000
抱歉,系统识别当前为高风险访问,暂不支持该操作

全部回复

上滑加载中

设置昵称

在此一键设置昵称,即可参与社区互动!

*长度不超过10个汉字或20个英文字符,设置后3个月内不可修改。

*长度不超过10个汉字或20个英文字符,设置后3个月内不可修改。