如何解决scrapy 2.3.0中可能存在错误语法无效async = False
当我尝试在AWS ubuntu 18.04实例中运行scrapy时,我不断收到语法错误:
scrapy crawl pcz -o px.csv
这是日志
ubuntu@ip-172-31-60-245:~/free_proxy/free_proxy$ scrapy crawl pcz -o px.csv
2020-08-27 14:09:37 [scrapy.utils.log] INFO: Scrapy 2.3.0 started (bot: free_proxy)
2020-08-27 14:09:37 [scrapy.utils.log] INFO: Versions: lxml 4.5.2.0,libxml2 2.9.10,cssselect 1.1.0,parsel 1.6.0,w3lib 1.22.0,Twisted 17.9.0,Python 3.8.5 (default,Jul 20 2020,19:48:14) - [GCC 7.5.0],pyOpenSSL 17.5.0 (OpenSSL 1.1.1 11 Sep 2018),cryptography 2.1.4,Platform Linux-5.3.0-1033-aws-x86_64-with-glibc2.27
2020-08-27 14:09:37 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.epollreactor.EPollReactor
2020-08-27 14:09:37 [scrapy.crawler] INFO: Overridden settings:
{'BOT_NAME': 'free_proxy','CONCURRENT_REQUESTS_PER_DOMAIN': 2,'DOWNLOAD_TIMEOUT': 10,'NEWSPIDER_MODULE': 'free_proxy.spiders','RETRY_HTTP_CODES': [500,503,504,400,403,404,408,401],'RETRY_TIMES': 10,'SPIDER_MODULES': ['free_proxy.spiders']}
2020-08-27 14:09:37 [scrapy.middleware] WARNING: Disabled TelnetConsole: TELNETCONSOLE_ENABLED setting is True but required twisted modules failed to import:
Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.8/site-packages/scrapy/extensions/telnet.py",line 15,in <module>
from twisted.conch import manhole,telnet
File "/usr/lib/python3/dist-packages/twisted/conch/manhole.py",line 154
def write(self,data,async=False):
^
SyntaxError: invalid syntax
2020-08-27 14:09:37 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats','scrapy.extensions.memusage.MemoryUsage','scrapy.extensions.feedexport.FeedExporter','scrapy.extensions.logstats.LogStats']
Unhandled error in Deferred:
Temporarily disabling observer LegacyLogObserverWrapper(<bound method PythonLoggingObserver.emit of <twisted.python.log.PythonLoggingObserver object at 0x7fe474901f10>>) due to exception: [Failure instance: Traceback: <class 'TypeError'>: _findCaller() takes from 1 to 2 positional arguments but 3 were given
/home/ubuntu/.local/lib/python3.8/site-packages/scrapy/cmdline.py:153:_run_command
/usr/lib/python3/dist-packages/twisted/internet/defer.py:954:__del__
/usr/lib/python3/dist-packages/twisted/logger/_logger.py:261:critical
/usr/lib/python3/dist-packages/twisted/logger/_logger.py:135:emit
--- <exception caught here> ---
/usr/lib/python3/dist-packages/twisted/logger/_observer.py:131:__call__
/usr/lib/python3/dist-packages/twisted/logger/_legacy.py:93:__call__
/usr/lib/python3/dist-packages/twisted/python/log.py:595:emit
/usr/lib/python3/dist-packages/twisted/logger/_legacy.py:154:publishToNewObserver
/usr/lib/python3/dist-packages/twisted/logger/_stdlib.py:115:__call__
/usr/lib/python3.8/logging/__init__.py:1500:log
/usr/lib/python3.8/logging/__init__.py:1565:_log
]
Temporarily disabling observer LegacyLogObserverWrapper(<bound method PythonLoggingObserver.emit of <twisted.python.log.PythonLoggingObserver object at 0x7fe474901f10>>) due to exception: [Failure instance: Traceback: <class 'TypeError'>: _findCaller() takes from 1 to 2 positional arguments but 3 were given
/home/ubuntu/.local/lib/python3.8/site-packages/scrapy/cmdline.py:153:_run_command
/usr/lib/python3/dist-packages/twisted/internet/defer.py:963:__del__
/usr/lib/python3/dist-packages/twisted/logger/_logger.py:181:failure
/usr/lib/python3/dist-packages/twisted/logger/_logger.py:135:emit
--- <exception caught here> ---
/usr/lib/python3/dist-packages/twisted/logger/_observer.py:131:__call__
/usr/lib/python3/dist-packages/twisted/logger/_legacy.py:93:__call__
/usr/lib/python3/dist-packages/twisted/python/log.py:595:emit
/usr/lib/python3/dist-packages/twisted/logger/_legacy.py:154:publishToNewObserver
/usr/lib/python3/dist-packages/twisted/logger/_stdlib.py:115:__call__
/usr/lib/python3.8/logging/__init__.py:1500:log
/usr/lib/python3.8/logging/__init__.py:1565:_log
]
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。