如何解决使用scrapy重定向strava.com中的授权问题 Log说strava将我从/ login重定向到/ login
我真的需要您的帮助:已经尝试了一切!目标-使用刮y授权https://www.strava.com/login
那是我的代码:
import scrapy
from scrapy.http import FormRequest
from scrapy.utils.response import open_in_browser
class StravaSpider(scrapy.Spider):
name = 'strava'
start_urls = ('https://www.strava.com/dashboard',)
def parse(self,response):
token = response.xpath('//*[@name="csrf-token"]/@content').get()
return FormRequest.from_response(response,formdata={
'authenticity_token': token,'plan': "",'email': 'login','password': 'password'},#dont_filter=True,#meta={'dont_redirect': True,'handle_httpstatus_list': [302]},callback=self.scrape_page)
def scrape_page(self,response):
print('okkkk','\n\n\n\n')
open_in_browser(response)
问题出在重定向:
2020-08-16 20:18:49 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (302) to <GET https://www.strava.com/dashboard> from <POST https://www.strava.com/session>
2020-08-16 20:18:49 [scrapy_user_agents.middlewares] DEBUG: Assigned User-Agent Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML,like Gecko) Chrome/67.0.3396.79 Safari/537.36
2020-08-16 20:18:49 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (301) to <GET https://www.strava.com/login> from <GET https://www.strava.com/dashboard>
2020-08-16 20:18:49 [scrapy_user_agents.middlewares] DEBUG: Assigned User-Agent Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML,like Gecko) Chrome/49.0.2623.110 Safari/537.36
2020-08-16 20:18:50 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (302) to <GET https://www.strava.com/dashboard> from <GET https://www.strava.com/login>
2020-08-16 20:18:50 [scrapy.dupefilters] DEBUG: Filtered duplicate request: <GET https://www.strava.com/dashboard> - no more duplicates will be shown (see DUPEFILTER_DEBUG to show all duplicates)
2020-08-16 20:18:50 [scrapy.core.engine] INFO: Closing spider (finished)
我关闭了重复过滤器,添加了handle_httpstatus_list,在设置中添加了scrapy-redirect...。 请不要使用bs4或selenium-我已经对它们进行过此程序,现在我只需要草皮,并且此授权...让我哭泣
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。