如何使用Python搜索HTML元素? Python Web Bot错误

如何解决如何使用Python搜索HTML元素? Python Web Bot错误

我正在按照教程创建一个可以在任何网页上执行任务的机器人。我正在使用Python3搜索任何随机网站,然后使用搜索结果(来自该网站)打印数据。我已经导入了一个硒Web驱动程序,并确保已正确设置它。

我面临的问题是我正在尝试创建一个遍历搜索结果的for循环。这个for循环使用的是我正在测试的网站上的类名称-这样该机器人就可以识别文章元素。问题是类名称是:c-entry-box--compact__title 这是造成 SyntaxError: Cannot assign to literal

有什么办法解决吗?该网站的搜索结果没有其他较短的类名或ID,也没有连字符或下划线。我正在测试网站“ theverge”的搜索结果上运行代码。

相关代码:

try:
    main = WebDriverWait(driver,10).until(
        EC.presence_of_element_located((By.CLASS_NAME,"c-entry-box--compact__body"))
    )

    articles = main.find_element_by_class_name("c-entry-box--compact__title")

    for "c-entry-box--compact__title" in articles:
        header = articles.find_element_by_class_name("c-entry-box--compact__title")
        print(header.text)
finally:
    driver.quit()

任何向我指出正确方向的提示或想法,将不胜感激!

更新:21/8下午11:44

我为类名创建了一个变量。现在的错误是

...line 28,in <module>
for article in articles:
TypeError: 'WebElement' object is not iterable

更新12:12 am 22/8

我对最近的海报进行了更改,并调整了一些代码。我现在得到的唯一错误是与键盘输入或键的使用有关。这是AttributeError: 'list' object has no attribute 'send_keys'我的代码是

search_button = driver.find_elements_by_id("icon-search")
search = driver.find_elements_by_name("q")
search.send_keys('facebook')
search.send_keys(Keys.RETURN)

解决方法

要获取所有标题文本,请引出WebDriverWait()并等待visibility_of_all_elements_located()并跟随 css 选择器。

driver.get("https://www.theverge.com/")
headerelements=WebDriverWait(driver,20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR,"div.c-entry-box--compact__body>h2>a")))
for head in headerelements:
    print(head.text)

控制台输出:

BRYDGE’S LATEST KEYBOARDS TURN A SURFACE PRO OR GO INTO A STANDARD LAPTOP
Ikea gives its 2021 catalog an Animal Crossing-themed makeover in Taiwan
School nurses are on the frontlines of the pandemic
AN INNOCENT TYPO LED TO A GIANT 212-STORY OBELISK IN MICROSOFT FLIGHT SIMULATOR
The epic campaign to win Elon Musk’s Tesla factory with memes
NASA is going to try to hunt down a leak on the International Space Station this weekend
What’s the best student laptop? We asked students
Goodbye to Patriot Act,a comedy show that was a different kind of angry
How to pick the right headphones for kids
Swipe left,Elon stans: that Tesla dating app is a joke,for now
Leaked Google Pixel 5 renders show dual rear camera and fingerprint sensor
Minecraft Education is perfectly suited for this surreal back-to-school moment
What we listen to while working from home
Samsung’s Galaxy S20 is receiving Note 20 features with new One UI update
Facebook’s old web design will disappear in September
Apple reportedly using cheaper iPhone battery parts to offset 5G cost
THE VERGE’S BACK TO SCHOOL SPECIAL
Epic to host a #FreeFortnite tournament with anti-Apple prizes
After inking a deal with Netflix,Trump impersonator Sarah Cooper is also getting a TV show
Magic Leap’s lost work The Last Light gets a surprise release after its developers were laid off
Android 11 phones will summon Android Auto wirelessly,no need to pull out your device
HOW FORTNITE’S EPIC BATTLE WITH APPLE COULD RESHAPE THE ANTITRUST FIGHT
Adobe accidentally deleted people’s photos in latest Lightroom update
Major news publishers ask Apple what can get them an App Store deal like Amazon’s
Tesla is working on a sensor that can detect a child left behind in a hot car
Fertility app Premom reportedly shared customer data with Chinese companies
Mark Zuckerberg testified before the FTC as part of its Facebook antitrust probe
How to get Microsoft’s xCloud and stream Xbox games on your phone right now
Where to sit on the school bus just got a lot more complicated
Former Uber security chief charged with paying hush money to cover up 2016 hack
Google confirms Android 11 will limit third-party camera apps because of location spying fears
Uber and Lyft shutdown in California averted as judge grants emergency stay
Netflix is re-creating iconic Stranger Things sets in LA,and you can drive your car through them
Google’s Pixel Buds are now available in more colors nearly four months after launch
Airbnb puts global ban on house parties to support social distancing guidelines
HOUSES ARE INFLUENCERS NOW,AND THIS ONE BURNED TO THE GROUND
Lyft will suspend its ride-hailing service in California
Reddit reports 18 percent reduction in hateful content after banning nearly 7,000 subreddits
A mail-in COVID-19 test company switched to FedEx because of USPS delays
Steve Bannon charged with fraud over crowdfunded border wall
Razer gets into the ergonomic game with its new $99.99 Pro Click wireless mouse
SAMSUNG GALAXY NOTE 20 ULTRA REVIEW: BIG PHONE,SMALL UPDATES
Google’s Pixel Buds get new transcribe mode,attention alerts,and sharing detection
Control’s publisher explains why it won’t offer a free next-gen upgrade
SpaceX still pressing ahead with its Air Force lawsuit,despite winning coveted Air Force contract
We're building great things,and we need your talent.
DoorDash launches grocery delivery to compete with Amazon and Instacart

您的脚本有问题 articles = main.find_element_by_class_name("c-entry-box--compact__title")

find_element_by_class_name()将返回单个Web元素。要获取元素列表,您需要使用find_elements_by_class_name()

因此应该是

articles = main.find_elements_by_class_name("c-entry-box--compact__title")

但是我建议使用线性的方法。

版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。

相关推荐


依赖报错 idea导入项目后依赖报错,解决方案:https://blog.csdn.net/weixin_42420249/article/details/81191861 依赖版本报错:更换其他版本 无法下载依赖可参考:https://blog.csdn.net/weixin_42628809/a
错误1:代码生成器依赖和mybatis依赖冲突 启动项目时报错如下 2021-12-03 13:33:33.927 ERROR 7228 [ main] o.s.b.d.LoggingFailureAnalysisReporter : *************************** APPL
错误1:gradle项目控制台输出为乱码 # 解决方案:https://blog.csdn.net/weixin_43501566/article/details/112482302 # 在gradle-wrapper.properties 添加以下内容 org.gradle.jvmargs=-Df
错误还原:在查询的过程中,传入的workType为0时,该条件不起作用 &lt;select id=&quot;xxx&quot;&gt; SELECT di.id, di.name, di.work_type, di.updated... &lt;where&gt; &lt;if test=&qu
报错如下,gcc版本太低 ^ server.c:5346:31: 错误:‘struct redisServer’没有名为‘server_cpulist’的成员 redisSetCpuAffinity(server.server_cpulist); ^ server.c: 在函数‘hasActiveC
解决方案1 1、改项目中.idea/workspace.xml配置文件,增加dynamic.classpath参数 2、搜索PropertiesComponent,添加如下 &lt;property name=&quot;dynamic.classpath&quot; value=&quot;tru
删除根组件app.vue中的默认代码后报错:Module Error (from ./node_modules/eslint-loader/index.js): 解决方案:关闭ESlint代码检测,在项目根目录创建vue.config.js,在文件中添加 module.exports = { lin
查看spark默认的python版本 [root@master day27]# pyspark /home/software/spark-2.3.4-bin-hadoop2.7/conf/spark-env.sh: line 2: /usr/local/hadoop/bin/hadoop: No s
使用本地python环境可以成功执行 import pandas as pd import matplotlib.pyplot as plt # 设置字体 plt.rcParams[&#39;font.sans-serif&#39;] = [&#39;SimHei&#39;] # 能正确显示负号 p
错误1:Request method ‘DELETE‘ not supported 错误还原:controller层有一个接口,访问该接口时报错:Request method ‘DELETE‘ not supported 错误原因:没有接收到前端传入的参数,修改为如下 参考 错误2:cannot r
错误1:启动docker镜像时报错:Error response from daemon: driver failed programming external connectivity on endpoint quirky_allen 解决方法:重启docker -&gt; systemctl r
错误1:private field ‘xxx‘ is never assigned 按Altʾnter快捷键,选择第2项 参考:https://blog.csdn.net/shi_hong_fei_hei/article/details/88814070 错误2:启动时报错,不能找到主启动类 #
报错如下,通过源不能下载,最后警告pip需升级版本 Requirement already satisfied: pip in c:\users\ychen\appdata\local\programs\python\python310\lib\site-packages (22.0.4) Coll
错误1:maven打包报错 错误还原:使用maven打包项目时报错如下 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-resources-plugin:3.2.0:resources (default-resources)
错误1:服务调用时报错 服务消费者模块assess通过openFeign调用服务提供者模块hires 如下为服务提供者模块hires的控制层接口 @RestController @RequestMapping(&quot;/hires&quot;) public class FeignControl
错误1:运行项目后报如下错误 解决方案 报错2:Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project sb 解决方案:在pom.
参考 错误原因 过滤器或拦截器在生效时,redisTemplate还没有注入 解决方案:在注入容器时就生效 @Component //项目运行时就注入Spring容器 public class RedisBean { @Resource private RedisTemplate&lt;String
使用vite构建项目报错 C:\Users\ychen\work&gt;npm init @vitejs/app @vitejs/create-app is deprecated, use npm init vite instead C:\Users\ychen\AppData\Local\npm-