如何解决从抓取的数据创建嵌套字典Scrapy Python 使用scrapy从this website刮下一个货币值表并创建此列表:到目前为止,我的代码:我想要的输出将是这样的:
我不确定我是否需要使用从网站抓取的数据直接生成字典,还是最好先创建一个列表,但这是我所做的(如果可能,我不想使用熊猫):
使用scrapy从this website刮下一个货币值表并创建此列表:
# the value order is "Currency_name",BRL,USD,EUR,GBP,JPY,CHF,CAD,AUD
[" BRL","1","0,1800",1517",1361","19,0340",1633",2372",2501"," USD","5,5571",8428",7564","105,78",9075","1,3181",3897"," EUR","6,5938",1866",8977","125,49",0769",5640",6488"," GBP","7,3460",3219",1140","139,82",1997",7424",8370"," JPY",05254",0095",0080",00715",0086",01246",01314"," CHF",1236",1019",9286",8335","116,53",4525",5312"," CAD","4,2160",7587",6394",5739","80,25",6886",0543"," AUD","3,9989",7196",6065",5444","76,12",6532",9485","1"]
到目前为止,我的代码:
import scrapy
# inside scrapy.Spider class
def parse(self,response):
currency_table = response.css("#exchange_rates_1 > tbody > tr > td::text").extract()
item_list = []
for data in currency_table:
items = data.strip("\t").strip("\n").strip("\t") # removing nonsense
if items.startswith("\xa0"): # country flag code
item_list.append(items.strip("\xa0"))
elif items != "":
item_list.append(items)
我想要的输出将是这样的:
currency_dict = {
"BRL": {
"Real": 1,"Dollar": value,"Euro": value,"Pound": value,"Yen": value,"Swiss Frank": value,"Canadian Dollar": value,"Australian Dollar": value,},"USD": {
"Real": value,"Dollar": 1,...
},# continues for all currencies
}
解决方法
类似的事情应该起作用:
import pprint
# from https://stackoverflow.com/a/312464/
def chunks(lst,n):
"""Yield successive n-sized chunks from lst."""
for i in range(0,len(lst),n):
yield lst[i:i + n]
cur_lst = ["Real","Dollar","Euro","Pound","Yen","Swiss Frank","Canadian Dollar","Australian Dollar"]
data = [" BRL","1","0,1800",1517",1361","19,0340",1633",2372",2501"," USD","5,5571",8428",7564","105,78",9075","1,3181",3897"," EUR","6,5938",1866",8977","125,49",0769",5640",6488"," GBP","7,3460",3219",1140","139,82",1997",7424",8370"," JPY",05254",0095",0080",00715",0086",01246",01314"," CHF",1236",1019",9286",8335","116,53",4525",5312"," CAD","4,2160",7587",6394",5739","80,25",6886",0543"," AUD","3,9989",7196",6065",5444","76,12",6532",9485","1"]
currency_dict = {}
for cur_name,*cur_data in chunks(data,9):
data_obj = {}
for name,value in zip(cur_lst,cur_data):
# Python doesn't like a comma used as a decimal point
data_obj[name] = float(value.replace(",","."))
currency_dict[cur_name.strip()] = data_obj
pprint.pprint(currency_dict)
输出:
{'AUD': {'Australian Dollar': 1.0,'Canadian Dollar': 0.9485,'Dollar': 0.7196,'Euro': 0.6065,'Pound': 0.5444,'Real': 3.9989,'Swiss Frank': 0.6532,'Yen': 76.12},'BRL': {'Australian Dollar': 0.2501,'Canadian Dollar': 0.2372,'Dollar': 0.18,'Euro': 0.1517,'Pound': 0.1361,'Real': 1.0,'Swiss Frank': 0.1633,'Yen': 19.034},'CAD': {'Australian Dollar': 1.0543,'Canadian Dollar': 1.0,'Dollar': 0.7587,'Euro': 0.6394,'Pound': 0.5739,'Real': 4.216,'Swiss Frank': 0.6886,'Yen': 80.25},'CHF': {'Australian Dollar': 1.5312,'Canadian Dollar': 1.4525,'Dollar': 1.1019,'Euro': 0.9286,'Pound': 0.8335,'Real': 6.1236,'Swiss Frank': 1.0,'Yen': 116.53},'EUR': {'Australian Dollar': 1.6488,'Canadian Dollar': 1.564,'Dollar': 1.1866,'Euro': 1.0,'Pound': 0.8977,'Real': 6.5938,'Swiss Frank': 1.0769,'Yen': 125.49},'GBP': {'Australian Dollar': 1.837,'Canadian Dollar': 1.7424,'Dollar': 1.3219,'Euro': 1.114,'Pound': 1.0,'Real': 7.346,'Swiss Frank': 1.1997,'Yen': 139.82},'JPY': {'Australian Dollar': 0.01314,'Canadian Dollar': 0.01246,'Dollar': 0.0095,'Euro': 0.008,'Pound': 0.00715,'Real': 0.05254,'Swiss Frank': 0.0086,'Yen': 1.0},'USD': {'Australian Dollar': 1.3897,'Canadian Dollar': 1.3181,'Dollar': 1.0,'Euro': 0.8428,'Pound': 0.7564,'Real': 5.5571,'Swiss Frank': 0.9075,'Yen': 105.78}}
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。