如何解决Python创建了字典,将大数据集的输出截断如何保存完整的词典?
我继承了此代码,该代码用于格式化和读取模型输出。但是,当我运行它并将输出保存到文本文件时,它仅给出每个位置的前5行和后5行(字典键)。我不确定代码或设置是否有问题。我正在Jupyter笔记本中运行代码。附带的是代码和输出。
def outflw1(path=''):
#get the starting year (old outflw1 format)
if path == '':
fin1 = open('input')
else:
fin1 = open(os.path.join(path,'input'))
s = fin1.readline()
while 'starting date of simulation' not in s:
s = fin1.readline()
s = s.replace(' ','').split(',')
year = int(s[2][:4])
fin1.close()
#read off 5 lines - don't need them. These are the headers and blank lines in the file
f = open(os.path.join(path,'outflw1'))
for i in range(5):
next(f)
#create dictionary for StringIO
sio = {} #denotes as a dictionary object
init = 0
for ln in f:
if not ln.strip():
init = 1
if s[0] == '12' and s[1] == '31' and s[2] == '23.0':
year += 1
continue
else:
s = ln.split()
#if first time through need to initialize StringIO objects
if init == 0:
sio[s[3]] = StringIO()
sio[s[3]].write(str(year) + '-' + s[0] + '-' + s[1] + ' ' + s[2].split('.')[0] + ':00:00' + ',')
else:
sio[s[3]].write(str(year) + '-' + s[0] + '-' + s[1] + ' ' + s[2].split('.')[0] + ':00:00' + ',')
sio[s[3]].write(','.join(s[4:8]) + ',')
if len(s) == 11:
sio[s[3]].write(','.join(s[9:]) + '\n')
else:
sio[s[3]].write(','.join([s[8][2:],s[9]]) + '\n')
continue
outflw1 = {}
for k in list(sio.keys()):
sio[k].seek(0)
outflw1[k] = pd.read_csv(sio[k],parse_dates=True,index_col=0,names = ['tide','elevation','depth','velocity','direction','salinity'])
outflw1[k].index.name = 'Date'
return(outflw1)
这是其中一个键的输出。当我使用print命令并将其保存到文本文件时,外观看起来相同。
{'10018': tide elevation depth velocity direction salinity
Date
1987-1-1 :00:00 0.40 0.06 3.06 0.01 121.87 8.48
1987-1-1 1:00:00 -0.47 0.09 3.09 0.02 122.65 8.50
1987-1-1 2:00:00 -1.54 0.13 3.13 0.03 134.67 8.51
1987-1-1 3:00:00 -1.83 0.18 3.18 0.02 133.44 8.53
1987-1-1 4:00:00 -1.75 0.21 3.21 0.01 334.74 8.55
... ... ... ... ... ... ...
2014-12-31 20:00:00 0.87 1.33 4.33 0.04 128.71 24.77
2014-12-31 21:00:00 0.86 1.40 4.40 0.05 169.77 24.84
2014-12-31 22:00:00 0.92 1.45 4.45 0.05 168.49 24.90
2014-12-31 23:00:00 1.02 1.37 4.37 0.03 311.43 24.91
2015-1-1 :00:00 0.97 1.34 4.34 0.02 161.63 24.93
[245449 rows x 6 columns],
解决方法
尝试使用pickle模块进行序列化:
pickle.dump