如何解决“您的输入数据用完”总是在第 # 次时代
我一直在尝试使用 Keras 为 model.fit() 制作我的第一个数据生成器。我试图制作的数据集有两个输入，一个图像和一个浮点值。我所有的图像名称和值都存储在一个 csv 文件中。我相信我错误地制作了我的生成器，因为无论我的批量大小是多少，我总是在与我的时代相同的步骤中收到错误“您的输入用完了数据”。因此，如果我的 epochs 设置为 100，我的模型将运行到第 100 步。我的数据集大约有 100000 个图像/值。如果有人能帮我找到一个很好的解决方案。
我目前正在使用：
蟒蛇 3.8
TF-GPU 2.4.0rc1
keras 2.4.3
熊猫 1.1.4
代码：
IMG_SIZE = 400
Version = 1
batch_size = 64
val = .05
val_aug = ImageDataGenerator(rescale=1/255)
aug = ImageDataGenerator(
rescale=1/255,rotation_range=30,width_shift_range=0.1,height_shift_range=0.1,shear_range=0.2,zoom_range=0.2,channel_shift_range=25,horizontal_flip=True,fill_mode='constant')
df = pd.read_csv('F:/DATA/Vote/Vote_Age.csv')
df = df.sample(frac = 1)
cut = int(len(df) * val)
train_df = df[cut:]
val_df = df[0:cut]
print(f'Training dataset: {len(train_df)}')
print(f'Val dataset: {len(val_df)}')
train_steps = int(len(train_df) / batch_size)
val_steps = int(len(val_df) / batch_size)
def data(df,generator,batch_size,IMG_SIZE):
z = 0
while True:
df = df.sample(frac = 1)
for i in range(int(len(df) / batch_size)):
images,ages,votes = [],[],[]
for x in range(batch_size):
csv_row = df.iloc[(z),:]
z += 1
image_path = f'F:/DATA/Vote/Images/{int(csv_row[0])}.jpg'
image = cv2.resize(cv2.imread(image_path),(int(IMG_SIZE),int(IMG_SIZE)))
image = cv2.cvtColor(image,cv2.COLOR_BGR2RGB)
image = image.reshape(-1,IMG_SIZE,3)
generator.fit(image)
image = generator.flow(image,batch_size=1)
image = image.next()
image = image.reshape(IMG_SIZE,3)
images.append(image)
ages.append(csv_row[1])
votes.append(int(csv_row[2]))
images = np.array(images)
ages = np.array(ages)
votes = np.array(votes)
return [[images,ages],[votes]]
#########
#Model was very big and unnecessary to include
#########
train_dataset = train_data(train_df,IMG_SIZE)
val_dataset = val_data(val_df,IMG_SIZE)
model.fit(
x = train_dataset[0],y = train_dataset[1],validation_data=(val_dataset[0],val_dataset[1]),steps_per_epoch=train_steps,validation_steps=val_steps,callbacks=earlyStop,epochs=100,batch_size=batch_size,workers=multiprocessing.cpu_count(),verbose=1)
model.save(f'F:/DATA/Vote/Models/YiffModel{Version}')
输出：
...
83/1484 [>.............................] - ETA: 4:46 - loss: 7578.7731
84/1484 [>.............................] - ETA: 4:46 - loss: 7575.5172
85/1484 [>.............................] - ETA: 4:46 - loss: 7572.2818
86/1484 [>.............................] - ETA: 4:46 - loss: 7569.0662
87/1484 [>.............................] - ETA: 4:46 - loss: 7565.8702
88/1484 [>.............................] - ETA: 4:45 - loss: 7562.6932
89/1484 [>.............................] - ETA: 4:45 - loss: 7559.5349
90/1484 [>.............................] - ETA: 4:45 - loss: 7556.3948
91/1484 [>.............................] - ETA: 4:45 - loss: 7553.2726
92/1484 [>.............................] - ETA: 4:44 - loss: 7550.1679
93/1484 [>.............................] - ETA: 4:44 - loss: 7547.0802
94/1484 [>.............................] - ETA: 4:44 - loss: 7544.0094
95/1484 [>.............................] - ETA: 4:44 - loss: 7540.9549
96/1484 [>.............................] - ETA: 4:43 - loss: 7537.9164
97/1484 [>.............................] - ETA: 4:43 - loss: 7534.8937
98/1484 [>.............................] - ETA: 4:43 - loss: 7531.8863
99/1484 [=>............................] - ETA: 4:43 - loss: 7528.8939
100/1484 [=>............................] - ETA: 4:43 - loss: 7525.9163
WARNING:tensorflow:Your input ran out of data; interrupting training. Make sure that your dataset or generator can generate at least `steps_per_epoch * epochs` batches (in this case,148400 batches). You may need to use the repeat() function when building your dataset.
WARNING:tensorflow:Your input ran out of data; interrupting training. Make sure that your dataset or generator can generate at least `steps_per_epoch * epochs` batches (in this case,78 batches). You may need to use the repeat() function when building your dataset.
1484/1484 [==============================] - 30s 15ms/step - loss: 7250.9988 - val_loss: 13595.9355
C:\Users\Tristan\anaconda3\envs\tf2\lib\site-packages\tensorflow\python\keras\engine\training.py:2325: UserWarning: `Model.state_updates` will be removed in a future version. This property should not be used in TensorFlow 2.0,as `updates` are applied automatically.
warnings.warn('`Model.state_updates` will be removed in a future version. '
2021-01-05 15:37:04.425018: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences,but this may change in the future,so consider avoiding using them.
C:\Users\Tristan\anaconda3\envs\tf2\lib\site-packages\tensorflow\python\keras\engine\base_layer.py:1402: UserWarning: `layer.updates` will be removed in a future version. This property should not be used in TensorFlow 2.0,as `updates` are applied automatically.
warnings.warn('`layer.updates` will be removed in a future version. '
WARNING:tensorflow:FOR KERAS USERS: The object that you are saving contains one or more Keras models or layers. If you are loading the SavedModel with `tf.keras.models.load_model`,continue reading (otherwise,you may ignore the following instructions). Please change your code to save with `tf.keras.models.save_model` or `model.save`,and confirm that the file "keras.metadata" exists in the export directory. In the future,Keras will only load the SavedModels that have this file. In other words,`tf.saved_model.save` will no longer write SavedModels that can be recovered as Keras models (this will apply in TF 2.5).
FOR DEVS: If you are overwriting _tracking_metadata in your class,this property has been used to save metadata in the SavedModel. The metadta field will be deprecated soon,so please move the metadata to a different file.
libpng warning: iCCP: known incorrect sRGB profile
版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 dio@foxmail.com 举报，一经查实，本站将立刻删除。