训练数据在model.fit_generator中面临错误

如何解决训练数据在model.fit_generator中面临错误

我正在使用Tensorflow训练10类不同图像的多类CNN模型。总训练图像为6000，测试图像为1600。当我尝试训练模型时，我正面临错误。以下是我的代码：

import os
import random
import tensorflow as tf
from tensorflow.keras.optimizers import RMSprop
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from shutil import copyfile

print(len(os.listdir('C:/Users/shweta/Desktop/characters/test'))) #test set
print(len(os.listdir('C:/Users/shweta/Desktop/characters/train'))) #train set

TRAINING_DIR = "C:/Users/shweta/Desktop/characters/train/"

train_datagen = ImageDataGenerator(rescale=1./255,shear_range=0.2,zoom_range=0.2,horizontal_flip=True)

train_generator = train_datagen.flow_from_directory(TRAINING_DIR,batch_size=60,class_mode='categorical',target_size=(64,64))

VALIDATION_DIR = "C:/Users/shweta/Desktop/characters/test/"

validation_datagen = ImageDataGenerator(rescale=1./255)
validation_generator = validation_datagen.flow_from_directory(VALIDATION_DIR,batch_size=64,64))



model = tf.keras.models.Sequential([
    tf.keras.layers.Conv2D(32,(3,3),activation='relu',input_shape=(64,64,3)),tf.keras.layers.MaxPooling2D(2,2),tf.keras.layers.Conv2D(32,activation='relu'),tf.keras.layers.Flatten(),tf.keras.layers.Dense(128,tf.keras.layers.Dense(10,activation='softmax')
])

model.compile(optimizer='adam',loss='categorical_crossentropy',metrics=['acc'])

model.summary()


history = model.fit_generator(train_generator,validation_data=validation_generator,steps_per_epoch=100,epochs=25,validation_steps=25)
                          

我在model.fit_generator中遇到以下错误：

        WARNING:tensorflow:From C:\Users\shweta\.spyder-py3\temp.py:55: Model.fit_generator (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.
Instructions for updating:
Please use Model.fit,which supports generators.
Epoch 1/25
  4/100 [>.............................] - ETA: 2:03 - loss: 2.3212 - acc: 0.1208Traceback (most recent call last):

  File "C:\Users\shweta\.spyder-py3\temp.py",line 55,in <module>
    validation_steps=25)

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\util\deprecation.py",line 324,in new_func
    return func(*args,**kwargs)

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\keras\engine\training.py",line 1479,in fit_generator
    initial_epoch=initial_epoch)

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\keras\engine\training.py",line 66,in _method_wrapper
    return method(self,*args,line 848,in fit
    tmp_logs = train_function(iterator)

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\eager\def_function.py",line 580,in __call__
    result = self._call(*args,**kwds)

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\eager\def_function.py",line 611,in _call
    return self._stateless_fn(*args,**kwds)  # pylint: disable=not-callable

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\eager\function.py",line 2420,in __call__
    return graph_function._filtered_call(args,kwargs)  # pylint: disable=protected-access

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\eager\function.py",line 1665,in _filtered_call
    self.captured_inputs)

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\eager\function.py",line 1746,in _call_flat
    ctx,args,cancellation_manager=cancellation_manager))

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\eager\function.py",line 598,in call
    ctx=ctx)

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\eager\execute.py",line 60,in quick_execute
    inputs,attrs,num_outputs)

UnknownError:  UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x00000180610E2E28>
Traceback (most recent call last):

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\ops\script_ops.py",line 243,in __call__
    ret = func(*args)

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\autograph\impl\api.py",line 309,in wrapper
    return func(*args,**kwargs)

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py",line 785,in generator_py_func
    values = next(generator_state.get_iterator(iterator_id))

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\keras\engine\data_adapter.py",line 801,in wrapped_generator
    for data in generator_fn():

  File "C:\Users\shweta\anaconda3\lib\site-packages\tensorflow\python\keras\engine\data_adapter.py",line 932,in generator_fn
    yield x[i]

  File "C:\Users\shweta\anaconda3\lib\site-packages\keras_preprocessing\image\iterator.py",line 65,in __getitem__
    return self._get_batches_of_transformed_samples(index_array)

  File "C:\Users\shweta\anaconda3\lib\site-packages\keras_preprocessing\image\iterator.py",line 230,in _get_batches_of_transformed_samples
    interpolation=self.interpolation)

  File "C:\Users\shweta\anaconda3\lib\site-packages\keras_preprocessing\image\utils.py",line 114,in load_img
    img = pil_image.open(io.BytesIO(f.read()))

  File "C:\Users\shweta\anaconda3\lib\site-packages\PIL\Image.py",line 2862,in open
    "cannot identify image file %r" % (filename if filename else fp)

PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x00000180610E2E28>


     [[{{node PyFunc}}]]
     [[IteratorGetNext]] [Op:__inference_train_function_877]

Function call stack:
train_function

请帮助我解决此问题。预先感谢。

解决方法

请将您的softmax输出更改为10，而不是将11作为类别数。 如果您希望添加否定类，然后在训练和测试数据集中再添加一个文件夹。

tf.keras.layers.Dense(10,activation='softmax')

,

一些观察。您有6000张训练图像，并且将批次大小指定为64，每个时期的步数=200。200 X 64 = 12,800，因此您将经历每个时期两次的训练集。将批次大小设置为60，将每个时期的步数设置为100，您将每个时期进行一次培训。对于验证数据，您有类似的问题。您只希望每个时期通过一次验证集。如果有1600张验证图像且批处理大小= 64，则1600/64 = 25，因此设置validation_steps = 25。现在，我不确定这是否可以解决您的问题。试试看，看看是否可以解决。如果不是，我怀疑您的输入数据集中可能存在无效内容。我开发了一个脚本来检查输入目录，以确保它们具有允许的扩展名，并且实际上是好的图像文件。脚本位于here.

中的答案中