2D CNN对3D灰度MRI数据进行分类，可能会出现数据标签问题

如何解决2D CNN对3D灰度MRI数据进行分类，可能会出现数据标签问题

我正在尝试对3D黑白MRI数据进行二进制分类。由于缺乏黑白数据固有的通道，我正在使用2D卷积。我添加了一个维以使维数对齐，并且本质上，此数据的深度充当批处理维。我使用的是数据的子样本，每个文件20个文件，每个189 x 233 x 197。

我有一个csv文件，其中包含大量信息，包括我尝试提取的每个文件的标签数据，如以下代码所示。

import numpy as np
import glob
import os
import tensorflow as tf
import pandas as pd
import glob

import SimpleITK as sitk

from sklearn.preprocessing import LabelEncoder
from sklearn.model_selection import train_test_split
from keras.preprocessing.image import ImageDataGenerator


from keras.utils import plot_model
from keras.utils import to_categorical
from keras.utils import np_utils

from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Dropout

from google.colab import drive
drive.mount('/content/gdrive')

datapath = ('/content/gdrive/My Drive/DirectoryTest/All Data/')
patients = os.listdir(datapath)
labels_df = pd.read_csv('/content/Data_Index.csv',index_col = 0 )

labelset = []

for i in patients:
  label = labels_df.loc[i,'Group']
  if label is 'AD':
    np.char.replace(label,['AD'],[0])
  if label is 'CN':
    np.char.replace(label,['CN'],[1])
  labelset.append(label)

label_encoder = LabelEncoder()
labelset = label_encoder.fit_transform(labelset)

labelset = np_utils.to_categorical(labelset,num_classes= 2)

FullDataSet = []

for i in patients:
  a = sitk.ReadImage(datapath + i)
  b = sitk.GetArrayFromImage(a)
  c = np.reshape(b,(189,233,197,1))
  FullDataSet.append(c)

training_data,testing_data,training_labels,testing_labels = train_test_split(FullDataSet,labelset,train_size=0.70,test_size=0.30)

dataset_train = tf.data.Dataset.from_tensor_slices((training_data,training_labels))
dataset_test = tf.data.Dataset.from_tensor_slices((testing_data,testing_labels))

CNN_model = tf.keras.Sequential(
  [
      #tf.keras.layers.Input(shape=(189,1),batch_size=2),#tf.keras.layers.Reshape((197,189,1)),tf.keras.layers.Conv2D(kernel_size=(7,7),data_format='channels_last',filters=64,activation='relu',padding='same',strides=( 3,3),input_shape=( 233,#tf.keras.layers.BatchNormalization(center=True,scale=False),tf.keras.layers.MaxPool2D(pool_size=(3,padding='same'),tf.keras.layers.Dropout(0.20),tf.keras.layers.Conv2D(kernel_size=( 7,filters=128,3)),filters=256,padding = 'same'),# last activation could be either sigmoid or softmax,need to look into this more. Sig for binary output,Soft for multi output 
      tf.keras.layers.Flatten(),tf.keras.layers.Dense(256,activation='relu'),tf.keras.layers.Dense(64,tf.keras.layers.Dense(2,activation='softmax')

  ])
# Compile the model
CNN_model.compile(optimizer=tf.keras.optimizers.Adam(lr=0.00001),loss='binary_crossentropy',metrics=['accuracy'])

# print model layers
CNN_model.summary()

CNN_history = CNN_model.fit(dataset_train,epochs=10,validation_data=dataset_test)

当我去拟合模型时，出现以下错误：

Epoch 1/10
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-35-a8b210ec2e72> in <module>()
      1 #running of the model
      2 #CNN_history = CNN_model.fit(dataset_train,epochs=100,validation_data =dataset_test,validation_steps=1)
----> 3 CNN_history = CNN_model.fit(dataset_train,validation_data=dataset_test)
      4 
      5 

10 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args,**kwargs)
    971           except Exception as e:  # pylint:disable=broad-except
    972             if hasattr(e,"ag_error_metadata"):
--> 973               raise e.ag_error_metadata.to_exception(e)
    974             else:
    975               raise

ValueError: in user code:

    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:806 train_function  *
        return step_function(self,iterator)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:796 step_function  **
        outputs = model.distribute_strategy.run(run_step,args=(data,))
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:1211 run
        return self._extended.call_for_each_replica(fn,args=args,kwargs=kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2585 call_for_each_replica
        return self._call_for_each_replica(fn,args,kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2945 _call_for_each_replica
        return fn(*args,**kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:789 run_step  **
        outputs = model.train_step(data)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:749 train_step
        y,y_pred,sample_weight,regularization_losses=self.losses)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/compile_utils.py:204 __call__
        loss_value = loss_obj(y_t,y_p,sample_weight=sw)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/losses.py:149 __call__
        losses = ag_call(y_true,y_pred)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/losses.py:253 call  **
        return ag_fn(y_true,**self._fn_kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/util/dispatch.py:201 wrapper
        return target(*args,**kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/losses.py:1605 binary_crossentropy
        K.binary_crossentropy(y_true,from_logits=from_logits),axis=-1)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/util/dispatch.py:201 wrapper
        return target(*args,**kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/backend.py:4829 binary_crossentropy
        bce = target * math_ops.log(output + epsilon())
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_ops.py:1141 binary_op_wrapper
        raise e
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_ops.py:1125 binary_op_wrapper
        return func(x,y,name=name)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_ops.py:1457 _mul_dispatch
        return multiply(x,name=name)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/util/dispatch.py:201 wrapper
        return target(*args,**kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_ops.py:509 multiply
        return gen_math_ops.mul(x,name)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_math_ops.py:6176 mul
        "Mul",x=x,y=y,name=name)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py:744 _apply_op_helper
        attrs=attr_protos,op_def=op_def)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py:593 _create_op_internal
        compute_device)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:3485 _create_op_internal
        op_def=op_def)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:1975 __init__
        control_input_ops,op_def)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:1815 _create_c_op
        raise ValueError(str(e))

    ValueError: Dimensions must be equal,but are 2 and 189 for '{{node binary_crossentropy/mul}} = Mul[T=DT_FLOAT](ExpandDims,binary_crossentropy/Log)' with input shapes: [2,1],[189,2].

我知道[189,2]中的2与最终的softmax层相关联，但我不知道该如何处理该信息，或从此处去哪里。 任何帮助将不胜感激，谢谢！

解决方法

以下是有关您代码的一些注释，希望对您有所帮助。

使用Conv3D和MaxPool3D

如果要处理3D图像，则几乎可以肯定使用Conv3D代替Conv2D，并使用MaxPool3D代替MaxPool2D。这是一个示例（使用随机数据），我刚刚对其进行了测试，它似乎运行良好：

import numpy as np
import tensorflow as tf
from tensorflow import keras

train_size = 20
val_size = 5

X_train = np.random.random([train_size,189,233,197]).astype(np.float32)
X_valid = np.random.random([val_size,197]).astype(np.float32)
y_train = np.random.randint(2,size=train_size).astype(np.float32)
y_valid = np.random.randint(2,size=val_size).astype(np.float32)

CNN_model = keras.Sequential([
      keras.layers.Reshape([189,197,1],input_shape=[189,197]),keras.layers.Conv3D(kernel_size=(7,7,7),filters=32,activation='relu',padding='same',strides=(3,3,3)),#keras.layers.BatchNormalization(),keras.layers.MaxPool3D(pool_size=(3,3),padding='same'),keras.layers.Dropout(0.20),keras.layers.Conv3D(kernel_size=(5,5,5),filters=64,keras.layers.MaxPool3D(pool_size=(2,2,2),keras.layers.Conv3D(kernel_size=(3,filters=128,strides=(1,1,1)),keras.layers.Flatten(),keras.layers.Dense(256,activation='relu'),keras.layers.Dense(64,keras.layers.Dense(1,activation='sigmoid')
  ])

# Compile the model
CNN_model.compile(optimizer=keras.optimizers.Adam(lr=0.00001),loss='binary_crossentropy',metrics=['accuracy'])

# print model layers
CNN_model.summary()

CNN_history = CNN_model.fit(X_train,y_train,epochs=10,validation_data=[X_valid,y_valid])

请勿重塑形状以替换尺寸

关于这两条注释行：

      #tf.keras.layers.Input(shape=(189,1),batch_size=2),#tf.keras.layers.Reshape((197,

将189x233x197x1图像重塑为197x233x189x1将无法正常工作。它将完全洗净周围的像素，使任务更加困难。这类似于将2x3图像重塑为3x2图像：

>>> img = np.array([[1,3],[4,6]])
>>> np.reshape(img,[3,2])
array([[1,2],4],[5,6]])

请注意，这与旋转图像不相同：像素完全混合在一起。

您要使用tf.keras.layers.Permute()代替这样：

CNN_model = tf.keras.Sequential([
      tf.keras.layers.Permute((3,4),input_shape=(189,...
])

由于这些注释掉的行是错误的，我怀疑以下行也可能是错误的：

c = np.reshape(b,(189,1))

我不知道b的形状，因此请绝对确保它与此np.reshape()操作兼容。例如，如果其形状为[189,197]，那很好。但是，例如，如果它是[197,189]，则需要在重新塑形之前对尺寸进行置换：

b_permuted = np.transpose(b,[2,0]) # permute dims
c = np.reshape(b_permuted,[189,1]) # then add the channels dim

np.transpose()函数与使用Permute()相似，不同之处在于尺寸是0索引而不是1索引。

它可能更加复杂。例如，如果将3D图像存储为并排的大2D图像，其中包含较小的2D切片，则b的形状可能类似于[189*197,233]。在这种情况下，您需要执行以下操作：

b_reshaped = np.reshape(b,1])
c = np.transpose(b_reshaped,[0,3])

我希望这些例子足够清楚。

使用tf.keras，而不是keras

Keras API有几种实现。一个是keras软件包，它是“多后端” Keras（使用pip install keras安装）。另一个是tf.keras，它随TensorFlow一起提供。您的程序似乎同时使用了两者。您应该绝对避免这种情况，否则会引起奇怪的问题。

from keras.utils import plot_model # this is multibackend Keras
...
CNN_model = tf.keras.Sequential(...) # this is tf.keras

我强烈建议您卸载多后端keras，以避免出现此类错误：pip uninstall keras。然后使用前缀tensorflow.来修复导入，例如：

from tensorflow.keras.models import Sequential
from tensorflow.keras.utils import to_categorical # note: not from np_utils
...

请勿使用to_categorical()进行二进制分类

对于二进制分类，标签应仅是包含0和1的一维数组，例如np.array([1.,0.,1.,1.])。代码可以非常简化：

labelset = []

for i in patients:
  label = labels_df.loc[i,'Group']
  if label == 'AD':  # use `==` instead of `is` to compare strings
    labelset.append(0.)
  elif label == 'CN':
    labelset.append(1.)
  else:
      raise "Oops,unknown label" # I recommend testing possible failure cases

labelset = np.array(labelset)

重要的是，对于二进制分类，您应该在输出层中使用单个神经元，还应使用"sigmoid"激活函数（不是用于多类分类的"softmax"）：

CNN_model = tf.keras.Sequential([
      ...
      tf.keras.layers.Dense(1,activation='sigmoid')
])

次要评论

• 调用train_size时不需要同时指定test_size和train_test_split()。

祝你好运！