在存在BatchNormalization层的情况下CNN意外预测

分类：编程问答

如何解决在存在BatchNormalization层的情况下CNN意外预测

我目前正在构建一个CNN，以便从CT图像（灰度为1024 x 1024像素）开始对2个不同的类别进行分类。我知道我的数据集很小：训练/验证集有100个样本（50个“ 0级”和50个“ 1类”），测试集有27个样本（6和21个）。经过多次仿真，这种网络配置似乎是最佳解决方案：

model = Sequential()
model.add(Conv2D(32,(3,3),activation='relu',input_shape=(1024,1024,1)))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.7))

model.add(Conv2D(32,activation='relu'))) 
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2,2)))

model.add(Flatten())
model.add(Dropout(0.7))
model.add(Dense(16,activation='relu'))
model.add(BatchNormalization())
model.add(Dense(8,activation='relu'))
model.add(BatchNormalization())
model.add(Dense(4,activation='relu'))
model.add(BatchNormalization())
model.add(Dense(2,activation='softmax'))
model.summary()

下面，您可以看到其他设置：

# compile model
model.compile(loss='categorical_crossentropy',optimizer='SGD',metrics=['accuracy'])

# callbacks
earlyStopping = EarlyStopping(min_delta=0.01,monitor = 'accuracy',patience=40,mode='max',restore_best_weights=True)
reduce_lr = ReduceLROnPlateau()

# Fitting
seed(100)
set_random_seed(100)
model.fit(X_train,y_train,batch_size=10,validation_split=0.1,epochs=50,shuffle = True,callbacks=[reduce_lr,earlyStopping])

如下所示，性能似乎不错；实际上，损失和验证损失会随着时代的发展而不断下降，而准确性却越来越高。

Epoch 1/50
90/90 [==============================] - 43s 478ms/step - loss: 0.8159 - accuracy: 0.4889 - val_loss: 0.6816 - val_accuracy: 0.6000
Epoch 2/50
90/90 [==============================] - 42s 470ms/step - loss: 0.7045 - accuracy: 0.6000 - val_loss: 0.5591 - val_accuracy: 1.0000
Epoch 3/50
90/90 [==============================] - 42s 467ms/step - loss: 0.6565 - accuracy: 0.6667 - val_loss: 0.4171 - val_accuracy: 1.0000
Epoch 4/50
90/90 [==============================] - 43s 474ms/step - loss: 0.6253 - accuracy: 0.6111 - val_loss: 0.0789 - val_accuracy: 1.0000
Epoch 5/50
90/90 [==============================] - 43s 474ms/step - loss: 0.6086 - accuracy: 0.6889 - val_loss: 0.0068 - val_accuracy: 1.0000
Epoch 6/50
90/90 [==============================] - 43s 474ms/step - loss: 0.6311 - accuracy: 0.6444 - val_loss: 0.0067 - val_accuracy: 1.0000
Epoch 7/50
90/90 [==============================] - 42s 468ms/step - loss: 0.5163 - accuracy: 0.7444 - val_loss: 5.7406e-04 - val_accuracy: 1.0000
Epoch 8/50
90/90 [==============================] - 43s 473ms/step - loss: 0.5752 - accuracy: 0.7222 - val_loss: 7.9747e-05 - val_accuracy: 1.0000
Epoch 9/50
90/90 [==============================] - 43s 475ms/step - loss: 0.5440 - accuracy: 0.7444 - val_loss: 1.9908e-06 - val_accuracy: 1.0000
Epoch 10/50
90/90 [==============================] - 42s 468ms/step - loss: 0.5481 - accuracy: 0.7444 - val_loss: 1.5497e-07 - val_accuracy: 1.0000
Epoch 11/50
90/90 [==============================] - 42s 471ms/step - loss: 0.5032 - accuracy: 0.7778 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 12/50
90/90 [==============================] - 43s 475ms/step - loss: 0.5036 - accuracy: 0.7889 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 13/50
90/90 [==============================] - 42s 471ms/step - loss: 0.4714 - accuracy: 0.8222 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 14/50
90/90 [==============================] - 42s 470ms/step - loss: 0.4353 - accuracy: 0.8667 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 15/50
90/90 [==============================] - 42s 467ms/step - loss: 0.4243 - accuracy: 0.8556 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 16/50
90/90 [==============================] - 42s 471ms/step - loss: 0.4046 - accuracy: 0.8444 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 17/50
90/90 [==============================] - 43s 479ms/step - loss: 0.4068 - accuracy: 0.8556 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 18/50
90/90 [==============================] - 43s 476ms/step - loss: 0.4336 - accuracy: 0.8111 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 19/50
90/90 [==============================] - 42s 470ms/step - loss: 0.3909 - accuracy: 0.8778 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 20/50
90/90 [==============================] - 42s 470ms/step - loss: 0.3343 - accuracy: 0.9111 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 21/50
90/90 [==============================] - 43s 475ms/step - loss: 0.4284 - accuracy: 0.8444 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 22/50
90/90 [==============================] - 42s 469ms/step - loss: 0.4517 - accuracy: 0.8111 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 23/50
90/90 [==============================] - 42s 465ms/step - loss: 0.3613 - accuracy: 0.9000 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 24/50
90/90 [==============================] - 43s 474ms/step - loss: 0.3489 - accuracy: 0.9111 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 25/50
90/90 [==============================] - 43s 474ms/step - loss: 0.3843 - accuracy: 0.8778 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 26/50
90/90 [==============================] - 42s 471ms/step - loss: 0.3456 - accuracy: 0.8889 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 27/50
90/90 [==============================] - 43s 474ms/step - loss: 0.3643 - accuracy: 0.8778 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 28/50
90/90 [==============================] - 43s 474ms/step - loss: 0.3856 - accuracy: 0.8889 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 29/50
90/90 [==============================] - 42s 471ms/step - loss: 0.3988 - accuracy: 0.8667 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 30/50
90/90 [==============================] - 43s 475ms/step - loss: 0.3948 - accuracy: 0.8778 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 31/50
90/90 [==============================] - 42s 470ms/step - loss: 0.4920 - accuracy: 0.7333 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 32/50
90/90 [==============================] - 42s 472ms/step - loss: 0.3573 - accuracy: 0.8889 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 33/50
90/90 [==============================] - 42s 465ms/step - loss: 0.4097 - accuracy: 0.8333 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 34/50
90/90 [==============================] - 42s 469ms/step - loss: 0.3378 - accuracy: 0.9111 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 35/50
90/90 [==============================] - 42s 472ms/step - loss: 0.3782 - accuracy: 0.8889 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 36/50
90/90 [==============================] - 43s 475ms/step - loss: 0.3891 - accuracy: 0.8333 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 37/50
90/90 [==============================] - 42s 470ms/step - loss: 0.3851 - accuracy: 0.8889 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 38/50
90/90 [==============================] - 42s 472ms/step - loss: 0.3569 - accuracy: 0.9000 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 39/50
90/90 [==============================] - 43s 474ms/step - loss: 0.3795 - accuracy: 0.8889 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 40/50
90/90 [==============================] - 42s 466ms/step - loss: 0.4399 - accuracy: 0.7667 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 41/50
90/90 [==============================] - 42s 466ms/step - loss: 0.4298 - accuracy: 0.8556 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 42/50
90/90 [==============================] - 43s 473ms/step - loss: 0.3553 - accuracy: 0.8778 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 43/50
90/90 [==============================] - 43s 474ms/step - loss: 0.3967 - accuracy: 0.8889 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 44/50
90/90 [==============================] - 43s 475ms/step - loss: 0.3783 - accuracy: 0.8556 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 45/50
90/90 [==============================] - 43s 474ms/step - loss: 0.3379 - accuracy: 0.9444 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 46/50
90/90 [==============================] - 43s 473ms/step - loss: 0.3679 - accuracy: 0.9000 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 47/50
90/90 [==============================] - 43s 475ms/step - loss: 0.3074 - accuracy: 0.9333 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 48/50
90/90 [==============================] - 43s 475ms/step - loss: 0.3210 - accuracy: 0.9333 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 49/50
90/90 [==============================] - 43s 476ms/step - loss: 0.4226 - accuracy: 0.8556 - val_loss: 0.0000e+00 - val_accuracy: 1.0000
Epoch 50/50
90/90 [==============================] - 43s 476ms/step - loss: 0.3811 - accuracy: 0.8889 - val_loss: 0.0000e+00 - val_accuracy: 1.0000

但是，当我尝试预测两个数据集（训练/验证集和测试集）的类别时，却得到了意料不到的令人惊讶的结果：CNN仅预测了一个类别。

model.predict(X_train)

array([[0.,1.],[0.,1.]],dtype=float32)


model.predict(X_test)

array([[0.,dtype=float32)

Here，我发现BatchNormalization()层可能会产生此问题。这篇文章中的解决方案是使用以下几行来预测样本：

#Prediction on training set
learning_phase = 1
sample_weights_TR = np.ones(100)
ins_TR = [X_train,sample_weights_TR,learning_phase]
model.test_function(ins_TR)

[0.39475524,0.8727273] # Average Accuracy = 0.873


#Prediction on test set
sample_weights_TS = np.ones(27)
ins_TS = [X_test,y_test,sample_weights_TS,learning_phase]
model.test_function(ins_TS)

[0.7853608,0.79562044]  # Average Accuracy = 0.796

但是，如果我一次又一次启动model.test_function(ins_TR)或model.test_function(ins_TS)，结果总是不同的；准确性不断下降！

model.test_function(ins_TS)
[0.7853608,0.79562044]

model.test_function(ins_TS)
[0.88033366,0.75]

 model.test_function(ins_TS)
[0.9703789,0.7068063]

model.test_function(ins_TS)
[0.86600196,0.69266057]

model.test_function(ins_TS)
[0.79449946,0.67755103]

model.test_function(ins_TS)
[0.77942985,0.6691176]

model.test_function(ins_TS)
[0.83834535,0.6588629]

因此，我的问题是：

  1. 您对改善CNN有何建议？
  2. 在存在BatchNormalization（）层的情况下，如何获得正确的预测？
  3. 使用此处提出的解决方案，如何获得每个样本的预测（我将评估CNN的敏感性和特异性）？

预先感谢， Mattia

解决方法

主要问题是，如@Masoud Erfani所述，您想训练一个二元分类器（只有两个可能的输出的分类器：0或1），但您正在使用S型激活函数和分类交叉熵损失函数，用于多分类器。

将最后一层的激活函数更改为sigmoid，将丢失函数更改为binary_crossentropy，这样可以解决您的问题。

保持联系！

版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 dio@foxmail.com 举报，一经查实，本站将立刻删除。

相关推荐

导入项目后报错问题
依赖报错 idea导入项目后依赖报错，解决方案：https://blog.csdn.net/weixin_42420249/article/details/81191861 依赖版本报错：更换其他版本 无法下载依赖可参考：https://blog.csdn.net/weixin_42628809/a
idea不能识别yaml文件
使用mybatis plus常见错误
错误1：代码生成器依赖和mybatis依赖冲突 启动项目时报错如下 2021-12-03 13:33:33.927 ERROR 7228 [ main] o.s.b.d.LoggingFailureAnalysisReporter : *************************** APPL
gradle常见问题与错误
错误1：gradle项目控制台输出为乱码 # 解决方案：https://blog.csdn.net/weixin_43501566/article/details/112482302 # 在gradle-wrapper.properties 添加以下内容 org.gradle.jvmargs=-Df
Mybatis Plus传入参数0不起作用
错误还原：在查询的过程中，传入的workType为0时，该条件不起作用 <select id="xxx"> SELECT di.id, di.name, di.work_type, di.updated... <where> <if test=&qu
linux中make编译源码包失败
报错如下，gcc版本太低 ^ server.c:5346:31: 错误：‘struct redisServer’没有名为‘server_cpulist’的成员 redisSetCpuAffinity(server.server_cpulist); ^ server.c: 在函数‘hasActiveC
微服务启动错误：Command line is too long. Shorten command line for
解决方案1 1、改项目中.idea/workspace.xml配置文件，增加dynamic.classpath参数 2、搜索PropertiesComponent，添加如下 <property name="dynamic.classpath" value="tru
vue常见错误
删除根组件app.vue中的默认代码后报错：Module Error (from ./node_modules/eslint-loader/index.js): 解决方案：关闭ESlint代码检测，在项目根目录创建vue.config.js，在文件中添加 module.exports = { lin
spark常见错误
查看spark默认的python版本 [root@master day27]# pyspark /home/software/spark-2.3.4-bin-hadoop2.7/conf/spark-env.sh: line 2: /usr/local/hadoop/bin/hadoop: No s
matplotlib报错：AttributeError: module 'backend_interagg' has no attribute 'FigureCanvas'. Did you mean: 'FigureCanvasAgg'?
使用本地python环境可以成功执行 import pandas as pd import matplotlib.pyplot as plt # 设置字体 plt.rcParams['font.sans-serif'] = ['SimHei'] # 能正确显示负号 p
gitlab登录失败，报错：This challenge page was accidentally cached by an intermediary and is no longer available.
设置时间 控制面板
后端开发常见错误
错误1：Request method ‘DELETE‘ not supported 错误还原：controller层有一个接口，访问该接口时报错：Request method ‘DELETE‘ not supported 错误原因：没有接收到前端传入的参数，修改为如下 参考 错误2：cannot r
docker常见错误
错误1：启动docker镜像时报错：Error response from daemon: driver failed programming external connectivity on endpoint quirky_allen 解决方法：重启docker -> systemctl r
idea常见错误
错误1：private field ‘xxx‘ is never assigned 按Altʾnter快捷键，选择第2项 参考：https://blog.csdn.net/shi_hong_fei_hei/article/details/88814070 错误2：启动时报错，不能找到主启动类 #
pip安装依赖失败
报错如下，通过源不能下载，最后警告pip需升级版本 Requirement already satisfied: pip in c:\users\ychen\appdata\local\programs\python\python310\lib\site-packages (22.0.4) Coll
maven常见错误
错误1：maven打包报错 错误还原：使用maven打包项目时报错如下 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-resources-plugin:3.2.0:resources (default-resources)
cloud项目中常见错误
错误1：服务调用时报错 服务消费者模块assess通过openFeign调用服务提供者模块hires 如下为服务提供者模块hires的控制层接口 @RestController @RequestMapping("/hires") public class FeignControl
springboot常见错误
错误1：运行项目后报如下错误 解决方案 报错2：Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project sb 解决方案：在pom.
springmvc拦截器中使用redisTemplate报空指针异常
参考 错误原因 过滤器或拦截器在生效时，redisTemplate还没有注入 解决方案：在注入容器时就生效 @Component //项目运行时就注入Spring容器 public class RedisBean { @Resource private RedisTemplate<String
vite报错：npm ERR! command C:WINDOWSsystem32cmd.exe /d /s /c C:UsersychenAppDataLocalTempnpx-dde1c848.cmd
使用vite构建项目报错 C:\Users\ychen\work>npm init @vitejs/app @vitejs/create-app is deprecated, use npm init vite instead C:\Users\ychen\AppData\Local\npm-
pythonJavaScriptjavaHTMLPHPreactjsC#AndroidCSSNode.jssqlrpython-3.xMysqLjQueryc++pandasFlutterangularIOSdjangolinuxswifttypescript路由器JSON路由器设置无线路由器h3c华三华三路由器设置华三路由器电脑软件教程arraysdocker软件图文教程Cvue.jslaravelspring-boot