如何解决我的预测值仅在梯度下降时才会降低
我目前正在编写梯度下降的实现,但遇到了一个问题,即我的预测值(y_hat)只会降低。即使在训练标签为1而不是0的情况下应该增加,它也永远不会增加。我的火车功能代码如下:
def sigma(self,a):
ans = 1/(1+np.exp(-a))
return ans
def get_loss(self,y_i,y_hat):
loss = -(y_i * np.log(y_hat) + (1 - y_i) * np.log(1 - y_hat))
return loss
def train(self,X,y,step_size,num_iterations):
b_0 = 0
rows = X.shape[0]
columns = X.shape[1]
weights = np.zeros(columns)
losses = []
for iteration in range(num_iterations):
#Step 1: calculate y hat for row
summation = 0
summation_k = np.zeros(columns)
total_loss = 0
for i in range(rows):
row_total = np.sum(np.multiply(X[i],weights))
y_hat = self.sigma(b_0 + row_total)
y_i = y[i]
# print('y_i: ',y_i)
# print('y_hat: ',y_hat)
# print()
total_loss += self.get_loss(y_i,y_hat)
diff = y_i - y_hat
summation += diff
# summation_k_i = summation_k_i + X[i] * diff
summation_k = np.add(summation_k,np.multiply(diff,X[i]))
# Compute change for each weight based on errors,then update the weights
# Update b_0
b_0 = b_0 + step_size * ((1/rows) * (-summation))
# Update b_k
# for j in range(columns):
# weights[j] = weights[j] + step_size * ((1/rows) * (-summation_k[j]))
weights = np.add(weights,np.multiply(summation_k,(-step_size/rows)))
# Keeping track of average loss for each iteration.
losses.append(total_loss/rows)
self.weights = np.insert(weights,b_0)
return np.array(losses)
运行此命令时,每行和每次迭代的y_hat值始终减小。我找不到导致此错误的错误。
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。