微信公众号搜"智元新知"关注
微信扫一扫可直接关注哦!

我的预测值仅在梯度下降时才会降低

如何解决我的预测值仅在梯度下降时才会降低

我目前正在编写梯度下降的实现,但遇到了一个问题,即我的预测值(y_hat)只会降低。即使在训练标签为1而不是0的情况下应该增加,它也永远不会增加。我的火车功能代码如下:

def sigma(self,a):
    ans = 1/(1+np.exp(-a))
    return ans

  def get_loss(self,y_i,y_hat):
    loss = -(y_i * np.log(y_hat) + (1 - y_i) * np.log(1 - y_hat))
    return loss

def train(self,X,y,step_size,num_iterations):
    b_0 = 0
    rows = X.shape[0]
    columns = X.shape[1]
    weights = np.zeros(columns)
    losses = []
    for iteration in range(num_iterations):
      #Step 1: calculate y hat for row 
      summation = 0
      summation_k = np.zeros(columns)
      total_loss = 0
      for i in range(rows):
        row_total = np.sum(np.multiply(X[i],weights))
        y_hat = self.sigma(b_0 + row_total)
        y_i = y[i]
        # print('y_i: ',y_i)
        # print('y_hat: ',y_hat)
        # print()
        total_loss += self.get_loss(y_i,y_hat)
        diff = y_i - y_hat
        summation += diff

        # summation_k_i = summation_k_i + X[i] * diff
        summation_k = np.add(summation_k,np.multiply(diff,X[i]))
        
      # Compute change for each weight based on errors,then update the weights
      # Update b_0
      b_0 = b_0 + step_size * ((1/rows) * (-summation))

      # Update b_k
      # for j in range(columns):
      #   weights[j] = weights[j] + step_size * ((1/rows) * (-summation_k[j]))
      weights = np.add(weights,np.multiply(summation_k,(-step_size/rows)))
      
      # Keeping track of average loss for each iteration.
      losses.append(total_loss/rows)
    
    self.weights = np.insert(weights,b_0)
    return np.array(losses)

运行此命令时,每行和每次迭代的y_hat值始终减小。我找不到导致此错误错误

版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。