如何解决您好,关于sklearn.Pipeline与时间序列的自定义转换器的两个问题
我应该如何修改下面的代码以使其起作用:
目标,预测= pipe.fit_predict(df)
编辑:
target,predicted = pipe.fit_transform(df,df)
我的代码:
import numpy as np
import pandas as pd
from sklearn.base import BaseEstimator
from sklearn.base import TransformerMixin
from sklearn.pipeline import Pipeline
np.random.seed(1)
rows,cols = 100,1
data = np.random.randint(100,size = (rows,cols))
tidx = pd.date_range('2019-01-01',periods=rows,freq='20min')
df = pd.DataFrame(data,columns=['num_orders'],index=tidx)
class MakeFeatures(BaseEstimator,TransformerMixin):
def __init__(self,X,y = None,max_lag = None,rolling_mean_day = None,rolling_mean_month = None):
self.X = X.resample('1H').sum()
self.max_lag = max_lag
self.rolling_mean_day = rolling_mean_day
self.rolling_mean_month = rolling_mean_month
def fit(self,y = None):
return self
def transform(self,y = None):
data = pd.DataFrame(index = self.X.index)
data['num_orders'] = self.X['num_orders']
data['year'] = self.X.index.year
data['month'] = self.X.index.month
data['day'] = self.X.index.day
data['dayofweek'] = self.X.index.dayofweek
data['detrend'] = self.X.shift() - self.X
if self.max_lag:
for lag in range(1,self.max_lag + 1):
data['lag_{}'.format(lag)] = data['detrend'].shift(lag)
if self.rolling_mean_day:
data['rolling_mean_24'] = data.detrend.shift().rolling(self.rolling_mean_day).mean()
if self.rolling_mean_month:
data['rolling_mean_24'] = data['detrend'].shift().rolling(self.rolling_mean_month).mean()
if data['year'].mean() == data['year'][1]:
data = data.drop('year',axis = 1)
data = data.dropna()
y = data.num_orders
data = data.drop('num_orders',1)
return data,y
pipe = Pipeline([
('features',MakeFeatures(df,df,2,24)),('scaler',StandardScaler())
])
target,df) # where ‘Target’ is y - the output from the Class
出局:
ValueError: could not broadcast input array from shape (9,7) into shape (9).
管道中的每个功能都工作正常。
我可以运行 MakeFeatures(df,df)和 StandardScaler()。fit_transform(df,df)。
我可以将MakeFeatures(df,df)的产品插入StandardScaler,并且没有错误。
解决方法
您不能使用
目标,预测= pipe.fit_predict(df)
与定义的管道一起使用,因为fit_predict()方法只能在估算器也实现了这种方法的情况下使用。 Reference in documentation
仅在最终估算器实现fit_predict时有效。
此外,它只会返回预测,因此您不能使用target,predicted =
,而应该使用predicted =
您遇到了错误
ValueError:设置具有序列的数组元素。
因为您要提供StandardScaler()
和pandas.TimeSeries
。
这是因为通过方法调用pipe.fit_predict(df)
,您仅向管道提供了“ X”而不是“ y”。这对于管道“ MakeFeatures”的第一个组件很好,因为它接受“ X”并返回“ data”和“ y”,但是在管道中将不使用“ y”,因为“ y”必须在fit_predict()调用的开头定义。
它表示“ y”参数
培训目标。必须满足所有步骤的标签要求 管道。
因此,“ y”将用作管道所有部分的“ y”,但您的未定义,所以None
。
因此,当前的管道基本上会发生以下情况:
makeF = MakeFeatures(df,2,24)
transformed_df = makeF.fit_transform(df)
sc = StandardScaler()
sc.fit(transformed_df)
并导致ValueError: setting an array element with a sequence.
所以我建议您像这样更新代码:
import numpy as np
import pandas as pd
from sklearn.base import BaseEstimator
from sklearn.base import TransformerMixin
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LinearRegression
np.random.seed(1)
rows,cols = 100,1
data = np.random.randint(100,size = (rows,cols))
tidx = pd.date_range('2019-01-01',periods=rows,freq='20min')
df = pd.DataFrame(data,columns=['num_orders'],index=tidx)
class MakeFeatures(BaseEstimator,TransformerMixin):
def __init__(self,X,max_lag = None,rolling_mean_day = None,rolling_mean_month = None):
self.X = X.resample('1H').sum()
self.max_lag = max_lag
self.rolling_mean_day = rolling_mean_day
self.rolling_mean_month = rolling_mean_month
def fit(self,X):
return self
def transform(self,X):
data = pd.DataFrame(index = self.X.index)
data['num_orders'] = self.X['num_orders']
data['year'] = self.X.index.year
data['month'] = self.X.index.month
data['day'] = self.X.index.day
data['dayofweek'] = self.X.index.dayofweek
data['detrend'] = self.X.shift() - self.X
if self.max_lag:
for lag in range(1,self.max_lag + 1):
data['lag_{}'.format(lag)] = data['detrend'].shift(lag)
if self.rolling_mean_day:
data['rolling_mean_24'] = data.detrend.shift().rolling(self.rolling_mean_day).mean()
if self.rolling_mean_month:
data['rolling_mean_24'] = data['detrend'].shift().rolling(self.rolling_mean_month).mean()
if data['year'].mean() == data['year'][1]:
data = data.drop('year',axis = 1)
data = data.dropna()
y = data.num_orders
data = data.drop('num_orders',1)
return data,list(y)
pipe = Pipeline([
('scaler',StandardScaler()),('Model',LinearRegression())
])
makeF = MakeFeatures(df,24)
makeF.fit(df)
data,y = makeF.transform(df)
pipe.fit(data,y) # where ‘Target’ is y - the output from the Class
然后,您可以使用管道来预测数据并评估性能,例如使用r2_score:
from sklearn.metrics import r2_score
predictions = pipe.predict(data)
r2_score(y,predictions)
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。