deepAR算法的批量转换错误

如何解决deepAR算法的批量转换错误

描述错误 你好 我正在为此Amazon SageMaker Batch Transform进行批量转换，以在教程 Stock Price Prediction,using SageMaker DeepAR中进行deepAR模型的实现。 批量转换代码

from sagemaker.transformer import Transformer
model = session.create_model_from_job(estimator._current_job_name,name='{}-test'.format(estimator._current_job_name))

test_transformer = Transformer(model,1,'ml.m4.xlarge',output_path='s3://sagemaker-eu-west-1-xxxxxxxxxxxx/sagemaker/stock-prediction/output',sagemaker_session=session,strategy='BatchStrategy',assemble_with='Line')
test_transformer.transform('s3://sagemaker-eu-west-1-xxxxxxxxxxxx/sagemaker/stock-prediction/source/D/test/test.json',split_type='Line')
test_transformer.wait()

JSON输入

{"start": "2018-07-24 00:00:00","target": [81.73,79.86,83.39,82.91,82.97,82.69,81.5,81.18,82.34,83.51,84.05,84.48,84.81,83.58,83.29,82.45,81.31,81.8,81.41,81.75,82.93,82.29,81.08,81.29,83.19,84.39,84.23,84.6,83.41,82.79,81.88,81.25,80.73,81.07,81.1,80.76,81.32,82.47,82.54,82.67,83.54,85.32,85.77,83.5,79.0,79.03,77.71,78.14,78.7,78.26,77.68,76.87,76.39,75.35,74.3,74.49,75.21,75.75,75.24,75.13,74.64,74.44,73.5,72.69,74.5,75.02,76.4,76.4],"dynamic_feat": [[82.2,83.12,83.27,82.28,81.66,83.02,84.38,84.79,84.68,85.42,84.19,84.02,82.15,81.78,83.15,83.66,81.76,81.46,83.33,85.22,85.75,83.98,82.89,82.95,82.11,81.84,81.19,81.05,81.45,83.35,83.25,83.05,83.81,83.82,85.48,86.74,85.26,83.2,79.16,78.44,79.44,78.99,77.94,77.39,76.78,76.0,75.52,75.86,75.98,75.47,75.2,75.94,74.53,74.09,74.97,79.38,79.38],[81.06,81.22,82.51,81.97,80.88,79.22,82.86,83.53,84.57,82.9,83.28,81.92,80.79,81.3,80.46,81.73,81.95,80.92,82.43,84.26,83.52,83.17,83.21,81.44,81.03,80.43,80.65,80.18,80.37,82.42,82.04,82.26,82.56,83.85,83.4,78.51,77.35,78.18,77.32,78.09,78.0,78.05,77.24,76.72,75.66,75.12,73.74,73.8,75.25,74.65,75.06,73.08,74.07,72.85,72.44,73.48,73.62,75.33,75.33],[81.13,81.59,84.18,82.55,82.6,79.95,82.32,82.96,84.56,84.09,84.77,83.83,83.48,83.46,81.86,82.1,82.06,83.43,81.26,82.44,84.85,83.59,83.63,82.46,82.68,81.48,80.85,80.67,81.02,82.58,82.14,82.72,83.93,86.52,85.21,82.98,77.98,78.19,77.59,78.31,78.85,78.55,77.64,77.16,76.65,75.89,73.86,74.63,75.42,75.27,76.19,74.16,73.16,73.42,73.59,75.48,75.48]]}
{"start": "2018-07-24 00:00:00","target": [196.9,193.9,197.8,197.3,197.45,197.0,189.85,186.45,184.45,186.65,188.15,188.05,189.0,186.5,187.45,185.9,182.9,184.05,182.65,184.9,185.3,160.85,153.95,155.45,160.1,159.9,160.35,161.1,158.05,156.6,155.15,152.7,151.4,150.15,150.35,147.0,149.6,149.35,150.7,151.1,150.0,151.9,158.0,159.1,156.8,152.9,151.7,151.5,149.95,152.75,146.95,144.3,143.85,143.1,139.2,137.0,140.15,138.4,140.6,138.95,137.75,131.5,133.3,129.45,138.5,136.1,143.55,143.55],"dynamic_feat": [[197.25,196.25,198.1,198.35,198.3,197.7,196.45,186.8,187.75,188.65,189.35,188.4,190.05,187.1,188.45,187.25,186.35,185.0,183.9,186.05,185.65,186.1,158.95,155.95,161.4,160.15,160.7,162.45,159.55,157.6,157.55,155.2,153.55,152.15,150.55,149.85,151.35,151.8,152.45,158.4,157.95,156.85,154.25,148.25,144.45,144.0,145.1,142.0,140.1,141.3,140.55,142.3,139.45,140.75,133.0,134.6,134.25,133.85,140.95,136.2,146.65,146.65],[194.0,192.5,195.7,196.15,196.75,195.25,189.3,183.0,185.25,187.7,186.55,187.85,185.35,184.55,182.05,183.15,181.25,184.6,184.0,157.9,153.35,154.1,157.05,157.75,159.3,154.45,149.75,148.85,150.25,146.75,149.25,148.8,149.65,154.0,158.3,156.35,151.6,148.5,148.95,150.4,151.15,145.05,141.4,142.5,142.35,138.8,137.6,137.5,139.35,137.7,127.9,132.25,130.6,129.15,130.9,133.1,135.9,135.9],[194.15,195.75,196.05,198.2,196.85,196.7,195.65,183.6,186.9,185.8,188.5,187.5,188.3,186.95,186.15,184.25,182.6,158.5,155.25,157.8,159.8,159.95,158.9,156.9,157.25,154.3,151.85,150.2,148.6,149.9,150.5,151.55,150.05,159.65,156.25,150.75,150.3,152.2,143.6,143.4,144.75,141.7,139.05,139.4,132.7,134.55,131.6,131.4,135.35,136.8,136.8]]}
{"start": "2018-07-24 00:00:00","target": [59.14,57.88,59.51,59.29,59.35,59.15,58.24,57.41,58.16,58.25,58.89,59.06,59.04,57.2,57.23,56.45,55.05,55.29,54.92,55.87,55.3,54.41,54.75,56.11,56.3,56.67,56.65,55.7,55.0,54.2,54.28,54.32,54.48,54.5,54.05,54.53,55.09,55.54,55.41,55.77,56.32,57.21,57.61,56.13,54.74,54.83,55.59,54.35,54.88,56.0,56.44,55.61,54.6,54.42,53.6,52.7,52.89,52.92,53.23,52.76,52.41,51.39,50.81,50.48,50.0,51.35,51.37,52.43,52.43],"dynamic_feat": [[59.72,58.79,59.75,59.6,59.62,59.59,58.85,57.71,58.53,58.99,59.4,59.27,59.38,58.11,57.86,57.35,56.71,55.72,55.99,56.37,54.97,56.21,56.98,56.73,57.33,56.27,55.44,54.98,54.66,54.68,54.45,54.54,56.14,55.85,56.47,57.56,58.81,57.08,56.16,55.36,55.1,56.43,57.09,55.94,55.26,54.3,53.39,53.84,53.21,53.54,52.88,53.52,52.25,52.22,50.86,51.77,54.53],[58.42,57.31,57.82,59.17,58.82,58.06,56.76,57.69,58.09,58.59,58.84,56.97,56.06,54.22,55.16,54.84,54.36,55.17,56.01,55.69,55.63,53.86,53.87,54.25,54.1,54.0,53.75,53.78,55.02,55.13,55.32,55.84,56.51,57.36,56.05,53.73,54.47,54.12,54.33,55.37,53.63,52.13,52.75,52.03,53.0,52.28,52.37,48.77,49.78,49.84,50.57,50.34,51.37],[58.43,59.54,59.21,59.46,58.54,57.16,57.78,58.31,59.25,58.88,59.0,57.81,57.52,57.26,56.69,55.48,55.51,55.18,56.62,56.35,55.83,55.21,54.85,53.91,54.37,54.09,54.44,54.11,55.35,56.09,58.56,57.03,55.97,54.94,56.7,55.89,55.11,54.52,54.13,52.15,53.22,53.14,52.77,53.31,51.83,52.07,50.09,50.83,50.69,50.74,51.4,51.4]]}
{"start": "2018-07-24 00:00:00","target": [57.1,55.92,58.18,58.38,58.32,57.9,56.2,56.02,55.86,57.14,56.88,55.42,55.06,54.8,53.74,53.72,53.4,53.76,53.92,53.24,53.44,55.76,55.28,52.9,52.06,52.0,51.6,51.7,52.84,54.72,55.56,58.36,59.08,59.12,58.34,57.54,57.58,58.72,58.0,58.4,58.3,57.1,56.04,53.42,57.0,56.46,55.9,54.62,53.88,50.78,51.76,51.56,53.52],"dynamic_feat": [[57.42,56.92,58.46,58.44,55.4,56.64,57.22,57.24,55.46,55.52,54.38,54.34,55.2,53.8,53.46,56.08,53.16,53.08,51.82,53.28,55.8,59.44,60.46,59.02,57.62,59.24,58.2,58.94,58.8,58.62,56.48,56.6,55.64,57.66,56.94,52.08,54.72],[55.94,55.88,57.98,58.1,57.7,55.78,55.5,56.74,53.3,53.32,52.36,53.62,53.7,53.34,53.12,54.82,52.98,52.38,52.48,52.14,51.22,51.66,51.32,51.44,57.4,58.66,58.08,56.9,56.66,57.68,57.74,56.96,52.86,53.26,53.2,55.74,53.48,54.26,51.24,50.42,51.56],[56.0,58.22,54.9,57.28,54.96,54.86,53.9,54.7,51.68,51.9,53.56,60.26,58.74,56.82,57.64,58.28,57.44,55.6,53.06,51.16,51.56]]}
{"start": "2018-07-24 00:00:00","target": [150.38,145.92,151.12,151.46,152.22,146.6,145.34,145.48,146.88,147.82,147.84,144.38,143.98,142.9,138.44,139.44,138.74,139.0,140.72,138.82,137.94,138.0,141.46,143.38,143.16,142.86,140.84,136.14,136.24,136.08,137.38,140.78,144.48,148.24,152.5,154.38,152.94,150.48,151.3,153.86,152.08,149.3,146.54,143.54,140.48,140.16,144.42,147.42,147.1,145.58,142.34,139.7,133.7,136.88,142.42,142.42],"dynamic_feat": [[151.6,148.98,152.34,153.42,153.36,143.9,145.62,147.08,149.1,148.72,146.78,144.92,144.68,143.36,140.74,139.32,140.62,141.16,138.52,142.14,143.52,145.36,139.18,138.3,137.2,138.04,136.32,138.94,137.76,138.56,141.88,144.7,145.26,150.24,151.96,157.2,153.62,155.0,151.54,155.68,152.52,154.46,154.26,150.68,147.76,148.06,146.1,142.8,142.62,146.72,149.32,147.4,144.84,139.54,137.88,146.34,146.34],[148.84,149.66,150.52,150.92,140.38,143.34,146.8,146.04,146.64,143.24,143.68,137.46,138.9,136.54,138.02,136.66,137.52,136.86,139.3,141.64,140.66,140.12,136.42,134.9,135.42,135.54,134.2,135.6,136.04,136.56,140.68,141.14,143.2,146.5,152.8,151.2,149.06,147.94,150.44,149.86,151.68,151.86,149.02,146.56,144.54,142.88,138.1,139.66,145.0,145.52,145.4,139.16,141.68,138.54,135.0,134.02,137.92,137.92],[149.02,148.64,153.32,151.82,151.32,150.12,142.12,143.76,145.3,147.72,146.94,147.5,145.56,144.06,140.06,138.06,140.3,141.78,138.16,138.08,139.34,144.72,141.74,138.26,135.94,140.98,141.38,143.64,149.14,151.72,153.92,148.38,150.46,151.34,152.58,154.06,153.48,147.6,147.8,138.32,141.8,146.96,148.96,139.58,138.58,135.3,137.98,137.98]]}

JSON.out 当我打开输出文件JSON.out时，它有5行，如下所示。 （我猜每个股票唯一实例的每一行）。 在控制台上运行批处理转换时，也会出现此错误。

{"error":"The field dynamic_feat needs to be provided in the full prediction range but request has dynamic_feat only for 0 time units in the prediction range when trying to predict for 91 time units."}

要复制 将DeepAR笔记本中的Forecasting and Plotting代码替换为Batch Transform。

预期的行为 对您期望发生的事情的简洁明了的描述。 JSON输出的目标库存值为3个月。

系统信息

• SageMaker Python SDK版本：
• 框架名称（例如PyTorch）或算法（例如KMeans）：
• 框架版本：
• Python版本：3.7.7
• CPU或GPU ：CPU
• 自定义Docker映像（是/否）：否

解决方法

我认为您已经对批处理转换作业的期望进行了描述。

批处理变换可预测将来的值。由于您具有91个时间单位的预测范围（错误消息中已提到），因此，期望为所有这些时间单位提供动态功能。

如documentation中所述：

如果模型是使用dynamic_feat字段训练的，则必须提供 该字段供推断。另外，每个特征都必须具有所提供目标的长度加上预测长度。换句话说，您将来必须提供功能值。

现在，目标和每个动态功能都有97个值。期望您为每个动态功能提供其他91个值。考虑到这些额外的91个值，将对目标进行预测。

,

正是！

如果您使用动态特征进行训练，您基本上是在为模型提供辅助回归。 即：对于预测未来 x 个周期的模型，它需要这些 x 个周期的辅助回归值。

因此，如果您使用动态特征进行训练，请想象您正在使用 10 个时间步长的时间序列进行推理并预测接下来的 3 个。

您输入的时间序列将为（长度为 10）：

[1,2,3,4,5,6,7,8,9,10]

但您输入的动态特征系列必须是（长度为 13）：

[1,10,x1,x2. x3]