RLlib Trainer 不提供 TensorBoard 的事件文件

分类：编程问答

如何解决RLlib Trainer 不提供 TensorBoard 的事件文件

当我运行 RLlib 训练器时，您将在下面看到日志输出中的前几行相关行，希望看到生成的 TfEvent 文件，以便我可以在 TensorBoard 上查看它们。

我使用的基本 docker 映像是 tensorflow/tensorflow:latest-gpu-py3，它给我 TensorFlow 2.5，我使用 Ray 1.3.0 进行训练。

WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/compat/v2_compat.py:88: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
/usr/local/lib/python3.6/dist-packages/ray/autoscaler/_private/cli_logger.py:61: FutureWarning: Not all Ray CLI dependencies were found. In Ray 1.4+,the Ray CLI,autoscaler,and dashboard will only be usable via `pip install 'ray[default]'`. Please update your install command.
  "update your install command.",FutureWarning)
2021-06-27 20:44:37,007 INFO services.py:1269 -- View the Ray dashboard at http://127.0.0.1:8265
/usr/local/lib/python3.6/dist-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN',msg % args),'yellow'))
2021-06-27 20:44:38,828 INFO logger.py:180 -- pip install 'ray[tune]' to see TensorBoard files.
2021-06-27 20:44:38,828 WARNING logger.py:317 -- Could not instantiate TBXLogger: No module named 'torch'.
2021-06-27 20:44:38,834 INFO trainer.py:669 -- Tip: set framework=tfe or the --eager flag to enable TensorFlow eager execution
(pid=211) WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/compat/v2_compat.py:88: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
(pid=211) Instructions for updating:
(pid=211) non-resource variables are not supported in the long term
(pid=211) 2021-06-27 20:44:40,339   INFO rollout_worker.py:497 -- Could not seed torch
(pid=211) 2021-06-27 20:44:40,390   INFO catalog.py:387 -- Wrapping <class 'my_vision_net.MyVisionNetwork'> as None
(pid=211) WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1635: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
(pid=211) Instructions for updating:
(pid=211) If using Keras pass *_constraint arguments to layers.
(pid=211) WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1635: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
(pid=211) Instructions for updating:
(pid=211) If using Keras pass *_constraint arguments to layers.
(pid=211) Constructing vision network...
(pid=211) 2021-06-27 20:44:40,612   INFO dynamic_tf_policy.py:463 -- Testing `compute_actions` w/ dummy batch.
(pid=211) 2021-06-27 20:44:40,612   INFO tf_run_builder.py:87 -- Executing TF run without tracing. To dump TF timeline traces to disk,set the TF_TIMELINE_DIR environment variable.
(pid=211) 2021-06-27 20:44:40,678   INFO dynamic_tf_policy.py:472 -- Adding extra-action-fetch `action_prob` to view-reqs.
(pid=211) 2021-06-27 20:44:40,678   INFO dynamic_tf_policy.py:472 -- Adding extra-action-fetch `action_logp` to view-reqs.
(pid=211) 2021-06-27 20:44:40,679   INFO dynamic_tf_policy.py:472 -- Adding extra-action-fetch `action_dist_inputs` to view-reqs.
(pid=211) 2021-06-27 20:44:40,679   INFO dynamic_tf_policy.py:472 -- Adding extra-action-fetch `vf_preds` to view-reqs.
(pid=211) 2021-06-27 20:44:40,679   INFO dynamic_tf_policy.py:479 -- Testing `postprocess_trajectory` w/ dummy batch.
2021-06-27 20:44:41,162 INFO rollout_worker.py:490 -- Env doesn't support env.seed(): None
2021-06-27 20:44:41,162 INFO rollout_worker.py:497 -- Could not seed torch
2021-06-27 20:44:41,450 INFO catalog.py:387 -- Wrapping <class 'my_vision_net.MyVisionNetwork'> as None
Constructing vision network...
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1635: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1635: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
2021-06-27 20:44:42,235 INFO dynamic_tf_policy.py:463 -- Testing `compute_actions` w/ dummy batch.
2021-06-27 20:44:42,235 INFO tf_run_builder.py:87 -- Executing TF run without tracing. To dump TF timeline traces to disk,set the TF_TIMELINE_DIR environment variable.
2021-06-27 20:44:43,892 INFO dynamic_tf_policy.py:472 -- Adding extra-action-fetch `action_prob` to view-reqs.
2021-06-27 20:44:43,893 INFO dynamic_tf_policy.py:472 -- Adding extra-action-fetch `action_logp` to view-reqs.
2021-06-27 20:44:43,893 INFO dynamic_tf_policy.py:472 -- Adding extra-action-fetch `action_dist_inputs` to view-reqs.
2021-06-27 20:44:43,894 INFO dynamic_tf_policy.py:472 -- Adding extra-action-fetch `vf_preds` to view-reqs.
2021-06-27 20:44:43,894 INFO dynamic_tf_policy.py:479 -- Testing `postprocess_trajectory` w/ dummy batch.
2021-06-27 20:44:44,455 INFO rollout_worker.py:1161 -- Built policy map: {'my_ppo': <ray.rllib.policy.tf_policy_template.PPOTFPolicy object at 0x7f5da47b3438>}
2021-06-27 20:44:44,455 INFO rollout_worker.py:1162 -- Built preprocessor map: {'my_ppo': <ray.rllib.models.preprocessors.NoPreprocessor object at 0x7f5da47b30b8>}
2021-06-27 20:44:44,456 INFO rollout_worker.py:563 -- Built filter map: {'my_ppo': <ray.rllib.utils.filter.NoFilter object at 0x7f5da479af60>}
2021-06-27 20:44:44,475 WARNING deprecation.py:34 -- DeprecationWarning: `simple_optimizer` has been deprecated. This will raise an error in the future!
(pid=202) WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/compat/v2_compat.py:88: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
(pid=202) Instructions for updating:
(pid=202) non-resource variables are not supported in the long term
(pid=202) 2021-06-27 20:44:45,909   INFO rollout_worker.py:497 -- Could not seed torch
(pid=202) 2021-06-27 20:44:45,941   INFO catalog.py:387 -- Wrapping <class 'my_vision_net.MyVisionNetwork'> as None
(pid=202) Constructing vision network...
(pid=202) WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1635: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
(pid=202) Instructions for updating:
(pid=202) If using Keras pass *_constraint arguments to layers.
(pid=202) WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1635: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
(pid=202) Instructions for updating:
(pid=202) If using Keras pass *_constraint arguments to layers.
(pid=202) 2021-06-27 20:44:46,118   INFO dynamic_tf_policy.py:463 -- Testing `compute_actions` w/ dummy batch.
(pid=202) 2021-06-27 20:44:46,118   INFO tf_run_builder.py:87 -- Executing TF run without tracing. To dump TF timeline traces to disk,set the TF_TIMELINE_DIR environment variable.
(pid=202) 2021-06-27 20:44:46,164   INFO dynamic_tf_policy.py:472 -- Adding extra-action-fetch `action_prob` to view-reqs.
(pid=202) 2021-06-27 20:44:46,165   INFO dynamic_tf_policy.py:472 -- Adding extra-action-fetch `action_logp` to view-reqs.
(pid=202) 2021-06-27 20:44:46,165   INFO dynamic_tf_policy.py:472 -- Adding extra-action-fetch `action_dist_inputs` to view-reqs.
(pid=202) 2021-06-27 20:44:46,165   INFO dynamic_tf_policy.py:472 -- Adding extra-action-fetch `vf_preds` to view-reqs.
(pid=202) 2021-06-27 20:44:46,166   INFO dynamic_tf_policy.py:479 -- Testing `postprocess_trajectory` w/ dummy batch.
2021-06-27 20:44:46,657 INFO rollout_worker.py:490 -- Env doesn't support env.seed(): None
2021-06-27 20:44:46,657 INFO rollout_worker.py:497 -- Could not seed torch
2021-06-27 20:44:46,661 INFO catalog.py:387 -- Wrapping <class 'my_vision_net.MyVisionNetwork'> as None

我可以看到主要问题是线路：

2021-06-27 20:44:38,828 WARNING logger.py:317 -- Could not instantiate TBXLogger: No module named 'torch'.

当我使用 TensorFlow 时，我不明白为什么它甚至需要 Torch。另外，我真的需要它来生成那些 TFevent 文件。这里有一个类似的问题 (https://github.com/ray-project/ray/issues/5546)，但我不能在这种情况下应用相同的解决方案。

编辑（相关代码供参考）：

def custom_log_creator(custom_path,custom_str):
    timestr = datetime.today().strftime("%Y-%m-%d_%H-%M-%S")
    logdir_prefix = "{}_{}".format(custom_str,timestr)

    def logger_creator(config):

        if not os.path.exists(custom_path):
            os.makedirs(custom_path)            os.makedirs(os.path.join(custom_path,logdir_prefix),exist_ok=True)
        logdir = os.path.join(custom_path,logdir_prefix)
        return UnifiedLogger(config,logdir,loggers=None)

    return logger_creator

log_path = os.path.join(pathlib.Path(__file__).parent.absolute(),'results')

trainer = ppo.PPOTrainer(env='test_env',config=policy_conf,logger_creator=custom_log_creator(custom_path=log_path,custom_str='res'))

版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 dio@foxmail.com 举报，一经查实，本站将立刻删除。

相关推荐

导入项目后报错问题
依赖报错 idea导入项目后依赖报错，解决方案：https://blog.csdn.net/weixin_42420249/article/details/81191861 依赖版本报错：更换其他版本 无法下载依赖可参考：https://blog.csdn.net/weixin_42628809/a
idea不能识别yaml文件
使用mybatis plus常见错误
错误1：代码生成器依赖和mybatis依赖冲突 启动项目时报错如下 2021-12-03 13:33:33.927 ERROR 7228 [ main] o.s.b.d.LoggingFailureAnalysisReporter : *************************** APPL
gradle常见问题与错误
错误1：gradle项目控制台输出为乱码 # 解决方案：https://blog.csdn.net/weixin_43501566/article/details/112482302 # 在gradle-wrapper.properties 添加以下内容 org.gradle.jvmargs=-Df
Mybatis Plus传入参数0不起作用
错误还原：在查询的过程中，传入的workType为0时，该条件不起作用 &lt;select id=&quot;xxx&quot;&gt; SELECT di.id, di.name, di.work_type, di.updated... &lt;where&gt; &lt;if test=&qu
linux中make编译源码包失败
报错如下，gcc版本太低 ^ server.c:5346:31: 错误：‘struct redisServer’没有名为‘server_cpulist’的成员 redisSetCpuAffinity(server.server_cpulist); ^ server.c: 在函数‘hasActiveC
微服务启动错误：Command line is too long. Shorten command line for
解决方案1 1、改项目中.idea/workspace.xml配置文件，增加dynamic.classpath参数 2、搜索PropertiesComponent，添加如下 &lt;property name=&quot;dynamic.classpath&quot; value=&quot;tru
vue常见错误
删除根组件app.vue中的默认代码后报错：Module Error (from ./node_modules/eslint-loader/index.js): 解决方案：关闭ESlint代码检测，在项目根目录创建vue.config.js，在文件中添加 module.exports = { lin
spark常见错误
查看spark默认的python版本 [root@master day27]# pyspark /home/software/spark-2.3.4-bin-hadoop2.7/conf/spark-env.sh: line 2: /usr/local/hadoop/bin/hadoop: No s
matplotlib报错：AttributeError: module 'backend_interagg' has no attribute 'FigureCanvas'. Did you mean: 'FigureCanvasAgg'?
使用本地python环境可以成功执行 import pandas as pd import matplotlib.pyplot as plt # 设置字体 plt.rcParams[&#39;font.sans-serif&#39;] = [&#39;SimHei&#39;] # 能正确显示负号 p
gitlab登录失败，报错：This challenge page was accidentally cached by an intermediary and is no longer available.
设置时间 控制面板
后端开发常见错误
错误1：Request method ‘DELETE‘ not supported 错误还原：controller层有一个接口，访问该接口时报错：Request method ‘DELETE‘ not supported 错误原因：没有接收到前端传入的参数，修改为如下 参考 错误2：cannot r
docker常见错误
错误1：启动docker镜像时报错：Error response from daemon: driver failed programming external connectivity on endpoint quirky_allen 解决方法：重启docker -&gt; systemctl r
idea常见错误
错误1：private field ‘xxx‘ is never assigned 按Altʾnter快捷键，选择第2项 参考：https://blog.csdn.net/shi_hong_fei_hei/article/details/88814070 错误2：启动时报错，不能找到主启动类 #
pip安装依赖失败
报错如下，通过源不能下载，最后警告pip需升级版本 Requirement already satisfied: pip in c:\users\ychen\appdata\local\programs\python\python310\lib\site-packages (22.0.4) Coll
maven常见错误
错误1：maven打包报错 错误还原：使用maven打包项目时报错如下 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-resources-plugin:3.2.0:resources (default-resources)
cloud项目中常见错误
错误1：服务调用时报错 服务消费者模块assess通过openFeign调用服务提供者模块hires 如下为服务提供者模块hires的控制层接口 @RestController @RequestMapping(&quot;/hires&quot;) public class FeignControl
springboot常见错误
错误1：运行项目后报如下错误 解决方案 报错2：Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project sb 解决方案：在pom.
springmvc拦截器中使用redisTemplate报空指针异常
参考 错误原因 过滤器或拦截器在生效时，redisTemplate还没有注入 解决方案：在注入容器时就生效 @Component //项目运行时就注入Spring容器 public class RedisBean { @Resource private RedisTemplate&lt;String
vite报错：npm ERR! command C:WINDOWSsystem32cmd.exe /d /s /c C:UsersychenAppDataLocalTempnpx-dde1c848.cmd
使用vite构建项目报错 C:\Users\ychen\work&gt;npm init @vitejs/app @vitejs/create-app is deprecated, use npm init vite instead C:\Users\ychen\AppData\Local\npm-
pythonJavaScriptjavaHTMLPHPreactjsC#AndroidCSSNode.jssqlrpython-3.xMysqLjQueryc++pandasFlutterangularIOSdjangolinuxswifttypescript路由器JSON路由器设置无线路由器h3c华三华三路由器设置华三路由器电脑软件教程arraysdocker软件图文教程Cvue.jslaravelspring-boot