如何解决使用 condor 匹配输出和错误文件制作当前文件和扩展名? 让 PBS 和 Slurm 具有相同的输出文件
如何让 condor 命名我的文件如下:
meta_learning_experiments_submission.py.e451863 meta_learning_experiments_submission.py.o444375
$(FILENAME).e$(CLUSTER)
$(FILENAME).e$(CLUSTER)
我试过了,但似乎不起作用。
例如以便在我执行 qsub
时匹配 PBS 的默认行为?
我也想知道如何让slurm也与之匹配,这样就完美了(所以三个集群都可以工作)。
我试过了:
#SBATCH --error="(filename).%j.%N.err"
但是没用
赏金
我想自动设置文件名而不用硬编码。
相关:
这里的上下文是我当前提交的文件:
####################
#
# Experiments script
# Simple HTCondor submit description file
#
#
# chmod a+x test_condor.py
# chmod a+x experiments_meta_model_optimization.py
# chmod a+x meta_learning_experiments_submission.py
# chmod a+x download_miniImagenet.py
# chmod a+x ~/meta-learning-lstm-pytorch/main.py
# chmod a+x /home/miranda9/automl-meta-learning/automl-proj/meta_learning/datasets/rand_fc_nn_vec_mu_ls_gen.py
# chmod a+x /home/miranda9/automl-meta-learning/automl-proj/experiments/meta_learning/supervised_experiments_submission.py
# chmod a+x /home/miranda9/automl-meta-learning/results_plots/is_rapid_learning_real.py
# condor_submit -i
# condor_submit job.sub
#
####################
Path = /home/miranda9/automl-meta-learning/
Path = /home/miranda9/ML4Coq/
# Executable = /home/miranda9/automl-meta-learning/automl-proj/experiments/meta_learning/supervised_experiments_submission.py
# Executable = /home/miranda9/automl-meta-learning/automl-proj/experiments/meta_learning/meta_learning_experiments_submission.py
# Executable = /home/miranda9/meta-learning-lstm-pytorch/main.py
# Executable = /home/miranda9/automl-meta-learning/automl-proj/meta_learning/datasets/rand_fc_nn_vec_mu_ls_gen.py
# Executable = /home/miranda9/automl-meta-learning/results_plots/is_rapid_learning_real.py
## Output Files
Log = experiment_output_job.$(CLUSTER).log.out
Output = experiment_output_job.$(CLUSTER).out.out
Error = experiment_output_job.$(CLUSTER).err.out
Output = %(FILENAME).o$(CLUSTER)
# Use this to make sure 1 gpu is available. The key words are case insensitive.
# REquest_gpus = 1
# requirements = (CUDADeviceName != "Tesla K40m")
# requirements = (CUDADeviceName == "Quadro RTX 6000")
# requirements = ((CUDADeviceName = "Tesla K40m")) && (TARGET.Arch == "X86_64") && (TARGET.OpSys == "LINUX") && (TARGET.Disk >= RequestDisk) && (TARGET.Memory >= RequestMemory) && (TARGET.Cpus >= RequestCpus) && (TARGET.gpus >= Requestgpus) && ((TARGET.FileSystemDomain == MY.FileSystemDomain) || (TARGET.HasFileTransfer))
# requirements = (CUDADeviceName == "Tesla K40m")
# requirements = (CUDADeviceName == "GeForce GTX TITAN X")
# Note: to use multiple CPUs instead of the default (one CPU),use request_cpus as well
# Request_cpus = 4
Request_cpus = 16
# E-mail option
Notify_user = me@gmail.com
Notification = always
Environment = MY_CONDOR_JOB_ID= $(CLUSTER)
# "Queue" means add the setup until this line to the queue (needs to be at the end of script).
Queue
或者,如果我可以使用我的可执行脚本作为提交脚本,顶部的参数也可以使用
我通常使用与我的可执行脚本相同的提交脚本,例如看我的 qsub 例子:
#!/homes/miranda9/.conda/envs/automl-meta-learning/bin/python
#PBS -V
#PBS -M mee@gmail.com
#PBS -m abe
#PBS -lselect=1:ncpus=112
import sys
import os
for p in sys.path:
print(p)
print(os.environ)
解决方法
HTCondor 默认不设置 FILENAME,但您可以自己设置,例如
FILENAME = meta_learning_experiments_submission.py
output = $(FILENAME).o.$(CLUSTER)
error = $(FILENAME).e.$(CLUSTER)
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。