如何解决达拉斯分布式调度程序-错误-无法收集密钥
import joblib
from sklearn.externals.joblib import parallel_backend
with joblib.parallel_backend('dask'):
from dask_ml.model_selection import GridSearchCV
import xgboost
from xgboost import XGBRegressor
grid_search = GridSearchCV(estimator= XGBRegressor(),param_grid = param_grid,cv = 3,n_jobs = -1)
grid_search.fit(df2,df3)
我使用两台本地计算机创建了一个dask集群
client = dask.distributed.client('tcp://191.xxx.xx.xxx:8786')
我正在尝试使用dask gridsearchcv查找最佳参数。我遇到以下错误。
istributed.scheduler - ERROR - Couldn't gather keys {"('xgbregressor-fit-score-7cb7087b3aff75a31f487cfe5a9cedb0',1202,2)": ['tcp://127.0.0.1:3738']} state: ['processing'] workers: ['tcp://127.0.0.1:3738']
NoneType: None
distributed.scheduler - ERROR - Workers don't have promised key: ['tcp://127.0.0.1:3738'],('xgbregressor-fit-score-7cb7087b3aff75a31f487cfe5a9cedb0',2)
NoneType: None
distributed.client - WARNING - Couldn't gather 1 keys,rescheduling {"('xgbregressor-fit-score-7cb7087b3aff75a31f487cfe5a9cedb0',2)": ('tcp://127.0.0.1:3738',)}
distributed.nanny - WARNING - Restarting worker
distributed.scheduler - ERROR - Couldn't gather keys {"('xgbregressor-fit-score-7cb7087b3aff75a31f487cfe5a9cedb0',1,2)": ['tcp://127.0.0.1:3730']} state: ['processing'] workers: ['tcp://127.0.0.1:3730']
NoneType: None
distributed.scheduler - ERROR - Couldn't gather keys {"('xgbregressor-fit-score-7cb7087b3aff75a31f487cfe5a9cedb0',1)": ['tcp://127.0.0.1:3730'],"('xgbregressor-fit-score-7cb7087b3aff75a31f487cfe5a9cedb0',5,1)": ['tcp://127.0.0.1:3729'],4,2)": ['tcp://127.0.0.1:3729'],2,1)": ['tcp://127.0.0.1:3730']} state: ['processing','processing','processing'] workers: ['tcp://127.0.0.1:3730','tcp://127.0.0.1:3729']
NoneType: None
distributed.scheduler - ERROR - Couldn't gather keys {'cv-n-samples-7cb7087b3aff75a31f487cfe5a9cedb0': ['tcp://127.0.0.1:3729']} state: ['processing'] workers: ['tcp://127.0.0.1:3729']
NoneType: None
distributed.scheduler - ERROR - Couldn't gather keys {"('xgbregressor-fit-score-7cb7087b3aff75a31f487cfe5a9cedb0',0)": ['tcp://127.0.0.1:3729'],0)": ['tcp://127.0.0.1:3729']} state: ['processing','processing'] workers: ['tcp://127.0.0.1:3729']
NoneType: None
distributed.scheduler - ERROR - Couldn't gather keys {"('xgbregressor-fit-score-7cb7087b3aff75a31f487cfe5a9cedb0',2)": ['tcp://127.0.0.1:3729']} state: ['processing','processing'] workers: ['tcp://127.0.0.1:3729']
NoneType: None
distributed.scheduler - ERROR - Workers don't have promised key: ['tcp://127.0.0.1:3730'],2)
NoneType: None
我希望有人能帮助解决这个问题。预先感谢。
解决方法
当我厌倦了在 ec2 实例上本地运行 dask 时,我遇到了同样的问题。为了解决它,我使用了:
from distributed import Client
from dask import config
config.set({'interface': 'lo'}) #<---found out to use 'lo' by running ifconfig in shell
client = Client()
这个问题帮我找到了解决方案:https://github.com/dask/distributed/issues/1281
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。