在具有大数据帧的XGBoost上运行BayesianOptimization时,为什么会出错?

如何解决在具有大数据帧的XGBoost上运行BayesianOptimization时,为什么会出错?

当我在较小的数据帧上运行带有XGBoost的BayesianOptimization时,一切工作都很好,但是当我使用较大的数据帧时,会遇到各种错误。我最近一次运行它收到了

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~\anaconda3\envs\tf-gpu\lib\site-packages\bayes_opt\target_space.py in probe(self,params)
    190         try:
--> 191             target = self._cache[_hashable(x)]
    192         except KeyError:

KeyError: (0.5414533613708468,0.4244312334264955,2.505215712179838,3.29866527650986,6937.176867016293,0.833081120152225)

During handling of the above exception,another exception occurred:

XGBoostError                              Traceback (most recent call last)
<ipython-input-2-ef2efb90fddb> in <module>
     53 # Use the expected improvement acquisition function to handle negative numbers
     54 # Optimally needs quite a few more initiation points and number of iterations
---> 55 xgb_bo.maximize(init_points=10,n_iter=100,acq='ei')
     56 print(xgb_bo.max)

~\anaconda3\envs\tf-gpu\lib\site-packages\bayes_opt\bayesian_optimization.py in maximize(self,init_points,n_iter,acq,kappa,xi,**gp_params)
    172                 iteration += 1
    173 
--> 174             self.probe(x_probe,lazy=False)
    175 
    176         self.dispatch(Events.OPTIMIZATION_END)

~\anaconda3\envs\tf-gpu\lib\site-packages\bayes_opt\bayesian_optimization.py in probe(self,params,lazy)
    110             self._queue.add(params)
    111         else:
--> 112             self._space.probe(params)
    113             self.dispatch(Events.OPTIMIZATION_STEP)
    114 

~\anaconda3\envs\tf-gpu\lib\site-packages\bayes_opt\target_space.py in probe(self,params)
    192         except KeyError:
    193             params = dict(zip(self._keys,x))
--> 194             target = self.target_func(**params)
    195             self.register(x,target)
    196         return target

<ipython-input-2-ef2efb90fddb> in xgb_evaluate(max_depth,gamma,colsample_bytree,subsample,eta,min_child_weight)
     38              }
     39     # Used around 1000 boosting rounds in the full model
---> 40     cv_result = xgb.cv(params,dtrain,num_boost_round=100,nfold=3)
     41 
     42 

~\anaconda3\envs\tf-gpu\lib\site-packages\xgboost\training.py in cv(params,num_boost_round,nfold,stratified,folds,metrics,obj,feval,maximize,early_stopping_rounds,fpreproc,as_pandas,verbose_eval,show_stdv,seed,callbacks,shuffle)
    496                            evaluation_result_list=None))
    497         for fold in cvfolds:
--> 498             fold.update(i,obj)
    499         res = aggcv([f.eval(i,feval) for f in cvfolds])
    500 

~\anaconda3\envs\tf-gpu\lib\site-packages\xgboost\training.py in update(self,iteration,fobj)
    224     def update(self,fobj):
    225         """"Update the boosters for one iteration"""
--> 226         self.bst.update(self.dtrain,fobj)
    227 
    228     def eval(self,feval):

~\anaconda3\envs\tf-gpu\lib\site-packages\xgboost\core.py in update(self,fobj)
   1367             _check_call(_LIB.XGBoosterUpdateOneIter(self.handle,1368                                                     ctypes.c_int(iteration),-> 1369                                                     dtrain.handle))
   1370         else:
   1371             pred = self.predict(dtrain,output_margin=True,training=True)

~\anaconda3\envs\tf-gpu\lib\site-packages\xgboost\core.py in _check_call(ret)
    188     """
    189     if ret != 0:
--> 190         raise XGBoostError(py_str(_LIB.XGBGetLastError()))
    191 
    192 
    >
    >XGBoostError: [07:47:25] C:/Users/Administrator/workspace/xgboost-win64_release_1.1.0/src/tree/updater_gpu_hist.cu:952: Exception in gpu_hist: bad allocation: temporary_buffer::allocate: get_temporary_buffer failed
Before that,I've received 

>OSError: [WinError -529697949] Windows Error 0xe06d7363

Here's the code that I am running.

        
    import pandas as pd
    import numpy as np
    import matplotlib.pyplot as plt
    
    number_of_rows = 10000000
    
    print('reading data')
    train_file = "D:\\reading_data/training_data.csv"
    df = pd.read_csv(train_file,sep=',',nrows=number_of_rows)
    df = df.replace([np.inf,-np.inf],np.nan).dropna(axis=0)
    print('done reading data')
    
    import xgboost as xgb
    from bayes_opt import BayesianOptimization
    from sklearn.metrics import mean_squared_error
    
    from sklearn.model_selection import train_test_split
    
    X_train,X_test,y_train,y_test = train_test_split(df.drop(['Symbol','new_date','old_dep_var'],axis=1),df['old_dep_var'],test_size=0.25)
    del(df)
    dtrain = xgb.DMatrix(X_train,label=y_train)
    del(X_train)
    dtest = xgb.DMatrix(X_test)
    del(X_test)
    
    def xgb_evaluate(max_depth,min_child_weight):
        params = {'objective': 'reg:squarederror','eval_metric': 'rmse','max_depth': int(max_depth),'subsample': 0.8,'eta': 0.1,'gamma': gamma,'colsample_bytree': colsample_bytree,'tree_method':'gpu_hist','min_child_weight': 100
                 }
        cv_result = xgb.cv(params,nfold=3)
        
        
        return -1.0 * cv_result['test-rmse-mean'].iloc[-1]
    
    xgb_bo = BayesianOptimization(xgb_evaluate,{'max_depth': (2,10),'gamma': (0,'colsample_bytree': (0.1,0.9),'subsample': (0.2,0.95),'eta': (0.05,0.5),'min_child_weight': (10,10000)
                                                })
    xgb_bo.maximize(init_points=10,acq='ei')
    print(xgb_bo.max)

我尝试过:1)重新安装贝叶斯优化,2)创建/使用新的conda环境,3)安装并运行Outbyte PC Repair来更新所有驱动程序,修复所有损坏的系统文件并消除恶意软件。

我不知道还能尝试什么。任何帮助将不胜感激。

版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。

相关推荐


依赖报错 idea导入项目后依赖报错,解决方案:https://blog.csdn.net/weixin_42420249/article/details/81191861 依赖版本报错:更换其他版本 无法下载依赖可参考:https://blog.csdn.net/weixin_42628809/a
错误1:代码生成器依赖和mybatis依赖冲突 启动项目时报错如下 2021-12-03 13:33:33.927 ERROR 7228 [ main] o.s.b.d.LoggingFailureAnalysisReporter : *************************** APPL
错误1:gradle项目控制台输出为乱码 # 解决方案:https://blog.csdn.net/weixin_43501566/article/details/112482302 # 在gradle-wrapper.properties 添加以下内容 org.gradle.jvmargs=-Df
错误还原:在查询的过程中,传入的workType为0时,该条件不起作用 &lt;select id=&quot;xxx&quot;&gt; SELECT di.id, di.name, di.work_type, di.updated... &lt;where&gt; &lt;if test=&qu
报错如下,gcc版本太低 ^ server.c:5346:31: 错误:‘struct redisServer’没有名为‘server_cpulist’的成员 redisSetCpuAffinity(server.server_cpulist); ^ server.c: 在函数‘hasActiveC
解决方案1 1、改项目中.idea/workspace.xml配置文件,增加dynamic.classpath参数 2、搜索PropertiesComponent,添加如下 &lt;property name=&quot;dynamic.classpath&quot; value=&quot;tru
删除根组件app.vue中的默认代码后报错:Module Error (from ./node_modules/eslint-loader/index.js): 解决方案:关闭ESlint代码检测,在项目根目录创建vue.config.js,在文件中添加 module.exports = { lin
查看spark默认的python版本 [root@master day27]# pyspark /home/software/spark-2.3.4-bin-hadoop2.7/conf/spark-env.sh: line 2: /usr/local/hadoop/bin/hadoop: No s
使用本地python环境可以成功执行 import pandas as pd import matplotlib.pyplot as plt # 设置字体 plt.rcParams[&#39;font.sans-serif&#39;] = [&#39;SimHei&#39;] # 能正确显示负号 p
错误1:Request method ‘DELETE‘ not supported 错误还原:controller层有一个接口,访问该接口时报错:Request method ‘DELETE‘ not supported 错误原因:没有接收到前端传入的参数,修改为如下 参考 错误2:cannot r
错误1:启动docker镜像时报错:Error response from daemon: driver failed programming external connectivity on endpoint quirky_allen 解决方法:重启docker -&gt; systemctl r
错误1:private field ‘xxx‘ is never assigned 按Altʾnter快捷键,选择第2项 参考:https://blog.csdn.net/shi_hong_fei_hei/article/details/88814070 错误2:启动时报错,不能找到主启动类 #
报错如下,通过源不能下载,最后警告pip需升级版本 Requirement already satisfied: pip in c:\users\ychen\appdata\local\programs\python\python310\lib\site-packages (22.0.4) Coll
错误1:maven打包报错 错误还原:使用maven打包项目时报错如下 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-resources-plugin:3.2.0:resources (default-resources)
错误1:服务调用时报错 服务消费者模块assess通过openFeign调用服务提供者模块hires 如下为服务提供者模块hires的控制层接口 @RestController @RequestMapping(&quot;/hires&quot;) public class FeignControl
错误1:运行项目后报如下错误 解决方案 报错2:Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project sb 解决方案:在pom.
参考 错误原因 过滤器或拦截器在生效时,redisTemplate还没有注入 解决方案:在注入容器时就生效 @Component //项目运行时就注入Spring容器 public class RedisBean { @Resource private RedisTemplate&lt;String
使用vite构建项目报错 C:\Users\ychen\work&gt;npm init @vitejs/app @vitejs/create-app is deprecated, use npm init vite instead C:\Users\ychen\AppData\Local\npm-