Elasticsearch索引红色状态

分类：编程问答

如何解决Elasticsearch索引红色状态

我的集群中一切正常，今天我发现我没有来自packetbeat的日志，并且碎片的健康状况是红色的：

enter image description here

当我运行GET _cat / shards时，我会得到类似的东西：

packetbeat-7.9.3-2020.10.28-000001                         2 p STARTED      11428    3.8mb 10.13.81.12 VSELK-MASTER-02
packetbeat-7.9.3-2020.10.28-000001                         2 r STARTED      11428    3.8mb 10.13.81.13 VSELK-MASTER-03
packetbeat-7.9.3-2020.10.28-000001                         9 r STARTED      11402    3.8mb 10.13.81.12 VSELK-MASTER-02
packetbeat-7.9.3-2020.10.28-000001                         9 p STARTED      11402    3.8mb 10.13.81.21 VSELK-DATA-01
packetbeat-7.9.3-2020.10.28-000001                         4 p STARTED      11619      4mb 10.13.81.21 VSELK-DATA-01
packetbeat-7.9.3-2020.10.28-000001                         4 r STARTED      11619    3.9mb 10.13.81.22 VSELK-DATA-02
packetbeat-7.9.3-2020.10.28-000001                         5 r STARTED      11567    3.8mb 10.13.81.21 VSELK-DATA-01
packetbeat-7.9.3-2020.10.28-000001                         5 p STARTED      11567    3.9mb 10.13.81.22 VSELK-DATA-02
packetbeat-7.9.3-2020.10.28-000001                         1 r STARTED      11553    3.8mb 10.13.81.11 VSELK-MASTER-01
packetbeat-7.9.3-2020.10.28-000001                         1 p STARTED      11553    3.9mb 10.13.81.22 VSELK-DATA-02
packetbeat-7.9.3-2020.10.28-000001                         7 r UNASSIGNED                              
packetbeat-7.9.3-2020.10.28-000001                         7 p UNASSIGNED                              
packetbeat-7.9.3-2020.10.28-000001                         6 r UNASSIGNED                              
packetbeat-7.9.3-2020.10.28-000001                         6 p UNASSIGNED                              
packetbeat-7.9.3-2020.10.28-000001                         8 r STARTED      11630      4mb 10.13.81.12 VSELK-MASTER-02
packetbeat-7.9.3-2020.10.28-000001                         8 p STARTED      11630    3.9mb 10.13.81.21 VSELK-DATA-01
packetbeat-7.9.3-2020.10.28-000001                         3 p STARTED      11495      4mb 10.13.81.12 VSELK-MASTER-02
packetbeat-7.9.3-2020.10.28-000001                         3 r STARTED      11495    3.7mb 10.13.81.13 VSELK-MASTER-03
packetbeat-7.9.3-2020.10.28-000001                         0 r STARTED      11713      4mb 10.13.81.11 VSELK-MASTER-01
packetbeat-7.9.3-2020.10.28-000001                         0 p STARTED      11713      4mb 10.13.81.22 VSELK-DATA-02

当我运行时，我得到：GET / _cluster / allocation / explain 

{
  "index" : "packetbeat-7.9.2-2020.10.22-000001","shard" : 6,"primary" : true,"current_state" : "unassigned","unassigned_info" : {
    "reason" : "ALLOCATION_FAILED","at" : "2020-10-28T13:22:03.006Z","failed_allocation_attempts" : 5,"details" : """failed shard on node [RCeMt0uXQie_ax_Sp22hLw]: failed to create shard,failure java.io.IOException: failed to obtain in-memory shard lock
    at org.elasticsearch.index.IndexService.createShard(IndexService.java:489)
    at org.elasticsearch.indices.IndicesService.createShard(IndicesService.java:763)
    at org.elasticsearch.indices.IndicesService.createShard(IndicesService.java:176)
    at org.elasticsearch.indices.cluster.IndicesClusterStateService.createShard(IndicesClusterStateService.java:607)
    at org.elasticsearch.indices.cluster.IndicesClusterStateService.createOrUpdateShards(IndicesClusterStateService.java:584)
    at org.elasticsearch.indices.cluster.IndicesClusterStateService.applyClusterState(IndicesClusterStateService.java:242)
    at org.elasticsearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:504)
    at org.elasticsearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:494)
    at org.elasticsearch.cluster.service.ClusterApplierService.applyChanges(ClusterApplierService.java:471)
    at org.elasticsearch.cluster.service.ClusterApplierService.runTask(ClusterApplierService.java:418)
    at org.elasticsearch.cluster.service.ClusterApplierService$UpdateTask.run(ClusterApplierService.java:162)
    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:674)
    at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:252)
    at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:215)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
    at java.lang.Thread.run(Thread.java:832)
Caused by: [packetbeat-7.9.2-2020.10.22-000001/RRAnRZrrRZiihscJ3bymig][[packetbeat-7.9.2-2020.10.22-000001][6]] org.elasticsearch.env.ShardLockObtainFailedException: [packetbeat-7.9.2-2020.10.22-000001][6]: obtaining shard lock for [starting shard] timed out after [5000ms],lock already held for [closing shard] with age [199852ms]
    at org.elasticsearch.env.NodeEnvironment$InternalShardLock.acquire(NodeEnvironment.java:869)
    at org.elasticsearch.env.NodeEnvironment.shardLock(NodeEnvironment.java:775)
    at org.elasticsearch.index.IndexService.createShard(IndexService.java:409)
    ... 16 more
""","last_allocation_status" : "no"
  },"can_allocate" : "no","allocate_explanation" : "cannot allocate because allocation is not permitted to any of the nodes that hold an in-sync shard copy","node_allocation_decisions" : [
    {
      "node_id" : "A_nOoYrdSSOAHNQrhfveNA","node_name" : "VSELK-DATA-02","transport_address" : "10.13.81.22:9300","node_attributes" : {
        "ml.machine_memory" : "8365424640","ml.max_open_jobs" : "20","xpack.installed" : "true","data" : "cold","transform.node" : "true"
      },"node_decision" : "no","store" : {
        "found" : false
      }
    },{
      "node_id" : "RCeMt0uXQie_ax_Sp22hLw","node_name" : "VSELK-MASTER-03","transport_address" : "10.13.81.13:9300","node_attributes" : {
        "ml.machine_memory" : "8365068288","data" : "hot","store" : {
        "in_sync" : true,"allocation_id" : "nMvn4c4vQp2efQQtIeKzlg"
      },"deciders" : [
        {
          "decider" : "max_retry","decision" : "NO","explanation" : """shard has exceeded the maximum number of retries [5] on failed allocation attempts - manually call [/_cluster/reroute?retry_failed=true] to retry,[unassigned_info[[reason=ALLOCATION_FAILED],at[2020-10-28T13:22:03.006Z],failed_attempts[5],failed_nodes[[hHHRtd5HTCKJgLTBtgDbOw,RCeMt0uXQie_ax_Sp22hLw]],delayed=false,details[failed shard on node [RCeMt0uXQie_ax_Sp22hLw]: failed to create shard,lock already held for [closing shard] with age [199852ms]
    at org.elasticsearch.env.NodeEnvironment$InternalShardLock.acquire(NodeEnvironment.java:869)
    at org.elasticsearch.env.NodeEnvironment.shardLock(NodeEnvironment.java:775)
    at org.elasticsearch.index.IndexService.createShard(IndexService.java:409)
    ... 16 more
],allocation_status[deciders_no]]]"""
        }
      ]
    },{
      "node_id" : "hHHRtd5HTCKJgLTBtgDbOw","node_name" : "VSELK-MASTER-01","transport_address" : "10.13.81.11:9300","transform.node" : "true","ml.max_open_jobs" : "20"
      },"allocation_id" : "ByqJGtQSQT-p8dCCfk3VlA"
      },{
      "node_id" : "k_SgmMDMRfGi-IFLbI-cRw","node_name" : "VSELK-MASTER-02","transport_address" : "10.13.81.12:9300","node_attributes" : {
        "ml.machine_memory" : "8365056000",{
      "node_id" : "r4V_KqZDQ7mYi7AZea5eXQ","node_name" : "VSELK-DATA-01","transport_address" : "10.13.81.21:9300","data" : "warm","store" : {
        "found" : false
      }
    }
  ]
}

有人可以告诉我这种错误的原因以及如何解决这些错误吗？ （知道我的集群中有5个节点，3个主节点和2个数据节点，并且它们都已启动）

感谢您的帮助！

解决方法

您可以按照相关的GitHub issue，特别是this的注释来解决此问题。

简而言之，您应该尝试使用以下更安全的命令

curl -XPOST'localhost：9200 / _cluster / reroute？retry_failed

版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 dio@foxmail.com 举报，一经查实，本站将立刻删除。

相关推荐

导入项目后报错问题
依赖报错 idea导入项目后依赖报错，解决方案：https://blog.csdn.net/weixin_42420249/article/details/81191861 依赖版本报错：更换其他版本 无法下载依赖可参考：https://blog.csdn.net/weixin_42628809/a
idea不能识别yaml文件
使用mybatis plus常见错误
错误1：代码生成器依赖和mybatis依赖冲突 启动项目时报错如下 2021-12-03 13:33:33.927 ERROR 7228 [ main] o.s.b.d.LoggingFailureAnalysisReporter : *************************** APPL
gradle常见问题与错误
错误1：gradle项目控制台输出为乱码 # 解决方案：https://blog.csdn.net/weixin_43501566/article/details/112482302 # 在gradle-wrapper.properties 添加以下内容 org.gradle.jvmargs=-Df
Mybatis Plus传入参数0不起作用
错误还原：在查询的过程中，传入的workType为0时，该条件不起作用 <select id="xxx"> SELECT di.id, di.name, di.work_type, di.updated... <where> <if test=&qu
linux中make编译源码包失败
报错如下，gcc版本太低 ^ server.c:5346:31: 错误：‘struct redisServer’没有名为‘server_cpulist’的成员 redisSetCpuAffinity(server.server_cpulist); ^ server.c: 在函数‘hasActiveC
微服务启动错误：Command line is too long. Shorten command line for
解决方案1 1、改项目中.idea/workspace.xml配置文件，增加dynamic.classpath参数 2、搜索PropertiesComponent，添加如下 <property name="dynamic.classpath" value="tru
vue常见错误
删除根组件app.vue中的默认代码后报错：Module Error (from ./node_modules/eslint-loader/index.js): 解决方案：关闭ESlint代码检测，在项目根目录创建vue.config.js，在文件中添加 module.exports = { lin
spark常见错误
查看spark默认的python版本 [root@master day27]# pyspark /home/software/spark-2.3.4-bin-hadoop2.7/conf/spark-env.sh: line 2: /usr/local/hadoop/bin/hadoop: No s
matplotlib报错：AttributeError: module 'backend_interagg' has no attribute 'FigureCanvas'. Did you mean: 'FigureCanvasAgg'?
使用本地python环境可以成功执行 import pandas as pd import matplotlib.pyplot as plt # 设置字体 plt.rcParams['font.sans-serif'] = ['SimHei'] # 能正确显示负号 p
gitlab登录失败，报错：This challenge page was accidentally cached by an intermediary and is no longer available.
设置时间 控制面板
后端开发常见错误
错误1：Request method ‘DELETE‘ not supported 错误还原：controller层有一个接口，访问该接口时报错：Request method ‘DELETE‘ not supported 错误原因：没有接收到前端传入的参数，修改为如下 参考 错误2：cannot r
docker常见错误
错误1：启动docker镜像时报错：Error response from daemon: driver failed programming external connectivity on endpoint quirky_allen 解决方法：重启docker -> systemctl r
idea常见错误
错误1：private field ‘xxx‘ is never assigned 按Altʾnter快捷键，选择第2项 参考：https://blog.csdn.net/shi_hong_fei_hei/article/details/88814070 错误2：启动时报错，不能找到主启动类 #
pip安装依赖失败
报错如下，通过源不能下载，最后警告pip需升级版本 Requirement already satisfied: pip in c:\users\ychen\appdata\local\programs\python\python310\lib\site-packages (22.0.4) Coll
maven常见错误
错误1：maven打包报错 错误还原：使用maven打包项目时报错如下 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-resources-plugin:3.2.0:resources (default-resources)
cloud项目中常见错误
错误1：服务调用时报错 服务消费者模块assess通过openFeign调用服务提供者模块hires 如下为服务提供者模块hires的控制层接口 @RestController @RequestMapping("/hires") public class FeignControl
springboot常见错误
错误1：运行项目后报如下错误 解决方案 报错2：Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project sb 解决方案：在pom.
springmvc拦截器中使用redisTemplate报空指针异常
参考 错误原因 过滤器或拦截器在生效时，redisTemplate还没有注入 解决方案：在注入容器时就生效 @Component //项目运行时就注入Spring容器 public class RedisBean { @Resource private RedisTemplate<String
vite报错：npm ERR! command C:WINDOWSsystem32cmd.exe /d /s /c C:UsersychenAppDataLocalTempnpx-dde1c848.cmd
使用vite构建项目报错 C:\Users\ychen\work>npm init @vitejs/app @vitejs/create-app is deprecated, use npm init vite instead C:\Users\ychen\AppData\Local\npm-
pythonJavaScriptjavaHTMLPHPreactjsC#AndroidCSSNode.jssqlrpython-3.xMysqLjQueryc++pandasFlutterangularIOSdjangolinuxswifttypescript路由器JSON路由器设置无线路由器h3c华三华三路由器设置华三路由器电脑软件教程arraysdocker软件图文教程Cvue.jslaravelspring-boot