在Apache IGNITE中获取间歇性的长期运行查询

分类：编程问答

如何解决在Apache IGNITE中获取间歇性的长期运行查询

在IGNITE中接收间歇性LRQ，同一查询整天运行良好。但是有时会收到LRQ，这会导致响应时间急剧增加并影响平台。

我们正在使用Ignite版本2.7.5，并且我们在每个节点中使用具有256个分区的分区，并且获得了两个以UDP多播模式相互通信的节点。

RAM为210GB，启用了持久模式，其他配置如下：

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd">
    <!-- Enable annotation-driven caching. -->


    <bean name="noOpFailureHandler" class="org.apache.ignite.failure.NoOpFailureHandler"/>
    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
        <property name="peerClassLoadingEnabled" value="true"/>
        <property name="igniteInstanceName" value="PincodeGrid"/>
        <property name="clientMode" value="false"/>
        <property name="failureDetectionTimeout" value="80000"/>
        <property name="clientFailureDetectionTimeout" value="120000"/>
        <property name="systemWorkerBlockedTimeout" value="30000" />
        <property name="longQueryWarningTimeout" value="3000"/>
        <property name="failureHandler" ref="noOpFailureHandler"/>
        <property name="metricsLogFrequency" value="#{600 * 10 * 1000}"/>
        <property name="rebalanceThreadPoolSize" value="16"/>
        <property name="dataStorageConfiguration">
            <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
                <!-- Redefining the default region's settings -->
                <property name="pageSize" value="#{4 * 1024}"/>
                <!--<property name="writeThrottlingEnabled" value="true"/>-->
                <property name="defaultDataRegionConfiguration">
                    <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                        <property name="persistenceEnabled" value="true"/>
                        <property name="initialSize" value="#{105L * 1024 * 1024 * 1024}"/>
                        <property name="name" value="Default_Region"/>
                        <!--Setting the size of the default region to 4GB. -->
                        <property name="maxSize" value="#{120L * 1024 * 1024 * 1024}"/>
                        <property name="checkpointPageBufferSize"
                                  value="#{4096L * 1024 * 1024}"/>
                        <!--<property name="pageEvictionMode" value="RANDOM_2_LRU"/>-->
                    </bean>
                </property>
                <property name="walPath" value="/wal/pincode"/>
                <property name="walArchivePath" value="/wal/pincode/archive"/>
                <property name="storagePath" value="/ignite/persistence"/>
                <property name="checkpointFrequency" value="180000"/>
                <property name="checkpointThreads" value="8"/>
                <property name="walMode" value="BACKGROUND"/>
                <property name="walSegmentSize" value="#{1L * 1024 * 1024 * 1024}"/>
                <!--<property name="authenticationEnabled" value="true"/>-->
            </bean>
        </property>

        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
                        <property name="multicastGroup" value="224.0.0.180"/>
                        <property name="multicastPort" value="47514"/>
                    </bean>
                </property>

            </bean>
        </property>
        <property name="communicationSpi">
            <bean class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
                <property name="messageQueueLimit" value="2048"/>
                <property name="socketWriteTimeout" value="10000"/>
                <property name="connectionsPerNode" value="10"/>
                <property name="usePairedConnections" value="true"/>
                <property name="socketReceiveBuffer" value="#{64L * 1024}"/>
            </bean>
        </property>
    </bean>


</beans>

这是正在成为LRQ的查询。

SPS__Z1.PRIORITY __C0_0,LOGISTIC__Z2.CODFLAG __C0_1,LOGISTIC__Z2.CODLIMIT __C0_2,LOGISTIC__Z2.PREPAIDLIMIT __C0_3,LOGISTIC__Z2.SLAVEID __C0_4,STOCKROOM__Z0.ORDERCUTOFFTIMESDD __C0_5,STOCKROOM__Z0.ORDERCUTOFFTIMEED __C0_6,STOCKROOM__Z0.ORDERCUTOFFTIMEHD __C0_7,LOGISTIC__Z2.LOGISTICSID __C0_8,LOGISTIC__Z2.LOGISTICPRIORITY __C0_9
FROM "SellerPincodeServiceabilityCache".SELLERPINCODESERVICEABILITY SPS__Z1 
 INNER JOIN "LogisticsServiceabilityCache".LOGISTICSSERVICEABILITY LOGISTIC__Z2 
 ON TRUE 
 INNER JOIN "StockRoomLocationCache".STOCKROOMLOCATIONS STOCKROOM__Z0 
 ON TRUE
WHERE (LOGISTIC__Z2.PREPAIDLIMIT >= ?3) AND ((LOGISTIC__Z2.ISLOGISTICACTIVE = 'Y') AND ((LOGISTIC__Z2.ISFRAGILE = ?8) AND ((LOGISTIC__Z2.ISPRECIOUS = ?7) AND ((LOGISTIC__Z2.DELIVERYMODE = ?6) AND ((LOGISTIC__Z2.TRANSPORTMODE = ?5) AND ((LOGISTIC__Z2.DESTNPINCODE = ?1) AND ((STOCKROOM__Z0.ACTIVE = 'Y') AND ((STOCKROOM__Z0.SELLERID = ?2) AND ((STOCKROOM__Z0.SLAVEID = LOGISTIC__Z2.SLAVEID) AND ((SPS__Z1.SLAVEID = LOGISTIC__Z2.SLAVEID) AND ((SPS__Z1.SHIPMENTTYPE = ?4) AND ((SPS__Z1.DELIVERYTYPE = ?6) AND ((SPS__Z1.PINCODE = ?1) AND (SPS__Z1.SELLERID = ?2))))))))))))))
ORDER BY 1,10',plan=
SELECT
    SPS__Z1.PRIORITY AS __C0_0,LOGISTIC__Z2.CODFLAG AS __C0_1,LOGISTIC__Z2.CODLIMIT AS __C0_2,LOGISTIC__Z2.PREPAIDLIMIT AS __C0_3,LOGISTIC__Z2.SLAVEID AS __C0_4,STOCKROOM__Z0.ORDERCUTOFFTIMESDD AS __C0_5,STOCKROOM__Z0.ORDERCUTOFFTIMEED AS __C0_6,STOCKROOM__Z0.ORDERCUTOFFTIMEHD AS __C0_7,LOGISTIC__Z2.LOGISTICSID AS __C0_8,LOGISTIC__Z2.LOGISTICPRIORITY AS __C0_9
FROM "SellerPincodeServiceabilityCache".SELLERPINCODESERVICEABILITY SPS__Z1
    /* "SellerPincodeServiceabilityCache".SELLER_PINCODE_SERVICE_INDX: SHIPMENTTYPE = ?4
        AND DELIVERYTYPE = ?6
        AND PINCODE = ?1
        AND SELLERID = ?2
     */
    /* WHERE (SPS__Z1.SELLERID = ?2)
        AND ((SPS__Z1.PINCODE = ?1)
        AND ((SPS__Z1.SHIPMENTTYPE = ?4)
        AND (SPS__Z1.DELIVERYTYPE = ?6)))
    */
INNER JOIN "LogisticsServiceabilityCache".LOGISTICSSERVICEABILITY LOGISTIC__Z2
    /* "LogisticsServiceabilityCache".LOGISTICS_SERVICEABILITY_INDX: PREPAIDLIMIT >= ?3
        AND ISLOGISTICACTIVE = 'Y'
        AND ISFRAGILE = ?8
        AND ISPRECIOUS = ?7
        AND DELIVERYMODE = ?6
        AND TRANSPORTMODE = ?5
        AND DESTNPINCODE = ?1
        AND SLAVEID = SPS__Z1.SLAVEID
     */
    ON 1=1
    /* WHERE (SPS__Z1.SLAVEID = LOGISTIC__Z2.SLAVEID)
        AND ((LOGISTIC__Z2.DESTNPINCODE = ?1)
        AND ((LOGISTIC__Z2.TRANSPORTMODE = ?5)
        AND ((LOGISTIC__Z2.DELIVERYMODE = ?6)
        AND ((LOGISTIC__Z2.ISPRECIOUS = ?7)
        AND ((LOGISTIC__Z2.ISFRAGILE = ?8)
        AND ((LOGISTIC__Z2.PREPAIDLIMIT >= ?3)
        AND (LOGISTIC__Z2.ISLOGISTICACTIVE = 'Y')))))))
    */
INNER JOIN "StockRoomLocationCache".STOCKROOMLOCATIONS STOCKROOM__Z0
    /* "StockRoomLocationCache".STOCKROOMLOCATIONS_SLAVEID_IDX: SLAVEID = LOGISTIC__Z2.SLAVEID */
    ON 1=1
WHERE (LOGISTIC__Z2.PREPAIDLIMIT >= ?3)
    AND ((LOGISTIC__Z2.ISLOGISTICACTIVE = 'Y')
    AND ((LOGISTIC__Z2.ISFRAGILE = ?8)
    AND ((LOGISTIC__Z2.ISPRECIOUS = ?7)
    AND ((LOGISTIC__Z2.DELIVERYMODE = ?6)
    AND ((LOGISTIC__Z2.TRANSPORTMODE = ?5)
    AND ((LOGISTIC__Z2.DESTNPINCODE = ?1)
    AND ((STOCKROOM__Z0.ACTIVE = 'Y')
    AND ((STOCKROOM__Z0.SELLERID = ?2)
    AND ((STOCKROOM__Z0.SLAVEID = LOGISTIC__Z2.SLAVEID)
    AND ((SPS__Z1.SLAVEID = LOGISTIC__Z2.SLAVEID)
    AND ((SPS__Z1.SHIPMENTTYPE = ?4)
    AND ((SPS__Z1.DELIVERYTYPE = ?6)
    AND ((SPS__Z1.PINCODE = ?1)
    AND (SPS__Z1.SELLERID = ?2))))))))))))))
ORDER BY 1,10,parameters=[533233,125112,2480.0,TSHIP,SUR,HD,N,N]] ```

解决方法

也许这个查询确实很慢？您是否检查了它通常返回多少行以及花费多长时间？也许您有一些离群值sellerId，它会运行更长的时间？

版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 dio@foxmail.com 举报，一经查实，本站将立刻删除。

相关推荐

导入项目后报错问题
依赖报错 idea导入项目后依赖报错，解决方案：https://blog.csdn.net/weixin_42420249/article/details/81191861 依赖版本报错：更换其他版本 无法下载依赖可参考：https://blog.csdn.net/weixin_42628809/a
idea不能识别yaml文件
使用mybatis plus常见错误
错误1：代码生成器依赖和mybatis依赖冲突 启动项目时报错如下 2021-12-03 13:33:33.927 ERROR 7228 [ main] o.s.b.d.LoggingFailureAnalysisReporter : *************************** APPL
gradle常见问题与错误
错误1：gradle项目控制台输出为乱码 # 解决方案：https://blog.csdn.net/weixin_43501566/article/details/112482302 # 在gradle-wrapper.properties 添加以下内容 org.gradle.jvmargs=-Df
Mybatis Plus传入参数0不起作用
错误还原：在查询的过程中，传入的workType为0时，该条件不起作用 &lt;select id=&quot;xxx&quot;&gt; SELECT di.id, di.name, di.work_type, di.updated... &lt;where&gt; &lt;if test=&qu
linux中make编译源码包失败
报错如下，gcc版本太低 ^ server.c:5346:31: 错误：‘struct redisServer’没有名为‘server_cpulist’的成员 redisSetCpuAffinity(server.server_cpulist); ^ server.c: 在函数‘hasActiveC
微服务启动错误：Command line is too long. Shorten command line for
解决方案1 1、改项目中.idea/workspace.xml配置文件，增加dynamic.classpath参数 2、搜索PropertiesComponent，添加如下 &lt;property name=&quot;dynamic.classpath&quot; value=&quot;tru
vue常见错误
删除根组件app.vue中的默认代码后报错：Module Error (from ./node_modules/eslint-loader/index.js): 解决方案：关闭ESlint代码检测，在项目根目录创建vue.config.js，在文件中添加 module.exports = { lin
spark常见错误
查看spark默认的python版本 [root@master day27]# pyspark /home/software/spark-2.3.4-bin-hadoop2.7/conf/spark-env.sh: line 2: /usr/local/hadoop/bin/hadoop: No s
matplotlib报错：AttributeError: module 'backend_interagg' has no attribute 'FigureCanvas'. Did you mean: 'FigureCanvasAgg'?
使用本地python环境可以成功执行 import pandas as pd import matplotlib.pyplot as plt # 设置字体 plt.rcParams[&#39;font.sans-serif&#39;] = [&#39;SimHei&#39;] # 能正确显示负号 p
gitlab登录失败，报错：This challenge page was accidentally cached by an intermediary and is no longer available.
设置时间 控制面板
后端开发常见错误
错误1：Request method ‘DELETE‘ not supported 错误还原：controller层有一个接口，访问该接口时报错：Request method ‘DELETE‘ not supported 错误原因：没有接收到前端传入的参数，修改为如下 参考 错误2：cannot r
docker常见错误
错误1：启动docker镜像时报错：Error response from daemon: driver failed programming external connectivity on endpoint quirky_allen 解决方法：重启docker -&gt; systemctl r
idea常见错误
错误1：private field ‘xxx‘ is never assigned 按Altʾnter快捷键，选择第2项 参考：https://blog.csdn.net/shi_hong_fei_hei/article/details/88814070 错误2：启动时报错，不能找到主启动类 #
pip安装依赖失败
报错如下，通过源不能下载，最后警告pip需升级版本 Requirement already satisfied: pip in c:\users\ychen\appdata\local\programs\python\python310\lib\site-packages (22.0.4) Coll
maven常见错误
错误1：maven打包报错 错误还原：使用maven打包项目时报错如下 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-resources-plugin:3.2.0:resources (default-resources)
cloud项目中常见错误
错误1：服务调用时报错 服务消费者模块assess通过openFeign调用服务提供者模块hires 如下为服务提供者模块hires的控制层接口 @RestController @RequestMapping(&quot;/hires&quot;) public class FeignControl
springboot常见错误
错误1：运行项目后报如下错误 解决方案 报错2：Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project sb 解决方案：在pom.
springmvc拦截器中使用redisTemplate报空指针异常
参考 错误原因 过滤器或拦截器在生效时，redisTemplate还没有注入 解决方案：在注入容器时就生效 @Component //项目运行时就注入Spring容器 public class RedisBean { @Resource private RedisTemplate&lt;String
vite报错：npm ERR! command C:WINDOWSsystem32cmd.exe /d /s /c C:UsersychenAppDataLocalTempnpx-dde1c848.cmd
使用vite构建项目报错 C:\Users\ychen\work&gt;npm init @vitejs/app @vitejs/create-app is deprecated, use npm init vite instead C:\Users\ychen\AppData\Local\npm-
pythonJavaScriptjavaHTMLPHPreactjsC#AndroidCSSNode.jssqlrpython-3.xMysqLjQueryc++pandasFlutterangularIOSdjangolinuxswifttypescript路由器JSON路由器设置无线路由器h3c华三华三路由器设置华三路由器电脑软件教程arraysdocker软件图文教程Cvue.jslaravelspring-boot