AI导航网
微信公众号搜"智元新知"关注
微信扫一扫可直接关注哦！

mysql – Innodb：48小时后优化10mb / sec的写入速度

分类：Mariadb作者：编程之家原文地址

我在具有NVME存储(2TB)和64 GB RAM的8核服务器上运行.
磁盘速度快,1.1 GB /秒seq和70-100k IOPS全双工.

因为我在mysql 5.7上有如此可怕的性能,所以我在一个纤薄的docker容器上安装了Mariadb 10.3.8.

总的来说,我要写的表格是2 TB和10亿行.但是让我说清楚：这个速度性能发生在前几千行的空磁盘上,它与一个大表无关.

在过去的一周,白天和黑夜,我投入了大约50个小时的工作,我阅读了我能找到的每个文档页面以及各种平台上的数百个指南和问题.
我完全测试了它,你几乎可以想到任何组合.
我尝试了纯内存缓冲,纯磁盘缓冲,有和没有大日志,日志缓冲区,各种刷新方法,没有刷新,所有这些设置,你可以想到.

我测试了导入使用：
我写的mydumper,MysqL控制台,mysqlimport,加载数据infile,PHP插入,多线程PDO脚本.

我测试了有和没有索引的表,只有主索引.

我尝试使用和不使用TRANSACTIONS进行导入,尝试单行和多行INSERT.

我尝试了不同的表类型,通常是20-30列,主要包含varchars和一些日期时间.

单线程性能为3-5k行/秒,多线程(荒谬……)10-25k /秒.
cpu和disK一直处于空闲状态,iostat显示3-20mb / sec的写入性能,通常约为7mb-12mb.取决于我尝试的设置.

因此,它应该比它应该执行的速度慢大约100倍,没有任何明显的阻止它.
    那是当前的配置：

innodb_buffer_pool_size = 14G
innodb_buffer_pool_chunk_size=1G
innodb_log_buffer_size  = 32M
innodb_file_per_table   = 1
innodb_open_files       = 600
#innodb_flush_method    = O_DIRECT
innodb_flush_method     = O_DSYNC 
innodb_log_file_size    = 512M
innodb_io_capacity=800 
innodb_io_capacity_max=3000 
innodb_flush_neighbors=0
innodb_write_io_threads=8 
innodb_read_io_threads=8 
innodb_change_buffer_max_size=70
innodb_doublewrite=0 # corruption risk

想象几乎每种组合几乎都可能,我尝试了一切.

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
xvda              0.50         0.00        19.00          0         76
xvdg            102.50     13120.00         0.00      52480          0
xvdh           1381.50     19708.00     30984.00      78832     123936
xvdf              0.00         0.00         0.00          0          0
nvme0n1         222.00         0.00     10957.00          0      43828

这里唯一相关的磁盘是nvme0n1,您可以使用多线程插入查看当前的写入性能.

| InnoDB |      | 
=====================================
2018-07-22 06:42:31 0x7fe7341c9700 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 39 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 4883 srv_active, 0 srv_shutdown, 1061 srv_idle
srv_master_thread log flush and writes: 5944
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 2215493
OS WAIT ARRAY INFO: signal count 3968391
RW-shared spins 0, rounds 6388873, OS waits 1674234
RW-excl spins 0, rounds 34932124, OS waits 431565
RW-sx spins 13782, rounds 169207, OS waits 2879
Spin rounds per wait: 6388873.00 RW-shared, 34932124.00 RW-excl, 12.28 RW-sx
------------
TRANSACTIONS
------------
Trx id counter 1036891
Purge done for trx's n:o < 1036891 undo n:o < 0 state: running but idle
History list length 53
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 422106005755512, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422106005754696, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
--------
FILE I/O
--------
I/O thread 0 state: waiting for completed aio requests (insert buffer thread)
I/O thread 1 state: waiting for completed aio requests (log thread)
I/O thread 2 state: waiting for completed aio requests (read thread)
I/O thread 3 state: waiting for completed aio requests (read thread)
I/O thread 4 state: waiting for completed aio requests (read thread)
I/O thread 5 state: waiting for completed aio requests (read thread)
I/O thread 6 state: waiting for completed aio requests (read thread)
I/O thread 7 state: waiting for completed aio requests (read thread)
I/O thread 8 state: waiting for completed aio requests (read thread)
I/O thread 9 state: waiting for completed aio requests (read thread)
I/O thread 10 state: waiting for completed aio requests (write thread)
I/O thread 11 state: waiting for completed aio requests (write thread)
I/O thread 12 state: waiting for completed aio requests (write thread)
I/O thread 13 state: waiting for completed aio requests (write thread)
I/O thread 14 state: waiting for completed aio requests (write thread)
I/O thread 15 state: waiting for completed aio requests (write thread)
I/O thread 16 state: waiting for completed aio requests (write thread)
I/O thread 17 state: waiting for completed aio requests (write thread)
Pending normal aio reads: [0, 0, 0, 0, 0, 0, 0, 0] , aio writes: [0, 0, 0, 0, 0, 0, 0, 0] ,
 ibuf aio reads:, log I/O's:, sync I/O's:
Pending flushes (fsync) log: 0; buffer pool: 0
263597 OS file reads, 2045302 OS file writes, 161983 OS fsyncs
0.00 reads/s, 0 avg bytes/read, 282.56 writes/s, 28.08 fsyncs/s
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf: size 1, free list len 1134, seg size 1136, 0 merges
merged operations:
 insert 0, delete mark 0, delete 0
discarded operations:
 insert 0, delete mark 0, delete 0
Hash table size 4425293, node heap has 1 buffer(s)
Hash table size 4425293, node heap has 4113 buffer(s)
Hash table size 4425293, node heap has 1 buffer(s)
Hash table size 4425293, node heap has 41738 buffer(s)
Hash table size 4425293, node heap has 1 buffer(s)
Hash table size 4425293, node heap has 9223 buffer(s)
Hash table size 4425293, node heap has 1 buffer(s)
Hash table size 4425293, node heap has 5900 buffer(s)
7968.05 hash searches/s, 13692.47 non-hash searches/s
---
LOG
---
Log sequence number 185044128669
Log flushed up to   185044077028
Pages flushed up to 185040571130
Last checkpoint at  185017989085
0 pending log flushes, 0 pending chkp writes
37148 log I/O's done, 6.79 log I/O's/second
----------------------
BUFFER POOL AND MEMORY
----------------------
Total large memory allocated 17649631232
Dictionary memory allocated 419728
Buffer pool size   1048576
Free buffers       8032
Database pages     979545
Old database pages 361426
Modified db pages  275
Percent of dirty pages(LRU & free pages): 0.028
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 817, not young 38547
0.00 youngs/s, 0.00 non-youngs/s
Pages read 263539, created 1493647, written 2008209
0.00 reads/s, 204.33 creates/s, 275.76 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 979545, unzip_LRU len: 0
I/O sum[116336]:cur[0], unzip sum[0]:cur[0]
----------------------
INDIVIDUAL BUFFER POOL INFO
----------------------
---BUFFER POOL 0
Buffer pool size   131072
Free buffers       1025
Database pages     122400
Old database pages 45162
Modified db pages  19
Percent of dirty pages(LRU & free pages): 0.015
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 70, not young 6961
0.00 youngs/s, 0.00 non-youngs/s
Pages read 33027, created 186873, written 272369
0.00 reads/s, 19.72 creates/s, 32.72 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 122400, unzip_LRU len: 0
I/O sum[14542]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 1
Buffer pool size   131072
Free buffers       1007
Database pages     122444
Old database pages 45179
Modified db pages  26
Percent of dirty pages(LRU & free pages): 0.021
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 74, not young 9194
0.00 youngs/s, 0.00 non-youngs/s
Pages read 32952, created 187093, written 242787
0.00 reads/s, 22.49 creates/s, 28.87 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 122444, unzip_LRU len: 0
I/O sum[14542]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 2
Buffer pool size   131072
Free buffers       1023
Database pages     122421
Old database pages 45170
Modified db pages  14
Percent of dirty pages(LRU & free pages): 0.011
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 45, not young 177
0.00 youngs/s, 0.00 non-youngs/s
Pages read 32936, created 186494, written 236773
0.00 reads/s, 23.97 creates/s, 30.69 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 122421, unzip_LRU len: 0
I/O sum[14542]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 3
Buffer pool size   131072
Free buffers       994
Database pages     122454
Old database pages 45182
Modified db pages  39
Percent of dirty pages(LRU & free pages): 0.032
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 173, not young 254
0.00 youngs/s, 0.00 non-youngs/s
Pages read 33012, created 185887, written 258491
0.00 reads/s, 26.49 creates/s, 39.49 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 122454, unzip_LRU len: 0
I/O sum[14542]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 4
Buffer pool size   131072
Free buffers       1006
Database pages     122449
Old database pages 45180
Modified db pages  46
Percent of dirty pages(LRU & free pages): 0.037
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 120, not young 12679
0.00 youngs/s, 0.00 non-youngs/s
Pages read 32937, created 187265, written 249442
0.00 reads/s, 29.79 creates/s, 39.18 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 122449, unzip_LRU len: 0
I/O sum[14542]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 5
Buffer pool size   131072
Free buffers       1012
Database pages     122452
Old database pages 45182
Modified db pages  25
Percent of dirty pages(LRU & free pages): 0.020
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 202, not young 5093
0.00 youngs/s, 0.00 non-youngs/s
Pages read 32867, created 187025, written 253352
0.00 reads/s, 30.69 creates/s, 40.20 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 122452, unzip_LRU len: 0
I/O sum[14542]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 6
Buffer pool size   131072
Free buffers       1021
Database pages     122423
Old database pages 45171
Modified db pages  13
Percent of dirty pages(LRU & free pages): 0.011
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 64, not young 2573
0.00 youngs/s, 0.00 non-youngs/s
Pages read 32909, created 187403, written 243809
0.00 reads/s, 25.82 creates/s, 31.64 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 122423, unzip_LRU len: 0
I/O sum[14542]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 7
Buffer pool size   131072
Free buffers       944
Database pages     122502
Old database pages 45200
Modified db pages  93
Percent of dirty pages(LRU & free pages): 0.075
Max dirty pages percent: 75.000
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 69, not young 1616
0.00 youngs/s, 0.00 non-youngs/s
Pages read 32899, created 185607, written 251186
0.00 reads/s, 25.36 creates/s, 32.97 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 122502, unzip_LRU len: 0
I/O sum[14542]:cur[0], unzip sum[0]:cur[0]
--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
0 read views open inside InnoDB
Process ID=21173, Main thread ID=140612953028352, state: sleeping
Number of rows inserted 68789564, updated 0, deleted 0, read 0
10659.42 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s
Number of system rows inserted 0, updated 0, deleted 0, read 0
0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s
----------------------------
END OF INNODB MONITOR OUTPUT

MysqL的行为就好像它的内部IO限制大约为25MB /秒.

我知道我没有添加基准测试,我现在已经连续工作了20个小时,手头没有结果.请相信我磁盘非常快.
分配给MysqL的内存不起作用,我从1 GB变为50 ..没有区别.

我已经花了半个星期的时间和16个小时的班次让这个跑步,我的斗智尽头.
我能想到的最后一件事是购买像甲骨文这样的商业数据库,但这是另一个经历的噩梦.

只有一次我有可接受的速度：
导入IBD文件“RAW”然后使用IMPORT TABLESPACE.
但是,这需要在源数据库上使用数小时的LOCK才能获得二进制快照,然后将其复制(通过网络获取时间),然后再将其全部导入.
IMPORT TABLESPACE性能本身还可以,大约600 MB /秒
总的来说这是最快的但是对我的事业来说是不可用的.

表格如下：

    CREATE TABLE `eval` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `intern_id` varchar(256) COLLATE utf8_bin DEFAULT NULL,
  `first_name` varchar(64) COLLATE utf8_bin DEFAULT NULL,
  `last_name` varchar(64) COLLATE utf8_bin DEFAULT NULL,
  `middle_name` varchar(64) COLLATE utf8_bin DEFAULT NULL,
  `location` varchar(196) COLLATE utf8_bin DEFAULT NULL,
  `i` varchar(128) COLLATE utf8_bin DEFAULT NULL,
  `e` varchar(256) COLLATE utf8_bin DEFAULT NULL,
  `country_code` varchar(4) COLLATE utf8_bin DEFAULT NULL,
  `country_name` varchar(64) COLLATE utf8_bin DEFAULT NULL,
  `state_name` varchar(64) COLLATE utf8_bin DEFAULT NULL,
  `city_name` varchar(64) COLLATE utf8_bin DEFAULT NULL,
  `education` varchar(256) COLLATE utf8_bin DEFAULT NULL,
  `num_c` smallint(6) DEFAULT NULL,
  `num_j` smallint(6) DEFAULT NULL,
  `j_t` varchar(256) COLLATE utf8_bin DEFAULT NULL,
  `c_name` varchar(256) COLLATE utf8_bin DEFAULT NULL,
  `e_a` varchar(256) COLLATE utf8_bin DEFAULT NULL,
  `flag_existent` tinyint(4) DEFAULT NULL COMMENT '1/0',
  `public_p_u` varchar(256) COLLATE utf8_bin DEFAULT NULL,
  `c_intern_id` varchar(256) COLLATE utf8_bin DEFAULT NULL,
  `unmatched_facts` varchar(2048) COLLATE utf8_bin DEFAULT NULL,
  `dt_snapshot` datetime DEFAULT NULL,
  `change_small` tinyint(4) DEFAULT NULL,
  `change_significant` tinyint(4) DEFAULT NULL,
  `j_t_auth` varchar(256) COLLATE utf8_bin DEFAULT NULL COMMENT 'sure about it',
  `c_name_auth` varchar(256) COLLATE utf8_bin DEFAULT NULL COMMENT 'sure about it',
  `c_intern_id_guess` varchar(256) COLLATE utf8_bin DEFAULT NULL,
  `ut_created` int(11) DEFAULT NULL,
  `reserve_int_2` int(11) DEFAULT NULL,
  `reserve_vc1` varchar(256) COLLATE utf8_bin DEFAULT NULL,
  `reserve_vc2` varchar(256) COLLATE utf8_bin DEFAULT NULL,
  `reserve_vc_3` varchar(256) COLLATE utf8_bin DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `intern_id` (`intern_id`),
  KEY `location` (`location`),
  KEY `country_name` (`country_name`),
  KEY `country_name_location_notnull` (`country_name`(1),`location`(1)),
  KEY `location_country_name` (`location`,`country_name`(1)),
  KEY `location_null_country_name` (`location`(1),`country_name`),
  KEY `dt_snapshot` (`dt_snapshot`),
  KEY `state_name` (`state_name`),
  KEY `city_name` (`city_name`),
  KEY `c_name` (`c_name`),
  KEY `c_name_auth` (`c_name_auth`),
  KEY `j_t` (`j_t`)
) ENGINE=InnoDB AUTO_INCREMENT=19883676 DEFAULT CHARSET=utf8 COLLATE=utf8_bin |

我测试了没有索引的导入,只测试了主键和完整索引.
基本上速度总是一样的.
唯一的区别是磁盘IO速度随着索引的增加而增加,但行/秒保持不变.

更新
使用“IMPORT TABLESPACE”导入表格工作得很快(可怕的方法,需要删除IBD文件,任何中断都会破坏表格,我必须锁定主要数据库一小时)
该方法允许大约350k行/秒.

现在我正在使用需要全面扫描的简单SELECTS来使用服务器上加载的数据库. (count(*)其中xxx不为null)
数据库仅以100mb /秒的速度完成全面扫描！
有一个瓶颈限制了90％的速度.

更新：
我尝试通过创建5个数据库会话绕过QUERY性能瓶颈,在同一个数据库表上执行SELECT查询,但是用ID分隔查询. 1-10000000,1000000-2000000,300000-4000000等
每次会话都会使磁盘负载增加100mb /秒.
因此,使用5个并发选择查询/会话的数据库比使用一个查询快5倍.
但实际上这应该是很慢的.这意味着需要大量随机IO,因为5个线程快速访问不同位置的同一文件,所以可能的顺序IO更少.

我对WRITE有类似的问题,写入5次SAME数据库文件的速度比写1次快5倍,但速度非常慢(1-5％的速度)

1在表格中选择：

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           1.21    0.00    0.35   10.66    0.26   87.52

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
xvda              0.67         0.00        21.33          0         64
xvdg              0.00         0.00         0.00          0          0
xvdh            134.33         5.33       721.33         16       2164
xvdf              4.67         0.00        25.33          0         76
nvme0n1        7032.00    112512.00         0.00     337536          0

5在SAME表上选择不同的主键位置：

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           1.98    0.00    0.63   43.35    0.77   53.28
    Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
    xvda              0.67         0.00        22.67          0         68
    xvdg              0.00         0.00         0.00          0          0
    xvdh            111.33        13.33       598.67         40       1796
    xvdf              0.00         0.00         0.00          0          0
    nvme0n1       30051.33    480821.33         0.00    1442464          0

最新结论
看来这个问题是MysqL和MariaDB中的一个缺陷,可能是由单线程设计引起的.
似乎随着列数的增加,最大性能降低,每列导致一些延迟开销.
InnoDb / XTradb似乎不是问题.
这是我能找到的唯一解释,除了编写多线程自定义代码之外无法解决.

所有全局变量和显示状态：
https://paste.ee/p/Yk1Le

在这里整个配置文件(当前的变种,我尝试了所有我认为)

[client]
port            = 3316
socket          = /maincache/share/MysqLd.sock


[MysqLd_safe]
socket          = /maincache/share/MysqLd.sock
nice            = 0

[MysqLd]
pid-file        = /var/run/MysqLd/MysqLd.pid
socket          = /maincache/share/MysqLd.sock
port            = 3316
basedir         = /usr
datadir         = /var/lib/MysqL
tmpdir          = /instance_store/tmp
lc_messages_dir = /usr/share/MysqL
lc_messages     = en_US
skip-external-locking
bind-address            = 127.0.0.1
max_connections         = 100
connect_timeout         = 5
wait_timeout            = 600
max_allowed_packet      = 160M
thread_cache_size       = 128
sort_buffer_size        = 4M
bulk_insert_buffer_size = 16M
tmp_table_size          = 16M
max_heap_table_size     = 16M
join_buffer_size=32k
sort_buffer_size=32k
myisam_recover_options = BACKUP
key_buffer_size         = 6M
table_open_cache        = 1000
table_open_cache_instances = 8
myisam_sort_buffer_size = 16M
concurrent_insert       = 2
read_buffer_size        = 2M
read_rnd_buffer_size    = 1M
query_cache_limit               = 16M
query_cache_size                = 256M
log_error = /maincache/share/cluster/MysqL_error.log
slow_query_log_file     = /var/log/MysqL/mariadb-slow.log
long_query_time = 10

expire_logs_days        = 10
max_binlog_size         = 100M
default_storage_engine  = InnoDB
innodb_force_recovery=1
innodb_buffer_pool_size = 10G
innodb_buffer_pool_chunk_size=512M
innodb_file_per_table   = 1
innodb_open_files       = 600
innodb_flush_method     = O_DIRECT_NO_FSYNC
innodb_log_file_size    = 512M
innodb_io_capacity=5000
innodb_io_capacity_max=80000
innodb_flush_neighbors=0
innodb_write_io_threads=64
innodb_read_io_threads=64
innodb_change_buffer_max_size=70
innodb_buffer_pool_instances=128
innodb_thread_concurrency=144

[galera]

[MysqLdump]
quick
quote-names
max_allowed_packet      = 16M

[MysqL]

[isamchk]
key_buffer              = 16M

服务器有64GB的内存,但我在这个版本中将MysqL服务器限制为10GB.
无论innodb缓冲区如何,它都表现出完全没有性能差异.
服务器空闲,也在插入80-90％空闲(IO / cpu)期间,当然不会交换.

解决方法:

根据提供的可见信息为my.cnf [MysqLd]部分考虑的建议.
整个块在[MysqLd]的END处去,并删除在该部分中出现较高的任何SAME NAMED VARIABLE以避免请求冲突.

innodb_io_capacity=40000  # from 5000 to open the door for NVME speed
read_rnd_buffer_size=256K  # from 1M to reduce handler_read_rnd_next RPS
innodb_lru_scan_depth=128  # from 1024 to conserve cpu every second
innodb_adaptive_max_sleep_delay=10000 # from 150000 for 1 sec sleep delay
innodb_flushing_avg_loops=4  # from 30 for reduce the loop delay
innodb_thread_concurrency=0  # from 144 see dba.stackexhange Question 5666
max_seeks_for_key=32  # to limit optimizer to nn vs ~ 4 Billion possible
max_write_locks_count=16  # to allow RD after nn lcks vs up to 4 Billion lcks
thread_concurrency=30  # from 10 for additional conc - may be DEPR
innodb_buffer_pool_instances=8  # from 64 see REFMAN for innodb_lru_scan_depth details
innodb_log_file_size=6G  # from ~ 512M to reduce log rotation
innodb_log_buffer_size=3G  # from 16M for ~ 30 minutes buffering
query_cache_type=OFF  # from ON no need to waste cpu for mgmt
query_cache_size=0  # from ~256M to conserve RAM and cpu cycles
slow_query_log=ON  # from OFF always good to have ON

当您每秒加载10,000行时,RAM使用策略

总RAM = 64GB,允许MysqLd高达48GB(~75％)

当你加载这个高音量时,

innodb_buffer_pool_size=30G  # for 62.5% of your 48G
innodb_change_buffer_max_size=50  # for 50% to have best insert rate per second

当装载完成时,

SET GLOBAL innodb_change_buffer_max_size = 15#,15％用于日常维护要求;

并且您将在合理的正常运行时间内解决innodb缓冲池中可用的典型数据.

有关其他建议,请查看我的个人资料,网络配置文件以获取联系信息,包括与SKYPE ID联系,请联系.谢谢

版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 [email protected] 举报，一经查实，本站将立刻删除。

相关推荐

2022年-5月22日centos7网络源安装mysql
安装开始...1.打开“https://dev.mysql.com/downloadsepo/yum/”下载Mysql源      将下载好的mysql源上传linux服务器 2.yumlocalinstallmysql80*#安装源 centos7中默认安装的是mariadb数据库，如果想要安装my...
K8S Helm安装部署 MariaDB 高可用、主从集群
安装Helm3#官网下载慢#wgethttps://get.helm.sh/helm-v3.5.4-linux-amd64.tar.gzwgethttp://qiniu.dev-share.top/helm-v3.5.4-linux-amd64.tar.gztar-zxvfhelm-v3.5.4-linux-amd64.tar.gzcplinux-amd64/helm/usr/loc...
安装MariaDB后无法启动怎么解决？
通过Linux命令行启动用的指令：systemctlstartmariadb.service反馈：Failedtostartmariadb.service:Unitmariadb.servicenotfound.MariaDB简介MariaDB是MySQL的一个分支，MariaDB打算保持与MySQL的高度兼容性，确保具...
Centos7安装mariaDB
InstallingMariaDBServer10.4TodeployMariaDBCommunityServer10.4onRHEL7orCentOS7,firstdownloadandusethe mariadb_repo_setup scripttoconfiguretheMariaDBrepositoriesforYUM:$sudoyuminstallwget$wgethttps://...
MySQL存储引擎
阅读目录一什么是存储引擎二mysql支持的存储引擎三使用存储引擎一什么是存储引擎mysql中建立的库--> 文件夹库中建立的表--> 文件现实生活中我们用来存储数据的文件有不同的类型，每种文件类型对应各自不同的...
centos7 Mariadb 安装
1、安装MariaDB安装命令yum-yinstallmariadbmariadb-server安装完成MariaDB，首先启动MariaDBsystemctlstartmariadb设置开机启动systemctlenablemariadb[root@node1~]#systemctlenablemariadbCreatedsymlinkfrom/etc...
ERROR 1045 (28000): Access denied for user &#039;root&#039;@&#039;localhost&#039; (using password: N
Centos7.5 刚安装的mariadb数据库报错报错:ERROR1045(28000):Accessdeniedforuser'root'@'localhost'(usingpassword:NO)原因一:没有启动数据库解决方法:systemctlstartmariadbsystemctlenablem...
mariadb多实例安装
基于YUM安装的mariadb多实例.=================================================================1.yum安装mariadb-server包#yuminstallmariadb-server2.创建多实例对应的目录结构#mkdir/mysql/{3306,3307,3308}/{d...
centos7 mysql数据库安装和配置
一、系统环境[root@localhost~]#cat/etcedhat-releaseCentOSLinuxrelease7.6.1810(Core)二、mysql安装#yuminstallmysqlmysql-servermysql-devel安装mysql-server失败，如下图：[root@localhost~]#yuminstallmysql-se...
MariaDB
数据库的选择两大点是：开源和跨平台，满足这三点MySQL、MongoDB和MariaDB。其中MariaDB是MySQL的分支，也是它的进阶产品，未来很有可能替代MySQL。与MySQL相比较，MariaDB更强的地方在于：Maria 存储引擎PBXT存储引...