无法运行 pktgen-dpdk错误：非法指令

如何解决无法运行 pktgen-dpdk错误：非法指令

我已按照以下步骤安装和运行 pktgen-dpdk。但是我收到“非法指令”错误并且应用程序停止。

系统信息（Centos 8）

$ uname -a
Linux localhost.localdomain 4.18.0-240.22.1.el8_3.x86_64 #1 SMP Thu Apr 8 19:01:30 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

安装 DPDK

[local@localhost dpdk-stable-20.11.1]$ cd dpdk-stable-20.11.1/
[local@localhost dpdk-stable-20.11.1]$ meson build
[local@localhost dpdk-stable-20.11.1]$ ninja -C build
[local@localhost dpdk-stable-20.11.1]$  ninja -C build install
[local@localhost dpdk-stable-20.11.1]$ sudo ldconfig
# echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages

构建 PKTGEN

[local@localhost dpdk-stable-20.11.1]$ export PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig
[local@localhost pktgen-dpdk]$ cd pktgen-dpdk/
[local@localhost pktgen-dpdk]$ cat VERSION 
21.01.2
[local@localhost pktgen-dpdk]$ make 
[local@localhost pktgen-dpdk]$ sudo vi /etc/ld.so.conf.d/x86_64-linux-gnu.conf 
    edited"/usr/local/lib64"
[local@localhost pktgen-dpdk]$ export RTE_SDK=/home/local/dpdk-stable-20.11.1/
[local@localhost pktgen-dpdk]$ export RTE_TARGET=build

# yum install dnf-plugins-core
# yum config-manager --set-enabled powertools
# yum repolist
repo id                                    repo name
appstream                                  CentOS Linux 8 - AppStream
baseos                                     CentOS Linux 8 - BaseOS
extras                                     CentOS Linux 8 - Extras
powertools                                 CentOS Linux 8 - PowerTools

编辑 default.cfg 文件以匹配系统设置

[local@localhost pktgen-dpdk]$ lspci | grep Eth
01:00.0 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 2-port Gigabit Ethernet PCIe
01:00.1 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 2-port Gigabit Ethernet PCIe
02:00.0 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 2-port Gigabit Ethernet PCIe
02:00.1 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 2-port Gigabit Ethernet PCIe
04:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
04:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
05:00.0 Ethernet controller: Mellanox Technologies MT28908 Family [ConnectX-6]
05:00.1 Ethernet controller: Mellanox Technologies MT28908 Family [ConnectX-6]
43:00.0 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]
43:00.1 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]

[local@localhost pktgen-dpdk]$ vi cfg/default.cfg

    'devices': (
            #'03:00.0','05:00.0','81:00.0','84:00.0'
            '04:00.0','04:00.1'
            ),'allowlist': (
                '04:00.0','04:00.1'
                #'05:00.0',#'84:00.0',#'03:00.0','81:00.0'
                ),'map': (
                '[3:4].0','[5:6].1',#'[16:17].2',#'[18:19].3'
                ),

设置端口并将它们附加到 DPDK

[local@localhost pktgen-dpdk]$ ./tools/run.py -s default
>>> sdk '/home/local/dpdk-stable-20.11.1/',target 'build'
<module 'cfg' from 'cfg/default.cfg'>
Setup DPDK to run 'pktgen' application from cfg/default.cfg file
[sudo] password for local: 

运行默认配置

[local@localhost pktgen-dpdk]$ ./tools/run.py default
>>> sdk '/home/local/dpdk-stable-20.11.1/',target 'build'
<module 'cfg' from 'cfg/default.cfg'>
   Trying ./usr/local/bin/pktgen
sudo -E ./usr/local/bin/pktgen -l 2,3-4,5-6,16-17,18-19 -n 4 --proc-type auto --log-level 7 --file-prefix pg -a 04:00.0 -a 04:00.1 -- -v -T -P -j -m [3:4].0 -m [5:6].1 -f themes/black-yellow.theme 



Copyright (c) <2010-2020>,Intel Corporation. All rights reserved. Powered by DPDK
EAL: Detected 24 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Auto-detected process type: PRIMARY
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/pg/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL:   using IOMMU type 1 (Type 1)
EAL: Ignore mapping IO port bar(2)
EAL: Probe PCI driver: net_ixgbe (8086:10fb) device: 0000:04:00.0 (socket 0)
EAL: Ignore mapping IO port bar(2)
EAL: Probe PCI driver: net_ixgbe (8086:10fb) device: 0000:04:00.1 (socket 0)
EAL: No legacy callbacks,legacy socket not created

直接运行pktgen导致非法指令错误

[local@localhost pktgen-dpdk]$ sudo -E ./usr/local/bin/pktgen -l 2,legacy socket not created
Illegal instruction

我也用 dpdk-stable-19.11.8 尝试了相同的步骤，但它不起作用。

[Vipin Check 下面]

[local@localhost pktgen-dpdk]$ find ./ -name pktgen
./Builddir/app/pktgen
./usr/local/bin/pktgen

rte_memcpy 发送非法指令错误 强调文字 /* 获取配置结构的干净副本 */ │ rte_memcpy(&conf,&default_port_conf,sizeof(struct rte_eth_conf));

Thread 1 "pktgen" received signal SIGILL,Illegal instruction.
0x000000000045a51e in _mm256_loadu_si256 (__P=0x6b5cc0 <default_port_conf>) at /usr/lib/gcc/x86_64-redhat-linux/8/include/avxintrin.h:922
(gdb) up
#1  rte_mov32 (src=0x6b5cc0 <default_port_conf> "",dst=0x7fffffffc2f0 "\300\355\377\377\377\377\377\377\214\312\001\367\377\177") at /usr/local/include/rte_memcpy.h:319
#2  rte_memcpy_generic (n=3064,src=0x6b5cc0 <default_port_conf>,dst=0x7fffffffc2f0) at /usr/local/include/rte_memcpy.h:461
#3  rte_memcpy (n=3080,dst=0x7fffffffc2f0) at /usr/local/include/rte_memcpy.h:874
#4  pktgen_config_ports () at ../app/pktgen-port-cfg.c:311

lscpu 输出：

[local@localhost pktgen-dpdk]$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit,64-bit
Byte Order:          Little Endian
CPU(s):              24
On-line CPU(s) list: 0-23
Thread(s) per core:  2
Core(s) per socket:  6
Socket(s):           2
NUMA node(s):        2
Vendor ID:           GenuineIntel
CPU family:          6
Model:               45
Model name:          Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
Stepping:            7
CPU MHz:             1199.646
CPU max MHz:         2500.0000
CPU min MHz:         1200.0000
BogoMIPS:            4000.03
Virtualization:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            15360K
NUMA node0 CPU(s):   0,2,4,6,8,10,12,14,16,18,20,22
NUMA node1 CPU(s):   1,3,5,7,9,11,13,15,17,19,21,23
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts md_clear flush_l1d

解决方法

Intel Xeon E5-2620 是 Sandy Bridge CPU，它正式支持 AVX 而不是 AVX2。

DPDK 20.11 meson build,ninja -C build 将生成带有 AVX 指令而不是 AVX2 的代码。但是（基于实时调试）PKTGEN 强制编译器添加要插入的 AVX2，从而导致非法指令。

解决方案：在第 meson.build 行编辑 22

来自

if get_option('enable-avx2') and cc.has_argument('-mavx2')
    add_project_arguments('-mavx2',language: 'c')
endif

到

if get_option('enable-avx2') and cc.has_argument('-mavx2')
    add_project_arguments('-mavx',language: 'c')
endif

,

上述更改适用于您，但并不适用于所有情况。

具有此更改的 enable-avx2 标志在支持 AVX2 的平台上仅使用 AVX 指令，而不使用 AVX2 指令。

在仅支持 AVX 的平台上运行时，我需要重新编写 meson.build 和 meson_options.txt 以启用 AVX 或 AVX2 指令。当平台支持 AVX2 时，也支持 AVX。

也许默认值是 AVX，当 enable-avx2 为 true 时，请改用 AVX2 标志。