如何解决使用 logstash 将日志文件推送到弹性
我们有一个来自 Jmeter 的错误日志文件，我们需要使用 logstash 将其推送到弹性
这是日志文件，我们想从第 5 行推送数据，
2021-05-14 20:32:49,822 INFO o.a.j.u.JMeterUtils: Setting Locale to en_EN
2021-05-14 20:32:49,842 INFO o.a.j.JMeter: Loading user properties from: user.properties
2021-05-14 20:32:49,843 INFO o.a.j.JMeter: Loading system properties from: system.properties
2021-05-14 20:32:49,844 WARN o.a.j.JMeter: LogLevel: ERROR
2021-05-14 20:32:51,824 ERROR o.a.j.e.J.JSR223 PostProcessor: Http URL/API Test 1-4: ; response received: {"timestamp":"2021-05-14T14:32:51.688+0000","status":404,"error":"Not Found","message":"No message available","path":"/healths"}
2021-05-14 20:32:51,824 ERROR o.a.j.e.J.JSR223 PostProcessor: Http URL/API Test 1-7: ; response received: {"timestamp":"2021-05-14T14:32:51.689+0000",824 ERROR o.a.j.e.J.JSR223 PostProcessor: Http URL/API Test 1-20: ; response received: {"timestamp":"2021-05-14T14:32:51.850+0000",824 ERROR o.a.j.e.J.JSR223 PostProcessor: LEX 2-1: ; response received: <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Header><ns3:systemContext xmlns:ns3="urn:abc.mon.utils.v1.context"><systemId>TREX</systemId><correlationId>hgfhjghj-c62dsfsdfdb-4aff-a871-8f32088943b3</correlationId></ns3:systemContext></SOAP-ENV:Header><SOAP-ENV:Body><ns3:retrieveStopPayResponse xmlns:ns3="urn:abc.tailhead.readyforcheck.processStopPay"><retrieveGagaPayResponse recordFound="false"><checkNumber/><stopPayDate/></retrieveGagaPayResponse></ns3:retrieveStopPayResponse></SOAP-ENV:Body></SOAP-ENV:Envelope>
2021-05-14 20:32:51,824 ERROR o.a.j.e.J.JSR223 PostProcessor: Http URL/API Test 1-5: ; response received: {"timestamp":"2021-05-14T14:32:51.687+0000","path":"/healths"}
我们希望将第 4 行的数据特别是 PostProcessor:,response received: 推送到弹性。以下配置仅用于收到的响应：
这里是logstash conf
if [type] == "errors"{
grok {
match => {"message" => "%{GREEDYDATA}#*response received: %{GREEDYDATA:error_message}"}
}
}
当我们尝试推送时，我们收到错误 Grok 解析失败 _grokparsefailure
不确定我在这里遗漏了什么
解决方法
这样的事情应该对你有用：
%{TIMESTAMP_ISO8601:timestamp} %{WORD:LogType} %{GREEDYDATA:data1} PostProcessor: %{GREEDYDATA:data2} ; response received: %{GREEDYDATA:responseRecd}
示例 1：
2021-05-14 20:32:51,824 ERROR o.a.j.e.J.JSR223 PostProcessor: Http URL/API Test 1-4: ; response received: {"timestamp":"2021-05-14T14:32:51.688+0000","status":404,"error":"Not Found","message":"No message available","path":"/healths"}
输出：
{
"timestamp": [
[
"2021-05-14 20:32:51,824"
]
],"LogType": [
[
"ERROR"
]
],"data1": [
[
"o.a.j.e.J.JSR223"
]
],"data2": [
[
"Http URL/API Test 1-4:"
]
],"responseRecd": [
[
"{"timestamp":"2021-05-14T14:32:51.688+0000","path":"/healths"}"
]
]
}
示例 2：
2021-05-14 20:32:51,824 ERROR o.a.j.e.J.JSR223 PostProcessor: LEX 2-1: ; response received: <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Header><ns3:systemContext xmlns:ns3="urn:abc.mon.utils.v1.context"><systemId>TREX</systemId><correlationId>hgfhjghj-c62dsfsdfdb-4aff-a871-8f32088943b3</correlationId></ns3:systemContext></SOAP-ENV:Header><SOAP-ENV:Body><ns3:retrieveStopPayResponse xmlns:ns3="urn:abc.tailhead.readyforcheck.processStopPay"><retrieveGagaPayResponse recordFound="false"><checkNumber/><stopPayDate/></retrieveGagaPayResponse></ns3:retrieveStopPayResponse></SOAP-ENV:Body></SOAP-ENV:Envelope>
输出：
{
"timestamp": [
[
"2021-05-14 20:32:51,"data2": [
[
"LEX 2-1:"
]
],"responseRecd": [
[
"<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Header><ns3:systemContext xmlns:ns3="urn:abc.mon.utils.v1.context"><systemId>TREX</systemId><correlationId>hgfhjghj-c62dsfsdfdb-4aff-a871-8f32088943b3</correlationId></ns3:systemContext></SOAP-ENV:Header><SOAP-ENV:Body><ns3:retrieveStopPayResponse xmlns:ns3="urn:abc.tailhead.readyforcheck.processStopPay"><retrieveGagaPayResponse recordFound="false"><checkNumber/><stopPayDate/></retrieveGagaPayResponse></ns3:retrieveStopPayResponse></SOAP-ENV:Body></SOAP-ENV:Envelope>"
]
]
}
我将虚拟名称指定为 data1 和 data2。您可以根据需要将它们进一步分离。 请使用 Grok Debugger 进行调试。
,^%{GREEDYDATA}ERROR%{GREEDYDATA}response received: %{GREEDYDATA:error_message}
版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 dio@foxmail.com 举报，一经查实，本站将立刻删除。