elastic4hpcclogs fields on Kibana question
I have Deployed elastic4hpcclogs 1.2.0 on an Azure AKS cluster, and found that the hpcc.log.* fields created don't seem to be acknowledged as searchable/filterable by Kibana, is there any way to fix this?
- g-pan
- Posts: 6
- Joined: Tue Apr 08, 2014 6:35 pm
We've noticed this once but we're unable to recreate.
However, one possible solution is to explicitly provide field types for the generated hpcc.log.* fields in the hpccpipeline.
In Kibana, under Stack Management | Ingest Pipelines, find the pre-configured "hpccpipeline" and choose to edit it. Change the Grok to include field types.
In this example we declare all fields as "strings":
Since the pipeline creates the hpcc.log fields on the target Elastic index(es), preexisting indexes will not update the field types. The field types will take affect on new indexes, pre-existing indexes will need to be removed.
If the issue continues, please let us know on the HPCC bug tracking system:
https://track.hpccsystems.com/browse/HPCC-27084
However, one possible solution is to explicitly provide field types for the generated hpcc.log.* fields in the hpccpipeline.
In Kibana, under Stack Management | Ingest Pipelines, find the pre-configured "hpccpipeline" and choose to edit it. Change the Grok to include field types.
In this example we declare all fields as "strings":
- Code: Select all
{{[
{ "grok":
{ "field": "message", "patterns": [ "%\{BASE16NUM:hpcc.log.sequence:string}
\\s+%{HPCC_LOG_AUDIENCE:hpcc.log.audience:string}\\s+%{HPCC_LOG_CLASS:hpcc.log.class:string}\\s+%{TIMESTAMP_ISO8601:hpcc.log.timestamp:string}\\s+%{POSINT:hpcc.log.procid:string}\\s+%{POSINT:hpcc.log.threadid:string}\\s+%{HPCC_LOG_WUID:hpcc.log.jobid:string}
s+%{QUOTEDSTRING:hpcc.log.message:string}" ], "pattern_definitions":
{ "HPCC_LOG_WUID": "([A-Z][0-9]\{8}
-[0-9]{6})|(UNK)", "HPCC_LOG_CLASS": "DIS|ERR|WRN|INF|PRO|MET|UNK", "HPCC_LOG_AUDIENCE": "OPR|USR|PRG|AUD|UNK" }
}
}
]}}
Since the pipeline creates the hpcc.log fields on the target Elastic index(es), preexisting indexes will not update the field types. The field types will take affect on new indexes, pre-existing indexes will need to be removed.
If the issue continues, please let us know on the HPCC bug tracking system:
https://track.hpccsystems.com/browse/HPCC-27084
- rodrigo.pastrana
- Posts: 29
- Joined: Tue Jun 10, 2014 2:19 pm
2 posts
• Page 1 of 1
Who is online
Users browsing this forum: No registered users and 1 guest