elastic4hpcclogs fields on Kibana question

[g-pan]

I have Deployed elastic4hpcclogs 1.2.0 on an Azure AKS cluster, and found that the hpcc.log.* fields created don't seem to be acknowledged as searchable/filterable by Kibana, is there any way to fix this?

[rodrigo.pastrana]

We've noticed this once but we're unable to recreate.
However, one possible solution is to explicitly provide field types for the generated hpcc.log.* fields in the hpccpipeline.

In Kibana, under Stack Management | Ingest Pipelines, find the pre-configured "hpccpipeline" and choose to edit it. Change the Grok to include field types.
In this example we declare all fields as "strings":

Code: Select all

{{[
{ "grok":
{ "field": "message", "patterns": [ "%\{BASE16NUM:hpcc.log.sequence:string}

\\s+%{HPCC_LOG_AUDIENCE:hpcc.log.audience:string}\\s+%{HPCC_LOG_CLASS:hpcc.log.class:string}\\s+%{TIMESTAMP_ISO8601:hpcc.log.timestamp:string}\\s+%{POSINT:hpcc.log.procid:string}\\s+%{POSINT:hpcc.log.threadid:string}\\s+%{HPCC_LOG_WUID:hpcc.log.jobid:string}
s+%{QUOTEDSTRING:hpcc.log.message:string}" ], "pattern_definitions":
{ "HPCC_LOG_WUID": "([A-Z][0-9]\{8}

-[0-9]{6})|(UNK)", "HPCC_LOG_CLASS": "DIS|ERR|WRN|INF|PRO|MET|UNK", "HPCC_LOG_AUDIENCE": "OPR|USR|PRG|AUD|UNK" }
}
}
]}}

Since the pipeline creates the hpcc.log fields on the target Elastic index(es), preexisting indexes will not update the field types. The field types will take affect on new indexes, pre-existing indexes will need to be removed.

If the issue continues, please let us know on the HPCC bug tracking system:
https://track.hpccsystems.com/browse/HPCC-27084