AWS Cloudwatch insight query to parse AWS Opensearch search log

Recently, I wanted to analyze our OpenSearch cluster using AWS CloudWatch Logs Insights. However, I was unable to find any official documentation that specifically addressed this topic.

After conducting a thorough search on Google and seeking assistance from ChatGPT, I came up with the following query:

The example search log from AWS Opensearch:
(You can setup this query following the official document)

[2023-05-15T02:24:59,493][WARN ][index.search.slowlog.query] [$node_name][$index_name][0] took[275.7ms], took_millis[275], total_hits[1736218+ hits], types[], stats[], search_type[QUERY_THEN_FETCH], total_shards[1], source[$query_content], id[]

Cloud watch query to count the total number of logs with logLevel = “WARN “ and aggregate by $index

fields @timestamp, @message
| parse @message "[*][*][*] [*] [*][*]" as _timestamp, logLevel, logType, node, index, message
| filter logLevel = "WARN "
| stats count(*) by index

Here I need to address some points:

• I only need the first five fields for my analysis, so I parsed all other fields into last message field
• Be aware that there are some random spaces in log