Multi tool use

logstash restrict search result to past day

I want to query Elasticsearch for an index a day before current date in Logstash using Elasticsearch input plugin.

I tried the following config for logstash,

input {
elasticsearch
{
hosts => ["localhost:9200"]
index => "logstash-%{+YYYY.MM.dd-6}"
query => '{ "query": { "query_string": { "query": "*" } } }'
size => 500
scroll => "5m"
docinfo => true
}
}
output { stdout { codec => rubydebug }
}


Can someone help me on how to do it?

input { elasticsearch { hosts => ["localhost:9200"] index => "logstash-%{+YYYY.MM.dd-6}" query => '{ "query": { "query_string": { "query": "*" } } }' size => 500 scroll => "5m" docinfo => true } } output { stdout { codec => rubydebug } }

1 Answer
1

You can use Date math index name within your elastic search query,

Date math index name resolution enables you to search a range of
time-series indices, rather than searching all of your time-series
indices and filtering the results or maintaining aliases. Limiting the
number of indices that are searched reduces the load on the cluster
and improves execution performance. For example, if you are searching
for errors in your daily logs, you can use a date math name template
to restrict the search to the past two days.

Almost all APIs that have an index parameter, support date math in the
index parameter value.

for instance to search for indices for yesterday, assuming the index use the default Logstash index name format, logstash-YYYY.MM.dd

logstash-YYYY.MM.dd

GET /<logstash-{now/d-1d}>/_search


input { elasticsearch { hosts => ["localhost:9200"] index => "logstash-{now/d-1d}" query => '{ "query": { "query_string": { "query": "*" } } }' size => 500 scroll => "5m" docinfo => true } } output { stdout { codec => rubydebug } }

index => "<logstash-{now/d}>"

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.