Docker is everywhere and Logstash is not going anywhere. So why not try running Logstash on Docker. I found that while Elasticsearch has done a good job of describing the details, the learning curve is still a bit steep for a person new to Docker. Hence this post.
There are some people out there who will like to connect to Snowflake via AWS Lambda to push data. This post is limited to connecting to Snowflake using AWS Lambda. The upcoming posts (if ever given my posting history) will detail how to send some actual data.
This is a brief post on how to configure the Elasticsearch Watcher alerts to be sent to Microsoft teams. We will use webhook action in Elasticsearch and corresponding incoming webook in Microsoft Teams. Documentation on these are rather sparse so I think it will be useful to detail the steps. WHY Elasticsearch Watcher is a… Read More »
Creating Box Plot in Kibana using Vega is what this tutorial all about. To keep it simple I will use hard coded data. However in the second part of this tutorial the data will come via aggregation from elasticsearch and will be plotted. WHY Box plots are very useful and manufacturing engineers especially love them.… Read More »
This tutorial on adding painless scripted field in kibana will give you a quick start on this rather useful feature. If you ever inherit an elasticsearch index and are wishing for some extra fields then scripted fields can save you lot of efforts. As usual we will start with why followed by how.
“Microsystems was unexpected at this time.” is a frustrating error which appears when you run Logstash 7.6.1 on a windows machine in command line window. Here is a workaround to get around it. Environment Logstash version: 7.6.1 Operating system: Windows 10 Java version : openjdk version “11.0.5” 2019-10-15 OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.5+10) OpenJDK… Read More »
We got some sample data for Elasticsearch. 1.5 million records to be precise. We will use Filebeats and Elasticsearch pipelines to load up the data into the cluster. The data has text, numbers and even geo points ! The data size on disk will be around 640MB (Windows environment). So let get on with it.
This short tutorial on Elasticsearch dynamic templates using match_mapping_type will teach you how to control the mappings of the dynamically added fields in Elasticsearch. If you are reading this then it means that you understand the importance of mappings and how to manage them using templates. Elasticsearch dynamic templates are a natural progression of templates.… Read More »
In this post on Elasticsearch Update by Query API, I will show the real world use of this API, what it can do and what it can’t. And why you should strive to not be in a situation which warrants the use of this API in the first place.
This post will show how to extract filename from filebeat shipped logs, using elasticsearch pipelines and grok. I will also show how to deal with the failures usually seen in real life. With that said lets get started.