Category Archives: ElasticSearch

Sample data for Elasticsearch

We got some sample data for Elasticsearch. 1.5 million records to be precise. We will use Filebeats and Elasticsearch pipelines to load up the data into the cluster. The data has text, numbers and even geo points ! The data size on disk will be around 640MB (Windows environment). So let get on with it.

Elasticsearch dynamic templates using match_mapping_type

This short tutorial on Elasticsearch dynamic templates using match_mapping_type will teach you how to control the mappings of the dynamically added fields in Elasticsearch. If you are reading this then it means that you understand the importance of mappings and how to manage them using templates. Elasticsearch dynamic templates are a natural progression of templates.… Read More »

Elasticsearch Update by Query API explained

In this post on Elasticsearch Update by Query API, I will show the real world use of this API, what it can do and what it can’t. And why you should strive to not be in a situation which warrants the use of this API in the first place.

How to extract filename from filebeat shipped logs

This post will show how to extract filename from filebeat shipped logs, using elasticsearch pipelines and grok. I will also show how to deal with the failures usually seen in real life. With that said lets get started.

Indexing csv files using Elasticsearch pipelines

In this tutorial on indexing csv files using Elasticsearch pipelines we will use painless script ingest a csv file. The painless script will run in a elasticsearch pipelines. This problem of ingesting csv logs shipped from filebeats directly into elasticsearch can be solved in many ways. I will discuss the usual method as well as… Read More »

Fixing Elasticsearch error: No handler for type [string] declared on field

This Elasticsearch error: No handler for type [string] declared on field is often seen after an “innocent” upgrade from Elasticsearch 5.x to 6.x. Classic sign is that the new indices do not get created. I faced this error when using Serilog to push data into the Elasticsearch cluster after upgrade. It is frustrating as it… Read More »

Managing Elasticsearch aliases using Curator

This tutorial on managing Elasticsearch aliases using Curator will help you to manage your Elasticsearch aliases better. There are not many detailed tutorials on this topic and hence this post. I hope that at the end of this tutorial you will appreciate the power curator brings to your hands.

Taking Elasticsearch snapshots using Curator

This tutorial on taking Elasticsearch snapshots using curator will be divided into sections. One obvious section is how to take snapshots. Other less obvious part will be on configuring a shared directory using Network file sharing on Linux. I will be using a RHEL 7 based cluster of three machines for this tutorial. Once you… Read More »

Authentication in Elasticsearch without shield or x-pack

Authentication in Elasticsearch without using x-pack or shield. Possible? Yes. In this post I will show you how to do it using excellent readonlyrest plugin written by sscarduzio. The reason I used this plugin was the ease of use as well as the way it works. That it is listed on Elastic website itself as… Read More »

ElasticSearch Sample Data

This ElasticSearch Sample Data is to be used for learning purpose only. It is randomly generated but still care has been taken to make it look like real world data. UPDATE: A follow up to this post has been published. I will highly recommend that you have a look at it. I use Elasticsearch 7.2.0… Read More »