Managing Elasticsearch aliases using Curator

By | August 15, 2017

This tutorial on managing Elasticsearch aliases using Curator will help you to manage your Elasticsearch aliases better. There are not many detailed tutorials on this topic and hence this post. I hope that at the end of this tutorial you will appreciate the power curator brings to your hands.

WHY

We cannot keep storing logs in an ELK stack. Old logs are phased out in stages. New logs take their place. If you have got analytics running on top of these logs (which you should) then there will be this issue of changing indices name.

In world of logging the name of logs follows a pattern like MyAwesomeLogs-2017-12-30. Now depending on the configuration indices for storing logs might be created on monthly, weekly or daily basis. To visualise the data in logs mostly Kibana is used. And dashboards in Kibana are based on index names. So if you create a Dashboard on the most common error for month of March then in April you have to change the dashboard source of data to the index created in April.

Managing elasticsearch aliases
Will it not be awesome if you just used current_month instead of March in making of your dashboard. And in backgound somehow current_month always keeps pointing to index for current month. In other words pointing to MyAwesomeLogs-2017-3-30 in month of march and pointing to MyAwesomeLogs-2017-4-30 in month of April.

HOW

Enter Elasticsearch aliases. Elasticsearch aliases allow you to assign an alias for a set of indices. So when you want to query something you can just issue the query against the alias instead of individually listing the target indices. Elasticsearch aliases allow us to solve the problem of rolling log indices. In the month of March if you have been creating an index everyday for storing logs then an alias of current_month targeted at MyAwesomeLogs-2017-04-* targets all the logs for the month of March!!! Write a query against the alias current_month and Elasticsearch in background hits all the indices for the month of March. Problem solved! Elasticsearch aliases here we come!
Managing elasticsearch aliases
You see the process of managing Elasticsearch aliases is as manual as it can be. Every month you have to add the new indices to alias and remove the old ones from it to keep the alias meaningful. You can do it via the Kibana dev window or if you are a terminal ninja then you can use curl. There has to be a better way. And it is called……drum rolls please.
managing elasticsearch aliases

Curator allows you to automate management of Elasticsearch aliases. You might want to go through the previous post on installation and configuration of curator.
Let us create some indices first so that we can put curator through paces.
This is a quick and dirty shell script which does just that.

As you can see it takes two parameters. First is either today, back or fwd. The second is the number of days.
./GenerateIndices.sh today will generate an index for the current date.
./GenerateIndices.sh back 5 will generate indices for last 5 days.
./GenerateIndices.sh fwd 5 will generate indices for coming 5 days.

Change the server ip and the port number to suit your setup.

Time to unleash Curator fire power.
Managing elasticsearch aliases
To demonstrate the management of Elasticsearch aliases I need some indices first. Let use create indices for last 75 days.

This a snipped version of indices now in our Elasticsearch cluster.

Now we will ask curator to create an alias called last_week which will target last week. As usual we will come up with an action file for the same. Let us call it action_alias.yml.

Line 14: The name of the alias is set to last_week
Line 16: This bit is to tell curator is that if no indices are found then please do not throw error and stop. Rather just carry on to the next action. By default curator will stop at first failed action.
Line 20: Setting the filter type to pattern so that I can target indices having a particular patter of naming. In my case all the indices will start with “logs_”.
Line 21: Setting the filter to match all the indices which start with “logs_”. You can use suffix to match the end if you want.
Line 22: This filter will now match all indices whose name starts with “logs_”.

Eagle eyed readers would have noticed the problem by now. This filter will match all of my indices!!! All of them. From 2 years old ones to the one generated today. We need more filtering.

Line 18: Instructs the curator to add the indices which match the below filters.

Line 24: Setting the filter type to period. This will take what has been passed to it by previous filter and then perform time based filtering on it. It is called filter chaining.
Line 25: Setting the source to be of type name. This means that it will look for the time string in the name of the index.
Line 28: Since my indices will have name like logs_2017-08-17 I will configure the timestring to be hyphen seperated.
Line 29: Since my alias is all about last week I will set the unit to weeks. Curator now will do all date related calculations. This is beautiful. Check why.
Line 30: You can set it to monday. But sunday is more common.

In the lines below, e.g. line 26 and 27 I am asking curator to add the last week indices to the alias.

Line 26: range_from is a bit tricky to explain. Here it means start from the week before the current one. Negative means we are talking about past. Imagine negative side of number line.
Line 27: range_to tells where the period ends. Here it means that it ends at last week. So essentially I have chosen the last week. If this value was -2 then it means I have chosen last two weeks.

Line 31: Instructs the curator to remove from the alias the indices which match the below filters.
Line 38-39: I am asking curator to remove the indices which are two week old.

Time for action.

Result

How to check that alias has been set?

Check the days on which the alias has been set. As per the date of writing it has picked up the last week correctly.

Now I will add another alias called this_week. I will add this section to the action_alias.yml.

Time for action.

First create a index for today.

Result

Check that alias has been set?

Check the days on which the alias this_week has been set. It is Thursday today and curator has put this_week alias correctly on Sun, Mon, Tue, Wed and Thrs.

Now I will add more to the action_alias.yml to create aliases this_day, this_month and last_month.

Run it

Result

Now to check if the Elasticsearch aliases are setup as expected.

I will leave the counting to you.

Now that is nice. Curator is managing Elasticsearch aliases as expected. The only thing left is to schedule curator to run as a cron job. This is easy.

and add this line

I am configuring it to run every minute for demo purpose.

Things to remember when managing your Elasticsearch aliases using curator:
1. Each action you see above has two parts. Add indices and remove indices. They are done atomically. So there is no time in between when the alias is not pointing to anything.
2. The action will fail if add or remove alias section fails. For example if you have index just for today. You will expect that setting the this_day alias should be a success. It will not be because the remove setion of action looks for the index of previous day. Since it does not find it the whole action of setting the this_day index fails. If this is a concern then separate add and remove in two separate actions.

And that is it. Curator will now take care of managing your Elasticsearch aliases on the indices.

One thought on “Managing Elasticsearch aliases using Curator

  1. Pingback: Taking Elasticsearch snapshots using Curator

Leave a Reply