Many people assume that with ChatGPT, there is no longer a need for people to write articles or blogs. That would be true if the content was purely meant for an audience. But to be honest, my motivation is a bit more selfish 🙂 This is my process when I want to learn something new. I write about it, which forces me to research, take notes and organize things a bit. On top of that I can always return to it later to refresh my memory, which I end up doing more often than not.
How to start?
Looking at Elastic documentation we have 4 approaches
Elastic Cloud – Elastic Cloud provides a hosted Elasticsearch and Kibana service available on AWS, GCP, and Azure. It offers a 14-day free trial.
Docker – A good option for local development and testing.
Serverless – Delivers a fully managed experience that abstracts away the underlying infrastructure.
Start-local – An option for quickly getting started and running Elasticsearch locally.
Ok, the easiest way it seems to be with a local setup, but given that I will need Docker and I do have some Org restrictions with Docker desktop, lets use instead the Elastic Cloud Trial.
Elastic Cloud Setup
The guide is very clear and it requires to sign up for a free trial, during which we get prompted for the type of deployment we need. I am pretty sure that serverless is just fine for this self-educational purposes.
Then we can pick the server location. We also can decide between AWS GCP or Azure. No Oracle?!!
Lets pick something geographically close to me.
A few minutes after everything is setup and ready to roll.
The Project
I have years of sport data spread across Garmin and Strava and I want to gather insights and do some extra analysis on my own data. Elasticsearch is perfect for this because it lets me add all my data, index it and label it, and Kibana will allow to search, aggregate, and visualize large datasets quickly.
Adding Data
After a bit of back and forward I learned that Indexing is the process of adding data to Elasticsearch. There are several options here, UI, programmatically and also the workflow guide which is our choice for the type of indexing we want, which will be reflected in the type of search.
For this exercise I choose the fastest option, which is with the UI. I see that we can use sample dataset, but I will upload my activities.csv from Strava to start with.
When I click Upload a file, I am taken to a different screen where I add my csv file and click Import. This will upload the data and create the Index.
Now what?
Clearly there is more to do here – the data has plenty of columns, most are useless I am sure. As someone with some Data Quality and MDM experience I know that data should be curated before being uploaded for processing, but that is beyond my goal here.
For now I will leave the data untouched and jump into Kibana for some dashboards.
Kibana
I must say it took me more than expected to figure out where the Kibana option was – It’s called Dashboards, who would have tought 🙂
In most screens we get the “Want to learn more? View documentation” – most of the times I do take the time to read the documentation, but I will wing it this time to see how intuitive the UI is.
I pick Lens to create a visualization by drag-n-drop,
Immediately I see the issue with not having clean data – I have 99 columns with no name, from which I can drag-n-drop and also explore/discover in detail. This makes it really hard to easily create a dashboard as I would need to search for the data I need. This is something I will correct soon, but for now it’s enough, I ll try to get a visualization on the number of activities per sport.
I drag the column with the data I need (activity type) and drop in on the main screen, and we get a visualization with the default bar graph.
We can easily change the visuals of the dashboard
That was it
This was a fast and easy introduction to Elasticsearch, but it does allow me to see the powerful platform behind it and I am looking forward to explore this use case further, also having my eyes set on what AI capabilities does Elastic has to offer.
Leave a Reply