How to Search Elasticsearch (ES) Index from ExpressJS

“Search is something that any application should have”
Shay Banon

Sharing is caring!

Overview

Searching is part of our daily lives. Especially in the internet where search engines are everywhere. Therefore, it is a fundamental need for any application to have search functionalities. And today, Elasticsearch is the most popular search engine in applications world.

As you may know, Elasticsearch itself comes with its own REST API. However, the results sometimes are too cumbersome to be parsed by another party. Another concern would be what if Elasticsearch comes with another version and response payloads will be different. Rather than we change all our API caller, it is better if we implement adapter pattern on top of that. In respect to those needs, I am using ExpressJS as our REST API adapter between Elasticsearch and outside ES world.

Why I am using NodeJS? Because it is commonly pure high IO between ES and NodeJS application. No heavy computational needs to parse the payload. But you may face different cases and NodeJS simply not enough.

Use Case

I want to search a collection of books by three criteria, which are by title, by category and by author.

Here is a list of books in my catalogue:

Book Category Author
First Name Middle Name Last Name
Cracking the Coding Interview Computer Programming Gayle Laakmann McDowell
Sapiens: A Brief History of Humankind Civilization Yuval Noah Harari
Homo Deus: A Brief History of Tomorrow Civilization Yuval Noah Harari

And our Book Search API will be:

ExpressJS

First, we will generate ExpressJS project with name es-sample.

Elasticsearch and Additional Libraries

We use official Elasticsearch 7 libraries and other libraries for compression, security and environment configuration

Elasticsearch.js

It is time to configure app.js to setup client connection with Elasticsearch.

Basically, you don’t need to configure SSL part, unless you used self-signed certificate. If you use self-signed, ES connection using .cer format. In general, the CA should consists of CA, Intermediate CA and Root CA.

Another important note is apiKey field. API Key should be in Base64, with combination with ID and api_key from host.

How to create API KEY in Elasticsearch, please visit here.

Routes

We have three routes for our book search API. They are routes/title.js, routes/category.js and routes/author.js.

routes/title.js

routes/category.js

routes/author.js

Run

Before we execute our node application, we need to run ES on Docker and export our testing data.

Next, execute es-sample.

Then begin searching.

Advance Topic

This advance topic will talk about putting es-sample project into Jenkins pipeline. As usual, the basic foundation of good CI/CD practice is to have test automation and good code coverage in place. Therefore, first we need to add testing libraries.

Testing Libraries

For testing, I am using mocha, chai, chai-http and istanbuljs.

And add this step into package.json.

Execute your test manually.

Jenkins Pipeline

Once your testing has been setup, we will create deployment pipeline using Jenkins. Our pipeline will use Trunk Based Development (TBD) approach, therefore our versioning will combine build-date and git-sha. We will run the integration test by using side car container. Then we will publish the report, check in sonar whether our code quality pass, and lastly ship our application via Docker image.

Import your Jenkinsfile and start your build automation. You will see something like this.

Closure

That’s all from my side. The full source code are already on my github repository.

Author: ru rocker

I am a professional software developer with more than 10 years experiences. I am a certified Java Developer (SCJP and SCWCD). However, In the recent months, I have more interest in DevOps and start to become a polyglot developer. Python and Go-lang become my favorite programming languages besides Java.

Leave a Reply

Your email address will not be published. Required fields are marked *