Secure Elasticsearch Communication

Learn how to enable TLS on the Transport and HTTP layers

Secure ElasticSearch communication

One of the biggest concerns of any software system is Security!

Clairvoyant always focuses on creating more reliable and secure systems. Here, we ensure that every internal and external communication is always encrypted, so no one can listen to what’s going on inside the system. It is at the heart of our operational model to provide highly secure solutions to our customers.

Borrowing from our expertise in exploring security concepts in-depth, in this blog, we’ll learn how to enable TLS on the Transport and HTTP layers in an Elasticsearch cluster.

Elasticsearch is a very commonly used open-source full-text search and analytics engine.

The Elasticsearch cluster usually consists of many nodes (Logstash, Kibana, and clients that communicate with the cluster). It should come as no surprise that securing such sets has many aspects and layers.

No security is enabled in Elasticsearch by default.

Once we enable the security, all traffic to, from, and within the Elasticsearch cluster will be encrypted.

Ok, let’s start enabling security now!

First of all, let’s enable security by adding/setting the below property in the “elasticsearch.yml” file, under the config folder:

xpack.security.enabled: true

We’ll need X.509 certificates to enable TLS. We can use the “elasticsearch-certutil” utility provided under the bin directory of Elasticsearch to generate the same!

Let’s generate a certificate authority for the Elasticsearch cluster. Using the ca command below, we can generate a new certificate authority (CA). Output file (“elastic-stack-ca.p12”) is a PKCS#12 Keystore that contains the public certificate for your CA and the private key that is used to sign the certificates for each node.

bin/elasticsearch-certutil ca

After executing the above command, it asks for a password to protect the file and key. If we plan to add more nodes to the cluster in the future, we should retain a copy of the file and remember/save its password.

Again, we can use the “elasticsearch-certutil” utility. Please execute the below command to create the certificate:

bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12

The output of the cert command will be a single PKCS#12 Keystore that includes the node certificate, node key, and CA certificate.

Generate additional certificates specifically for encrypting HTTP client communications. Again, we can use “elasticsearch-certutil” utility. Please execute the below command to create the certificate:

bin/elasticsearch-certutil http

This command guides you through the process of generating the appropriate certificates for use in Elasticsearch and Kibana. We can re-use CA (which was created in earlier steps) for the cluster by supplying its location when prompted.

The output file of the HTTP command is a zip that contains certificates and keys for use in Elasticsearch and Kibana. It also contains a readme as a usage guide.

So, we are done generating all the certifications!

For official documentation, please check this out:

https://www.elastic.co/guide/en/elasticsearch/reference/7.x/configuring-tls.html#node-certificates

Let’s enable TLS on the Transport and HTTP layers:

By enabling the TLS on the transport layer, communication between the elastic search nodes will be encrypted.

By enabling the TLS on the HTTP layer, communication between HTTP clients and the cluster will be encrypted. Enabling TLS on the HTTP layer is strongly recommended but not required.

Now, we should add the following information to the elasticsearch.yml file on each node:

xpack.security.transport.ssl.enabled: true

xpack.security.http.ssl.enabled: true

xpack.security.transport.ssl.verification_mode: certificate

xpack.security.transport.ssl.keystore.path: elastic-certificates.p12

xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

xpack.security.http.ssl.keystore.path: http.p12

If we have secured the node’s certificate with a password while generating certificates, we should add the password to our Elasticsearch Keystore. If the signed certificate is in PKCS#12 format, we can use the following commands:

Add Password to ES KeyStore

bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password

bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password

bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password

So, now that we are done with all the config changes, let’s restart Elasticsearch! If it’s not already running, start Elasticsearch!

Elasticsearch cluster is now secure! So, we’ll need users who can communicate with secured ES. Let’s set the passwords for all the built-in users:

Using the “elasticsearch-setup-passwords” command, we can set built-in users’ passwords. We can choose auto or interactive mode while executing it! The auto will generate all users’ passwords automatically but the interactive mode prompts you to enter passwords.

bin/elasticsearch-setup-passwords interactive|auto

Once we have enabled TLS on the HTTP layer, all communications with Elasticsearch will happen through HTTPS instead of HTTP.

So, we need to hit “https://localhost:9200" instead of “http://localhost:9200"

Integration of other clients (e.g. Kibana, SpringBoot, etc.) with secured Elasticsearch will be covered in the coming next blog!

Conclusion:

• Enabled security in Elasticsearch
• Generated certifications required for enabling TLS on Transport and HTTP layers
• Enabled TLS on the transport and HTTP layers
• Setup users' passwords

References:

https://www.elastic.co/guide/en/elasticsearch/reference/7.x/elasticsearch-security.html

https://www.elastic.co/guide/en/elasticsearch/reference/7.x/built-in-users.html