Elasticsearch
Elasticsearch standalone installation and configuration.

Installation

Download and install the Elastic public signing key :
1
sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
Copied!
Create a new yum repo file and add the following lines :
1
sudo vi /etc/yum.repos.d/elastic.repo
2
3
[elasticsearch-7.x]
4
name=Elasticsearch repository for 7.x packages
5
baseurl=https://artifacts.elastic.co/packages/7.x/yum
6
gpgcheck=1
7
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
8
enabled=1
9
autorefresh=1
10
type=rpm-md
Copied!
Install Elasticsearch :
1
sudo yum install elasticsearch
Copied!
Enable and start Elasticsearch :
1
sudo systemctl enable elasticsearch
2
sudo systemctl start elasticsearch
Copied!

Configuration

Basic configuration is not suitable for production environment. In production authentication, encryption and tuning are required for Elasticsearch nodes.

Basic configuration

Modify the Elasticsearch configuration file and set the following parameters with your values:
1
sudo vi /etc/elasticsearch/elasticsearch.yml
2
3
# Set Cluster Name
4
cluster.name: Nybble-Analytics
5
# Set Node Name
6
node.name: Trantor
7
# Lock memory on startup
8
bootstrap.memory_lock: true
9
# Set Node IP or Address to listen on
10
network.host: $Your_IP
11
# Set Network port to listen on (Default 9200)
12
http.port: 9200
13
# Set Master Node. In case of Standalone it's the only node.
14
cluster.initial_master_nodes: ["Trantor"]
Copied!

Security configuration

Native authentication can't be enabled without SSL for transport layer, both need to be configured.
Stop Elasticsearch and edit the elasticsearch.yml configuration file :
1
sudo systemctl stop elasticsearch
2
3
sudo vi /etc/elasticsearch/elasticsearch.yml
Copied!
Enable XPack security to be able to use the free Elasticsearch feature. Add the following line at the end of elasticsearch.yml file :
1
# ---------------------------------- Security ----------------------------------
2
#
3
xpack.security.enabled: true
4
Copied!

SSL Configuration

If you followed the instruction for Kafka SSL configuration, you can reuse the CA certificate, else you can follow the "Root CA" steps in Kafka section.
Create a truststore which will be used by all your Elasticsearch nodes (You can also reuse the truststore created during Kafka or Kibana secured configuration steps):
1
sudo keytool -keystore nybble.truststore.jks -alias CARoot -import -file nybble-ca.crt
Copied!
Truststore password will be used in Elasticsearch configuration.
Create a keystore file for each Elasticsearch node:
1
sudo keytool -keystore es-node1.keystore.jks -alias localhost -validity 3650 -genkey -keyalg RSA -ext SAN=DNS:es-node1.nybble.local
Copied!
Use your own node FQDN for the Subject Alternative Name (SAN).
Use your own node FQDN for response to "What is your first and last name?" prompt.
Keystore password will be used in Elasticsearch configuration.
Export the Elasticsearch node's certificate to signed it with the root CA:
1
sudo keytool -keystore es-node1.keystore.jks -alias localhost -certreq -file es-node1.unsigned.crt
Copied!
Sign the Elasticsearch node's certificate with the Root CA:
1
sudo openssl x509 -req -CA nybble-ca.crt -CAkey nybble-root.key -in es-node1.unsigned.crt -out es-node1.signed.crt -days 3650 -CAcreateserial
Copied!
Import the Root CA certificate in the Elasticsearch node's keystore:
1
sudo keytool -keystore es-node1.keystore.jks -alias CARoot -import -file nybble-ca.crt
Copied!
Import the signed Elasticsearch node's certificate in the Elasticsearch node's keystore:
1
sudo keytool -keystore es-node1.keystore.jks -alias localhost -import -file es-node1.signed.crt
Copied!
Copy the Elasticsearch node's keystore and the truststore (local or SFTP) in the Elasticsearch configuration folder and set elasticsearch service account as owner:
1
sudo cp nybble.truststore.jks /etc/elasticsearch/
2
sudo cp es-node1.keystore.jks /etc/elasticsearch/
3
4
sudo chown elasticsearch:elasticsearch /etc/elasticsearch/nybble.truststore.jks
5
sudo chown elasticsearch:elasticsearch /etc/elasticsearch/es-node1.keystore.jks
Copied!
Edit Elasticsearch configuration and add following lines in "Security" section created above:
Change "network.host" value to match the FQDN used during certificate creation in the previous steps.
1
#
2
# Set Node IP or Address to listen on. Need to match the FQDN used during certificate creation.
3
#
4
network.host: $Your_Node_FQDN
5
#
6
# Enable SSL for transport layer
7
#
8
xpack.security.transport.ssl.enabled: true
9
#
10
# Set SSL verification mode for transport layer
11
#
12
xpack.security.transport.ssl.verification_mode: certificate
13
#
14
# Set SSL keystore path for transport layer
15
#
16
xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/es-node1.keystore.jks
17
#
18
# Set SSL keystore password for transport layer
19
#
20
xpack.security.transport.ssl.keystore.password: $your_keystore_pwd
21
#
22
# Set SSL truststore path for transport layer
23
#
24
xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/nybble.truststore.jks
25
#
26
# Set SSL truststore password for transport layer
27
#
28
xpack.security.transport.ssl.truststore.password: $your_trustore_pwd
29
#
30
# Enable SSL for HTTP layer
31
#
32
xpack.security.http.ssl.enabled: true
33
#
34
# Set SSL keystore path for HTTP layer
35
#
36
xpack.security.http.ssl.keystore.path: /etc/elasticsearch/es-node1.keystore.jks
37
#
38
# Set SSL keystore password for HTTP layer
39
#
40
xpack.security.http.ssl.keystore.password: $your_keystore_pwd
41
#
42
# Set client authentication mode for HTTP layer
43
#
44
xpack.security.http.ssl.client_authentication: optional
Copied!

Native authentication configuration

Add the following lines in "Security" section created above to enable Native realm:
1
xpack:
2
security:
3
authc:
4
realms:
5
native:
6
native1:
7
order: 0
Copied!
Restart Elasticsearch after SSL and Native authentication configuration:
1
sudo systemctl start elasticsearch
Copied!
Modify built-in user passwords:
1
sudo /usr/share/elasticsearch/bin/elasticsearch-setup-passwords interactive
Copied!
Modify your Kibana configuration to use "kibana" user to authenticate with Elasticsearch.
Kibana configuration guide can be found in Appendix chapter.
Access Kibana interface and log in with "elastic" user.
Go to Management > Security > Users and create a new user with "transport_client" role for Nybble sink:
Elasticsearch sink user creation

Tuning/Options

Heap memory

Set the min and max allocated heap space. It's recommended to not allocate more than 50% of the total server memory and no more than 32Gb.
1
sudo vi /etc/elasticsearch/jvm.options
2
3
-Xms2g
4
-Xmx2g
Copied!

Limits

Modify the limits.conf file and add the following lines at the end:
1
sudo vi /etc/security/limits.conf
2
3
elasticsearch soft memlock unlimited
4
elasticsearch hard memlock unlimited
Copied!

Sysconfig

Set MAX_LOCKED_MEMORY to unlimited:
1
sudo vi /etc/sysconfig/elasticsearch
2
3
MAX_LOCKED_MEMORY=unlimited
Copied!

Elasticsearch service

Add the following lines in elasticsearch.service script:
1
sudo vi /usr/lib/systemd/system/elasticsearch.service
2
3
# Specifies the maximum reserved size of memory
4
LimitMEMLOCK=infinity
Copied!

Disable swap

1
sudo swapoff -a
Copied!
To permanently disable swap, comment the swap line in fstab file :
1
sudo vi /etc/fstab
2
3
#UUID=871908ba-890a-4a9a-9b93-f471629d2940 swap swap defaults,noatime 0 0
Copied!
Last modified 1yr ago