Configuring Elasticsearch with Security

If you're using Elasticsearch and want security over SSL/TLS to add encryption for node to node communication and client communication, you'll need to complete some additional setup and configuration steps.

This document provides information about implementing security for Elasticsearch. Depending on your API Platform version, you'll use one of these:

  • X-Pack: Version 2018.0.0 uses Elasticsearch 6.3.1, which uses X-Pack for security.

    X-Pack is bundled with the Elasticsearch install, but requires a license for activation.

  • Shield: Versions prior to 2018.0.0 use Elasticsearch 1.5, which uses Shield for security. Shield is not supported in version 2018.0.0.

    For full details about installing Shield, see https://www.elastic.co/guide/en/shield/shield-1.3/installing-shield.html.

Installing and Configuring Elasticsearch (2018.0.0) Configuring Elasticsearch with Security

Supported Platforms: 8.4x, 2018.0.0

Table of Contents

  1. Setting up X-Pack (2018.0.0 and later)
    1. Step 1: Change the bootstrap.password value for built-in users
    2. Step 2: Set up user and user roles
    3. Step 3: Add user with specific roles
    4. Step 4: Modify the Elasticsearch YAML file
    5. Step 5: Set up encryption for communications between nodes in a cluster/encrypting HTTP client communications
    6. Step 6: Set up encryption for communications from the client to the Elasticsearch cluster
  2. Installing and configuring the Elasticsearch Shield plug-in (8.4x, prior to 2018.0.0)
    1. Step 1: Download the Elasticsearch Shield plug-in
    2. Step 2: Unzip installation files
    3. Step 3: Set up user and user roles
    4. Step 4: Configure the Elasticsearch YAML file
    5. Step 5: Configure transport client security settings

Note: For general information about installation and configuration of Elasticsearch, see Installing and Configuring Elasticsearch.

Setting up X-Pack

X-Pack is bundled with the Elasticsearch 6.3.1 install, but requires a license for activation.

As a starting point, here is the API call you can use to start a trial license for 30 days:

POST {http://localhost:9200}/_xpack/license/start_trial?acknowledge=true

However, X-Pack is a paid product and you'll need to renew the license.

For the hostname in the above, use the address for the Elasticsearch nodes. If you have a cluster with multiple Elasticsearch nodes, use the address for any one of the nodes.

To set up the X-Pack plug-in, complete the following steps:

  1. Step 1: Change the bootstrap.password value for built-in users
  2. Step 2: Set up user and user roles
  3. Step 3: Add user with specific roles
  4. Step 4: Modify the Elasticsearch YAML file
  5. Step 5: Set up encryption for communications between nodes in a cluster/encrypting HTTP client communications
  6. Step 6: Set up encryption for communications from the client to the Elasticsearch cluster

Step 1: Change the bootstrap.password value for built-in users

Elasticsearch comes with a built-in default user. As a first step, change the bootstrap.password value.

For information about changing the bootstrap.password value for built-in users, refer to the Elasticsearch help: https://www.elastic.co/guide/en/elastic-stack-overview/current/get-started-built-in-users.html.

back to top

Step 2: Set up user and user roles

You can:

To add new roles using the X-Pack role management API

Use the built-in user, which you updated in the previous step, to call this API. To use this API, you must have at least the manage_security cluster privilege.

Adding a new role: example

The following API adds a new role, read-only-user, with a privilege of read for the search indexes. This can be used on the containers where search is performed but indexing is not (Community Manager Scheduled Jobs feature is not installed).

URL:

POST /_xpack/security/role/read_only_user

Message body:

{
  "cluster":[
    "all or the cluster name in elasticsearch.yml"
  ],
  "indices":[
    {
      "names":[
        "default*"
      ],
      "privileges":[
        "read"
      ]
    }
  ]
}

Example: adding a role es-admin with indices privilege all

URL:

POST /_xpack/security/role/es-admin

Message body:

{
  "cluster":[
    "all or the cluster name in elasticsearch.yml"
  ],
  "indices":[
    {
      "names":[
        "default*"
      ],
      "privileges":[
        "all"
      ]
    }
  ]
}

The container with Community Manager Scheduled jobs feature installed needs a user with indices privileges all. Use the above es-admin role on these containers.

You might also need a user with a built-in role of superuser to perform different tasks on the cluster. For information on setting this up, refer to https://www.elastic.co/guide/en/elastic-stack-overview/6.3/built-in-roles.html.

To add users and roles using role mapping files

For information about using this approach, see https://www.elastic.co/guide/en/elasticsearch/reference/6.3/users-command.html (Elasticsearch help).

back to top

Step 3: Add user with specific roles

Use the built-in user above to call this api. To use this API, you must have at least the manage_security cluster privilege.

For information about using this approach, see https://www.elastic.co/guide/en/elasticsearch/reference/6.3/security-api-put-user.html (Elasticsearch help).

Example #1: adding adminuser

The following example creates a user adminuser.

URL:

POST /_xpack/security/user/adminuser

Message body:

{
  "password":"Specify Password",
  "roles":[
    "admin",
    "other_role1"
  ],
  "full_name":"Admin User",
  "email":"adminuser@example.com"
}

Example #2: adding superuser

The following example creates a user superuser.

URL:

POST /_xpack/security/user/mysuperuser

Message body:

{
  "password":"Specify Password",
  "roles":[
    "superuser"
  ],
  "full_name":"Some Super User",
  "email":"superuser@example.com"
}

back to top

Step 4: Modify the Elasticsearch YAML file

The next step is to manually change some of the settings in one of the Elasticsearch configuration files, elasticsearch.yml, to enable X-Pack security.

The elasticsearch.yml file is generally stored in the {elasticsearch_home}/config folder. It might have some default placeholder content, but not all the placeholder values.

As a starting point to model your changes, you can use the example in Sample Elasticsearch YAML file with security settings below.

Note: The first part of the file, up to the section titled Various, is needed for all Elasticsearch installation, even if security is not added. The last section, Various, is where you set up the values for your secure installation.

To enable X-Pack security, you'll at least need to provide username/password.

In the YAML file, change the value for the xpack.security.enabled setting to true, as shown below.

xpack.security.enabled: true

You'll also need to set up other values. For details on how to modify the elasticsearch.yml file with the general settings for an Akana API Platform implementation, see What changes do I need to make to the Elasticsearch YAML file?

For general information about the elasticsearch.yml file, see https://www.elastic.co/guide/en/elasticsearch/reference/current/settings.html.

To configure the Elasticsearch YAML file
  1. Modify the properties in the elasticsearch.yml file, using the example in Sample Elasticsearch YAML file with security settings below (you can copy and paste). The secure values are set in the last section, Various, below.
  2. Restart the Elasticsearch cluster.
Sample Elasticsearch YAML file with security settings

Note: The example below excludes the first part of the YAML file, and shows only the last part, with the security settings. For an example of the first part of the file, see Sample Elasticsearch YAML file.

# ======================== Elasticsearch Configuration =========================
#
...
#
# ---------------------------------- Various -----------------------------------
# (This section is needed only if you want to configure security)
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
xpack.security.enabled: true

#xpack.ssl.client_authentication
#xpack.ssl.verification_mode: certificate

#xpack.security.transport.ssl.enabled: true

#pkcs12 or jks
#xpack.security.transport.ssl.verification_mode: certificate 
#xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12 
#xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12

#pem
#xpack.security.transport.ssl.verification_mode: certificate 
#xpack.security.transport.ssl.key: certs\instance.key 
#xpack.security.transport.ssl.certificate: certs\instance.crt 
#xpack.security.transport.ssl.certificate_authorities: [ "certs/ca.crt" ]

xpack.security.http.ssl.enabled: true
xpack.ssl.verification_mode: certificate
xpack.security.http.ssl.keystore.path: certs/elastic-certificates.p12 
xpack.security.http.ssl.truststore.path: certs/elastic-certificates.p12

back to top

Step 5: Set up encryption for communications between nodes in a cluster/encrypting HTTP client communications

For information about encryption, please follow the Elasticsearch instructions at this URL: https://www.elastic.co/guide/en/elasticsearch/reference/6.3/configuring-tls.html#node-certificates.

back to top

Step 6: Set up encryption for communications from the client to the Elasticsearch cluster

  1. Log in to the Akana Administration Console for the container.
  2. Go to Configuration > com.akana.es.client.security. You'll see the settings, as shown below:

    Elasticsearch configuration settings

  3. For elastic.client.keystoreType, specify one of the following values:
    • PKCS12 (the default)
    • JKS
    • PEM
  4. For elastic.client.transportSSLVerificationMode, specify one of the following values:
    • none (the default)
    • certificate
    • full

    For details about these values, please refer to the Elasticsearch documentation.

  5. If the Mode is Transport Client, and you are using PKCS12 or JKS, provide values for the following:
    • elastic.client.keystorePath
    • elastic.client.keystorePassword
    • elastic.client.truststorePath
    • elastic.client.truststorePassword
  6. If the Mode is Transport Client, and you are using PEM format, provide values for the following:
    • elastic.client.sslKey
    • elastic.client.sslCertificate
    • elastic.client.certificateAuthorities
    • elastic.client.keyPassphrase - if applicable
  7. If the mode is Rest Client, only PKCS12 and JKS formats are supported. Provide the following:
    • elastic.client.keystorePath
    • elastic.client.keystorePassword
    • elastic.client.sslHostNameVerifierMode—The default is allow_all. The other possible value is default, which enforces host name verification.

    Note: For Rest Client, only PKCS12 and JKS formats are accepted at this time. The PEM formats are not supported.

back to top

Installing and configuring the Elasticsearch Shield plug-in

To install and configure the Shield plug-in, complete the following steps:

  1. Step 1: Download the Elasticsearch Shield plug-in
  2. Step 2: Unzip installation files
  3. Step 3: Set up user and user roles
  4. Step 4: Configure the Elasticsearch YAML file
  5. Step 5: Configure transport client security settings

Step 1: Download the Elasticsearch Shield plug-in

The first step is to download the Shield plug-in.

To download the Shield plug-in
  1. Go to the Elasticsearch site (https://www.elastic.co/) and download the Shield feature:
  2. Download the license ZIP file:

    Note: This is the trial license, and can be used for development only. For the Production environment, you'll need to get a license from Elasticsearch support.

  3. Copy the ZIP files to the machine where Elasticsearch is installed.

back to top

Step 2: Unzip installation files

To install the plugins using the ZIP files, run bin/plugin with the -u option.

For example:

  1. Change directory (cd) to the Elasticsearch installation folder.
  2. Run the following two commands:
    1. bin/plugin -i license -u file:///{path_to_file}/license-1.0.0.zip
    2. bin/plugin -i shield -u file:///{path_to_file}/shield-1.3.3.zip

Note: You must specify an absolute path to the ZIP file after the file:// protocol.

back to top

Step 3: Set up user and user roles

The next step is to add user and user roles for the client user.

For detailed documentation about managing Elasticsearch users and user roles, see https://www.elastic.co/guide/en/shield/shield-1.3/_managing_users_in_an_esusers_realm.html.

Note: The Transport client user must have at least the user role of power_user for the container where the Community Manager Scheduled Jobs feature is installed. For the containers that are read-only (only search), the user role user is enough.

  1. CD to the {elasticsearch}/bin/shield folder.
  2. Run the following command to add a user:
    esusers useradd {username} -p {secret}

    For example:

    esusers useradd es_power_user -p Password1
  3. Assign roles to the user. The example below assigns the power_user role to the user es_power_user:
    esusers roles es_power_user -a power_user
  4. If needed, change user password:
    esusers passwd {username} -p {secret}
  5. If needed, remove roles assigned to a user. In the example below, -r removes role admin for user es_admin:
    esusers roles es_admin -r admin
  6. If needed, delete users, using the esusers userdel command:
    userdel {username}

back to top

Step 4: Configure the Elasticsearch YAML file

You'll need to manually change some of the settings in one of the Elasticsearch configuration files, elasticsearch.yml, generally stored in the {elasticsearch_home}/config folder.

You can have Shield installed and enabled but might choose not to enable SSL; for example, if the cluster is within the firewall. If Shield is installed and SSL is disabled, Shield still expects the transport client user authentication header. In the Akana Administration Console, see the Transport Client user settings.

For general information about the elasticsearch.yml file, see https://www.elastic.co/guide/en/elasticsearch/reference/current/settings.html.

To configure the Elasticsearch YAML file
  1. Modify the properties in the elasticsearch.yml file, using the examples below (you can copy and paste):
  2. Restart the Elasticsearch cluster.
Sample YAML with SSL enabled
shield.enabled: true
shield.ssl.keystore.path: {path}\node01.jks
shield.ssl.keystore.password: changeit
shield.ssl.keystore.key_password: changeit
shield.transport.ssl: true
shield.http.ssl: true
shield.https.ssl: true
shield.ssl.hostname_verification: false
#shield.ssl.hostname_verification.resolve_name: true
#client specific settings
#This enables SSL on the client profile. The default value for this setting is the value of shield.transport.ssl.
transport.profiles.client.ssl: false
#This setting keeps certificate authentication active for node-to-node traffic, but removes the requirement to distribute a signed certificate to transport clients. 
#transport.profiles.client.shield.ssl.client.auth: no
shield.http.ssl.client.auth: optional
Sample YAML with SSL disabled
shield.enabled: true
shield.ssl.keystore.path: C:\SOA\ES\NodeCert\node01.jks
shield.ssl.keystore.password: changeit
shield.ssl.keystore.key_password: changeit
shield.transport.ssl: false
shield.http.ssl: false
shield.https.ssl: false
shield.ssl.hostname_verification: false
#shield.ssl.hostname_verification.resolve_name: true
#client specific settings
#This enables SSL on the client profile. The default value for this setting is the value of shield.transport.ssl.
transport.profiles.client.ssl: false
#This setting keeps certificate authentication active for node-to-node traffic, but removes the requirement to distribute a signed certificate to transport clients. 
#transport.profiles.client.shield.ssl.client.auth: no
shield.http.ssl.client.auth: no

back to top

Step 5: Configure transport client security settings

Follow the steps below.

Note: If Shield is not installed on the Elasticsearch cluster, you don't need to do anything. The default for enabling SSL is false.

To configure the transport client security settings
  1. Log in to the Akana Administration Console.
  2. Click the Configuration tab.
  3. On the left, under Configuration Categories, scroll down to find the com.akana.es.client.security category. You'll see the settings, as shown below:
    elastic.client.alias
    elastic.client.aliasPassword
    elastic.client.clientUser:
    elastic.client.clientUserPassword:
    elastic.client.enableSSL: false
    elastic.client.keystorePassword
    elastic.client.keystorePath
  4. Change the values as needed. Sample settings when SSL is disabled are shown below:

    Elasticsearch configuration settings (SSL disabled)

    Sample settings when SSL is enabled are shown below:

    Elasticsearch configuration settings with SSL enabled

  5. Click Apply Changes.

Notes:

  • By default, SSL is disabled.
  • If SSL is disabled on the Elasticsearch cluster and Shield is installed, you must at minimum provide values for clientUser and clientUserPassword.
  • The transport client user is added in the above steps, with appropriate roles.

back to top