Elasticsearch: Information for Site Admins

Plan and manage the configuration and deployment of Elasticsearch, the platform's search/indexing tool.

API Platform Version: 8.1 and later

Table of Contents

Elasticsearch feature overview:
  1. What is Elasticsearch?
  2. Why transition to Elasticsearch?
  3. What configuration modes are available for Elasticsearch?
  4. What is standalone mode?
  5. What is embedded mode?
  6. In Elasticsearch, what is a data node?
  7. How many Elasticsearch data nodes should I configure?
  8. In Elasticsearch, what is a master eligible node?
  9. Why would I configure a node so that it is not master eligible?
  10. In Elasticsearch, what is a client node?
  11. What is the default configuration mode for Elasticsearch?
  12. What deployment options are available for Elasticsearch in standalone mode?
  13. What deployment options are available for Elasticsearch in embedded mode?
  14. Elasticsearch limitations: reserved characters
Installing and Configuring Elasticsearch:
  1. I'm upgrading; what are my options?
  2. What is the basic configuration needed to get Elasticsearch working out of the box with the API Platform?
  3. What do I need to use Elasticsearch?
  4. How do I transition from Compass to Elasticsearch standalone mode?
  5. How do I transition from Compass to Elasticsearch embedded mode?
  6. How do I install the Elasticsearch feature?
  7. How do I configure Elasticsearch?
  8. What are the default ports for Elasticsearch configuration?
  9. Elasticsearch Global Configuration: Field Values
  10. Configuring an Elasticsearch Embedded Node: Field Values
  11. How do I delete an Elasticsearch search index?
  12. How do I force a rebuild of an Elasticsearch search index?
  13. How do I configure Elasticsearch in embedded mode with a single CM container?
  14. How do I configure Elasticsearch in embedded mode with multiple CM containers (two or more)?
  15. How do I configure the number of nodes and shards?
Troubleshooting Elasticsearch:
  1. Pre-upgrade search results are showing even though the index folder is populated with the Elasticsearch index: what do I do?
  2. Elasticsearch client can't connect to the nodes
  3. Related Topics

Elasticsearch feature information:

What is Elasticsearch?

In version 8.0, the Akana API Platform moved from Compass to Elasticsearch for managing the search feature.

Elasticsearch is a search engine based on Apache Lucene. It is more robust and extensible tool than Compass, which the platform used previously, and allows faster indexing and more responsive updating. It's an extremely popular tool in very broad use.

The platform provides several options for implementation flexibility: Elasticsearch can start in an embedded mode, or remotely to the Java process in a distributed mode or a single node mode.

If you have an Elasticsearch server that's used for other applications, and administrators are trained to manage the standalone Elasticsearch server, use the standalone Elasticsearch server. Otherwise, it's best to use embedded mode.

With the Embedded Elasticsearch Container feature, Elasticsearch runs within the container process. It supports all the features that a standalone Elasticsearch process would support. Customers should install this feature in every container where API Platform feature or API Platform Schedule Jobs Add-On is installed. In this scenario, the Administrator's activities to start and stop the process are the usual container management processes. Administration of Elasticsearch is done with the configuration wizard in the Akana Administration Console.

For more information on Elasticsearch terminology, refer to the Elasticsearch glossary: https://www.elastic.co/guide/en/elasticsearch/reference/current/glossary.html.

Back to top

Why transition to Elasticsearch?

Elasticsearch is more robust and extensible than the previous implementation of Compass.

The platform supports both standalone and embedded modes. In embedded mode, Elasticsearch is integrated and run within an individual container. The container opens new TCP ports for receiving Elasticsearch requests to do the search and receive the results. Elasticsearch supports TCP requests and REST API requests at separate ports. The API Platform uses TCP requests for searching and indexing. An HTTP port for REST API is needed only if external clients need the REST Elasticsearch interface.

However, there is a deployment impact. Sys admins must make deployment decisions; at minimum, they must determine where to save the index in the filesystem of each node and how to configure Elasticsearch in a cluster.

Back to top

What configuration modes are available for Elasticsearch?

Elasticsearch performance is very efficient and extensible. However, it requires implementation decisions and configuration tasks.

You can set up your installation to run Elasticsearch in these modes:

  • Standalone: In standalone mode, Elasticsearch works like a database server. It requires separate installation and is run as a different process, like an LDAP server. Standalone mode uses Transport Client. If you use Standalone mode, embedded mode is not needed. For more information, see What is standalone mode? below.
  • Embedded: In embedded mode, all you need to do is install the Embedded Elasticsearch feature in all containers. For more information, see What is embedded mode? below.

You can use a shared filesystem, or you can have a local filesystem on each machine. If you choose to use a separate filesystem for each machine, you don't need to do anything for replication of the index. If you're using a shared filesystem, be aware that if the shared filesystem were to go down it would be a single point of failure. you should only use a shared filesystem if it gives better IO performance with little to no compromise in reliability.

There are other options in a multi-tenant installation. For example, you can choose to have different indexes for different tenants. You might have ten small tenants sharing one index, and another large tenant with its own dedicated index.

Elasticsearch configuration is supported in the Akana Administration Console, with UI for global settings and also for configuration of each node.

Back to top

What is standalone mode?

With standalone mode, your installation will need to include a standalone external Elasticsearch server. Just as with a relational database, you'll need to provide the software and hardware required.

If you choose to run Elasticsearch in standalone mode, like a database server, all containers running the Akana API Platform can use it. In this scenario, it's important to provide a cluster capability to help prevent outage.

Generally, in this scenario, all containers are working as client nodes. There is no master eligible node or data node within the product.

Back to top

What is embedded mode?

In embedded mode, all you need to do is install the embedded feature in one or more containers. In this mode, there is no external software needed. Elasticsearch runs within each container it's installed in.

Back to top

In Elasticsearch, what is a data node?

A data node is where the index data resides.

Regardless of where the API platform feature is running, when an object is indexed, the data for that index is in the data node.

When a user is searching, the data is accessed from the data node. For example, let's say the API platform feature is running in Container 1, but when the user performs a search, Container 1 doesn’t have the data. The request is sent to Container 2, where the data is, and Container 2 returns the results back to Container 1, which in turn returns the results to the caller.

Back to top

How many Elasticsearch data nodes should I configure?

It's best to configure as many nodes as possible as data nodes, because if the data node goes down and there are no other data nodes, the search feature goes down. If there are no data nodes available, the data is inaccessible, so the index fails and the search feature fails entirely.

When you configure multiple data nodes, the internal logic of the Elasticsearch feature determines which data nodes will hold which data. Generally, all data is available in all data nodes.

Another scenario is that the Elasticsearch feature might be configured to use multiple index partitions, called shards. Each data node could contain certain shards.

Back to top

In Elasticsearch, what is a master eligible node?

A master eligible node is one that can become a master of the cluster.

There is always a cluster, even if there is only one node, so a single node will always be master eligible.

The master nodes of the cluster manage the cluster, including such tasks as:

  • Keeping track of all containers that are part of the cluster, and updating as needed when nodes join or leave the cluster
  • Keeping track of which nodes are master eligible
  • Keeping track of which shards are in which data node.

There can be more than one master node inside a cluster.

Here are some scenarios:

Four containers, four master eligible nodes, two master nodes

If the Elasticsearch feature is installed in four containers, with each container being configured as master eligible, but the Elasticsearch settings are such that only two containers are required to be masters, the cluster uses the Elasticsearch logic to determine which two will act as master. In this scenario, the additional nodes are master eligible, but not master nodes.

If one of the master nodes goes down, one of the additional master eligible nodes will become a master.

Two containers, one master eligible node, one master node

There must always be a master node; so, if only one node is configured as master eligible, that node is always the master node. Note that this scenario doesn't have any accommodation for redundancy.

For additional information, see Why would I configure a node so that it is not master eligible? below.

Back to top

Why would I configure a node so that it is not master eligible?

It might be that a specific container already has a processing burden and you don't want to add the cluster management task. In this scenario, it would be better to configure that node so that it is not master eligible.

In this scenario, you could install the Elasticsearch embedded feature in a dedicated container and set that to be a master node, taking on the extra processing work associated with managing the Elasticsearch index. The Elasticsearch node manages the cluster, and other nodes process the normal traffic for the installation.

However, in most cases, customers will choose to make every node a master eligible node. Additional master eligible nodes will only become the master node if other nodes go down. Unless the installation is very large, managing the cluster is not a very great overhead.

Back to top

In Elasticsearch, what is a client node?

The client node is the node that makes the search or indexing request. All nodes can be client nodes.

Any container, even if it isn't configured to be a master eligible node or a data node, is at least a client node. If it is a data node, it does the local searches; if it isn't a data node, it is still a client node. As such, it sends a search request or index request to a remote data node and receives the results.

The client node makes the call to the platform's Search API, which in turn requests the search results from the data node.

Back to top

What is the default configuration mode for Elasticsearch?

The default configuration mode is embedded mode.

Back to top

What deployment options are available for Elasticsearch in standalone mode?

The Elasticsearch server is run outside the API Platform, in the same way as a database server. If you choose to run Elasticsearch in standalone mode, it's important to provide a cluster capability to prevent outage. In this scenario, all API Platform containers run in Client mode; in this mode, they send both search and index requests to the standalone Elasticsearch server.

Back to top

What deployment options are available for Elasticsearch in embedded mode?

If you choose to run Elasticsearch in embedded mode, you can deploy multiple containers for this feature. This feature can run along with the Akana API Platform in the same container, or it can run in a completely different container by itself. In this scenario, you'll have to configure each container separately. See To configure an Elasticsearch embedded node in the Akana Administration Console.

Note: If you're using an external Elasticsearch server with Transport Client mode, you do not need to install embedded Elasticsearch on your containers.

Back to top

Elasticsearch limitations: reserved characters

Elasticsearch has certain characters that are reserved. If any of these characters are included in search terms, they'll need to be escaped, or the search feature will not work as expected.

The reserved characters are:

+ - = && || > < ! ( ) { } [ ] ^ " ~ * ? : \ /

To use one of these characters in search results, users must escape them with a leading backslash.

For example, to search for (123)app, a user would have to write this query: \(123\)app.

It's best if users steer clear of special characters in naming platform resources such as apps and APIs.

This information is documented for platform users here: How do I use special characters in search?

Back to top

Installing and Configuring Elasticsearch:

I'm upgrading; what are my options?

In setting up the Elasticsearch feature, there are three possible scenarios; follow the applicable set of steps as listed below.

Elasticsearch, embedded mode (most likely scenario): If you are using Elasticsearch (recommended) in embedded mode (you don't have a standalone Elasticsearch server) you'll need to:

Elasticsearch, standalone mode: If you are using Elasticsearch (recommended) in standalone mode (you have a standalone Elasticsearch server) you don't need to install the feature. You'll need to:

No Elasticsearch: If you do not want to use Elasticsearch, you'll just need to do a database update:

Installing the Akana Embedded Elasticsearch Node feature
  1. Log in to the Akana Administration Console.
  2. Click the Available Features tab.

  3. Choose Akana Embedded Elasticsearch Node and click Install Feature.
  4. When installation is complete, at the prompt, restart the system.
Elasticsearch configuration: global settings

When you install the Elasticsearch feature, the product runs in embedded mode by just accepting the default settings. You don't need to change anything. Follow the applicable procedure below:

  • Information for embedded mode
  • Information for standalone mode
Embedded mode: Global configuration
  1. In the Akana Administration Console, on the Configuration tab, under Configuration Actions, choose Configure Elasticsearch Global Configuration. The wizard opens.
  2. Make sure the Deployment Mode is set to Embedded, as shown below.

Standalone mode: Global configuration

If you choose to run in standalone mode, you don't need to install the Akana Embedded Elasticsearch Node feature. However, you must configure Elasticsearch.

  1. In the Akana Administration Console, on the Configuration tab, under Configuration Actions, choose Configure Elasticsearch Global Configuration. The wizard opens.
  2. Make sure the Deployment Mode is set to Transport Client.
  3. Provide the ES Server URL (see Elasticsearch Global Configuration: Field Values below).
Elasticsearch embedded node settings

If you're using Elasticsearch in embedded mode, the platform automatically configures. You don't need to do anything to configure the embedded node settings.

Note: If you are using Embedded Node configuration, the developer portal must be available on the default port or whichever port you configure in the Admin Console.

Database queries to switch the search index to Elasticsearch

Run the following queries to switch the search index to Elasticsearch and to trigger the index.

  1. Change the index target to Elasticsearch only:
    update TENANTS set INDEX_TARGET='elastic', LASTMODIFIEDDTS=<current-timestamp>;

    current-timestamp above is:

    • utc_timestamp() for MySQL
    • sys_extract_utc(current_timestamp) for Oracle
    • getutcdate() for MSSQL
    • current_timestamp – current_timezone for DB2
  2. Note down the number of records in the INDEX_STATUS table, then delete the index metadata so the platform can trigger the indexing again, this time with Elasticsearch:
    delete from INDEX_STATUS;
  3. Wait for a few minutes to see that the INDEX_STATUS table is populated to as many records as it had before emptying the table, and that the INDEX_QUEUE table is empty. This indicates that indexing is complete.
  4. Now, run the query below to change the platform so that it uses the Elasticsearch index. Until you complete this step, the platform continues to service the search results with the Compass index that was built before the upgrade.
    update TENANTS set SEARCH_SOURCE='elastic', LASTMODIFIEDDTS= <current-timestamp>;

    current-timestamp above is:

    • utc_timestamp() for MySQL
    • sys_extract_utc(current_timestamp) for Oracle
    • getutcdate() for MSSQL
    • current_timestamp – current_timezone for DB2

Example: database queries for MySQL database

For a MySQL database, the above database queries are as follows:

update TENANTS set INDEX_TARGET='elastic', LASTMODIFIEDDTS=utc_timestamp();
delete from INDEX_STATUS;
update TENANTS set SEARCH_SOURCE='elastic', LASTMODIFIEDDTS=utc_timestamp();
Steps to follow if you want to continue using Compass for search

If you don't want to move to Elasticsearch, but prefer to stay with the older Compass search feature, you'll need to check some values in the database and make changes if needed.

Make sure that the TENANTS table has these two fields set to compass:

  • INDEX_TARGET field: value must be compass (post-upgrade default value is compass,elastic; remove elastic)
  • SEARCH_SOURCE field, value must be compass (post-upgrade default value is elastic; change it)

Back to top

What is the basic configuration needed to get Elasticsearch working out of the box with the API Platform?

When you install the Elasticsearch feature (see How do I install the Elasticsearch feature?), the product runs in embedded mode by just accepting the default settings.

All containers need to have the embedded Elasticsearch feature installed. In this scenario, the container fulfills two roles:

  • Data node
  • Master eligible node

If you want to run in standalone mode, or want to customize your installation, you'll need to specify values; if you just want to accept the default settings, no additional configuration is needed.

Back to top

What do I need to use Elasticsearch?

If you want to run Elasticsearch in embedded mode (see What is embedded mode?), you'll need to install the Elasticsearch feature once you've installed or upgraded to version 8.0 or later. See How do I install the Elasticsearch feature?

If you are using standalone mode (see What is standalone mode?), you don't need to install the Akana Embedded Elasticsearch Node feature.

Back to top

How do I transition from Compass to Elasticsearch standalone mode?

After installing an upgrade from a version before 8.0 to version 8.0 or later, you'll need to complete some post-upgrade tasks.

  1. Install the Embedded Elasticsearch Container feature on all containers that have either the API Platform APIs installed or the API Platform Scheduled Jobs feature installed. See How do I install the Elasticsearch feature?
  2. Configure the global settings. See To set up global configuration of Elasticsearch in the Akana Administration Console.
  3. Review the settings for each node to run in client mode. You should not need to change the settings.
  4. Change the INDEX_TARGET for all tenants to either just elastic (to index using only Elasticsearch) or elastic,compass (to index using both Elasticsearch and Compass).
  5. Empty the INDEX_STATUS table so that all entries are indexed again.

Back to top

How do I transition from Compass to Elasticsearch embedded mode?

After installing the upgrade, you'll need to complete some post-upgrade tasks.

  1. Install the Embedded Elasticsearch Container feature on all containers that have either the API Platform APIs installed or the API Platform Scheduled Jobs feature installed. See How do I install the Elasticsearch feature?
  2. Configure the global settings. See To set up global configuration of Elasticsearch in the Akana Administration Console.
  3. Configure the settings for each node, with some nodes configured as a data node, some nodes configured as a master node, and some nodes configured as a client node. See To configure an Elasticsearch embedded node in the Akana Administration Console.

Back to top

How do I install the Elasticsearch feature?

Installation of the Elasticsearch feature is done in the Akana Administration Console. If you don't have access, get this step completed by the Administrator, following the procedure below.

This feature must be installed on every container that's running the Akana API Platform and also on the containers where the Community Manager Scheduled Jobs feature is installed.

To install the Elasticsearch feature
  1. Log in to the Akana Administration Console.
  2. Click the Available Features tab.
  3. Choose Akana Embedded Elasticsearch Node and click Install Feature.
  4. When installation is complete, at the prompt, restart the system.

Back to top

How do I configure Elasticsearch?

In some cases, if you are using Elasticsearch in embedded mode, you won't even need to do any configuration, just leave the default settings. See How do I configure Elasticsearch in embedded mode with a single CM container? below. However, you might have multiple containers or special requirements and want to configure Elasticsearch a certain way.

First, review the information above and determine your deployment strategy.

Then, when you've installed, or installed an upgrade, you're ready to configure Elasticsearch. Depending on the configuration you choose, you might have to follow one or both of the procedures listed below. If you do them both, complete the tasks in the order given below.

  1. Set up Elasticsearch global configuration.
  2. Configure Elasticsearch in embedded mode.
To set up global configuration of Elasticsearch in the Akana Administration Console
  1. In the Akana Administration Console, on the Configuration tab, under Configuration Actions, choose Configure Elasticsearch Global Configuration. The wizard opens.
  2. Specify the deployment mode and then set up values that will apply to all containers running Elasticsearch:
    • Embedded (the default): specify cluster name and minimum number of master nodes, and indicate whether it is a multicast scenario.
    • Transport Client: The node will not be in a cluster, but will communicate with a cluster. Specify cluster name and Elasticsearch server URL. Transport Client mode is used when the container uses a TCP port to connect to a standalone Elasticsearch server.
    • Client Only: The node will be in the cluster, but will not be a master node. Specify cluster name and master host URL, and indicate whether it is a multicast scenario. Client-only mode is used when a container running the embedded Elasticsearch feature becomes a node as part of an external standalone Elasticsearch server

    For more information, refer to Elasticsearch Global Configuration: Field Values below.

  3. Click Save.
  4. Restart.
To configure an Elasticsearch embedded node in the Akana Administration Console

Note: Run the steps below on every container that has an Elasticsearch embedded node.

  1. In the Akana Administration Console, on the Configuration tab, under Configuration Actions, choose Configure Elasticsearch Embedded Mode. The wizard opens.
  2. Set up values that will apply to Elasticsearch in embedded mode. For more information, refer to Configuring an Elasticsearch Embedded Node: Field Values below.

    Note: Check that the default value for Network Publish Address is correct for your installation, and modify if needed.

  3. Click Save.
  4. Restart the container.

Back to top

What are the default ports for Elasticsearch configuration?

The default ports for Elasticsearch configuration are as follows:

  • HTTP: Default is 9200, default range is 9200–9299
  • TCP: Default is 9300, default range is TCP is 9300–9399

Range implies that if the first port is busy, the platform tries the next one and so on.

Note to Administrators: When setting up your implementation, make sure that all ports in the range 9300–9305 are open, on all containers that the Elasticsearch feature is installed on.

Back to top

Elasticsearch Global Configuration: Field Values

Information about the field values is given below.

Deployment Mode
Choose from the available options:
  • Embedded: Only the embedded Elasticsearch feature is in use; no external software is required.
  • Transport Client (external with transport client interface): This setting indicates that the platform is integrating with an external Elasticsearch server. In this scenario, the Elasticsearch feature uses the TCP interface to get the search data or get the index requests. In that case, the request always goes from the container to the standalone Elasticsearch server. In this mode, there is no local node. There are only API calls to the external Elasticsearch server.

    If you are using Transport Client mode, you don't need to configure any of the embedded node features, because all requests go to the standalone external master node.

  • Client Only (external with client-only node interface): In this scenario, the platform starts a node locally, but the node is a client node. The client node can connect directly to the external Elasticsearch server's data node. In this model, the request always goes directly to the data node. This deployment mode performs better than Transport Client mode.
Cluster Name (all modes)
Name of the cluster for the Elasticsearch feature. Required for all deployment modes. Default for embedded mode: default_xxxxxx. When you install the feature, a value is defaulted in this field.
ES Server URL (Transport Client deployment mode only)
The transport address for the Elasticsearch server (one or more, separated by commas), in the format: {hostname}:{port}. For example, 10.12.121.116:9300 or 10.12.121.116:9300,10.12.122.140:9300.
Master Host URL (Client-Only deployment mode only)
The URL for the master host that this is a client for.
Minimum Master Nodes (Embedded mode only)
The minimum number of master nodes. Default: 2. Example: let's say there are five master eligible nodes, but Minimum Master Nodes is set to 2. In this scenario, two nodes are master nodes. If there is a specific need, such as increased load, an additional node might be used as a master node; if there is an outage, an additional node will replace the one that is down. However, if the minimum number is configured to be a smaller number, such as 1, only one master node might be used (depending on the Elasticsearch logic).
Multicast (Embedded or Client Only modes)
If this is a multicast scenario, check the box. In this scenario, the cluster name is defaulted.

Transport Client versus Client Only mode

Here is an example to illustrate the difference between these Transport Client and Client Only modes, both of which reference an external Elasticsearch server. Let's say there are two shards; one shard is on one node and the other shard is on a different node.

In the Transport Client scenario, the request is sent to the master node. The master node then sends the request to both data nodes, receives the responses, merges the data, and sends back the merged data as the response to the query.

In the Client Only scenario, the node has the information regarding which two data nodes are needed. It sends the request to both data nodes, receives the responses, and merges locally. The request is fulfilled with a smaller number of hops. Client-only mode is more efficient than transport client model.

Transport Client mode is used when the container uses a TCP port to connect to a standalone Elasticsearch server.

Client-only mode is used when a container running the embedded Elasticsearch feature becomes a node as part of an external standalone Elasticsearch server.

Back to top

Configuring an Elasticsearch Embedded Node: Field Values

Information about the field values is given below.

Name
The name of the node; for example, ES_Data01. Each embedded Elasticsearch node uses the container key for the name.
Index Location
The path to the folder where index data for this node is stored.
Note: if the location changes after the indexes are built, all the indexes must be rebuilt.
Bind to all
Check the box to bind to all.
Network Bind Address
The address that the server is binding the TCP port to; the address that is used to access the REST API on an HTTP port if an HTTP port is enabled. If Bind to all is selected, the Elasticsearch feature binds itself to the value 0.0.0.0, which indicates all addresses of the machine; it's not necessary to specify a network bind address.
If the machine has multiple network interfaces, it can bind to all or to a specific IP address. To bind to a specific address of the machine, provide the address in this field.
Network Publish Address
The address that other nodes will use to communicate with this node. It can be the machine host name or a valid IP address for the machine. If not specified, the address is automatically derived.
Transport Port
The port used for node-to-node communication, if there are multiple nodes. Default: 9300. For more information, see What are the default ports for Elasticsearch configuration?
HTTP Port
The port used for the REST API. Default: 9200. For more information, see What are the default ports for Elasticsearch configuration?
HTTP Enabled
Indicates that the Elasticsearch node is HTTP-enabled.
There are two ways that the data node or master node receives the Elasticsearch search request; TCP or HTTP. TCP is always enabled for all nodes, but HTTP is a higher-level protocol and is optional. To enable it, check the box.
The product does not depend on the HTTP port of any container. The HTTP enabled option really only comes into play to facilitate using some other interface, from outside the product, with HTTP interfaces. For example, you might want to use the HTTP interface of the Elasticsearch node for a third-party client, or for troubleshooting purposes. Internally, the platform does not depend on the HTTP port.
Master Eligible
Indicates whether the current node could be the master node.
There must be at least one master eligible node in a cluster.
Data Node
Indicates whether the current node is a data node only. The data node is where the index is stored.
There must be at least one data node in a cluster.
Index location is required only for data nodes. It's vital because if the node doesn't hold the data, there is no storage of index location. Index location is where the index is saved, and the index is only saved on the data nodes. Therefore, for a data node, the index location needs to be provided.
The default index location is a subfolder to the installation folder, /index, at the same level in the folder structure as the /lib and /bin folders. Index data is stored in this folder.
If the node is not a data node, index location is not needed.

Back to top

How do I delete an embedded Elasticsearch search index I'm no longer using?

You might need to delete an Elasticsearch index that you previously created; for example, if you configure an index and then change to use an index with a different name. If you have a search index that you no longer want to use, you'll need to delete it using the Akana Administrative Console. For instructions, see below.

Note: This is only applicable to embedded Elasticsearch nodes. If you're using external Elasticsearch nodes, you'll need to delete redundant indexes using the Elasticsearch commands or user interface provided by your external Elasticsearch engine.

To delete an Elasticsearch index in the Akana Administration Console
  1. In the Akana Administration Console, on the Configuration tab, under Configuration Actions, choose Delete Elasticsearch Index. The wizard opens.
  2. From the Available Index drop-down list, choose the index you want to delete.
  3. Click the Delete check box, and then click Finish. The index is deleted.

Back to top

How do I force a rebuild of an Elasticsearch search index?

If you're working with the Elasticsearch index and want to force a rebuild, you can delete everything from the INDEX_STATUS table. This will force a reindex of all the objects in the index.

Back to top

How do I configure Elasticsearch in embedded mode with a single CM container?

To run Elasticsearch in embedded mode, you don't need to change any configuration settings at all. Just go with the default settings and Elasticsearch takes care of everything in the background.

Back to top

How do I configure Elasticsearch in embedded mode with multiple CM containers (two or more)?

If there are two CM containers in your installation, you'll need to install the embedded Elasticsearch feature in both containers, and then configure the feature in both containers.

In this scenario, you'll need to decide whether you want to replicate the index in both containers or have it in only one container. Akana recommends configuring both containers, so that you have a failover for the index data.

To configure both containers, follow the steps below.

To configure two containers so that the index is replicated in both containers
  1. For each container, choose Configure Elasticsearch Embedded Node and configure the settings below:
    • Name: Enter any unique name for the node name in the cluster.
    • Index Location: Either leave blank or enter a folder path.
    • Bind to All: Check the check box, but leave the Network Bind Address blank.
    • Network Publish Address: Enter the hostname/ipaddress.
    • Transport Port: Enter an unused port. The default is 9300. For more information, see What are the default ports for Elasticsearch configuration?
    • Http Port: Leave blank.
    • Http Enabled: Cleared.
    • Master Eligible: Checked.
    • Data Node: Checked.
  2. Click Finish to save the configuration.
  3. Restart the containers. This replicates the index in both containers.

In case there is an issue with search working in one container but not the other, follow the additional steps below:

  1. Delete the index folder on both instances.
  2. Restart both containers.
  3. Empty the INDEX_STATUS table.

In some cases, you might find that the index is built with the earlier Compass feature as well as the new Elasticsearch feature. To avoid building the index with Compass unnecessarily, follow the steps below.

To ensure the search index is only Elasticsearch, not Compass
  1. In the TENANTS table, change the INDEX_TARGET field to elastic.
  2. Empty the index_objects table.
  3. Restart both containers.
  4. Empty the INDEX_STATUS table.

Back to top

How do I configure the number of nodes and shards?

New in version: 8.0.3

The platform includes configuration settings that you can use to manage your Elasticsearch setup. These are controlled by the Administrator for the Akana Admin Console; if you don't have access, coordinate as needed to get the settings modified if needed.

In the Akana Administration Console, the configuration category is: com.akana.elasticsearch.

In the default configuration, shown below, there are two shards and one replica. Let's say there are two nodes in the cluster. One shard, approximately half the index, is stored on each node. The one replica includes a replica of each shard:

  • Node 1 has Shard 1 and the replica of Shard 2
  • Node 2 has Shard 2 and the replica of Shard 1

In this scenario, if one node goes down, the other node has the full search index. Additional nodes add additional safety.

There are two settings, as shown below.

elastic.config.index.number.of.replicas
The number of replicas (additional copies) of the Elasticsearch index. Each replica includes a replica of each shard, so one replica might be distributed across multiple nodes, just as the index itself is split into shards which are distributed across nodes.
Default: 1
elastic.config.index.number.of.shards
The number of shards (splits) for the Elasticsearch index.
Note: This is a one-time setting. An Elasticsearch index cannot be re-sharded; if you want to change the number of shards, after changing the setting you'll need to delete the /index folder that the search index is stored in and then rebuild the index.
Default: 2

For additional information about configuration settings in the Akana Administration Console, see Admin Console Settings

Back to top

Troubleshooting Elasticsearch:

Pre-upgrade search results are showing even though the index folder is populated with the Elasticsearch index: what do I do?

If you upgrade to Elasticsearch from a previous version of the product, and the index folder is populated with Elasticsearch results after upgrading, but Compass search results are still showing up, check these two fields in the TENANTS table in the database:

  • INDEX_TARGET: This determines what the platform uses for search indexing. To use Elasticsearch, make sure INDEX_TARGET is set to elastic.
  • SEARCH_SOURCE: This determines which search index the platform uses to deliver search results. It can have either of these values: elastic or compass. If the value is compass, the platform uses the Compass index even if there is an Elasticsearch index available also. To use Elasticsearch, make sure SEARCH_SOURCE is set to elastic.

Back to top

Elasticsearch client can't connect to the nodes

If the Elasticsearch client can't connect to the nodes, and/or the nodes can't connect to each other, check that you have the default Elasticsearch ports (all ports in the range 9300–9305) available on every container that the Elasticsearch feature is installed on.

For more information, see What are the default ports for Elasticsearch configuration?

Back to top