Upgrade Standard Index to Elasticsearch 7.*
You have two options to perform the upgrade: either via full reindexation or via search index dump.
Keep in mind that Elasticsearch 7.* can be used with version 3.1 as well, but in this case application uses old index structure (with mapping types). If you need to upgrade to the current version from a such application, then you should change index prefix in config/parameters.yml file during the upgrade and allow application to create new indices with a new structure (without mapping types). Both upgrade options can be used to do that. Indices with old structure (with old prefix) should be removed after the upgrade.
This option is suitable for upgrades from version lower than 2.6, or if you have a small number of entities (fewer than a hundred thousand).
Search index upgrade is a part of the application upgrade.
So, once you have turned on maintenance mode through
app/console lexik:maintenance:lock --env=prod, you need to perform the following actions:
- Stop old Elasticsearch
- Modify credentials for search engine configuration in the config/parameters.yml file
- Start the Elasticsearch 7.* service.
Proceed with the standard upgrade procedure.
Search Index Dump
Search index dump is suitable only if you perform upgrade from versions 2.6 to 3.1, and you have a large number of entities. The biggest advantage of this approach is that you do not need to schedule reindexation and wait until it is finished.
Generating the search index dump is also a part of standard procedure of application upgrade. But you should note that the elastic index dump must be created from the old version of the code (2.6 to 3.1). So follow next step of upgrade procedure:
Turn on maintenance mode to switch the application to the maintenance mode.
app/console lexik:maintenance:lock --env=prod
bin/console lexik:maintenance:lock --env=prod
Create Elastic search index dump. Consider you must do this before updating code to new version.
app/console oro:elasticsearch:dump-standard-index elasticsearch7 standard-index-es7.dump --env=prod
bin/console oro:elasticsearch:dump-standard-index elasticsearch7 standard-index-es7.dump --env=prod
It creates the standard-index-es7.dump file (in application directory) with search index dump in the Elasticsearch bulk API format which is applicable for Elasticsearch version 7.*.
Here is an example:
- Stop old Elasticsearch service.
- Proceed with the standard upgrade procedure which includes creating needed backups and updating code to new version, updating composer dependencies (all actions needed before running update command).
5. Then modify credentials for search engine configuration in the config/parameters.yml file. Consider you should do this after updating code to new version.
Start the Elasticsearch 7.* service.
Execute update command from standard upgrade procedure but pay attention to skip-search-reindexation (it will prevent full reindexation start):
bin/console oro:platform:update --skip-search-reindexation --env=prod
Now you need to execute command which will create an empty indexes (without any data) with correct elastic search mappings:
bin/console oro:elasticsearch:create-standard-index --env=prod
Upload the dump data to the Elasticsearch 7.* index, the Elasticsearch 7.* bulk API, and the dump file created previously using a standard curl CLI command:
curl -XPOST http://localhost:9200/_bulk -H 'Content-Type: application/json' --data-binary @standard-index-es7.dump > /dev/null
To speed up this process you may split the dump file into smaller chunks and upload them in parallel. In this case, each chunk has to contain an even number of lines because each document is represented by two lines in the dump file.
- Finish the standard upgrade procedure.
You may adjust this procedure according to your needs, but keep in mind that you need to:
- create index dump before upgrading to 4.+ and ensure that old Elasticsearch service is running at this time;
- create and upload index dump during maintenance mode to avoid data loss.