...

• Not all OpenSearch API operations supported

• can't take snapshot or restore snapshots

• index refresh interval varies between 10-30 seconds, depending on size of requests

  • opposed to the 10 sec default in Elasticsearch

  • might affect user_rule processing

• OpenSearch Serverless automatically upgrades your collections to new OpenSearch versions. Upgrades don't necessarily happen as soon as a new version is available.

  • this can be potentially bad as it can break HySDS if it doesnt support the latest version

• Not a mature product, still in a preview mode for users

  • documentation a little incomplete

  • will have to test out features to see if they exist in OpenSearch Serverless

Still needs testing

• HySDS UI

  • currently we need a reverse proxy for hysds_ui to communicate with AWS ES

    • we use a forked version of aws-es-proxy: https://github.com/hysds/aws-es-proxy

    • will need to make changes to code to accommodate for OpenSearch Serverless

      • have a flag for --service which can be set to aoss (default will be es)

    • Logstash

      • Code Block
        logstash-plugin install --version 0.1.0.pre logstash-integration-aws logstash-plugin install --version 2.0.0 logstash-output-opensearch

• will need to test if the existing elasticsearch-python (>=7.0.0,<7.14.0) will work with OpenSearch Serverless

  • https://elasticsearch-py.readthedocs.io/en/v8.5.2/#

• Deep pagination:

  • Starting from ElasticSearch 7.0, the limit of records returned by the _search API is 10,000 records

    • This led to us relying on the scroll API for deep pagination, and eventually the PIT + search_after API

      • not sure if that still applies to OpenSearch Serverless, or if they do not impose a hard limit on the _search API

    • a lot of HySDS core + PCM code uses it

...