Cost

OpenSearch serverless needs a "server" running called a OCU (OpenSearch Compute Unit)
- billed at $0.24/hr, with a minimum of 2 OCUs running at once
- “Each OCU is a combination of 6 GiB of memory and corresponding virtual CPU (vCPU), as well as data transfer to Amazon S3.”
- The cost per month of data stored in Amazon S3 is $0.024 per GB
- no charges for OpenSearch fashboard usage

Supported Regions

as of Dec '23, it is supported in these regions

Region Name	Region	Endpoint	Protocol

Region Name	Region	Endpoint	Protocol
US East (Ohio)	us-east-2	aoss.us-east-2.amazonaws.com	HTTPS
US East (N. Virginia)	us-east-1	aoss.us-east-1.amazonaws.com	HTTPS
US West (Oregon)	us-west-2	aoss.us-west-2.amazonaws.com	HTTPS
Asia Pacific (Tokyo)	ap-northeast-1	aoss.ap-northeast-1.amazonaws.com	HTTPS
Europe (Ireland)	eu-west-1	aoss.eu-west-1.amazonaws.com	HTTPS

Benefits

Simpler than provisioned – OpenSearch Serverless removes much of the complexity of managing OpenSearch clusters and capacity. It automatically sizes and tunes your clusters, and takes care of shard and index lifecycle management. It also manages service software updates and OpenSearch version upgrades. All updates and upgrades are non-disruptive.
- Cost-effective – When you use OpenSearch Serverless, you only pay for the resources that you consume. This removes the need for upfront provisioning and overprovisioning for peak workloads.
- Highly available – OpenSearch Serverless supports production workloads with redundancy to protect against Availability Zone outages and infrastructure failures.
- Scalable – OpenSearch Serverless automatically scales resources to maintain consistently fast data ingestion rates and query response times.

Logstash

You must use version 2.0.0 or later of the logstash-output-opensearch plugin to publish logs to OpenSearch Serverless collections.
- https://github.com/opensearch-project/logstash-output-opensearch
- unsure if the OpenSearch logstash plugin works with other plugins such as redis, etc.

input {
  s3  {
    bucket => "my-s3-bucket"
    region => "us-east-1"
  }
}

output {
  opensearch {
    ecs_compatibility => disabled
    hosts => "https://my-collection-endpoint.us-east-1.aoss.amazonaws.com:443"
    index => my-index
    auth_type => {
      type => 'aws_iam'
      aws_access_key_id => 'your-access-key'
      aws_secret_access_key => 'your-secret-key'
      region => 'us-east-1'
      service_name => 'aoss'
    }
    default_server_major_version => 2
    legacy_template => false
  }
}

Signing Requests

You must specify the service name as aoss.
You can't include Content-Length as a signed header, otherwise you'll get an invalid signature error.
The x-amz-content-sha256 header is required for all AWS Signature Version 4 requests. It provides a hash of the request payload. For OpenSearch Serverless, include it with one of these values when you build the canonical request for signing:
If there's a request payload, set the value to its Secure Hash Algorithm (SHA) cryptographic hash (SHA256).
If there's no request payload, set the value to e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855, which is the hash of an empty string.
In either of the above two cases, you can also use the literal string UNSIGNED-PAYLOAD as the value of the x-amz-content-sha256 header.

API Operations

Not sure if it supports certain endpoints, such as:

Point-in-time (PIT) API: POST /<index>/_pit?keep_alive=1m
Scroll API: GET /_search/scroll

`aoss:CreateIndex`	PUT <index>	Create indexes. For more information, see Create index.
`aoss:DescribeIndex`	GET <index> GET <index>/_mapping GET <index>/_mappings GET <index>/_setting GET <index>/_setting/<setting> GET <index>/_settings GET <index>/_settings/<setting> GET _cat/indices GET _mapping GET _mappings GET _resolve/index/<index>	Describe indexes. For more information, see the following resources: Get index Get a mapping Get settings CAT indices
`aoss:WriteDocument`	DELETE <index>/_doc/<id> (for search collection types only) POST <index>/_bulk POST <index>/_create/<id> (for search collection types only) POST <index>/_doc POST <index>/_update/<id> (for search collection types only) POST _bulk PUT <index>/_create/<id> (for search collection types only) PUT <index>/_doc/<id> (for search collection types only)	Write and update documents. For more information, see the following resources: Bulk Index data Note Some operations are only allowed for collections of type `SEARCH`. For more information, see Choosing a collection type.
`aoss:ReadDocument`	GET <index>/_analyze GET <index>/_doc/<id> GET <index>/_explain/<id> GET <index>/_mget GET <index>/_source/<id> GET <index>/_count GET <index>/_field_caps GET <index>/_msearch GET <index>/_rank_eval GET <index>/_search GET <index>/_validate/<query> GET _analyze GET _field_caps GET _mget GET _search HEAD <index>/_doc/<id> HEAD <index>/_source/<id> POST <index>/_analyze POST <index>/_explain/<id> POST <index>/_count POST <index>/_field_caps POST <index>/_rank_eval POST <index>/_search POST _analyze POST _field_caps POST _search	Read documents. For more information, see the following resources: Perform text analysis Get document Count Query DSL Ranking evaluation Analyze API Explain
`aoss:DeleteIndex`	DELETE <target>	Delete indexes. For more information, see Delete index.
`aoss:UpdateIndex`	POST _mapping POST <index>/_mapping/ POST <index>/_mappings/ POST <index>/_setting POST <index>/_settings POST _setting POST _settings PUT _mapping PUT <index>/_mapping PUT <index>/_mappings/ PUT <index>/_setting PUT <index>/_settings PUT _setting PUT _settings	Update index settings. For more information, see the following resources: Mapping Update settings
`aoss:CreateCollectionItems`	POST _aliases	Create index aliases. For more information, see Create aliases.
`aoss:DescribeCollectionItems`	GET <index>/_alias/<alias> GET _alias GET _alias/<alias> GET _cat/aliases GET _cat/templates GET _cat/templates/<template_name> GET _component_template GET _component_template/<component-template> GET _index_template GET _index_template/<index-template> HEAD _alias/<alias> HEAD _component_template/<component-template> HEAD _index_template/<name> HEAD <index>/_alias/<alias>	Describe aliases and index templates. For more information, see the following resources: Manage aliases Index templates
`aoss:UpdateCollectionItems`	POST <index>/_alias/<alias> POST <index>/_aliases/<alias> POST _component_template/<component-template> POST _index_template/<index-template> PUT <index>/_alias/<alias> PUT <index>/_aliases/<alias> PUT _component_template/<component-template> PUT _index_template/<index-template>	Update aliases and index templates. For more information, see the following resources: Index aliases Index templates
`aoss:DeleteCollectionItems`	DELETE <index>/_alias/<alias> DELETE _component_template/<component-template> DELETE _index_template/<index-template> DELETE <index>/_aliases/<alias>	Delete aliases and index templates. For more information, see the following resources: Delete aliases Delete a template

Drawbacks

Not all OpenSearch API operations supported
can't take snapshot or restore snapshots
index refresh interval varies between 10-30 seconds, depending on size of requests
- opposed to the 10 sec default in Elasticsearch
- might affect user_rule processing
OpenSearch Serverless automatically upgrades your collections to new OpenSearch versions. Upgrades don't necessarily happen as soon as a new version is available.
- this can be potentially bad as it can break HySDS if it doesnt support the latest version
Not a mature product, still in a preview mode for users
- documentation a little incomplete
- will have to test out features to see if they exist in OpenSearch Serverless

Still needs testing

HySDS UI
- currently we need a reverse proxy for hysds_ui to communicate with AWS ES
  - we use a forked version of aws-es-proxy: https://github.com/hysds/aws-es-proxy
  - will need to make changes to code to accommodate for OpenSearch Serverless
    - have a flag for --service which can be set to aoss (default will be es)
  - Logstash
    - logstash-plugin install --version 0.1.0.pre logstash-integration-aws logstash-plugin install --version 2.0.0 logstash-output-opensearch
will need to test if the existing elasticsearch-python (>=7.0.0,<7.14.0) will work with OpenSearch Serverless
Deep pagination:
- Starting from ElasticSearch 7.0, the limit of records returned by the _search API is 10,000 records
  - This led to us relying on the scroll API for deep pagination, and eventually the PIT + search_after API
    - not sure if that still applies to OpenSearch Serverless, or if they do not impose a hard limit on the _search API
  - a lot of HySDS core + PCM code uses it