OpenSearch Sizing Calculator

Worked example: Amazon OpenSearch Service, hot only

Reference scenario the calculator's defaults: 250 GB/day of source data, 90 days kept hot, 1 replica, index / source factor 1.1 and 25% growth. Open this scenario.

AWS simplifies the same formula to source × (1 + replicas) × 1.45. The exact terms give a multiplier of 2.9 for one replica.

Reference scenarios

Reference scenario comparison points, not recommended layouts. Index / source factor 1.1, 1 replica, 25% growth, 85% planning target for the self-managed row.

Keeping a year hot needs 330.2 TB; moving 351 days to S3-backed tiers needs 133.3 TB, 2.48× less, because warm and cold data carry no replicas and none of the hot-storage overhead. The same 90 days on a self-managed cluster need 72.8 TB: the AWS service overhead does not apply there, but a planning utilization target does.

Shard planning example

OpenSearch documents 10–50 GB per shard as the rule of thumb (AWS: 10–30 GiB for latency-sensitive search, 30–50 GiB for write-heavy log analytics). The per-node shard limit depends on deployment and version. The planner divides the indexed data of each rollover by a target shard size and rounds up.

Illustrative example 2 GB/day with daily rollover indexes 2.2 GB per rollover, so each shard is about 2.2 GB, below the 10 GB rule of thumb, and 90 days produce 180 shards on data nodes. Rolling over every 14 days gives shards of about 15.4 GB and 28 shards. Open the 14-day rollover.

Common mistakes

1. Applying the Amazon formula to a self-managed cluster, or the reverse. The 5% Linux and 20% service overhead terms are AWS's. On the 250 GB/day scenario the Amazon service model gives 81.4 TB and the self-managed model 72.8 TB. Pick the deployment that matches yours.
2. Giving UltraWarm and cold storage replicas. They are backed by S3 and count primary shards only. For the one-year tiered scenario, UltraWarm needs 26.1 TB; counting a replica would show 52.3 TB.
3. Keeping a year of rarely searched data hot. All hot, a year needs 330.2 TB; tiered it needs 133.3 TB.
4. Planning with zero replicas. Hypothetical example with no replica the 250 GB/day scenario needs 40.7 TB instead of 81.4 TB, but a node loss then loses data. AWS recommends at least one replica.
5. Assuming no indexing overhead. Hypothetical example with a factor of 1.0 instead of the documented 1.1 the 250 GB/day scenario shrinks to 74 TB, 10% short of the AWS guidance. Measure your own factor.
6. Daily indexes for small volumes. Many small shards add per-shard overhead: 180 shards of about 2.2 GB, against 28 with a 14-day rollover.
7. Mixing open source and Amazon service defaults. Open source OpenSearch creates one primary and one replica by default; AWS documents that the Amazon service divides an index into five primary shards by default. Set the shard count explicitly before indexing.

How OpenSearch sizing is calculated

Amazon OpenSearch Service
  source(tier)   = daily ingest × tier days × (1 + growth)
  hot to provision = source(hot) × (1 + replicas) × index factor ÷ 0.95 ÷ 0.80
  UltraWarm        = source(warm) × index factor        primary shards only
  cold storage     = source(cold) × index factor        primary shards only

Self-managed OpenSearch (hot only)
  disk to provision = source × (1 + replicas) × index factor ÷ utilization target (85%)

shards per rollover = ceil(daily ingest × index factor × rollover days ÷ target shard size)

• Decimal units: 1,000 GB = 1 TB.
• The 0.95 and 0.80 terms are Linux reserved space and the service's reserved storage, applied as AWS's worst case.
• Not modeled: node and instance counts, EBS volume types, UltraWarm node counts, the 25-shards-per-GiB-of-heap guideline and self-managed warm tiers.

The common model behind every calculator is in the methodology.