S4 Logs
Archive CloudWatch Logs to zstd S3
When does this pay back?
Example: $5,000/month in CloudWatch Logs spend
At $5,000/month in CloudWatch Logs spend, a 70–90% reduction is roughly $3,500–$4,500/mo — gross avoided usage charge before the S4 software fee, EC2, storage/transfer, and workload differences.
Estimate your savings
Enter your relevant monthly spend or usage for a rough estimate — no bill upload needed.
Cut your CloudWatch Logs bill 70–90%: multi-account drain to zstd S3, WORM/Object Lock compliance, and a cost dashboard.
S4 Logs Commercial archives CloudWatch Logs to zstd-compressed S3 and avoids ingest with a PutLogEvents-compatible gateway. The commercial layer adds AWS Organizations multi-account drain, S3 Object Lock (Governance and Compliance, SEC 17a-4) WORM retention, and a cost-savings plus restore-search dashboard. Archives stay readable with plain zstd and Athena — no lock-in.
The problem
CloudWatch Logs charges $0.50/GB to ingest and $0.03/GB-month to store. Most logs are written once and almost never read again, so you keep paying to store data nobody queries. Ingest you have already paid is not recoverable, so growing log volume pushes the bill up month after month.
How it works
- 1
Drain or bypass at the source
Drain existing log groups with Mode A, or point agents at the PutLogEvents-compatible gateway (Mode B) with an endpoint override only and no application changes.
- 2
Archive to zstd on S3
Events are written as standard zstd-compressed JSONL to S3 with byte-range and timestamp index sidecars, at storage costs one to two orders of magnitude below CloudWatch.
- 3
Search and restore on demand
Grep and restore read only the frames a query needs via S3 range reads, you can query the archive in place with Athena, and the dashboard shows your savings.
Highlights
AWS Organizations multi-account drain into one lock-in-free archive.
WORM / S3 Object Lock (Governance + Compliance), verified before locking.
Cost-savings dashboard + restore-search UI (single binary).
What's included
- PutLogEvents-compatible gateway plus Mode A drain and Mode B bypass routing
- AWS Organizations multi-account drain that fans out via STS assume-role into one aggregate bucket
- S3 Object Lock WORM retention in Governance and Compliance modes (SEC 17a-4) with a per-object audit log
- Cost-savings and restore-search dashboard as a single self-contained binary with embedded assets
- Archives stay plain zstd and Athena-readable, with no S4 tooling required to read your data
- The open-source s4logs CLI bundled on the AMI (drain, serve, grep, restore, report, plan)
- One-click CloudFormation that provisions an Object-Lock-enabled bucket, IAM role, and the dashboard host
Use cases
Centralize and archive CloudWatch Logs from every account in an AWS Organization into one bucket
Hold regulated logs under SEC 17a-4 WORM retention that no one, including root, can alter or delete early
Cut the bill on high-ingest accounts by routing log traffic through the gateway to skip the $0.50/GB ingest toll
Shrink the storage line on write-once, rarely-read log groups, then shorten CloudWatch retention safely
FAQ
How much will I actually save?
It depends on the mode. Mode B bypasses ingest, the dominant cost, so it can take roughly 70–90% off the bill because the $0.50/GB ingest charge dominates. Mode A on its own only cuts the storage line, about 50–70% on S3 Standard and up to ~90% with Glacier Instant Retrieval, and ingest you already paid is not recoverable.
Is my data locked into your product?
No. Archives are concatenated standard RFC 8878 zstd frames containing JSONL, readable with zstd -dc and queryable in place with Athena, no S4 tooling required. The on-disk format is frozen for the 1.x series, so cancelling the subscription leaves your data fully readable.
Does it satisfy compliance and WORM requirements?
Yes. The lock command applies S3 Object Lock in Governance or Compliance mode; Compliance mode is irreversible and cannot be deleted or shortened by anyone, including the root account, which supports regimes such as SEC 17a-4. Each object is integrity-checked against its manifest before locking.
Does it work across multiple AWS accounts?
Yes. The Organizations multi-account drain enumerates member accounts (all, by OU, or an explicit list), assumes a cross-account role into each via STS, and drains them into one aggregate bucket partitioned by account id.
Where does it run, and do my applications have to change?
It runs as an AMI in your own AWS account and VPC, billed hourly plus annual via Marketplace, with no license-key check and no data leaving your account. Applications do not change: the gateway speaks the CloudWatch Logs PutLogEvents wire protocol, so Fluent Bit, the CloudWatch Agent, or an SDK migrate with an endpoint override only.
Why it's cheaper
Assumes 5 TB / month of log ingest, 30-day hot retention, text logs that compress ~10× with zstd, in us-east-1.
- CloudWatch Logs ingest (5 TB × $0.50/GB)
- $2,560 / mo
- CloudWatch Logs storage (5 TB × 30 days)
- $155 / mo
- Monthly total
- $2,715 / mo
- S3 storage (0.5 TB compressed)
- $12 / mo
- S4 Logs instance (m5.large + software fee)
- $100 / mo
- Monthly total
- $112 / mo
Sizing the S4 Logs instance by ingest rate
| Ingest rate | Recommended instance | S4 Logs instance cost | Total vs. CloudWatch Logs |
|---|---|---|---|
| ~1 TB / mo | t3.medium | $45 / mo | $48 / mo (CWL $543, −91%) |
| ~5 TB / mo | m5.large | $100 / mo | $112 / mo (CWL $2,715, −96%) |
| ~50 TB / mo | m5.2xlarge | $310 / mo | $427 / mo (CWL $27,140, −98%) |
Illustrative example. CloudWatch Logs uses us-east-1 list pricing ($0.50/GB ingest + $0.03/GB-month storage); S3 and EC2 use on-demand pricing in the same region. The S4 Logs software fee is billed hourly via AWS Marketplace, separate from the EC2 charge. The 10× compression figure is mid-range for text logs under zstd (typical 4–10×). Actual savings depend on log content, retention window and how often you run long-tail searches.
Pricing model
Hourly software fee + EC2 (t3 / m5 class). Metered per instance type, annual option available.
Other S4 products
S4 — Squished S3
Transparent GPU S3-compression gateway
S4 Metrics
Govern CloudWatch metric cardinality
S4 NAT
Cost-optimized NAT for Amazon VPC