Recent elasticsearch licensing change ensures that the Beats modules are sending data to an officially supported versions of Elasticsearch and Kibana where Elastic can attest to the quality and scale of the products. Does AWS have any plans to fork a version filebeat?
https://www.elastic.co/guide/en/beats/libbeat/current/breaking-changes-7.13.html
https://www.reddit.com/r/aws/comments/nn95aq/elastic_has_broken_filebeat_as_of_713_it_no/
What are the alternatives?
- Host elasticsearch on EC2 instances, why not?
- CloudWatch, slow… https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_ES_Stream.html
- Kubernetes options like ECK or helm chart deployments…tricky for production usage
- Loki? Is it mature enough?
UPDATE 2021-07-12:
OpenSearch 1.0 launches with Apache License, Version 2.0 and AWS huge contribution efforts
https://aws.amazon.com/blogs/opensource/opensearch-1-0-launches/
UPDATE 2024-04-21:
- Quickwit - very interesting alternative for common logging tools written on rust
- Elasticsearch can be deployed with helm chart but based on experience you should consider using single-AZ. This might relevant to logs which are not as critical as business data.
- Loki is pretty good solution, it integrates well with Grafana Tempo for tracing purposes and Opentelemetry. But based on my experience, developers for some reasons don’t love search capabilities in Grafana Loki as it’s available in kibana or other commercial logging solutions like datadog.