Writing Logs to ELK Stack
ELK Stack is the acronym that represents three open source projects: Elasticsearch, Logstash, and Kibana. Elasticsearch is a search and analytics engine. Logstash is a server‑side data processing pipeline that ingests data from multiple sources simultaneously, transforms it, and then sends it to a “stash” like Elasticsearch. Kibana lets users visualize data with charts and graphs in Elasticsearch.
Logstash supports a variety of inputs that pull in all events from a multitude of common sources simultaneously. For more detailed documentation, check the Logstash product page.
Configure Monolog Logstash formatter in your
# path/to/config_prod.yml monolog: handlers: consumer: type: stream path: "%kernel.logs_dir%/consumer.log" level: debug channel: ["consumer"] formatter: monolog.formatter.logstash
Logs will be formatted like:
Filebeat helps you keep the simple things simple by offering a lightweight way to forward and centralize logs and files. Filebeat comes with internal modules (auditd, Apache, NGINX, System, MySQL, and more) that simplify the collection, parsing, and visualization of common log formats down to a single command.
output.logstash enabled: true hosts: ["localhost:5044"] filebeat.prospectors: - type: log paths: - /path/to/the/application/var/logs/consumer*.log json: keys_under_root: true overwrite_keys: true
Kibana works in pair with Elasticsearch and gives the possibility to search, view, and interact with data stored in Elasticsearch indices. You can easily perform advanced data analysis and visualize your data in a variety of charts, tables, and maps.
- Interactive charts
- Mapping support
- Pre-built aggregations and filters
- Easily Accessible dashboards
You can easily set up dashboards and reports and share them with others. All you need is a browser to view and explore the data.
Using Kibana’s pre-built aggregations and filters, you can run a variety of analytics like histograms, top-N queries, and trends with just a few clicks.
For more information, see the following external resources: