Elastic SIEM Part I - Install and Configure Elasticsearch and Kibana on Debian Linux

Elasticsearch logo

A SIEM for Security information and event management is a security solution which centralizes the logs and events of numerous devices in a computing network in order to process them and generate alerts in the event that abnormal behavior is detected.

We will see here how to deploy Elasticsearch solution.

Elastic SIEM Architecture

Elasticsearch is a real-time, distributed storage, search, and analytics engine.

Kibana is an open source analytics and visualization platform designed to work with Elasticsearch. Kibana will be used to search, view, and interact with data stored in Elasticsearch indices.

SIEM | Elasticsearch architecture

Notes

Links

Components

Main

  • Elasticsearch : Distributed, RESTful search and analytics.
  • Kibana : Visualize your data. Navigate the Stack.
  • Beats : Collect, parse, and ship in a lightweight fashion.

Others

  • Logstash : Ingest, transform, enrich, and output.
  • Filebeat : Real-time insight into log data.

Ports

  • ElastiSearch default port : http://IP_ADDRESS:9200
  • Kibana web access : http://IP_ADDRESS:5601
  • Logstash default port : 9600

Installing Elasticsearch (Debian Server)

We can install it, on a Debian system from raw packages or via Package Managers. We will see both methods.

Installing From raw packages (dpkg)

Elasticsearch

  • Download Elasticsearch package and the associate sha512 message digest to /tmp/ :
root@host:~# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.12.0-amd64.deb https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.12.0-amd64.deb.sha512 -P /tmp/
  • Install Elasticsearch service :
root@host:~# dpkg -i /tmp/elasticsearch-7.12.0-amd64.deb

Kibana

  • Download Kibana package and the associate sha512 message digest to /tmp/ :
root@host:~# wget https://artifacts.elastic.co/downloads/kibana/kibana-7.12.0-amd64.deb https://artifacts.elastic.co/downloads/kibana/kibana-7.12.0-amd64.deb.sha512 -P /tmp/
  • Install Kibana service :
root@host:~# dpkg -i /tmp/kibana-7.12.0-amd64.deb

Installing From Package Managers (apt)

Prerequisites

  • Import the Elasticsearch PGP key :
root@host:~# wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
  • Install apt-transport-https :
root@host:~# apt update && apt-get install apt-transport-https
  • Save the repository definition :
root@host:~# echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-7.x.list

Elasticsearch

  • Install Elasticsearch :
root@host:~# apt update && apt-get install elasticsearch

Kibana

  • Install Kibana :
root@host:~# apt update && apt-get install kibana

Configuring

Elasticsearch

  • Edit /etc/elasticsearch/elasticsearch.yml to :
    • Set the bind address to all : 0.0.0.0
    • Declare one node configuration : single-node
    • List of hosts : ["0.0.0.0"]
network.host: 0.0.0.0
discovery.type: single-node
discovery.seed_hosts: ["0.0.0.0"]
  • Still in file configuration we can also set the name for the node :
node.name: std
  • Start service :
root@host:~# systemctl start elasticsearch.service
  • Check elastic service is running :
root@host:~# curl http://127.0.0.1:9200
{
  "name" : "std",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "StdgreaTBanDKphU4S0ceg",
  "version" : {
    "number" : "7.12.0",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "ff17057114c2199c9c1bbecc727003a907c0db7a",
    "build_date" : "2021-02-15T13:44:09.394032Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Kibana

  • Edit /etc/kibana/kibana.yml to :
    • Set the bind address to all : 0.0.0.0
server.host: "0.0.0.0"
  • Start service :
root@host:~# systemctl start kibana.service
  • Check everything is fine :
root@host:~# curl -XGET 'http://localhost:9200/_cluster/health'
{"cluster_name":"elasticsearch","status":"yellow","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":18,"active_shards":18,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":4,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":81.81818181818183}
  • Open Firefox and go to http://IP_SERVER:5601/, after few seconds, you should be able to see this web page :
ElasticSearch | Kibana : select your space

AutoStart

To make our Elasticsearch and Kibana services start at boot, we need to do some modifications.

Elasticsearch

  • Because it's slow to start, edit /usr/lib/systemd/system/elasticsearch.service file, and change the TimeoutStartSec value :
TimeoutStartSec=300
  • Set Elasticsearch service start when the server boots :
root@host:~# systemctl enable elasticsearch.service

Kibana

  • Set Kibana service start when the server boots :
root@host:~# systemctl enable kibana.service

Log Files

  • Elasticsearch :
root@host:~# tail /var/log/elasticsearch/elasticsearch.log
  • Kibana :
root@host:~# tail /var/log/kibana/kibana.log
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Contact :