ElasticSearch+Filebeat+Kibana 搭建手记

2019-04-10 41次浏览 发表评论

实验环境:

  • 192.168.0.4:安装 ElasticSearch、Kibana
  • 192.168.0.6:安装 Filebeat、Nginx

系统环境准备

安装Java环境:

tar zxvf jdk-8u151-linux-x64.tar.gz
mv jdk1.8.0_151/ /usr/local/jdk

vim /etc/profile
#Add JDK
export JAVA_HOME=/usr/local/jdk
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib

source /etc/profil

node环境:

wget https://npm.taobao.org/mirrors/node/latest-v4.x/node-v4.2.1-linux-x64.tar.x
xz -d node-v4.2.1-linux-x64.tar.xz && tar xvf node-v4.2.1-linux-x64.tar
mv node-v4.2.1-linux-x64 /usr/local/node
ln -s /usr/local/node/bin/* /usr/local/bin

ElasticSearch 和 Kibana 安装

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.7.0.rpm
rpm -ivh elasticsearch-6.7.0.rpm

wget http://47.75.109.163/kibana-6.7.0-x86_64.rpm
rpm -ivh kibana-6.7.0-x86_64.rpm

ElasticSearch 配置文件:

vim /etc/elasticsearch/elasticsearch.yml
cluster.name: my-application
node.name: node-1
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
http.port: 9200

Kibana 配置文件:

vim /etc/kibana/kibana.yml   
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: ["http://localhost:9200"]

汉化 Kibana:

Kibana 中文汉化地址: https://github.com/anbai-inc/Kibana_Hanization

git clone https://github.com/anbai-inc/Kibana_Hanization.git
mv Kibana_Hanization/translations /usr/share/kibana/src/legacy/core_plugins/kibana/

在 kibana.yml 中添加:
i18n.locale: "zh_CN

启动:

systemctl enable elasticsearch.service
systemctl start elasticsearch

systemctl enable kibana.service
systemctl start kibana

安装 Filebeat

wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.7.0-x86_64.rpm
rpm -ivh filebeat-6.7.0-x86_64.rpm

filebeat 配置文件:

# 配置输入
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /home/wwwlogs/access.log
  json.keys_under_root: true
  json.add_error_key: true
  fields:
    source: 'nginx-access'

- type: log
  enabled: true
  paths:
    - /home/wwwlogs/kibana.log
  json.keys_under_root: true
  json.add_error_key: true
  fields:
    source: 'kibana-access'

# 配置输出
output.elasticsearch:
  hosts: ["192.168.0.4:9200"]
  indices:
    - index: "%{[fields.source]}-%{+yyyy.MM.dd}"
      when.contains:
        fields.source: "nginx-access"
    - index: "%{[fields.source]}-%{+yyyy.MM.dd}"
      when.contains:
        fields.source: "kibana-access"

修改好配置文件,重启生效:

systemctl restart filebeat 

收集nginx日志

修改nginx日志为JSON格式:

log_format json escape=json '{ "@timestamp": "$time_iso8601", '
                             '"time": "$time_iso8601", '
                             '"remote_addr": "$remote_addr", '
                             '"remote_user": "$remote_user", '
                             '"body_bytes_sent": "$body_bytes_sent", '
                             '"request_time": "$request_time", '
                             '"status": "$status", '
                             '"host": "$host", '
                             '"request": "$request", '
                             '"request_method": "$request_method", '
                             '"uri": "$uri", '
                             '"http_referrer": "$http_referer", '
                             '"body_bytes_sent":"$body_bytes_sent", '
                             '"http_x_forwarded_for": "$http_x_forwarded_for", '
                             '"http_user_agent": "$http_user_agent" '
                            '}';


评论

随便说点,聊聊天~