Elastic Stack安装 - Filebeat

in 编程
关注公众号【好便宜】( ID:haopianyi222 ),领红包啦~
阿里云,国内最大的云服务商,注册就送数千元优惠券:https://t.cn/AiQe5A0g
腾讯云,良心云,价格优惠: https://t.cn/AieHwwKl
搬瓦工,CN2 GIA 优质线路,搭梯子、海外建站推荐: https://t.cn/AieHwfX9

环境:

OS: Centos 7.6

JDK: 1.8.0_91

下载地址:

https://artifacts.elastic.co/downloads/beats/heartbeat/heartbeat-7.4.2-linux-x86_64.tar.gz
https://artifacts.elastic.co/downloads/beats/packetbeat/packetbeat-7.4.2-linux-x86_64.tar.gz
https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-7.4.2-linux-x86_64.tar.gz
https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.4.2-linux-x86_64.tar.gz
https://artifacts.elastic.co/downloads/logstash/logstash-7.4.2.tar.gz
https://artifacts.elastic.co/downloads/kibana/kibana-7.4.2-linux-x86_64.tar.gz
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.4.2-linux-x86_64.tar.gz

安装Filebeat


tar -xzvf filebeat-7.4.2-linux-x86_64.tar.gz
cd filebeat-7.4.2-linux-x86_64/

vim filebeat.yml
# 配置如下信息
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/*.log
  multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
  multiline.negate: true
  multiline.match: after
  multiline.timeout: 10s


output.elasticsearch:
  hosts: ["myEShost:9200"]
setup.kibana:
  host: "mykibanahost:5601"

#For example, the following command enables the system, nginx, and mysql modules:
./filebeat modules enable system nginx mysql

#Set up the initial environment:
./filebeat setup -e

# setup dashboards is needed
./filebeat setup --dashboards

# Run Filebeat.
./filebeat -e -c filebeat.yml -d "publish"

#for more:https://www.elastic.co/guide/en/beats/filebeat/6.4/filebeat-modules-quickstart.html

要了解关于安装和配置其他Beat的更多信息,请参阅入门文档:

Elastic Beats To capture
Auditbeat Audit data
Filebeat Log files
Heartbeat Availability monitoring
Metricbeat Metrics
Packetbeat Network traffic
Winlogbeat Windows event logs

 

匹配多行:

  multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
  multiline.negate: true
  multiline.match: after
  multiline.timeout: 10s

 完整配置:

filebeat.inputs:

# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.

- type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    #- /var/log/*.log
    - /data/logs-run/carte/server*.log
    #- c:\programdata\elasticsearch\logs\*

  # Exclude lines. A list of regular expressions to match. It drops the lines that are
  # matching any regular expression from the list.
  #exclude_lines: ['^DBG']

  # Include lines. A list of regular expressions to match. It exports the lines that are
  # matching any regular expression from the list.
  #include_lines: ['^ERR', '^WARN']

  # Exclude files. A list of regular expressions to match. Filebeat drops the files that
  # are matching any regular expression from the list. By default, no files are dropped.
  #exclude_files: ['.gz$']

  # Optional additional fields. These fields can be freely picked
  # to add additional information to the crawled log files for filtering
  #fields:
  #  level: debug
  #  review: 1

  ### Multiline options

  # Multiline can be used for log messages spanning multiple lines. This is common
  # for Java Stack Traces or C-Line Continuation

  # The regexp Pattern that has to be matched. The example pattern matches all lines starting with [
  #multiline.pattern: ^\[
  multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'

  # Defines if the pattern set under pattern should be negated or not. Default is false.
  multiline.negate: true

  # Match can be set to "after" or "before". It is used to define if lines should be append to a pattern
  # that was (not) matched before or after or as long as a pattern is not matched based on negate.
  # Note: After is the equivalent to previous and before is the equivalent to to next in Logstash
  multiline.match: after
  multiline.timeout: 10s

 

格式化日志: https://www.elastic.co/guide/en/beats/filebeat/6.4/dissect.html

processors:
- dissect:
    tokenizer: "%{key1} %{key2}"
    field: "message"
    target_prefix: "dissect"

完整配置信息. 

#================================ Processors =====================================

# Configure processors to enhance or manipulate events generated by the beat.

processors:
  #- add_host_metadata: ~
  #- add_cloud_metadata: ~
- dissect:
    tokenizer: "%{entry_datetime} [%{thread_info}] %{log_level}  -  [%{trans_id}] [%{task_id}] [%{task_uuid|mapping_id}] %{description}"
    field: "message"
    target_prefix: "out"

 

配置模板Index: https://www.elastic.co/guide/en/beats/filebeat/current/configuration-template.html

https://www.elastic.co/guide/en/beats/filebeat/6.4/filebeat-template.html#filebeat-template

#顶格
setup.template.name: "test-01"
setup.template.pattern: "test-01-*"

#重要, 必须配置, 否则不生效
setup.ilm.enabled: false


output.elasticsearch.index: "test-01-%{+yyyy.MM.dd}"
#或者配置成如下:
output.elasticsearch:
  hosts: ["192.168.1.163:9200"]
  index: "test-01-%{+yyyy.MM.dd}"

其它可行或者相关配置:

setup.ilm.enabled: false

#另一种配置方法
#setup.ilm.enabled: auto
#setup.ilm.rollover_alias: "filebeat-worker001"
#setup.ilm.pattern: "{now/d}-000001"

#其它可用配置
#setup.template.name: "UAT"
#setup.template.pattern: "UAT-*"
#setup.dashboards.index: "UAT-DSBS-*"
#setup.template.overwrite: true
#setup.template.enabled: false

 

 

 

关注公众号【好便宜】( ID:haopianyi222 ),领红包啦~
阿里云,国内最大的云服务商,注册就送数千元优惠券:https://t.cn/AiQe5A0g
腾讯云,良心云,价格优惠: https://t.cn/AieHwwKl
搬瓦工,CN2 GIA 优质线路,搭梯子、海外建站推荐: https://t.cn/AieHwfX9
扫一扫关注公众号添加购物返利助手,领红包
Comments are closed.

推荐使用阿里云服务器

超多优惠券

服务器最低一折,一年不到100!

朕已阅去看看