菜单
开源

drop

注意

Promtail 已被弃用,并将在 2026 年 2 月 28 日之前获得长期支持 (LTS)。Promtail 将于 2026 年 3 月 2 日到达生命周期结束 (EOL)。您可以在此处找到迁移资源。

drop 阶段是一个过滤阶段,允许您根据多个选项丢弃日志。

请注意,如果您提供多个选项,它们将被视为 AND 子句,即每个选项都必须为真才能丢弃日志。

如果您希望使用 OR 子句进行丢弃,则指定多个 drop 阶段。

下面有一些示例可以帮助解释。

Drop 阶段 Schema

yaml
drop:
  # Single name or names list of extracted data. If empty, uses the log message.
  [source: [<string>] | <string>]

  # Separator placed between concatenated extracted data names. The default separator is a semicolon.
  [separator: <string> | default = ";"]

  # RE2 regular expression. If `source` is provided and it's a list, the regex will attempt to match
  # the concatenated sources. If no source is provided, then the regex attempts
  # to match the log line.
  # If the provided regex matches the log line or the source, the line will be dropped.
  [expression: <string>]

  # value can only be specified when source is specified. If `source` is provided and it's a list,
  # the value will attempt to match the concatenated sources. It is an error to specify value and expression.
  # If the value provided is an exact match for the `source` the line will be dropped.
  [value: <string>]

  # older_than will be parsed as a Go duration: https://golang.ac.cn/pkg/time/#ParseDuration
  # If the log line timestamp is older than the current time minus the provided duration it will be dropped.
  [older_than: <duration>]

  # longer_than is a value in bytes, any log line longer than this value will be dropped.
  # Can be specified as an exact number of bytes in integer format: 8192
  # Or can be expressed with a suffix such as 8kb
  [longer_than: <string>|<int>]

  # Every time a log line is dropped the metric `logentry_dropped_lines_total`
  # will be incremented.  By default the reason label will be `drop_stage`
  # however you can optionally specify a custom value to be used in the `reason`
  # label of that metric here.
  [drop_counter_reason: <string> | default = "drop_stage"]

示例

以下是展示 drop 阶段用法的示例。

简单丢弃

简单的 drop 阶段配置仅指定一个选项,或在使用 source 选项时指定两个选项。

给定以下管道

yaml
- drop:
    source: ["level","msg"]

丢弃任何包含至少 levelmsg 提取数据字段的日志行。

正则表达式匹配行

此示例管道会丢弃任何包含子字符串“debug”的日志行

yaml
- drop:
    expression: ".*debug.*"

正则表达式匹配连接的源

给定以下管道

yaml
- json:
    expressions:
     level:
     msg:
- drop:
    source:     ["level","msg"]
    separator:   "#"
    expression:  "(error|ERROR)#.*\/loki\/api\/push.*"

丢弃这两行日志

{"time":"2019-01-01T01:00:00.000000001Z", "level": "error", "msg":"11.11.11.11 - "POST /loki/api/push/ HTTP/1.1" 200 932 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB6"}
{"time":"2019-01-01T01:00:00.000000001Z", "level": "ERROR", "msg":"11.11.11.11 - "POST /loki/api/push/ HTTP/1.1" 200 932 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB6"}

值匹配源

给定以下管道

yaml
- json:
    expressions:
     level:
     msg:
- drop:
    source: "level"
    value:  "error"

会丢弃此日志行

{"time":"2019-01-01T01:00:00.000000001Z", "level": "error", "msg":"11.11.11.11 - "POST /loki/api/push/ HTTP/1.1" 200 932 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB6"}

丢弃旧日志行

注意

为了让 older_than 生效,您必须在使用 drop 阶段之前使用 timestamp 阶段从摄入的日志行中设置时间戳。

给定以下管道

yaml
- json:
    expressions:
     time:
     msg:
- timestamp:
    source: time
    format: RFC3339
- drop:
    older_than: 24h
    drop_counter_reason: "line_too_old"

如果当前的摄入时间是 2020-08-12T12:00:00Z,则从文件读取时会丢弃此日志行

{"time":"2020-08-11T11:00:00Z", "level": "error", "msg":"11.11.11.11 - "POST /loki/api/push/ HTTP/1.1" 200 932 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB6"}

但不会丢弃此日志行

{"time":"2020-08-11T13:00:00Z", "level": "error", "msg":"11.11.11.11 - "POST /loki/api/push/ HTTP/1.1" 200 932 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB6"}

在此示例中,当前时间是 2020-08-12T12:00:00Z,并且 older_than 是 24 小时。所有时间戳早于 2020-08-11T12:00:00Z 的日志行都将被丢弃。

此 drop 阶段丢弃的所有行也会增加 logentry_dropped_lines_total 指标,并带有标签 reason="line_too_old"

丢弃长日志行

给定以下管道

yaml
- drop:
    longer_than: 8kb
    drop_counter_reason: "line_too_long"

会丢弃任何超过 8kb 的日志行,这在 Loki 会因日志行过长而拒绝时非常有用。

此 drop 阶段丢弃的所有行也会增加 logentry_dropped_lines_total 指标,并带有标签 reason="line_too_long"

复杂丢弃

复杂的 drop 阶段配置可以在一个阶段中指定多个选项,或者指定多个 drop 阶段

按正则表达式和长度丢弃日志 (AND 条件)

给定以下管道

yaml
- drop:
    expression: ".*debug.*"
    longer_than: 1kb

会丢弃所有包含单词 'debug' 且长度超过 1kb 的日志行

按时间、长度或正则表达式丢弃日志 (OR 条件)

给定以下管道

yaml
- json:
    expressions:
     time:
     msg:
- timestamp:
    source: time
    format: RFC3339
- drop:
    older_than: 24h
- drop:
    longer_than: 8kb
- drop:
    source: "msg"
    expression: ".*trace.*"

会丢弃所有早于 24 小时 OR 长度超过 8kb OR json msg 字段包含单词 'trace' 的日志行