drop
注意
Promtail 已被弃用,并将在 2026 年 2 月 28 日之前获得长期支持 (LTS)。Promtail 将于 2026 年 3 月 2 日到达生命周期结束 (EOL)。您可以在此处找到迁移资源。
drop
阶段是一个过滤阶段,允许您根据多个选项丢弃日志。
请注意,如果您提供多个选项,它们将被视为 AND 子句,即每个选项都必须为真才能丢弃日志。
如果您希望使用 OR 子句进行丢弃,则指定多个 drop 阶段。
下面有一些示例可以帮助解释。
Drop 阶段 Schema
drop:
# Single name or names list of extracted data. If empty, uses the log message.
[source: [<string>] | <string>]
# Separator placed between concatenated extracted data names. The default separator is a semicolon.
[separator: <string> | default = ";"]
# RE2 regular expression. If `source` is provided and it's a list, the regex will attempt to match
# the concatenated sources. If no source is provided, then the regex attempts
# to match the log line.
# If the provided regex matches the log line or the source, the line will be dropped.
[expression: <string>]
# value can only be specified when source is specified. If `source` is provided and it's a list,
# the value will attempt to match the concatenated sources. It is an error to specify value and expression.
# If the value provided is an exact match for the `source` the line will be dropped.
[value: <string>]
# older_than will be parsed as a Go duration: https://golang.ac.cn/pkg/time/#ParseDuration
# If the log line timestamp is older than the current time minus the provided duration it will be dropped.
[older_than: <duration>]
# longer_than is a value in bytes, any log line longer than this value will be dropped.
# Can be specified as an exact number of bytes in integer format: 8192
# Or can be expressed with a suffix such as 8kb
[longer_than: <string>|<int>]
# Every time a log line is dropped the metric `logentry_dropped_lines_total`
# will be incremented. By default the reason label will be `drop_stage`
# however you can optionally specify a custom value to be used in the `reason`
# label of that metric here.
[drop_counter_reason: <string> | default = "drop_stage"]
示例
以下是展示 drop
阶段用法的示例。
简单丢弃
简单的 drop
阶段配置仅指定一个选项,或在使用 source
选项时指定两个选项。
给定以下管道
- drop:
source: ["level","msg"]
丢弃任何包含至少 level
或 msg
提取数据字段的日志行。
正则表达式匹配行
此示例管道会丢弃任何包含子字符串“debug”的日志行
- drop:
expression: ".*debug.*"
正则表达式匹配连接的源
给定以下管道
- json:
expressions:
level:
msg:
- drop:
source: ["level","msg"]
separator: "#"
expression: "(error|ERROR)#.*\/loki\/api\/push.*"
丢弃这两行日志
{"time":"2019-01-01T01:00:00.000000001Z", "level": "error", "msg":"11.11.11.11 - "POST /loki/api/push/ HTTP/1.1" 200 932 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB6"}
{"time":"2019-01-01T01:00:00.000000001Z", "level": "ERROR", "msg":"11.11.11.11 - "POST /loki/api/push/ HTTP/1.1" 200 932 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB6"}
值匹配源
给定以下管道
- json:
expressions:
level:
msg:
- drop:
source: "level"
value: "error"
会丢弃此日志行
{"time":"2019-01-01T01:00:00.000000001Z", "level": "error", "msg":"11.11.11.11 - "POST /loki/api/push/ HTTP/1.1" 200 932 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB6"}
丢弃旧日志行
注意
为了让
older_than
生效,您必须在使用drop
阶段之前使用 timestamp 阶段从摄入的日志行中设置时间戳。
给定以下管道
- json:
expressions:
time:
msg:
- timestamp:
source: time
format: RFC3339
- drop:
older_than: 24h
drop_counter_reason: "line_too_old"
如果当前的摄入时间是 2020-08-12T12:00:00Z,则从文件读取时会丢弃此日志行
{"time":"2020-08-11T11:00:00Z", "level": "error", "msg":"11.11.11.11 - "POST /loki/api/push/ HTTP/1.1" 200 932 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB6"}
但不会丢弃此日志行
{"time":"2020-08-11T13:00:00Z", "level": "error", "msg":"11.11.11.11 - "POST /loki/api/push/ HTTP/1.1" 200 932 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB6"}
在此示例中,当前时间是 2020-08-12T12:00:00Z,并且 older_than
是 24 小时。所有时间戳早于 2020-08-11T12:00:00Z 的日志行都将被丢弃。
此 drop 阶段丢弃的所有行也会增加 logentry_dropped_lines_total
指标,并带有标签 reason="line_too_old"
丢弃长日志行
给定以下管道
- drop:
longer_than: 8kb
drop_counter_reason: "line_too_long"
会丢弃任何超过 8kb 的日志行,这在 Loki 会因日志行过长而拒绝时非常有用。
此 drop 阶段丢弃的所有行也会增加 logentry_dropped_lines_total
指标,并带有标签 reason="line_too_long"
复杂丢弃
复杂的 drop
阶段配置可以在一个阶段中指定多个选项,或者指定多个 drop 阶段
按正则表达式和长度丢弃日志 (AND 条件)
给定以下管道
- drop:
expression: ".*debug.*"
longer_than: 1kb
会丢弃所有包含单词 'debug' 且长度超过 1kb 的日志行
按时间、长度或正则表达式丢弃日志 (OR 条件)
给定以下管道
- json:
expressions:
time:
msg:
- timestamp:
source: time
format: RFC3339
- drop:
older_than: 24h
- drop:
longer_than: 8kb
- drop:
source: "msg"
expression: ".*trace.*"
会丢弃所有早于 24 小时 OR 长度超过 8kb OR json msg
字段包含单词 'trace' 的日志行