菜单
文档breadcrumb arrow Grafana Lokibreadcrumb arrow 发送数据breadcrumb arrow Promtailbreadcrumb arrow 流水线阶段breadcrumb arrow multiline
开源

multiline

注意

Promtail 已被弃用，并将通过长期支持 (LTS) 持续到 2026 年 2 月 28 日。Promtail 将于 2026 年 3 月 2 日达到生命周期结束 (EOL)。您可以在此处找到迁移资源。

multiline 阶段在将多行传递到流水线中的下一个阶段之前，会将它们合并为一个多行块。

新块由 firstline 正则表达式标识。任何与该表达式*不*匹配的行都被视为前一个匹配块的一部分。

Schema

yaml 
multiline:
  # RE2 regular expression, if matched will start a new multiline block.
  # This expression must be provided.
  firstline: <string>

  # The maximum wait time will be parsed as a Go duration: https://golang.ac.cn/pkg/time/#ParseDuration.
  # If no new logs arrive within this maximum wait time, the current block will be sent on.
  # This is useful if the observed application dies with, for example, an exception.
  # No new logs will arrive and the exception
  # block is sent *after* the maximum wait time expires.
  # It defaults to 3s.
  max_wait_time: <duration>

  # Maximum number of lines a block can have. If the block has more lines, a new block is started.
  # The default is 128 lines.
  max_lines: <integer>

示例

预定义日志格式

考虑来自一个简单 flask 服务的这些日志。

[2020-12-03 11:36:20] "GET /hello HTTP/1.1" 200 -
[2020-12-03 11:36:23] ERROR in app: Exception on /error [GET]
Traceback (most recent call last):
  File "/home/pallets/.pyenv/versions/3.8.5/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/pallets/.pyenv/versions/3.8.5/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/pallets/.pyenv/versions/3.8.5/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/pallets/.pyenv/versions/3.8.5/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/pallets/.pyenv/versions/3.8.5/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/pallets/.pyenv/versions/3.8.5/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/pallets/src/deployment_tools/hello.py", line 10, in error
    raise Exception("Sorry, this route always breaks")
Exception: Sorry, this route always breaks
[2020-12-03 11:36:23] "GET /error HTTP/1.1" 500 -
[2020-12-03 11:36:26] "GET /hello HTTP/1.1" 200 -
[2020-12-03 11:36:27] "GET /hello HTTP/1.1" 200 -

我们希望将堆栈跟踪的所有行合并为一个多行块。在此示例中，所有块都以方括号中的时间戳开头。因此，我们使用 firstline 正则表达式 ^\[\d{4}-\d{2}-\d{2} \d{1,2}:\d{2}:\d{2}\] 配置了一个 multiline 阶段。这将匹配堆栈跟踪的开头，但不匹配直到 Exception: Sorry, this route always breaks 的后续行。这些行将成为一个多行块，并在 Loki 中成为一个日志条目。

yaml 
- multiline:
    # Identify timestamps as first line of a multiline block. Enclose the string in single quotes.
    firstline: '^\[\d{4}-\d{2}-\d{2} \d{1,2}:\d{2}:\d{2}\]'
    max_wait_time: 3s
- regex:
    # Flag (?s:.*) needs to be set for regex stage to capture full traceback log in the extracted map.
    expression: '^(?P<time>\[\d{4}-\d{2}-\d{2} \d{1,2}:\d{2}:\d{2}\]) (?P<message>(?s:.*))$'

自定义日志格式

该示例假设您无法控制日志格式。因此，需要一个更精心设计的正则表达式来匹配第一行。如果您可以控制被监控系统的日志格式，我们可以简化第一行匹配。

这次我们来看看一个简单的 Akka HTTP 服务的日志。

​​[2021-01-07 14:17:43,494] [DEBUG] [akka.io.TcpListener] [HelloAkkaHttpServer-akka.actor.default-dispatcher-26] [akka://HelloAkkaHttpServer/system/IO-TCP/selectors/$a/0] - New connection accepted
​​[2021-01-07 14:17:43,499] [ERROR] [akka.actor.ActorSystemImpl] [HelloAkkaHttpServer-akka.actor.default-dispatcher-3] [akka.actor.ActorSystemImpl(HelloAkkaHttpServer)] - Error during processing of request: 'oh no! oh is unknown'. Completing with 500 Internal Server Error response. To change default exception handling behavior, provide a custom ExceptionHandler.
java.lang.Exception: oh no! oh is unknown
	at com.grafana.UserRoutes.$anonfun$userRoutes$6(UserRoutes.scala:28)
	at akka.http.scaladsl.server.Directive$.$anonfun$addByNameNullaryApply$2(Directive.scala:166)
	at akka.http.scaladsl.server.ConjunctionMagnet$$anon$2.$anonfun$apply$3(Directive.scala:234)
	at akka.http.scaladsl.server.directives.BasicDirectives.$anonfun$mapRouteResult$2(BasicDirectives.scala:68)
	at akka.http.scaladsl.server.directives.BasicDirectives.$anonfun$textract$2(BasicDirectives.scala:161)
	at akka.http.scaladsl.server.RouteConcatenation$RouteWithConcatenation.$anonfun$$tilde$2(RouteConcatenation.scala:47)
	at akka.http.scaladsl.util.FastFuture$.strictTransform$1(FastFuture.scala:40)
  ...

初看起来，这些日志似乎与其他日志类似。我们来看看日志格式。

xml 
<configuration>
    <appender name="FILE" class="ch.qos.logback.core.FileAppender">
        <file>crasher.log</file>
        <append>true</append>
        <encoder>
            <pattern>&ZeroWidthSpace;[%date{ISO8601}] [%level] [%logger] [%thread] [%X{akkaSource}] - %msg%n</pattern>
        </encoder>
    </appender>

    <appender name="ASYNC" class="ch.qos.logback.classic.AsyncAppender">
        <queueSize>1024</queueSize>
        <neverBlock>true</neverBlock>
        <appender-ref ref="STDOUT" />
    </appender>

    <root level="DEBUG">
        <appender-ref ref="ASYNC"/>
    </root>

</configuration>

除了每行日志开头都有 &ZeroWidthSpace; 之外，这个 Logback 配置没有什么特别之处。这是零宽空格字符的 HTML 代码。它使识别第一行变得简单得多，并且不可见。因此，它不会改变日志的视图。新的第一行匹配正则表达式是 \x{200B}\[。200B 是零宽空格字符的 Unicode 码点。

yaml 
multiline:
  # Identify zero-width space as first line of a multiline block.
  # Note the string should be in single quotes.
  firstline: '^\x{200B}\['

  max_wait_time: 3s

零宽空格可能不适合所有人。任何不太可能出现在常规日志中的特殊字符应该都能很好地工作。