参数

默认值

解释

jobmanager.memory.process.size

None

JM的总内存

如果没有设置，会使用旧的配置jobmanager.heap.size

jobmanager.memory.flink.size

None

jobmanager.memory.process.size= jobmanager.memory.flink.size+ JVM Metaspace and JVM Overhead

jobmanager.memory.heap.size

None

jobmanager.memory.off-heap.size

128

jobmanager.memory.enable-jvm-direct-memory-limit

false

如果设置了-XX:MaxDirectMemorySize且enable-jvm-direct-memory-limit=true，那么MaxDirectMemorySize=off-heap.size

jobmanager.memory.jvm-metaspace.size

256

jobmanager.memory.jvm-overhead.min

192

jobmanager.memory.jvm-overhead.max

1g

jobmanager.memory.jvm-overhead.fraction

0.1f

参数

默认值

解释

taskmanager.memory.process.size

None

TM的总内存

如果没有设置，会使用旧的配置taskmanager.heap.size

taskmanager.memory.flink.size

None

taskmanager.memory.process.size= taskmanager.memory.flink.size+ JVM Metaspace and JVM Overhead

taskmanager.memory.jvm-overhead.min

192m

Overhead的最小值

taskmanager.memory.jvm-overhead.max

1g

Overhead的最大值

taskmanager.memory.jvm-overhead.fraction

0.1

通过计算taskmanager.memory.process.size*0.1

Overhead的具体值= fraction* process.size，如果fraction* process.size小于overhead.min，那么Overhead的具体值就是overhead.min，如果fraction* process.size大于overhead.max，那么Overhead的具体值就是overhead.max

taskmanager.memory.jvm-metaspace.size

256m

Metaspace大小

taskmanager.memory.framework.heap.size

128m

framework的堆内

taskmanager.memory.framework.off-heap.size

128m

framework的堆外

taskmanager.memory.network.min

64m

network的最小内存

taskmanager.memory.network.max

1g

network的最大内存

taskmanager.memory.network.fraction

network在1.12版本以后都使用offheap，不需要再配置offheap=true/false

0.1f

计算(taskmanager.memory.process.size-overhead- -metaspace)* network.fraction得到真正的network内存大小，如果内存小于network.min或者大于network.max，将被替代

taskmanager.memory.managed.size

Long最大值

taskmanager.memory.managed.fraction

0.4f

(taskmanager.memory.process.size-overhead- -metaspace)* managed.fraction得到真正的managed内存大小，如果内存大小大于managed.size，将被managed.size替代

taskmanager.memory.task.off-heap.size

0

Task的offheap

taskmanager.memory.task.heap.size

None

默认的计算公式taskmanager.memory.process.size- overhead-metaspace-frameworkHeapMemorySize-frameworkOffHeapMemorySize-taskOffHeapMemorySize-managedMemorySize-networkMemorySize

参数

默认值

解释

taskmanager.memory.segment-size

32kb

Segment大小

最终segment个数=networksize/segment，不能大于LongMax

taskmanager.network.netty.sendReceiveBufferSize

0

默认使用cat /proc/sys/net/ipv4/tcp_[rw]mem操作系统大小

taskmanager.network.retries

0

两个TM之间建立连接时的重试次数

该参数不解决连接断开后的重连场景

taskmanager.network.netty.client.numThreads

-1

默认值是slot数

taskmanager.network.netty.server.numThreads

-1

默认值是slot数

taskmanager.network.netty.num-arenas

-1

默认值是slot数

taskmanager.network.numberOfBuffers

2048

Buffer数量，最终按照networksize/segment计算，该参数在1.12已经Deprecated

taskmanager.network.memory.max-buffers-per-channel

10

每个channel最大能持有多少buffers，如果segment有很多空闲，可以适当调大该值，否则channel会因为拿不到segment而blocking

taskmanager.network.memory.buffers-per-channel

2

每个channel的buffer数，LocalBufferPool最多申请的数量= numberOfSubpartitions* buffers-per-channel+floating-buffers-per-gate(Bounded)或者是IntMax(!Bounded)

taskmanager.network.memory.floating-buffers-per-gate

8

每个channel的floating buffers数，只在ResultPartitionType为Bounded类型时有效

taskmanager.network.sort-shuffle.min-parallelism

IntMax

sort-shuffle的最小并发