Kubernetes descheduler 几种使用方式

Kubernetes descheduler 几种使用方式

descheduler是对于现有的 kube-scheduler 精细化调度丰富. de-scheduler 当业务长期运行后, 出现warning到一定情况，通过预先设置的自动干预法策略，来重新调度资源。

主要是丰富了: 掉漆中Filter、strategy环节中，实现插件middleware功能。从而影响调度决策。

descheduler架构

Descheduler的配置主要由Policies组成. Policies由Evictor和Strategies组成。

策略决定在什么情况下应该重新分配 Pod，而 Evictor 则确定哪些 Pod 有资格重新分配.

Evictor 驱逐者

Evictor 目前采用两种内置机制，即 Filter 和 PreEvicitionFilter 两套interface接口插件。

其中 Filter 默认放通以下pod. 有些特殊的pod，是不是被descheduler调度的. 包括：

• Static pods
• DaemonSet Pods
• Terminating Pods
• Success Pods
• 满足特定命名空间/优先级/标签条件的 Pod

还有：

• 具有本地存储的 Pod 永远不会被驱逐
• 带有 PVC 的 Pod 会被驱逐
• 带有 PDB 的 Pod 会被驱逐（利用齐本身的PDB做驱逐）

而，PreEvicitionFilter 会检查 Pod 的 NodeSelector/Affinity/Taint/Request 是否有可用于重新部署的节点。

Strategies 策略

Strategy 内置两种策略：取消计划Remove和资源均衡Balance。

具体策略介绍

建议使用：发现pod不知原因的N多次重启，将此pod驱逐到其他机器上

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: ProfileName
    pluginConfig:
    - name: "RemovePodsHavingTooManyRestarts"
      args:
        podRestartThreshold: 50   # 重启次数高达50次以上
        includingInitContainers: true # 是否check InitContainer的重启次数
        states:
        - CrashLoopBackOff
    plugins:
      deschedule:
        enabled:
          - "RemovePodsHavingTooManyRestarts"

建议使用: 确保只有一个 pod 与在同一节点上运行, 删除重复Pod主动驱逐

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: ProfileName
    pluginConfig:
    - name: "RemoveDuplicates"
      args:
        excludeOwnerKinds: # ReplicationController/ReplicaSet/StatefulSet/Job
          - "ReplicaSet"
          - "ReplicationController"
        namespaces: # namespaces 白名单和黑名单
        	  include:
          - "namespace1"
          - "namespace2"
          exclude:
          - "namespace3"
          - "namespace4"
    plugins:
      balance:
        enabled:
          - "RemoveDuplicates"

谨慎使用: 查找未充分利用的节点，并在可能的情况下从其他节点逐出 pod

PS: 依赖metrice-server Node利用率资源

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: ProfileName
    pluginConfig:
    - name: "LowNodeUtilization"
      args:
        useDeviationThresholds: true # 如果设置为true, 是集群平均资源的偏差值;如果为false, 是节点node绝对值百分比;
        thresholds:
          "cpu" : 20
          "memory": 20
          "pods": 20
        targetThresholds:
          "cpu" : 50
          "memory": 50
          "pods": 50
        numberOfNodes: 5 # 仅当未充分利用的节点数高于5时, 才激活该策略
        evictableNamespaces: # namespaces 白名单和黑名单
          include:
          - "namespace1"
          - "namespace2"
          exclude:
          - "kube-system"
    plugins:
      balance:
        enabled:
          - "LowNodeUtilization"

不建议使用: 找到未充分利用的节点，并将 pod 从节点中逐出，将 pod 紧凑地调度到更少利用率的节点

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: ProfileName
    pluginConfig:
    - name: "HighNodeUtilization"
      args:
        numberOfNodes: 5 # 仅当未充分利用的节点数高于5时, 才激活该策略
        thresholds:
          "cpu" : 20
          "memory": 20
          "pods": 20
        evictableNamespaces: # namespaces 白名单和黑名单
          exclude:
          - "kube-system"
          - "namespace1"
       	  include:
          - "namespace2"
          - "namespace3"
    plugins:
      balance:
        enabled:
          - "HighNodeUtilization"

不建议使用: 删除违反 Pod 间反亲和性的 Pod（PodA、PodB有亲和性，会祸及 PodC）

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: ProfileName
    pluginConfig:
    - name: "RemovePodsViolatingInterPodAntiAffinity"
      args:
        numberOfNodes: 5 # 仅当未充分利用的节点数高于5时, 才激活该策略
        labelSelector:
          matchLabels:   # label match
            component: xxxx  
          matchExpressions: 
            - {key: tier, operator: In, values: [cache]}
            - {key: environment, operator: NotIn, values: [dev]}
        namespaces: # namespaces 白名单和黑名单
          exclude:
          - "kube-system"
          - "namespace1"
       	  include:
          - "namespace2"
          - "namespace3"
    plugins:
      deschedule:
        enabled:
          - "RemovePodsViolatingInterPodAntiAffinity"

谨慎使用: 删除节点上违反 NoSchedule 污点的 Pod，将有NoSchedule的Node排除考虑调度之外

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: ProfileName
    pluginConfig:
    - name: "RemovePodsViolatingNodeTaints"
      args:
        excludedTaints:
        - key1=value1 # 剔除 taints 包含 key "key1" and value "value1"
        - reserved    # 剔除所有 taints 包含 key "reserved"
        namespaces:   # namespaces 白名单和黑名单
          exclude:
          - "kube-system"
          - "namespace1"
       	  include:
          - "namespace2"
          - "namespace3"
    plugins:
      deschedule:
        enabled:
          - "RemovePodsViolatingNodeTaints"

不建议使用: Pod提前驱逐，早于 的 pod maxPodLifeTimeSeconds进行驱逐

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: ProfileName
    pluginConfig:
    - name: "PodLifeTime"
      args:
        maxPodLifeTimeSeconds: 86400
        states:
        - "Pending"
        - "PodInitializing"
        labelSelector:
          matchLabels:   # label match
            component: xxxx  
          matchExpressions: 
            - {key: tier, operator: In, values: [cache]}
            - {key: environment, operator: NotIn, values: [dev]}
        namespaces: # namespaces 白名单和黑名单
          exclude:
          - "kube-system"
          - "namespace1"
       	  include:
          - "namespace2"
          - "namespace3"
    plugins:
      deschedule:
        enabled:
          - "PodLifeTime"

建议使用: 驱逐处于失败状态阶段的 Pod

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: ProfileName
    pluginConfig:
    - name: "RemoveFailedPods"
      args:
        reasons: # 错位原因
        - "NodeAffinity"
        - "CrashLoopBackOff"
        includingInitContainers: true # 也需要check InitContainers的错误原因
        excludeOwnerKinds: # 过滤类型，Job类型Pod不考虑
        - "Job"
        minPodLifetimeSeconds: 360 # 逐出错位指定秒数的 Pod，单位秒
          labelSelector:
          matchLabels:   # label match
            component: xxxx  
          matchExpressions: 
            - {key: tier, operator: In, values: [cache]}
            - {key: environment, operator: NotIn, values: [dev]}
        namespaces: # namespaces 白名单和黑名单
          exclude:
          - "kube-system"
          - "namespace1"
       	  include:
          - "namespace2"
          - "namespace3"
    plugins:
      deschedule:
        enabled:
          - "RemoveFailedPods"

其他

• descheduler
• Koordinator
• strategy