前置知识: Git

Kubernetes

3 min高级

K8s 架构、核心资源、存储、网络策略、Helm 与 Operator 模式。

学习目标

本文是「运维与中间件」模块的第 8 篇,难度定位为高级。重点内容:K8s 架构、核心资源、存储、网络策略、Helm 与 Operator 模式。

主要章节:

    1. Kubernetes 架构
    1. 核心资源
    1. ConfigMap 与 Secret
    1. 自动扩缩容
    1. 存储
    1. 网络策略
  • ……共 20 个章节

1. Kubernetes 架构

1.1 整体架构

flowchart TD
    subgraph CP[Control Plane]
        API[API Server] SCH[Scheduler] CM[Controller Manager]
        ETCD[etcd 集群状态存储]
    end
    N1[Node 1<br/>kubelet Proxy Pods]
    N2[Node 2<br/>kubelet Proxy Pods]
    N3[Node 3<br/>kubelet Proxy Pods]
    NN[Node N<br/>kubelet Proxy Pods]
    CP --> N1
    CP --> N2
    CP --> N3
    CP --> NN

1.2 核心组件

组件职责
API Server集群入口,RESTful API
etcd分布式 KV 存储,保存集群状态
SchedulerPod 调度到 Node
Controller Manager控制器(Deployment/ReplicaSet/Node)
kubelet节点代理,管理 Pod 生命周期
kube-proxy网络代理,Service 转发

2. 核心资源

2.1 Pod

apiVersion: v1
kind: Pod
metadata:
  name: web-app
  labels:
    app: web
    version: v1
spec:
  containers:
    - name: web
      image: nginx:1.28-alpine
      ports:
        - containerPort: 80
      resources:
        requests:
          cpu: 100m
          memory: 128Mi
        limits:
          cpu: 500m
          memory: 512Mi
      env:
        - name: NODE_ENV
          value: 'production'
        - name: DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: db-secret
              key: password
      livenessProbe:
        httpGet:
          path: /health
          port: 80
        initialDelaySeconds: 15
        periodSeconds: 10
      readinessProbe:
        httpGet:
          path: /ready
          port: 80
        initialDelaySeconds: 5
        periodSeconds: 5
      volumeMounts:
        - name: config
          mountPath: /etc/config
          readOnly: true
  volumes:
    - name: config
      configMap:
        name: app-config
  restartPolicy: Always

2.2 Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
        - name: web
          image: myapp:v2
          ports:
            - containerPort: 8080
          resources:
            requests:
              cpu: 100m
              memory: 128Mi
            limits:
              cpu: 500m
              memory: 512Mi

2.3 Service

# ClusterIP(集群内部访问)
apiVersion: v1
kind: Service
metadata:
  name: web-service
spec:
  type: ClusterIP
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 8080

---
# NodePort(节点端口暴露)
apiVersion: v1
kind: Service
metadata:
  name: web-nodeport
spec:
  type: NodePort
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 8080
      nodePort: 30080

---
# LoadBalancer(云厂商负载均衡)
apiVersion: v1
kind: Service
metadata:
  name: web-lb
spec:
  type: LoadBalancer
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 8080

2.4 Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-ingress
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: 'true'
    nginx.ingress.kubernetes.io/rate-limit: '100'
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - example.com
      secretName: example-tls
  rules:
    - host: example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web-service
                port:
                  number: 80
          - path: /api
            pathType: Prefix
            backend:
              service:
                name: api-service
                port:
                  number: 80

3. ConfigMap 与 Secret

3.1 ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  # 键值对
  DATABASE_HOST: 'postgres.default.svc.cluster.local'
  DATABASE_PORT: '5432'
  LOG_LEVEL: 'info'
  # 完整配置文件
  nginx.conf: |
    server {
      listen 80;
      location / {
        proxy_pass http://web-service:8080;
      }
    }

3.2 Secret

apiVersion: v1
kind: Secret
metadata:
  name: db-secret
type: Opaque
data:
  # Base64 编码
  username: YWRtaW4= # admin
  password: c2VjcmV0MTIz # secret123
stringData:
  # 明文(创建时自动编码)
  api-key: 'my-api-key-12345'
# 创建 Secret
kubectl create secret generic db-secret \
  --from-literal=username=admin \
  --from-literal=password=secret123

# 从文件创建
kubectl create secret generic tls-secret \
  --from-file=tls.crt=./cert.pem \
  --from-file=tls.key=./key.pem

4. 自动扩缩容

4.1 HPA(水平扩缩容)

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-deployment
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 60
      policies:
        - type: Percent
          value: 100
          periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 10
          periodSeconds: 60

4.2 VPA(垂直扩缩容)

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: web-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-deployment
  updatePolicy:
    updateMode: Auto # Off / Initial / Recreate / Auto
  resourcePolicy:
    containerPolicies:
      - containerName: web
        minAllowed:
          cpu: 100m
          memory: 128Mi
        maxAllowed:
          cpu: '2'
          memory: 2Gi

5. 存储

5.1 PV 与 PVC

# PersistentVolume
apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-pv
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteMany
  nfs:
    server: nfs-server.default.svc.cluster.local
    path: /data/share

---
# PersistentVolumeClaim
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  storageClassName: standard

5.2 StorageClass

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast-ssd
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp3
  iopsPerGB: '50'
reclaimPolicy: Retain
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer

6. 网络策略

# 默认拒绝所有入站
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
  namespace: production
spec:
  podSelector: {}
  policyTypes:
    - Ingress

---
# 允许特定 Pod 访问
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-frontend-to-api
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: api
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: frontend
      ports:
        - protocol: TCP
          port: 8080

7. Helm

7.1 Chart 结构

flowchart TD
    T0["mychart/"]
    T1["Chart.yaml          # Chart 元数据"]
    T2["values.yaml         # 默认值"]
    T3["templates/"]
    T4["deployment.yaml"]
    T5["service.yaml"]
    T6["ingress.yaml"]
    T7["configmap.yaml"]
    T8["_helpers.tpl    # 模板辅助函数"]
    T9["NOTES.txt       # 安装说明"]
    T10[".helmignore"]
    T0 --> T1
    T0 --> T2
    T0 --> T3
    T9 --> T10

7.2 Chart.yaml

apiVersion: v2
name: myapp
description: My Application Helm Chart
type: application
version: 1.0.0
appVersion: '2.0.0'
dependencies:
  - name: postgresql
    version: '14.x.x'
    repository: 'https://charts.bitnami.com/bitnami'
    condition: postgresql.enabled

7.3 values.yaml

replicaCount: 3

image:
  repository: myapp
  pullPolicy: IfNotPresent
  tag: '2.0.0'

service:
  type: ClusterIP
  port: 80

ingress:
  enabled: true
  className: nginx
  hosts:
    - host: example.com
      paths:
        - path: /
          pathType: Prefix

resources:
  requests:
    cpu: 100m
    memory: 128Mi
  limits:
    cpu: 500m
    memory: 512Mi

postgresql:
  enabled: true
  auth:
    database: myapp
    username: app
    password: changeme

7.4 Helm 命令

# 仓库管理
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update

# 安装/升级
helm install myapp ./mychart                    # 本地 Chart
helm install myapp bitnami/nginx                # 仓库 Chart
helm upgrade myapp ./mychart --set replicaCount=5  # 升级
helm upgrade --install myapp ./mychart -f values-prod.yaml  # 安装或升级

# 管理
helm list                                       # 列出发布
helm status myapp                               # 查看状态
helm rollback myapp 1                           # 回滚
helm uninstall myapp                            # 卸载

# 调试
helm template myapp ./mychart                   # 渲染模板
helm lint ./mychart                             # 检查 Chart
helm diff upgrade myapp ./mychart               # 查看变更(需插件)

8. Operator 模式

8.1 Operator 原理

Operator = CRD(自定义资源) + Controller(控制器逻辑),将运维知识编码为软件。

flowchart LR
    O[Operator]
    CRD[CRD 自定义资源] <--> C[Controller 协调循环 Reconcile]
    O --- CRD
    O --- C

8.2 CRD 定义

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: webapps.apps.example.com
spec:
  group: apps.example.com
  versions:
    - name: v1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                replicas:
                  type: integer
                  minimum: 1
                  maximum: 100
                image:
                  type: string
  scope: Namespaced
  names:
    plural: webapps
    singular: webapp
    kind: WebApp
    shortNames:
      - wa

8.3 常用 Operator

Operator功能适用场景
Prometheus Operator管控监控栈可观测性
Cert Manager证书管理TLS 自动化
ArgoCDGitOps 部署持续交付
MySQL OperatorMySQL 集群管理数据库运维
Kafka OperatorKafka 集群管理消息队列运维

9. 常用 kubectl 命令

# 资源查看
kubectl get pods -A                        # 所有命名空间的 Pod
kubectl get pods -o wide                   # 详细信息
kubectl get all -n production              # 命名空间所有资源
kubectl describe pod web-app               # 详细描述
kubectl top pods                           # 资源使用

# 调试
kubectl logs -f web-app                    # 查看日志
kubectl logs web-app -c sidecar            # 指定容器
kubectl exec -it web-app -- bash           # 进入容器
kubectl port-forward svc/web 8080:80       # 端口转发

# 资源管理
kubectl apply -f deployment.yaml           # 应用配置
kubectl delete -f deployment.yaml          # 删除资源
kubectl scale deployment web --replicas=5  # 扩缩容
kubectl rollout restart deployment web     # 重启
kubectl rollout undo deployment web        # 回滚
kubectl rollout status deployment web      # 查看状态

# 上下文管理
kubectl config get-contexts
kubectl config use-context prod-cluster

10. 小结

Kubernetes 是容器编排的事实标准:

  1. 架构理解 Control Plane 和 Node 组件的职责是排障基础
  2. 核心资源(Pod/Deployment/Service/Ingress)覆盖了大部分应用场景
  3. ConfigMap/Secret 管理配置和敏感信息,避免硬编码
  4. HPA/VPA 实现自动扩缩容,应对流量波动
  5. Helm 是 K8s 应用的包管理器,简化部署和管理
  6. Operator 将运维知识编码化,适合有状态应用

Pod 资源定义

基本写法:定义 Pod

`apiVersion: v1
kind: Pod
metadata:
  name: <名称>
spec:
  containers:
    - name: <容器名>
      image: <镜像>`
# 定义 nginx Pod
apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
    - name: nginx
      image: nginx:1.28
      ports:
        - containerPort: 80

基本写法:多容器 Pod

`spec:
  containers:
    - name: <容器1>
      image: <镜像1>
    - name: <容器2>
      image: <镜像2>`
# 定义多容器 Pod
apiVersion: v1
kind: Pod
metadata:
  name: app-with-sidecar
spec:
  containers:
    - name: app
      image: myapp:v1
    - name: log-sidecar
      image: busybox
      args: [/bin/sh, -c, "tail -f /log/app.log"]

Deployment 资源定义

基本写法:定义 Deployment

`apiVersion: apps/v1
kind: Deployment
metadata:
  name: <名称>
spec:
  replicas: <副本数>
  selector:
    matchLabels:
      app: <标签>
  template:
    metadata:
      labels:
        app: <标签>
    spec:
      containers:
        - name: <容器名>
          image: <镜像>`
# 定义 nginx Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:1.28
          ports:
            - containerPort: 80

Service 资源定义

基本写法:定义 ClusterIP Service

`apiVersion: v1
kind: Service
metadata:
  name: <名称>
spec:
  selector:
    app: <标签>
  ports:
    - port: <端口>
      targetPort: <目标端口>`
# 定义 ClusterIP Service
apiVersion: v1
kind: Service
metadata:
  name: nginx-svc
spec:
  selector:
    app: nginx
  ports:
    - port: 80
      targetPort: 80

基本写法:定义 NodePort Service

`spec:
  type: NodePort
  ports:
    - port: <端口>
      targetPort: <目标端口>
      nodePort: <节点端口>`
# 定义 NodePort Service
apiVersion: v1
kind: Service
metadata:
  name: nginx-nodeport
spec:
  type: NodePort
  selector:
    app: nginx
  ports:
    - port: 80
      targetPort: 80
      nodePort: 30080

基本写法:定义 LoadBalancer Service

`spec:
  type: LoadBalancer
  ports:
    - port: <端口>`
# 定义 LoadBalancer Service
apiVersion: v1
kind: Service
metadata:
  name: nginx-lb
spec:
  type: LoadBalancer
  selector:
    app: nginx
  ports:
    - port: 80
      targetPort: 80

ConfigMap 资源定义

基本写法:定义 ConfigMap

`apiVersion: v1
kind: ConfigMap
metadata:
  name: <名称>
data:
  <键>: <值>`
# 定义应用配置 ConfigMap
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  APP_ENV: production
  LOG_LEVEL: info
  config.yaml: |
    server:
      port: 8080
      host: 0.0.0.0

Secret 资源定义

基本写法:定义 Opaque Secret

`apiVersion: v1
kind: Secret
metadata:
  name: <名称>
type: Opaque
data:
  <键>: <Base64值>`
# 定义数据库密码 Secret
apiVersion: v1
kind: Secret
metadata:
  name: db-secret
type: Opaque
data:
  username: YWRtaW4=
  password: c2VjcmV0MTIz

基本写法:使用 stringData 明文

`apiVersion: v1
kind: Secret
metadata:
  name: <名称>
type: Opaque
stringData:
  <键>: <明文值>`
# 使用明文定义 Secret
apiVersion: v1
kind: Secret
metadata:
  name: db-secret
type: Opaque
stringData:
  username: admin
  password: secret123

Ingress 资源定义

基本写法:定义 Ingress

`apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: <名称>
spec:
  rules:
    - host: <域名>
      http:
        paths:
          - path: <路径>
            pathType: Prefix
            backend:
              service:
                name: <服务名>
                port:
                  number: <端口>`
# 定义 nginx Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-ingress
spec:
  rules:
    - host: example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: nginx-svc
                port:
                  number: 80

资源配额与限制

基本写法:定义资源请求和限制

`spec:
  containers:
    - name: <名称>
      resources:
        requests:
          cpu: <CPU请求>
          memory: <内存请求>
        limits:
          cpu: <CPU限制>
          memory: <内存限制>`
# 设置容器资源配额
apiVersion: v1
kind: Pod
metadata:
  name: app
spec:
  containers:
    - name: app
      image: myapp
      resources:
        requests:
          cpu: 100m
          memory: 128Mi
        limits:
          cpu: 500m
          memory: 512Mi

健康检查

基本写法:定义存活探针

`spec:
  containers:
    - name: <名称>
      livenessProbe:
        httpGet:
          path: <路径>
          port: <端口>
        initialDelaySeconds: <秒数>`
# 定义 HTTP 存活探针
apiVersion: v1
kind: Pod
metadata:
  name: app
spec:
  containers:
    - name: app
      image: myapp
      livenessProbe:
        httpGet:
          path: /health
          port: 8080
        initialDelaySeconds: 30
        periodSeconds: 10

基本写法:定义就绪探针

`spec:
  containers:
    - name: <名称>
      readinessProbe:
        httpGet:
          path: <路径>
          port: <端口>`
# 定义 HTTP 就绪探针
spec:
  containers:
    - name: app
      image: myapp
      readinessProbe:
        httpGet:
          path: /ready
          port: 8080
        initialDelaySeconds: 5
        periodSeconds: 10

持久化存储

基本写法:定义 PersistentVolumeClaim

`apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: <名称>
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: <大小>`
# 定义 10GB 的 PVC
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi

基本写法:在 Pod 中使用 PVC

`spec:
  containers:
    - name: <名称>
      volumeMounts:
        - mountPath: <挂载路径>
          name: <卷名>
  volumes:
    - name: <卷名>
      persistentVolumeClaim:
        claimName: <PVC名称>`
# 在 Pod 中挂载 PVC
spec:
  containers:
    - name: app
      image: myapp
      volumeMounts:
        - mountPath: /data
          name: data-volume
  volumes:
    - name: data-volume
      persistentVolumeClaim:
        claimName: data-pvc

命名空间

基本写法:定义 Namespace

`apiVersion: v1
kind: Namespace
metadata:
  name: <名称>`
# 定义开发环境命名空间
apiVersion: v1
kind: Namespace
metadata:
  name: dev
  labels:
    name: dev

基本写法:定义 ResourceQuota

`apiVersion: v1
kind: ResourceQuota
metadata:
  name: <名称>
  namespace: <命名空间>
spec:
  hard:
    requests.cpu: <CPU总量>
    requests.memory: <内存总量>`
# 限制命名空间资源配额
apiVersion: v1
kind: ResourceQuota
metadata:
  name: dev-quota
  namespace: dev
spec:
  hard:
    requests.cpu: "4"
    requests.memory: 8Gi
    limits.cpu: "8"
    limits.memory: 16Gi