K8s安装Kuboard

发表于 更新于 分类于 Waline：

安装

  1. 在Mac或其他机器安装管理工具kuboard-spray

    1
    2
    3
    4
    5
    6
    7
    8
    docker run -d \
      --privileged \
      --restart=unless-stopped \
      --name=kuboard-spray \
      -p 80:80/tcp \
      -v /var/run/docker.sock:/var/run/docker.sock \
      -v ~/kuboard-spray-data:/data \
      eipwork/kuboard-spray:latest-amd64

  2. 访问http://localhost/#/login，输入用户名 admin，默认密码 Kuboard123，即可登录 Kuboard-Spray 界面。

  3. 点击Add Cluster Installation Plad输入集群名称，选择spray-v2.21.0c_k8s-v1.26.4_v4.4-amd64点击OK。

  4. 点击Add Node添加一个节点，勾选control plane，etcd node，worker node点击OK。

  5. 右侧输入安装节点的ip=172.16.3.165，端口，密码，最底部输入etcd的名字etcd_exxk，点击Validate Connection

  6. 最后点击save，然后点击Install/Setup K8S Cluster按钮进行安装。

  7. 等待安装完成，失败可以重复6

  8. 访问http://172.16.3.165默认用户名: admin默认密 码: Kuboard123

配置

方案一 ：修改Kuboard端口

  1. 找到Kuboard的部署配置文件，vi /etc/kubernetes/manifests/kuboard.yaml修改

    1
    2
    3
    4
    5
    6
    7
    8
    9
    - env:
        - name: KUBOARD_ENDPOINT
          value: "http://172.16.3.165:14001" #把80修改为14001
      name: kuboard
      ports:
        - containerPort: 80
          hostPort: 14001     #hostPort修改为14001  
          name: web
          protocol: TCP

  2. 保存，等待自动重启。

    知识点：static-pod：静态 Pod 在指定的节点上由 kubelet 守护进程直接管理，不需要 API 服务器监管。 与由控制面管理的 Pod（例如，Deployment） 不同；kubelet 监视每个静态 Pod（在它失败之后重新启动）。

    特点：更改配置自动重启pod，无法删除pod，只能把配置文件移除目录才能删除，默认静态pod目录/etc/kubernetes/manifests。

方案二：修改Kuboard走ingress-nginx(失败，能访问界面，但是bash相关功能用不了)

  1. 在Kuboard管理界面的Kuboard命名空间创建service

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    apiVersion: v1
    kind: Service
    metadata:
      labels:
        k8s.kuboard.cn/name: kuboard-v3
      name: kuboard-v3
      namespace: kuboard
    spec:
      ports:
        - protocol: TCP
          port: 80
          targetPort: 80
      selector:
        k8s.kuboard.cn/name: kuboard-v3
      type: ClusterIP

  2. 在Kuboard管理界面的Kuboard命名空间创建Ingress

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      labels:
        k8s.kuboard.cn/name: kuboard-v3
      name: kuboard
      namespace: kuboard
    spec:
      ingressClassName: myingresscontroller  #需要先安装IngressNginxController，使用安装时候的名字
      rules:
        - host: kuboard.iexxk.io #安装时候的域名后缀
          http:
            paths:
              - backend:
                  service:
                    name: kuboard-v3
                    port:
                      number: 80
                path: /
                pathType: Prefix

  3. 找到Kuboard的部署配置文件，vi /etc/kubernetes/manifests/kuboard.yaml修改

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    ....省略其他配置
        - env:
            - name: KUBOARD_ENDPOINT
              value: "http://172.16.3.165:14001" #把172.16.3.165:80修改为kuboard.iexxk.io
          name: kuboard
          ports:
            - containerPort: 80
            #  hostPort: 14001     #hostPort这一行删除  
              name: web
              protocol: TCP   
    .....省略

  4. 保存，等待自动重启。

    知识点：

    1. 本来想用静态pod的方式配置，发现添加到配置文件不生效，后来只能在界面的模式添加。

    2. IngressNginxController简单来说就是一个nginx，进入pod容器里面可以看到nginx的相关配置，在使用服务配置了ingress后，会自动在ingress的pod里面生成相应的nginx配置，样例如下：

      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      39
      40
      41
      42
      43
      44
      45
      46
      47
      48
      49
      50
      51
      52
      53
      54
      55
      56
      57
      58
      59
      60
      61
      62
      63
      64
      65
      66
      67
      68
      69
      70
      71
      72
      73
      74
      75
      76
      77
      78
      79
      80
      81
      82
      83
      84
      85
      86
      87
      88
      89
      90
      91
      92
      93
      94
      95
      96
      97
      98
      99
      100
      101
      102
      103
      104
      105
      106
      107
      108
      109
      110
      111
      112
      113
      114
      115
      116
      117
      118
      119
      120
      121
      122
      123
      124
      125
      126
      127
      128
      129
      130
      131
      132
      133
      134
      135
      ## start server web.iexxk.io
       server {
               server_name web.iexxk.io ;
            
               listen 80  ;
               listen [::]:80  ;
               listen 443  ssl http2 ;
               listen [::]:443  ssl http2 ;
            
               set $proxy_upstream_name "-";
            
               ssl_certificate_by_lua_block {
                       certificate.call()
               }
            
               location / {
            
                       set $namespace      "exxk";
                       set $ingress_name   "web";
                       set $service_name   "web";
                       set $service_port   "80";
                       set $location_path  "/";
                       set $global_rate_limit_exceeding n;
            
                       rewrite_by_lua_block {
                               lua_ingress.rewrite({
                                       force_ssl_redirect = false,
                                       ssl_redirect = true,
                                       force_no_ssl_redirect = false,
                                       preserve_trailing_slash = false,
                                       use_port_in_redirects = false,
                                       global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } },
                               })
                               balancer.rewrite()
                               plugins.run()
                       }
            
                       # be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any
                       # will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)`
                       # other authentication method such as basic auth or external auth useless - all requests will be allowed.
                       #access_by_lua_block {
                       #}
            
                       header_filter_by_lua_block {
                               lua_ingress.header()
                               plugins.run()
                       }
            
                       body_filter_by_lua_block {
                               plugins.run()
                       }
            
                       log_by_lua_block {
                               balancer.log()
            
                               monitor.call()
            
                               plugins.run()
                       }
            
                       port_in_redirect off;
            
                       set $balancer_ewma_score -1;
                       set $proxy_upstream_name "exxk-web-80";
                       set $proxy_host          $proxy_upstream_name;
                       set $pass_access_scheme  $scheme;
            
                       set $pass_server_port    $server_port;
            
                       set $best_http_host      $http_host;
                       set $pass_port           $pass_server_port;
            
                       set $proxy_alternative_upstream_name "";
            
                       client_max_body_size                    1m;
            
                       proxy_set_header Host                   $best_http_host;
            
                       # Pass the extracted client certificate to the backend
            
                       # Allow websocket connections
                       proxy_set_header                        Upgrade           $http_upgrade;
            
                       proxy_set_header                        Connection        $connection_upgrade;
            
                       proxy_set_header X-Request-ID           $req_id;
                       proxy_set_header X-Real-IP              $remote_addr;
            
                       proxy_set_header X-Forwarded-For        $remote_addr;
            
                       proxy_set_header X-Forwarded-Host       $best_http_host;
                       proxy_set_header X-Forwarded-Port       $pass_port;
                       proxy_set_header X-Forwarded-Proto      $pass_access_scheme;
                       proxy_set_header X-Forwarded-Scheme     $pass_access_scheme;
            
                       proxy_set_header X-Scheme               $pass_access_scheme;
            
                       # Pass the original X-Forwarded-For
                       proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for;
            
                       # mitigate HTTPoxy Vulnerability
                       # https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
                       proxy_set_header Proxy                  "";
            
                       # Custom headers to proxied server
            
                       proxy_connect_timeout                   5s;
                       proxy_send_timeout                      60s;
                       proxy_read_timeout                      60s;
            
                       proxy_buffering                         off;
                       proxy_buffer_size                       4k;
                       proxy_buffers                           4 4k;
            
                       proxy_max_temp_file_size                1024m;
            
                       proxy_request_buffering                 on;
                       proxy_http_version                      1.1;
            
                       proxy_cookie_domain                     off;
                       proxy_cookie_path                       off;
            
                       # In case of errors try the next upstream server before returning an error
                       proxy_next_upstream                     error timeout;
                       proxy_next_upstream_timeout             0;
                       proxy_next_upstream_tries               3;
            
                       proxy_pass http://upstream_balancer;
            
                       proxy_redirect                          off;
            
               }
            
       }
       ## end server web.iexxk.io

安装IngressNginxController

  1. 在集群的 集群管理 –> 网络 –> IngressClass 列表页点击图中的 安装 IngressNginxController 并创建 IngressClass 的按钮，输入名称myingresscontroller。

  2. 查看界面上的端口提示信息

    1
    2
    3
    4
    5
    负载均衡映射
    建议使用 Kubernetes 集群外的负载均衡器，对如下端口设置 L4 转发（不能通过 X-FORWARDED-FOR 追溯源地址） 或 L7 转发（部分负载均衡产品配置 L7 转发较繁琐）
    （如果您已完成转发设置，请忽略此消息）。
    负载均衡的 80 端口转发至 Kubernetes 集群任意节点的 32211
    负载均衡的 443 端口转发至 Kubernetes 集群任意节点的 31612

  3. 方案一(比较节约资源，但是80端口被占用，不能做更多的用途)，修改容器的hostPort端口为80，然后直接通过域名即可访问。（修改myingresscontroller的no dePort32211端口为80，但是k8s集群的nodePort端口为30000～40000）

  4. 方案二(多搭建了一个nginx服务，但是灵活性更高)

    配置外部nginx，创建一个static-pod的nginx服务。

    在/etc/kubernetes/manifests/目录创建一个static-nginx.yaml

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    apiVersion: v1
    kind: Pod
    metadata:
      annotations: {}
      labels:
        k8s.kuboard.cn/name: static-nginx
      name: static-nginx
      namespace: ingress-nginx
    spec:
      containers:
        - name: web
          image: nginx:alpine
          ports:
            - name: web
              containerPort: 80
              hostPort: 80
              protocol: TCP
          volumeMounts:
            - mountPath: /etc/nginx/conf.d/default.conf
              name: nginx-conf
      volumes:
        - hostPath:
            path: "/root/static-nginx/nginx.conf"
          name: nginx-conf

    在目录/root/static-nginx创建nginx.conf

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    server {
        listen       80;
        server_name .iexxk.io;
        #access_log  /var/log/nginx/hoddst.access.log  main;

        location / {
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_pass http://172.16.3.165:32211/;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   /usr/share/nginx/html;
        }
    }

  5. 创建个测试web，ingress设置为web.iexxk.io进行访问即可，记得映射域名*.iexxk.io到172.16.3.165主机上。

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90
    91
    92
    93
    94
    95
    96
    97
    98
    99
    100
    101
    102
    103
    104
    105
    106
    107
    108
    109
    110
    111
    112
    113
    114
    115
    116
    117
    118
    119
    120
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      annotations: {}
      labels:
        k8s.kuboard.cn/name: web
      name: web
      namespace: exxk
      resourceVersion: '126840'
    spec:
      progressDeadlineSeconds: 600
      replicas: 1
      revisionHistoryLimit: 10
      selector:
        matchLabels:
          k8s.kuboard.cn/name: web
      strategy:
        rollingUpdate:
          maxSurge: 25%
          maxUnavailable: 25%
        type: RollingUpdate
      template:
        metadata:
          creationTimestamp: null
          labels:
            k8s.kuboard.cn/name: web
        spec:
          containers:
            - image: 'nginx:alpine'
              imagePullPolicy: IfNotPresent
              name: web
              ports:
                - containerPort: 80
                  protocol: TCP
              resources: {}
              terminationMessagePath: /dev/termination-log
              terminationMessagePolicy: File
          dnsPolicy: ClusterFirst
          restartPolicy: Always
          schedulerName: default-scheduler
          securityContext: {}
          terminationGracePeriodSeconds: 30
    status:
      availableReplicas: 1
      conditions:
        - lastTransitionTime: '2023-08-30T03:52:17Z'
          lastUpdateTime: '2023-08-30T03:52:17Z'
          message: Deployment has minimum availability.
          reason: MinimumReplicasAvailable
          status: 'True'
          type: Available
        - lastTransitionTime: '2023-08-30T03:52:16Z'
          lastUpdateTime: '2023-08-30T03:52:17Z'
          message: ReplicaSet "web-6f8fdd7f55" has successfully progressed.
          reason: NewReplicaSetAvailable
          status: 'True'
          type: Progressing
      observedGeneration: 1
      readyReplicas: 1
      replicas: 1
      updatedReplicas: 1

    ---
    apiVersion: v1
    kind: Service
    metadata:
      annotations: {}
      labels:
        k8s.kuboard.cn/name: web
      name: web
      namespace: exxk
      resourceVersion: '126824'
    spec:
      clusterIP: 10.233.80.181
      clusterIPs:
        - 10.233.80.181
      internalTrafficPolicy: Cluster
      ipFamilies:
        - IPv4
      ipFamilyPolicy: SingleStack
      ports:
        - name: gre3pw
          port: 80
          protocol: TCP
          targetPort: 80
      selector:
        k8s.kuboard.cn/name: web
      sessionAffinity: None
      type: ClusterIP
    status:
      loadBalancer: {}

    ---
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations: {}
      labels:
        k8s.kuboard.cn/name: web
      name: web
      namespace: exxk
      resourceVersion: '128138'
    spec:
      ingressClassName: myingresscontroller
      rules:
        - host: web.iexxk.io
          http:
            paths:
              - backend:
                  service:
                    name: web
                    port:
                      number: 80
                path: /
                pathType: Prefix
    status:
      loadBalancer:
        ingress:
          - ip: 172.16.3.165

  6. 额外，如果要不同域名对应不同的集群，nginx设置如下

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    server {
        listen       80;
        server_name .iexxk.io;
        #access_log  /var/log/nginx/hoddst.access.log  main;

        location / {
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_pass http://172.16.3.165:32211/;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   /usr/share/nginx/html;
        }
    }
    server {
        listen       80;
        server_name .test.io;
        #access_log  /var/log/nginx/hoddst.access.log  main;

        location / {
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_pass http://172.16.3.160:32211/;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   /usr/share/nginx/html;
        }
    }