k8s volume挂载卷

常用命令

1
2
3
4
5
6
7
8
9
10
#查询所有pvc
kubectl get pvc -n kubesphere-devops-system
#查看pvc详情信息
kubectl describe pvc ks-jenkins -n kubesphere-devops-system
#查询所有pv
kubectl get pv
#查看pv详情信息
kubectl describe pv pvc-fe2e8742-a736-4134-b89a-85eef0b6edce
kubectl get sc
kubectl describe sc local

基础概念

jQhr6g.png](https://imgtu.com/i/jQhr6g)](https://imgtu.com/i/jQfkqA)

  • PV(PersistentVolume) :描述的是持久化存储卷,是集群中的资源。
  • PVC(persistentVolumeClaim):申领是对这些资源的请求,也被用来执行对资源的申领检查。
  • SC(StorageClass):为管理员提供了描述存储 “类” 的方法。

pv和sc都属于集群所有,没有namespace之分,只有pvc才各属于每个空间的资源

简单nginx挂载持久卷实例

  1. 创建pvc:nginx-pvc.yaml文件

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
    name: nginx-pvc
    spec:
    accessModes: #访问模式的支持由storageClassName进行限定的
    - ReadWriteOnce
    #ReadWriteOnce(RWO)卷可以被一个节点以读写方式挂载。
    #ReadOnlyMany(ROX)卷可以被多个节点以只读方式挂载。
    #ReadWriteMany(RWX)卷可以被多个节点以读写方式挂载。
    #ReadWriteOncePod(RWOP)卷可以被单个 Pod 以读写方式挂载。
    resources:
    requests:
    storage: 1Gi #申请的资源大小
    storageClassName: local #指定存储类
  2. 执行kubectl apply -f nginx-pvc.yaml,可以通过kubectl get pvc进行查看

  3. 创建pod:nginx-pod-volumes.yaml文件

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    apiVersion: v1
    kind: Pod
    metadata:
    name: nginx
    labels: #标签
    app: nginx #标签为了后面service的selector
    spec:
    volumes:
    - name: nginx-data
    persistentVolumeClaim:
    claimName: nginx-pvc
    containers:
    - name: nginx
    image: nginx:alpine
    ports:
    - containerPort: 80
    volumeMounts:
    - mountPath: "/usr/share/nginx/html"
    name: nginx-data
  4. 执行kubectl apply -f nginx-pod-volumes.yaml,可以通过kubectl get po进行查看

  5. 创建对应的服务,暴露端口,见k8s 部署基础

  6. 访问nginx服务

  7. 通过kubectl get pvkubectl describe pv pvc-...查看source:path的路径/var/openebs/local/pvc-***

  8. 该路径在其中一个节点,找到有该路径的节点,在该目录新建一个index.html,里面输入hello pvc

  9. 重新访问nginx服务,就能在网页上看到hello pvc

pvc部署文件
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: ks-jenkins
namespace: kubesphere-devops-system
labels:
app: ks-jenkins
app.kubernetes.io/managed-by: Helm
chart: jenkins-0.19.0
heritage: Helm
release: ks-jenkins
annotations:
meta.helm.sh/release-name: ks-jenkins
meta.helm.sh/release-namespace: kubesphere-devops-system
pv.kubernetes.io/bind-completed: 'yes'
pv.kubernetes.io/bound-by-controller: 'yes'
volume.beta.kubernetes.io/storage-provisioner: openebs.io/local
volume.kubernetes.io/selected-node: k8s-worker-242-17 #在那个节点上
finalizers:
- kubernetes.io/pvc-protection
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
volumeName: pvc-fe2e8742-a736-4134-b89a-85eef0b6edce #pv的name
storageClassName: local
volumeMode: Filesystem
Pod部署文件:
1
2
3
4
5
6
7
8
9
10
11
12
kind: Pod
apiVersion: v1
spec:
volumes:
- name: jenkins-home #定义挂载卷
persistentVolumeClaim: #使用的pvc
claimName: ks-jenkins #使用的pvc名字
containers:
- name: ks-jenkins
volumeMounts:
- name: jenkins-home #使用的挂载卷
mountPath: /var/jenkins_home #容器内需要挂载的目录
pvc信息解析
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Name:          ks-jenkins #pvc名字
Namespace: kubesphere-devops-system
StorageClass: local
Status: Bound
Volume: pvc-fe2e8742-a736-4134-b89a-85eef0b6edce #挂载卷名字也就是pv名字
Labels: app=ks-jenkins
app.kubernetes.io/managed-by=Helm
chart=jenkins-0.19.0
heritage=Helm
release=ks-jenkins
Annotations: meta.helm.sh/release-name: ks-jenkins
meta.helm.sh/release-namespace: kubesphere-devops-system
pv.kubernetes.io/bind-completed: yes
pv.kubernetes.io/bound-by-controller: yes
volume.beta.kubernetes.io/storage-provisioner: openebs.io/local
volume.kubernetes.io/selected-node: k8s-worker-242-17 #那个节点
Finalizers: [kubernetes.io/pvc-protection]
Capacity: 20Gi #大小
Access Modes: RWO
VolumeMode: Filesystem
Mounted By: ks-jenkins-76d6b84fdf-qvj98
Events: <none>
pv信息解析
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Name:              pvc-fe2e8742-a736-4134-b89a-85eef0b6edce #pv名字
Labels: openebs.io/cas-type=local-hostpath
Annotations: pv.kubernetes.io/provisioned-by: openebs.io/local
Finalizers: [kubernetes.io/pv-protection]
StorageClass: local
Status: Bound
Claim: kubesphere-devops-system/ks-jenkins
Reclaim Policy: Delete
Access Modes: RWO
VolumeMode: Filesystem
Capacity: 20Gi
Node Affinity:
Required Terms:
Term 0: kubernetes.io/hostname in [k8s-worker-242-17] #在那个节点
Message:
Source:
Type: LocalVolume (a persistent volume backed by local storage on a node) #类型
Path: /var/openebs/local/pvc-fe2e8742-a736-4134-b89a-85eef0b6edce #在节点的路径
Events: <none>
SC信息解析
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Name:            local
IsDefaultClass: Yes
Annotations: cas.openebs.io/config=- name: StorageType
value: "hostpath"
- name: BasePath
value: "/var/openebs/local/"
,kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"storage.k8s.io/v1","kind":"StorageClass","metadata":{"annotations":{"cas.openebs.io/config":"- name: StorageType\n value: \"hostpath\"\n- name: BasePath\n value: \"/var/openebs/local/\"\n","openebs.io/cas-type":"local","storageclass.beta.kubernetes.io/is-default-class":"true","storageclass.kubesphere.io/supported-access-modes":"[\"ReadWriteOnce\"]"},"name":"local"},"provisioner":"openebs.io/local","reclaimPolicy":"Delete","volumeBindingMode":"WaitForFirstConsumer"}
,openebs.io/cas-type=local,storageclass.beta.kubernetes.io/is-default-class=true,storageclass.kubesphere.io/support-snapshot=false,storageclass.kubesphere.io/supported-access-modes=["ReadWriteOnce"] #限定pvc的访问模式
Provisioner: openebs.io/local #制备器(包含NFS、CephFS、AzureFile、Local等等)
Parameters: <none>
AllowVolumeExpansion: <unset>
MountOptions: <none>
ReclaimPolicy: Delete
VolumeBindingMode: WaitForFirstConsumer #卷绑定模式
Events: <none>

常见问题

  1. 错误信息,申明错误的访问模式,访问模式由sc存储类定义,local存储类不支持该访问模式

    1
    Warning  ProvisioningFailed    48s (x4 over 2m33s)    openebs.io/local_openebs-localpv-provisioner-7bbb56d7dc-zsq5k_8a90d62c-5e9f-4544-a7bb-50ca898c55a0  failed to provision volume with StorageClass "local": Only support ReadWriteOnce access mode
  2. 使用local存储卷时,节点不会从一个节点游走到另一个节点,因此节点挂机,该服务直接不可用了,损失了高可用,这是因为sc限制了访问模式为ReadWriteOnce,影响了节点亲和性