基础镜像支持

Longhorn supports creation of block devices backed by a base image. Longhorn base images are packaged as Docker images. Public or private registries may be used as a distribution mechanism for your Docker base images.

Usage

Volumes backed by a base image can be created in three ways.

  1. UI - Create Longhorn volumes exposed as block device or iSCSI target
  2. Flexvolume Driver - Create Longhorn block devices and consume in Kubernetes pods
  3. CSI Driver - (Newer) Create Longhorn block devices and consume in Kubernetes pods

UI

On the Volume tab, click the Create Volume button. The Base Image field expects a Docker image name such as rancher/vm-ubuntu:16.04.4-server-amd64.

Flexvolume Driver

The flexvolume driver supports volumes backed by base image. Below is a sample Flexvolume definition including baseImage option.

flexVolume:
  driver: "rancher.io/longhorn"
  fsType: "ext4"
  options:
    size: "32Mi"
    numberOfReplicas: "3"
    staleReplicaTimeout: "20"
    fromBackup: ""
    baseImage: "rancher/longhorn-test:baseimage-ext4"
1
2
3
4
5
6
7
8
9

You do not need to (and probably shouldn't) explicitly set filesystem type fsType when base image is present. If you do, it must match the base image's filesystem or the flexvolume driver will return an error.

Try it out for yourself. Make sure the Longhorn driver deployer specifies flag --driver flexvolume, otherwise a different driver may be deployed. The following example creates an nginx pod serving content from a flexvolume with a base image and is accessible from a service.

kubectl --kubeconfig=kube_configxxx.yml create  -f https://raw.githubusercontent.com/rancher/longhorn-manager/master/examples/flexvolume/example_baseimage.yml
1

Wait until the pod is running.

kubectl --kubeconfig=kube_configxxx.yml  get  po/flexvol-baseimage -w
1

Query for the service you created.

kubectl --kubeconfig=kube_configxxx.yml  get  svc/flexvol-baseimage
1

Your service should look similar.

NAME                        TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
service/flexvol-baseimage   LoadBalancer   10.43.153.186   <pending>     80:31028/TCP   2m
1
2

Now let's access something packaged inside the base image through the Nginx webserver, exposed by the LoadBalancer service. If you have LoadBalancer support and EXTERNAL-IP is set, navigate to the following URL.

http://<EXTERNAL-IP>/guests/hd/party-wizard.gif
1

Otherwise, navigate to the following URL where NODE-IP is the external IP address of any Kubernetes node and NODE-PORT is the second port in the service (31028 in the example service above).

http://<NODE-IP>:<NODE-PORT>/guests/hd/party-wizard.gif
1

Finally, tear down the pod and service.

kubectl --kubeconfig=kube_configxxx.yml delete  -f https://raw.githubusercontent.com/rancher/longhorn-manager/master/examples/flexvolume/example_baseimage.yml
1

CSI Driver

The CSI driver supports volumes backed by base image. Below is a sample StorageClass definition including baseImage option.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
provisioner: rancher.io/longhorn
parameters:
  numberOfReplicas: '3'
  staleReplicaTimeout: '30'
  fromBackup: ''
  baseImage: rancher/longhorn-test:baseimage-ext4
1
2
3
4
5
6
7
8
9

Let's walk through an example. First, ensure the CSI Plugin is deployed.

kubectl --kubeconfig=kube_configxxx.yml -n   longhorn-system get daemonset.apps/longhorn-csi-plugin
1

The following example creates an nginx statefulset with two replicas serving content from two csi-provisioned volumes backed by a base image. The statefulset is accessible from a service.

kubectl --kubeconfig=kube_configxxx.yml create  -f https://raw.githubusercontent.com/rancher/longhorn-manager/master/examples/provisioner_with_baseimage.yml
1

Wait until both pods are running.

kubectl -l app=provisioner-baseimage get po -w
1

Query for the service you created.

kubectl --kubeconfig=kube_configxxx.yml  get  svc/csi-baseimage
1

Your service should look similar.

NAME            TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
csi-baseimage   LoadBalancer   10.43.47.129   <pending>     80:32768/TCP   4m
1
2

Now let's access something packaged inside the base image through the Nginx webserver, exposed by the LoadBalancer service. If you have LoadBalancer support and EXTERNAL-IP is set, navigate to the following URL.

http://<EXTERNAL-IP>/guests/hd/party-wizard.gif
1

Otherwise, navigate to the following URL where NODE-IP is the external IP address of any Kubernetes node and NODE-PORT is the second port in the service (32768 in the example service above).

http://<NODE-IP>:<NODE-PORT>/guests/hd/party-wizard.gif
1

Finally, tear down the pod and service.

kubectl --kubeconfig=kube_configxxx.yml delete  -f https://raw.githubusercontent.com/rancher/longhorn-manager/master/examples/provisioner_with_baseimage.yml
1

Building

Creating and packaging an empty base image is a very simple process.

  1. Install QEMU.
  2. Create a qcow2 image.
qemu-img create -f qcow2 example.qcow2 4G
1
  1. Create the Dockerfile file with the following contents:
FROM busybox
COPY example.qcow2 /base_image/example.qcow2
1
2
  1. Build and publish the image:
DOCKERHUB_ACCT=rancher
docker build -t ${DOCKERHUB_ACCT}/longhorn-example:baseimage .
docker push ${DOCKERHUB_ACCT}/longhorn-example:baseimage
1
2
3

That's it! Your (empty) base image is ready for (no) use. Let's now explore some use cases for a base image and what we should do to our example.qcow2 before building and publishing.

Simple Filesystem

Suppose we want to store some static web assets in a volume. We have our qcow2 image and the web assets, but how to put the assets in the image?

On a Linux machine, load the network block device module.

sudo modprobe nbd
1

Use qemu-nbd to expose the image as a network block device.

sudo qemu-nbd -f qcow2 -c /dev/nbd0 example.qcow2
1

The raw block device needs a filesystem. Consider your infrastructure and choose an appropriate filesystem. We will use EXT4 filesystem.

sudo mkfs -t ext4 /dev/nbd0
1

Mount the filesystem.

mkdir -p example
sudo mount /dev/nbd0 example
1
2

Copy web assets to filesystem.

cp /web/images/* example/
1

Unmount the filesystem, shutdown qemu-nbd, cleanup.

sudo umount example
sudo killall qemu-nbd
rmdir example
1
2
3

Optionally, compress the image.

qemu-img convert -c -O qcow2 example.qcow2 example.compressed.qcow2
1

Follow the build and publish image steps and you are done. Example script.

Virtual Machine

See this document for the basic procedure of preparing Virtual Machine images.