Now Available: Ceph RBD as Persistent Storage for Kubernetes

You can now use Ceph RBD as Persistent Storage for containers in Kubernetes.  Examples and instructions can be found here.

Now you have the following storage options for Kubernets.

Type Format Duration Provider
EmptyDir File Ephermal Local host
HostDir File Persistent Local host
GitRepo File Persistent Git repository
GCE PD Block Persistent GCE
AWS EBS Block Persistent AWS
NFS File Persistent NFS Server
iSCSI Block Persistent iSCSI target provider
Glusterfs File Persistent Glusterfs Servers
Ceph RBD Block Persistent Ceph Cluster
Advertisements

5 thoughts on “Now Available: Ceph RBD as Persistent Storage for Kubernetes

  1. Hi,
    Thank you for your Ceph plugin, we use it for production use @ hebus.io , but we have some trouble when a container crash and restart, especialy when using database, instance become extremly slow. Do you have any advice about it ? We use Ext4.
    Thank you

    Like

    1. Without looking at your instance, I am blind to provide the right answer. But I think it is likely that :
      – Depending on the kubernetes release, fsck was kicked off[0] upon mount.
      – Your database container was recovering from crash and that could cause database read from logs and repair possible corruptions.

      In both cases, you can run top command to see which process is running on the top.

      [0]. https://github.com/kubernetes/kubernetes/blob/master/pkg/util/mount/mount_linux.go#L251

      Like

      1. Thank you for reply ,
        I’m running Kubernetes 1.2.0, with postgres database ( image sameersbn/postgresql:9.4-12 ) With Odoo.
        Looks you are right the database try to recover, but it appear that Odoo check structure too and rebuild relation.
        Do you know if we can prevent recover from log on the database side or it’s bad idea ?

        Like

      2. I suggest you keep postgres doing its things and leave no remorse for later. The slowness could be resolved by tuning ext4 with some tricks (e.g. with format your RBD image with ext4 options like discard, nobarrier and stripe width match RBD stripe). If your RBD image is formatted by Kubernetes, only the most reliable (yet low performing) options are applied.

        Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s