On March 25th, Chris Mellor broke the news that SUSE and Ceph are parting ways.
It’s a head-scratcher, but it matches up with what we have also heard from channel partners and prospects participating in the Ceph community.
If Mellor’s analysis is correct, SUSE has decided to go all-in on Longhorn. But that leaves a huge gap in their enterprise storage coverage. According to last year’s Ceph user survey, 3% of users reported exclusively running Ceph with Kubernetes. The other 38% that reported using Kubernetes also reported using Ceph with a wide variety of other workloads including OpenStack, VMware, Windows, and direct attach to Linux.
And while those numbers are bound to increase in this year’s survey as container adoption changes, it goes to show that multi-workload use is the norm.
Since Longhorn is Kubernetes focused, it just won’t solve those problems, potentially putting lots of users in the position of choosing whether to run disparate storage stacks or determining whether they can get the functionality they need from one.
To help with that decision, I’ve highlighted some key differences between Ceph (without Rook) and Longhorn. Where possible, I based the contrast on the CNCF Storage whitepaper definitions.
Distributed vs. hyperconverged storage stack
- Ceph is a distributed storage stack.
- Longhorn is typically a hyperconverged and distributed storage stack.
- This provides data locality for cases where that’s valuable, but it comes with some caveats and drawbacks. It’s great for small, independent software teams who have both hardware and software purchasing authority. Simpler is better, especially when it comes to managing the complexity of cloud-native technologies.
- But the CNCF Storage Whitepaper calls out an important consideration – “a reduction in the separation of concerns in hyperconverged systems can have an impact on security and operational complexity as maintenance operations, or any node failure, not only impacts the workload on that node, but also the underlying storage system.”
- For cloud-native acolytes, this sounds an awful lot like turning your cattle into pets.
- For larger enterprises, especially those where the DevOps team isn’t the one that has to purchase and maintain the infrastructure, having a cluster to care and feed that can only serve containers, might come with unwanted overhead. It’s operationally expensive to have a different storage stack for every workload.
- Data locality is best-effort only – meaning that longhorn will try to keep a replica on the same node as the attached volume. But it can’t guarantee that. And as containers shuffle around, it becomes unreasonable to expect it to.
Storage media options
Data access interfaces
- Ceph provides block, file, and object storage. With SoftIron Storage Router, you get additional SMB, iSCSI, and NFS functionality which may be important to enterprises making the transition from traditional applications to cloud-native architectures and want to make an investment that will support them through that change.
- Longhorn provides block storage only, although they do support backup to file or object-based storage systems
- Ceph favors consistency and correctness over performance and supports both erasure-coding and replication-based data durability mechanisms.
- Longhorn uses replication-based data protection only today, although I suspect that’s a feature that will come in time as more requests come in.
- This is critical to the storage purchasing calculation because it has a massive impact on the $/GB metric. That becomes even more critical when you’re dealing with SSD-only architectures.
- What’s more, this is a tunable feature in Ceph, which makes it somewhat unique in the industry. It’s especially useful for organizations that practice differentiated services delivery like managed service providers, providing different service level agreements at different price objectives to their users or customers
- Ceph has a diverse ecosystem of vendors providing support services.
- SoftIron, for example, provides two models – our pre-optimized HyperDrive appliances which integrate into any Ceph environment and provides the deepest integration between Ceph and hardware, and we offer HyperSafe as a pure Ceph software support offering for those who need to use other hardware.
- But there’s also Red Hat, Canonical, and several other regional organizations that can provide Ceph support
- By contrast, Longhorn is supported by a single vendor today. As with other open-source projects, that may change in the future. But for the moment, if you want Longhorn support, you must get it from SUSE. That sounds like vendor lock-in.
Important similarities between the two
- Cloud-native: both storage implementations are considered “cloud-native” in the industry. In fact, some believe that Ceph is the most popular cloud-native storage system that isn’t delivered only as a service
- Snapshots: both products provide snapshots at the volume level. This is especially useful for stateful applications
- Both support ARM-based architectures: We were the first to productize Ceph on ARM64, and in January the Longhorn team announced similar support. Right-sizing your hardware for your task is critical for efficient computing, especially at the edge.
That’s where SoftIron has excelled and will continue to focus – building right-sized hardware to run open-source infrastructure. We’re happy to provide HyperSafe as a support path for any SUSE customers left hanging. Open a chat, or drop us a line.