I thought it would be useful to present some of the benchmarking work I’ve been undertaking at this year’s virtual FOSDEM 2021 event. Usually held in Brussels, FOSDEM is an annual free and non-commercial two-day event organised by volunteers to promote the widespread use of free and open source software, and one of the leading open source events in Europe.
Benchmarking reliably and sensibly gets more challenging as you scale out storage clusters, gateways, and benchmarking drivers. For S3 in particular there are many moving parts to get right to ensure that you’re not benchmarking your load-generation clients and that your system isn’t a bottleneck. Cosbench has been one of the primary tools to get this right in recent years, and has historically had wide-ranging support for a plethora of object storage systems.
While the tool itself is very useful and even has support for Ceph’s Librados, it’s definitely a bit rough around the edges – it hasn’t seen real core development in many years and is lacking many of the simple usability tools most open source projects typically provide. In this short overview, I go over how you typically architect a Cosbench benchmarking system, how to structure a workload file, and some of the tooling that we’ve built to make this easier to use – I even run a few benchmarks on my test system. I’ll also talk a bit more about where the project leaves much to be desired and how we might be able to fix some of these aspects.
Here’s the video of my talk. I hope you find it useful.
If you’re undertaking some Ceph benchmarking work and want to dig a little deeper into the topic, find me on Twitter and I’ll be happy to help if I can (@dabukalam) or, contact SoftIron here.
