13 521
правка
Изменения
Нет описания правки
== Bluestore vs Filestore ==
Bluestore is Ceph’s the «new» storage layerof Ceph. It’s reasonable to expect better performance and basically better everything from something «new».Is it correct for Bluestore?
Yes, Bluestore is really 2x faster than Filestore for linear write workloads, because it has no double-writes — big blocks are written only once, not twice as in Filestore. Filestore journals everything, so all writes first go to the journal and the get copied to the main device. Bluestore is also more feature-rich: it has checksums, compression, erasure-coded overwrites and virtual clones (which make «fast» snapshots faster(thanks to the «virtual clones»).
However, the random write performance varies:
* Bluestore is 2x faster than Filestore in HDD-only (or bad-SSD-only) configurations, because it can do 1 commit per write, at least if you remove the handbrake from hereapply this patch: https://github.com/ceph/ceph/pull/26909 or if you wait for the backportand turn . In fact it’s OK to say that Bluestore’s deferred-write algorithm is really optimal for HDD-only transactional write.
* In HDD+SDD configurations, Bluestore performance is very consistent, but it’s worse than the peak performance of Filestore for the same hardware. This is because Bluestore’s deferred write queue doesn’t act as a «temporary buffer» that can smooth burst random write loads. In other words, Bluestore OSD refuses to do random writes faster than the HDD can do on average.
*: So with Filestore you can easily get 1000—2000 iops (iodepth=1). But after the journal becomes full it will drop very low, to 30-50 iops. With Bluestore you only get 100-300 100—300 iops for a single HDD regardless of the SSD journal, but these are absolutely stable over time.*: This is caused by the very short deferred write queue being very short(it’s only 64 requests) and the lack of any background flush thread.* In SSD-only (All-Flash) clusters Bluestore’s latency is 1.5-2 times greater than Filestore’s. However, even though the latency is greater, the total throughput is usually slightly better (+5..10 %) and total CPU usage is slightly lower (-5..10 %).
== RAID WRITE HOLE ==
