Изменения

Single-threaded random reads and writes are where the latency matters, and the latency doesn't doesn’t scale with the number of servers. Whenever you're you’re benchmarking your cluster with iodepth=1 you're you’re benchmarking only ONE placement group (triplet or pair of OSDs) at a time. The result is only affected by how fast 1 OSD is responding to 1 request.

The latency really matters because not many applications can do random writes with high parallelism/iodepth. For example, a DBMS can'tcan’t, because it's it’s transactional and it needs to serialize its writes to the journal.

The naive expectation is that Ceph should be almost as fast as drives and network are, because everyone is used to the idea that I/O is slow and the software is fast. And this is true with Ceph only until you don't don’t use SSDs.

The general rule is: with Ceph it's is an SDS. With Ceph it’s hard to achieve random read latencies less than 0.5ms and random write latencies less than 1ms, ''no matter what drives or network you use''. This is stands for only 2000 iops random read and 1000 iops random write, and it's already even this result is good if you manage to achieve it. With BIS hardware and some tuning you may be able to improve it twice, but not moreonly twice or so.

There is Nick Fisk's Fisk’s presentation titled "Low«Low-latency Ceph"Ceph». By "low«low-latency" latency» he only means 0.7ms, which is only ~1500 iops.