13 494
правки
Изменения
Нет описания правки
Main test cases for benchmarking are:
* Linear read and write (big blocks, big long queue) in MB/s
* Highly parallel random read and write of small blocks (4-8kb, iodepth=32-128) in IOPS (Input/Output ops per second)
* Single-threaded transactional random write (4-8kb, iodepth=1) and read (though single-threaded reads are more rare) in IOPS
Recommended benchmarking tools:
* fio -ioengine=rbd. Run the following:
*# fio -ioengine=rbd -direct=1 -name=test -bs=4M -iodepth=16 -rw=write -pool=rpool_hdd -runtime=60 -rbdname=testimg
*# fio -ioengine=rbd -direct=1 -name=test -bs=4k -iodepth=1 -rw=randwrite -pool=rpool_hdd -runtime=60 -rbdname=testimg
*# fio -ioengine=rbd -direct=1 -name=test -bs=4k -iodepth=128 -rw=randwrite -pool=rpool_hdd -runtime=60 -rbdname=testimg
*: Then repeat for rw=read/randread.
*: The idea is to test a) the best possible latency b) linear bandwidth c) random access iops.
*: Reading from an empty RBD image is very fast :) so pre-fill it before testing.
*: Run tests from node(s) where your actual RBD users will reside. The results are usually slightly better when you run tests from a separate physical server.
* The same through krbd:
*# fio -ioengine=libaio -sync=1 -direct=1 -name=test -bs=4M -iodepth=16 -rw=write -pool=rpool_hdd -runtime=60 -rbdname=testimg
*# fio -ioengine=rbd -direct=1 -name=test -bs=4k -iodepth=1 -rw=randwrite -pool=rpool_hdd -runtime=60 -rbdname=testimg
*# fio -ioengine=rbd -direct=1 -name=test -bs=4k -iodepth=128 -rw=randwrite -pool=rpool_hdd -runtime=60 -rbdname=testimg
* Всё то же самое изнутри виртуалки или через krbd:
*# fio -ioengine=libaio -direct=1 -name=test -bs=4M -iodepth=16 -rw=write -runtime=60 -filename=/dev/rbdX
*# fio -ioengine=libaio -direct=1 -sync=1 -name=test -bs=4k -iodepth=1 -rw=randwrite -runtime=60 -filename=/dev/rbdX
*# fio -ioengine=libaio -direct=1 -name=test -bs=4k -iodepth=128 -rw=randwrite -runtime=60 -filename=/dev/rbdX
*: Заметьте, что при тестировании задержки добавилась опция -sync=1. Это не случайно, а соответствует режиму работы СУБД (транзакционная запись в 1 поток). В ioengine=rbd понятие sync отсутствует, там всё всегда «sync».
* The first recommended tool is again `fio` with `-ioengine=rbd -pool=<your pool> -rbdname=<your image>`. All of the above tests valid for raw drives can be repeated for RBD and they mean the same things. Sync, direct and invalidate flags can be omitted, because RBD has no concept of «sync» — all operations are always «sync». And there’s no page cache involved either, so «direct» also doesn’t mean anything.
