Commit Graph

845 Commits (nbd-vmsplice)

Author SHA1 Message Date
Vitaliy Filippov 1f6c4c79d6 vmsplice+splice experiment in stub_osd to test it too 2021-11-22 01:20:12 +03:00
Vitaliy Filippov 4936c42132 Splice via io_uring - bad result too
40% CPU according to perf is lost inside do_splice() -> unix_stream_sendpage()
without io_uring and in various exc_page_fault() with io_uring
2021-11-22 00:13:27 +03:00
Vitaliy Filippov 6c3248a36c Experiment: vmsplice+splice "zero-copy" read in NBD 2021-11-22 00:12:39 +03:00
Vitaliy Filippov a863013cb2 Add a patch for qemu 6.1 and replace _ with - in qemu options 2021-11-21 16:16:46 +03:00
Vitaliy Filippov 660c3f7b0d Change default RDMA settings to 128x 129K buffers
129K to leave extra space for the header

The problem with 8x 1M buffers is that the following happens with,
for example, 2 OSDs and 4M T1Q1 write:
- Server posts 8 receives
- Client posts 8 sends
- WRs are processed by the RDMA stack, but the OSD doesn't have the time
  to handle them and doesn't refill buffers
- Client posts 1 more send
- RNR retransmission happens and performance drops to zero

Overall it seems that RDMA support should be reworked to use real 'RDMA'
operations i.e. operations writing into remote memory. This has an
additional advantage of avoiding a copy at the receive side of the OSD.
2021-11-21 12:05:52 +03:00
Vitaliy Filippov f0ebfae3b8 Fix vitastor-cli alloc-osd, use vitastor-cli in make-osd.sh 2021-11-21 00:01:03 +03:00
Vitaliy Filippov eb7ad2c114 Fix empty size syntax, use C version of simple-offsets in tests 2021-11-20 23:51:26 +03:00
Vitaliy Filippov cd21ff0b6a Rewrite simple-offsets.js in C/C++ 2021-11-19 02:39:56 +03:00
Vitaliy Filippov d3903f039c Implement alloc-osd (allocate a new OSD number) command 2021-11-19 02:39:37 +03:00
Vitaliy Filippov 66fe1a469b Additionally balance parity chunks over OSDs using round-robin when generating initial distribution 2021-11-16 21:02:39 +03:00
Vitaliy Filippov 24409bd4c4 Oops # 2. Fix vitastor-cli create broken due to json11 :D 2021-11-16 12:52:35 +03:00
Vitaliy Filippov c5029961ea Oops. Fix vitastor-cli ls 2021-11-16 12:39:41 +03:00
Vitaliy Filippov 1ca1143d4a Add fio_version and qemu_version variable example files back 2021-11-15 17:21:01 +03:00
Vitaliy Filippov 920345f7b6 Release 0.6.8
- Build separate packages for OSD, monitor, client, C header, fio and QEMU drivers
  instead of one package which included everything
2021-11-15 00:49:21 +03:00
Vitaliy Filippov 75b47a6298 Generate pkg-config file 2021-11-15 00:49:21 +03:00
Vitaliy Filippov 6e446653ae Include README 2021-11-15 00:00:02 +03:00
Vitaliy Filippov e51edf2542 Split into multiple rpm packages 2021-11-14 23:48:50 +03:00
Vitaliy Filippov ce170af91f Split into multiple Debian packages 2021-11-14 23:31:30 +03:00
Vitaliy Filippov 7eabc364bf Release 0.6.7
- Implement CLI commands for listing, viewing I/O statistics, creating,
  snapshotting, cloning, resizing and modifying images. All these operations
  are covered by 3 commands: ls, create, modify
- Implement an important fix to prior OSD set tracking for PGs. The previous
  version had an issue which could lead to data loss due to an OSD with older
  copy of the data thinking it has the newest copy
- Fix I/O statistics aggregation in the monitor
- Several minor fixes for Cinder driver
- Fix QEMU driver to be compatible with QEMU 2.x > 2.0
- Fix stalls sometimes possible in configurations without immediate_commit due
  to insufficient amount of automatic internal fsync operations
- Add `vita` alias for `vitastor-cli`
2021-11-13 23:23:55 +03:00
Vitaliy Filippov a346f84c69 Allow to show only specific images in listing 2021-11-13 23:23:55 +03:00
Vitaliy Filippov 71a0c1a7b9 Fix list sorting 2021-11-13 23:23:55 +03:00
Vitaliy Filippov 20e86c7d84 Note about vitastor-cli 2021-11-13 22:39:17 +03:00
Vitaliy Filippov 110b39900b Rename the new "set" command to "modify" 2021-11-13 22:39:17 +03:00
Vitaliy Filippov 697ee30a26 Add ARG REL into build Dockerfiles 2021-11-13 22:39:17 +03:00
Vitaliy Filippov 42479b4590 Fix vitastor-nbd list, add ls alias 2021-11-13 22:39:17 +03:00
Vitaliy Filippov 6e82044e84 Add `vita` symlink 2021-11-13 22:39:17 +03:00
Vitaliy Filippov 2cb3e84882 Implement CLI set (resize, change readonly status) command 2021-11-13 22:39:17 +03:00
Vitaliy Filippov 32614c5bc8 Add a test for the last change ("split-brain avoidance", lol) 2021-11-13 22:39:17 +03:00
Vitaliy Filippov aa436027c8 Report pg/history from OSD on every degraded activation
Required to prevent data loss due to activation of an OSD with older data
when PG OSD set change doesn't occur. I.e. fixes the simplest case:
- Run 2 OSDs with 1 PG
- Start writing into the PG
- Stop OSD 2
- Stop OSD 1
- Start OSD 2

After this change the PG will refuse to start after the last step.
2021-11-13 22:39:17 +03:00
Vitaliy Filippov 577a563b91 Allow to disable colored output 2021-11-11 01:41:58 +03:00
Vitaliy Filippov e4efa2c08a Improve vitastor-cli ls - show I/O statistics, allow to sort & limit output 2021-11-11 01:41:58 +03:00
Vitaliy Filippov 0f3f0a9d29 Calculate average statistics in mon, remove buggy "fix_stat_overflows" 2021-11-11 00:20:57 +03:00
Vitaliy Filippov 0544a16f95
Merge pull request #30 from moly7x/moly7x-fix-No-weighed-backend-found
Fix No weighed backend found
2021-11-10 17:13:17 +03:00
Tân Lê 30d8930958
Revert spacing in comment 2021-11-10 10:47:58 +07:00
Tân Lê baf003fbd3
Revert newline in comment 2021-11-10 10:43:24 +07:00
Tân Lê ba39a38dc4
Fix No weighed backend found 2021-11-10 10:27:01 +07:00
Vitaliy Filippov d528cd77f1 Fix install_symlink 2021-11-09 16:42:29 +03:00
Vitaliy Filippov 6e6f407df3 Simplify & fix monitor stats aggregation 2021-11-09 01:41:22 +03:00
Vitaliy Filippov 4d43774cbb Use 5s etcd_report_interval by default 2021-11-09 01:27:12 +03:00
Vitaliy Filippov a1488f7217 Fix qemu_driver to build with QEMU 2.x (previously it was only correct for QEMU 2.0) 2021-11-08 23:07:31 +03:00
Vitaliy Filippov 404e07d365 Implement image/snapshot/clone creation and listing by pool 2021-11-07 01:01:07 +03:00
Vitaliy Filippov b3dcee0d43 Also print "bare" inodes with missing config if they occupy space 2021-11-06 14:56:41 +03:00
Vitaliy Filippov 609bd4eb59 Remove naggy RDMA messages when log level is zero 2021-11-06 14:36:23 +03:00
Vitaliy Filippov 8e445ddc9a Begin to implement CLI: implement listing, add help, add create stub 2021-11-06 14:32:19 +03:00
Vitaliy Filippov ffb06536ff Revoke lease in mon on SIGINT & SIGTERM, fix raw_to_usable calculation 2021-11-06 13:54:35 +03:00
Vitaliy Filippov eeecab20c2
Merge pull request #28 from moly7x/fix
Fix building QEMU 3.1
2021-11-06 11:12:17 +03:00
Tân Lê e889ac4209
Fix building QEMU 3.1 2021-11-05 13:45:51 +07:00
Vitaliy Filippov cfe8de9b84 Autosync based on number of unstable ops to prevent journal stalls 2021-10-30 14:26:48 +03:00
Vitaliy Filippov 24b9b19066 Change rm to rm-data as it was renamed 2021-10-23 10:39:20 +03:00
Vitaliy Filippov ef645ee0c2
Merge pull request #26 from moly7x/fix
FIX ERROR cinder.volume.drivers.vitastor KeyError: 'total_raw_tb'
2021-10-22 21:29:52 +03:00