Commit Graph

44 Commits (master)

Author SHA1 Message Date
Vitaliy Filippov ae5af04fde Add noout flag for OSDs 2024-04-16 02:19:55 +03:00
Vitaliy Filippov 6366972fe8 Warn about full and almost full OSDs in status
Test / test_splitbrain (push) Successful in 18s Details
Test / test_snapshot_chain (push) Successful in 2m23s Details
Test / test_snapshot_chain_ec (push) Successful in 2m53s Details
Test / test_rebalance_verify_imm (push) Successful in 3m21s Details
Test / test_rebalance_verify (push) Successful in 3m46s Details
Test / test_switch_primary (push) Successful in 33s Details
Test / test_write (push) Successful in 54s Details
Test / test_write_xor (push) Successful in 48s Details
Test / test_write_no_same (push) Successful in 14s Details
Test / test_rebalance_verify_ec (push) Successful in 4m38s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m22s Details
Test / test_heal_pg_size_2 (push) Successful in 3m34s Details
Test / test_heal_ec (push) Successful in 3m38s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m44s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m51s Details
Test / test_heal_csum_32k (push) Successful in 6m45s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m34s Details
Test / test_enospc (push) Successful in 1m47s Details
Test / test_enospc_xor (push) Successful in 2m41s Details
Test / test_enospc_imm (push) Successful in 1m31s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m39s Details
Test / test_heal_csum_4k (push) Successful in 6m15s Details
Test / test_scrub_zero_osd_2 (push) Successful in 32s Details
Test / test_scrub (push) Successful in 35s Details
Test / test_scrub_xor (push) Successful in 26s Details
Test / test_enospc_imm_xor (push) Successful in 1m13s Details
Test / test_nfs (push) Successful in 24s Details
Test / test_scrub_ec (push) Successful in 33s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 34s Details
Test / test_scrub_pg_size_3 (push) Successful in 42s Details
2024-04-07 19:39:51 +03:00
Vitaliy Filippov 2b863fb715 Add ENOSPC handling tests 2024-04-07 19:39:33 +03:00
Vitaliy Filippov 3629dbc54d Plug the new PG combinator into monitor
Test / test_move_reappear (push) Successful in 22s Details
Test / test_snapshot_down (push) Successful in 25s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 2m46s Details
Test / test_snapshot_down_ec (push) Successful in 24s Details
Test / test_splitbrain (push) Successful in 17s Details
Test / test_snapshot_chain (push) Successful in 2m36s Details
Test / test_snapshot_chain_ec (push) Successful in 3m1s Details
Test / test_rebalance_verify_imm (push) Successful in 3m17s Details
Test / test_rebalance_verify (push) Successful in 3m50s Details
Test / test_switch_primary (push) Successful in 33s Details
Test / test_write (push) Successful in 50s Details
Test / test_write_xor (push) Successful in 56s Details
Test / test_write_no_same (push) Successful in 14s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m29s Details
Test / test_rebalance_verify_ec (push) Successful in 5m23s Details
Test / test_heal_pg_size_2 (push) Successful in 4m23s Details
Test / test_heal_ec (push) Successful in 4m57s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m21s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m33s Details
Test / test_heal_csum_32k (push) Successful in 6m55s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m54s Details
Test / test_scrub (push) Successful in 1m32s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m12s Details
Test / test_heal_csum_4k_dj (push) Successful in 7m12s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m1s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m41s Details
Test / test_heal_csum_4k (push) Successful in 6m22s Details
Test / test_scrub_ec (push) Successful in 44s Details
Test / test_nfs (push) Successful in 16s Details
Test / test_scrub_xor (push) Successful in 18s Details
2024-04-07 02:44:17 +03:00
Vitaliy Filippov 5d3aaf016b Add administration docs 2024-03-31 01:54:52 +03:00
Vitaliy Filippov 989675a780 s/etcd_ws_keepalive_timeout/etcd_ws_keepalive_interval/ in docs 2024-03-26 01:56:08 +03:00
Vitaliy Filippov f8c403ec9e Add newer benchmark results 2024-03-23 18:28:48 +03:00
Vitaliy Filippov 6783d4a13c Implement fool protection for FS pools 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 4cf6dceed7 Merge branch 'rel-1.4'
Test / test_minsize_1 (push) Has been cancelled Details
Test / test_move_reappear (push) Has been cancelled Details
Test / test_rm (push) Has been cancelled Details
Test / test_snapshot_chain (push) Has been cancelled Details
Test / test_snapshot_chain_ec (push) Has been cancelled Details
Test / test_snapshot_down (push) Has been cancelled Details
Test / test_snapshot_down_ec (push) Has been cancelled Details
Test / test_splitbrain (push) Has been cancelled Details
Test / test_rebalance_verify (push) Has been cancelled Details
Test / test_rebalance_verify_imm (push) Has been cancelled Details
Test / test_rebalance_verify_ec (push) Has been cancelled Details
Test / test_rebalance_verify_ec_imm (push) Has been cancelled Details
Test / test_switch_primary (push) Has been cancelled Details
Test / test_write (push) Has been cancelled Details
Test / test_write_xor (push) Has been cancelled Details
Test / test_write_no_same (push) Has been cancelled Details
Test / test_heal_pg_size_2 (push) Has been cancelled Details
Test / test_heal_ec (push) Has been cancelled Details
Test / test_heal_csum_32k_dmj (push) Has been cancelled Details
Test / test_heal_csum_32k_dj (push) Has been cancelled Details
Test / test_heal_csum_32k (push) Has been cancelled Details
Test / test_heal_csum_4k_dmj (push) Has been cancelled Details
Test / test_heal_csum_4k_dj (push) Has been cancelled Details
Test / test_heal_csum_4k (push) Has been cancelled Details
Test / test_scrub (push) Has been cancelled Details
Test / test_scrub_zero_osd_2 (push) Has been cancelled Details
Test / test_scrub_xor (push) Has been cancelled Details
Test / test_scrub_pg_size_3 (push) Has been cancelled Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Has been cancelled Details
Test / test_scrub_ec (push) Has been cancelled Details
2024-02-29 09:59:01 +03:00
Vitaliy Filippov 6bf1f539a6 Add EIO retry timeout and allow to disable these retries, rename up_wait_retry_interval to client_retry_interval 2024-02-28 13:10:02 +03:00
Vitaliy Filippov 4eab26f968 Add documentation and a very basic test for pool management commands
Test / test_snapshot_ec (push) Successful in 31s Details
Test / test_rm (push) Successful in 17s Details
Test / test_move_reappear (push) Successful in 24s Details
Test / test_snapshot_down (push) Successful in 27s Details
Test / test_snapshot_down_ec (push) Successful in 33s Details
Test / test_splitbrain (push) Successful in 20s Details
Test / test_snapshot_chain (push) Successful in 2m15s Details
Test / test_snapshot_chain_ec (push) Successful in 2m58s Details
Test / test_rebalance_verify_imm (push) Successful in 5m3s Details
Test / test_rebalance_verify (push) Successful in 5m36s Details
Test / test_switch_primary (push) Successful in 37s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m3s Details
Test / test_write_no_same (push) Successful in 21s Details
Test / test_write (push) Successful in 58s Details
Test / test_write_xor (push) Successful in 1m31s Details
Test / test_rebalance_verify_ec (push) Successful in 6m20s Details
Test / test_heal_pg_size_2 (push) Successful in 4m7s Details
Test / test_heal_ec (push) Successful in 4m33s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m53s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m17s Details
Test / test_heal_csum_32k (push) Successful in 7m23s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m56s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m26s Details
Test / test_scrub (push) Successful in 1m29s Details
Test / test_heal_csum_4k_dj (push) Successful in 7m1s Details
Test / test_scrub_xor (push) Successful in 1m1s Details
Test / test_heal_csum_4k (push) Successful in 6m34s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 32s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m19s Details
Test / test_scrub_ec (push) Successful in 24s Details
2024-02-28 13:08:04 +03:00
Vitaliy Filippov f6cd9f9153 Add a note about pg_minsize 2024-02-15 23:38:52 +03:00
Vitaliy Filippov bb2f395f1e Add cutoff threshold for recovery auto-tuning 2024-02-11 16:13:52 +03:00
Vitaliy Filippov d2b43cb118 Change default etcd_mon_ttl
Test / test_move_reappear (push) Successful in 35s Details
Test / test_interrupted_rebalance_ec (push) Successful in 3m29s Details
Test / test_interrupted_rebalance (push) Successful in 4m47s Details
Test / test_snapshot_down (push) Successful in 29s Details
Test / test_snapshot_down_ec (push) Successful in 25s Details
Test / test_splitbrain (push) Successful in 24s Details
Test / test_snapshot_chain (push) Successful in 2m46s Details
Test / test_snapshot_chain_ec (push) Failing after 3m10s Details
Test / test_rebalance_verify_imm (push) Successful in 4m24s Details
Test / test_rebalance_verify (push) Successful in 4m54s Details
Test / test_switch_primary (push) Successful in 35s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m38s Details
Test / test_write (push) Successful in 46s Details
Test / test_write_xor (push) Successful in 49s Details
Test / test_write_no_same (push) Successful in 18s Details
Test / test_rebalance_verify_ec (push) Successful in 7m14s Details
Test / test_heal_pg_size_2 (push) Successful in 4m10s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m10s Details
Test / test_heal_csum_32k_dj (push) Successful in 4m52s Details
Test / test_heal_csum_32k (push) Successful in 5m20s Details
Test / test_heal_csum_4k_dmj (push) Successful in 5m8s Details
Test / test_heal_ec (push) Failing after 10m21s Details
Test / test_scrub (push) Successful in 1m2s Details
Test / test_scrub_xor (push) Successful in 54s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m4s Details
Test / test_heal_csum_4k_dj (push) Successful in 4m48s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m26s Details
Test / test_scrub_ec (push) Successful in 50s Details
Test / test_scrub_pg_size_3 (push) Failing after 2m5s Details
Test / test_heal_csum_4k (push) Successful in 4m33s Details
2024-01-29 23:45:19 +03:00
Vitaliy Filippov 1c322b33ed Change default up_wait_retry_interval to 50 ms
Test / test_rm (push) Successful in 14s Details
Test / test_interrupted_rebalance_ec (push) Successful in 3m59s Details
Test / test_snapshot_chain (push) Successful in 1m34s Details
Test / test_snapshot_down (push) Successful in 25s Details
Test / test_snapshot_down_ec (push) Successful in 29s Details
Test / test_splitbrain (push) Successful in 19s Details
Test / test_snapshot_chain_ec (push) Successful in 2m35s Details
Test / test_interrupted_rebalance (push) Successful in 8m15s Details
Test / test_rebalance_verify_imm (push) Successful in 3m54s Details
Test / test_switch_primary (push) Successful in 36s Details
Test / test_write (push) Successful in 35s Details
Test / test_rebalance_verify_ec (push) Successful in 4m48s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 2m51s Details
Test / test_write_no_same (push) Successful in 14s Details
Test / test_write_xor (push) Failing after 3m9s Details
Test / test_heal_pg_size_2 (push) Successful in 3m55s Details
Test / test_heal_ec (push) Successful in 3m50s Details
Test / test_rebalance_verify (push) Failing after 9m30s Details
Test / test_heal_csum_32k_dmj (push) Failing after 5m40s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m12s Details
Test / test_heal_csum_32k (push) Successful in 6m25s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m56s Details
Test / test_scrub (push) Successful in 1m4s Details
Test / test_scrub_zero_osd_2 (push) Successful in 55s Details
Test / test_scrub_xor (push) Successful in 56s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m19s Details
Test / test_scrub_pg_size_3 (push) Failing after 2m14s Details
Test / test_heal_csum_4k_dj (push) Successful in 5m53s Details
Test / test_scrub_ec (push) Successful in 1m1s Details
Test / test_heal_csum_4k (push) Successful in 5m17s Details
2024-01-26 01:51:08 +03:00
Vitaliy Filippov c17f76a3e4 Add documentation for recovery auto-tuning
Test / test_snapshot_ec (push) Successful in 26s Details
Test / test_move_reappear (push) Successful in 19s Details
Test / test_rm (push) Successful in 15s Details
Test / test_snapshot_down (push) Successful in 24s Details
Test / test_snapshot_down_ec (push) Successful in 26s Details
Test / test_snapshot_chain (push) Successful in 1m50s Details
Test / test_splitbrain (push) Successful in 52s Details
Test / test_snapshot_chain_ec (push) Successful in 2m31s Details
Test / test_rebalance_verify_imm (push) Successful in 2m28s Details
Test / test_rebalance_verify (push) Successful in 3m25s Details
Test / test_rebalance_verify_ec (push) Successful in 3m31s Details
Test / test_write (push) Successful in 1m17s Details
Test / test_write_no_same (push) Successful in 17s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m36s Details
Test / test_heal_pg_size_2 (push) Successful in 4m12s Details
Test / test_heal_ec (push) Successful in 5m20s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m36s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m11s Details
Test / test_heal_csum_32k (push) Successful in 6m13s Details
Test / test_scrub (push) Successful in 56s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m6s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m31s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m58s Details
Test / test_scrub_xor (push) Successful in 43s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m10s Details
Test / test_scrub_ec (push) Successful in 49s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m40s Details
Test / test_heal_csum_4k (push) Successful in 5m59s Details
Test / test_write_xor (push) Successful in 34s Details
Test / test_interrupted_rebalance (push) Successful in 1m19s Details
2023-12-31 01:23:17 +03:00
Vitaliy Filippov 53de2bbd0f Support VDUSE in CSI
VDUSE has multiple advantages:
- Better performance
- Lack of timeout problems
- And even the ability to recover after restart of the vitastor-csi pod!
2023-12-04 00:41:24 +03:00
Vitaliy Filippov 225eb2fe3d Support RDMA without ODP by stupidly copying memory. Disable ODP by default
ODP is slower than regular RDMA even with memory copy overhead

Example numbers:
- 3950000 random read iops without ODP vs 240000 iops with ODP
- 1447000 random write iops without ODP vs 101000 iops with ODP

Reference: https://tkygtr6.github.io/pub/ISPASS21_slides.pdf
2023-11-12 15:03:47 +03:00
Vitaliy Filippov eae81bbda6 Fix typo 2023-10-28 01:09:20 +03:00
Vitaliy Filippov a883e79507 Make docs to add etcd_stats_interval 2023-10-27 14:09:26 +03:00
Vitaliy Filippov be7e76f849 Split etcd_stats_interval out of etcd_report_interval
Test / test_interrupted_rebalance_ec (push) Successful in 1m46s Details
Test / test_snapshot_ec (push) Successful in 36s Details
Test / test_move_reappear (push) Successful in 19s Details
Test / test_rm (push) Successful in 15s Details
Test / test_snapshot_down (push) Successful in 29s Details
Test / test_snapshot_down_ec (push) Successful in 30s Details
Test / test_splitbrain (push) Successful in 26s Details
Test / test_snapshot_chain (push) Successful in 2m15s Details
Test / test_snapshot_chain_ec (push) Successful in 2m57s Details
Test / test_rebalance_verify_imm (push) Successful in 2m29s Details
Test / test_rebalance_verify (push) Successful in 3m40s Details
Test / test_write (push) Successful in 1m0s Details
Test / test_write_no_same (push) Successful in 13s Details
Test / test_write_xor (push) Successful in 50s Details
Test / test_rebalance_verify_ec (push) Successful in 4m58s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m14s Details
Test / test_heal_pg_size_2 (push) Successful in 4m21s Details
Test / test_heal_ec (push) Successful in 4m5s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m36s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m28s Details
Test / test_heal_csum_32k (push) Successful in 6m38s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m46s Details
Test / test_scrub_zero_osd_2 (push) Successful in 59s Details
Test / test_scrub (push) Successful in 1m16s Details
Test / test_scrub_xor (push) Successful in 53s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m57s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m18s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m7s Details
Test / test_heal_csum_4k (push) Successful in 5m43s Details
Test / test_scrub_ec (push) Successful in 32s Details
2023-10-27 01:26:26 +03:00
Vitaliy Filippov 6fd2cf5df6 Add documentation for the write-back cache 2023-10-27 01:26:26 +03:00
Vitaliy Filippov ff479a102d Make MON filter OSDs by block layout to prevent "refusing to start PGs of this pool" errors on misconfiguration 2023-09-16 17:52:17 +03:00
Vitaliy Filippov 7333022adf Add a third I/O mode: O_DIRECT|O_SYNC, change parameters to data_io/meta_io/journal_io 2023-08-09 00:57:08 +03:00
Vitaliy Filippov b8a1734465 Reword checksum docs one more time 2023-07-29 14:42:56 +03:00
Vitaliy Filippov 564df2eb5d Support using buffered I/O with O_SYNC instead of direct I/O 2023-07-29 12:17:18 +03:00
Vitaliy Filippov 21b5124a4b Document data_csum_type and csum_block_size parameters 2023-07-29 12:17:18 +03:00
Vitaliy Filippov ee0c78fd74 Fix default HDD block size in docs (actual size is 1 MB) 2023-07-09 13:12:30 +03:00
Vitaliy Filippov 2f9353df60 Markdown preprocessor to compile multiple .md files into one 2023-06-22 01:27:38 +03:00
Vitaliy Filippov 3f60fecd7c Fix typo 2023-05-21 18:37:01 +03:00
Vitaliy Filippov aea567cfbd Slightly improve scrub docs
Test / test_cas (push) Successful in 9s Details
Test / test_change_pg_count (push) Successful in 52s Details
Test / test_change_pg_count_ec (push) Successful in 1m0s Details
Test / test_change_pg_size (push) Successful in 16s Details
Test / test_create_nomaxid (push) Successful in 16s Details
Test / test_etcd_fail (push) Successful in 56s Details
Test / test_failure_domain (push) Successful in 13s Details
Test / test_interrupted_rebalance (push) Successful in 1m24s Details
Test / test_interrupted_rebalance_imm (push) Successful in 1m10s Details
Test / test_interrupted_rebalance_ec (push) Successful in 1m9s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m6s Details
Test / test_minsize_1 (push) Failing after 19s Details
Test / test_move_reappear (push) Successful in 28s Details
Test / test_rebalance_verify (push) Successful in 2m25s Details
Test / test_rebalance_verify_imm (push) Successful in 2m19s Details
Test / test_rebalance_verify_ec (push) Successful in 3m3s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 2m20s Details
Test / test_rm (push) Successful in 16s Details
Test / test_snapshot (push) Successful in 21s Details
Test / test_snapshot_ec (push) Successful in 28s Details
Test / test_splitbrain (push) Successful in 20s Details
Test / test_write_xor (push) Has started running Details
Test / test_heal_pg_size_2 (push) Has started running Details
Test / test_write (push) Has started running Details
Test / test_scrub (push) Has been cancelled Details
Test / test_scrub_zero_osd_2 (push) Has been cancelled Details
Test / test_scrub_xor (push) Has been cancelled Details
Test / test_scrub_pg_size_3 (push) Has been cancelled Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Has been cancelled Details
Test / test_scrub_ec (push) Has been cancelled Details
2023-05-21 12:52:30 +03:00
Vitaliy Filippov ce02f47de6 Allow to disable scrub_find_best 2023-05-21 12:33:38 +03:00
Vitaliy Filippov 8d40ad99a6 Add scrub documentation 2023-05-20 23:19:39 +03:00
Vitaliy Filippov 3475772b07 Add configuration online update documentation 2023-05-20 23:19:39 +03:00
Vitaliy Filippov 14d6acbcba Set default rdma_max_recv/send to 16/8, fix documentation 2023-02-28 11:00:56 +03:00
Vitaliy Filippov 998e24adf8 Add a new recovery_pg_switch setting to mix all PGs during recovery 2022-12-30 02:03:33 +03:00
Vitaliy Filippov 9491f81419 Add missing documentation for OSD tags 2022-11-20 00:50:13 +03:00
Vitaliy Filippov 5a10d135f3 Allow to configure block_size, bitmap_granularity and immediate_commit per-pool 2022-08-11 01:56:33 +03:00
Vitaliy Filippov 342517d126 Fix typo 2022-06-05 00:45:02 +03:00
Vitaliy Filippov 404e43dd2d Note that ISA-L does not need to be enabled separately 2022-06-04 22:58:02 +03:00
Vitaliy Filippov 1efbbb0c36 Make deleted inodes vanish from statistics after 60 seconds 2022-06-04 13:27:06 +03:00
Vitaliy Filippov a0cae4c180 Rename "jerasure" to "ec" in pool configuration, function names, fix documentation and Debian build scripts
Old pool configurations with "jerasure" also remain supported as an alias for "ec"
2022-06-03 15:40:00 +03:00
Vitaliy Filippov f826831282 Describe OSD placement tree and reweights 2022-05-17 10:42:11 +03:00
Vitaliy Filippov 5d47bbe04c Add documentation 2022-05-17 01:10:49 +03:00