Commit Graph

176 Commits (master)

Author SHA1 Message Date
Vitaliy Filippov 9ad6822353 Release 1.5.0
Test / test_rm (push) Successful in 14s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m36s Details
Test / test_snapshot_down (push) Successful in 31s Details
Test / test_snapshot_down_ec (push) Successful in 30s Details
Test / test_splitbrain (push) Successful in 24s Details
Test / test_snapshot_chain (push) Successful in 2m20s Details
Test / test_snapshot_chain_ec (push) Successful in 3m5s Details
Test / test_rebalance_verify_imm (push) Successful in 5m11s Details
Test / test_rebalance_verify (push) Successful in 5m55s Details
Test / test_switch_primary (push) Successful in 33s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m26s Details
Test / test_write (push) Successful in 54s Details
Test / test_write_xor (push) Successful in 57s Details
Test / test_write_no_same (push) Successful in 19s Details
Test / test_rebalance_verify_ec (push) Successful in 7m21s Details
Test / test_heal_pg_size_2 (push) Successful in 4m36s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m33s Details
Test / test_heal_ec (push) Successful in 6m15s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m31s Details
Test / test_heal_csum_32k (push) Successful in 6m29s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m15s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m16s Details
Test / test_scrub (push) Successful in 1m18s Details
Test / test_scrub_xor (push) Successful in 1m13s Details
Test / test_heal_csum_4k_dj (push) Successful in 7m10s Details
Test / test_scrub_ec (push) Successful in 56s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 59s Details
Test / test_heal_csum_4k (push) Successful in 6m2s Details
Test / test_scrub_pg_size_3 (push) Successful in 2m11s Details
Test / test_nfs (push) Successful in 11s Details
After half a year of hard work, VitastorFS is finally here ! :-)

New features:
- VitastorFS, a full-featured clustered (read-write-many) file system.
  Documentation: [VitastorFS](docs/usage/nfs.en.md)
- Embedded key-value database implementation based on Parallel Optimistic B-Tree
  algorithm and used for the metadata of VitastorFS
- Pool management commands in vitastor-cli (create-pool, list-pools, rm-pool, modify-pool).
  Thanks MIND Software (https://mindsw.io) for their contribution!
  [Documentation](docs/usage/cli.en.md#create-pool)

Bug fixes:
- Fix a very rare "infinite loop" in the client library
- Fix a rare OSD hang on during start when zeroing out bad metadata entries left from the previous run
2024-03-16 15:35:10 +03:00
Vitaliy Filippov 8840c84572 Fix "bad key in etcd" in mon for FS pools 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 3aee37eadd Allow to disable per-inode stats for VitastorFS pools 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 38b8963330 Release 1.4.8
Test / test_rm (push) Successful in 19s Details
Test / test_move_reappear (push) Successful in 26s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m40s Details
Test / test_snapshot_down (push) Successful in 31s Details
Test / test_snapshot_down_ec (push) Successful in 34s Details
Test / test_splitbrain (push) Successful in 27s Details
Test / test_snapshot_chain (push) Successful in 2m18s Details
Test / test_snapshot_chain_ec (push) Successful in 2m59s Details
Test / test_rebalance_verify_imm (push) Successful in 5m32s Details
Test / test_rebalance_verify (push) Successful in 6m11s Details
Test / test_switch_primary (push) Successful in 41s Details
Test / test_write (push) Successful in 45s Details
Test / test_write_no_same (push) Successful in 23s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 5m2s Details
Test / test_write_xor (push) Successful in 55s Details
Test / test_rebalance_verify_ec (push) Successful in 6m22s Details
Test / test_heal_pg_size_2 (push) Successful in 5m41s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m59s Details
Test / test_heal_csum_32k_dj (push) Successful in 7m19s Details
Test / test_heal_csum_32k (push) Successful in 7m17s Details
Test / test_heal_csum_4k_dmj (push) Successful in 7m14s Details
Test / test_scrub (push) Successful in 1m12s Details
Test / test_heal_ec (push) Successful in 9m2s Details
Test / test_scrub_xor (push) Successful in 56s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m8s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 2m1s Details
Test / test_heal_csum_4k_dj (push) Successful in 4m45s Details
Test / test_scrub_pg_size_3 (push) Successful in 2m31s Details
Test / test_heal_csum_4k (push) Successful in 4m54s Details
Test / test_scrub_ec (push) Successful in 46s Details
- Do not use \r if output is not a terminal (should fix unexpected job output in proxmox)
- Fix rm/rm-data error return code, add --down-ok option to bypass the error
- Add EIO retry timeout and allow to disable these retries, rename up_wait_retry_interval to client_retry_interval
- Add ubuntu jammy build
- Wait for blockstore initialisation before starting OSD (prevent timeouts when init takes time)
- Fix a rare use-after-free in automatic sync after delete in blockstore
2024-02-29 09:58:34 +03:00
Vitaliy Filippov 6bf1f539a6 Add EIO retry timeout and allow to disable these retries, rename up_wait_retry_interval to client_retry_interval 2024-02-28 13:10:02 +03:00
Vitaliy Filippov 5e934264cf Release 1.4.7
- Fix another old "BUG: Attempt to overwrite used offset" in a very simple
  case: bs=4k rw=write iodepth=16 from OSD start; add this case to tests
- Fix a rare crash with "unexpected state during flush: 0x51" possible with
  EC since 1.4.2 during rebalance and OSD outages
- Fix a rare write stall with EC & immediate_commit=none caused by sync
  operations reserving unneeded space in the journal
- Fix 32-bit build warnings, most in printf/scanf format strings
2024-02-22 12:45:52 +03:00
Vitaliy Filippov f87964861d Release 1.4.6
Test / test_snapshot_ec (push) Successful in 29s Details
Test / test_rm (push) Successful in 18s Details
Test / test_move_reappear (push) Successful in 26s Details
Test / test_snapshot_down (push) Successful in 28s Details
Test / test_snapshot_down_ec (push) Successful in 32s Details
Test / test_splitbrain (push) Successful in 23s Details
Test / test_snapshot_chain (push) Successful in 2m3s Details
Test / test_snapshot_chain_ec (push) Successful in 2m46s Details
Test / test_rebalance_verify_imm (push) Successful in 3m1s Details
Test / test_rebalance_verify (push) Successful in 3m30s Details
Test / test_switch_primary (push) Successful in 38s Details
Test / test_write (push) Successful in 32s Details
Test / test_write_no_same (push) Successful in 17s Details
Test / test_write_xor (push) Successful in 38s Details
Test / test_rebalance_verify_ec (push) Successful in 4m38s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m57s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m14s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m21s Details
Test / test_heal_csum_32k (push) Successful in 5m45s Details
Test / test_heal_csum_4k_dmj (push) Successful in 5m27s Details
Test / test_scrub (push) Successful in 1m30s Details
Test / test_heal_csum_4k_dj (push) Successful in 5m26s Details
Test / test_scrub_zero_osd_2 (push) Successful in 38s Details
Test / test_scrub_xor (push) Successful in 40s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m8s Details
Test / test_scrub_ec (push) Successful in 1m5s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m49s Details
Test / test_heal_csum_4k (push) Successful in 5m41s Details
Test / test_heal_ec (push) Successful in 4m11s Details
Test / test_heal_pg_size_2 (push) Successful in 4m22s Details
Unwavering stabilization of 1.4.x, continued :-)

- Include the accidentally lost part of 1.4.5 journal trimming fix
- Fix a possible OSD crash with "BUG: Attempt to overwrite used offset"
  which was probably present for long time, but became apparent after
  fixing flapping tests in CI
- Fix remaining flapping tests in CI. It was the first time when tests
  actually passed without retries :-)
2024-02-20 17:01:26 +03:00
Vitaliy Filippov f882c7dd87 Release 1.4.5
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m23s Details
Test / test_rm (push) Successful in 15s Details
Test / test_move_reappear (push) Successful in 21s Details
Test / test_snapshot_down (push) Successful in 26s Details
Test / test_snapshot_down_ec (push) Successful in 30s Details
Test / test_splitbrain (push) Successful in 29s Details
Test / test_snapshot_chain (push) Successful in 2m17s Details
Test / test_snapshot_chain_ec (push) Successful in 3m14s Details
Test / test_rebalance_verify_imm (push) Successful in 3m24s Details
Test / test_rebalance_verify (push) Successful in 3m59s Details
Test / test_switch_primary (push) Successful in 35s Details
Test / test_write_xor (push) Successful in 32s Details
Test / test_write_no_same (push) Successful in 13s Details
Test / test_rebalance_verify_ec (push) Successful in 3m46s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m13s Details
Test / test_heal_pg_size_2 (push) Successful in 3m52s Details
Test / test_heal_ec (push) Successful in 5m25s Details
Test / test_heal_csum_32k_dj (push) Successful in 4m24s Details
Test / test_heal_csum_4k_dmj (push) Successful in 4m23s Details
Test / test_heal_csum_4k_dj (push) Successful in 4m17s Details
Test / test_scrub (push) Successful in 38s Details
Test / test_scrub_zero_osd_2 (push) Successful in 29s Details
Test / test_scrub_xor (push) Successful in 30s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 43s Details
Test / test_scrub_ec (push) Successful in 32s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m46s Details
Test / test_heal_csum_4k (push) Successful in 4m4s Details
Test / test_write (push) Successful in 1m38s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m5s Details
Test / test_heal_csum_32k (push) Successful in 4m15s Details
- Fix a write stall caused by incorrect journal trimming introduced in 1.4.4 :)
- Fix PGs sometimes hanging in "starting" state on mass OSD restarts
- Fix a rare crash with "map::at" during OSD pings
- Use new defaults for non-capacitor (desktop) SSDs - improves T1Q256 random write from ~6k iops to ~45k iops
- Make journal_trim_interval configurable
2024-02-16 10:13:33 +03:00
Vitaliy Filippov c777a0041a Release 1.4.4
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m23s Details
Test / test_move_reappear (push) Successful in 21s Details
Test / test_rm (push) Successful in 16s Details
Test / test_snapshot_down (push) Successful in 30s Details
Test / test_snapshot_down_ec (push) Successful in 30s Details
Test / test_splitbrain (push) Successful in 25s Details
Test / test_snapshot_chain (push) Successful in 2m18s Details
Test / test_snapshot_chain_ec (push) Successful in 3m13s Details
Test / test_rebalance_verify_imm (push) Successful in 3m8s Details
Test / test_rebalance_verify (push) Successful in 3m41s Details
Test / test_switch_primary (push) Successful in 36s Details
Test / test_write (push) Successful in 40s Details
Test / test_write_no_same (push) Successful in 16s Details
Test / test_write_xor (push) Successful in 39s Details
Test / test_rebalance_verify_ec (push) Successful in 4m56s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m21s Details
Test / test_heal_pg_size_2 (push) Successful in 4m15s Details
Test / test_heal_ec (push) Successful in 5m1s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m32s Details
Test / test_heal_csum_32k (push) Successful in 5m38s Details
Test / test_heal_csum_4k_dmj (push) Successful in 5m43s Details
Test / test_scrub (push) Successful in 1m31s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m17s Details
Test / test_heal_csum_4k_dj (push) Successful in 5m57s Details
Test / test_scrub_xor (push) Successful in 30s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m7s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 41s Details
Test / test_scrub_ec (push) Successful in 24s Details
Test / test_heal_csum_32k_dmj (push) Successful in 3m56s Details
Test / test_heal_csum_4k (push) Successful in 3m16s Details
A couple of fixes for EC pools

- Fix a segfault possible on partial EC overwrite in 1234 -> 5030 rebalance scenario
- Fix two problems leading to EC pools stalling on rebalance & parallel sudden stops
  of OSDs, for example during a sudden poweroff of a host:
  - Recovery auto-tuning (1.4.0 feature) could apply too large delays and stall
    the EC journal - fixed by limiting delays with a new recovery_tune_sleep_cutoff_us
    parameter (10 seconds by default) and applying recovery pauses before write
    operations, not after them, to not occupy space in the journal for long time
  - Dynamic journal space reservation (1.3.0 feature) wasn't accounting new writes
    when checking the limit so OSDs could still fill the journal fully and stall -
    fixed by including new writes into the limit
- Print etcd dbSize instead of dbSizeInUse in status
2024-02-11 16:23:08 +03:00
Vitaliy Filippov 27e9f244ec Release 1.4.3
Test / test_move_reappear (push) Successful in 22s Details
Test / test_rm (push) Successful in 15s Details
Test / test_snapshot_down (push) Successful in 36s Details
Test / test_snapshot_down_ec (push) Successful in 30s Details
Test / test_interrupted_rebalance (push) Successful in 5m3s Details
Test / test_splitbrain (push) Successful in 20s Details
Test / test_snapshot_chain (push) Successful in 3m1s Details
Test / test_snapshot_chain_ec (push) Successful in 3m13s Details
Test / test_rebalance_verify_imm (push) Successful in 3m0s Details
Test / test_rebalance_verify (push) Successful in 3m29s Details
Test / test_switch_primary (push) Successful in 37s Details
Test / test_write (push) Successful in 44s Details
Test / test_write_xor (push) Successful in 39s Details
Test / test_write_no_same (push) Successful in 16s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m13s Details
Test / test_rebalance_verify_ec (push) Successful in 5m31s Details
Test / test_heal_ec (push) Successful in 4m54s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m25s Details
Test / test_heal_csum_32k (push) Successful in 6m8s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m17s Details
Test / test_scrub (push) Successful in 1m8s Details
Test / test_scrub_zero_osd_2 (push) Successful in 55s Details
Test / test_scrub_xor (push) Successful in 45s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m22s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m11s Details
Test / test_scrub_ec (push) Successful in 46s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m39s Details
Test / test_heal_csum_4k (push) Successful in 6m8s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m15s Details
Test / test_heal_pg_size_2 (push) Successful in 4m41s Details
Hotfix for hotfix O:-)

- "Write stall fix" was incomplete and EC write stalls could
  continue even on 1.4.2. Now they're finally fixed O:-)
- Make monitor ignore statistics of stopped OSDs. Previously if you stopped all
  OSDs the last total I/O numbers would remain the same indefinitely
2024-02-09 00:29:31 +03:00
Vitaliy Filippov 8e25a28a08 Ignore down OSDs in monitor statistics aggregation
Test / test_move_reappear (push) Successful in 20s Details
Test / test_interrupted_rebalance_ec (push) Successful in 2m50s Details
Test / test_snapshot_down (push) Successful in 22s Details
Test / test_snapshot_down_ec (push) Successful in 25s Details
Test / test_splitbrain (push) Successful in 18s Details
Test / test_snapshot_chain (push) Successful in 2m10s Details
Test / test_snapshot_chain_ec (push) Failing after 3m8s Details
Test / test_rebalance_verify (push) Successful in 3m6s Details
Test / test_interrupted_rebalance (push) Failing after 10m52s Details
Test / test_rebalance_verify_imm (push) Successful in 5m28s Details
Test / test_switch_primary (push) Successful in 37s Details
Test / test_write (push) Successful in 42s Details
Test / test_write_xor (push) Successful in 38s Details
Test / test_write_no_same (push) Successful in 16s Details
Test / test_rebalance_verify_ec (push) Successful in 6m7s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 6m3s Details
Test / test_heal_pg_size_2 (push) Successful in 4m12s Details
Test / test_heal_ec (push) Successful in 5m20s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m53s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m23s Details
Test / test_heal_csum_32k (push) Successful in 5m59s Details
Test / test_scrub_zero_osd_2 (push) Has been cancelled Details
Test / test_scrub_xor (push) Has been cancelled Details
Test / test_heal_csum_4k_dmj (push) Has been cancelled Details
Test / test_scrub_pg_size_3 (push) Has been cancelled Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Has been cancelled Details
Test / test_scrub_ec (push) Has been cancelled Details
Test / test_scrub (push) Has been cancelled Details
Test / test_heal_csum_4k_dj (push) Has been cancelled Details
Test / test_heal_csum_4k (push) Has been cancelled Details
2024-02-09 00:22:36 +03:00
Vitaliy Filippov 016115c0d4 Release 1.4.2
Test / test_rm (push) Successful in 16s Details
Test / test_move_reappear (push) Successful in 20s Details
Test / test_snapshot_down (push) Successful in 25s Details
Test / test_snapshot_down_ec (push) Successful in 39s Details
Test / test_interrupted_rebalance (push) Successful in 4m52s Details
Test / test_splitbrain (push) Successful in 20s Details
Test / test_snapshot_chain (push) Successful in 3m11s Details
Test / test_rebalance_verify_imm (push) Successful in 3m16s Details
Test / test_rebalance_verify (push) Successful in 3m45s Details
Test / test_switch_primary (push) Successful in 36s Details
Test / test_write (push) Successful in 40s Details
Test / test_write_xor (push) Successful in 40s Details
Test / test_write_no_same (push) Successful in 17s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m8s Details
Test / test_rebalance_verify_ec (push) Successful in 5m57s Details
Test / test_heal_pg_size_2 (push) Successful in 4m22s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m20s Details
Test / test_heal_ec (push) Successful in 5m54s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m24s Details
Test / test_heal_csum_32k (push) Successful in 6m3s Details
Test / test_heal_csum_4k_dmj (push) Successful in 5m54s Details
Test / test_scrub_zero_osd_2 (push) Successful in 53s Details
Test / test_scrub (push) Successful in 55s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m14s Details
Test / test_scrub_xor (push) Successful in 1m1s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m50s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 57s Details
Test / test_scrub_ec (push) Successful in 52s Details
Test / test_heal_csum_4k (push) Successful in 5m47s Details
Test / test_snapshot_chain_ec (push) Successful in 1m24s Details
- Log to systemd by default
- Fix excessive autosyncs after every operation with disabled immediate_commit (introduced in 1.1.0)
- Fix a possible write stall with EC due to the lack of OSD wakeup after stabilizing previous writes
- Change sync operation semantics as a final fix to possible write stalls with EC and disabled immediate_commit
- Sync after deleting data in CLI rm / rm-data if immediate_commit is disabled
- Fix OSDs ignoring syncs & autosyncs for delete operations
- Fix OSD space reporting sometimes adding garbage zeros for deleted inodes (causing extra pool/stats etcd keys for deleted pools)
- Speed up monitor failover - change default etcd_mon_ttl from 30 to 5 seconds
- Speed up operation retries - change default up_wait_retry_interval to 50 ms
- Add patch for libvirt 9.10
2024-02-04 02:23:49 +03:00
Vitaliy Filippov e026de95d5 Log to systemd by default
Test / test_move_reappear (push) Successful in 20s Details
Test / test_etcd_fail (push) Successful in 5m19s Details
Test / test_snapshot_chain (push) Successful in 1m26s Details
Test / test_snapshot_down (push) Successful in 26s Details
Test / test_snapshot_down_ec (push) Successful in 28s Details
Test / test_splitbrain (push) Successful in 19s Details
Test / test_snapshot_chain_ec (push) Failing after 3m8s Details
Test / test_interrupted_rebalance (push) Successful in 7m44s Details
Test / test_rebalance_verify_imm (push) Successful in 3m11s Details
Test / test_switch_primary (push) Successful in 34s Details
Test / test_write (push) Successful in 34s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 2m41s Details
Test / test_rebalance_verify_ec (push) Successful in 3m18s Details
Test / test_write_no_same (push) Successful in 22s Details
Test / test_write_xor (push) Successful in 1m41s Details
Test / test_heal_pg_size_2 (push) Failing after 3m54s Details
Test / test_rebalance_verify (push) Successful in 9m38s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m4s Details
Test / test_heal_csum_32k_dj (push) Successful in 4m23s Details
Test / test_heal_csum_32k (push) Successful in 5m24s Details
Test / test_heal_ec (push) Failing after 10m18s Details
Test / test_heal_csum_4k_dmj (push) Successful in 5m31s Details
Test / test_scrub (push) Successful in 1m18s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m0s Details
Test / test_scrub_xor (push) Successful in 51s Details
Test / test_heal_csum_4k_dj (push) Successful in 5m10s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 48s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m59s Details
Test / test_scrub_ec (push) Successful in 48s Details
Test / test_heal_csum_4k (push) Successful in 4m39s Details
2024-02-04 01:21:31 +03:00
Vitaliy Filippov d2b43cb118 Change default etcd_mon_ttl
Test / test_move_reappear (push) Successful in 35s Details
Test / test_interrupted_rebalance_ec (push) Successful in 3m29s Details
Test / test_interrupted_rebalance (push) Successful in 4m47s Details
Test / test_snapshot_down (push) Successful in 29s Details
Test / test_snapshot_down_ec (push) Successful in 25s Details
Test / test_splitbrain (push) Successful in 24s Details
Test / test_snapshot_chain (push) Successful in 2m46s Details
Test / test_snapshot_chain_ec (push) Failing after 3m10s Details
Test / test_rebalance_verify_imm (push) Successful in 4m24s Details
Test / test_rebalance_verify (push) Successful in 4m54s Details
Test / test_switch_primary (push) Successful in 35s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m38s Details
Test / test_write (push) Successful in 46s Details
Test / test_write_xor (push) Successful in 49s Details
Test / test_write_no_same (push) Successful in 18s Details
Test / test_rebalance_verify_ec (push) Successful in 7m14s Details
Test / test_heal_pg_size_2 (push) Successful in 4m10s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m10s Details
Test / test_heal_csum_32k_dj (push) Successful in 4m52s Details
Test / test_heal_csum_32k (push) Successful in 5m20s Details
Test / test_heal_csum_4k_dmj (push) Successful in 5m8s Details
Test / test_heal_ec (push) Failing after 10m21s Details
Test / test_scrub (push) Successful in 1m2s Details
Test / test_scrub_xor (push) Successful in 54s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m4s Details
Test / test_heal_csum_4k_dj (push) Successful in 4m48s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m26s Details
Test / test_scrub_ec (push) Successful in 50s Details
Test / test_scrub_pg_size_3 (push) Failing after 2m5s Details
Test / test_heal_csum_4k (push) Successful in 4m33s Details
2024-01-29 23:45:19 +03:00
Vitaliy Filippov 1c322b33ed Change default up_wait_retry_interval to 50 ms
Test / test_rm (push) Successful in 14s Details
Test / test_interrupted_rebalance_ec (push) Successful in 3m59s Details
Test / test_snapshot_chain (push) Successful in 1m34s Details
Test / test_snapshot_down (push) Successful in 25s Details
Test / test_snapshot_down_ec (push) Successful in 29s Details
Test / test_splitbrain (push) Successful in 19s Details
Test / test_snapshot_chain_ec (push) Successful in 2m35s Details
Test / test_interrupted_rebalance (push) Successful in 8m15s Details
Test / test_rebalance_verify_imm (push) Successful in 3m54s Details
Test / test_switch_primary (push) Successful in 36s Details
Test / test_write (push) Successful in 35s Details
Test / test_rebalance_verify_ec (push) Successful in 4m48s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 2m51s Details
Test / test_write_no_same (push) Successful in 14s Details
Test / test_write_xor (push) Failing after 3m9s Details
Test / test_heal_pg_size_2 (push) Successful in 3m55s Details
Test / test_heal_ec (push) Successful in 3m50s Details
Test / test_rebalance_verify (push) Failing after 9m30s Details
Test / test_heal_csum_32k_dmj (push) Failing after 5m40s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m12s Details
Test / test_heal_csum_32k (push) Successful in 6m25s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m56s Details
Test / test_scrub (push) Successful in 1m4s Details
Test / test_scrub_zero_osd_2 (push) Successful in 55s Details
Test / test_scrub_xor (push) Successful in 56s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m19s Details
Test / test_scrub_pg_size_3 (push) Failing after 2m14s Details
Test / test_heal_csum_4k_dj (push) Successful in 5m53s Details
Test / test_scrub_ec (push) Successful in 1m1s Details
Test / test_heal_csum_4k (push) Successful in 5m17s Details
2024-01-26 01:51:08 +03:00
Vitaliy Filippov ba55f91409 Release 1.4.1
Test / test_move_reappear (push) Successful in 22s Details
Test / test_snapshot_chain (push) Successful in 1m27s Details
Test / test_interrupted_rebalance_ec (push) Successful in 4m41s Details
Test / test_snapshot_down (push) Successful in 25s Details
Test / test_snapshot_chain_ec (push) Successful in 2m0s Details
Test / test_splitbrain (push) Successful in 18s Details
Test / test_snapshot_down_ec (push) Successful in 25s Details
Test / test_rebalance_verify_ec (push) Failing after 2m21s Details
Test / test_rebalance_verify_imm (push) Successful in 2m30s Details
Test / test_switch_primary (push) Successful in 39s Details
Test / test_write (push) Successful in 35s Details
Test / test_interrupted_rebalance (push) Failing after 10m8s Details
Test / test_write_xor (push) Successful in 36s Details
Test / test_write_no_same (push) Successful in 17s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m4s Details
Test / test_heal_pg_size_2 (push) Successful in 3m55s Details
Test / test_rebalance_verify (push) Successful in 8m31s Details
Test / test_heal_ec (push) Successful in 5m9s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m27s Details
Test / test_heal_csum_32k (push) Successful in 5m42s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m1s Details
Test / test_scrub (push) Successful in 59s Details
Test / test_scrub_zero_osd_2 (push) Successful in 38s Details
Test / test_heal_csum_4k_dmj (push) Successful in 7m5s Details
Test / test_scrub_xor (push) Successful in 58s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m25s Details
Test / test_scrub_ec (push) Failing after 42s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m32s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m38s Details
Test / test_heal_csum_4k (push) Successful in 5m38s Details
- Fix a monitor crash on primary OSD switching introduced in 1.4.0
- Fix "partly outside array bounds" warnings for GCC 12 in cpp-btree
- Fix a realloc memory leak in theory possible with too large listings (OSD_OP_LIST)
2024-01-18 02:31:42 +03:00
Vitaliy Filippov 2aa5aa7ab6 Add a test for simple master switching without PG reconfiguration
Test / test_move_reappear (push) Successful in 20s Details
Test / test_snapshot_chain (push) Successful in 1m27s Details
Test / test_snapshot_down (push) Successful in 23s Details
Test / test_snapshot_chain_ec (push) Successful in 1m56s Details
Test / test_snapshot_down_ec (push) Successful in 23s Details
Test / test_splitbrain (push) Successful in 17s Details
Test / test_interrupted_rebalance_ec (push) Successful in 6m40s Details
Test / test_interrupted_rebalance (push) Successful in 8m12s Details
Test / test_rebalance_verify_imm (push) Successful in 3m12s Details
Test / test_switch_primary (push) Successful in 34s Details
Test / test_write (push) Successful in 46s Details
Test / test_rebalance_verify_ec (push) Successful in 3m18s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 2m42s Details
Test / test_write_no_same (push) Successful in 15s Details
Test / test_rebalance_verify (push) Successful in 6m36s Details
Test / test_heal_ec (push) Successful in 5m2s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m33s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m58s Details
Test / test_heal_csum_32k (push) Successful in 6m6s Details
Test / test_scrub (push) Successful in 47s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m17s Details
Test / test_scrub_zero_osd_2 (push) Successful in 43s Details
Test / test_scrub_xor (push) Successful in 47s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m44s Details
Test / test_scrub_ec (push) Successful in 41s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m18s Details
Test / test_scrub_pg_size_3 (push) Successful in 2m11s Details
Test / test_heal_csum_4k (push) Successful in 6m12s Details
Test / test_heal_pg_size_2 (push) Successful in 3m16s Details
Test / test_write_xor (push) Successful in 34s Details
Also use osd_out_time:1 only in select tests and restart mon in tests only on connection errors
2024-01-17 00:19:01 +03:00
Vitaliy Filippov 3ca3b8a8d8 Fix recheck_pgs bug introduced in 1.4.0
Test / test_rm (push) Successful in 14s Details
Test / test_interrupted_rebalance_ec (push) Successful in 3m27s Details
Test / test_snapshot_chain (push) Successful in 1m24s Details
Test / test_snapshot_down (push) Successful in 25s Details
Test / test_snapshot_chain_ec (push) Successful in 1m54s Details
Test / test_snapshot_down_ec (push) Successful in 20s Details
Test / test_splitbrain (push) Successful in 15s Details
Test / test_rebalance_verify_imm (push) Successful in 2m42s Details
Test / test_etcd_fail (push) Failing after 10m8s Details
Test / test_interrupted_rebalance (push) Failing after 10m9s Details
Test / test_write (push) Successful in 1m22s Details
Test / test_rebalance_verify_ec (push) Failing after 1m51s Details
Test / test_write_no_same (push) Successful in 16s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m27s Details
Test / test_write_xor (push) Failing after 3m13s Details
Test / test_heal_pg_size_2 (push) Successful in 3m22s Details
Test / test_rebalance_verify (push) Failing after 10m9s Details
Test / test_heal_ec (push) Successful in 4m41s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m42s Details
Test / test_heal_csum_32k_dj (push) Successful in 4m58s Details
Test / test_heal_csum_32k (push) Successful in 6m34s Details
Test / test_scrub (push) Successful in 54s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m56s Details
Test / test_scrub_zero_osd_2 (push) Successful in 49s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m1s Details
Test / test_scrub_ec (push) Has been cancelled Details
Test / test_heal_csum_4k (push) Has been cancelled Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Has been cancelled Details
Test / test_scrub_xor (push) Has been cancelled Details
Test / test_scrub_pg_size_3 (push) Has been cancelled Details
2024-01-16 23:49:21 +03:00
Vitaliy Filippov 5280d1d561 Release 1.4.0
Test / test_snapshot (push) Successful in 26s Details
Test / test_snapshot_ec (push) Successful in 26s Details
Test / test_rm (push) Successful in 16s Details
Test / test_move_reappear (push) Successful in 24s Details
Test / test_snapshot_down (push) Successful in 26s Details
Test / test_snapshot_down_ec (push) Successful in 30s Details
Test / test_splitbrain (push) Successful in 28s Details
Test / test_snapshot_chain (push) Successful in 2m41s Details
Test / test_rebalance_verify_imm (push) Successful in 2m48s Details
Test / test_rebalance_verify (push) Successful in 3m28s Details
Test / test_write (push) Successful in 47s Details
Test / test_write_no_same (push) Successful in 14s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m5s Details
Test / test_rebalance_verify_ec (push) Successful in 3m41s Details
Test / test_heal_pg_size_2 (push) Successful in 3m45s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m52s Details
Test / test_heal_ec (push) Successful in 5m11s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m42s Details
Test / test_heal_csum_32k (push) Successful in 5m56s Details
Test / test_scrub (push) Successful in 1m25s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m18s Details
Test / test_scrub_xor (push) Successful in 42s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m49s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m32s Details
Test / test_heal_csum_4k (push) Successful in 5m31s Details
Test / test_scrub_ec (push) Successful in 50s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m2s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m5s Details
Test / test_snapshot_chain_ec (push) Successful in 1m21s Details
Test / test_write_xor (push) Successful in 36s Details
New features:
- Intelligent recovery/rebalance speed auto-tuning to reduce its impact on clients (see README -> Features)
- Auto-restoration of dead VDUSE daemons in CSI plugin
- Add vitastor-disk update-sb command
- Update QEMU for Debian Bookworm to 8.1 and use it for CSI plugin

Bug fixes:
- Fix pools SOMETIMES staying inactive after stopping a node due to OSDs not reacting
  to PG state changes caused by incorrect full reload of state from etcd on reconnection
- Make monitors retry pool configuration changes quickier which fixes them being unable
  to apply changes when an ongoing rebalance is quickly making a lot of PGs clean
- Fix CSI plugin not accepting array of strings as etcd address in /etc/vitastor/vitastor.conf
- Allow multiple interfaces with the same IP address, for "simple routed" full mesh network
- Do not ignore loopback addresses for OSD network (to make ECMP setups with frr possible)
- Fix a rare client crash during OSD reconnections
- Only treat data partitions as existing OSDs in vitastor-disk prepare
- Remove etcd parameter from default command examples
- Fix reported free space sometimes changing non-immediately after deletion of data from OSDs
- Fix a possible OSD crash on print_slow when bs_op is NULL
- Use the same etcd_ws_keepalive_interval in mon as in OSD
- Fix mon not using values from config when /config/global is not present
- Remove pve-storage-portal-dns-list format for vitastor_etcd_address
- Parse log_level in cluster_client
- Fix vitastor-nbd image existence check not working because of non-zeroed inode_watch fields
- Do not warn on EPIPE in client unless log_level is raised explicitly
- Fix incorrect error in CSI when searching for the device in /sys
- Remove 2 last prints to stdout in etcd_state_client
- Fix a possible OSD crash when checking corrupted journal entries
2024-01-12 01:28:33 +03:00
Vitaliy Filippov 99ee8596ea Rename min/max_util to util_low/high 2023-12-31 01:23:17 +03:00
Vitaliy Filippov f757a35a8d Retry PG changes without re-running lpsolve when pool configuration and OSD tree don't change
OSDs often change their /pg/history keys during rebalance, so monitor receives additional
transaction failures from etcd if it re-runs lpsolve which sometimes may even lead to monitor
being unable to apply PG changes at all until rebalance completes
2023-12-31 01:23:17 +03:00
Vitaliy Filippov 1edf86ed26 Aggregate recovery delay using simple mean over last 10 observations (EWMA is shit) 2023-12-31 01:23:17 +03:00
Vitaliy Filippov 751935ddd8 WIP Auto-tune recovery speed 2023-12-31 01:23:17 +03:00
Vitaliy Filippov 1299373988 Use the same etcd_ws_keepalive_interval in OSD and mon
Test / test_snapshot_ec (push) Successful in 33s Details
Test / test_interrupted_rebalance_ec (push) Successful in 1m58s Details
Test / test_move_reappear (push) Successful in 22s Details
Test / test_rm (push) Successful in 16s Details
Test / test_snapshot_down (push) Successful in 32s Details
Test / test_snapshot_down_ec (push) Successful in 32s Details
Test / test_splitbrain (push) Successful in 25s Details
Test / test_snapshot_chain (push) Successful in 2m36s Details
Test / test_snapshot_chain_ec (push) Failing after 3m8s Details
Test / test_rebalance_verify_imm (push) Successful in 2m58s Details
Test / test_rebalance_verify (push) Successful in 3m55s Details
Test / test_write (push) Successful in 39s Details
Test / test_write_no_same (push) Successful in 15s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m18s Details
Test / test_rebalance_verify_ec (push) Successful in 4m8s Details
Test / test_write_xor (push) Failing after 3m11s Details
Test / test_heal_pg_size_2 (push) Successful in 3m47s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m58s Details
Test / test_heal_ec (push) Successful in 6m21s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m11s Details
Test / test_heal_csum_32k (push) Successful in 6m22s Details
Test / test_scrub (push) Successful in 1m17s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m17s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m35s Details
Test / test_scrub_xor (push) Successful in 57s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m27s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m3s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m33s Details
Test / test_scrub_ec (push) Successful in 44s Details
Test / test_heal_csum_4k (push) Successful in 6m9s Details
2023-12-23 20:07:29 +03:00
Vitaliy Filippov 4ece4dfdd0 Fix mon not using values from config when /config/global is not present
Test / test_snapshot_ec (push) Successful in 27s Details
Test / test_rm (push) Successful in 17s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m24s Details
Test / test_move_reappear (push) Successful in 22s Details
Test / test_snapshot_down (push) Successful in 32s Details
Test / test_snapshot_down_ec (push) Successful in 33s Details
Test / test_splitbrain (push) Successful in 26s Details
Test / test_snapshot_chain (push) Successful in 2m13s Details
Test / test_snapshot_chain_ec (push) Successful in 3m0s Details
Test / test_rebalance_verify_imm (push) Successful in 2m57s Details
Test / test_rebalance_verify (push) Successful in 3m47s Details
Test / test_write (push) Successful in 44s Details
Test / test_write_no_same (push) Successful in 14s Details
Test / test_write_xor (push) Successful in 56s Details
Test / test_rebalance_verify_ec (push) Successful in 5m4s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m29s Details
Test / test_heal_pg_size_2 (push) Successful in 3m42s Details
Test / test_heal_ec (push) Successful in 5m1s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m10s Details
Test / test_heal_csum_32k_dmj (push) Successful in 6m57s Details
Test / test_heal_csum_32k (push) Successful in 6m21s Details
Test / test_scrub (push) Successful in 1m16s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m8s Details
Test / test_scrub_xor (push) Successful in 1m13s Details
Test / test_heal_csum_4k_dmj (push) Successful in 8m15s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m1s Details
Test / test_heal_csum_4k (push) Successful in 5m47s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m14s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m24s Details
Test / test_scrub_ec (push) Successful in 27s Details
2023-12-22 02:25:09 +03:00
Vitaliy Filippov a1c7cc3d8d Release 1.3.1
Test / test_interrupted_rebalance_ec (push) Successful in 1m46s Details
Test / test_move_reappear (push) Successful in 21s Details
Test / test_rm (push) Successful in 15s Details
Test / test_snapshot_ec (push) Successful in 35s Details
Test / test_snapshot_down (push) Successful in 30s Details
Test / test_snapshot_down_ec (push) Successful in 31s Details
Test / test_splitbrain (push) Successful in 23s Details
Test / test_snapshot_chain (push) Successful in 2m22s Details
Test / test_snapshot_chain_ec (push) Successful in 2m59s Details
Test / test_rebalance_verify_imm (push) Successful in 3m3s Details
Test / test_rebalance_verify (push) Successful in 3m47s Details
Test / test_write (push) Successful in 44s Details
Test / test_write_no_same (push) Successful in 13s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m36s Details
Test / test_rebalance_verify_ec (push) Successful in 4m20s Details
Test / test_heal_pg_size_2 (push) Successful in 3m43s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m45s Details
Test / test_heal_ec (push) Successful in 6m22s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m51s Details
Test / test_heal_csum_32k (push) Successful in 6m2s Details
Test / test_scrub (push) Successful in 1m14s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m19s Details
Test / test_heal_csum_4k_dmj (push) Successful in 5m54s Details
Test / test_scrub_xor (push) Successful in 1m1s Details
Test / test_heal_csum_4k_dj (push) Successful in 5m59s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m54s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m2s Details
Test / test_scrub_ec (push) Successful in 34s Details
Test / test_heal_csum_4k (push) Successful in 6m0s Details
Test / test_write_xor (push) Successful in 32s Details
Hotfix to 1.3.0 - new "journal space reservation" had a bug which
caused OSDs to crash with EC and without immediate_commit.
2023-12-04 18:35:09 +03:00
Vitaliy Filippov 7972502eaf Release 1.3.0
Test / test_rm (push) Successful in 12s Details
Test / test_snapshot_chain (push) Successful in 1m1s Details
Test / test_snapshot_down (push) Successful in 19s Details
Test / test_splitbrain (push) Successful in 12s Details
Test / test_snapshot_down_ec (push) Failing after 3m10s Details
Test / test_rebalance_verify (push) Successful in 2m45s Details
Test / test_rebalance_verify_imm (push) Successful in 2m17s Details
Test / test_write (push) Successful in 1m11s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 2m41s Details
Test / test_write_no_same (push) Successful in 12s Details
Test / test_write_xor (push) Failing after 3m6s Details
Test / test_rebalance_verify_ec (push) Failing after 5m27s Details
Test / test_heal_pg_size_2 (push) Failing after 3m7s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m36s Details
Test / test_heal_csum_32k_dj (push) Failing after 4m53s Details
Test / test_heal_csum_32k (push) Failing after 5m27s Details
Test / test_heal_ec (push) Failing after 10m15s Details
Test / test_heal_csum_4k_dmj (push) Successful in 5m14s Details
Test / test_scrub (push) Successful in 1m11s Details
Test / test_heal_csum_4k_dj (push) Successful in 5m15s Details
Test / test_scrub_zero_osd_2 (push) Successful in 56s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m4s Details
Test / test_heal_csum_4k (push) Failing after 5m31s Details
Test / test_scrub_xor (push) Failing after 3m17s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Failing after 3m6s Details
Test / test_change_pg_count_ec (push) Failing after 3m5s Details
Test / test_snapshot_ec (push) Failing after 3m5s Details
Test / test_scrub_ec (push) Failing after 3m5s Details
Test / test_snapshot_chain_ec (push) Failing after 3m5s Details
Test / test_interrupted_rebalance_ec (push) Failing after 10m5s Details
New features:
- RDMA without ODP - much faster and all cards are now supported, not just Mellanox
- VDUSE in CSI - faster, more stable and can even recover after CSI pod restart!
- Reserve journal space for stabilize requests dynamically to prevent stalls under load with EC
- Raise default NBD timeout from 30 to 300 seconds and allow to take it from /etc/vitastor/vitastor.conf
- Remove explicit etcdUrl/etcdPrefix K8S storage class parameter support to prevent
  etcd migration issues for volumes created with these parameters
- Support QEMU 8.1 and pve-qemu 8.1

Bug fixes:
- Fix RDMA connection (and thus memory) leak
- Fix rare crashes under load due to incorrect io_uring queue size tracking
- Fix monitor statistics aggregation in case of empty /osd/stats keys
- Fix crash on unknown long argument to vitastor-disk
- Allow trailing comma in JSONs again
- Fix crash on attempts to dump a long listing of objects "to stabilize" or "to rollback" in a slow op
2023-12-04 02:36:43 +03:00
Vitaliy Filippov 7da4868b37 Fix monitor statistics aggregation in case of empty /osd/stats keys
Test / test_move_reappear (push) Successful in 18s Details
Test / test_rm (push) Successful in 13s Details
Test / test_snapshot_chain (push) Successful in 1m1s Details
Test / test_snapshot_down (push) Successful in 20s Details
Test / test_snapshot_ec (push) Failing after 3m6s Details
Test / test_splitbrain (push) Successful in 12s Details
Test / test_snapshot_chain_ec (push) Failing after 3m6s Details
Test / test_snapshot_down_ec (push) Failing after 3m7s Details
Test / test_rebalance_verify_ec (push) Failing after 48s Details
Test / test_rebalance_verify (push) Successful in 3m30s Details
Test / test_rebalance_verify_imm (push) Successful in 3m3s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 2m36s Details
Test / test_write_no_same (push) Successful in 12s Details
Test / test_interrupted_rebalance_ec (push) Failing after 10m7s Details
Test / test_write (push) Failing after 3m7s Details
Test / test_write_xor (push) Failing after 3m6s Details
Test / test_heal_csum_32k_dmj (push) Failing after 4m5s Details
Test / test_heal_csum_32k (push) Failing after 3m16s Details
Test / test_heal_pg_size_2 (push) Failing after 10m7s Details
Test / test_heal_ec (push) Failing after 10m10s Details
Test / test_heal_csum_32k_dj (push) Failing after 10m14s Details
Test / test_scrub (push) Successful in 59s Details
Test / test_scrub_zero_osd_2 (push) Successful in 54s Details
Test / test_heal_csum_4k_dj (push) Successful in 4m27s Details
Test / test_scrub_pg_size_3 (push) Successful in 41s Details
Test / test_heal_csum_4k (push) Failing after 4m56s Details
Test / test_scrub_xor (push) Failing after 3m11s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Failing after 3m5s Details
Test / test_scrub_ec (push) Failing after 3m5s Details
Test / test_heal_csum_4k_dmj (push) Failing after 10m8s Details
2023-11-24 01:05:21 +03:00
Vitaliy Filippov 5524dbdab7 Release 1.2.0
Test / test_snapshot_ec (push) Successful in 25s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m18s Details
Test / test_rm (push) Successful in 15s Details
Test / test_snapshot_down (push) Successful in 22s Details
Test / test_snapshot_down_ec (push) Successful in 23s Details
Test / test_splitbrain (push) Successful in 18s Details
Test / test_snapshot_chain (push) Successful in 2m13s Details
Test / test_snapshot_chain_ec (push) Successful in 2m57s Details
Test / test_rebalance_verify_imm (push) Successful in 2m51s Details
Test / test_write (push) Successful in 38s Details
Test / test_rebalance_verify (push) Successful in 3m39s Details
Test / test_write_no_same (push) Successful in 12s Details
Test / test_rebalance_verify_ec (push) Successful in 3m56s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m6s Details
Test / test_heal_pg_size_2 (push) Successful in 3m43s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m35s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m44s Details
Test / test_heal_csum_32k (push) Successful in 5m50s Details
Test / test_heal_csum_4k_dmj (push) Successful in 5m44s Details
Test / test_scrub_zero_osd_2 (push) Successful in 57s Details
Test / test_scrub (push) Successful in 1m0s Details
Test / test_scrub_xor (push) Successful in 1m5s Details
Test / test_heal_csum_4k_dj (push) Successful in 5m9s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m38s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 54s Details
Test / test_scrub_ec (push) Successful in 52s Details
Test / test_heal_csum_4k (push) Successful in 5m8s Details
Test / test_heal_ec (push) Successful in 3m17s Details
Test / test_write_xor (push) Successful in 35s Details
Test / test_move_reappear (push) Failing after 48s Details
New features:

- Implement CSI volume expansion
- Implement CSI volume snapshots
- CSI driver now requires Kubernetes >= 1.20

Bug fixes:

- Important bug fix for EC: fix EC n+k, k>=2 read recovery in ISA-L version returning
  incorrect data when reading at least the second chunk out of multiple missing chunks
  without reading the first one. All users of EC n+k, k>=2 should upgrade as soon as
  possible, and upgrade should be conducted with downtime: first stop all clients
  (VMs/containers), then all OSDs, then upgrade and restart everything.
- Fix unstable statistics aggregation in monitor (affecting vitastor-cli status and df)
- Make udev not wait for OSDs to start during boot
- Do not report negative numbers of offline PGs in vitastor-cli status when changing PG count
- Report both old and new PG counts in vitastor-cli df when changing it
- Fix OSDs sometimes not starting with "The code only supports journal versions 1 and 2,
  but it is 2 on disk" error after upgrading from pre-1.0 versions and letting OSDs run
  for some time
- Fix monitors sometimes returning old PG count back after OSD configuration changes
- Make monitor PG changes more stable and timeout errors less probable
2023-11-05 01:48:57 +03:00
Vitaliy Filippov 0e888e6c60 Prevent spamming etcd with last_clean_pgs update requests
Test / test_snapshot_ec (push) Successful in 34s Details
Test / test_interrupted_rebalance_ec (push) Successful in 1m49s Details
Test / test_move_reappear (push) Successful in 19s Details
Test / test_rm (push) Successful in 14s Details
Test / test_snapshot_down (push) Successful in 30s Details
Test / test_snapshot_down_ec (push) Successful in 31s Details
Test / test_splitbrain (push) Successful in 24s Details
Test / test_snapshot_chain (push) Successful in 2m23s Details
Test / test_snapshot_chain_ec (push) Successful in 3m4s Details
Test / test_rebalance_verify_imm (push) Successful in 2m54s Details
Test / test_rebalance_verify (push) Successful in 3m48s Details
Test / test_write (push) Successful in 35s Details
Test / test_write_no_same (push) Successful in 14s Details
Test / test_write_xor (push) Successful in 55s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m15s Details
Test / test_rebalance_verify_ec (push) Successful in 5m3s Details
Test / test_heal_pg_size_2 (push) Successful in 3m59s Details
Test / test_heal_ec (push) Successful in 4m56s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m48s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m44s Details
Test / test_heal_csum_32k (push) Successful in 6m35s Details
Test / test_scrub (push) Successful in 1m14s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m54s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m2s Details
Test / test_scrub_xor (push) Successful in 49s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m9s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m54s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m17s Details
Test / test_heal_csum_4k (push) Successful in 6m18s Details
Test / test_scrub_ec (push) Successful in 37s Details
2023-11-05 00:12:00 +03:00
Vitaliy Filippov 408c21d8f0 Scale last_clean_pgs PG count even if current PGs already contain the new number of PGs
Test / test_interrupted_rebalance_ec (push) Successful in 1m49s Details
Test / test_snapshot_ec (push) Successful in 31s Details
Test / test_rm (push) Successful in 13s Details
Test / test_snapshot_down (push) Successful in 23s Details
Test / test_move_reappear (push) Failing after 50s Details
Test / test_snapshot_down_ec (push) Successful in 22s Details
Test / test_splitbrain (push) Successful in 20s Details
Test / test_snapshot_chain (push) Successful in 2m15s Details
Test / test_snapshot_chain_ec (push) Successful in 2m56s Details
Test / test_rebalance_verify_imm (push) Successful in 2m59s Details
Test / test_write (push) Successful in 34s Details
Test / test_rebalance_verify (push) Successful in 3m44s Details
Test / test_write_no_same (push) Successful in 13s Details
Test / test_write_xor (push) Successful in 52s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m5s Details
Test / test_rebalance_verify_ec (push) Successful in 5m1s Details
Test / test_heal_pg_size_2 (push) Successful in 4m1s Details
Test / test_heal_ec (push) Successful in 5m3s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m13s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m37s Details
Test / test_heal_csum_32k (push) Successful in 6m19s Details
Test / test_scrub (push) Successful in 1m11s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m13s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m5s Details
Test / test_scrub_xor (push) Successful in 48s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m11s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m10s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m29s Details
Test / test_heal_csum_4k (push) Successful in 6m9s Details
Test / test_scrub_ec (push) Successful in 35s Details
2023-11-04 23:45:59 +03:00
Vitaliy Filippov 43cb9ae212 Prevent multiple parallel recheck_pgs in case of timeouts
Test / test_snapshot_ec (push) Successful in 37s Details
Test / test_minsize_1 (push) Successful in 13s Details
Test / test_rm (push) Successful in 12s Details
Test / test_move_reappear (push) Successful in 17s Details
Test / test_snapshot_down (push) Successful in 30s Details
Test / test_snapshot_down_ec (push) Successful in 31s Details
Test / test_splitbrain (push) Successful in 22s Details
Test / test_snapshot_chain (push) Successful in 2m25s Details
Test / test_snapshot_chain_ec (push) Failing after 3m7s Details
Test / test_rebalance_verify_imm (push) Successful in 3m0s Details
Test / test_rebalance_verify (push) Successful in 3m54s Details
Test / test_write (push) Successful in 34s Details
Test / test_write_no_same (push) Successful in 14s Details
Test / test_write_xor (push) Successful in 52s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m6s Details
Test / test_rebalance_verify_ec (push) Successful in 5m10s Details
Test / test_heal_pg_size_2 (push) Successful in 4m1s Details
Test / test_heal_ec (push) Successful in 4m21s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m10s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m51s Details
Test / test_heal_csum_32k (push) Successful in 6m54s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m38s Details
Test / test_scrub (push) Successful in 1m9s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m2s Details
Test / test_scrub_xor (push) Successful in 43s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m24s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m16s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m38s Details
Test / test_scrub_ec (push) Successful in 37s Details
Test / test_heal_csum_4k (push) Successful in 6m2s Details
2023-11-04 20:59:56 +03:00
Vitaliy Filippov 1fe678e57b Add --no-block to udev rule
Test / test_minsize_1 (push) Successful in 13s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 2m0s Details
Test / test_move_reappear (push) Successful in 21s Details
Test / test_rm (push) Successful in 15s Details
Test / test_snapshot_down (push) Successful in 30s Details
Test / test_snapshot_down_ec (push) Successful in 32s Details
Test / test_splitbrain (push) Successful in 23s Details
Test / test_snapshot_chain (push) Successful in 2m29s Details
Test / test_snapshot_chain_ec (push) Successful in 3m4s Details
Test / test_rebalance_verify_ec_imm (push) Failing after 18s Details
Test / test_write (push) Successful in 29s Details
Test / test_rebalance_verify_imm (push) Successful in 2m53s Details
Test / test_write_no_same (push) Successful in 12s Details
Test / test_rebalance_verify (push) Successful in 3m46s Details
Test / test_write_xor (push) Failing after 3m6s Details
Test / test_rebalance_verify_ec (push) Successful in 5m1s Details
Test / test_heal_pg_size_2 (push) Successful in 4m50s Details
Test / test_heal_ec (push) Successful in 4m34s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m5s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m57s Details
Test / test_heal_csum_32k (push) Successful in 6m56s Details
Test / test_heal_csum_4k_dmj (push) Successful in 7m28s Details
Test / test_scrub (push) Successful in 1m10s Details
Test / test_scrub_zero_osd_2 (push) Successful in 57s Details
Test / test_scrub_xor (push) Successful in 53s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m34s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m8s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m37s Details
Test / test_scrub_ec (push) Successful in 41s Details
Test / test_heal_csum_4k (push) Successful in 6m6s Details
2023-10-30 12:18:29 +03:00
Vitaliy Filippov 2e592a2f22 Fix undefined variable "timeout"
Test / test_snapshot_ec (push) Successful in 44s Details
Test / test_rm (push) Successful in 17s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 2m9s Details
Test / test_snapshot_down (push) Successful in 23s Details
Test / test_move_reappear (push) Failing after 51s Details
Test / test_snapshot_down_ec (push) Successful in 25s Details
Test / test_splitbrain (push) Successful in 24s Details
Test / test_snapshot_chain (push) Successful in 2m26s Details
Test / test_snapshot_chain_ec (push) Failing after 3m6s Details
Test / test_rebalance_verify_imm (push) Successful in 3m2s Details
Test / test_write (push) Successful in 35s Details
Test / test_rebalance_verify (push) Successful in 3m56s Details
Test / test_write_no_same (push) Successful in 12s Details
Test / test_write_xor (push) Successful in 38s Details
Test / test_rebalance_verify_ec (push) Successful in 5m2s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m13s Details
Test / test_heal_pg_size_2 (push) Successful in 4m17s Details
Test / test_heal_ec (push) Successful in 5m2s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m43s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m36s Details
Test / test_heal_csum_32k (push) Successful in 7m4s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m47s Details
Test / test_scrub (push) Successful in 1m11s Details
Test / test_scrub_zero_osd_2 (push) Successful in 59s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m26s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m32s Details
Test / test_heal_csum_4k (push) Successful in 6m31s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 27s Details
Test / test_scrub_ec (push) Successful in 25s Details
Test / test_scrub_xor (push) Failing after 3m7s Details
2023-10-29 01:30:55 +03:00
Vitaliy Filippov b92f644e3a Fix statistics aggregation, calculate inode stats by first deriving per-OSD stats, too 2023-10-29 01:30:55 +03:00
Vitaliy Filippov 8222e3c77d Release 1.1.0
Test / test_interrupted_rebalance_ec (push) Successful in 1m49s Details
Test / test_snapshot_ec (push) Successful in 38s Details
Test / test_rm (push) Successful in 15s Details
Test / test_snapshot_down (push) Successful in 23s Details
Test / test_move_reappear (push) Failing after 49s Details
Test / test_snapshot_down_ec (push) Successful in 23s Details
Test / test_splitbrain (push) Successful in 22s Details
Test / test_snapshot_chain (push) Successful in 2m25s Details
Test / test_snapshot_chain_ec (push) Successful in 3m5s Details
Test / test_rebalance_verify_imm (push) Successful in 2m51s Details
Test / test_write (push) Successful in 34s Details
Test / test_rebalance_verify (push) Successful in 3m38s Details
Test / test_write_no_same (push) Successful in 14s Details
Test / test_write_xor (push) Successful in 50s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m3s Details
Test / test_rebalance_verify_ec (push) Successful in 5m0s Details
Test / test_heal_pg_size_2 (push) Successful in 4m2s Details
Test / test_heal_ec (push) Successful in 4m49s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m27s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m44s Details
Test / test_heal_csum_32k (push) Successful in 6m57s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m50s Details
Test / test_scrub (push) Successful in 1m12s Details
Test / test_scrub_xor (push) Successful in 48s Details
Test / test_scrub_zero_osd_2 (push) Successful in 54s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m14s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m32s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m38s Details
Test / test_heal_csum_4k (push) Successful in 6m20s Details
Test / test_scrub_ec (push) Successful in 27s Details
New features:

- Implement [client writeback cache](docs/config/client.en.md#client_enable_writeback)
- Add the third I/O mode: [O_DIRECT|O_SYNC](docs/config/osd.en.md#data_io) (good for Optane)
- Reduce load on etcd by splitting OSD lease and statistics reporting intervals:
  [etcd_stats_interval](docs/config/osd.en.md#etcd_stats_interval) (default 30 sec)
- Make MON automatically filter OSDs by layout (block_size/immediate_commit/bitmap_granularity)
  to prevent "refusing to start PGs of this pool" errors on misconfiguration
- Support running fio benchmarks on systems without io_uring
- Make QEMU driver compatible with QEMU 8.1
- Document usage of [vhost-user-blk](docs/usage/qemu.en.md#vhost-user-blk)

Bug fixes:

- Fix resizing disks in QEMU driver (for example, in Proxmox)
- Fix "unexpected result" in Proxmox driver by making CLI flush output on exit
- Remove unneeded block_size mismatch warnings on pools without matching PGs
- Fix possible segfault in vitastor-cli ls -l (usually with deleted pools)
- Fix QEMU driver compatibility with systems without io_uring
- Fix monitor eating 100% CPU when etcd is down (caused by infinite retries)
- Fix potential incorrect write processing with snapshots (not caught in tests
  but could probably lead to client hangs)
- Fix buffer insertion in cluster_client (not caught in tests but could
  probably lead to incorrect writes in rare cases)
- Fix rare OSD crash during sync operation processing
- Fix a reenterability issue in cluster_client not reproducible in QEMU/fio,
  but reproducible with the currently developed K/V database implementation
- Fix deletion of the first modified object - OSDs could crash if you modified
  the same object a lot of times, then deleted it, and then modified it again
- Fix the fio_sec_osd test tool
2023-10-28 00:33:06 +03:00
Vitaliy Filippov be7e76f849 Split etcd_stats_interval out of etcd_report_interval
Test / test_interrupted_rebalance_ec (push) Successful in 1m46s Details
Test / test_snapshot_ec (push) Successful in 36s Details
Test / test_move_reappear (push) Successful in 19s Details
Test / test_rm (push) Successful in 15s Details
Test / test_snapshot_down (push) Successful in 29s Details
Test / test_snapshot_down_ec (push) Successful in 30s Details
Test / test_splitbrain (push) Successful in 26s Details
Test / test_snapshot_chain (push) Successful in 2m15s Details
Test / test_snapshot_chain_ec (push) Successful in 2m57s Details
Test / test_rebalance_verify_imm (push) Successful in 2m29s Details
Test / test_rebalance_verify (push) Successful in 3m40s Details
Test / test_write (push) Successful in 1m0s Details
Test / test_write_no_same (push) Successful in 13s Details
Test / test_write_xor (push) Successful in 50s Details
Test / test_rebalance_verify_ec (push) Successful in 4m58s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m14s Details
Test / test_heal_pg_size_2 (push) Successful in 4m21s Details
Test / test_heal_ec (push) Successful in 4m5s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m36s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m28s Details
Test / test_heal_csum_32k (push) Successful in 6m38s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m46s Details
Test / test_scrub_zero_osd_2 (push) Successful in 59s Details
Test / test_scrub (push) Successful in 1m16s Details
Test / test_scrub_xor (push) Successful in 53s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m57s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m18s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m7s Details
Test / test_heal_csum_4k (push) Successful in 5m43s Details
Test / test_scrub_ec (push) Successful in 32s Details
2023-10-27 01:26:26 +03:00
Vitaliy Filippov 38db53f5ee Implement client writeback cache
- Disabled by default, enable with client_enable_writeback=true
- Even then only enabled in FIO when -direct is disabled and in QEMU when
  block device cache is enabled in settings
- Can also be enabled in other clients like vitastor-cli using parameter
  client_writeback_allowed=true, but not recommended
2023-09-16 17:52:17 +03:00
Vitaliy Filippov ff479a102d Make MON filter OSDs by block layout to prevent "refusing to start PGs of this pool" errors on misconfiguration 2023-09-16 17:52:17 +03:00
Vitaliy Filippov ab8627c9fa Fix monitor retrying failed etcd connection in an infinite loop without pauses 2023-08-09 00:57:08 +03:00
Vitaliy Filippov 25a15d24cf Fix incorrect EC space statistics in `vitastor-cli status`
Test / test_etcd_fail (push) Successful in 1m21s Details
Test / test_interrupted_rebalance_imm (push) Successful in 2m9s Details
Test / test_interrupted_rebalance_ec (push) Successful in 1m52s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m25s Details
Test / test_failure_domain (push) Successful in 10s Details
Test / test_snapshot (push) Successful in 28s Details
Test / test_snapshot_ec (push) Successful in 30s Details
Test / test_minsize_1 (push) Successful in 15s Details
Test / test_move_reappear (push) Successful in 17s Details
Test / test_rm (push) Successful in 11s Details
Test / test_snapshot_chain (push) Successful in 2m1s Details
Test / test_snapshot_chain_ec (push) Successful in 2m41s Details
Test / test_snapshot_down (push) Successful in 23s Details
Test / test_snapshot_down_ec (push) Successful in 24s Details
Test / test_splitbrain (push) Successful in 17s Details
Test / test_rebalance_verify (push) Successful in 3m9s Details
Test / test_rebalance_verify_imm (push) Successful in 3m9s Details
Test / test_rebalance_verify_ec (push) Successful in 3m23s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 5m38s Details
Test / test_write (push) Successful in 33s Details
Test / test_write_xor (push) Successful in 43s Details
Test / test_write_no_same (push) Successful in 14s Details
Test / test_heal_pg_size_2 (push) Successful in 4m16s Details
Test / test_heal_ec (push) Successful in 5m0s Details
Test / test_scrub (push) Successful in 56s Details
Test / test_scrub_zero_osd_2 (push) Successful in 41s Details
Test / test_scrub_xor (push) Successful in 32s Details
Test / test_scrub_pg_size_3 (push) Successful in 53s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 45s Details
Test / test_scrub_ec (push) Successful in 40s Details
2023-07-27 02:26:17 +00:00
Vitaliy Filippov 2f999d8607 Reduce etcd memory usage
Test / test_etcd_fail (push) Successful in 1m34s Details
Test / test_interrupted_rebalance (push) Successful in 1m52s Details
Test / test_interrupted_rebalance_imm (push) Successful in 2m4s Details
Test / test_interrupted_rebalance_ec (push) Successful in 2m13s Details
Test / test_failure_domain (push) Successful in 12s Details
Test / test_snapshot (push) Successful in 19s Details
Test / test_snapshot_ec (push) Successful in 20s Details
Test / test_minsize_1 (push) Successful in 13s Details
Test / test_move_reappear (push) Successful in 19s Details
Test / test_rm (push) Successful in 11s Details
Test / test_snapshot_chain (push) Successful in 1m56s Details
Test / test_snapshot_chain_ec (push) Successful in 2m36s Details
Test / test_snapshot_down (push) Successful in 22s Details
Test / test_snapshot_down_ec (push) Successful in 23s Details
Test / test_splitbrain (push) Successful in 15s Details
Test / test_rebalance_verify (push) Successful in 3m6s Details
Test / test_rebalance_verify_imm (push) Successful in 3m9s Details
Test / test_rebalance_verify_ec (push) Successful in 3m27s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 5m30s Details
Test / test_write (push) Successful in 35s Details
Test / test_write_xor (push) Successful in 41s Details
Test / test_write_no_same (push) Successful in 14s Details
Test / test_heal_pg_size_2 (push) Successful in 3m48s Details
Test / test_heal_ec (push) Successful in 4m15s Details
Test / test_scrub (push) Successful in 33s Details
Test / test_scrub_zero_osd_2 (push) Successful in 31s Details
Test / test_scrub_xor (push) Successful in 28s Details
Test / test_scrub_pg_size_3 (push) Successful in 53s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 35s Details
Test / test_scrub_ec (push) Successful in 28s Details
With default --snapshot-count 100000 and GOGC=100 it easily reaches 6.6 GB
even when we only store 1-2 MB of data in it
2023-07-06 00:46:26 +03:00
Vitaliy Filippov d007a374f2 Delete extra /pool/stats/ keys for non-existing pools
Test / test_interrupted_rebalance (push) Failing after 10m5s Details
Test / test_interrupted_rebalance_imm (push) Successful in 1m29s Details
Test / test_interrupted_rebalance_ec (push) Failing after 10m7s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m32s Details
Test / test_failure_domain (push) Successful in 8s Details
Test / test_snapshot (push) Successful in 19s Details
Test / test_snapshot_ec (push) Successful in 19s Details
Test / test_minsize_1 (push) Successful in 12s Details
Test / test_move_reappear (push) Successful in 17s Details
Test / test_rm (push) Successful in 11s Details
Test / test_snapshot_chain (push) Successful in 1m1s Details
Test / test_snapshot_chain_ec (push) Successful in 1m25s Details
Test / test_snapshot_down (push) Successful in 20s Details
Test / test_snapshot_down_ec (push) Successful in 19s Details
Test / test_splitbrain (push) Successful in 12s Details
Test / test_rebalance_verify (push) Successful in 3m1s Details
Test / test_rebalance_verify_imm (push) Successful in 4m11s Details
Test / test_rebalance_verify_ec (push) Successful in 4m19s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m51s Details
Test / test_write (push) Successful in 31s Details
Test / test_write_xor (push) Successful in 41s Details
Test / test_write_no_same (push) Successful in 12s Details
Test / test_heal_pg_size_2 (push) Successful in 4m10s Details
Test / test_heal_ec (push) Failing after 10m11s Details
Test / test_scrub (push) Successful in 43s Details
Test / test_scrub_zero_osd_2 (push) Successful in 36s Details
Test / test_scrub_xor (push) Successful in 37s Details
Test / test_scrub_pg_size_3 (push) Successful in 48s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 23s Details
Test / test_scrub_ec (push) Successful in 22s Details
2023-07-06 00:40:13 +03:00
Vitaliy Filippov f12b8e45a9 Remove /usr/local/bin path from make-etcd
Test / test_create_nomaxid (push) Successful in 6s Details
Test / test_etcd_fail (push) Successful in 51s Details
Test / test_interrupted_rebalance (push) Successful in 7m28s Details
Test / test_interrupted_rebalance_imm (push) Failing after 10m10s Details
Test / test_interrupted_rebalance_ec (push) Successful in 7m21s Details
Test / test_interrupted_rebalance_ec_imm (push) Failing after 10m7s Details
Test / test_failure_domain (push) Successful in 7s Details
Test / test_snapshot (push) Successful in 17s Details
Test / test_snapshot_ec (push) Successful in 18s Details
Test / test_minsize_1 (push) Successful in 11s Details
Test / test_move_reappear (push) Successful in 17s Details
Test / test_rm (push) Successful in 10s Details
Test / test_snapshot_chain (push) Successful in 1m19s Details
Test / test_snapshot_chain_ec (push) Successful in 1m43s Details
Test / test_splitbrain (push) Successful in 12s Details
Test / test_rebalance_verify (push) Successful in 3m9s Details
Test / test_rebalance_verify_imm (push) Successful in 3m3s Details
Test / test_rebalance_verify_ec (push) Successful in 3m31s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 5m33s Details
Test / test_write (push) Successful in 49s Details
Test / test_write_xor (push) Successful in 58s Details
Test / test_write_no_same (push) Successful in 16s Details
Test / test_heal_pg_size_2 (push) Successful in 3m47s Details
Test / test_heal_ec (push) Successful in 3m59s Details
Test / test_scrub (push) Successful in 58s Details
Test / test_scrub_zero_osd_2 (push) Successful in 42s Details
Test / test_scrub_xor (push) Successful in 30s Details
Test / test_scrub_pg_size_3 (push) Successful in 39s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 31s Details
Test / test_scrub_ec (push) Successful in 31s Details
2023-06-29 23:49:31 +03:00
Vitaliy Filippov a4186e20aa First derive, then sum per-OSD statistics instead of first summing and then deriving
Test / test_change_pg_count (push) Successful in 43s Details
Test / test_change_pg_count_ec (push) Successful in 37s Details
Test / test_change_pg_size (push) Successful in 8s Details
Test / test_create_nomaxid (push) Successful in 8s Details
Test / test_failure_domain (push) Successful in 16s Details
Test / test_interrupted_rebalance (push) Successful in 1m49s Details
Test / test_interrupted_rebalance_imm (push) Successful in 1m38s Details
Test / test_interrupted_rebalance_ec (push) Successful in 1m49s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m23s Details
Test / test_minsize_1 (push) Successful in 13s Details
Test / test_move_reappear (push) Successful in 16s Details
Test / test_rebalance_verify (push) Successful in 3m2s Details
Test / test_rebalance_verify_imm (push) Successful in 2m53s Details
Test / test_rebalance_verify_ec (push) Successful in 3m9s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 5m27s Details
Test / test_rm (push) Successful in 17s Details
Test / test_snapshot (push) Successful in 34s Details
Test / test_snapshot_ec (push) Successful in 29s Details
Test / test_splitbrain (push) Successful in 22s Details
Test / test_write (push) Successful in 37s Details
Test / test_write_xor (push) Successful in 44s Details
Test / test_write_no_same (push) Successful in 16s Details
Test / test_heal_pg_size_2 (push) Successful in 3m31s Details
Test / test_heal_ec (push) Successful in 4m20s Details
Test / test_scrub (push) Successful in 38s Details
Test / test_scrub_zero_osd_2 (push) Successful in 30s Details
Test / test_scrub_xor (push) Successful in 32s Details
Test / test_scrub_pg_size_3 (push) Successful in 42s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 37s Details
Test / test_scrub_ec (push) Successful in 34s Details
This makes statistics reported by vitastor-cli status much smoother
2023-06-18 01:32:24 +03:00
Vitaliy Filippov aea567cfbd Slightly improve scrub docs
Test / test_cas (push) Successful in 9s Details
Test / test_change_pg_count (push) Successful in 52s Details
Test / test_change_pg_count_ec (push) Successful in 1m0s Details
Test / test_change_pg_size (push) Successful in 16s Details
Test / test_create_nomaxid (push) Successful in 16s Details
Test / test_etcd_fail (push) Successful in 56s Details
Test / test_failure_domain (push) Successful in 13s Details
Test / test_interrupted_rebalance (push) Successful in 1m24s Details
Test / test_interrupted_rebalance_imm (push) Successful in 1m10s Details
Test / test_interrupted_rebalance_ec (push) Successful in 1m9s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m6s Details
Test / test_minsize_1 (push) Failing after 19s Details
Test / test_move_reappear (push) Successful in 28s Details
Test / test_rebalance_verify (push) Successful in 2m25s Details
Test / test_rebalance_verify_imm (push) Successful in 2m19s Details
Test / test_rebalance_verify_ec (push) Successful in 3m3s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 2m20s Details
Test / test_rm (push) Successful in 16s Details
Test / test_snapshot (push) Successful in 21s Details
Test / test_snapshot_ec (push) Successful in 28s Details
Test / test_splitbrain (push) Successful in 20s Details
Test / test_write_xor (push) Has started running Details
Test / test_heal_pg_size_2 (push) Has started running Details
Test / test_write (push) Has started running Details
Test / test_scrub (push) Has been cancelled Details
Test / test_scrub_zero_osd_2 (push) Has been cancelled Details
Test / test_scrub_xor (push) Has been cancelled Details
Test / test_scrub_pg_size_3 (push) Has been cancelled Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Has been cancelled Details
Test / test_scrub_ec (push) Has been cancelled Details
2023-05-21 12:52:30 +03:00
Vitaliy Filippov ce02f47de6 Allow to disable scrub_find_best 2023-05-21 12:33:38 +03:00
Vitaliy Filippov 8d40ad99a6 Add scrub documentation 2023-05-20 23:19:39 +03:00
Vitaliy Filippov 3475772b07 Add configuration online update documentation 2023-05-20 23:19:39 +03:00
Vitaliy Filippov 6648f6bb6e Implement ambiguity detection during scrub 2023-05-20 23:19:39 +03:00