Vitaliy Filippov
3b84dcaedd
Handle ENOSPC during write - rollback partial EC writes, remember partial replica writes
Test / test_rm (push) Successful in 14s
Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m59s
Details
Test / test_snapshot_down (push) Successful in 28s
Details
Test / test_snapshot_down_ec (push) Successful in 30s
Details
Test / test_splitbrain (push) Successful in 27s
Details
Test / test_snapshot_chain (push) Successful in 2m41s
Details
Test / test_snapshot_chain_ec (push) Successful in 3m12s
Details
Test / test_rebalance_verify_imm (push) Successful in 3m33s
Details
Test / test_rebalance_verify (push) Successful in 4m24s
Details
Test / test_switch_primary (push) Successful in 34s
Details
Test / test_write (push) Successful in 53s
Details
Test / test_write_xor (push) Successful in 51s
Details
Test / test_write_no_same (push) Successful in 11s
Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m11s
Details
Test / test_rebalance_verify_ec (push) Successful in 6m3s
Details
Test / test_heal_pg_size_2 (push) Successful in 4m57s
Details
Test / test_heal_ec (push) Successful in 4m52s
Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m37s
Details
Test / test_heal_csum_32k_dj (push) Successful in 6m55s
Details
Test / test_heal_csum_32k (push) Successful in 6m42s
Details
Test / test_heal_csum_4k_dj (push) Successful in 6m41s
Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m45s
Details
Test / test_scrub_zero_osd_2 (push) Successful in 44s
Details
Test / test_scrub (push) Successful in 48s
Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m6s
Details
Test / test_scrub_pg_size_3 (push) Successful in 1m30s
Details
Test / test_scrub_ec (push) Successful in 51s
Details
Test / test_nfs (push) Successful in 39s
Details
Test / test_heal_csum_4k (push) Successful in 5m22s
Details
Test / test_scrub_xor (push) Successful in 18s
Details
2024-04-07 18:02:05 +03:00
Vitaliy Filippov
f20564b44b
Fix 32-bit build warnings (99.9% in printf)
2024-02-22 12:22:16 +03:00
Vitaliy Filippov
581d02e581
Mark secondary OSDs with deletions as dirty to not forget to sync & autosync them
Test / test_change_pg_count (push) Has been cancelled
Details
Test / test_rm (push) Has been cancelled
Details
Test / test_snapshot_chain (push) Has been cancelled
Details
Test / test_snapshot_chain_ec (push) Has been cancelled
Details
Test / test_snapshot_down (push) Has been cancelled
Details
Test / test_snapshot_down_ec (push) Has been cancelled
Details
Test / test_splitbrain (push) Has been cancelled
Details
Test / test_rebalance_verify (push) Has been cancelled
Details
Test / test_rebalance_verify_imm (push) Has been cancelled
Details
Test / test_rebalance_verify_ec (push) Has been cancelled
Details
Test / test_rebalance_verify_ec_imm (push) Has been cancelled
Details
Test / test_switch_primary (push) Has been cancelled
Details
Test / test_write (push) Has been cancelled
Details
Test / test_write_xor (push) Has been cancelled
Details
Test / test_write_no_same (push) Has been cancelled
Details
Test / test_heal_pg_size_2 (push) Has been cancelled
Details
Test / test_heal_ec (push) Has been cancelled
Details
Test / test_heal_csum_32k_dmj (push) Has been cancelled
Details
Test / test_cas (push) Has been cancelled
Details
Test / test_heal_csum_32k_dj (push) Has been cancelled
Details
Test / test_heal_csum_32k (push) Has been cancelled
Details
Test / test_heal_csum_4k_dmj (push) Has been cancelled
Details
Test / test_heal_csum_4k_dj (push) Has been cancelled
Details
Test / test_heal_csum_4k (push) Has been cancelled
Details
Test / test_scrub (push) Has been cancelled
Details
Test / test_scrub_zero_osd_2 (push) Has been cancelled
Details
Test / test_scrub_xor (push) Has been cancelled
Details
Test / test_scrub_pg_size_3 (push) Has been cancelled
Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Has been cancelled
Details
Test / test_scrub_ec (push) Has been cancelled
Details
2024-02-03 20:31:08 +03:00
Vitaliy Filippov
c8d61568b5
Fix primary_read bitmap buffers being freed too early (use-after-free)
Test / test_etcd_fail (push) Successful in 1m43s
Details
Test / test_interrupted_rebalance (push) Successful in 1m23s
Details
Test / test_interrupted_rebalance_imm (push) Successful in 1m33s
Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m39s
Details
Test / test_failure_domain (push) Successful in 10s
Details
Test / test_snapshot (push) Successful in 19s
Details
Test / test_snapshot_ec (push) Successful in 17s
Details
Test / test_minsize_1 (push) Successful in 13s
Details
Test / test_move_reappear (push) Successful in 17s
Details
Test / test_rm (push) Successful in 11s
Details
Test / test_snapshot_chain (push) Successful in 1m7s
Details
Test / test_snapshot_chain_ec (push) Successful in 2m1s
Details
Test / test_snapshot_down (push) Successful in 22s
Details
Test / test_snapshot_down_ec (push) Successful in 20s
Details
Test / test_splitbrain (push) Successful in 12s
Details
Test / test_rebalance_verify (push) Successful in 2m51s
Details
Test / test_rebalance_verify_imm (push) Successful in 2m49s
Details
Test / test_rebalance_verify_ec (push) Successful in 4m6s
Details
Test / test_rebalance_verify_ec_imm (push) Successful in 5m3s
Details
Test / test_write (push) Successful in 32s
Details
Test / test_write_xor (push) Successful in 34s
Details
Test / test_write_no_same (push) Successful in 12s
Details
Test / test_heal_pg_size_2 (push) Successful in 3m11s
Details
Test / test_heal_ec (push) Successful in 3m54s
Details
Test / test_scrub (push) Successful in 44s
Details
Test / test_scrub_zero_osd_2 (push) Successful in 30s
Details
Test / test_scrub_xor (push) Successful in 31s
Details
Test / test_scrub_pg_size_3 (push) Successful in 37s
Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 32s
Details
Test / test_scrub_ec (push) Successful in 1m31s
Details
2023-06-30 12:47:45 +03:00
Vitaliy Filippov
8603b5cb1d
Do not hang on inactive OSDs during delete, report and skip them instead
Test / test_create_nomaxid (push) Successful in 8s
Details
Test / test_etcd_fail (push) Successful in 46s
Details
Test / test_interrupted_rebalance (push) Successful in 3m13s
Details
Test / test_interrupted_rebalance_imm (push) Successful in 1m30s
Details
Test / test_interrupted_rebalance_ec (push) Successful in 2m18s
Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m34s
Details
Test / test_failure_domain (push) Successful in 7s
Details
Test / test_snapshot (push) Successful in 29s
Details
Test / test_snapshot_ec (push) Successful in 32s
Details
Test / test_minsize_1 (push) Successful in 12s
Details
Test / test_move_reappear (push) Successful in 17s
Details
Test / test_rm (push) Successful in 12s
Details
Test / test_snapshot_chain (push) Successful in 1m37s
Details
Test / test_snapshot_chain_ec (push) Successful in 2m21s
Details
Test / test_splitbrain (push) Successful in 13s
Details
Test / test_rebalance_verify (push) Successful in 3m14s
Details
Test / test_rebalance_verify_imm (push) Successful in 3m8s
Details
Test / test_rebalance_verify_ec (push) Successful in 3m18s
Details
Test / test_rebalance_verify_ec_imm (push) Successful in 5m20s
Details
Test / test_write (push) Successful in 32s
Details
Test / test_write_xor (push) Successful in 43s
Details
Test / test_write_no_same (push) Successful in 13s
Details
Test / test_heal_pg_size_2 (push) Successful in 3m55s
Details
Test / test_heal_ec (push) Successful in 4m2s
Details
Test / test_scrub (push) Successful in 56s
Details
Test / test_scrub_zero_osd_2 (push) Successful in 51s
Details
Test / test_scrub_xor (push) Successful in 29s
Details
Test / test_scrub_pg_size_3 (push) Successful in 1m2s
Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 37s
Details
Test / test_scrub_ec (push) Successful in 32s
Details
2023-06-30 00:15:16 +03:00
Vitaliy Filippov
63c2b9832c
Fix chained (snapshot) reads often not working at all with chain size > 2
Test / test_change_pg_count (push) Successful in 42s
Details
Test / test_change_pg_count_ec (push) Successful in 36s
Details
Test / test_change_pg_size (push) Successful in 9s
Details
Test / test_create_nomaxid (push) Successful in 8s
Details
Test / test_etcd_fail (push) Successful in 1m6s
Details
Test / test_failure_domain (push) Successful in 10s
Details
Test / test_interrupted_rebalance (push) Successful in 1m52s
Details
Test / test_interrupted_rebalance_imm (push) Successful in 1m49s
Details
Test / test_interrupted_rebalance_ec (push) Successful in 2m2s
Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m19s
Details
Test / test_minsize_1 (push) Successful in 13s
Details
Test / test_rebalance_verify (push) Successful in 3m10s
Details
Test / test_rebalance_verify_imm (push) Successful in 2m56s
Details
Test / test_rebalance_verify_ec (push) Successful in 3m6s
Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m9s
Details
Test / test_rm (push) Successful in 14s
Details
Test / test_snapshot (push) Successful in 24s
Details
Test / test_snapshot_ec (push) Successful in 27s
Details
Test / test_splitbrain (push) Successful in 25s
Details
Test / test_write (push) Successful in 1m30s
Details
Test / test_write_xor (push) Successful in 2m14s
Details
Test / test_write_no_same (push) Successful in 20s
Details
Test / test_heal_pg_size_2 (push) Successful in 4m5s
Details
Test / test_heal_ec (push) Successful in 4m11s
Details
Test / test_scrub (push) Successful in 48s
Details
Test / test_scrub_zero_osd_2 (push) Successful in 32s
Details
Test / test_scrub_xor (push) Successful in 30s
Details
Test / test_scrub_pg_size_3 (push) Successful in 53s
Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 37s
Details
Test / test_scrub_ec (push) Successful in 41s
Details
2023-06-28 18:54:03 +03:00
Vitaliy Filippov
6ca20aa194
Allow scrub to fix corrupted object states
2023-05-20 23:19:39 +03:00
Vitaliy Filippov
6648f6bb6e
Implement ambiguity detection during scrub
2023-05-20 23:19:39 +03:00
Vitaliy Filippov
43b77d7619
Implement scrubbing "data path" - OSD_OP_SCRUB
2023-05-20 23:19:39 +03:00
Vitaliy Filippov
8dc427b43c
Retry failed reads (including chained and RMW) from other replicas
2023-05-20 23:19:39 +03:00
Vitaliy Filippov
bf2112653b
Refcount object_states
2023-05-20 23:19:39 +03:00
Vitaliy Filippov
5da1d8e1b5
Fix EC just-bitmap reads (len=0) (fixes SCHEME=ec test_snapshot.sh)
2023-05-07 14:00:08 +03:00
Vitaliy Filippov
629200b0cc
Return ENOSPC as the primary OSD
2022-12-30 02:03:33 +03:00
Vitaliy Filippov
a0cae4c180
Rename "jerasure" to "ec" in pool configuration, function names, fix documentation and Debian build scripts
...
Old pool configurations with "jerasure" also remain supported as an alias for "ec"
2022-06-03 15:40:00 +03:00
Vitaliy Filippov
83146fa3e2
Fix the same HUGE bug for regular reads during rebalance
2022-04-08 11:50:09 +03:00
Vitaliy Filippov
7bdd92ca4f
Fix build under clang and some warnings
...
Build problems fixed:
- void* pointer arithmetic which is a GNU extension (works as byte*)
- "variable size object may not be initialized" which is OK under GCC
- nullptr_t related error in json11 (it lacks 'operator <' in clang)
Warnings fixed:
- empty nested struct initializer { 0 } replaced by {}
- removed several unused lambda captures
2022-01-16 00:02:54 +03:00
Vitaliy Filippov
5cf1157f16
Return real version on CAS failure
2021-08-01 20:05:19 +03:00
Vitaliy Filippov
acf637950c
Implement layer merge
...
A new command merges multiple snapshot/clone layers into one of them,
so merged layers can be deleted after this procedure
2021-07-31 00:23:30 +03:00
Vitaliy Filippov
aad7792d3f
Check for loops in parent inode chains
2021-06-20 00:23:03 +03:00
Vitaliy Filippov
891250d355
Implement CAS writes
...
From now on, reads will return the server-side object version numbers
and writes and deletes will have an additional "version" parameter
which, if set to a non-zero value, will be atomically compared with
the current version of the object plus 1 and the modification will
fail if it doesn't match.
This feature opens the road to correct online flattening of snapshot
layers and other interesting things.
2021-06-15 00:12:35 +03:00
Vitaliy Filippov
38a3df4a0e
Implement chained (optimized) read in the primary OSD code
2021-04-10 17:44:12 +03:00
Vitaliy Filippov
d6524670e1
Introduce data distribution locality
2021-04-10 17:44:12 +03:00
Vitaliy Filippov
ab39ce2bbb
Use clean_entry_bitmap_size instead of entry_attr_size back because of changed bitmap handling
2021-04-10 17:44:12 +03:00
Vitaliy Filippov
d0c2e31312
Add a test for snapshots, fix bugs. Now the test passes
2021-04-10 17:44:12 +03:00
Vitaliy Filippov
9038d42327
Fix several snapshot I/O bugs
2021-04-10 17:44:12 +03:00
Vitaliy Filippov
0aa2dd2890
Send bitmaps with primary-reads, actually read bitmaps for READ ops
2021-04-10 17:44:12 +03:00
Vitaliy Filippov
6bf88883ac
Allocate bitmaps along with stripes to avoid memory fragmentation
2021-04-10 17:44:12 +03:00
Vitaliy Filippov
004f265393
Remove cryptic bitmap inlining from bs_op_t and osd_op_t, use bitmap in primary OSD code
2021-04-10 17:44:12 +03:00
Vitaliy Filippov
95c29b9dc3
Add "external" bitmap support to osd_rmw
2021-04-10 17:44:12 +03:00
Vitaliy Filippov
97efb9e299
Do not crash on PG re-peering events when operations are in progress
2021-04-07 11:06:31 +03:00
Vitaliy Filippov
54f2353f24
Use bitmap granularity for alignment checks
2021-04-03 14:36:04 +03:00
Vitaliy Filippov
883bf84a16
Fix build
2021-04-03 01:47:15 +03:00
Vitaliy Filippov
0949f08407
Extract osd_primary write and sync code into separate files
2021-03-24 14:20:56 +03:00
Vitaliy Filippov
cf9a641d66
Skip disconnected OSDs during sync
2021-03-24 14:20:56 +03:00
Vitaliy Filippov
05db1308aa
Fix two potential read/write ordering problems (even though not yet seen in tests)
...
- Write operations could be 'stabilized' and previous versions could be
purged from OSDs before the removal of version_override and following
reads could potentially hit different version in EC pools
- Object was marked clean after completing the delete during recovery, so
reads could in theory hit a deleted version and return nothing
2021-03-24 14:20:56 +03:00
Vitaliy Filippov
435045751d
Delete objects only after a SYNC during rebalance in the non-immediate_commit mode
...
Previously OSDs could commit deletes before writes during recovery or rebalance
in the "lazy fsync" (immediate_commit=off) mode which could result in lost objects
2021-03-16 12:48:26 +03:00
Vitaliy Filippov
1be94da437
Check & remove extra chunks for degraded / incomplete objects, too
2021-03-08 17:04:10 +03:00
Vitaliy Filippov
21e7686037
Fix possible "assertion failed: pg.inflight >= 0" error during PG stop
2021-03-08 17:04:10 +03:00
Vitaliy Filippov
ab21a1908b
Check for the dirty PG flag when trying to continue to stop it after sync
2021-03-08 17:04:10 +03:00
Vitaliy Filippov
6155b23a7e
Replace pgs[id] with pgs.at(id) to prevent accidental auto-vivification
2021-02-28 19:36:59 +03:00
Vitaliy Filippov
bf9a175efc
Move C/C++ sources to src subdirectory
2021-02-25 23:59:03 +03:00
Vitaliy Filippov