Vitaliy Filippov
0471b09b9c
Add license notices to all source code files
1 year ago
Vitaliy Filippov
242d9a42a2
Change object format in prints to %lx:%lx v%lu
1 year ago
Vitaliy Filippov
ec7acc8f3a
Add WRITE_STABLE operation for future replication support
2 years ago
Vitaliy Filippov
416a80b099
Make blockstore object state a combination of type and workflow
2 years ago
Vitaliy Filippov
571be0f380
Make deletions instantly stable
"2-phase" (write->stabilize) process is pointless for deletions because it
doesn't protect us from incomplete objects. This happens because it removes
the version information from metadata after stabilization. Deletions require
"3-phase" process with a potentially very long 3rd phase.
So, deletions will be allowed to generate degraded and incomplete objects,
and for it to not affect users' ability to delete something, the cluster
will allow to delete whole inodes while storing a list of them in etcd.
Proper TRIM will be impossible until the implementation of the aforementioned
"3-phase" process, though.
By the way, this change also fixes a possible write stall after rebalancing
which was caused by the lack of "stabilize delete" operations.
2 years ago
Vitaliy Filippov
c22e096943
Output journal offsets in debug trace in hex, add detailed "still waiting" messages
2 years ago
Vitaliy Filippov
eba053febe
Do not start small writes before finishing the last big write to the same object
2 years ago
Vitaliy Filippov
3dd1b22d55
Fix segfault with concurrent OP_SYNCs
2 years ago
Vitaliy Filippov
3f522c66e6
Implement immediate commit mode
2 years ago
Vitaliy Filippov
c3737ae3ff
Add journal fsync to stabilize/rollback
2 years ago
Vitaliy Filippov
1696446545
Rename min/max _used to _flushed
2 years ago
Vitaliy Filippov
2be4824a7a
Fix a small memory leak and BS_OP_SYNC mishandling, now fio does not hang during primary-osd test
2 years ago
Vitaliy Filippov
f4707709c6
Move ack_sync() near sync_state = SYNC_DONE so it does not hang with disable_journal_fsync
2 years ago
Vitaliy Filippov
1e286eed08
Wait for writes to complete before issuing an fsync in blockstore_sync
Also fix a dormant bug (OP_SYNC could clear unsynced_*_writes and not be added into syncs_in_progress)
2 years ago
Vitaliy Filippov
47663bd1dc
Add (empty) osd_primary.cpp, rename osd_read to osd_receive, add FIXMEs for fsync
2 years ago
Vitaliy Filippov
d0ab2a20b2
Make fsync flags separate for data, metadata and journal
2 years ago
Vitaliy Filippov
43f6cfeb73
Extract alignments to options
2 years ago
Vitaliy Filippov
a3d3949dce
Do not overwrite same journal sector multiple times
It doesn't reduce actual WA, but it reduces tail latency (Q=32, 10% / 50% / 90% / 99% / 99.95%):
- write: 766us/979us/1090us/1303us/1729us vs 1074us/1450us/2212us/3261us/4113us
- sync: 701us/881us/1188us/1762us/2540us vs 269us/955us/1663us/2638us/4146us
2 years ago
Vitaliy Filippov
36d8c8724f
Fix sparse reads using bitmap, fix journal replay (we could sometimes lose its end)
2 years ago
Vitaliy Filippov
b3f2102f33
Add queue stall tracking
2 years ago
Vitaliy Filippov
a7e74670a5
Split blockstore implementation and interface header
2 years ago
Vitaliy Filippov
749ab6e2c6
Rename blockstore_operation to blockstore_op_t
2 years ago
Vitaliy Filippov
a7a0946ba8
WIP OP_DELETE
2 years ago
Vitaliy Filippov
00eeedae90
Add "fsync disabled" mode
2 years ago
Vitaliy Filippov
9ba243b3ee
Add debug prints
2 years ago
Vitaliy Filippov
e2b91968c5
Fix sync confirmations and some pipeline-stall bugs
2 years ago
Vitaliy Filippov
ce5cd13bc8
Use fdatasync (just for testing over an FS)
2 years ago
Vitaliy Filippov
06634054c9
Note to add no-fsync mode in the future
2 years ago
Vitaliy Filippov
2831d40edb
Fix repeated syncs
2 years ago
Vitaliy Filippov
1080cbdf12
Fix used journal sector tracking again
2 years ago
Vitaliy Filippov
b8e53f8c67
Fix op validation, fix journal sector usage tracking
Now it runs for a while with fio but then crashes with ENOSPC because
of an apparent bug in the allocator
2 years ago
Vitaliy Filippov
be3015169f
Safe stop procedure
2 years ago
Vitaliy Filippov
50cf3667fa
Track unstable writes
2 years ago
Vitaliy Filippov
7e87290fca
Clear second sector of the journal, init iov for callbacks
2 years ago
Vitaliy Filippov
3bfa2f5f39
Fix io_uring submission, journal sector selection
2 years ago
Vitaliy Filippov
e40a71b2ce
Check result to be equal to iov_len
2 years ago
Vitaliy Filippov
c2de733e35
Copy io_uring_prep_* to my_uring_prep_* so they do not clear user_data
2 years ago
Vitaliy Filippov
eec1c35ea4
Begin to debug ... all of this stuff :)
2 years ago
Vitaliy Filippov
0627dd0f5e
Used journal sector tracking
2 years ago
Vitaliy Filippov
f1e236c6e8
Sync metadata & data after copying from journal
2 years ago
Vitaliy Filippov
1c6b9778a4
Handle all io_uring events using lambdas
2 years ago
Vitaliy Filippov
ae77a228c7
Rename big_write.block to location
2 years ago
Vitaliy Filippov
46e96c5128
Remove duplicate journal buffer submission code
2 years ago
Vitaliy Filippov
8edb9e9d6f
Remove duplicate journal writing code (and fix it at the same time)
2 years ago
Vitaliy Filippov
7aabe11ef9
Switch object states after sync
2 years ago
Vitaliy Filippov
2d0334f9b9
Remove in_progress_ops
2 years ago
Vitaliy Filippov
ae56fe1067
Finish sync algorithm in theory
2 years ago
Vitaliy Filippov
24f5d71595
Move handle_event code into _read.cpp, _write.cpp; move sync code into _sync.cpp
2 years ago
