Vitaliy Filippov
0471b09b9c
Add license notices to all source code files
2020-09-17 23:07:06 +03:00
Vitaliy Filippov
242d9a42a2
Change object format in prints to %lx:%lx v%lu
2020-09-05 17:44:05 +03:00
Vitaliy Filippov
416a80b099
Make blockstore object state a combination of type and workflow
2020-07-04 22:20:32 +03:00
Vitaliy Filippov
985c309d7f
Remove duplicate code between blockstore_{rollback,stable} and blockstore_init
2020-06-02 20:37:00 +03:00
Vitaliy Filippov
0f43f6d3f6
Fix crashes, print some stats
...
Notably:
- fix the `delete op` inside lambda callback crash (it frees the lambda itself
which results in use-after-free with g++)
- fix stop_client() reenterability
- fix a bug in the blockstore layer which resulted in always returning version=0
for zero-length reads
- change error codes for blockstore_stabilize
2020-03-31 17:55:31 +03:00
Vitaliy Filippov
8a8b619875
Handle secondary OSD connection errors [in theory]
2020-03-30 19:51:34 +03:00
Vitaliy Filippov
c0a22d825d
Fix degraded object recovery (it seems to work now)
2020-03-25 02:17:41 +03:00
Vitaliy Filippov
46f9bd2a69
Make blockstore list operation return consistent snapshots
2020-03-14 02:10:25 +03:00
Vitaliy Filippov
3f522c66e6
Implement immediate commit mode
2020-03-10 01:59:15 +03:00
Vitaliy Filippov
c3737ae3ff
Add journal fsync to stabilize/rollback
2020-03-09 00:35:58 +03:00
Vitaliy Filippov
1696446545
Rename min/max _used to _flushed
2020-03-07 16:41:58 +03:00
Vitaliy Filippov
1e21555343
Add FIXME with Oops
2020-03-04 20:34:45 +03:00
Vitaliy Filippov
2b09710d6f
Implement blockstore rollback operation
...
Rollback operation is required for the primary OSD to kill unstable
object versions in OSD peers so they don't occupy journal space
2020-01-24 20:18:14 +03:00
Vitaliy Filippov
43f6cfeb73
Extract alignments to options
2020-01-16 00:54:25 +03:00
Vitaliy Filippov
a3d3949dce
Do not overwrite same journal sector multiple times
...
It doesn't reduce actual WA, but it reduces tail latency (Q=32, 10% / 50% / 90% / 99% / 99.95%):
- write: 766us/979us/1090us/1303us/1729us vs 1074us/1450us/2212us/3261us/4113us
- sync: 701us/881us/1188us/1762us/2540us vs 269us/955us/1663us/2638us/4146us
2020-01-15 02:53:01 +03:00
Vitaliy Filippov
b3f2102f33
Add queue stall tracking
2020-01-10 01:23:46 +03:00
Vitaliy Filippov
a7e74670a5
Split blockstore implementation and interface header
2019-12-15 14:57:18 +03:00
Vitaliy Filippov
749ab6e2c6
Rename blockstore_operation to blockstore_op_t
2019-12-15 14:57:18 +03:00
Vitaliy Filippov
f4d06ba102
OP_DELETE flushing
2019-12-02 02:41:14 +03:00
Vitaliy Filippov
a7a0946ba8
WIP OP_DELETE
2019-12-01 17:25:59 +03:00
Vitaliy Filippov
35a6ed728d
Fix another stall due to bad unstable_writes tracking, do not try to write beyond the end of the journal
2019-11-28 00:28:08 +03:00
Vitaliy Filippov
9ba243b3ee
Add debug prints
2019-11-27 18:07:51 +03:00
Vitaliy Filippov
ce5cd13bc8
Use fdatasync (just for testing over an FS)
2019-11-27 02:41:30 +03:00
Vitaliy Filippov
1080cbdf12
Fix used journal sector tracking again
2019-11-27 01:20:09 +03:00
Vitaliy Filippov
b8e53f8c67
Fix op validation, fix journal sector usage tracking
...
Now it runs for a while with fio but then crashes with ENOSPC because
of an apparent bug in the allocator
2019-11-26 20:38:25 +03:00
Vitaliy Filippov
3e46728321
Continue fio engine
2019-11-26 00:03:09 +03:00
Vitaliy Filippov
50cf3667fa
Track unstable writes
2019-11-25 01:16:34 +03:00
Vitaliy Filippov
82a2b8e7d9
Fix some extra bugs and it seems now it is even able to trim the journal
2019-11-22 12:08:44 +03:00
Vitaliy Filippov
2b12428cb1
Debug OP_STABLE so the basic case passes without problem
2019-11-21 02:09:18 +03:00
Vitaliy Filippov
e40a71b2ce
Check result to be equal to iov_len
2019-11-18 02:09:34 +03:00
Vitaliy Filippov
eec1c35ea4
Begin to debug ... all of this stuff :)
2019-11-16 02:32:29 +03:00
Vitaliy Filippov
c38c8ab029
Prevent two parallel flushes of the same object
2019-11-15 13:58:35 +03:00
Vitaliy Filippov
f1e236c6e8
Sync metadata & data after copying from journal
2019-11-14 14:09:07 +03:00
Vitaliy Filippov
bc549553e4
Rename dirty_entry.size to len, remove some unneeded methods and fields, read metadata correctly
2019-11-14 01:13:07 +03:00
Vitaliy Filippov
1c6b9778a4
Handle all io_uring events using lambdas
2019-11-13 22:46:42 +03:00
Vitaliy Filippov
214da03735
Move flusher into a separate file
2019-11-13 17:46:05 +03:00
Vitaliy Filippov
75398414d1
Add grouping flusher class
2019-11-13 17:41:57 +03:00
Vitaliy Filippov
db66b3916e
Begin metadata read-modify-write
2019-11-13 16:07:29 +03:00
Vitaliy Filippov
bb55a7fbf4
Zero-fill new objects and write them to the main storage
2019-11-12 20:49:57 +03:00
Vitaliy Filippov
4afa95b0e3
FSM is a dreadful unreadable thing, reimplement using gotos
2019-11-12 18:16:41 +03:00
Vitaliy Filippov
34451b6e44
Try to implement journal flusher as a FSM
2019-11-12 18:16:38 +03:00
Vitaliy Filippov
f892104aed
Begin journal flusher
2019-11-12 12:02:11 +03:00
Vitaliy Filippov
71552adf79
Do not use deque, test cpp-btree
2019-11-11 19:42:18 +03:00
Vitaliy Filippov
46e96c5128
Remove duplicate journal buffer submission code
2019-11-11 18:38:57 +03:00
Vitaliy Filippov
1f04db7d68
Begin to implement stabilize as a batch operation
2019-11-11 16:30:47 +03:00
Vitaliy Filippov
d2d8d6e7fb
Comments about stabilize operation, track unsynced_writes correctly
2019-11-11 02:53:19 +03:00
Vitaliy Filippov
8edb9e9d6f
Remove duplicate journal writing code (and fix it at the same time)
2019-11-11 00:28:14 +03:00
Vitaliy Filippov
ff57dd420e
Rename object_db to clean_db
2019-11-10 22:41:40 +03:00
Vitaliy Filippov
69581e6d0c
Send STABLE entry to the journal
2019-11-10 22:41:40 +03:00
Vitaliy Filippov
890335bff6
Begin implementation of the STABLE operation
2019-11-10 14:37:45 +03:00
Vitaliy Filippov