Compare commits

...

522 Commits

Author SHA1 Message Date
Vitaliy Filippov d29b5d2d04 Add Russian translation of VNPL-1.1 1 week ago
Vitaliy Filippov 65b0e8e940 Fix typo in VNPL-1.1 1 week ago
Vitaliy Filippov bce357e2a5 Do not read all metadata into memory when dumping 3 weeks ago
Vitaliy Filippov 0876ca09cd Fix dumper includes and print format 3 weeks ago
Vitaliy Filippov dac12d8a4c Implement metadata dump tool 3 weeks ago
Vitaliy Filippov 1eec4407ab Fix inode creation when /index/maxid is out of sync 4 weeks ago
huy 3b7c6dcac2 Fix volume creation from snapshots in Cinder driver 6 months ago
Vitaliy Filippov 342517d126 Fix typo 4 weeks ago
Vitaliy Filippov 675bc12a13 Add extern "C" for systems like Gentoo which miss it in jerasure includes 4 weeks ago
Vitaliy Filippov 101592bbff Release 0.7.1 4 weeks ago
Vitaliy Filippov be4087d9d2 Add a FIXME to test_interrupted_rebalance 4 weeks ago
Vitaliy Filippov 404e43dd2d Note that ISA-L does not need to be enabled separately 4 weeks ago
Vitaliy Filippov 87613ed590 Add ISA-L into RPM specs 4 weeks ago
Vitaliy Filippov 2a2e914ef9 Show no_recovery, no_rebalance and readonly flags in status 4 weeks ago
Vitaliy Filippov 0cdc9292c8 Fix true/false parsing in json configs like no_recovery/no_rebalance 4 weeks ago
Vitaliy Filippov 3e1b03bb5c Show all etcd addresses in the "reporting to..." message 4 weeks ago
Vitaliy Filippov 36e851505a Make monitor delete pool statistics when the pool is deleted 4 weeks ago
Vitaliy Filippov 1efbbb0c36 Make deleted inodes vanish from statistics after 60 seconds 4 weeks ago
Vitaliy Filippov 088dd15449 Exclude empty inodes from stats 4 weeks ago
Vitaliy Filippov 4a531d7b8b Fix listings sending too many parallel requests to OSDs, fix rm-data crashing with --wait-list 4 weeks ago
Vitaliy Filippov a0cae4c180 Rename "jerasure" to "ec" in pool configuration, function names, fix documentation and Debian build scripts 4 weeks ago
Vitaliy Filippov c4eb46600d Merge run_3osds and run_7osds scripts 4 weeks ago
Vitaliy Filippov 21b306e25f Add ISA-L support 1 month ago
Vitaliy Filippov d8313e939a Release 0.7.0 1 month ago
huynnp911 3e92c3f082 Add patch for libvirt 6.0 1 month ago
Vitaliy Filippov 82b9f4c52d Add a test with OSD kills 1 month ago
Vitaliy Filippov 2bdf415eb3 Fix unknown OSD numbers on error 1 month ago
Vitaliy Filippov f826831282 Describe OSD placement tree and reweights 2 months ago
Vitaliy Filippov 5d47bbe04c Add documentation 5 months ago
Vitaliy Filippov 93a9f1ef89 Fix NFS socket read hangs 2 months ago
Vitaliy Filippov 2697aae909 Fix free_down_raw in cli status 2 months ago
Vitaliy Filippov 6b69db73ac Remove getrandom() usage 2 months ago
Vitaliy Filippov d48a824846 Fix some warnings 2 months ago
Vitaliy Filippov 40985282ff Fix build under GCC 8 2 months ago
Vitaliy Filippov acf403e886 Add install target for NFS proxy 2 months ago
Vitaliy Filippov cf03b9c84d Implement "primary affinity tags" 2 months ago
Vitaliy Filippov 7c2379d458 Simplified NFS proxy based on own NFS/XDR implementation 5 months ago
Vitaliy Filippov a2189100dd Make CLI functions usable in library form 3 months ago
Vitaliy Filippov bb84379db6 Release 0.6.17 2 months ago
Vitaliy Filippov 714dda8151 Fix slow connection ramp-up caused by up_wait_retry_interval pausing operations on first connection attempt 2 months ago
Vitaliy Filippov 834554c523 LD_PRELOAD=libasan.so.5 fio in tests fails when vitastor is built with ASan 2 months ago
Vitaliy Filippov e718116f54 Fix incorrect reading of extra metadata block 2 months ago
Vitaliy Filippov 98e3528a14 Add block PVC and pod examples 3 months ago
Vitaliy Filippov 8e88f77101 Fix CSI driver volumeMode: Block support 3 months ago
Vitaliy Filippov caa2cc2e6c Fix 32bit build error 3 months ago
Vitaliy Filippov 842ba8b831 Use (uint64_t)1 instead of 1l / 1ul 3 months ago
Vitaliy Filippov 1493823f9e Note about starting monitors 3 months ago
Vitaliy Filippov c857272f44 Comment: epoch is uint64_t 3 months ago
Vitaliy Filippov 340a4b4f27 Release 0.6.16 3 months ago
Vitaliy Filippov 5118980315 Add a script to run all tests 3 months ago
Vitaliy Filippov d71cc174e3 Implement CLI status command 3 months ago
Vitaliy Filippov 0eb929f1ba Fix change_pg_count test (statistic reporting may take some time) 3 months ago
Vitaliy Filippov 83146fa3e2 Fix the same HUGE bug for regular reads during rebalance 3 months ago
Vitaliy Filippov 15dcaf7903 Add the same "rebalance" test with regular reads 3 months ago
Vitaliy Filippov cd18ef7323 Disconnect NBD proxy correctly without leaving a zombie [vitastor-nbd] process in D state 3 months ago
Vitaliy Filippov 39531ef1a6 Fix incorrect chained reads during rebalance (the bug detected by test_rebalance_verify.sh) 3 months ago
Vitaliy Filippov d334914948 Fix the test so it actually fails indicating a bug :-) 3 months ago
Vitaliy Filippov c373425562 Fix nbd log 3 months ago
Vitaliy Filippov 3615e57879 Register standby monitors in etcd in /mon/member 3 months ago
Vitaliy Filippov 0edc6fe5a6 Add notes about the new script 3 months ago
Vitaliy Filippov 9c30df83e3 Fix a HUGE :) bug in NBD proxy 3 months ago
Vitaliy Filippov a420c77107 Add rebalance-verify test 3 months ago
Vitaliy Filippov 4100d829c7 Allow to override log file for daemonized NBD proxy 3 months ago
Vitaliy Filippov 79ebda933e Fix a write hang with throttling due to timer reenterability / triggerability 3 months ago
Vitaliy Filippov 65d08e067e Add a script for preparing hybrid (HDD+SSD) OSDs 3 months ago
Vitaliy Filippov d289753df4 Implement snapshot deletion for Cinder driver 4 months ago
Vitaliy Filippov 85298ddae2 Release 0.6.15 4 months ago
Vitaliy Filippov e23296a327 Rename cli_rm -> cli_rm_data, cli_snap_rm -> cli_rm 4 months ago
Vitaliy Filippov 839ec9e6e0 Shard clean_db by PGs to speedup listings 4 months ago
Vitaliy Filippov 7cbfdff41a Replace some throws with force_stop 4 months ago
Vitaliy Filippov 951272f27f Try to process PG one after another 4 months ago
Vitaliy Filippov a3fb1d4c98 Fix reenterability around set_timer 4 months ago
Vitaliy Filippov 88402e6eb6 Move next_request to run_cb_and_clear 4 months ago
Vitaliy Filippov 390239c51b Don't terminate HTTP requests with timeouts if response is already available in the socket 4 months ago
Vitaliy Filippov b7b2adfa32 Fix http client not continuing requests in case of failure to connect 4 months ago
Vitaliy Filippov 36c276358b Attempt to fix "head-of-line blocking" by LIST operations 4 months ago
Vitaliy Filippov 117d6f0612 Release 0.6.14 5 months ago
Vitaliy Filippov 7d79c58095 Use the larger sockaddr_storage structure 5 months ago
Vitaliy Filippov 46d2bc100f Add some tolerance to stat calculation so it does not fail on a fresh DB 5 months ago
Vitaliy Filippov 732e2804e9 Fix operation dependency counter underflow for reads without immediate_commit=all mode 5 months ago
Vitaliy Filippov abaec2008c Fix OSDs missing misplaced recovery 5 months ago
Vitaliy Filippov 8129d238a4 Different fio versions have different types for xfer_buflen, but Vitastor anyway does not support 128-bit offsets 5 months ago
Vitaliy Filippov 61ebed144a Fix OSDs possibly dying with "map::at" errors when other OSDs are stopped 5 months ago
Vitaliy Filippov 9d3ba113aa Extract bind socket code into a utility function 5 months ago
Vitaliy Filippov 9788045dc9 Fix division by zero in ls if all pool OSDs are down 5 months ago
Vitaliy Filippov d6b0d29af6 4k MEM_ALIGNMENT 5 months ago
Vitaliy Filippov 36f352f06f Release 0.6.13 5 months ago
Vitaliy Filippov 318cc463c2 Fix warnings 5 months ago
Vitaliy Filippov 145e5cfb86 MCL_ONFAULT is not available under centos 7 5 months ago
Vitaliy Filippov 73ae578981 Add osd_memlock option 5 months ago
Vitaliy Filippov 20ee4ed758 Update some parameter docs 5 months ago
Vitaliy Filippov 63de79d1b2 Change > to | to preserve newlines 5 months ago
Vitaliy Filippov f712967079 And one more sqe starvation fix 5 months ago
Vitaliy Filippov df0cd85352 Fix another part of the "async sqe clear" bug (followup to d9857a5340) 5 months ago
Vitaliy Filippov ebaf4d7a72 Fix compatibility with fio 3.28+ 5 months ago
Vitaliy Filippov d4bc10542c Fix compatibility with liburing >= 2.1 where it only has __pad2[2] 5 months ago
Vitaliy Filippov 140309620a Free recv_buf in nbd_proxy 5 months ago
Vitaliy Filippov 0a610ee943 Destroy the client after completing CLI command 5 months ago
Vitaliy Filippov f3ce166064 Do not print nan% in df when a pool has no available OSDs 5 months ago
Vitaliy Filippov 717d303370 Handle get_sqe failures, don't die with "will fall out of sync" in epoll_manager 5 months ago
Vitaliy Filippov d9857a5340 Check for SQEs, not for completions 5 months ago
Vitaliy Filippov eb5d9153e8 Fix build under centos 7 5 months ago
Vitaliy Filippov ae6d1ed1d5 Remove completed items 5 months ago
Vitaliy Filippov d123e58ea3 Fix yaml syntax - remove ` in default 5 months ago
Vitaliy Filippov d9869d8116 Add parameter documentation 5 months ago
Vitaliy Filippov 4047ca606f Add missing cancel_op(currently being read op) when stopping a client 5 months ago
Vitaliy Filippov 218e294e9c > 0, of course 5 months ago
Vitaliy Filippov c1929cabe0 Release 0.6.12 5 months ago
Vitaliy Filippov cc6b24e03a Allow to configure NBD timeout, max devices and partitions 5 months ago
Vitaliy Filippov 0757ba630a Do not happily NBD "map" non-existing images, do not try to benchmark them too 5 months ago
Vitaliy Filippov 2a0b881685 Respect max_write_iodepth for deletes 5 months ago
Vitaliy Filippov 9a15b843ff Do not set pg_real_size to 0 5 months ago
Vitaliy Filippov 8dc1ffb13b Try to connect with PG peers before deciding it's incomplete :) 5 months ago
Vitaliy Filippov ba63af49b4 Add etcd retries everywhere (they were missing in some places) 5 months ago
Vitaliy Filippov 31b9c683ee Fix flushing of unclean objects 5 months ago
Vitaliy Filippov 3abcac058f Check for double response_callback call more 5 months ago
Vitaliy Filippov e01c4db702 Add paranoic if()s to prevent accidental double free of etcd_watch_ws 5 months ago
Vitaliy Filippov a5cf06acd0 Remove etcd timeout and keepalive interval hardcode 5 months ago
Vitaliy Filippov 9c3653b1e1 Handle EINTR 5 months ago
Vitaliy Filippov 23e578b6a2 Fix common.sh 5 months ago
Vitaliy Filippov 7920414bee Fix build under older gcc (debian buster) 5 months ago
Vitaliy Filippov 098e369a3b Fix rand initialization, add etcd connection/disconnection logging 5 months ago
Vitaliy Filippov a43ef525a2 Remove two last end()s from http_client (should have been removed in the keepalive patch) 5 months ago
Vitaliy Filippov 8a6b07d8f7 Add a 2/5 etcd failure test 5 months ago
Vitaliy Filippov 2c930d55fb
Merge pull request #41 from promobit-bitblaze/1-small-fix 6 months ago
Mikhail Koshel d798e0821e #1 fix deps 6 months ago
Vitaliy Filippov e591a3e9f7 Include sys/stat.h in messenger.cpp 6 months ago
Vitaliy Filippov 77cc18420a Fix leaks detected by clang scan-build (only 1 of 4 may be important though) 6 months ago
Vitaliy Filippov 7bdd92ca4f Fix build under clang and some warnings 6 months ago
Vitaliy Filippov 8f64fc61e7 Ignore empty events in mon 6 months ago
Vitaliy Filippov 4a9f001d9e Make mon also ping etcd websockets regularly 6 months ago
Vitaliy Filippov 8c908316d9 Add a test with an OSD being added 6 months ago
Vitaliy Filippov 515a2e6e33 Only die when detecting a real race condition, not just a CAS failure 6 months ago
Vitaliy Filippov 68b6763ebe Add asserts for lp-optimizer tests, pass `ordered` from the monitor 6 months ago
Vitaliy Filippov 9c6168bf17 Remove fill_parsed_response 6 months ago
Vitaliy Filippov 08e467270a Fix pg_size changing from 3 to 2 6 months ago
Vitaliy Filippov 5473d5b4a2 Rework HTTP client to use keepalive, move getifaddr_list to addr_util 6 months ago
Vitaliy Filippov c3304bce27
Merge pull request #38 from mirrorll/master 6 months ago
Vitaliy Filippov ec2852c598 Add minsize_1 test 6 months ago
Vitaliy Filippov b9f5c2a823 Support zero-copy send in fio_sec_osd to allow testing it 6 months ago
Vitaliy Filippov e9d2f79aa7 Support reading bitmaps in fio_sec_osd 6 months ago
Vitaliy Filippov 0785bdf8b3 Release 0.6.11 6 months ago
Vitaliy Filippov b57e44748b Send 4 byte bitmap in stub_uring_osd 6 months ago
Vitaliy Filippov 1bbe62f29c Fix uninitialized listen_backlog which was leading to REALLY SLOW send speeds!!! 6 months ago
lihai 3061c30132 journal check_available error 6 months ago
Vitaliy Filippov 20a4406acc Support IPv6 OSD addresses 7 months ago
Vitaliy Filippov f93491bc6c Implement journal write batching and slightly refactor journal writes 7 months ago
Vitaliy Filippov 999bed8514 Fix opening regular files as blockstore 7 months ago
Vitaliy Filippov 3f33095fd7 Do not try to initialize client in simple-offsets 7 months ago
Vitaliy Filippov dd74c5ce1b Fix OSDs marking PGs incomplete instead of trying to connect with peers 7 months ago
Vitaliy Filippov c6d104ecd6 Print object version on fatal overwrite 7 months ago
Vitaliy Filippov e544aef7d0 Fix test rw_blocking 7 months ago
Vitaliy Filippov 616c18c786 Fix stub_uring_osd 7 months ago
Vitaliy Filippov fa687d3878 Allow to configure OSD placement in node_placement 7 months ago
Vitaliy Filippov 2c7556e536 Allow to run with 4k sector size. Natural, but it was forbidden 7 months ago
Vitaliy Filippov 2020608a39 Release 0.6.10 7 months ago
Vitaliy Filippov 139b98d80f Exclude block/vitastor.c from patches and add script to easily re-add it 7 months ago
Vitaliy Filippov f54ff6ad5d Do not crash in simple-offsets when some options are empty, too 7 months ago
Vitaliy Filippov b376ef2ed9 Do not crash on empty matched_addrs 7 months ago
Vitaliy Filippov 5a234588b9 Do not die when invoked via `vita` symlink 7 months ago
Vitaliy Filippov b82c30328f Use vitastor-cli df to show pool stats in Proxmox 7 months ago
Vitaliy Filippov 0ee5e0a7fe Implement vitastor-cli df command 7 months ago
Vitaliy Filippov 0a1640d169 Some important fixes for our new Proxmox driver 7 months ago
Vitaliy Filippov 3482bb0860 Fix readonly/readwrite option parsing 7 months ago
Vitaliy Filippov 526995f486 Do not skip empty iops in listings 7 months ago
Vitaliy Filippov 073b505928 Package Proxmox plugin as pve-storage-vitastor 7 months ago
Vitaliy Filippov a8b21a22d0 Add patch for pve-qemu 6.1 7 months ago
Vitaliy Filippov 0b1ffba62b Add Proxmox storage driver 7 months ago
Vitaliy Filippov 8dfbd7943c Use logical block size = 512 bytes by default 7 months ago
Vitaliy Filippov 39e7f98e54 Allow to change etcd IP in tests 7 months ago
Vitaliy Filippov 3a83a32cb7 Aaand now fix create --parent :D 7 months ago
Vitaliy Filippov 20d5ed799a Add glob pattern matching for ls 7 months ago
Vitaliy Filippov b262938bca Fix naggy "Failed to get RDMA device list: Unknown error -38" 7 months ago
Vitaliy Filippov 7e54242251 Add patches for Proxmox QEMU 5.1 and 52 7 months ago
Vitaliy Filippov c3c2e68cc1 Now fix resize command :D 7 months ago
Vitaliy Filippov aa1e21dd99 Release 0.6.9 7 months ago
Vitaliy Filippov f4b57d487f Remove +deb10u1 from libvirt version 7 months ago
Vitaliy Filippov 711ecd2f8e Add a Dockerfile to build libvirt 7 months ago
Vitaliy Filippov 9fca01dc62 Add a forgotten return statement 7 months ago
Vitaliy Filippov 0bd3a94efd Use qdict_get_try_int because qdict_get_int may segfault on a missing key 7 months ago
Vitaliy Filippov 9ffdeef93b Install the built liburing version in el8 dockerfile 7 months ago
Vitaliy Filippov 589892d501 Fix rpm dockerfiles 7 months ago
Vitaliy Filippov 5fe3a40416 More fixes for QEMU 2.x :) 7 months ago
Vitaliy Filippov a453db9c8e An attempt to automatically build patched specs inside Docker is mostly broken for now 7 months ago
Vitaliy Filippov e6498a52ca Rename 4.2 el7 spec patch 7 months ago
Vitaliy Filippov 4bc41aed9d Add patches for QEMU 6.0 and for 6.0 RPM spec 7 months ago
Vitaliy Filippov 4da51f9c4c Update QEMU 3.1 patch 7 months ago
Vitaliy Filippov c6cee6f734 Update QEMU 5.0 patch 7 months ago
Vitaliy Filippov 6fc08f5581 Update CentOS 8 QEMU 4.2 spec patch 7 months ago
Vitaliy Filippov 15957b7d13 Update QEMU 4.2 patch and CentOS 7 QEMU 4.2 spec patch 7 months ago
Vitaliy Filippov 09a3987e83 Remove vitastor-qemu from RPM specs 7 months ago
Vitaliy Filippov cd6820c439 Update QEMU 5.1/5.2 patch to include internal vitastor driver 7 months ago
Vitaliy Filippov dcd8f5e76c Remove qemu shenanigans from vitastor build dockerfile 7 months ago
Vitaliy Filippov 5859f913fc Fix client failover in case of etcd shutdown or crash 7 months ago
Vitaliy Filippov cac6a1d8d1 Don't need to download fio in qemu dockerfile anymore 7 months ago
Vitaliy Filippov a0c32e7de9 Update patch for libvirt 7.6 7 months ago
Vitaliy Filippov 8b37610dd0 Rename patch to Nova 23 because it's actually closer 7 months ago
Vitaliy Filippov ae82ca3b08 Pass config path in <config file="" /> element instead of an attribute 7 months ago
Vitaliy Filippov 92362027a8 Build vitastor driver as part of the QEMU package by default 7 months ago
Vitaliy Filippov c4aeeda143 Fix index removal in vitastor-cli rm 7 months ago
Vitaliy Filippov 24f0f8278a Fix modify --readwrite 7 months ago
Vitaliy Filippov 95496d0845 Implement renaming images in CLI (vitastor-cli modify --rename) 7 months ago
Vitaliy Filippov 94b1f09ef2 Create snapshots in the same pool by default 7 months ago
Vitaliy Filippov 32b1312abb Remove stale deleted inode statistics in monitor 7 months ago
Vitaliy Filippov d5c8fde5de Remove kludgy $IP and $ETCD_MON parsing from make-osd.sh, suggest to use vitastor.conf 7 months ago
Vitaliy Filippov 7a0b5212fe Exit if unable to restart watches 7 months ago
Vitaliy Filippov a8f5c71ae8 Use the same etcd address selection algorithm in the monitor 7 months ago
Vitaliy Filippov ce5b6253ab Make OSDs stick to the last successful etcd address 7 months ago
Vitaliy Filippov 8398ad0117 Fix #36 - Fix old version data sometimes overriding new version data 7 months ago
Vitaliy Filippov fea451b4db Prefer local etcd in OSD 7 months ago
Vitaliy Filippov 6e12aca53b Remove the total PG count restriction in optimize_change which was leading to unfeasible problems sometimes 7 months ago
Vitaliy Filippov 8b007d531f
Merge pull request #33 from moly7x/fix-TypeError 7 months ago
Vitaliy Filippov 7b7f20fb89
Merge pull request #34 from mirrorll/master 7 months ago
Vitaliy Filippov 300d507026 Fix capture of out in alloc_osd 7 months ago
harley 6886171289
report pg state failed 7 months ago
Vitaliy Filippov 43f8ea47a0 Ok, something is not allowed somewhere in C99 7 months ago
Vitaliy Filippov 6e0e172e15 Implement OSD address selection from a specified subnet 7 months ago
Vitaliy Filippov 655a2c871d Move make-osd.sh into vitastor-client package 7 months ago
Vitaliy Filippov 879fe9b2b4 Add a patch for qemu 6.1 and replace _ with - in qemu options 7 months ago
Vitaliy Filippov 660c3f7b0d Change default RDMA settings to 128x 129K buffers 7 months ago
Vitaliy Filippov f0ebfae3b8 Fix vitastor-cli alloc-osd, use vitastor-cli in make-osd.sh 7 months ago
Vitaliy Filippov eb7ad2c114 Fix empty size syntax, use C version of simple-offsets in tests 7 months ago
Tân Lê b4235b4edf
Fixed TypeError 8 months ago
Vitaliy Filippov cd21ff0b6a Rewrite simple-offsets.js in C/C++ 8 months ago
Vitaliy Filippov d3903f039c Implement alloc-osd (allocate a new OSD number) command 8 months ago
Vitaliy Filippov 66fe1a469b Additionally balance parity chunks over OSDs using round-robin when generating initial distribution 8 months ago
Vitaliy Filippov 24409bd4c4 Oops # 2. Fix vitastor-cli create broken due to json11 :D 8 months ago
Vitaliy Filippov c5029961ea Oops. Fix vitastor-cli ls 8 months ago
Vitaliy Filippov 1ca1143d4a Add fio_version and qemu_version variable example files back 8 months ago
Vitaliy Filippov 920345f7b6 Release 0.6.8 8 months ago
Vitaliy Filippov 75b47a6298 Generate pkg-config file 8 months ago
Vitaliy Filippov 6e446653ae Include README 8 months ago
Vitaliy Filippov e51edf2542 Split into multiple rpm packages 8 months ago
Vitaliy Filippov ce170af91f Split into multiple Debian packages 8 months ago
Vitaliy Filippov 7eabc364bf Release 0.6.7 8 months ago
Vitaliy Filippov a346f84c69 Allow to show only specific images in listing 8 months ago
Vitaliy Filippov 71a0c1a7b9 Fix list sorting 8 months ago
Vitaliy Filippov 20e86c7d84 Note about vitastor-cli 8 months ago
Vitaliy Filippov 110b39900b Rename the new "set" command to "modify" 8 months ago
Vitaliy Filippov 697ee30a26 Add ARG REL into build Dockerfiles 8 months ago
Vitaliy Filippov 42479b4590 Fix vitastor-nbd list, add ls alias 8 months ago
Vitaliy Filippov 6e82044e84 Add `vita` symlink 8 months ago
Vitaliy Filippov 2cb3e84882 Implement CLI set (resize, change readonly status) command 8 months ago
Vitaliy Filippov 32614c5bc8 Add a test for the last change ("split-brain avoidance", lol) 8 months ago
Vitaliy Filippov aa436027c8 Report pg/history from OSD on every degraded activation 8 months ago
Vitaliy Filippov 577a563b91 Allow to disable colored output 8 months ago
Vitaliy Filippov e4efa2c08a Improve vitastor-cli ls - show I/O statistics, allow to sort & limit output 8 months ago
Vitaliy Filippov 0f3f0a9d29 Calculate average statistics in mon, remove buggy "fix_stat_overflows" 8 months ago
Vitaliy Filippov 0544a16f95
Merge pull request #30 from moly7x/moly7x-fix-No-weighed-backend-found 8 months ago
Tân Lê 30d8930958
Revert spacing in comment 8 months ago
Tân Lê baf003fbd3
Revert newline in comment 8 months ago
Tân Lê ba39a38dc4
Fix No weighed backend found 8 months ago
Vitaliy Filippov d528cd77f1 Fix install_symlink 8 months ago
Vitaliy Filippov 6e6f407df3 Simplify & fix monitor stats aggregation 8 months ago
Vitaliy Filippov 4d43774cbb Use 5s etcd_report_interval by default 8 months ago
Vitaliy Filippov a1488f7217 Fix qemu_driver to build with QEMU 2.x (previously it was only correct for QEMU 2.0) 8 months ago
Vitaliy Filippov 404e07d365 Implement image/snapshot/clone creation and listing by pool 8 months ago
Vitaliy Filippov b3dcee0d43 Also print "bare" inodes with missing config if they occupy space 8 months ago
Vitaliy Filippov 609bd4eb59 Remove naggy RDMA messages when log level is zero 8 months ago
Vitaliy Filippov 8e445ddc9a Begin to implement CLI: implement listing, add help, add create stub 8 months ago
Vitaliy Filippov ffb06536ff Revoke lease in mon on SIGINT & SIGTERM, fix raw_to_usable calculation 8 months ago
Vitaliy Filippov eeecab20c2
Merge pull request #28 from moly7x/fix 8 months ago
Tân Lê e889ac4209
Fix building QEMU 3.1 8 months ago
Vitaliy Filippov cfe8de9b84 Autosync based on number of unstable ops to prevent journal stalls 8 months ago
Vitaliy Filippov 24b9b19066 Change rm to rm-data as it was renamed 8 months ago
Vitaliy Filippov ef645ee0c2
Merge pull request #26 from moly7x/fix 8 months ago
Tân Lê 8a9bae5216 FIX ERROR cinder.volume.drivers.vitastor KeyError: 'total_raw_tb' 8 months ago
Vitaliy Filippov da99686a15 Correctly aggregate pool statistics for unknown pools 8 months ago
Vitaliy Filippov dcc03ee41f Add go.sum 9 months ago
Vitaliy Filippov fb2f7a0d3c Release 0.6.6 9 months ago
Vitaliy Filippov 38d85da19a Fix build for older gcc 9 months ago
Vitaliy Filippov dc3caee284 Add Dockerfile 9 months ago
Vitaliy Filippov 89dcda1fed Remove "bool" from the C header 9 months ago
Vitaliy Filippov 1526e2055e Do not crash with RDMA when receiving garbage, free RDMA buffers when connection is closed 9 months ago
Vitaliy Filippov 74cb3911db Rebase children of the "inverse" child when it is removed, change /index/image/%s keys during metadata ops 9 months ago
Vitaliy Filippov d5efbbb6b9 Rename commands and add CLI help 9 months ago
Vitaliy Filippov 4319091bd3 Implement "inverse merge" optimisation 9 months ago
Vitaliy Filippov 6d307d5391 Ignore "readonly" flag when merging snapshots 9 months ago
Vitaliy Filippov 065dfef683 Rename vitastor-cmd to vitastor-cli 9 months ago
Vitaliy Filippov 4d6b85fe67 Split one big cmd.cpp into multiple files 9 months ago
Vitaliy Filippov 2dd2f29f46 Move get_inode_cfg to cli_tool_t 9 months ago
Vitaliy Filippov fc3a1e076a Fix minor bugs in snapshot removal, check it in tests 9 months ago
Vitaliy Filippov 3a3e168c42 Implement high-level snapshot flatten and remove commands 9 months ago
Vitaliy Filippov 95c55da0ad Implement merge with CAS 11 months ago
Vitaliy Filippov 5cf1157f16 Return real version on CAS failure 11 months ago
Vitaliy Filippov acf637950c Implement layer merge 11 months ago
Vitaliy Filippov a02b02eb04 Use new listing methods in rm_inode 12 months ago
Vitaliy Filippov 7d3d696110 Implement object listing with controllable parallelism in cluster_client 12 months ago
Vitaliy Filippov 712576ca75
Merge pull request #13 from lnsyyj/wip-vitastor-debug 12 months ago
Vitaliy Filippov 28bd94d2c2 Make diagnostics slightly better 12 months ago
Vitaliy Filippov 148ff04aa8 Do not lose flusher queue entries when an "older object rescan" happens in parallel with flushing of an older version of another object 12 months ago
JiangYu e86df4a2a2 fix BLOCKSTORE_DEBUG, error: ‘dirty_it’ was not declared in this scope 12 months ago
Vitaliy Filippov e74af9745e Print journal flusher diagnostics on slow ops 12 months ago
Vitaliy Filippov 0e0509e3da Dump op states in slow operation log 12 months ago
Vitaliy Filippov cb282d25e0 Release 0.6.5 12 months ago
Vitaliy Filippov 8b2a4c9539 Fix centos builds (yum-builddep stopped working in el7, cmake in el8..) 12 months ago
Vitaliy Filippov b66a079892 State basic OpenStack support 12 months ago
Vitaliy Filippov e90bbe6385 Implement OpenStack Cinder driver for Vitastor 12 months ago
Vitaliy Filippov 4be761254c Move patches to patches/ 12 months ago
Vitaliy Filippov 7a45c5f86c buster-backports has broken mesa 12 months ago
Vitaliy Filippov bff413584d Fix qemuBlockStorageSourceGetVitastorProps 12 months ago
Vitaliy Filippov bb31050ab5 Add missing image, config_path options to QEMU QAPI 12 months ago
Vitaliy Filippov b52dd6843a Rename qemu_rbd_unescape and qemu_rbd_next_tok to *_vitastor_* 1 year ago
Vitaliy Filippov b66160a7ad Aggregate per-pool statistics in mon 1 year ago
Vitaliy Filippov 30bb602681 Add _VITASTOR to missing switches in libvirt 7.0 patch 1 year ago
Vitaliy Filippov eb0a3adafc Patch libvirt schema, add an example to test libvirt 1 year ago
Vitaliy Filippov 24301b116c Add libvirt 5.0 patch 1 year ago
Vitaliy Filippov 1d00c17d68 Add libvirt 7.5 patch 1 year ago
Vitaliy Filippov 24f19c4b80 Add libvirt 7.0 patch 1 year ago
Vitaliy Filippov dfdf5c1f9c Fix comments in mon.js 1 year ago
Vitaliy Filippov aad7792d3f Check for loops in parent inode chains 1 year ago
Vitaliy Filippov 6ca8afffe5 Add CAS version parameter to the C wrapper 1 year ago
Vitaliy Filippov 511a89948b Rework qemu_proxy into a C wrapper library with public header 1 year ago
Vitaliy Filippov 3de553ecd7 Add a test for CAS write operation 1 year ago
Vitaliy Filippov 9c45d43e74 Extract common 3 OSD code from several test scripts 1 year ago
Vitaliy Filippov 891250d355 Implement CAS writes 1 year ago
Vitaliy Filippov f9fe72d40a Release 0.6.4 1 year ago
Vitaliy Filippov 10ee4f7c1d Add notes about CSI to README 1 year ago
Vitaliy Filippov fd8244699b Implement basic CSI driver 1 year ago
Vitaliy Filippov eaac1fc5d1 Log to stderr in etcd_state_client, too 1 year ago
Vitaliy Filippov 57be1923d3 Daemonize NBD_DO_IT process, correctly cleanup unmounted NBD clients 1 year ago
Vitaliy Filippov c467acc388 Fix /v3 appendage to etcd URLs without /v3 1 year ago
Vitaliy Filippov bf591ba3ee Fix nbd module load check 1 year ago
Vitaliy Filippov 699a0fbbc7 Log to stderr instead of stdout in client 1 year ago
Vitaliy Filippov 6b2dd50f27 Fix build without RDMA 1 year ago
Vitaliy Filippov caf2f3c56f Release 0.6.3 1 year ago
Vitaliy Filippov 9174f188b1 Build packages with libibverbs 1 year ago
Vitaliy Filippov d3978c6d0e Do not print RDMA connection messages when log_level=0 1 year ago
Vitaliy Filippov 4a7365660d Do not wait for down OSDs during sync 1 year ago
Vitaliy Filippov 818ae5d61d Some config parsing fixes 1 year ago
Vitaliy Filippov 6810e93c3f Add RDMA options to mon.js list 1 year ago
Vitaliy Filippov f6f35f4127 Pass options correctly to not override /etc/vitastor/vitastor.conf 1 year ago
Vitaliy Filippov 72aa2fd819 Make OSD and client read common configuration from /etc/vitastor/vitastor.conf 1 year ago
Vitaliy Filippov 5010b0dd75 Use json11 instead of blockstore_config_t 1 year ago
Vitaliy Filippov 483c5ab380 Negotiate max_msg instead of max_sge, make buffer settings more conservative :-) 1 year ago
Vitaliy Filippov 6a6fd6544d Add RDMA options to the QEMU driver 1 year ago
Vitaliy Filippov 971aa4ae4f Implement RDMA receive with memory copying (send remains zero-copy) 1 year ago
Vitaliy Filippov 9e6cbc6ebc Negotiate max_sge between RDMA client & server 1 year ago
Vitaliy Filippov ce777319c3 WIP RDMA support 1 year ago
Vitaliy Filippov f8ff39b0ab Rework continue_ops() to remove a CPU hot spot 1 year ago
Vitaliy Filippov d749159585 Linked list experiment 1 year ago
Vitaliy Filippov 9703773a63 Fix has_flushes setting 1 year ago
Vitaliy Filippov 5d8d486f7c Add SOVERSION 1 year ago
Vitaliy Filippov 2b546cdd55 Link vitastor_blk with vitastor_common for timerfd_manager_t 1 year ago
Vitaliy Filippov bd7b177707 Report sensitive configuration values instead of the configuration source 1 year ago
Vitaliy Filippov 33f9d03d22 Update documentation regarding image names and vitastor-nbd 1 year ago
Vitaliy Filippov 82e6aff17b Support mapping NBD by the image name 1 year ago
Vitaliy Filippov 57e2c503f7 Rename osd_t::c_cli to msgr 1 year ago
Vitaliy Filippov 715bc8d53d Release 0.6.2 1 year ago
Vitaliy Filippov 0af077701c Fix a possible crash during SYNC when journal fsyncs are enabled 1 year ago
Vitaliy Filippov cac976ce25 Fix a memory leak in the chained read implementation 1 year ago
Vitaliy Filippov acf0646542 Build common sources once 1 year ago
Vitaliy Filippov ede1c1d667 Release 0.6.1 1 year ago
Vitaliy Filippov 38bd51c97f Remove aio_context assertion, it seems it is unneeded 1 year ago
Vitaliy Filippov 8c9f32cd45 Add run_vm test bash scripts 1 year ago
Vitaliy Filippov 966fb763ca Oooops, fix chained reads 1 year ago
Vitaliy Filippov 0b41ffc08d Release 0.6.0 1 year ago
Vitaliy Filippov 64eeb79051 Prevent 0.6.x OSDs from talking to 0.5.x 1 year ago
Vitaliy Filippov 2a02f3c4c7 Add metadata superblock and check it on start 1 year ago
Vitaliy Filippov f684d9101a Refuse to start with old journal version 1 year ago
Vitaliy Filippov c72fddd714 Notes about master/0.5.x 1 year ago
Vitaliy Filippov a1f2f19489 Do not increment inode statistics if the object already exists 1 year ago
Vitaliy Filippov 82c1a7ec67 Fix statistics reporting, split inode number into pool & inode 1 year ago
Vitaliy Filippov 2ab423d4ef Implement journaled write throttling for the SSD+HDD case 1 year ago
Vitaliy Filippov 4694811eab Add microsecond accuracy to set_timer 1 year ago
Vitaliy Filippov 6b988de17d Remove timerfd_interval 1 year ago
Vitaliy Filippov 37efdc2a83 Fix bitmap_set for replicated pools 1 year ago
Vitaliy Filippov 591cad09c9 Fix bitmaps for objects larger than 128K 1 year ago
Vitaliy Filippov b907ad50aa Oops, forgot to add external bitmaps to blockstore in some places 1 year ago
Vitaliy Filippov 7308d6a6c0 Note about etcd 3.4.15 1 year ago
Vitaliy Filippov 5f5b6ef150 Enable chained reads in the client 1 year ago
Vitaliy Filippov 38a3df4a0e Implement chained (optimized) read in the primary OSD code 1 year ago
Vitaliy Filippov 6950b8e3a0 Watch inode metadata revisions 1 year ago
Vitaliy Filippov 0cea3576fb Add "read bitmaps" operation to secondary OSD protocol 1 year ago
Vitaliy Filippov f01eea07d3 Add simplified interface to read blockstore bitmaps synchronously 1 year ago
Vitaliy Filippov 2c2f08aca2 Shorten some structure names 1 year ago
Vitaliy Filippov d6524670e1 Introduce data distribution locality 1 year ago
Vitaliy Filippov 879ecfa74d Fix wording 1 year ago
Vitaliy Filippov aea2d19d35 Change Telegram chat link 1 year ago
Vitaliy Filippov 04f86dc00b Fix Russian README for CMake build 1 year ago
Vitaliy Filippov 7aeb2cbac7 Capture all by value in qemu_proxy 1 year ago
Vitaliy Filippov 519f081006 Add LICENSE 1 year ago
Vitaliy Filippov e50f703e1d Add Russian version of the README 1 year ago
Vitaliy Filippov 2612d3198a Introduce image names and metadata storage in etcd 1 year ago
Vitaliy Filippov ab39ce2bbb Use clean_entry_bitmap_size instead of entry_attr_size back because of changed bitmap handling 1 year ago
Vitaliy Filippov d0c2e31312 Add a test for snapshots, fix bugs. Now the test passes 1 year ago
Vitaliy Filippov 9038d42327 Fix several snapshot I/O bugs 1 year ago
Vitaliy Filippov 691f066055 Actual snapshot support (untested) 1 year ago
Vitaliy Filippov ffe1cd4c79 Report inode I/O statistics, aggregate it in the monitor 1 year ago
Vitaliy Filippov 4ae1b84c67 Report inode space usage statistics to etcd, aggregate it in the monitor 1 year ago
Vitaliy Filippov c35963967f Add inode space usage statistics tracking to blockstore 1 year ago
Vitaliy Filippov 0aa2dd2890 Send bitmaps with primary-reads, actually read bitmaps for READ ops 1 year ago
Vitaliy Filippov 6bf88883ac Allocate bitmaps along with stripes to avoid memory fragmentation 1 year ago
Vitaliy Filippov 004f265393 Remove cryptic bitmap inlining from bs_op_t and osd_op_t, use bitmap in primary OSD code 1 year ago
Vitaliy Filippov 860ac24762 Add "external" bitmap support to the secondary OSD protocol 1 year ago
Vitaliy Filippov 6107a4d07b Add "external" bitmap support to blockstore 1 year ago
Vitaliy Filippov 95c29b9dc3 Add "external" bitmap support to osd_rmw 2 years ago
Vitaliy Filippov d99407dcec Check QEMU block-vitastor.so during the test 1 year ago
Vitaliy Filippov 6909807068 Allow to start the OSD just to flush the journal completely 1 year ago
Vitaliy Filippov ec90fe6ec1 Release 0.5.13 1 year ago
Vitaliy Filippov 18c72f4835 Correct reenterability fix (now verified with a test) 1 year ago
Vitaliy Filippov 59fbcef734 Release 0.5.12 1 year ago
Vitaliy Filippov 40b7c21fb1 Followup to 307c1731c1 - fix mark_stable 1 year ago
Vitaliy Filippov efb3678606 Fix qemu-img broken in 0.5.11 1 year ago
Vitaliy Filippov 462650134e Release 0.5.11 1 year ago
Vitaliy Filippov 8d87e32175 Fix msgr_op.h includes 1 year ago
Vitaliy Filippov b0b2e7df3c Fix use-after-free in keepalive_timer and rework stop_client() 1 year ago
Vitaliy Filippov 97efb9e299 Do not crash on PG re-peering events when operations are in progress 1 year ago
Vitaliy Filippov f6d705383a Fix client connection recovery bugs, add dirty_ops limit 1 year ago
Vitaliy Filippov 68567c0e1f Fix messenger possibly trying to connect to the same OSD twice 1 year ago
Vitaliy Filippov 04b00003e9 Log ping failures 1 year ago
Vitaliy Filippov 307c1731c1 Forget all dirty_entries before stable big_write or delete during initialisation 1 year ago
Vitaliy Filippov 75a6a556b5 Shuffle PGs for better data device utilisation 1 year ago
Vitaliy Filippov a48e2bbf18 Fix write replay ordering when immediate_commit != all 1 year ago
Vitaliy Filippov 688821665a Remove stoull_full() from etcd_state_client.cpp 1 year ago
Vitaliy Filippov 3e162d95a0 Remove http_client.h include from etcd_state_client.h 1 year ago
Vitaliy Filippov 829381b335 Extract some definitions to msgr_op.{cpp,h} 1 year ago
Vitaliy Filippov 54f2353f24 Use bitmap granularity for alignment checks 1 year ago
Vitaliy Filippov e47f6fba60 Remove cluster_client_t::stop() 1 year ago
Vitaliy Filippov 883bf84a16 Fix build 1 year ago
Vitaliy Filippov 52097c4856 Stop flushing when less than min_flusher_count operations are available (unless a trim is forced) 1 year ago
Vitaliy Filippov e1355cbc74 Report failed operation name in cluster_client 1 year ago
Vitaliy Filippov 8f8b90be7a Add min_flusher_count configuration 1 year ago
Vitaliy Filippov ad9f619370 Skip double allocs when reading journal 1 year ago
Vitaliy Filippov f4769ba7c7 Collapse create+delete journal entry pairs if they're already flushed 1 year ago
Vitaliy Filippov 843b7052d2 Add an assertion when clearing deleted metadata entries, add debug details when freeing blocks 1 year ago
Vitaliy Filippov df99e232ee Deduplicate osd_sets in pg history + raise request size limit for etcd 1 year ago
Vitaliy Filippov 3a40fa4127 Fix monitor errors in case of OSD removal 1 year ago
Vitaliy Filippov 4095bcc558 Do not ignore object deletion journal entries when they are preceded by a big write 1 year ago
Vitaliy Filippov 564d64e271 Add some details for debug prints 1 year ago
Vitaliy Filippov cf54741c95 Followup to 05db1308aa 2 years ago
Vitaliy Filippov 18a5fafa2a Fix rollback 1 year ago
Vitaliy Filippov 06f4978085 Fix fsync check in blockstore_flush (data fsyncs were disabled instead of journal fsyncs) 2 years ago
Vitaliy Filippov 7ebf1588c5 Check for immediate_commit==small in the OSD code 1 year ago
Vitaliy Filippov b0ad1e1e6d Remember writes as "unsynced" only after completing them 2 years ago
Vitaliy Filippov 0949f08407 Extract osd_primary write and sync code into separate files 1 year ago
Vitaliy Filippov 04a1f18fa5 Assign .req as a whole to always zero out the remaining part 1 year ago
Vitaliy Filippov cf9a641d66 Skip disconnected OSDs during sync 2 years ago
Vitaliy Filippov 05db1308aa Fix two potential read/write ordering problems (even though not yet seen in tests) 1 year ago
Vitaliy Filippov 98b54ca948 Don't try to "recover" misplaced objects if it would make them degraded 1 year ago
Vitaliy Filippov 23225c5e62 Do not run ping on clients that are not yet connected 1 year ago
Vitaliy Filippov 7e6e1a5a82 Release 0.5.10 1 year ago
Vitaliy Filippov 435045751d Delete objects only after a SYNC during rebalance in the non-immediate_commit mode 1 year ago
Vitaliy Filippov c5fb1d5987 Do not duplicate blockstore operations when io_uring fills up 1 year ago
Vitaliy Filippov 9f59381bea Re-distribute PG primaries over OSDs that come up after a short downtime 1 year ago
Vitaliy Filippov 9ac7e75178 Allow to specify etcd URLs for OSDs with http://, do not die with a strange error if -etcd option is missing for fio 1 year ago
Vitaliy Filippov 88671cf745 Fix a bug causing all flushers to wait for an fsync without actually trying to do it 1 year ago
Vitaliy Filippov fe1749c427 Fix the multiple_interrupted_rebalance test 1 year ago
Vitaliy Filippov ceb9c28de7 Set default log_level before passing config to etcd_state_client 1 year ago
Vitaliy Filippov 299d7d7c95 Use common macro for get_sqe 1 year ago
Vitaliy Filippov d1526b415f Correctly resume writes when OSD is full to return an error 1 year ago
Vitaliy Filippov f49fd53d55 Fix a bug where allocator was unable to allocate up to last (n%64) blocks, add tests for it 1 year ago
Vitaliy Filippov dd76eda5e5 Test multiple interrupted rebalancings 2 years ago
Vitaliy Filippov 87dbd8fa57 Use empty hash as the default value for some etcd keys in the monitor 2 years ago
Vitaliy Filippov b44f49aab2 Ignore zero OSDs in history osd_sets 2 years ago
Vitaliy Filippov 036555638e Release 0.5.9 1 year ago
Vitaliy Filippov af5155fcd9 Implement "no_recovery" and "no_rebalance" flags 1 year ago
Vitaliy Filippov 0d2efbecc9 Preserve previous PG history when changing PG distribution 1 year ago
Vitaliy Filippov e62e8b6bae Use real pg configuration instead of the "last clean" one for generating PG history 1 year ago
Vitaliy Filippov c4ba24c305 Do not print ping op latency 1 year ago
Vitaliy Filippov 19e47a0279 Release 0.5.8 2 years ago
Vitaliy Filippov bd178ac20f Fix history osd_set check - local OSD is always available! 1 year ago
Vitaliy Filippov 7006875a24 Make monitor stick to one etcd until the restart 1 year ago
Vitaliy Filippov ad577c4aac Add PING operation and timeouts to detect OSD failures when a host goes down 1 year ago
Vitaliy Filippov 836635c518 Use osd_out_time = 10 minutes by default 1 year ago
Vitaliy Filippov 88a03f4e98 Release 0.5.7 1 year ago
Vitaliy Filippov 2a5036669d Fix PG count change procedure 1 year ago
Vitaliy Filippov 2e0c853180 Make test_change_pg_count check if any objects are lost during the test 1 year ago
Vitaliy Filippov e91ff2a9ec Only forget offline PGs if their state is not changed during reporting 1 year ago
Vitaliy Filippov 086667f568 Do not check PG state key ownership if it doesn't exist yet 1 year ago
Vitaliy Filippov 73ce20e246 Add a test for the "reappear after move" case 1 year ago
Vitaliy Filippov 1be94da437 Check & remove extra chunks for degraded / incomplete objects, too 1 year ago
Vitaliy Filippov 80e12358a2 Use pg_data_size instead of pg_minsize for object state calculation 1 year ago
Vitaliy Filippov 36c935ace6 Use std::vector for the blockstore submission queue 1 year ago
Vitaliy Filippov 0d8b5e2ef9 Remove unused enqueue_op_first() 1 year ago
Vitaliy Filippov 98f1e2c277 Rework write/sync ordering 1 year ago
Vitaliy Filippov 21e7686037 Fix possible "assertion failed: pg.inflight >= 0" error during PG stop 1 year ago
Vitaliy Filippov ab21a1908b Check for the dirty PG flag when trying to continue to stop it after sync 1 year ago
Vitaliy Filippov 30d1ccd43e Fix an infinite loop when discarding list operations during stop_pg() 1 year ago
Vitaliy Filippov 8bdd6d8d78 Reset PG state when stopping them 1 year ago
Vitaliy Filippov 09b3e4e789 Fix OSDs being unable to stop PGs that are 'peering', not 'active' 1 year ago
Vitaliy Filippov 07912fd670 Use history/last_clean_pgs to avoid extra data move when observing a series of changes in the cluster 1 year ago
Vitaliy Filippov bc742ccf8c Fix a small memory leak in etcd_state_client 1 year ago
Vitaliy Filippov 314b20437b Do not break subsequent small writes badly when a big write is canceled 1 year ago
Vitaliy Filippov 29bac892ad Add .gitignore 1 year ago
Vitaliy Filippov cf7547faf3 Fix *.sh build scripts 1 year ago
Vitaliy Filippov ab90ed747f Release 0.5.6 1 year ago
Vitaliy Filippov 29d8ac8b1b Do not report statistics for the empty operation 1 year ago
Vitaliy Filippov 97795ea1b1 Use pg_minsize=2 in the pg_count change test 1 year ago
Vitaliy Filippov 24e7075f08 Fix monitor's statistics aggregation 1 year ago
Vitaliy Filippov 6155b23a7e Replace pgs[id] with pgs.at(id) to prevent accidental auto-vivification 1 year ago
Vitaliy Filippov 7d49706c07 Improve the pg_count change test: add more OSDs and actually move data between them 1 year ago
Vitaliy Filippov 46e79f3306 Wait for PGs to become clean before stopping them 1 year ago
Vitaliy Filippov 41fd14e024 Fix deletes not increasing write_iodepth 1 year ago
Vitaliy Filippov bb2d9a3afe Release 0.5.5 1 year ago
Vitaliy Filippov e899ed2c25 Make OSDs with 256 flushers (as they are now dynamic) 1 year ago
Vitaliy Filippov e21b14b72c Fix rpm specs for building with CMake 1 year ago
Vitaliy Filippov 5af8eddaa9 Add the remaining build script for Debian 1 year ago
Vitaliy Filippov 4f5a94c07a Modify instructions for the CMake build 1 year ago
Vitaliy Filippov e16b87ecc8 Rename random_combinations() parameter from "unordered" to "ordered" as it's more correct 1 year ago
Vitaliy Filippov fcb4aa0a11 Fix Monitor being unable to change PG sizes 1 year ago
Vitaliy Filippov 12adfa470c Add a test for changing PG size 1 year ago
Vitaliy Filippov 7f15e0c084 Add a simple test for the PG optimizer 1 year ago
Vitaliy Filippov 08d4bef419 Fix PG optimizer removing PGs without adding new ones 1 year ago
Vitaliy Filippov 2d73b19a6c Fix online PG count change bugs 1 year ago
Vitaliy Filippov 69c87009e9 Add a test for changing PG count 1 year ago
Vitaliy Filippov c974cb539c Make flusher_count adaptive and limit write iodepth 1 year ago
Vitaliy Filippov 00e98f64f3 A little better systemd unit generator 1 year ago
Vitaliy Filippov 91a70dfb1b Add a test for the no_same_sector_overwrites mode 1 year ago
Vitaliy Filippov 178388ac8c Use packages/ subdir instead of build/ for Docker package builds 1 year ago
Vitaliy Filippov bf9a175efc Move C/C++ sources to src subdirectory 1 year ago
Vitaliy Filippov 08aed962de Use CMake 1 year ago
Vitaliy Filippov 8c65e890b9 Slightly clean up the build script 1 year ago
Vitaliy Filippov 8cda70b889 Allow to enable AddressSanitizer with "ASAN=1 make" 1 year ago
Vitaliy Filippov 61ab22403a Use w=8 with jerasure 1 year ago
Vitaliy Filippov 16da663a66 Add another test for failure domains 1 year ago
Vitaliy Filippov 4a2dcf7b6b Update the license to VNPL 1.1 1 year ago
Vitaliy Filippov 8d48cc56b0 Generate randomly permutated OSD combinations when optimizing for compressed chunks 1 year ago
Vitaliy Filippov 9f58f01425 Mirror afr.js from /vitalif/ceph-afr-calc 1 year ago
Vitaliy Filippov b9e7d31aa1 Release v0.5.4 1 year ago
Vitaliy Filippov 2d9f09dcb6 Attempt forced trim when stopping an overrun flusher 1 year ago
Vitaliy Filippov 7cc59260c5 Fix no_same_sector_overwrites related bug 1 year ago
  1. 2
      .dockerignore
  2. 18
      .gitignore
  3. 7
      CMakeLists.txt
  4. 27
      LICENSE
  5. 46
      Make-gen.pl
  6. 195
      Makefile
  7. 100
      README-ru.md
  8. 496
      README.md
  9. 680
      VNPL-1.1-RU.txt
  10. 23
      VNPL-1.1.txt
  11. 765
      cluster_client.cpp
  12. 106
      cluster_client.h
  13. 2
      copy-fio-includes.sh
  14. 2
      copy-qemu-includes.sh
  15. 2
      cpp-btree
  16. 2
      csi/.dockerignore
  17. 32
      csi/Dockerfile
  18. 9
      csi/Makefile
  19. 5
      csi/deploy/000-csi-namespace.yaml
  20. 9
      csi/deploy/001-csi-config-map.yaml
  21. 37
      csi/deploy/002-csi-nodeplugin-rbac.yaml
  22. 72
      csi/deploy/003-csi-nodeplugin-psp.yaml
  23. 140
      csi/deploy/004-csi-nodeplugin.yaml
  24. 102
      csi/deploy/005-csi-provisioner-rbac.yaml
  25. 60
      csi/deploy/006-csi-provisioner-psp.yaml
  26. 159
      csi/deploy/007-csi-provisioner.yaml
  27. 11
      csi/deploy/008-csi-driver.yaml
  28. 19
      csi/deploy/009-storage-class.yaml
  29. 13
      csi/deploy/example-pvc-block.yaml
  30. 12
      csi/deploy/example-pvc.yaml
  31. 17
      csi/deploy/example-test-pod-block.yaml
  32. 17
      csi/deploy/example-test-pod.yaml
  33. 35
      csi/go.mod
  34. 448
      csi/go.sum
  35. 22
      csi/src/config.go
  36. 530
      csi/src/controllerserver.go
  37. 137
      csi/src/grpc.go
  38. 60
      csi/src/identityserver.go
  39. 293
      csi/src/nodeserver.go
  40. 36
      csi/src/server.go
  41. 39
      csi/vitastor-csi.go
  42. 7
      debian/build-vitastor-bullseye.sh
  43. 7
      debian/build-vitastor-buster.sh
  44. 28
      debian/changelog
  45. 42
      debian/control
  46. 13
      debian/copyright
  47. 1
      debian/fio_version
  48. 5
      debian/install
  49. 11
      debian/libisal.pc
  50. 40
      debian/libvirt.Dockerfile
  51. 58
      debian/patched-qemu.Dockerfile
  52. 1
      debian/pve-storage-vitastor.install
  53. 19
      debian/raw.h
  54. 3
      debian/rules
  55. 2
      debian/substvars
  56. 2
      debian/vitastor-client-dev.install
  57. 7
      debian/vitastor-client.install
  58. 1
      debian/vitastor-fio.install
  59. 1
      debian/vitastor-mon.install
  60. 3
      debian/vitastor-osd.install
  61. 64
      debian/vitastor.Dockerfile
  62. 9
      docker/Dockerfile
  63. 1
      docker/apt.conf
  64. BIN
      docker/vitastor.gpg
  65. 1
      docker/vitastor.list
  66. 3
      docker/vitastor.pref
  67. 37
      docs/config.en.md
  68. 39
      docs/config.ru.md
  69. 46
      docs/config/common.en.md
  70. 45
      docs/config/common.ru.md
  71. 32
      docs/config/inode.en.md
  72. 34
      docs/config/inode.ru.md
  73. 124
      docs/config/layout-cluster.en.md
  74. 134
      docs/config/layout-cluster.ru.md
  75. 176
      docs/config/layout-osd.en.md
  76. 185
      docs/config/layout-osd.ru.md
  77. 79
      docs/config/monitor.en.md
  78. 80
      docs/config/monitor.ru.md
  79. 214
      docs/config/network.en.md
  80. 224
      docs/config/network.ru.md
  81. 305
      docs/config/osd.en.md
  82. 318
      docs/config/osd.ru.md
  83. 257
      docs/config/pool.en.md
  84. 257
      docs/config/pool.ru.md
  85. 3
      docs/config/src/common.en.md
  86. 3
      docs/config/src/common.ru.md
  87. 35
      docs/config/src/common.yml
  88. 4
      docs/config/src/layout-cluster.en.md
  89. 4
      docs/config/src/layout-cluster.ru.md
  90. 200
      docs/config/src/layout-cluster.yml
  91. 4
      docs/config/src/layout-osd.en.md
  92. 5
      docs/config/src/layout-osd.ru.md
  93. 206
      docs/config/src/layout-osd.yml
  94. 120
      docs/config/src/make.js
  95. 3
      docs/config/src/monitor.en.md
  96. 3
      docs/config/src/monitor.ru.md
  97. 65
      docs/config/src/monitor.yml
  98. 4
      docs/config/src/network.en.md
  99. 4
      docs/config/src/network.ru.md
  100. 225
      docs/config/src/network.yml

2
.dockerignore

@ -1,5 +1,6 @@
.git
build
packages
mon/node_modules
*.o
*.so
@ -15,3 +16,4 @@ fio
qemu
rpm/*.Dockerfile
debian/*.Dockerfile
Dockerfile

18
.gitignore

@ -0,0 +1,18 @@
*.o
*.so
package-lock.json
fio
qemu
osd
stub_osd
stub_uring_osd
stub_bench
osd_test
osd_peering_pg_test
dump_journal
nbd_proxy
rm_inode
test_allocator
test_blockstore
test_shit
osd_rmw_test

7
CMakeLists.txt

@ -0,0 +1,7 @@
cmake_minimum_required(VERSION 2.8)
project(vitastor)
set(VERSION "0.7.1")
add_subdirectory(src)

27
LICENSE

@ -0,0 +1,27 @@
Copyright (c) Vitaliy Filippov (vitalif [at] yourcmc.ru), 2019+
All server-side code (OSD, Monitor and so on) is licensed under the terms of
Vitastor Network Public License 1.1 (VNPL 1.1), a copyleft license based on
GNU GPLv3.0 with the additional "Network Interaction" clause which requires
opensourcing all programs directly or indirectly interacting with Vitastor
through a computer network and expressly designed to be used in conjunction
with it ("Proxy Programs"). Proxy Programs may be made public not only under
the terms of the same license, but also under the terms of any GPL-Compatible
Free Software License, as listed by the Free Software Foundation.
This is a stricter copyleft license than the Affero GPL.
Please note that VNPL doesn't require you to open the code of proprietary
software running inside a VM if it's not specially designed to be used with
Vitastor.
Basically, you can't use the software in a proprietary environment to provide
its functionality to users without opensourcing all intermediary components
standing between the user and Vitastor or purchasing a commercial license
from the author 😀.
Client libraries (cluster_client and so on) are dual-licensed under the same
VNPL 1.1 and also GNU GPL 2.0 or later to allow for compatibility with GPLed
software like QEMU and fio.
You can find the full text of VNPL-1.1 in the file [VNPL-1.1.txt](VNPL-1.1.txt).
GPL 2.0 is also included in this repository as [GPL-2.0.txt](GPL-2.0.txt).

46
Make-gen.pl

@ -1,46 +0,0 @@
#!/usr/bin/perl
use strict;
my $deps = {};
for my $line (split /\n/, `grep '^#include "' *.cpp *.h`)
{
if ($line =~ /^([^:]+):\#include "([^"]+)"/s)
{
$deps->{$1}->{$2} = 1;
}
}
my $added;
do
{
$added = 0;
for my $file (keys %$deps)
{
for my $dep (keys %{$deps->{$file}})
{
if ($deps->{$dep})
{
for my $subdep (keys %{$deps->{$dep}})
{
if (!$deps->{$file}->{$subdep})
{
$added = 1;
$deps->{$file}->{$subdep} = 1;
}
}
}
}
}
} while ($added);
for my $file (sort keys %$deps)
{
if ($file =~ /\.cpp$/)
{
my $obj = $file;
$obj =~ s/\.cpp$/.o/s;
print "$obj: $file ".join(" ", sort keys %{$deps->{$file}})."\n";
print "\tg++ \$(CXXFLAGS) -c -o \$\@ \$\<\n";
}
}

195
Makefile

@ -1,195 +0,0 @@
BINDIR ?= /usr/bin
LIBDIR ?= /usr/lib/x86_64-linux-gnu
QEMU_PLUGINDIR ?= /usr/lib/x86_64-linux-gnu/qemu
BLOCKSTORE_OBJS := allocator.o blockstore.o blockstore_impl.o blockstore_init.o blockstore_open.o blockstore_journal.o blockstore_read.o \
blockstore_write.o blockstore_sync.o blockstore_stable.o blockstore_rollback.o blockstore_flush.o crc32c.o ringloop.o
# -fsanitize=address
CXXFLAGS := -g -O3 -Wall -Wno-sign-compare -Wno-comment -Wno-parentheses -Wno-pointer-arith -fPIC -fdiagnostics-color=always -I/usr/include/jerasure
all: libfio_blockstore.so osd libfio_sec_osd.so libfio_cluster.so stub_osd stub_uring_osd stub_bench osd_test dump_journal qemu_driver.so nbd_proxy rm_inode
clean:
rm -f *.o libblockstore.so libfio_blockstore.so osd libfio_sec_osd.so libfio_cluster.so stub_osd stub_uring_osd stub_bench osd_test dump_journal qemu_driver.so nbd_proxy rm_inode
install: all
mkdir -p $(DESTDIR)$(LIBDIR)/vitastor
install -m 0755 libfio_sec_osd.so $(DESTDIR)$(LIBDIR)/vitastor/
install -m 0755 libfio_cluster.so $(DESTDIR)$(LIBDIR)/vitastor/
install -m 0755 libfio_blockstore.so $(DESTDIR)$(LIBDIR)/vitastor/
install -m 0755 libblockstore.so $(DESTDIR)$(LIBDIR)/vitastor/
mkdir -p $(DESTDIR)$(BINDIR)
install -m 0755 osd $(DESTDIR)$(BINDIR)/vitastor-osd
install -m 0755 dump_journal $(DESTDIR)$(BINDIR)/vitastor-dump-journal
install -m 0755 nbd_proxy $(DESTDIR)$(BINDIR)/vitastor-nbd
install -m 0755 rm_inode $(DESTDIR)$(BINDIR)/vitastor-rm
mkdir -p $(DESTDIR)$(QEMU_PLUGINDIR)
install -m 0755 qemu_driver.so $(DESTDIR)$(QEMU_PLUGINDIR)/block-vitastor.so
dump_journal: dump_journal.cpp crc32c.o blockstore_journal.h
g++ $(CXXFLAGS) -o $@ $< crc32c.o
libblockstore.so: $(BLOCKSTORE_OBJS)
g++ $(CXXFLAGS) -o $@ -shared $(BLOCKSTORE_OBJS) -ltcmalloc_minimal -luring
libfio_blockstore.so: ./libblockstore.so fio_engine.o json11.o
g++ $(CXXFLAGS) -Wl,-rpath,'$(LIBDIR)/vitastor',-rpath,'$$ORIGIN' -shared -o $@ fio_engine.o json11.o libblockstore.so -ltcmalloc_minimal -luring
OSD_OBJS := osd.o osd_secondary.o msgr_receive.o msgr_send.o osd_peering.o osd_flush.o osd_peering_pg.o \
osd_primary.o osd_primary_subops.o etcd_state_client.o messenger.o osd_cluster.o http_client.o osd_ops.o pg_states.o \
osd_rmw.o json11.o base64.o timerfd_manager.o epoll_manager.o
osd: ./libblockstore.so osd_main.cpp osd.h osd_ops.h $(OSD_OBJS)
g++ $(CXXFLAGS) -Wl,-rpath,'$(LIBDIR)/vitastor',-rpath,'$$ORIGIN' -o $@ osd_main.cpp $(OSD_OBJS) libblockstore.so -ltcmalloc_minimal -luring -lJerasure
stub_osd: stub_osd.o rw_blocking.o
g++ $(CXXFLAGS) -o $@ stub_osd.o rw_blocking.o -ltcmalloc_minimal
osd_rmw_test: osd_rmw_test.o
g++ $(CXXFLAGS) -o $@ osd_rmw_test.o -lJerasure -fsanitize=address
STUB_URING_OSD_OBJS := stub_uring_osd.o epoll_manager.o messenger.o msgr_send.o msgr_receive.o ringloop.o timerfd_manager.o json11.o
stub_uring_osd: $(STUB_URING_OSD_OBJS)
g++ $(CXXFLAGS) -o $@ -ltcmalloc_minimal $(STUB_URING_OSD_OBJS) -luring
stub_bench: stub_bench.cpp osd_ops.h rw_blocking.o
g++ $(CXXFLAGS) -o $@ stub_bench.cpp rw_blocking.o -ltcmalloc_minimal
osd_test: osd_test.cpp osd_ops.h rw_blocking.o
g++ $(CXXFLAGS) -o $@ osd_test.cpp rw_blocking.o -ltcmalloc_minimal
osd_peering_pg_test: osd_peering_pg_test.cpp osd_peering_pg.o
g++ $(CXXFLAGS) -o $@ $< osd_peering_pg.o -ltcmalloc_minimal
libfio_sec_osd.so: fio_sec_osd.o rw_blocking.o
g++ $(CXXFLAGS) -ltcmalloc_minimal -shared -o $@ fio_sec_osd.o rw_blocking.o
FIO_CLUSTER_OBJS := cluster_client.o epoll_manager.o etcd_state_client.o \
messenger.o msgr_send.o msgr_receive.o ringloop.o json11.o http_client.o osd_ops.o pg_states.o timerfd_manager.o base64.o
libfio_cluster.so: fio_cluster.o $(FIO_CLUSTER_OBJS)
g++ $(CXXFLAGS) -ltcmalloc_minimal -shared -o $@ $< $(FIO_CLUSTER_OBJS) -luring
nbd_proxy: nbd_proxy.o $(FIO_CLUSTER_OBJS)
g++ $(CXXFLAGS) -ltcmalloc_minimal -o $@ $< $(FIO_CLUSTER_OBJS) -luring
rm_inode: rm_inode.o $(FIO_CLUSTER_OBJS)
g++ $(CXXFLAGS) -ltcmalloc_minimal -o $@ $< $(FIO_CLUSTER_OBJS) -luring
qemu_driver.o: qemu_driver.c qemu_proxy.h
gcc -I qemu/b/qemu `pkg-config glib-2.0 --cflags` \
-I qemu/include $(CXXFLAGS) -c -o $@ $<
qemu_driver.so: qemu_driver.o qemu_proxy.o $(FIO_CLUSTER_OBJS)
g++ $(CXXFLAGS) -ltcmalloc_minimal -shared -o $@ $(FIO_CLUSTER_OBJS) qemu_driver.o qemu_proxy.o -luring
test_blockstore: ./libblockstore.so test_blockstore.cpp timerfd_interval.o
g++ $(CXXFLAGS) -Wl,-rpath,'$(LIBDIR)/vitastor',-rpath,'$$ORIGIN' -o test_blockstore test_blockstore.cpp timerfd_interval.o libblockstore.so -ltcmalloc_minimal -luring
test_shit: test_shit.cpp osd_peering_pg.o
g++ $(CXXFLAGS) -o test_shit test_shit.cpp -luring -lm
test_allocator: test_allocator.cpp allocator.o
g++ $(CXXFLAGS) -o test_allocator test_allocator.cpp allocator.o
crc32c.o: crc32c.c crc32c.h
g++ $(CXXFLAGS) -c -o $@ $<
json11.o: json11/json11.cpp
g++ $(CXXFLAGS) -c -o json11.o json11/json11.cpp
# Autogenerated
allocator.o: allocator.cpp allocator.h
g++ $(CXXFLAGS) -c -o $@ $<
base64.o: base64.cpp base64.h
g++ $(CXXFLAGS) -c -o $@ $<
blockstore.o: blockstore.cpp allocator.h blockstore.h blockstore_flush.h blockstore_impl.h blockstore_init.h blockstore_journal.h cpp-btree/btree_map.h crc32c.h malloc_or_die.h object_id.h ringloop.h
g++ $(CXXFLAGS) -c -o $@ $<
blockstore_flush.o: blockstore_flush.cpp allocator.h blockstore.h blockstore_flush.h blockstore_impl.h blockstore_init.h blockstore_journal.h cpp-btree/btree_map.h crc32c.h malloc_or_die.h object_id.h ringloop.h
g++ $(CXXFLAGS) -c -o $@ $<
blockstore_impl.o: blockstore_impl.cpp allocator.h blockstore.h blockstore_flush.h blockstore_impl.h blockstore_init.h blockstore_journal.h cpp-btree/btree_map.h crc32c.h malloc_or_die.h object_id.h ringloop.h
g++ $(CXXFLAGS) -c -o $@ $<
blockstore_init.o: blockstore_init.cpp allocator.h blockstore.h blockstore_flush.h blockstore_impl.h blockstore_init.h blockstore_journal.h cpp-btree/btree_map.h crc32c.h malloc_or_die.h object_id.h ringloop.h
g++ $(CXXFLAGS) -c -o $@ $<
blockstore_journal.o: blockstore_journal.cpp allocator.h blockstore.h blockstore_flush.h blockstore_impl.h blockstore_init.h blockstore_journal.h cpp-btree/btree_map.h crc32c.h malloc_or_die.h object_id.h ringloop.h
g++ $(CXXFLAGS) -c -o $@ $<
blockstore_open.o: blockstore_open.cpp allocator.h blockstore.h blockstore_flush.h blockstore_impl.h blockstore_init.h blockstore_journal.h cpp-btree/btree_map.h crc32c.h malloc_or_die.h object_id.h ringloop.h
g++ $(CXXFLAGS) -c -o $@ $<
blockstore_read.o: blockstore_read.cpp allocator.h blockstore.h blockstore_flush.h blockstore_impl.h blockstore_init.h blockstore_journal.h cpp-btree/btree_map.h crc32c.h malloc_or_die.h object_id.h ringloop.h
g++ $(CXXFLAGS) -c -o $@ $<
blockstore_rollback.o: blockstore_rollback.cpp allocator.h blockstore.h blockstore_flush.h blockstore_impl.h blockstore_init.h blockstore_journal.h cpp-btree/btree_map.h crc32c.h malloc_or_die.h object_id.h ringloop.h
g++ $(CXXFLAGS) -c -o $@ $<
blockstore_stable.o: blockstore_stable.cpp allocator.h blockstore.h blockstore_flush.h blockstore_impl.h blockstore_init.h blockstore_journal.h cpp-btree/btree_map.h crc32c.h malloc_or_die.h object_id.h ringloop.h
g++ $(CXXFLAGS) -c -o $@ $<
blockstore_sync.o: blockstore_sync.cpp allocator.h blockstore.h blockstore_flush.h blockstore_impl.h blockstore_init.h blockstore_journal.h cpp-btree/btree_map.h crc32c.h malloc_or_die.h object_id.h ringloop.h
g++ $(CXXFLAGS) -c -o $@ $<
blockstore_write.o: blockstore_write.cpp allocator.h blockstore.h blockstore_flush.h blockstore_impl.h blockstore_init.h blockstore_journal.h cpp-btree/btree_map.h crc32c.h malloc_or_die.h object_id.h ringloop.h
g++ $(CXXFLAGS) -c -o $@ $<
cluster_client.o: cluster_client.cpp cluster_client.h etcd_state_client.h http_client.h json11/json11.hpp malloc_or_die.h messenger.h object_id.h osd_id.h osd_ops.h ringloop.h timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<
dump_journal.o: dump_journal.cpp allocator.h blockstore.h blockstore_flush.h blockstore_impl.h blockstore_init.h blockstore_journal.h cpp-btree/btree_map.h crc32c.h malloc_or_die.h object_id.h ringloop.h
g++ $(CXXFLAGS) -c -o $@ $<
epoll_manager.o: epoll_manager.cpp epoll_manager.h ringloop.h timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<
etcd_state_client.o: etcd_state_client.cpp base64.h etcd_state_client.h http_client.h json11/json11.hpp object_id.h osd_id.h osd_ops.h pg_states.h timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<
fio_cluster.o: fio_cluster.cpp cluster_client.h epoll_manager.h etcd_state_client.h fio/arch/arch.h fio/fio.h fio/optgroup.h fio_headers.h http_client.h json11/json11.hpp malloc_or_die.h messenger.h object_id.h osd_id.h osd_ops.h ringloop.h timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<
fio_engine.o: fio_engine.cpp blockstore.h fio/arch/arch.h fio/fio.h fio/optgroup.h fio_headers.h json11/json11.hpp object_id.h ringloop.h
g++ $(CXXFLAGS) -c -o $@ $<
fio_sec_osd.o: fio_sec_osd.cpp fio/arch/arch.h fio/fio.h fio/optgroup.h fio_headers.h object_id.h osd_id.h osd_ops.h rw_blocking.h
g++ $(CXXFLAGS) -c -o $@ $<
http_client.o: http_client.cpp http_client.h json11/json11.hpp timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<
messenger.o: messenger.cpp json11/json11.hpp malloc_or_die.h messenger.h object_id.h osd_id.h osd_ops.h ringloop.h timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<
msgr_receive.o: msgr_receive.cpp json11/json11.hpp malloc_or_die.h messenger.h object_id.h osd_id.h osd_ops.h ringloop.h timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<
msgr_send.o: msgr_send.cpp json11/json11.hpp malloc_or_die.h messenger.h object_id.h osd_id.h osd_ops.h ringloop.h timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<
nbd_proxy.o: nbd_proxy.cpp cluster_client.h epoll_manager.h etcd_state_client.h http_client.h json11/json11.hpp malloc_or_die.h messenger.h object_id.h osd_id.h osd_ops.h ringloop.h timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<
osd.o: osd.cpp blockstore.h cpp-btree/btree_map.h epoll_manager.h etcd_state_client.h http_client.h json11/json11.hpp malloc_or_die.h messenger.h object_id.h osd.h osd_id.h osd_ops.h osd_peering_pg.h pg_states.h ringloop.h timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<
osd_cluster.o: osd_cluster.cpp base64.h blockstore.h cpp-btree/btree_map.h epoll_manager.h etcd_state_client.h http_client.h json11/json11.hpp malloc_or_die.h messenger.h object_id.h osd.h osd_id.h osd_ops.h osd_peering_pg.h pg_states.h ringloop.h timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<
osd_flush.o: osd_flush.cpp blockstore.h cpp-btree/btree_map.h epoll_manager.h etcd_state_client.h http_client.h json11/json11.hpp malloc_or_die.h messenger.h object_id.h osd.h osd_id.h osd_ops.h osd_peering_pg.h pg_states.h ringloop.h timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<
osd_main.o: osd_main.cpp blockstore.h cpp-btree/btree_map.h epoll_manager.h etcd_state_client.h http_client.h json11/json11.hpp malloc_or_die.h messenger.h object_id.h osd.h osd_id.h osd_ops.h osd_peering_pg.h pg_states.h ringloop.h timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<
osd_ops.o: osd_ops.cpp object_id.h osd_id.h osd_ops.h
g++ $(CXXFLAGS) -c -o $@ $<
osd_peering.o: osd_peering.cpp base64.h blockstore.h cpp-btree/btree_map.h epoll_manager.h etcd_state_client.h http_client.h json11/json11.hpp malloc_or_die.h messenger.h object_id.h osd.h osd_id.h osd_ops.h osd_peering_pg.h pg_states.h ringloop.h timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<
osd_peering_pg.o: osd_peering_pg.cpp cpp-btree/btree_map.h object_id.h osd_id.h osd_ops.h osd_peering_pg.h pg_states.h
g++ $(CXXFLAGS) -c -o $@ $<
osd_peering_pg_test.o: osd_peering_pg_test.cpp cpp-btree/btree_map.h object_id.h osd_id.h osd_ops.h osd_peering_pg.h pg_states.h
g++ $(CXXFLAGS) -c -o $@ $<
osd_primary.o: osd_primary.cpp blockstore.h cpp-btree/btree_map.h epoll_manager.h etcd_state_client.h http_client.h json11/json11.hpp malloc_or_die.h messenger.h object_id.h osd.h osd_id.h osd_ops.h osd_peering_pg.h osd_primary.h osd_rmw.h pg_states.h ringloop.h timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<
osd_primary_subops.o: osd_primary_subops.cpp blockstore.h cpp-btree/btree_map.h epoll_manager.h etcd_state_client.h http_client.h json11/json11.hpp malloc_or_die.h messenger.h object_id.h osd.h osd_id.h osd_ops.h osd_peering_pg.h osd_primary.h osd_rmw.h pg_states.h ringloop.h timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<
osd_rmw.o: osd_rmw.cpp malloc_or_die.h object_id.h osd_id.h osd_rmw.h xor.h
g++ $(CXXFLAGS) -c -o $@ $<
osd_rmw_test.o: osd_rmw_test.cpp malloc_or_die.h object_id.h osd_id.h osd_rmw.cpp osd_rmw.h test_pattern.h xor.h
g++ $(CXXFLAGS) -c -o $@ $<
osd_secondary.o: osd_secondary.cpp blockstore.h cpp-btree/btree_map.h epoll_manager.h etcd_state_client.h http_client.h json11/json11.hpp malloc_or_die.h messenger.h object_id.h osd.h osd_id.h osd_ops.h osd_peering_pg.h pg_states.h ringloop.h timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<
osd_test.o: osd_test.cpp object_id.h osd_id.h osd_ops.h rw_blocking.h test_pattern.h
g++ $(CXXFLAGS) -c -o $@ $<
pg_states.o: pg_states.cpp pg_states.h
g++ $(CXXFLAGS) -c -o $@ $<
qemu_proxy.o: qemu_proxy.cpp cluster_client.h etcd_state_client.h http_client.h json11/json11.hpp malloc_or_die.h messenger.h object_id.h osd_id.h osd_ops.h qemu_proxy.h ringloop.h timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<
ringloop.o: ringloop.cpp ringloop.h
g++ $(CXXFLAGS) -c -o $@ $<
rm_inode.o: rm_inode.cpp cluster_client.h etcd_state_client.h http_client.h json11/json11.hpp malloc_or_die.h messenger.h object_id.h osd_id.h osd_ops.h ringloop.h timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<
rw_blocking.o: rw_blocking.cpp rw_blocking.h
g++ $(CXXFLAGS) -c -o $@ $<
stub_bench.o: stub_bench.cpp object_id.h osd_id.h osd_ops.h rw_blocking.h
g++ $(CXXFLAGS) -c -o $@ $<
stub_osd.o: stub_osd.cpp object_id.h osd_id.h osd_ops.h rw_blocking.h
g++ $(CXXFLAGS) -c -o $@ $<
stub_uring_osd.o: stub_uring_osd.cpp epoll_manager.h json11/json11.hpp malloc_or_die.h messenger.h object_id.h osd_id.h osd_ops.h ringloop.h timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<
test_allocator.o: test_allocator.cpp allocator.h
g++ $(CXXFLAGS) -c -o $@ $<
test_blockstore.o: test_blockstore.cpp blockstore.h object_id.h ringloop.h timerfd_interval.h
g++ $(CXXFLAGS) -c -o $@ $<
test_shit.o: test_shit.cpp allocator.h blockstore.h blockstore_flush.h blockstore_impl.h blockstore_init.h blockstore_journal.h cpp-btree/btree_map.h crc32c.h malloc_or_die.h object_id.h osd_id.h osd_ops.h osd_peering_pg.h pg_states.h ringloop.h
g++ $(CXXFLAGS) -c -o $@ $<
timerfd_interval.o: timerfd_interval.cpp ringloop.h timerfd_interval.h
g++ $(CXXFLAGS) -c -o $@ $<
timerfd_manager.o: timerfd_manager.cpp timerfd_manager.h
g++ $(CXXFLAGS) -c -o $@ $<

100
README-ru.md

@ -0,0 +1,100 @@
## Vitastor
[Read English version](README.md)
## Идея
Вернём былую скорость кластерному блочному хранилищу!
Vitastor - распределённая блочная SDS (программная СХД), прямой аналог Ceph RBD и
внутренних СХД популярных облачных провайдеров. Однако, в отличие от них, Vitastor
быстрый и при этом простой. Только пока маленький :-).
Vitastor архитектурно похож на Ceph, что означает атомарность и строгую консистентность,
репликацию через первичный OSD, симметричную кластеризацию без единой точки отказа
и автоматическое распределение данных по любому числу дисков любого размера с настраиваемыми схемами
избыточности - репликацией или с произвольными кодами коррекции ошибок.
Vitastor нацелен на SSD и SSD+HDD кластеры с как минимум 10 Гбит/с сетью, поддерживает
TCP и RDMA и на хорошем железе может достигать задержки 4 КБ чтения и записи на уровне ~0.1 мс,
что примерно в 10 раз быстрее, чем Ceph и другие популярные программные СХД.
Vitastor поддерживает QEMU-драйвер, протоколы NBD и NFS, драйверы OpenStack, Proxmox, Kubernetes.
Другие драйверы могут также быть легко реализованы.
Подробности смотрите в документации по ссылкам ниже.
## Презентации и записи докладов
- DevOpsConf'2021: презентация ([на русском](https://vitastor.io/presentation/devopsconf/devopsconf.html),
[на английском](https://vitastor.io/presentation/devopsconf/devopsconf_en.html)),
[видео](https://vitastor.io/presentation/devopsconf/talk.webm)
- Highload'2022: презентация ([на русском](https://vitastor.io/presentation/highload/highload.html)),
[видео](https://vitastor.io/presentation/highload/talk.webm)