vitalif
/
era_copy
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Notes, systemd unit

master
Vitaliy Filippov 2021-10-25 14:30:17 +03:00
parent 50ec364f5b
commit 5af1368bb6
2 changed files with 42 additions and 19 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

49
README.md
View File

@ -3,6 +3,9 @@
dm-era is a Device Mapper target that acts as a proxy to an existing block device, like dm-linear, dm-era is a Device Mapper target that acts as a proxy to an existing block device, like dm-linear,
but also keeps track of which blocks were written to. It is included in mainline Linux kernel since 3.15. but also keeps track of which blocks were written to. It is included in mainline Linux kernel since 3.15.
Update: dm-era seems to be unsafe in kernel versions before 5.12 because of
[https://www.spinics.net/lists/dm-devel/msg45023.html](two bugs) which were only fixed in 5.12.
# era_copy and era_apply # era_copy and era_apply
era_copy parses dm-era metadata from `era_invalidate` output and saves changed blocks into a stream era_copy parses dm-era metadata from `era_invalidate` output and saves changed blocks into a stream
@ -15,34 +18,40 @@ With dm-era and these two small utilities you can perform incremental backups of
without the need for a COW FS, LVM or a storage hypervisor. dm-era almost doesn't hurt performance without the need for a COW FS, LVM or a storage hypervisor. dm-era almost doesn't hurt performance
and seems to handle fsyncs correctly. and seems to handle fsyncs correctly.
# How to try dm-era for incremental backups (Debian, GRUB) It's not a one-click solution, but it works :-)
1. Setup a small partition for dm-era metadata. We'll use 1024 block (512 KB) granularity # How to try dm-era for incremental backups
1. Install dm-era tools (`era_invalidate`) with `apt-get install thin-provisioning-tools` on your target host.
2. Install `era_copy` and `era_apply` (`make install` from this repository) on both hosts (target host and backup host).
3. Setup a small partition for dm-era metadata. We'll use 1024 block (512 KB) granularity
so a bitmap for 1 TB device will only take 8 MB on disk. dm-era keeps slightly more than one bitmap so a bitmap for 1 TB device will only take 8 MB on disk. dm-era keeps slightly more than one bitmap
on disk at a time, but anyway, a 512 MB or 1 GB partition will be more than enough. on disk at a time, but anyway, a 512 MB or 1 GB partition will be more than enough.
For example you can shrink the main partition a bit and add the dm-era partition after it. For example you can shrink the main partition a bit and add the dm-era partition after it.
2. Zero out the new metadata partition: `dd if=/dev/zero of=<META_PARTITION> bs=1048576` 4. Zero out the new metadata partition: `dd if=/dev/zero of=<META_PARTITION> bs=1048576`
3. Copy `zz_dm-era.sh` to `/etc/initramfs-tools/scripts/local-block` 5. Copy `dm-era.service` to `/etc/systemd/system` and edit the ExecStart command.
and adjust `DATA_DEVICE`, `META_DEVICE` and `ERA_DEVICE_NAME` in it. The syntax is: `/sbin/dmsetup create ERA_DEVICE_NAME --table "0 SIZE_IN_SECTORS era METADATA_DEVICE DATA_DEVICE 1024"`.
Use partition IDs (`/dev/disk/by-partuuid/*` for GPT partitions, `/dev/disk/by-id/md-uuid-*` for mdadm, etc) 6. Enable the unit: `systemctl enable dm-era`.
to be safe because dm-era doesn't check if you supply correct partitions to it. 7. Change the desired /etc/fstab entry and reboot (or unmount the partition, stop services, mount it back and start services).
4. Add `dm_era` to `/etc/initramfs-tools/modules`. 8. Do an initial full partition backup with block-level copy. For example, to copy an ext4 filesystem
5. Edit `/etc/fstab` and change your actual device to `/dev/mapper/<ERA_DEVICE_NAME>`,
for example `/dev/mapper/root_era`.
6. Repeat it for more partitions if you want.
7. Run `update-initramfs -u -k all`.
8. If you do it for the root partition also change `/etc/default/grub`:
`GRUB_CMDLINE_LINUX="root=/dev/mapper/root_era"` and refresh grub config with `update-grub`.
9. Reboot.
10. Install dm-era tools (`era_invalidate`) with `apt-get install thin-provisioning-tools` on your target host.
11. Install `era_copy` and `era_apply` (`make install` from this repository) on both hosts (target host and backup host).
12. Do an initial full partition backup with block-level copy. For example, to copy an ext4 filesystem
to another host over ssh run: `ssh root@host "e2image -f -p -ra /dev/mapper/root_era - | gzip" | gzip -d | cp --sparse=always /dev/stdin rootfs.bin`. to another host over ssh run: `ssh root@host "e2image -f -p -ra /dev/mapper/root_era - | gzip" | gzip -d | cp --sparse=always /dev/stdin rootfs.bin`.
13. Now you can use `backup.sh` to perform incremental backups of the dm-era device 9. Now you can use `backup.sh` to perform incremental backups of the dm-era device
over ssh from the backup host. Just change variables at the top of the script so it matches over ssh from the backup host. Just change variables at the top of the script so it matches
your device configuration. your device configuration.
It's not a one-click solution, but it works :-) # dm-era on root partition (Debian, GRUB)
1. Copy `zz_dm-era.sh` to `/etc/initramfs-tools/scripts/local-block`
and adjust `DATA_DEVICE`, `META_DEVICE` and `ERA_DEVICE_NAME` in it.
Use partition IDs (`/dev/disk/by-partuuid/*` for GPT partitions, `/dev/disk/by-id/md-uuid-*` for mdadm, etc)
to be safe because dm-era doesn't check if you supply correct partitions to it.
2. Add `dm_era` to `/etc/initramfs-tools/modules`.
3. Edit `/etc/fstab` and change your actual device to `/dev/mapper/<ERA_DEVICE_NAME>`,
for example `/dev/mapper/root_era`.
4. Run `update-initramfs -u -k all`.
5. Change `/etc/default/grub`: set
`GRUB_CMDLINE_LINUX="root=/dev/mapper/root_era"` and refresh grub config with `update-grub`.
6. Reboot.
# Author and license # Author and license

14
dm-era.service Normal file
View File

@ -0,0 +1,14 @@
[Unit]
Description=dm-era for /var
DefaultDependencies=no
StartLimitInterval=0
Before=local-fs-pre.target
DefaultDependencies=no
Conflicts=shutdown.target
[Service]
Type=oneshot
ExecStart=/sbin/modprobe dm_era ; /sbin/dmsetup create md2p4_era --table "0 3625547743 era /dev/md2p3 /dev/md2p4 1024"
[Install]
WantedBy=sysinit.target