vitastor/docs/usage/cli.en.md

[Documentation](../../README.md#documentation) → Usage → Vitastor CLI

-----

[Читать на русском](cli.ru.md)

# Vitastor CLI

vitastor-cli is a command-line tool for administrative tasks like image management.

It supports the following commands:

- [status](#status)
- [df](#df)
- [ls](#ls)
- [create](#create)
- [snap-create](#create)
- [modify](#modify)
- [rm](#rm)
- [flatten](#flatten)
- [rm-data](#rm-data)
- [merge-data](#merge-data)
- [describe](#describe)
- [fix](#fix)
- [alloc-osd](#alloc-osd)
- [rm-osd](#rm-osd)
- [create-pool](#create-pool)
- [modify-pool](#modify-pool)
- [ls-pools](#ls-pools)
- [rm-pool](#rm-pool)

Global options:

```
--config_file FILE   Path to Vitastor configuration file
--etcd_address URL   Etcd connection address
--iodepth N          Send N operations in parallel to each OSD when possible (default 32)
--parallel_osds M    Work with M osds in parallel when possible (default 4)
--progress 1|0       Report progress (default 1)
--cas 1|0            Use CAS writes for flatten, merge, rm (default is decide automatically)
--no-color           Disable colored output
--json               JSON output
```

## status

`vitastor-cli status`

Show cluster status.

Example output:

```
  cluster:
    etcd: 1 / 1 up, 1.8 M database size
    mon:  1 up, master stump
    osd:  8 / 12 up

  data:
    raw:   498.5 G used, 301.2 G / 799.7 G available, 399.8 G down
    state: 156.6 G clean, 97.6 G misplaced
    pools: 2 / 3 active
    pgs:   30 active
           34 active+has_misplaced
           32 offline

  io:
    client:    0 B/s rd, 0 op/s rd, 0 B/s wr, 0 op/s wr
    rebalance: 989.8 M/s, 7.9 K op/s
```

## df

`vitastor-cli df`

Show pool space statistics.

Example output:

```
NAME      SCHEME  PGS  TOTAL    USED    AVAILABLE  USED%   EFFICIENCY
testpool  2/1     32   100 G    34.2 G  60.7 G     39.23%  100%
size1     1/1     32   199.9 G  10 G    121.5 G    39.23%  100%
kaveri    2/1     32   0 B      10 G    0 B        100%    0%
```

In the example above, "kaveri" pool has "zero" efficiency because all its OSD are down.

## ls

`vitastor-cli ls [-l] [-p POOL] [--sort FIELD] [-r] [-n N] [<glob> ...]`

List images (only matching `<glob>` pattern(s) if passed).

Options:

```
-p|--pool POOL  Filter images by pool ID or name
-l|--long       Also report allocated size and I/O statistics
--del           Also include delete operation statistics
--sort FIELD    Sort by specified field (name, size, used_size, <read|write|delete>_<iops|bps|lat|queue>)
-r|--reverse    Sort in descending order
-n|--count N    Only list first N items
```

Example output:

```
NAME                 POOL      SIZE  USED    READ   IOPS  QUEUE  LAT   WRITE  IOPS  QUEUE  LAT   FLAGS  PARENT
debian9              testpool  20 G  12.3 G  0 B/s  0     0      0 us  0 B/s  0     0      0 us     RO
pve/vm-100-disk-0    testpool  20 G  0 B     0 B/s  0     0      0 us  0 B/s  0     0      0 us      -  debian9
pve/base-101-disk-0  testpool  20 G  0 B     0 B/s  0     0      0 us  0 B/s  0     0      0 us     RO  debian9
pve/vm-102-disk-0    testpool  32 G  36.4 M  0 B/s  0     0      0 us  0 B/s  0     0      0 us      -  pve/base-101-disk-0
debian9-test         testpool  20 G  36.6 M  0 B/s  0     0      0 us  0 B/s  0     0      0 us      -  debian9
bench                testpool  10 G  10 G    0 B/s  0     0      0 us  0 B/s  0     0      0 us      -
bench-kaveri         kaveri    10 G  10 G    0 B/s  0     0      0 us  0 B/s  0     0      0 us      -
```

## create

`vitastor-cli create -s|--size <size> [-p|--pool <id|name>] [--parent <parent_name>[@<snapshot>]] <name>`

Create an image. You may use K/M/G/T suffixes for `<size>`. If `--parent` is specified,
a copy-on-write image clone is created. Parent must be a snapshot (readonly image).
Pool must be specified if there is more than one pool.

```
vitastor-cli create --snapshot <snapshot> [-p|--pool <id|name>] <image>
vitastor-cli snap-create [-p|--pool <id|name>] <image>@<snapshot>
```

Create a snapshot of image `<name>` (either form can be used). May be used live if only a single writer is active.

See also about [how to export snapshots](qemu.en.md#exporting-snapshots).

## modify

`vitastor-cli modify <name> [--rename <new-name>] [--resize <size>] [--readonly | --readwrite] [-f|--force] [--down-ok]`

Rename, resize image or change its readonly status. Images with children can't be made read-write.
If the new size is smaller than the old size, extra data will be purged.
You should resize file system in the image, if present, before shrinking it.

* `-f|--force` - Proceed with shrinking or setting readwrite flag even if the image has children.
* `--down-ok` - Proceed with shrinking even if some data will be left on unavailable OSDs.

## rm

`vitastor-cli rm <from> [<to>] [--writers-stopped] [--down-ok]`

Remove `<from>` or all layers between `<from>` and `<to>` (`<to>` must be a child of `<from>`),
rebasing all their children accordingly. --writers-stopped allows merging to be a bit
more effective in case of a single 'slim' read-write child and 'fat' removed parent:
the child is merged into parent and parent is renamed to child in that case.
In other cases parent layers are always merged into children.

Other options:

* `--down-ok` - Continue deletion/merging even if some data will be left on unavailable OSDs.

## flatten

`vitastor-cli flatten <layer>`

Flatten a layer, i.e. merge data and detach it from parents.

## rm-data

`vitastor-cli rm-data --pool <pool> --inode <inode> [--wait-list] [--min-offset <offset>]`

Remove inode data without changing metadata.

```
--wait-list   Retrieve full objects listings before starting to remove objects.
              Requires more memory, but allows to show correct removal progress.
--min-offset  Purge only data starting with specified offset.
```

## merge-data

`vitastor-cli merge-data <from> <to> [--target <target>]`

Merge layer data without changing metadata. Merge `<from>`..`<to>` to `<target>`.
`<to>` must be a child of `<from>` and `<target>` may be one of the layers between
`<from>` and `<to>`, including `<from>` and `<to>`.

## describe

`vitastor-cli describe [OPTIONS]`

Describe unclean object locations in the cluster. Options:

```
--osds <osds>
    Only list objects from primary OSD(s) <osds>.
--object-state <states>
    Only list objects in given state(s). State(s) may include:
    degraded, misplaced, incomplete, corrupted, inconsistent.
--pool <pool name or number>
    Only list objects in the given pool.
--pg <pg number>
    Only list objects in the given PG of the pool.
--inode, --min-inode, --max-inode
    Restrict listing to specific inode numbers.
--min-offset, --max-offset
    Restrict listing to specific offsets inside inodes.
```

## fix

`vitastor-cli fix [--objects <objects>] [--bad-osds <osds>] [--part <part>] [--check no]`

Fix inconsistent objects in the cluster by deleting some copies.

```
--objects <objects>
    Objects to fix, either in plain text or JSON format. If not specified,
    object list will be read from STDIN in one of the same formats.
    Plain text format: 0x<inode>:0x<stripe> <any delimiter> 0x<inode>:0x<stripe> ...
    JSON format: [{"inode":"0x...","stripe":"0x..."},...]
--bad-osds <osds>
    Remove inconsistent copies/parts of objects from these OSDs, effectively
    marking them bad and allowing Vitastor to recover objects from other copies.
--part <number>
    Only remove EC part <number> (from 0 to pg_size-1), required for extreme
    edge cases where one OSD has multiple parts of a EC object.
--check no
    Do not recheck that requested objects are actually inconsistent,
    delete requested copies/parts anyway.
```

## alloc-osd

`vitastor-cli alloc-osd`

Allocate a new OSD number and reserve it by creating empty `/osd/stats/<n>` key.

## rm-osd

`vitastor-cli rm-osd [--force] [--allow-data-loss] [--dry-run] <osd_id> [osd_id...]`

Remove metadata and configuration for specified OSD(s) from etcd.

Refuses to remove OSDs with data without `--force` and `--allow-data-loss`.

With `--dry-run` only checks if deletion is possible without data loss and
redundancy degradation.

## create-pool

`vitastor-cli create-pool|pool-create <name> (-s <pg_size>|--ec <N>+<K>) -n <pg_count> [OPTIONS]`

Create a pool. Required parameters:

| <!-- -->                 | <!-- -->                                                                              |
|--------------------------|---------------------------------------------------------------------------------------|
| `-s R` or `--pg_size R`  | Number of replicas for replicated pools                                               |
| `--ec N+K`               | Number of data (N) and parity (K) chunks for erasure-coded pools                      |
| `-n N` or `--pg_count N` | PG count for the new pool (start with 10*<OSD count>/pg_size rounded to a power of 2) |

Optional parameters:

| <!-- -->                       | <!-- -->                                                                   |
|--------------------------------|----------------------------------------------------------------------------|
| `--pg_minsize <number>`        | R or N+K minus number of failures to tolerate without downtime ([details](../config/pool.en.md#pg_minsize)) |
| `--failure_domain host`        | Failure domain: host, osd or a level from placement_levels. Default: host  |
| `--root_node <node>`           | Put pool only on child OSDs of this placement tree node                    |
| `--osd_tags <tag>[,<tag>]...`  | Put pool only on OSDs tagged with all specified tags                       |
| `--block_size 128k`            | Put pool only on OSDs with this data block size                            |
| `--bitmap_granularity 4k`      | Put pool only on OSDs with this logical sector size                        |
| `--immediate_commit none`      | Put pool only on OSDs with this or larger immediate_commit (none < small < all) |
| `--level_placement <rules>`    | Use additional failure domain rules (example: "dc=112233")                 |
| `--raw_placement <rules>`      | Specify raw PG generation rules ([details](../config/pool.en.md#raw_placement)) |
| `--primary_affinity_tags tags` | Prefer to put primary copies on OSDs with all specified tags               |
| `--scrub_interval <time>`      | Enable regular scrubbing for this pool. Format: number + unit s/m/h/d/M/y  |
| `--used_for_fs <name>`         | Mark pool as used for VitastorFS with metadata in image <name>             |
| `--pg_stripe_size <number>`    | Increase object grouping stripe                                            |
| `--max_osd_combinations 10000` | Maximum number of random combinations for LP solver input                  |
| `--wait`                       | Wait for the new pool to come online                                       |
| `-f` or `--force`              | Do not check that cluster has enough OSDs to create the pool               |

See also [Pool configuration](../config/pool.en.md) for detailed parameter descriptions.

Examples:

`vitastor-cli create-pool test_x4 -s 4 -n 32`

`vitastor-cli create-pool test_ec42 --ec 4+2 -n 32`

## modify-pool

`vitastor-cli modify-pool|pool-modify <id|name> [--name <new_name>] [PARAMETERS...]`

Modify an existing pool. Modifiable parameters:

```
[-s|--pg_size <number>] [--pg_minsize <number>] [-n|--pg_count <count>]
[--failure_domain <level>] [--root_node <node>] [--osd_tags <tags>] [--no_inode_stats 0|1]
[--max_osd_combinations <number>] [--primary_affinity_tags <tags>] [--scrub_interval <time>]
```

Non-modifiable parameters (changing them WILL lead to data loss):

```
[--block_size <size>] [--bitmap_granularity <size>]
[--immediate_commit <all|small|none>] [--pg_stripe_size <size>]
```

These, however, can still be modified with -f|--force.

See [create-pool](#create-pool) for parameter descriptions.

Examples:

`vitastor-cli modify-pool pool_A --name pool_B`

`vitastor-cli modify-pool 2 --pg_size 4 -n 128`

## rm-pool

`vitastor-cli rm-pool|pool-rm [--force] <id|name>`

Remove a pool. Refuses to remove pools with images without `--force`.

## ls-pools

`vitastor-cli ls-pools|pool-ls|ls-pool|pools [-l] [--detail] [--sort FIELD] [-r] [-n N] [--stats] [<glob> ...]`

List pools (only matching <glob> patterns if passed).

| <!-- -->             | <!-- -->                                              |
|----------------------|-------------------------------------------------------|
| `-l` or `--long`     | Also report I/O statistics                            |
| `--detail`           | Use list format (not table), show all details         |
| `--sort FIELD`       | Sort by specified field (see fields in --json output) |
| `-r` or `--reverse`  | Sort in descending order                              |
| `-n` or `--count N`  | Only list first N items                               |