Getting to know Logical Volume Manager on Ubuntu Precise
Notes
You can use LVM for /boot, /, and any other filesystem. GRUB in lucid and later supports booting from LVM.
You don't need to partition disks to use them with LVM. In fact, this is usually just annoying and not useful. Use pvcreate directly on the block devices.
Directly using the block devices also overcomes the MBR format's limit of 2.1 terabytes. LVM supports PVs on block devices up to who knows how big.
-
For 32-bit CPUs on 2.6 kernels, the maximum LV size is 16TB. That'll do you.
-
For 64-bit CPUs on 2.6 kernels, the maximum LV size is 8EB.
See also:
TLDP: http://tldp.org/HOWTO/LVM-HOWTO/lvm2faq.html#AEN407
Now, let's start playing around with LVM.
Setup a KVM with some extra block devices to use as an LVM sandbox
I've installed 64-bit Precise Server into a VM called precise-x64. I've added the block devices as SCSI devices
#!/bin/bash
VM="precise-x64"
for i in {a..f}; do
sudo qemu-img create -f qcow2 /var/lib/libvirt/images/${VM}_vd${i}.img 4G
sudo virsh attach-disk ${VM} /var/lib/libvirt/images/${VM}_vd${i}.img vd${i} --type disk --subdriver qcow2 --persistent
done
sudo virsh start ${VM}
#EOF
-
Setting up LVM inside the VM
$ sudo vgcreate vg_lvm_test /dev/vd{a..f} No physical volume label read from /dev/vda Physical volume "/dev/vda" successfully created [...] Volume group "vg_lvm_test" successfully created $ sudo lvcreate -n "lv_lvm_test" -L 5G vg_lvm_test Logical volume "lv_lvm_test" created
The Basics
-
pvs: show physical volumes:
$ sudo pvs PV VG Fmt Attr PSize PFree /dev/vda vg_lvm_test lvm2 a- 4.00g 4.00g /dev/vdb vg_lvm_test lvm2 a- 4.00g 4.00g /dev/vdc vg_lvm_test lvm2 a- 4.00g 4.00g /dev/vdd vg_lvm_test lvm2 a- 4.00g 4.00g /dev/vde vg_lvm_test lvm2 a- 4.00g 4.00g /dev/vdf vg_lvm_test lvm2 a- 4.00g 4.00g -
vgs: volume groups
$ sudo vgs VG #PV #LV #SN Attr VSize VFree vg_lvm_test 6 1 0 wz--n- 23.98g 18.98g
PV: Number of physical volumes in this group
SN: Snapshots
Attributes: See man vgs for the attributes columns
Notice "VSize" and "VFree". It's always a nice feeling to have a bit of free space in your volume groups.
-
lvs: show logical volumes
$ sudo lvs LV VG Attr LSize Origin Snap% Move Log Copy% Convert lv_lvm_test vg_lvm_test -wi-a- 5.00g
Filesystem resizing
Resizing filesystems on LVM is almost trivial.
Resize the LV into your VG's free space (add free space with vgextend if there is none in the VG).
$ sudo vgs
VG #PV #LV #SN Attr VSize VFree
vg_lvm_test 4 1 0 wz--n- 15.98g 5.98g
$ sudo mkfs.ext4 /dev/vg_lvm_test/lv_lvm_test
...
$ sudo mount /dev/vg_lvm_test/lv_lvm_test /mnt/
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 7.0G 1.1G 5.6G 17% /
udev 490M 4.0K 490M 1% /dev
tmpfs 200M 304K 199M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 498M 0 498M 0% /run/shm
/dev/mapper/vg_lvm_test-lv_lvm_test 5.0G 204M 4.6G 5% /mnt
OK, we've got a 5.0G filesystem mounted at /mnt. Let's give him a couple
more megabytes.
$ sudo lvextend vg_lvm_test/lv_lvm_test -L +2G
Extending 2 mirror images.
Extending logical volume lv_lvm_test to 7.00 GiB
Logical volume lv_lvm_test successfully resized
$ sudo resize2fs /dev/vg_lvm_test/lv_lvm_test
$ sudo resize2fs /dev/vg_lvm_test/lv_lvm_test
resize2fs 1.42 (29-Nov-2011)
Filesystem at /dev/vg_lvm_test/lv_lvm_test is mounted on /mnt; on-line
resizing required
old_desc_blocks = 1, new_desc_blocks = 1
Performing an on-line resize of /dev/vg_lvm_test/lv_lvm_test to 1835008 (4k)
blocks.
$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 7.0G 1.1G 5.6G 17% /
udev 490M 4.0K 490M 1% /dev
tmpfs 200M 304K 199M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 498M 0 498M 0% /run/shm
/dev/mapper/vg_lvm_test-lv_lvm_test 7.0G 205M 6.5G 4% /mnt
You can do this with mounted filesystems, even the root filesystem.
Mirroring
We're going to mirror the logical volume that we just created, lv_lvm_test,
using the lvconvert command:
$ sudo lvconvert -m1 vg_lvm_test/lv_lvm_test
vg_lvm_test/lv_lvm_test: Converted: 0.0%
vg_lvm_test/lv_lvm_test: Converted: 4.9%
You can ^C this process and it will continue in the background. Check its progress with the "lvs" command.
$ sudo lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
lv_lvm_test vg_lvm_test mwi-a- 5.00g lv_lvm_test_mlog 16.02
The mirrored volume has 3 components: 2 mirror copies and the disklog. The disklog tracks changes and keeps the volumes in sync.
You can see the different components, and their physical locations, using
the pvs command.
$ sudo pvs -o lv_name,pv_name
LV PV
[lv_lvm_test_mimage_0] /dev/vda
[lv_lvm_test_mimage_0] /dev/vdb
[lv_lvm_test_mlog] /dev/vdb
/dev/vdb
[lv_lvm_test_mimage_1] /dev/vdc
[lv_lvm_test_mimage_1] /dev/vdd
/dev/vdd
/dev/vde
/dev/vdf
Note that the mirrored segments are named $LVNAME_mimage_0 and
$LVNAME_mimage_1. The logical volume is 5GB, which is bigger than the
physical volumes, which are all 4GB. The 5GB segments are split across a
couple of physical volumes each. LVM will try to "do the right thing" and
prevent mimage segments from occupying the same physical devices.
There's a third part, the disklog, named $LVNAME_mlog. In this example, part
of mimage_0 and mlog are on the same physical volume, vdb.
Corelog
When you create a mirror, you can also use "corelog" instead of disklog. This keeps the mirror log in RAM instead of on a disk. This way, you don't need to have a disklog segment.
The downside is that the mirror will have to be synchronized at every boot.
$ sudo pvs -o +lv_name
[sudo] password for eric:
PV VG Fmt Attr PSize PFree LV
/dev/vda vg_lvm_test lvm2 a- 4.00g 0 [lv_lvm_test_mimage_0]
/dev/vdb vg_lvm_test lvm2 a- 4.00g 2.99g [lv_lvm_test_mimage_0]
/dev/vdb vg_lvm_test lvm2 a- 4.00g 2.99g [lv_lvm_test_mlog]
/dev/vdb vg_lvm_test lvm2 a- 4.00g 2.99g
/dev/vdc vg_lvm_test lvm2 a- 4.00g 0 [lv_lvm_test_mimage_1]
/dev/vdd vg_lvm_test lvm2 a- 4.00g 2.99g [lv_lvm_test_mimage_1]
/dev/vdd vg_lvm_test lvm2 a- 4.00g 2.99g
/dev/vde vg_lvm_test lvm2 a- 4.00g 4.00g
/dev/vdf vg_lvm_test lvm2 a- 4.00g 4.00g
Since /dev/vde and /dev/vdf don't have any LV segments on them, we'll take
them out of vg_lvm_test with the vgreduce command, and put them into a new
volume group, vg_corelog_test.
$ sudo vgreduce vg_lvm_test /dev/vde /dev/vdf
Removed "/dev/vde" from volume group "vg_lvm_test"
Removed "/dev/vdf" from volume group "vg_lvm_test"
$ sudo vgcreate vg_corelog_test /dev/vde
Volume group "vg_corelog_test" successfully created
Create the LV:
$ sudo lvcreate -n lv_corelog -l 100%FREE vg_corelog_test
Logical volume "lv_corelog" created
$ sudo vgs
VG #PV #LV #SN Attr VSize VFree
vg_corelog_test 1 1 0 wz--n- 4.00g 0
vg_lvm_test 4 1 0 wz--n- 15.98g 5.98g
Now we double the amount of space in the VG by adding another PV, /dev/vdf:
$ sudo vgextend vg_corelog_test /dev/vdf
Volume group "vg_corelog_test" successfully extended
$ sudo vgs
VG #PV #LV #SN Attr VSize VFree
vg_corelog_test 2 1 0 wz--n- 7.99g 4.00g
vg_lvm_test 4 1 0 wz--n- 15.98g 5.98g
We convert the volume to be mirrored:
$ sudo lvconvert -m1 vg_corelog_test/lv_corelog
Insufficient suitable allocatable extents for logical volume : 1023 more required
Unable to allocate extents for mirror(s).
We don't have enough room for the mirror in our volume group, since it
requires a disklog by default.
$ sudo lvconvert -m1 vg_corelog_test/lv_corelog --corelog
vg_corelog_test/lv_corelog: Converted: 0.0%
vg_corelog_test/lv_corelog: Converted: 7.2%
vg_corelog_test/lv_corelog: Converted: 14.3%
vg_corelog_test/lv_corelog: Converted: 21.4%
Moving PVs around with pvmove
Let's make room by getting rid of our corelog thing.
$ sudo vgremove vg_corelog_test
Do you really want to remove volume group "vg_corelog_test" containing 1
logical volumes? [y/n]: y
Do you really want to remove active logical volume lv_corelog? [y/n]: y
Logical volume "lv_corelog" successfully removed
Volume group "vg_corelog_test" successfully removed
Now we'll add the old PVs back into the volume group, vg_lvm_test.
$ sudo vgextend vg_lvm_test /dev/vde /dev/vdf
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/vda vg_lvm_test lvm2 a- 4.00g 0
/dev/vdb vg_lvm_test lvm2 a- 4.00g 1012.00m
/dev/vdc vg_lvm_test lvm2 a- 4.00g 0
/dev/vdd vg_lvm_test lvm2 a- 4.00g 1016.00m
/dev/vde vg_lvm_test lvm2 a- 4.00g 4.00g
/dev/vdf vg_lvm_test lvm2 a- 4.00g 4.00g
Let's say that we've got the sick feeling that /dev/vdb is about to die on us. Let's move it over to the much newer /dev/vde.
$ sudo pvmove /dev/vdb /dev/vde
Skipping mirror LV lv_lvm_test
Skipping mirror log LV lv_lvm_test_mlog
Skipping mirror image LV lv_lvm_test_mimage_0
Skipping mirror image LV lv_lvm_test_mimage_1
All data on source PV skipped. It contains locked, hidden or non-top level
LVs only.
No data to move for vg_lvm_test
D'oh! It's our mirror. We'll have to break it first.
$ sudo lvconvert -m0 vg_lvm_test/lv_lvm_test
$ sudo pvmove /dev/vdb /dev/vde
/dev/vdb: Moved: 0.0%
You can ^C this. You can check back on the move at any time by just running
pvmove:
$ sudo pvmove
/dev/vdb: Moved: 9.5%
/dev/vdb: Moved: 18.7%
pvmove works by temporarily mirroring the source and target PVs. Check the
output of dmsetup ls --tree:
$ sudo dmsetup ls --tree
vg_lvm_test-lv_lvm_test (252:0)
├─vg_lvm_test-pvmove0 (252:1)
│ ├─ (253:64)
│ └─ (253:16)
└─ (253:0)
Even if the machine crashes in the middle of a pvmove, it won't affect
it. It will just pick up where it left off when the machine gets rebooted.
pvmove works on online devices and mounted filesystems, including the root
filesystem. Cool!