Fortech I.T. Solutions - SANDBOX

(Additional disclaimer, using 3rd party drives in your Dell gear is clearly not supported by Dell, you do so at your own risk, etc etc. Your server may implode if you attempt this. Don't blame me if you decide to do this and things don't work well for you :-)

'''Note''': Why would you want to do this?
* Single ProxVE host with generous amount of local storage in HW raid
* Note that most Linux distros are not happy to boot from a volume > 2Tb in size
* so a work-around is to carve out 2 LUNs on the raid-controller; the first is some 'modest' size like 100Gigs for the OS; and the second LUN is intended for "bulk data". In the case of ProxVE, I'm installing ProxVE initially into just the 100Gig LUN; but subsequently want to reconfigure it so that the ~3+Tb LUN is used for VM Image storage; and the 100Gig LUN is used for ProxVE platform, swap, etc.

TDC Oct-1-2010

!!!! CONTEXT

These notes are here partially for my own reference / for future; and in case they are of use to someone else.

The machine used for this work was a Dell t300 with internal Perc6i SATA HW Raid Controller (LSI MegaRaid chipset based)

In case it is of interest to anyone, it appears this controller is OK with the 3rd party drives used to upgrade storage capacity: "HITACHI Deskstar 7K2000 HDS722020ALA330 (0F10311) 2TB 7200 RPM 32MB Cache SATA 3.0Gb/s 3.5" Internal Hard Drive -Bare Drive" -- as per the NewEgg URL for reference: http://www.newegg.ca/Product/Product.aspx?Item=N82E16822145298&nm_mc=TEMC-RMA-Approvel&cm_mmc=TEMC-RMA-Approvel-_-Content-_-text-_-

(This is not an endorsement of Newegg, etc, just a link for clarity in case someone else wants to consider using such disks as 'possibly OK' in their Dell 6i raid config).

--------------------

!!!! CONCISE SUMMARY OF LVM REARRANGE ON PROXVE:


TARBALL BACKUP:

[@
proxmox:~# cd /var/lib/vz
proxmox:/var/lib/vz# tar cvf /var-lib-vz.tar ./*
@]

CHECK STATUS OF LVs:
[@
proxmox:/# lvdisplay
--- Logical volume ---
LV Name /dev/pve/swap
VG Name pve
LV UUID cj7Dfz-9y1C-6cSw-0wGa-clta-k91i-a2lygU
LV Write Access read/write
LV Status available
# open 1
LV Size 4.00 GB
Current LE 1024
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 254:0

--- Logical volume ---
LV Name /dev/pve/root
VG Name pve
LV UUID zeaXvP-J4cO-3ybZ-AseL-V7E3-ZcHE-9PGdts
LV Write Access read/write
LV Status available
# open 1
LV Size 24.25 GB
Current LE 6208
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 254:1

proxmox:/#
@]

CHECK VOLUME GROUP STATUS - total capacity ?

[@
proxmox:/# vgdisplay
--- Volume group ---
VG Name pve
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 5
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 1
Act PV 1
VG Size 97.15 GB
PE Size 4.00 MB
Total PE 24871
Alloc PE / Size 7232 / 28.25 GB
Free PE / Size 17639 / 68.90 GB
VG UUID 6s2HNY-mWlw-RVve-XlYC-z2LU-PW1G-Gpbpe3

proxmox:/#

@]
so: add the free space - 68.90 GB - to the root LV. Thus:

[@
proxmox:/# lvresize -L +68.90G /dev/pve/root
Rounding up size to full physical extent 68.90 GB
Extending logical volume root to 93.15 GB
Logical volume root successfully resized
proxmox:/#
@]

Now check status:

[@
proxmox:/# vgdisplay
--- Volume group ---
VG Name pve
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 6
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 1
Act PV 1
VG Size 97.15 GB
PE Size 4.00 MB
Total PE 24871
Alloc PE / Size 24871 / 97.15 GB
Free PE / Size 0 / 0
VG UUID 6s2HNY-mWlw-RVve-XlYC-z2LU-PW1G-Gpbpe3

@]

NEXT UP: PREP THE SDB DRIVE FOR ADDITION

USE PARTED BECAUSE IT IS >2Tb in size, Fdisk will choke.

[@
26 apt-get install parted
@]
RUN PARTED AND....
[@
proxmox:/# parted /dev/sdb
GNU Parted 1.8.8
Using /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p
Model: DELL PERC 6/i Adapter (scsi)
Disk /dev/sdb: 3895GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags

(parted) mklabel

Warning: The existing disk label on /dev/sdb will be destroyed and all data on this disk will be lost. Do you want to continue?

Yes/No? yes

New disk label type? [msdos]? gpt

(parted) mkpart primary 0% 100%
(parted) set 1 lvm on
(parted) p

Model: DELL PERC 6/i Adapter (scsi)
Disk /dev/sdb: 3895GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 17.4kB 3895GB 3895GB primary lvm

(parted) q
Information: You may need to update /etc/fstab.
@]

ADD NEW STORAGE INTO LVM:

[@
proxmox:/# pvcreate /dev/sdb1
Physical volume "/dev/sdb1" successfully created


proxmox:/# vgextend pve /dev/sdb1
Volume group "pve" successfully extended

proxmox:/# vgdisplay
--- Volume group ---
VG Name pve
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 11
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 2
Act PV 2
VG Size 3.64 TB
PE Size 4.00 MB
Total PE 953470
Alloc PE / Size 24871 / 97.15 GB
Free PE / Size 928599 / 3.54 TB
VG UUID 6s2HNY-mWlw-RVve-XlYC-z2LU-PW1G-Gpbpe3

proxmox:/#

@]
OK: Create the new LV, pve-data:

[@
lvcreate -L 3.54T -n /dev/pve/data pve
@]
check that space is allocated:
[@
proxmox:~# vgdisplay
--- Volume group ---
VG Name pve
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 12
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 3
Open LV 3
Max PV 0
Cur PV 2
Act PV 2
VG Size 3.64 TB
PE Size 4.00 MB
Total PE 953470
Alloc PE / Size 952861 / 3.63 TB
Free PE / Size 609 / 2.38 GB
VG UUID 6s2HNY-mWlw-RVve-XlYC-z2LU-PW1G-Gpbpe3

@]
Check the LVs to be certain all is well:
[@
proxmox:~# lvdisplay
--- Logical volume ---
LV Name /dev/pve/swap
VG Name pve
LV UUID cj7Dfz-9y1C-6cSw-0wGa-clta-k91i-a2lygU
LV Write Access read/write
LV Status available
# open 1
LV Size 4.00 GB
Current LE 1024
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 254:0

--- Logical volume ---
LV Name /dev/pve/root
VG Name pve
LV UUID zeaXvP-J4cO-3ybZ-AseL-V7E3-ZcHE-9PGdts
LV Write Access read/write
LV Status available
# open 1
LV Size 93.15 GB
Current LE 23847
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 254:1

--- Logical volume ---
LV Name /dev/pve/data
VG Name pve
LV UUID ZWW2x1-UegY-EUAT-8Mk8-T1BM-M9dj-0rds2W
LV Write Access read/write
LV Status available
# open 1
LV Size 3.54 TB
Current LE 927990
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 254:2

proxmox:~#
@]

FORMAT THE THING:

(note this takes a few minutes - 15min? - it is a big filesystem after all)

[@
proxmox:/# mke2fs -j /dev/pve/data
mke2fs 1.41.3 (12-Oct-2008)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
237568000 inodes, 950261760 blocks
47513088 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
29000 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848, 512000000, 550731776, 644972544

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 37 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
proxmox:/#
@]

WHILE THAT IS GRINDING ALONG;

-> Open another SSH session
-> Resize the ROOT FILESYSTEM to use up the space available to it.

another shell session: resize my root fs.

[@
proxmox:~# resize2fs /dev/pve/root 24419328
resize2fs 1.41.3 (12-Oct-2008)
Filesystem at /dev/pve/root is mounted on /; on-line resizing required
old desc_blocks = 2, new_desc_blocks = 6
Performing an on-line resize of /dev/pve/root to 24419328 (4k) blocks.
The filesystem on /dev/pve/root is now 24419328 blocks long.
@]
(NOTE THIS STEP TAKES A LITTLE WHILE ALSO - GO GET A COFFEE/ETC)



ONCE FS FORMAT IS DONE (IN OTHER SHELL SESSION:)
Finally, mount it, untar, we're good.


[@
proxmox:/# more /etc/fstab
# <file system> <mount point> <type> <options> <dump> <pass>
/dev/pve/root / ext3 errors=remount-ro 0 1
/dev/pve/data /var/lib/vz ext3 defaults 0 1
UUID=9249231d-b67c-4dae-9691-cc66aa63be64 /boot ext3 defaults 0 1
/dev/pve/swap none swap sw 0 0
proc /proc proc defaults 0 0

proxmox:/# mount /var/lib/vz

proxmox:/# df -k
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/pve-root 96143116 765168 90494968 1% /
tmpfs 2019720 0 2019720 0% /lib/init/rw
udev 10240 592 9648 6% /dev
tmpfs 2019720 0 2019720 0% /dev/shm
/dev/sda1 516040 45016 444812 10% /boot
/dev/mapper/pve-data 3741402976 201364 3551149260 1% /var/lib/vz

proxmox:/# cd /var/lib/vz
proxmox:/var/lib/vz# tar xvf /var-lib-vz.tar
./dump/
./images/
./lock/
./lost+found/
./private/
./root/
./template/
./template/qemu/
./template/iso/
./template/cache/

proxmox:/var/lib/vz# pwd
/var/lib/vz

proxmox:/var/lib/vz# ls -la
total 48
drwxr-xr-x 9 root root 4096 Sep 30 21:46 .
drwxr-xr-x 32 root root 4096 Sep 30 20:19 ..
drwxr-xr-x 2 root root 4096 Sep 3 09:21 dump
drwxr-xr-x 2 root root 4096 Aug 31 06:14 images
drwxr-xr-x 2 root root 4096 Sep 30 20:29 lock
drwx------ 2 root root 16384 Sep 30 20:17 lost+found
drwxr-xr-x 2 root root 4096 Sep 20 07:03 private
drwxr-xr-x 2 root root 4096 Sep 20 07:03 root
drwxr-xr-x 5 root root 4096 Sep 30 20:19 template
proxmox:/var/lib/vz#
@]

SO:

- Root FS has 700meg used, 90 gig free.
- VZ Data FS has 3.4 Tb free.

[@
proxmox:/var/lib/vz# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/pve-root 92G 748M 87G 1% /
tmpfs 2.0G 0 2.0G 0% /lib/init/rw
udev 10M 592K 9.5M 6% /dev
tmpfs 2.0G 0 2.0G 0% /dev/shm
/dev/sda1 504M 44M 435M 10% /boot
/dev/mapper/pve-data 3.5T 197M 3.4T 1% /var/lib/vz

proxmox:/var/lib/vz#
@]

-----------