CONTEXT
In June-2008, in preparation for a number of client projects I was developing, it seemed prudent to perform some base installs of a number of 'standard' virtualization solutions, with the intent of confirming feature set; ease of installation; and performance of the various virtualization platforms on consistent hardware.
Clearly, benchmarks are imperfect and ultimately tell you only one thing (at best) - how well a given environment will run the chosen benchmark. This may or may not reflect on how well the environment will perform in other roles / for other benchmarks. However, such information may be of use / interest to compare and contrast features / environments / etc.
Virtualization platforms tested were:
- Xen Express version 4.10
- Xen Express Version 5.0 (results added Sept-08)
- Virtual Iron (Single Server Edition v 4.2, and Enterprise Edition, v 4.3)
- Major Vendor (Free edition) version XXXX (*)
- Major Vendor (Free "embedded" version, Aug-08 release (*) (Results added Nov-08) )
- Parallels Virtuozzo 4.0
- ProxMox VE ver 0.9
(*) Note that no less than one major vendor does not like benchmarks being published on their product(s), so their name(s) cannot be given here
Hardware used for tests was definitely "not" high end:
- MSI K9MM-V mainboard (socket AM2 / AMD)
- AMD Athlon 64 "3200+" 2.0ghz cpu
- 2 gigs DDR2 553 ram
- Local HDD, 320gig Seagate ATA
Test setup:
- Install virtualization layer
- Install Win2003 Server Std Trial edition in a virtual machine, with 512mb RAM available
- Patch Windows to latest state via Windows Update / Offline Update
- Run basic benchmark via RDP session to Win2003 host.
- Benchmark app used was "PassMark Performance Test" (ver.6.1), from URL, http://www.passmark.com/products/pt.htm
- Rationale for this benchmark app:
- - It wasn't a huge download, and was easy to install- seemed fairly broad based, covering a variety of CPU and Disk I/O performance metrics (which were of most interest to me)- note that graphics/video performance is inherently of NO interest to me (sessions are in RDP) - I'm concerned only with "server performance" (CPU/IO) here.
Declared inconsistencies:
- "Bare Metal" install has 1 gig RAM available, rather than 512mb - smallest part I had
- Virtual Iron Enterprise Edition uses iSCSI as the backing store for data, not a local disk - and thus, IMHO, it is not appropriate to compare the IO performance of VI_EE with the other IO results here. In particular, I was using a test-grade OpenFiler iSCSI target on old-grade hardware which doesn't offer same performance as local storage.
- Note that the products here break out into 4 'bins' of virtualization, as far as I'm concerned, and as a result, we should anticipate 'inherently' some performance differences between these approaches to virtualization: (At least, according to 'dogma' ?)
- 1 - Major Vendor Free Edition: full hardware abstraction / emulation; no requirement for VT assist hardware; anticipated to have more abstraction / performance penalty. (*)2 - XEN hypervisor: paravirtualization and full virtualization with VT assist. Xen and Virtual Iron are both in this category. Anticipated to have some performance hit.3 - KVM - Linux Kernel Virtualization Module: requires VT assist in this config; hypothetically anticipated to have less performance hit than Xen style hypervisor (?)4 - Container based: (Parallels/Virtuozzo) is not abstracting hardware, so is expected to have performance closest to bare metal.
(*) Unclear if (1) really should be 'that much slower' than (2), and possibly even (3) as well; I'm ultimately not qualified to really understand the precise inner works and nitty grit detail involved. In general - for this page and this work - usual benchmark disclaimers apply of course :-)
Take Note:
- I am running only the single virtualized system on the virtualization layer.
- Typical production Virtualization deployments would be expected to have multiple virtual hosts on one physical host, which may impact performance metrics, etc.
- Clearly I'm not using a "big" / production class physical host - and the intent of these 'basic tests' is simply to establish the "general overhead penalty" of the virtualization layer itself.
| BenchMark - Measure | Bare Metal | KVM Prox | VI_EE | M.Vendr | VI_SSE | VZHOST | Xen Xprss | P4 REF | M.Vendr - sata disk | XenX_5 | ||
| CPU - Integer Math | 62.5 | 59.2 | 59.1 | 29.1 | 60.4 | 58.8 | 61.6 | 43.4 | 59.3 | 61.5 | ||
| CPU - Floating Point Math | 359.7 | 298 | 339.4 | 167.7 | 315.8 | 344.9 | 335.9 | 235.2 | 335.1 | 342.3 | ||
| CPU - Find Prime Numbers | 147.4 | 136 | 142.2 | 74.1 | 137.2 | 135.1 | 144.5 | 156.2 | 135.9 | 142.6 | ||
| CPU - SSE/3DNow! | 1307.5 | 1142.6 | 1254.6 | 630.1 | 1154.1 | 1211.9 | 1269.3 | 1079.7 | 1187.6 | 1292.1 | ||
| CPU - Compression | 1779 | 1712.5 | 1659.8 | 812.5 | 1692.3 | 1652.2 | 1691.2 | 1659 | 1730 | 1740.4 | ||
| CPU - Encryption | 12.9 | 12.3 | 12.7 | 6.2 | 12.5 | 12.4 | 12.6 | 11 | 12.5 | 12.9 | ||
| CPU - Image Rotation | 286.5 | 270.6 | 282.4 | 132.5 | 263.2 | 272.9 | 284.3 | 171.6 | 272.8 | 285.4 | ||
| CPU - String Sorting | 1156.8 | 1071.6 | 1153.9 | 538.7 | 1059.3 | 1089.8 | 1105.4 | 572.4 | 1092.8 | 1152.5 | ||
| ---------- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- |
| Graphics 2D - Lines | 649 | 15.6 | 26 | 156.8 | 26.5 | 42.8 | 27.3 | 22.9 | 28.4 | 759.6 | ||
| Graphics 2D - Rectangles | 513.4 | 17.2 | 36.8 | 142.8 | 38.7 | 41.7 | 37.8 | 27.6 | 43.5 | 695.8 | ||
| Graphics 2D - Shapes | 358.8 | 0.3 | 0.3 | 94.2 | 0.3 | 1.2 | 0.2 | 10.8 | 1 | 393.5 | ||
| Graphics 2D - Fonts and Text | 1319.5 | 18.2 | 47.6 | 413.9 | 54 | 60.8 | 44.9 | 165 | 53.9 | 1232.2 | ||
| Graphics 2D - GUI | 1295.4 | 81.9 | 227 | 366.9 | 196.4 | 243.2 | 278.9 | 132.1 | 246.8 | 1209.9 | ||
| ---------- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- |
| Memory - Allocate Small Block | 1845.2 | 1739.1 | 1762.6 | 804.2 | 1637.6 | 1780.5 | 1762 | 841.2 | 1689.8 | 1784.3 | ||
| Memory - Read Cached | 973.4 | 801.9 | 957.6 | 463.5 | 858.3 | 886.4 | 955.8 | 1477.8 | 924.7 | 958.6 | ||
| Memory - Read Uncached | 884.1 | 740.3 | 852.1 | 375.7 | 840.1 | 837.6 | 855.7 | 1268.6 | 795.9 | 869.6 | ||
| Memory - Write | 861.1 | 719.8 | 938.1 | 420.5 | 885.8 | 845.6 | 811.4 | 533.2 | 857 | 958.7 | ||
| Memory - Large RAM | 198.8 | 7.7 | 7.1 | 19.6 | 32.5 | 50.7 | 54.9 | 206.5 | 39 | 57.5 | ||
| ---------- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- |
| Disk - Sequential Read | 44 | 44.9 | 24.5 | 46.8 | 49.8 | 43.8 | 2.4 | 52.1 | 152.7 | 54.1 | ||
| Disk - Sequential Write | 44.6 | 40.2 | 26.3 | 25.7 | 50.1 | 43 | 2.5 | 48.3 | 25.4 | 54.1 | ||
| Disk - Random Seek + RW | 2.7 | 12.8 | 8.5 | 8.8 | 2.9 | 2.7 | 1.4 | 3.4 | 183.6 | 2.9 | ||
| ---------- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- |
| CD - Read | --- | 9.3 | 4.3 | --- | 1.6 | --- | --- | --- | 3.6 | -- | ||
| CPU Mark | 414.5 | 373.8 | 400.6 | 197.1 | 375.7 | 388.1 | 401.7 | 317.8 | 385.5 | 408.6 | ||
| 2D Graphics Mark | 2884.6 | 73.2 | 170.6 | 803.2 | 167.5 | 208.4 | 187.7 | 213.4 | 191.3 | 3151.1 | ||
| Memory Mark | 352.1 | 296.4 | 334 | 154 | 314.5 | 325.4 | 328.2 | 319.9 | 318.4 | 342.2 | ||
| Disk Mark | 330.2 | 354.1 | 214.3 | 294.2 | 371.8 | 323.8 | 22.8 | 375.3 | 1307.7 | 401.7 | ||
| CD Mark | --- | 1139.1 | 527.2 | --- | 201.2 | --- | --- | --- | 435.6 | --- | ||
| ---------- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- |
| PassMark Rating | 645.2 | 337.9 | 282.5 | 252.2 | 274 | 257.3 | 198.4 | 248.3 | 488.9 | 693.4 |
DISCLAIMER:
This was my own testing work, for my own use. It is not intended as a perfect test, nor ultimately for anyone else for that matter. Material here is released purely for the possible interest of others interested in virtualization. As-is / Where-Is / no implied nor express anything. Your results likely will vary, I'm sure mine will (at least somewhat) if I did this again tomorrow using the same gear. So take this all with a grain of salt and don't worry to much about the specific details. Please do not reproduce this work in any significant part anywhere else in any way. This web site is public so link to this page if you feel you need to 'distribute this data as it stands / in its entirety'.
TODO:
- add M.Vendr commercial non-free (30 day trial) bench results
- tidy up order, sort them by PassMark rating from left to right (declining scores)
- add notes here / update & tidy disclaimers.
CONFIGURATION NOTES:
Bare Metal = Athlon 64bit socket AM2 2ghz / 1gig Ram KVM ProxMox = As above, running ProxMox ver 0.9 / 512mb Virtual Machine Xen Express = as above, running Xen Express v4.10 / 512mb Virtual Machine P4 ref - bare metal older ref box, Classic socket 478 P4 @ 2.4ghz / 2gig ram / intel onboard video Proxmox has ParaVirt NIC but not disk Xen has Paravirt drivers for Nic and Disk no special drivers on bare metal install (windows defaults) - appeared to autodetect fine for disk and NIC in particular. all cases running Win2003 Std except P4 box was Standard Winxp 32bit something is fishy with KVM disk? And also with Xen disk ? Humm. More difference here than expected. VERY Skeptical that SATA disk really made so much performance difference to M.Vendr-Server, but maybe possible? (more likely cache or some other feature(s) obscure 'real' performance?) VI_SSE is Virtual Iron ver 4.2 / Single Server edition, Win2003, fully patched. Maj.Vendr ver XXXX cannot be explicitly given here (recent download) VERY surprised with performance of VZHOST/Virtuozzo - had been under impression it should be among the top of the performance curve, based on 'dogma and documentation'. Unclear if this result is utterly bogus, or if this is just an unexpected surprise. Already have re-tested this twice now on this hardware with similar results. Maybe need to test on different hardware? Or something else ? Note that XEN EXPRESS results were added Sept-19-08, since that new available release. It was tested against a SATA disk/local storage; same hardware otherwise as everything else.
ROUGH RESULTS ADDENDUM:
This will be integrated to table above shortly. But for now:
Some new bench results from Nov-1-08, running
- "Major vendor" "Embedded Server eXtension Infrastructure" platform; on same 2ghz athlon /2gig ram system. Hypervisor is run from 2gig USB memory stick. No local storage. NFS/iSCSI via Intel Gig-Ether nic. Public/Management NIC interface is also via another intel GigEther Nic.
- 2 different bench results below, the first is using iSCSI backed storage on openfiler 2.3 server; the second is NFS backed storage on openfiler 2.3 server. Storage network is dedicated gig-ether.
iSCSI backed storage - benchmark results:
Test Name This Computer CPU - Integer Math 59.9 CPU - Floating Point Math 335.9 CPU - Find Prime Numbers 145.0 CPU - SSE/3DNow! 1303.8 CPU - Compression 1714.7 CPU - Encryption 12.9 CPU - Image Rotation 195.2 CPU - String Sorting 1003.5 Graphics 2D - Lines 27.7 Graphics 2D - Rectangles 32.8 Graphics 2D - Shapes 0.3 Graphics 2D - Fonts and Text 44.3 Graphics 2D - GUI 244.4 Memory - Allocate Small Block 1730.4 Memory - Read Cached 952.8 Memory - Read Uncached 664.5 Memory - Write 915.6 Memory - Large RAM 48.4 Disk - Sequential Read 30.0 Disk - Sequential Write 25.5 Disk - Random Seek + RW 1.5 CPU Mark 387.2 2D Graphics Mark 171.6 Memory Mark 318.8 Disk Mark 206.5 PassMark Rating 227.2
NFS backed storage - benchmark results:
Test Name This Computer CPU - Integer Math 58.9 CPU - Floating Point Math 329.1 CPU - Find Prime Numbers 137.4 CPU - SSE/3DNow! 1296.7 CPU - Compression 1667.6 CPU - Encryption 12.2 CPU - Image Rotation 237.8 CPU - String Sorting 1126.5 Graphics 2D - Lines 27.8 Graphics 2D - Rectangles 40.9 Graphics 2D - Shapes 0.2 Graphics 2D - Fonts and Text 43.4 Graphics 2D - GUI 245.5 Memory - Allocate Small Block 1681.1 Memory - Read Cached 901.4 Memory - Read Uncached 651.0 Memory - Write 907.8 Memory - Large RAM 61.6 Disk - Sequential Read 30.2 Disk - Sequential Write 33.8 Disk - Random Seek + RW 24.0 CPU Mark 394.4 2D Graphics Mark 180.1 Memory Mark 310.7 Disk Mark 318.0 Pa ssMark Rating 251.2