In the December 2015 issue of GURU advisor, we covered Proxmox Virtualization Environment with an in-depth review of version 4.0 of the Open Source virtualization platform developed by the Austrian Proxmox Server Solutions. Let’s analyze the new features of the 4.1 release that make it a reasonable competitor to the renowned solutions by Microsoft and VMware.

Proxmox VE vs. vSphere

A comparison with the market-leading hypervisor might seem unfair at first glance, but a careful analysis of the feature of Proxmox VE can let you reconsider it as an alternative, in particular in situations where the budget/performances ratio is the main choice parameter.

Proxmox VE (Proxmox from here onwards) is based on the Debian operating system and brings with itself all pros and cons of this renowned Linux distro: a stable, secure, widespread and well-tested operating system with a very large choice of software packages -albeit sometimes not updated to the latest version in order to guarantee a perfect compatibility and stability- and the “Social Contract” which guarantees all the advantages of the Free Software (https://www.debian.org/social_contract). Other advantages include a well-founded community which is very active in code management and a rich and detailed documentation. For it is an open source project, the source code repository is available to whoever wants to implement new features or adapt the existing ones to his/her own needs. Moreover, the GPL license allows commercial use.

Those who are familiar with the VMware vSphere platform, and its most advanced features, like clustering, High Availability, storage and VM live migration, know that they are not part of the free version of ESXi.
In order to obtain access to these features, the purchase of a commercial license (Essentials Plus, Standard, Enterprise Plus, etc.. with related requirements and costs) is required.

On the other hand, with Proxmox you can use for free and natively most of the aforementioned features, and some additional ones: VM backup and restore with vzdump, cloning VMs, creating templates, HA simulation (to test if requirements are met) and distributed network storage.

Moreover, still offering a valuable alternative to the vSphere ecosystem, each Proxmox installation comes with a Web interface for the centralized management of clusters and hosts. We stress the fact that there is no dependence upon a vCenter as each host can act this role independently from the others.
Speaking about features that aren’t found within the solution by VMware, LXC Containers and the ZFS file system are the ones that offer the most consistent added value. vSphere, on its turn, is a mature product that has risen as de-facto standard in the virtualization world: it’s used in mission-critical environments, support by VMware and hardware manufacturers is excellent and the array of offered features is vast.

However, features like vMotion, storage vMotion, High Availability, Fault Tolerance, vSwitch and integrations with other products by VMware are a prerogative of paid versions that could cost tens of thousands of euro, adding up license costs and complexity of the required infrastructure.

Being price the only discriminating factor, Proxmox is more convenient that the free version of ESXi, too, given the more favourable price/feature ratio. The presence of additional free features also allows to experiment situations and scenarios more or less complex without a time limitation: the free version of ESXi offers a full-feature demo for only 60 days.
In this sense, SMBs and freelancers should consider Proxmox as a solution for the consolidation of their servers or for the creation of their own infrastructure with a tight budget, possibly considering the purchase -if required- of the commercial support offered by the Austrian company on its product.

It ain’t just ‘plus’, you get a few disadvantages

When making a decision it’s important to ponder about the fact that Proxmox requires to “get your hands dirty”. This hypervisor is essentially a Debian distro, so you need to possess a good knowledge of the Linux OS and a natural confidence with the command line.
Proxmox is a project developed by a team that, no matter being very active, professional and fit in a strong community, is no way comparable in terms of size and resources to VMware or Microsoft (talking about Hyper-V).
Despite being production-ready and many realities use it to manage a virtualized infrastructure, keep in mind that bumping into bugs and small problems which are not expected with the concurrent products is not a remote likelihood.
Support too might turn critical: VMware offers a wide and tested coverage and MSPs, resellers and specialized technicians composing a global network of pros: it really can’t be compared to a niche and still not very well known product like the Austrian.
Support by Proxmox Server Solutions, as in the case of vSphere, is commercially offered with a licensing model that is simpler and clearer than VMware’s. The four Subscription options are on an annual and per socket basis and also include the access to the Enterprise repository.

Proxmox VE 4.1: the new release

One of the peculiarities of Proxmox is, without any doubt, the presence of LXC Container. Containers (CT) are a virtualization system that allows the creation of virtualized instances sharing resources with the underlying OS: in other words, it’s possible to create Linux virtual machine on top of the same OS of the hypervisor. Indeed, LXC stands for Linux Containers.

The advantages of CTs with respect to traditional VMs are a minor impact in terms of required computational resources and a very quick deployment time. After completing the creation wizard in Proxmox, the newly-created CT takes only a few seconds to be fully functioning. Boot time is lower as well because the kernel, which is shared with the underlying hypervisor, has already been loaded by Proxmox, thus making the creation, use and destruction of Containers a quick and easily replicable process.
Among the disadvantages we can find a minor isolation from the underlying operating system: a VM has its own kernel that can be radically different from the host one (think about a Windows VM on Proxmox, which is Linux), while a CT shares its kernel with the host thus (partially) depend on it.

An approach of this kind sets some questions in terms of security: can a user, somehow, manage to “escape” from a CT and end up directly in the host? The answer is yes and, should it be a root user, the user could have complete control of the hypervisor. This scenario can happen when the UID (User ID, a number that univocally distinguishes the user and defines its privileges) is mapped to 0 (zero), which is root. That is Privileged Containers. Privileged CTs were the first kind of CT to be implemented and, regarding this virtualization approach, some doubts still remains.
With Unprivileged CTs instead you can give admin privileges to the user inside a Container, and remove them should the user manage to escape into the host, thus increasing the overall security level.
Unprivileged Containers have been introduced in Proxmox 4.1 as a technology preview, albeit they can’t be created from the Web graphical interface. You need to pass the --unprivileged parameter to the pct command (the command to manage LXC Containers) from the command line when creating a CT: for instance, pct create 130 local:vztmpl/centos-7-default_20150829_amd64.tar.xz --unprivileged.
Another procedure that can’t be natively done is the conversion of a Privileged CT to an Unrivileged CT: from the command line you need to backup the CT and then restore it with the --unprivileged parameter again, for instance pct restore 127 local:backup/vzdump-lxc-111-2016_04_07-13_18_30.tar.gz --unprivileged. The support to Unprivileged Containers should be complete with the next Proxmox version.

Among the other news we must cite an improved support to ZFS, with the option to install up to 8 disks in RAID-Z configurations. The NTP (Network Time Protocol) service, which is used to synchronize VMs, CTs and hosts in clusters, is now enabled by default, and an improved startup/shutdown behaviour thanks to systemd -the analogous to the init process that manages the start of the various processes after the boot phase- and VM shutdown from GUI has been improved: now no errors should arise. Talking about system behaviour, the GUI now has the option to active QEMU Agent (for KVM VMs), whose implementation has room for improvement; installation is cumbersome and requires a VirtIO driver that is usually not installed.

In addition to Unprivileged CT, LXC now supports the new Fedora 22 and Ubuntu 15.10 OSes, in addition to the disk resize option with rootfs from GUI (and not just CLI anymore).
Lastly, the Thing Provisioning technology for LVM-partitioned disks is introduced as technology preview: the implementation will be enhanced in the upcoming release; at the moment it’s still too immature for a production use (for instance, CT can’t be migrated on LVM-thin supports at the moment).
The roadmap of the next releases has additional improvements: Proxmox 4.2 will be based on the Linux 4.4 kernel and many interesting features are expected, with CT live migration and LVM-thin provisioned partitioning (which suffers from bugs and implementation issues at the moment) amongst them.
As far as security is concerned, the support to Let’s Encrypt will be introduced (LE is the Open Source project by Internet Security Research Group that offers free SSL certificates with an automated request, activation and renown procedure. More information at https://letsencrypt.org/about/), in terms of High Availability external fencing devices will be supported thus offering a broader HA management and the implementation of DRBD (storage block replica) will be improved.
The improvement of both the documentation framework and the Web interface with the appealing Sencha Ext JS framework are scheduled.

Proxmox and Turney: quick deployment of VMs and Containers

Turnkey Linux (more information at https://www.turnkeylinux.org) is a really interesting project that offers several Virtual Appliances (both as Virtual Machines and Containers) with different preconfigured services: LAMP/LEMP stacks, CMS, BugZilla, Ansible, MySQL, wiki systems, ownCloud, openLDAP, OpenVPN etc.
The integration with Proxmox has been featured since version 2.0, and CT were based on OpenVZ. With Proxmox 4.0 -the release that saw the switch from OpenVZ to LXC- all appliances based on the old Container system had been removed, leaving only OS images based on the newer LXC. With Proxmox 4.1, TurnKey offers both resource technologies.
Deployment is rather easy: all you need is to access to the Storage View, choose an host and a storage and select the Templates voice in the Content tab. At the moment, a noteworthy 97 different virtual appliances are available, in addition to the aforementioned OSes.
The pvereport command, introduced with this release, offers an useful and very detailed overview of the installation analyzing all the information about the system, virtual machines, containers, storage, bios, clusters, firewall, interfaces and network scheme. The output is quite sizeable, we suggest to pipe it to a text file (pvereport > report.txt) for a quick consultation, for instance, in case of troubleshooting.

A note about lifecycle

The interval between a Proxmox release and the next one is usually 6 months. Being an on-going and young project (it started in 2008), each release introduces an important amount of new feature and bug-fixes: the advice is to constantly update the infrastructure, but after verifying the compatibility between the different versions.

At the end of this month (30th April, 2016) the main support to Proxmox 3.4, the most important release before Proxmox 4.x, will end. From that time onwards, only security updates will be released, nothing about new features: Debian 7 “Wheezy” -the OS at the basis of Proxmox 3.4- will enter the Long Term Support on 26th April, 2016 and will extend the duration of security updates, managed by a team of volunteers and not by the development team, until 31st May, 2018.