Difference between revisions of "Nested esxi in kvm"
Michael.mast (talk | contribs) |
Michael.mast (talk | contribs) |
||
| (4 intermediate revisions by the same user not shown) | |||
| Line 22: | Line 22: | ||
yum -y update | yum -y update | ||
</pre> | </pre> | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
===Install Virtualization Packages=== | ===Install Virtualization Packages=== | ||
| Line 45: | Line 28: | ||
yum install -y qemu-kvm qemu-img libvirt-python python-virtinst libvirt-client virt-install virt-viewer device-mapper-libs libvirt-daemon-driver-qemu libvirt-daemon-config-network libvirt-daemon-kvm OVMF | yum install -y qemu-kvm qemu-img libvirt-python python-virtinst libvirt-client virt-install virt-viewer device-mapper-libs libvirt-daemon-driver-qemu libvirt-daemon-config-network libvirt-daemon-kvm OVMF | ||
</pre> | </pre> | ||
| − | Now we can enable nested virtualization <ref>https://www.server-world.info/en/note?os=CentOS_7&p=kvm&f=7</ref> | + | Now we can enable nested virtualization <ref>https://www.server-world.info/en/note?os=CentOS_7&p=kvm&f=7</ref><ref>https://www.linuxtechi.com/enable-nested-virtualization-kvm-centos-7-rhel-7/</ref> |
<pre> | <pre> | ||
| − | echo -e "options kvm_intel nested=1 | + | echo -e "options kvm_intel nested=1 \ |
| − | + | enable_shadow_vmcs=1 \ | |
| − | + | enable_apicv=1 \ | |
| − | + | ept=1\n \ | |
| − | options kvm | + | options kvm ignore_msrs=1" >> /etc/modprobe.d/kvm-nested.conf |
| + | |||
modprobe -r kvm_intel | modprobe -r kvm_intel | ||
modprobe kvm_intel | modprobe kvm_intel | ||
| Line 81: | Line 65: | ||
--features kvm_hidden=on \ | --features kvm_hidden=on \ | ||
--machine q35 | --machine q35 | ||
| + | </pre> | ||
| + | Then edit the config<ref>https://www.cloudgardens.eu/blog/vmware-esxi-running-under-qemu-kvm/</ref> | ||
| + | <pre> | ||
| + | virsh edit esxi | ||
| + | <features> | ||
| + | . | ||
| + | . | ||
| + | <kvm> | ||
| + | <hidden state='on'/> | ||
| + | </kvm> | ||
| + | </features> | ||
| + | . | ||
| + | . | ||
| + | . | ||
| + | <cpu mode='custom' match='exact'> | ||
| + | . | ||
| + | . | ||
| + | <feature policy='require' name='vmx'/> | ||
| + | </cpu> | ||
</pre> | </pre> | ||
Latest revision as of 17:34, 31 January 2019
Contents
Purpose
Purchased an refurbished Dell server with 24 drives, 20 physical cores, and ~386GB RAM.The purpose for this server has changed many times during it's time in the lab, but now it will be a storage backend for both VMware and KVM hypervisors. ZFS will be used as the filsesystem with dedup and compression enabled. Obviously the server is overkill for such a use case. So I would like to run the following
- LXD Containers for my many CentOS 7 based servers. Makes sense to use fast kernel based "virtualization" since they run the same kernel.
- KVM machines that are essentially dedicated to the site the server is hosted in.
- VMWare esxi as a nested hypervisor. This is to maintain compatibility with the existing data center.
Ultimately this will act as a replica and DR site.
Prepare Hardware
If you happen to have used the Dell PERC storage controller and wanted something ZFS friendly, please wipe the meta data from the drives. Otherwise CentOS 7 will not install with an error during Anaconda initialization.
- Boot from rescue media and perform the following on all drives.
dd if=/dev/zero of=$YOUR_DEV bs=512 seek=$(( $(blockdev --getsz $YOUR_DEV) - 1024 )) count=1024
I do not have the orignal link to reference. But this would have saved me a lot of time as I was using shred on all the drives instead.
At this point I had installed the "IT Mode LSI SAS 9207-8i SATA/SAS 6Gb/s PCI-E 3.0 Host Bus Adapter LSI00301" I bought off ebay, which gave me direct disk access to all drives. ZFS is happy.
Start Installing Software
Enable epel-release and update
yum -y install epel-release yum -y update
Install Virtualization Packages
yum install -y qemu-kvm qemu-img libvirt-python python-virtinst libvirt-client virt-install virt-viewer device-mapper-libs libvirt-daemon-driver-qemu libvirt-daemon-config-network libvirt-daemon-kvm OVMF
Now we can enable nested virtualization [3][4]
echo -e "options kvm_intel nested=1 \ enable_shadow_vmcs=1 \ enable_apicv=1 \ ept=1\n \ options kvm ignore_msrs=1" >> /etc/modprobe.d/kvm-nested.conf modprobe -r kvm_intel modprobe kvm_intel
You should now see the nested module enabled. At this point we can install the updated qemu packages as mentioned here [5]
yum -y install centos-release-qemu-ev sed -i -e "s/enabled=1/enabled=0/g" /etc/yum.repos.d/CentOS-QEMU-EV.repo yum --enablerepo=centos-qemu-ev -y install qemu-kvm-ev systemctl restart libvirtd
esxi
At this point we install esxi as a VM.[6]
virt-install \ --name esxi \ --ram 160000 \ --disk path=/tank/images/esxi.qcow2,bus=sata \ --cpu host-passthrough \ --vcpus=10 \ --os-type linux \ --os-variant=virtio26 \ --network bridge=br3,model=e1000 \ --graphics spice \ --video qxl \ --cdrom /tank/iso/VMware-VMvisor-Installer-6.7.0.update01-10302608.x86_64.iso \ --features kvm_hidden=on \ --machine q35
Then edit the config[7]
virsh edit esxi
<features>
.
.
<kvm>
<hidden state='on'/>
</kvm>
</features>
.
.
.
<cpu mode='custom' match='exact'>
.
.
<feature policy='require' name='vmx'/>
</cpu>
- ↑ https://fabianlee.org/2018/09/19/kvm-deploying-a-nested-version-of-vmware-esxi-6-7-inside-kvm/
- ↑ https://www.server-world.info/en/note?os=CentOS_7&p=kvm&f=12
- ↑ https://www.server-world.info/en/note?os=CentOS_7&p=kvm&f=7
- ↑ https://www.linuxtechi.com/enable-nested-virtualization-kvm-centos-7-rhel-7/
- ↑ https://www.server-world.info/en/note?os=CentOS_7&p=kvm&f=12
- ↑ https://lists.centos.org/pipermail/centos-virt/2015-November/004780.html
- ↑ https://www.cloudgardens.eu/blog/vmware-esxi-running-under-qemu-kvm/
