Differences

This shows you the differences between two versions of the page.

--- start:virtualization [2022/05/10 16:32] – peter
+++ start:virtualization [2023/09/14 20:18] (current) – peter
@@ Line 1: / Line 1: @@
+DATE CHECKED THIS PAGE WAS VALID: 14/09/2023
 So there is quite a bit to do when setting up for KVM.
 In the [[Start:ZramSwap| How To Setup Zramswap And Make Your PC Awesome]] we touch on setting up zram so that you can get more VM's but there is actually a way to get even better performance that I did not mention in that guide.
@@ Line 10: / Line 12: @@
 <code>
-sudo apt-get install virt-manager
+sudo apt-get install virt-manager qemu-kvm ovmf swtpm-tools
 </code>
@@ Line 59: / Line 61: @@
 eg: here is one on a test machine I am using:
-{{:images:grubiommudeb.png?400|}}
+{{:images:grubiommudeb.png|}}
 You will see that on that line as I have an intel chip I have added the intel_iommu=on parameter onto that line. I also have a couple other parameters such as mitigations=off and nvidia-drm.modeset=1 but these should not be added as they are generally not required and decrease the security of the system. Im simply showing you what I had on a test system so you can see where to make modifications that you may need.
@@ Line 86: / Line 88: @@
 is not a super useful value as it is a little under 400KB or 19.5MB per second or around 1GB per minute. This might sound like a lot but if you have 128GB of RAM assigned to VM's then for it to scan all the way through the RAM would take about 2 hours to do one complete scan through your RAM. While this will eventually get through everything it can be more efficient to find a slightly more reasonable value.
-The cost of this is that the CPU will be more taxed. The ksmd service appears to be single threaded so it will use 1 core of your available CPU cores only, so once it reaches 100%, increasing the value will make it less efficient as you are asking it to do more than the CPU can keep up with. During my testing I have found that a value of CPU at 40% (on a single core) is alright if you have a lot of spare cores, or perhaps less if you dont have a large number of free cpu cores. I have 12 on my system I tested from so giving 1 core up for better memory management seemed like an alright trade off (leaving it running at 40% all the time). However I did feel this was a little on the high side and would be happy with a lower value as well. You can experiment yourself based on what I will show you here but probably a good low value would be around 10% CPU, a mid value around 25% and a high value around 40%). The higher the value the more the processor will scale and use power, cooling to compensate etc.
+The cost of this is that the CPU will be more taxed. The ksmd service appears to be single threaded so it will use 1 core of your available CPU cores only, so once it reaches 100%, increasing the value will make it less efficient as you are asking it to do more than the CPU can keep up with. During my testing I have found that a value of CPU at 40% (on a single core) is alright if you have a lot of spare cores, or perhaps less if you dont have a large number of free cpu cores. I have 12 on my system I tested from so giving 1 core up for better memory management seemed like an alright trade off (leaving it running at 40% all the time). However I did feel this was a little on the high side and would be happy with a lower value as well. You can experiment yourself based on what I will show you here but probably a good low value would be around 10% CPU (for servers with high uptime etc that can over several hours normalize memory), a mid value around 25% and a high value around 40% (more agressive for desktop pc that spins up vm's for a lab then turns them off after a couple hours.). The higher the value the more the processor will scale and use power, cooling to compensate etc.
 To check what is reasonable to you just increase the pages_to_scan value eg:
@@ Line 95: / Line 97: @@
 On my system a value of 4096 is 16 megabytes every cycle or 800MB a second (4096*4/1024 = 16 and the default sleep is every 20 milliseconds or 50 times a second). Thus every minute 46GB is being scanned and deduplicated if possible.
-On a modern system with lots of RAM getting through all the RAM in around 15 minutes would be good, so that you are checking the RAM can be optimized again and again. As its not uncommon for systems with around 256GB of RAM these days this value seems fairly reasonable to me at this time. Mine is probably a little high but the final optimization I leave to you, try aim to keep CPU under 40% and the amount of time around 15 minutes or so. That way when you fire up a VM, 15 minutes later some of the RAM can be claimed back. Setting this value too high is detrimental so try experiment to find the lowest value that is reasonable. There is no point scanning so quickly that everything is done every minute for example, try think of a reasonable goal like getting through the RAM in 15 minutes and work to find the lowest value that hits that. My test value above is high and not something I would use unless I was testing to see what I saved quickly just to benchmark if it was worth it at all.
+On a modern system with lots of RAM getting through all the RAM in around 15 minutes would be good, so that you are checking the RAM can be optimized again and again. As its not uncommon for systems with around 256GB of RAM these days this value seems fairly reasonable to me at this time. Mine is probably a little high but the final optimization I leave to you, try aim to keep CPU under 40% and the amount of time around 15 minutes or so. That way when you fire up a VM, 15 minutes later some of the RAM can be claimed back. Setting this value too high is detrimental so try experiment to find the lowest value that is reasonable. There is no point scanning so quickly that everything is done every minute for example, try think of a reasonable goal like getting through the RAM in 15 minutes and work to find the lowest value that hits that. My test value above is high and not something I would use unless I was testing to see what I saved quickly just to benchmark if it was worth it at all. Also not my system I was testing on was a desktop not a server. As mentioned a server would want a low value of CPU consumed like 10% or under. Setting this value high is detrimental to performance in terms of CPU so on a server where you can take a long time and let it optimize over many hours there is no benefit to an aggressive value.
 Here is htop showing the CPU being taxed:
-{{:images:htopksm.png?400|}}
+{{:images:htopksm.png|}}
 If you dont see the ksmd process check that htop does not have "hide kernel threads" set in the setup.
@@ Line 173: / Line 175: @@
 sudo apt-get install virt-manager
 </code>
+I did go and add this in the beginning of the guide in case you didn't know but it does not address how to actually use it.
+After installing you will want to give the box a reboot and then run virt-manager. It will also be in the apps menu.
+By default Debian (not Ubuntu) asks for your password when you open it. If you don't like this you can add your account to the relevant groups:
+<code>
+sudo usermod -a -G libvirt $(whoami)
+sudo usermod -a -G kvm $(whoami)
+</code>
+But if you don't mind you can authenticate each time also. Either option works.
+The first screen is very blank if you haven't used it before:
+{{:images:virtmanblank.png?600|}}
+The first thing we should do is define where we want to place machines, and the second thing is create a network for them. I recommend using a logical location for disk images/ISO images in your home, and a macvtap for the network as it prevents communication to the host which is safer. For temporary connunication we can create a bridge then unplug the bridge when not using it. This should be a general setup for like 99% of machines and will get you going so let me show this and you can fiddle around with other more complex setups yourself.
+Oh and download an ISO as a test also. For this I downloded the fedora iso image for fun.
+In my screenshot above you can see I am "not connected". If this happens just right click on the QEMU/KVM line and choose connect. Its no big deal its just being silly.
+Next choose edit and connection details.
+We want to make a new virtual network here so lets make our own. I want a bridge network. The 'default' network is actually already a bridge on 192.168.122.0/24 range but in case you didn't know the way to create one is to just click the + and fill in details as such:
+{{:images:bridgenat.png?600|}}
+When you finish it will have autostart and active enabled. I will use this one (virbr1) going forward but you could use the default on if you wanted. Its just nice to know how to make your own and understand all steps if need be.
+Next lets define our storage area. Click on the storage tab and instead of the default location we will create a new one. I have a folder in my home already with the Fedora ISO in it I will be using.
+Click the + and give the pool a name like "MyHomePool" or something and browse to the folder location as such:
+{{:images:poolcreate.png?600|}}
+Click finish. If you select that pool you will see whatever files are in there listed and it will be active and enabled on boot:
+{{:images:showthepool.png?600|}}
+Now we are ready to make a VM. As I mentioned the general kind of setup most people want (or me at least) is a VM that is on the same local LAN as other computers in their network. You can obviously not do this and put it on its own private lan if you like. However its generally a good idea to not have the VM guests also communicate with the host as a compromised VM then can try compromise the host and by extension all VM's running on it. However you can setup things exactly how you feel they should be as you go on after this simple tutorial. For us we will have the VM on the local lan but separate from the host, and have a bridge that we turn on or off if we want to communicate with it temporarily. This is pretty much a generalized blend of usability and functionality and security while you learn the ins and outs of VM management.
+So click File - New Virtual Machine and on the first screen choose Local Install. Browse for the ISO on the next screen by selecting your pool and the ISO in question. The installer should detect what you are installing but you can search the bottom dialogue for your OS if need be or choose a generic option.
+On the next screen give it some CPU and Memory as you feel appropriate and choose next.
+On the next screen I choose "select or create custom storage" and then click "manage". I then click on my pool and click the + so that on the next screen i can give the disk image a name in the location I wanted:
+{{:images:diskimagecreation.png?600|}}
+As you can see I created a small disk image in my home location I wanted. Click finish and the disk image should appear and you can click choose volume after selecting it so that this is where the OS will be installed.
+Back on the New VM screen you can click forward and then tick the box that says "customize configuration before install" so we can add our specific networking.
+Under the Networking selection you can change the 'default' to the 'bridge' network we made earlier and click finish:
+{{:images:finalcustomvirtman.png?600|}}
+On the customization screen you can fiddle around if you want but in this tutorial we are just interested in creating a new network. The current NIC is already the bridge which will facilitate communication to the host machine and provide the VM internet if need be but I want to also add a macvtap adapter so it can be on my local lan, and communicate with devices on my lan such as my NAS or things like that.
+So choose "add hardware" and choose "network". On the settings of the network we want Macvtap device and under device name you will choose the appropriate device of your Host machines network (ie when you use the 'ip a' command:
+{{:images:networkselection.png?600|}}
+I have highlighted the network name on the host machine in the terminal which is on my local lan 192.168.2.x range above. Hopefully this makes sense. You are telling it what network to place that device on with the identifier you type there.
+You can then click finish and "begin installation" At any rate once you are done you will be able to install your VM of choice and have in this case 2 networks - the VM on your local LAN which can communicate with all pc's on your lan except the host machine and a bridge which can allow communication between the host and the VM if need be temporarily.
+Here we can see the networks and I ping the host (on .1) (and it also got an ip of 192.168.2.74 on my LAN):
+{{:images:pingtest.png?600|}}
+Now what you can do is when you are done with the bridge click on the light bulb and just choose the bridge network and untick 'active' and then 'apply'. This is the same as unplugging a cable when you dont need the bridge. You can see that you wont have a connection to that network anymore on the guest (ip a command).
+{{:images:activedisablevmm.png?600|}}
+Thats the basics, good luck :)
 === Notes ===
-Any further notes you can add here.
+Any VM with the following in the XML will not use the ksmd optimizations:
+  <memoryBacking>
+    <nosharepages/>
+  </memoryBacking>
+Adding that can exclude certain VM's from being memory optimized.
+For guest VM's dont forget to install the guest additions on the VM. In windows guet VMs you have to download them but on linux guests you can install them by:
+<code>
+sudo apt-get install qemu-guest-agent spice-vdagent
+</code>
+If you dont install them the guests tend to run sub optimally and slower.
+=== Notes ===
+Notes:
+Suggested value for pages to scan is:
+echo 1024 > /sys/kernel/mm/ksm/pages_to_scan
+Works well most systems.

Make Debian Fun Again And Learn How To Do Other Cool Stuff Too

User Tools

Site Tools

Differences

Page Tools