VirtualBox: !!Assertion Failed!!? Try disabling SME!

Submitted by eazrael on Sat, 04. May 2019 11:47:05

After upgrading from an old FX-8350 to a new ThreadRipper VirtualBox was not able to start any VM in Linux. The error shown in VM was something like "NS_ERROR_FAILURE (0x80004005)"

And in the VM logs were these entries:

!!Assertion Failed!!
Expression: pVM->pVMR0 == CreateVMReq.pVMR0
Location  : /home/vbox/tinderbox/6.0-lnx64-rel/src/VBox/VMM/VMMR3/VM.cpp(591) int vmR3CreateU(UVM*, uint32_t, int (*)(UVM*, VM*, void*), void*)
Stack     :
00007f8970304133 VBoxRT.so + 0x1f1133

For me this was solved by disabling AMDs Secure Memory Encryption (SME) .You can do this in the BIOS/UEFI Setup, via the linux boot parameter mem_encrypt=off or by disabling this during in your next self-compiled kernel.

This was solved by trying to work around a kernel/firmware/AMD bug which caused errors when trimming/discarding from a NVME with an activated IOMMU. After setting the iommu to passthrough discarding worked, but the Broadcom/LSI/Avago RAID controllers and the Radeon driver failed to start:

mpt3sas 0000:09:00.0: SME is active, device will require DMA bounce buffers
mpt2sas_cm0: reply_post_free pool: dma_pool_alloc failed
mpt2sas_cm0: failure at drivers/scsi/mpt3sas/mpt3sas_scsih.c:10506/_scsih_probe()!
radeon 0000:07:00.0: SME is active, device will require DMA bounce buffers

radeon 0000:07:00.0: SME is active, device will require DMA bounce buffers

software IO TLB: SME is active and system is using DMA bounce buffers

[drm:r600_ring_test [radeon]] *ERROR* radeon: ring 0 test failed (scratch(0x8504)=0xCAFEDEAD)

radeon 0000:07:00.0: disabling GPU acceleration

And after disabling SME VirtualBox also worked..

Given my fun with my Lenovo notebook last year and the unblogged fun I have/had with my successor Dell notebook, it seems that the AMD IOMMU is quite a PITA.

Tags:

amd