KVM and VGA Passthrough problem - Can't unbind


#1

Hi all,

I’m facing a problem. I would attach my grahic card to a KVM guest via PCI Passthrough.
Here is my configuration :
Host: Debian 2.6.32-5-amd64 #1 SMP Mon Mar 7 21:35:22 UTC 2011 x86_64 GNU/Linux
Intel i5 Processor, VT-d ready
VT-d option active in BIOS (the following command show it)

#dmesg | grep -e DMAR -e IOMMU 
    0.044867] DMAR: Host address width 36
    0.044869] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
    0.044874] IOMMU fed90000: ver 1:0 cap c9008020e30272 ecap 1000
    0.044876] DMAR: DRHD base: 0x000000fed93000 flags: 0x1
    0.044881] IOMMU fed93000: ver 1:0 cap c9008020630272 ecap 1000
    0.044883] DMAR: RMRR base: 0x00000000000000 end: 0x00000000000fff
    0.044884] DMAR: No ATSR found

References: http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM

The problem occurs when I try to unbind the graphical card : The screen goes black as if the card cas successfully removed of the host, but dmesg immediately print this backtrace :

  843.862619] Console: switching to colour VGA+ 80x25
  843.863105] unregistered panic notifier
  845.006494] ------------ cut here ]------------
  845.006500] kernel BUG at /build/buildd-linux-2.6_2.6.32-31-amd64-vrfdM4/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/gpu/drm/ttm/ttm_bo.c:1539!
  845.006505] invalid opcode: 0000 #1] SMP
  845.006510] last sysfs file: /sys/bus/pci/drivers/nouveau/unbind
  845.006513] CPU 1
  845.006515] Modules linked in: pci_stub ip6table_filter ip6_tables iptable_filter ip_tables ebtable_nat ebtables x_tables ppdev lp acpi_cpufreq sco cpufreq_stats bridge stp bnep cpufreq_conservative cpufreq_powersave rfcomm l2cap crc16 cpufreq_userspace binfmt_misc uinput fuse loop firewire_sbp2 arc4 snd_hda_codec_idt ecb iwlagn ide_cs snd_hda_intel iwlcore snd_hda_codec ide_core snd_hwdep snd_pcm nouveau mac80211 snd_seq snd_timer pcmcia uvcvideo snd_seq_device btusb ttm snd cfg80211 joydev soundcore bluetooth hp_accel videodev video snd_page_alloc v4l1_compat yenta_socket drm_kms_helper tpm_infineon v4l2_compat_ioctl32 output rsrc_nonstatic rfkill parport_pc lis3lv02d parport psmouse tpm drm tpm_bios pcmcia_core pcspkr serio_raw i2c_algo_bit i2c_core wmi input_polldev evdev ac button battery container processor ext3 jbd mbcache dm_mod sg sr_mod sd_mod cdrom crc_t10dif sdhci_pci sdhci ahci xhci libata firewire_ohci mmc_core ricoh_mmc firewire_core led_class crc_itu_t scsi_mod ehci_hcd e1000e usbcore nls_base thermal thermal_sys [last unloaded: kvm]
  845.006591] Pid: 2623, comm: libvirtd Not tainted 2.6.32-5-amd64 #1 HP EliteBook 8540p
  845.006593] RIP: 0010:<ffffffffa02c092f>]  <ffffffffa02c092f>] ttm_bo_device_release+0x10a/0x13d [ttm]
  845.006599] RSP: 0018:ffff88011b809d58  EFLAGS: 00010246
  845.006601] RAX: 0000000000000000 RBX: ffff880136466e08 RCX: 00000000c0000104
  845.006602] RDX: 00000000000017e8 RSI: 0000000000000001 RDI: ffff880135980510
  845.006604] RBP: ffff8801359800d0 R08: ffff880136466de8 R09: ffffffff814d68e0
  845.006605] R10: ffff88000524fae0 R11: 0000000000000400 R12: ffff880135980500
  845.006606] R13: ffff880136466d80 R14: 0000000000000000 R15: ffffffff8149b620
  845.006608] FS:  00007f98ab7fe700(0000) GS:ffff880005240000(0000) knlGS:0000000000000000
  845.006610] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  845.006611] CR2: 00007fde654bd000 CR3: 00000001343f2000 CR4: 00000000000006e0
  845.006612] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  845.006614] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  845.006615] Process libvirtd (pid: 2623, threadinfo ffff88011b808000, task ffff8801343fe2e0)
  845.006616] Stack:
  845.006617]  ffff8801359800d0 ffff88013399c000 ffff88013399c020 ffff880135980000
  845.006620] <0> ffff8801343416c0 ffffffffa032701c ffff88013399c000 ffff8801359800d0
  845.006622] <0> ffff88013399c000 ffff88013399c020 ffff880135980000 ffffffffa0324b76
  845.006624] Call Trace:
  845.006633]  <ffffffffa032701c>] ? nouveau_mem_close+0x34/0x142 [nouveau]
  845.006638]  <ffffffffa0324b76>] ? nouveau_card_takedown+0xfe/0x152 [nouveau]
  845.006642]  <ffffffffa0324c0e>] ? nouveau_unload+0x44/0x70 [nouveau]
  845.006649]  <ffffffffa01bf37d>] ? drm_put_dev+0xdc/0x1a7 [drm]
  845.006654]  <ffffffff811a2d81>] ? pci_device_remove+0x27/0x4a
  845.006658]  <ffffffff81220bb0>] ? __device_release_driver+0x96/0xeb
  845.006660]  <ffffffff81220cbd>] ? device_release_driver+0x1e/0x2a
  845.006663]  <ffffffff812205c9>] ? driver_unbind+0x56/0x98
  845.006666]  <ffffffff8113f078>] ? sysfs_write_file+0xd0/0x107
  845.006669]  <ffffffff810ef2b8>] ? vfs_write+0xa9/0x102
  845.006671]  <ffffffff810ef3cd>] ? sys_write+0x45/0x6e
  845.006674]  <ffffffff81010b42>] ? system_call_fastpath+0x16/0x1b
  845.006675] Code: ef 49 8d 9d 88 00 00 00 48 89 df e8 8f b8 03 e1 66 41 ff 85 88 00 00 00 4c 8d a5 30 04 00 00 4c 89 e7 e8 d4 1d f0 ff 85 c0 75 04 <0f> 0b eb fe 48 8d 5d 20 48 89 df e8 7f b8 03 e1 4c 89 e7 e8 1a
  845.006691] RIP  <ffffffffa02c092f>] ttm_bo_device_release+0x10a/0x13d [ttm]
  845.006695]  RSP <ffff88011b809d58>
  845.006697] --- end trace 7a4a82736647bf39 ]---
 1149.327156] BUG: unable to handle kernel NULL pointer dereference at 0000000000000008
 1149.327400] IP: <ffffffffa01c3436>] drm_fb_release+0x65/0xa9 [drm]
 1149.327578] PGD 0
 1149.327729] Oops: 0002 #2] SMP
 1149.327967] last sysfs file: /sys/bus/pci/drivers/nouveau/unbind
 1149.328068] CPU 0
 1149.328232] Modules linked in: pci_stub ip6table_filter ip6_tables iptable_filter ip_tables ebtable_nat ebtables x_tables ppdev lp acpi_cpufreq sco cpufreq_stats bridge stp bnep cpufreq_conservative cpufreq_powersave rfcomm l2cap crc16 cpufreq_userspace binfmt_misc uinput fuse loop firewire_sbp2 arc4 snd_hda_codec_idt ecb iwlagn ide_cs snd_hda_intel iwlcore snd_hda_codec ide_core snd_hwdep snd_pcm nouveau mac80211 snd_seq snd_timer pcmcia uvcvideo snd_seq_device btusb ttm snd cfg80211 joydev soundcore bluetooth hp_accel videodev video snd_page_alloc v4l1_compat yenta_socket drm_kms_helper tpm_infineon v4l2_compat_ioctl32 output rsrc_nonstatic rfkill parport_pc lis3lv02d parport psmouse tpm drm tpm_bios pcmcia_core pcspkr serio_raw i2c_algo_bit i2c_core wmi input_polldev evdev ac button battery container processor ext3 jbd mbcache dm_mod sg sr_mod sd_mod cdrom crc_t10dif sdhci_pci sdhci ahci xhci libata firewire_ohci mmc_core ricoh_mmc firewire_core led_class crc_itu_t scsi_mod ehci_hcd e1000e usbcore nls_base thermal thermal_sys [last unloaded: kvm]
 1149.337016] Pid: 1582, comm: Xorg Tainted: G      D    2.6.32-5-amd64 #1 HP EliteBook 8540p
 1149.337137] RIP: 0010:<ffffffffa01c3436>]  <ffffffffa01c3436>] drm_fb_release+0x65/0xa9 [drm]
 1149.337335] RSP: 0018:ffff880136083c38  EFLAGS: 00010283
 1149.337434] RAX: 0000000000000000 RBX: ffff880134b9ed40 RCX: dead000000200200
 1149.337535] RDX: 0000000000000000 RSI: dead000000100100 RDI: ffff880137164900
 1149.337635] RBP: ffffffffffffff70 R08: 0000000000000000 R09: ffff88013399c000
 1149.337736] R10: 0000000000000292 R11: ffffffff810c2cb5 R12: ffff88013399c408
 1149.337836] R13: ffff880134b9ecc0 R14: ffff88013399c000 R15: ffff88013399c300
 1149.337937] FS:  0000000000000000(0000) GS:ffff880005200000(0000) knlGS:0000000000000000
 1149.338054] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 1149.338151] CR2: 0000000000000008 CR3: 0000000001001000 CR4: 00000000000006f0
 1149.338252] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
 1149.338351] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
 1149.338451] Process Xorg (pid: 1582, threadinfo ffff880136082000, task ffff880134008e20)
 1149.338565] Stack:
 1149.338655]  ffff880134b9ed60 ffff88013399c000 ffff880134b9ed68 ffffffffa01bb9e8
 1149.338981] <0> 0000000000000001 0000000000000296 ffff88013399c310 0000000000000008
 1149.339459] <0> ffff880134b9ef00 ffff8801374d9c00 ffff88013388b028 ffff88013388b028
 1149.340031] Call Trace:
 1149.340137]  <ffffffffa01bb9e8>] ? drm_release+0x385/0x5f5 [drm]
 1149.340245]  <ffffffff810efb91>] ? __fput+0x100/0x1af
 1149.340348]  <ffffffff810ecff6>] ? filp_close+0x5b/0x62
 1149.340454]  <ffffffff8104fa04>] ? put_files_struct+0x64/0xc1
 1149.340560]  <ffffffff810512c9>] ? do_exit+0x236/0x6c6
 1149.340665]  <ffffffff810e5af1>] ? virt_to_head_page+0x9/0x2a
 1149.340770]  <ffffffff810517cf>] ? do_group_exit+0x76/0x9d
 1149.340876]  <ffffffff8105e05f>] ? get_signal_to_deliver+0x310/0x339
 1149.340984]  <ffffffff8101004f>] ? do_notify_resume+0x87/0x73f
 1149.341090]  <ffffffff810fec98>] ? dput+0x40/0x15e
 1149.341192]  <ffffffff8105b52b>] ? sigprocmask+0x9b/0xc3
 1149.341295]  <ffffffff8105b848>] ? sys_rt_sigprocmask+0x4c/0xc4
 1149.341399]  <ffffffff8105b52b>] ? sigprocmask+0x9b/0xc3
 1149.341503]  <ffffffff81010e0e>] ? int_signal+0x12/0x17
 1149.341601] Code: 48 8d a8 70 ff ff ff eb 4e 48 8b 87 98 00 00 00 48 8b 97 90 00 00 00 48 be 00 01 10 00 00 00 ad de 48 b9 00 02 20 00 00 00 ad de <48> 89 42 08 48 89 10 48 8b 47 20 48 89 b7 90 00 00 00 48 89 8f
 1149.346559] RIP  <ffffffffa01c3436>] drm_fb_release+0x65/0xa9 [drm]
 1149.346741]  RSP <ffff880136083c38>
 1149.346835] CR2: 0000000000000008
 1149.346949] --- end trace 7a4a82736647bf3a ]---

The following manipulation doesn’t work, and I suppose this is linked to the previous error :
Pour Rappel voici les commandes que je tente d’effectuer :

    echo "10de 0a2c" > /sys/bus/pci/drivers/pci-stub/new_id
    echo 0000:01:00.0 > /sys/bus/pci/devices/0000:01:00.0/driver/unbind
    echo 0000:01:00.0 > /sys/bus/pci/drivers/pci-stub/bind [/code]

When I try to bind the PCI device, I get "Write Error : No such device"

Here is the listing of my PCI devices, with and without name resolution :

[code]aerendil@aerendil-domU:~$ lspci
00:00.0 Host bridge: Intel Corporation Core Processor DRAM Controller (rev 02)
00:01.0 PCI bridge: Intel Corporation Core Processor PCI Express x16 Root Port (rev 02)
00:16.0 Communication controller: Intel Corporation 5 Series/3400 Series Chipset HECI Controller (rev 06)
00:19.0 Ethernet controller: Intel Corporation 82577LM Gigabit Network Connection (rev 05)
00:1a.0 USB Controller: Intel Corporation 5 Series/3400 Series Chipset USB2 Enhanced Host Controller (rev 05)
00:1b.0 Audio device: Intel Corporation 5 Series/3400 Series Chipset High Definition Audio (rev 05)
00:1c.0 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 1 (rev 05)
00:1c.1 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 2 (rev 05)
00:1c.3 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 4 (rev 05)
00:1c.7 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 8 (rev 05)
00:1d.0 USB Controller: Intel Corporation 5 Series/3400 Series Chipset USB2 Enhanced Host Controller (rev 05)
00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev a5)
00:1f.0 ISA bridge: Intel Corporation Mobile 5 Series Chipset LPC Interface Controller (rev 05)
00:1f.2 SATA controller: Intel Corporation 5 Series/3400 Series Chipset 6 port SATA AHCI Controller (rev 05)
00:1f.6 Signal processing controller: Intel Corporation 5 Series/3400 Series Chipset Thermal Subsystem (rev 05)
01:00.0 VGA compatible controller: nVidia Corporation GT216 [NVS 5100M] (rev a2)
01:00.1 Audio device: nVidia Corporation High Definition Audio Controller (rev a1)
44:00.0 Network controller: Intel Corporation Centrino Advanced-N 6200 (rev 35)
45:00.0 USB Controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 03)
46:06.0 FireWire (IEEE 1394): Ricoh Co Ltd R5C832 IEEE 1394 Controller (rev 06)
46:06.1 SD Host controller: Ricoh Co Ltd R5C822 SD/SDIO/MMC/MS/MSPro Host Adapter (rev 25)
46:06.2 System peripheral: Ricoh Co Ltd R5C843 MMC Host Controller (rev 14)
46:06.3 System peripheral: Ricoh Co Ltd R5C592 Memory Stick Bus Host Adapter (rev 14)
46:06.4 System peripheral: Ricoh Co Ltd xD-Picture Card Controller (rev bb)
46:06.5 CardBus bridge: Ricoh Co Ltd RL5c476 II (rev ff)
ff:00.0 Host bridge: Intel Corporation Core Processor QuickPath Architecture Generic Non-core Registers (rev 02)
ff:00.1 Host bridge: Intel Corporation Core Processor QuickPath Architecture System Address Decoder (rev 02)
ff:02.0 Host bridge: Intel Corporation Core Processor QPI Link 0 (rev 02)
ff:02.1 Host bridge: Intel Corporation Core Processor QPI Physical 0 (rev 02)
ff:02.2 Host bridge: Intel Corporation Core Processor Reserved (rev 02)
ff:02.3 Host bridge: Intel Corporation Core Processor Reserved (rev 02)
aerendil@aerendil-domU:~$ lspci -n
00:00.0 0600: 8086:0044 (rev 02)
00:01.0 0604: 8086:0045 (rev 02)
00:16.0 0780: 8086:3b64 (rev 06)
00:19.0 0200: 8086:10ea (rev 05)
00:1a.0 0c03: 8086:3b3c (rev 05)
00:1b.0 0403: 8086:3b56 (rev 05)
00:1c.0 0604: 8086:3b42 (rev 05)
00:1c.1 0604: 8086:3b44 (rev 05)
00:1c.3 0604: 8086:3b48 (rev 05)
00:1c.7 0604: 8086:3b50 (rev 05)
00:1d.0 0c03: 8086:3b34 (rev 05)
00:1e.0 0604: 8086:2448 (rev a5)
00:1f.0 0601: 8086:3b07 (rev 05)
00:1f.2 0106: 8086:3b2f (rev 05)
00:1f.6 1180: 8086:3b32 (rev 05)
01:00.0 0300: 10de:0a2c (rev a2)
01:00.1 0403: 10de:0be2 (rev a1)
44:00.0 0280: 8086:4239 (rev 35)
45:00.0 0c03: 1033:0194 (rev 03)
46:06.0 0c00: 1180:0832 (rev 06)
46:06.1 0805: 1180:0822 (rev 25)
46:06.2 0880: 1180:0843 (rev 14)
46:06.3 0880: 1180:0592 (rev 14)
46:06.4 0880: 1180:0852 (rev bb)
46:06.5 0607: 1180:0476 (rev ff)
ff:00.0 0600: 8086:2c62 (rev 02)
ff:00.1 0600: 8086:2d01 (rev 02)
ff:02.0 0600: 8086:2d10 (rev 02)
ff:02.1 0600: 8086:2d11 (rev 02)
ff:02.2 0600: 8086:2d12 (rev 02)
ff:02.3 0600: 8086:2d13 (rev 02)

I’ll be please if anyone have an idea :slight_smile:
If you need any information, just tell me !

Thanks in advance