Hi guys!
I installed Ubuntu 12.04 LTS(Kernel version: 3.8.1) and Xen 4.4 to make VM using VF.
I use 2 82599 10G nic.
So I made some VFs and set "iommu=1" in GRUB like these.
root@NewXebra2:/guest# lspci | grep Eth
01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
82:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
82:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
82:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
82:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
82:10.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
82:10.3 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
82:10.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
82:10.5 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
82:10.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
82:10.7 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
84:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
84:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
84:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
84:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
84:10.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
84:10.3 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
84:10.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
84:10.5 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
84:10.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
84:10.7 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
And I also made VM(Paravirtualized Ubuntu 10.04 LTS) with 2 VFs. But...
it doesn't work...
Below, there are some information of Eth0(VF) by ethtool.
driver: ixgbevf
version: 2.11.3
firmware-version: N/A
bus-info: 0000:00:00.0
As you can see, bus address is null.
I also checked xen & ubuntu dmesg, but I can't find problem.
This is xen dmesg.
(XEN) Processor #34 6:12 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x09] lapic_id[0x24] enabled)
(XEN) Processor #36 6:12 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x30] enabled)
(XEN) Processor #48 6:12 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x32] enabled)
(XEN) Processor #50 6:12 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x34] enabled)
(XEN) Processor #52 6:12 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x01] enabled)
(XEN) Processor #1 6:12 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x03] enabled)
(XEN) Processor #3 6:12 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x05] enabled)
(XEN) Processor #5 6:12 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x10] lapic_id[0x11] enabled)
(XEN) Processor #17 6:12 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x11] lapic_id[0x13] enabled)
(XEN) Processor #19 6:12 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x12] lapic_id[0x15] enabled)
(XEN) Processor #21 6:12 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x13] lapic_id[0x21] enabled)
(XEN) Processor #33 6:12 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x14] lapic_id[0x23] enabled)
(XEN) Processor #35 6:12 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x15] lapic_id[0x25] enabled)
(XEN) Processor #37 6:12 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x16] lapic_id[0x31] enabled)
(XEN) Processor #49 6:12 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x17] lapic_id[0x33] enabled)
(XEN) Processor #51 6:12 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x18] lapic_id[0x35] enabled)
(XEN) Processor #53 6:12 APIC version 21
(XEN) ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
(XEN) Overriding APIC driver with bigsmp
(XEN) ACPI: IOAPIC (id[0x06] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 6, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode: Phys. Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a301 base: 0xfed00000
(XEN) Failed to get Error Log Address Range.
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 24 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 24 GSI, 4600 MSI/MSI-X
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 3066.814 MHz processor.
(XEN) Initing memory sharing.
(XEN) mce_intel.c:717: MCA Capability: BCAST 1 SER 0 CMCI 1 firstbank 0 extended MCE MSR 0
(XEN) Intel machine check reporting enabled
(XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - ff
(XEN) PCI: MCFG area at e0000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-ff
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB.
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB.
(XEN) Intel VT-d Snoop Control enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Shared EPT tables not enabled.
(XEN) I/O virtualisation enabled
(XEN) - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using old ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) Platform timer is 14.318MHz HPET
(XEN) Defaulting to alternative key handling; send 'A' to switch to normal mode.
(XEN) Allocated console ring of 256 KiB.
(XEN) mwait-idle: MWAIT substates: 0x1120
(XEN) mwait-idle: v0.4 model 0x2c
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) VMX: Supported advanced features:
(XEN) - APIC MMIO access virtualisation
(XEN) - APIC TPR shadow
(XEN) - Extended Page Tables (EPT)
(XEN) - Virtual-Processor Identifiers (VPID)
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) Brought up 24 CPUs
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) *** LOADING DOMAIN 0 ***
(XEN) elf_parse_binary: phdr: paddr=0x1000000 memsz=0xb03000
(XEN) elf_parse_binary: phdr: paddr=0x1c00000 memsz=0xef0f0
(XEN) elf_parse_binary: phdr: paddr=0x1cf0000 memsz=0x14c40
(XEN) elf_parse_binary: phdr: paddr=0x1d05000 memsz=0x63e000
(XEN) elf_parse_binary: memory: 0x1000000 -> 0x2343000
(XEN) elf_xen_parse_note: GUEST_OS = "linux"
(XEN) elf_xen_parse_note: GUEST_VERSION = "2.6"
(XEN) elf_xen_parse_note: XEN_VERSION = "xen-3.0"
(XEN) elf_xen_parse_note: VIRT_BASE = 0xffffffff80000000
(XEN) elf_xen_parse_note: ENTRY = 0xffffffff81d05210
(XEN) elf_xen_parse_note: HYPERCALL_PAGE = 0xffffffff81001000
(XEN) elf_xen_parse_note: FEATURES = "!writable_page_tables|pae_pgdir_above_4gb"
(XEN) elf_xen_parse_note: PAE_MODE = "yes"
(XEN) elf_xen_parse_note: LOADER = "generic"
(XEN) elf_xen_parse_note: unknown xen elf note (0xd)
(XEN) elf_xen_parse_note: SUSPEND_CANCEL = 0x1
(XEN) elf_xen_parse_note: HV_START_LOW = 0xffff800000000000
(XEN) elf_xen_parse_note: PADDR_OFFSET = 0x0
(XEN) elf_xen_addr_calc_check: addresses:
(XEN) virt_base = 0xffffffff80000000
(XEN) elf_paddr_offset = 0x0
(XEN) virt_offset = 0xffffffff80000000
(XEN) virt_kstart = 0xffffffff81000000
(XEN) virt_kend = 0xffffffff82343000
(XEN) virt_entry = 0xffffffff81d05210
(XEN) p2m_base = 0xffffffffffffffff
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2343000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 0000000c00000000->0000000c04000000 (395510 pages to be allocated)
(XEN) Init. ramdisk: 0000000c248f6000->0000000c3ffff800
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff81000000->ffffffff82343000
(XEN) Init. ramdisk: ffffffff82343000->ffffffff9da4c800
(XEN) Phys-Mach map: ffffffff9da4d000->ffffffff9de4d000
(XEN) Start info: ffffffff9de4d000->ffffffff9de4d4b4
(XEN) Page tables: ffffffff9de4e000->ffffffff9df41000
(XEN) Boot stack: ffffffff9df41000->ffffffff9df42000
(XEN) TOTAL: ffffffff80000000->ffffffff9e000000
(XEN) ENTRY ADDRESS: ffffffff81d05210
(XEN) Dom0 has maximum 24 VCPUs
(XEN) elf_load_binary: phdr 0 at 0xffffffff81000000 -> 0xffffffff81b03000
(XEN) elf_load_binary: phdr 1 at 0xffffffff81c00000 -> 0xffffffff81cef0f0
(XEN) elf_load_binary: phdr 2 at 0xffffffff81cf0000 -> 0xffffffff81d04c40
(XEN) elf_load_binary: phdr 3 at 0xffffffff81d05000 -> 0xffffffff81def000
(XEN) Scrubbing Free RAM: ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 252kB init memory.
(XEN) PCI add device 0000:00:00.0
(XEN) PCI add device 0000:00:01.0
(XEN) PCI add device 0000:00:03.0
(XEN) PCI add device 0000:00:07.0
(XEN) PCI add device 0000:00:09.0
(XEN) PCI add device 0000:00:14.0
(XEN) PCI add device 0000:00:16.0
(XEN) PCI add device 0000:00:16.1
(XEN) PCI add device 0000:00:16.2
(XEN) PCI add device 0000:00:16.3
(XEN) PCI add device 0000:00:16.4
(XEN) PCI add device 0000:00:16.5
(XEN) PCI add device 0000:00:16.6
(XEN) PCI add device 0000:00:16.7
(XEN) PCI add device 0000:00:1a.0
(XEN) PCI add device 0000:00:1a.1
(XEN) PCI add device 0000:00:1a.2
(XEN) PCI add device 0000:00:1a.7
(XEN) PCI add device 0000:00:1b.0
(XEN) PCI add device 0000:00:1c.0
(XEN) PCI add device 0000:00:1d.0
(XEN) PCI add device 0000:00:1d.1
(XEN) PCI add device 0000:00:1d.2
(XEN) PCI add device 0000:00:1d.7
(XEN) PCI add device 0000:00:1e.0
(XEN) PCI add device 0000:00:1f.0
(XEN) PCI add device 0000:00:1f.2
(XEN) PCI add device 0000:00:1f.3
(XEN) PCI add device 0000:00:1f.5
(XEN) PCI add device 0000:01:00.0
(XEN) PCI add device 0000:01:00.1
(XEN) PCI add device 0000:06:00.0
(XEN) PCI add device 0000:07:05.0
(XEN) PCI add device 0000:80:00.0
(XEN) PCI add device 0000:80:01.0
(XEN) PCI add device 0000:80:03.0
(XEN) PCI add device 0000:80:05.0
(XEN) PCI add device 0000:80:07.0
(XEN) PCI add device 0000:80:14.0
(XEN) PCI add device 0000:80:14.1
(XEN) PCI add device 0000:80:14.2
(XEN) PCI add device 0000:80:14.3
(XEN) PCI add device 0000:80:16.0
(XEN) PCI add device 0000:80:16.1
(XEN) PCI add device 0000:80:16.2
(XEN) PCI add device 0000:80:16.3
(XEN) PCI add device 0000:80:16.4
(XEN) PCI add device 0000:80:16.5
(XEN) PCI add device 0000:80:16.6
(XEN) PCI add device 0000:80:16.7
(XEN) PCI add device 0000:82:00.0
(XEN) PCI add device 0000:82:00.1
(XEN) PCI add device 0000:84:00.0
(XEN) PCI add device 0000:84:00.1
(XEN) PCI add device 0000:87:00.0
(XEN) PCI add device 0000:87:00.1
(XEN) PCI add device 0000:fe:00.0
(XEN) PCI add device 0000:fe:00.1
(XEN) PCI add device 0000:fe:02.0
(XEN) PCI add device 0000:fe:02.1
(XEN) PCI add device 0000:fe:02.2
(XEN) PCI add device 0000:fe:02.3
(XEN) PCI add device 0000:fe:02.4
(XEN) PCI add device 0000:fe:02.5
(XEN) PCI add device 0000:fe:03.0
(XEN) PCI add device 0000:fe:03.1
(XEN) PCI add device 0000:fe:03.2
(XEN) PCI add device 0000:fe:03.4
(XEN) PCI add device 0000:fe:04.0
(XEN) PCI add device 0000:fe:04.1
(XEN) PCI add device 0000:fe:04.2
(XEN) PCI add device 0000:fe:04.3
(XEN) PCI add device 0000:fe:05.0
(XEN) PCI add device 0000:fe:05.1
(XEN) PCI add device 0000:fe:05.2
(XEN) PCI add device 0000:fe:05.3
(XEN) PCI add device 0000:fe:06.0
(XEN) PCI add device 0000:fe:06.1
(XEN) PCI add device 0000:fe:06.2
(XEN) PCI add device 0000:fe:06.3
(XEN) PCI add device 0000:ff:00.0
(XEN) PCI add device 0000:ff:00.1
(XEN) PCI add device 0000:ff:02.0
(XEN) PCI add device 0000:ff:02.1
(XEN) PCI add device 0000:ff:02.2
(XEN) PCI add device 0000:ff:02.3
(XEN) PCI add device 0000:ff:02.4
(XEN) PCI add device 0000:ff:02.5
(XEN) PCI add device 0000:ff:03.0
(XEN) PCI add device 0000:ff:03.1
(XEN) PCI add device 0000:ff:03.2
(XEN) PCI add device 0000:ff:03.4
(XEN) PCI add device 0000:ff:04.0
(XEN) PCI add device 0000:ff:04.1
(XEN) PCI add device 0000:ff:04.2
(XEN) PCI add device 0000:ff:04.3
(XEN) PCI add device 0000:ff:05.0
(XEN) PCI add device 0000:ff:05.1
(XEN) PCI add device 0000:ff:05.2
(XEN) PCI add device 0000:ff:05.3
(XEN) PCI add device 0000:ff:06.0
(XEN) PCI add device 0000:ff:06.1
(XEN) PCI add device 0000:ff:06.2
(XEN) PCI add device 0000:ff:06.3
(XEN) PCI add virtual function 0000:82:10.0
(XEN) PCI add virtual function 0000:82:10.2
(XEN) PCI add virtual function 0000:82:10.4
(XEN) PCI add virtual function 0000:82:10.6
(XEN) PCI add virtual function 0000:82:11.0
(XEN) PCI add virtual function 0000:82:11.2
(XEN) PCI add virtual function 0000:82:11.4
(XEN) PCI add virtual function 0000:82:10.1
(XEN) PCI add virtual function 0000:82:10.3
(XEN) PCI add virtual function 0000:82:10.5
(XEN) PCI add virtual function 0000:82:10.7
(XEN) PCI add virtual function 0000:82:11.1s-
(XEN) PCI add virtual function 0000:82:11.3
(XEN) PCI add virtual function 0000:82:11.5
If you have any solutions, please....
Sincerely.