Protection Strategies for Direct Access to Virtualized I/O Devices ------------------------------------------------------------------ Five different strategies: - Single-use I/O memory mappings (create and destroy before and after each access) - Shared I/O mappings (can be reused by multiple concurrent I/O operations) - Persistent I/O mappings: Keep a mapping around for longer than the I/O completes. - Static Direct Map of the guest's physical memory - Software-based strategy which uses VMM-managed DMA descriptors DMA Descriptor: each DMA descriptor typically includes an address, a length, and a few device-specific flags. For example, to transmit a network packet, the network interface's device driver might create two DMA descriptors, one pointing to the packet headers and the other pointing to the packet's payload. Three potential memory access violations: - Device driver creates DMA descriptor with incorrect address causing device to fault - Device driver could create a DMA descriptor pointing to some other unsuited memory region (for example, the OS could have repurposed an original DMA target location) - Device (hardware) itself may be buggy and could initiate DMA to some arbitrary guest location Nearly 85% of all system crashes of the Windows XP operating system are caused by drivers The VMM or OS must also maintain another structure for the IOMMU that maps each I/O device to one of the page tables. Although every device must have an entry in this mapping, not every device must have its own page table. If two or more devices are entitled to access the same memory, then they can share a single page table. Single-Use Mappings: Even when separate DMA descriptors refer to the same physical page, the single-use strategy always creates distinct IOMMU mappings for each descriptor. Discuss the 10 steps for an IOMMU-assisted virtual I/O operation in Section 3.1 (page 18) Shared Mappings: Check if mapping exists before creation. Similarly, check if mapping in use before destruction (e.g., maintain reference count for each mapping). For reused mapping, can skip steps 1-4 and 7-10. Persistent Mappings: Introduces a window of vulnerability due to stale mappings. To limit this exposure, the guest OS can implement a reclamation policy that eventually removes mappings that are not currently in use by an I/O operation. Direct Map: Simple. offers no intra-guest protection. Software-based Protection: The guest should use VMM-specific DMA descriptors. The VMM can then make only those regions referenced by the DMA descriptor available to the device by programming the host DMA descriptor appropriately. This is a paravirtualization approach and requires changes to guest device driver. All these five strategies provide inter-guest protection. The only exception is if the device is faulty and is shared among multiple guests. Then the device can potentially write to the other guest's memory (there is no IOMMU). Intra-guest protection: Direct-map offers none. "In all other strategies, at least one specific request for memory by the driver is required before the OS will approve construction of the necessary IOMMU mapping, and hence, all other strategies can protect against bad-address faults." The distinction between "bad-address" and "invalid-use" is fuzzy. Invalid Use: In all IOMMU strategies, the driver is responsible for informing the OS when it is done with an IOMMU mapping. Even if the OS was modified to automatically revoke an IOMMU mapping when it detected the completion of a corresponding I/O event (e.g., sendfile() or free()), the driver could still invalidly reuse a mapping after the original I/O event finished, but before the OS could intervene to terminate the IOMMU mapping. In the software strategy, however, the VMM automatically detects when the device has completed a specific I/O transaction and ensures that individual DMA descriptors can never be reused. Discuss Table 2 -- differences between different IOMMU architectures. Table 3: Notice that the CPU utilization for implementing protection is highest in the "Single-Use" strategy. Surprisingly, the overhead incurred by the software-based technique is noticeably less than the IOMMU-based shared-mapping and single-use strategies. The software-based technique certainly requires far more hypercalls per DMA than the IOMMU-based strategies. However, the software strategy does not incur the additional overhead of flushing the IOMMU's IOTLB via a programmed I/O write. VoIP Server: Sends and receives very small TCP messages in order to initiate and terminate VoIP phone calls. Hence reuse is only 4% compared to 39% in TCP streaming transmit. Table 5: Larger average messages lead to larger amounts of reuse for transmitted buffers under the shared strategy. write() versue sendfile(): write() copies to kernel buffers before invoking device driver. Zero-copy sendfile() appends the application's file buffers (rather than copying) allowing any pointer in guest physical memory to be a potential DMA target. For transmitted packet data, spatial reuse happens when either the packet size is less than that of a physical page, or when the network stack and network card are not utilizing TCP Segmentation Offloading. For packet headers prepended onto zero-copy sendfile packets, spatial reuse can be fairly common because many of the small TCP/IP headers can be allocated from a single physical page. Spatial and temporal use of skbuffs is entiredly dependent on the behaviour of the skbuff allocator.