__________________________________________________________________

Copyright © OASIS Open 2023. All Rights Reserved.

All capitalized terms in the following text have the meanings assigned to them in the OASIS Intellectual Property Rights Policy (the "OASIS IPR Policy"). The full Policy may be found at the OASIS website.

This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published, and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this section are included on all such copies and derivative works. However, this document itself may not be modified in any way, including by removing the copyright notice or references to OASIS, except as needed for the purpose of developing any document or deliverable produced by an OASIS Technical Committee (in which case the rules applicable to copyrights, as set forth in the OASIS IPR Policy, must be followed) or as required to translate it into languages other than English.

The limited permissions granted above are perpetual and will not be revoked by OASIS or its successors or assigns.

This document and the information contained herein is provided on an "AS IS" basis and OASIS DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY OWNERSHIP RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. OASIS AND ITS MEMBERS WILL NOT BE LIABLE FOR ANY DIRECT, INDIRECT, SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF ANY USE OF THIS DOCUMENT OR ANY PART THEREOF.

As stated in the OASIS IPR Policy, the following three paragraphs in brackets apply to OASIS Standards Final Deliverable documents (Committee Specifications, OASIS Standards, or Approved Errata).

[OASIS requests that any OASIS Party or any other party that believes it has patent claims that would necessarily be infringed by implementations of this OASIS Committee Specification or OASIS Standard, to notify OASIS TC Administrator and provide an indication of its willingness to grant patent licenses to such patent claims in a manner consistent with the IPR Mode of the OASIS Technical Committee that produced this specification.]

[OASIS invites any party to contact the OASIS TC Administrator if it is aware of a claim of ownership of any patent claims that would necessarily be infringed by implementations of this specification by a patent holder that is not willing to provide a license to such patent claims in a manner consistent with the IPR Mode of the OASIS Technical Committee that produced this specification. OASIS may include such claims on its website, but disclaims any obligation to do so.]

[OASIS takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on OASIS’ procedures with respect to rights in any document or deliverable produced by an OASIS Technical Committee can be found on the OASIS website. Copies of claims of rights made available for publication and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this OASIS Committee Specification or OASIS Standard, can be obtained from the OASIS TC Administrator. OASIS makes no representation that any information or list of intellectual property rights will at any time be complete, or that any claims in such list are, in fact, Essential Claims.]

The name "OASIS" is a trademark of OASIS, the owner and developer of this specification, and should be used only to refer to the organization and its official outputs. OASIS welcomes reference to, and implementation and use of, specifications, while reserving the right to enforce its marks against misleading uses. Please see https://www.oasis-open.org/policies-guidelines/trademark/ for above guidance.

__________________________________________________________________

Table of Contents

---

1 Introduction

The purpose of virtio and this specification is that virtual environments and guests should have a straightforward, efficient, standard and extensible mechanism for virtual devices, rather than boutique per-environment or per-OS mechanisms.

Straightforward:

Virtio devices use normal bus mechanisms of interrupts and DMA which should be familiar to any device driver author. There is no exotic page-flipping or COW mechanism: it’s just a normal device.¹

Efficient:

Virtio devices consist of rings of descriptors for both input and output, which are neatly laid out to avoid cache effects from both driver and device writing to the same cache lines.

Standard:

Virtio makes no assumptions about the environment in which it operates, beyond supporting the bus to which device is attached. In this specification, virtio devices are implemented over MMIO, Channel I/O and PCI bus transports ², earlier drafts have been implemented on other buses not included here.

Extensible:

Virtio devices contain feature bits which are acknowledged by the guest operating system during device setup. This allows forwards and backwards compatibility: the device offers all the features it knows about, and the driver acknowledges those it understands and wishes to use.

1.1 Normative References

1.2 Non-Normative References

1.3 Terminology

The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “NOT RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC2119] and [RFC8174] when, and only when, they appear in all capitals, as shown here.

1.3.1 Legacy Interface: Terminology

Specification drafts preceding version 1.0 of this specification (e.g. see [Virtio PCI Draft]) defined a similar, but different interface between the driver and the device. Since these are widely deployed, this specification accommodates OPTIONAL features to simplify transition from these earlier draft interfaces.

Specifically devices and drivers MAY support:

Legacy Interface

is an interface specified by an earlier draft of this specification (before 1.0)

Legacy Device

is a device implemented before this specification was released, and implementing a legacy interface on the host side

Legacy Driver

is a driver implemented before this specification was released, and implementing a legacy interface on the guest side

Legacy devices and legacy drivers are not compliant with this specification.

To simplify transition from these earlier draft interfaces, a device MAY implement:

Transitional Device

a device supporting both drivers conforming to this specification, and allowing legacy drivers.

Similarly, a driver MAY implement:

Transitional Driver

a driver supporting both devices conforming to this specification, and legacy devices.

Devices or drivers with no legacy compatibility are referred to as non-transitional devices and drivers, respectively.

1.3.2 Transition from earlier specification drafts

For devices and drivers already implementing the legacy interface, some changes will have to be made to support this specification.

In this case, it might be beneficial for the reader to focus on sections tagged "Legacy Interface" in the section title. These highlight the changes made since the earlier drafts.

1.4 Structure Specifications

Many device and driver in-memory structure layouts are documented using the C struct syntax. All structures are assumed to be without additional padding. To stress this, cases where common C compilers are known to insert extra padding within structures are tagged using the GNU C __attribute__((packed)) syntax.

For the integer data types used in the structure definitions, the following conventions are used:

u8, u16, u32, u64

An unsigned integer of the specified length in bits.

le16, le32, le64

An unsigned integer of the specified length in bits, in little-endian byte order.

be16, be32, be64

An unsigned integer of the specified length in bits, in big-endian byte order.

Some of the fields to be defined in this specification don’t start or don’t end on a byte boundary. Such fields are called bit-fields. A set of bit-fields is always a sub-division of an integer typed field.

Bit-fields within integer fields are always listed in order, from the least significant to the most significant bit. The bit-fields are considered unsigned integers of the specified width with the next in significance relationship of the bits preserved.

For example:

struct S { 
 
        be16 { 
 
                A : 15; 
 
                B : 1; 
 
        } x; 
 
        be16 y; 
 
};

documents the value A stored in the low 15 bit of x and the value B stored in the high bit of x, the 16-bit integer x in turn stored using the big-endian byte order at the beginning of the structure S, and being followed immediately by an unsigned integer y stored in big-endian byte order at an offset of 2 bytes (16 bits) from the beginning of the structure.

Note that this notation somewhat resembles the C bitfield syntax but should not be naively converted to a bitfield notation for portable code: it matches the way bitfields are packed by C compilers on little-endian architectures but not the way bitfields are packed by C compilers on big-endian architectures.

Assuming that CPU_TO_BE16 converts a 16-bit integer from a native CPU to the big-endian byte order, the following is the equivalent portable C code to generate a value to be stored into x:

CPU_TO_BE16(B << 15 | A)

1.5 Constant Specifications

In many cases, numeric values used in the interface between the device and the driver are documented using the C #define and /* */ comment syntax. Multiple related values are grouped together with a common name as a prefix, using _ as a separator. Using _XXX as a suffix refers to all values in a group. For example:

/* Field Fld value A description */ 
 
#define VIRTIO_FLD_A        (1 << 0) 
 
/* Field Fld value B description */ 
 
#define VIRTIO_FLD_B        (1 << 1)

documents two numeric values for a field Fld, with Fld having value 1 referring to A and Fld having value 2 referring to B. Note that << refers to the shift-left operation.

Further, in this case VIRTIO_FLD_A and VIRTIO_FLD_B refer to values 1 and 2 of Fld respectively. Further, VIRTIO_FLD_XXX refers to either VIRTIO_FLD_A or VIRTIO_FLD_B.

---

2 Basic Facilities of a Virtio Device

• Device status field

• Feature bits

• Notifications

• Device Configuration space

• One or more virtqueues

2.1 Device Status Field

During device initialization by a driver, the driver follows the sequence of steps specified in 3.1.

The device status field provides a simple low-level indication of the completed steps of this sequence. It’s most useful to imagine it hooked up to traffic lights on the console indicating the status of each device. The following bits are defined (listed below in the order in which they would be typically set):

ACKNOWLEDGE (1)

Indicates that the guest OS has found the device and recognized it as a valid virtio device.

DRIVER (2)

Indicates that the guest OS knows how to drive the device. Note: There could be a significant (or infinite) delay before setting this bit. For example, under Linux, drivers can be loadable modules.

DRIVER_OK (4)

Indicates that the driver is set up and ready to drive the device.

FEATURES_OK (8)

Indicates that the driver has acknowledged all the features it understands, and feature negotiation is complete.

SUSPEND (16)

When VIRTIO_F_SUSPEND is negotiated, indicates that the device has been suspended by the driver.

DEVICE_NEEDS_RESET (64)

Indicates that the device has experienced an error from which it can’t recover.

FAILED (128)

Indicates that something went wrong in the guest, and it has given up on the device. This could be an internal error, or the driver didn’t like the device for some reason, or even a fatal error during device operation.

The device status field starts out as 0, and is reinitialized to 0 by the device during reset.

2.1.1 Driver Requirements: Device Status Field

The driver MUST update device status, setting bits to indicate the completed steps of the driver initialization sequence specified in 3.1. The driver MUST NOT clear a device status bit. If the driver sets the FAILED bit, the driver MUST later reset the device before attempting to re-initialize.

The driver SHOULD NOT rely on completion of operations of a device if DEVICE_NEEDS_RESET is set. Note: For example, the driver can’t assume requests in flight will be completed if DEVICE_NEEDS_RESET is set, nor can it assume that they have not been completed. A good implementation will try to recover by issuing a reset.

2.1.2 Device Requirements: Device Status Field

The device MUST NOT consume buffers or send any used buffer notifications to the driver before DRIVER_OK.

The device SHOULD set DEVICE_NEEDS_RESET when it enters an error state that a reset is needed. If DRIVER_OK is set, after it sets DEVICE_NEEDS_RESET, the device MUST send a device configuration change notification to the driver.

2.2 Feature Bits

Each virtio device offers all the features it understands. During device initialization, the driver reads this and tells the device the subset that it accepts. The only way to renegotiate is to reset the device.

This allows for forwards and backwards compatibility: if the device is enhanced with a new feature bit, older drivers will not write that feature bit back to the device. Similarly, if a driver is enhanced with a feature that the device doesn’t support, it see the new feature is not offered.

Feature bits are allocated as follows:

0 to 23, 41, 42 and 50 to 127

Feature bits for the specific device type

24 to 40, and 43

Feature bits reserved for extensions to the queue and feature negotiation mechanisms, see 6

44 to 49, and 128 and above

Feature bits reserved for future extensions.

In particular, new fields in the device configuration space are indicated by offering a new feature bit.

To keep the feature negotiation mechanism extensible, it is important that devices do not offer any feature bits that they would not be able to handle if the driver accepted them (even though drivers are not supposed to accept any unspecified, reserved, or unsupported features even if offered, according to the specification.) Likewise, it is important that drivers do not accept feature bits they do not know how to handle (even though devices are not supposed to offer any unspecified, reserved, or unsupported features in the first place, according to the specification.) The preferred way for handling reserved and unexpected features is that the driver ignores them.

In particular, this is especially important for features limited to specific transports, as enabling these for more transports in future versions of the specification is highly likely to require changing the behaviour from drivers and devices. Drivers and devices supporting multiple transports need to carefully maintain per-transport lists of allowed features.

2.2.1 Driver Requirements: Feature Bits

The driver MUST NOT accept a feature which the device did not offer, and MUST NOT accept a feature which requires another feature which was not accepted.

The driver MUST validate the feature bits offered by the device. The driver MUST ignore and MUST NOT accept any feature bit that is

• not described in this specification,

• marked as reserved,

• not supported for the specific transport,

• not defined for the device type.

The driver SHOULD go into backwards compatibility mode if the device does not offer a feature it understands, otherwise MUST set the FAILED device status bit and cease initialization.

By contrast, the driver MUST NOT fail solely because a feature it does not understand has been offered by the device.

2.2.2 Device Requirements: Feature Bits

The device MUST NOT offer a feature which requires another feature which was not offered. The device SHOULD accept any valid subset of features the driver accepts, otherwise it MUST fail to set the FEATURES_OK device status bit when the driver writes it.

The device MUST NOT offer feature bits corresponding to features it would not support if accepted by the driver (even if the driver is prohibited from accepting the feature bits by the specification); for the sake of clarity, this refers to feature bits not described in this specification, reserved feature bits and feature bits reserved or not supported for the specific transport or the specific device type, but this does not preclude devices written to a future version of this specification from offering such feature bits should such a specification have a provision for devices to support the corresponding features.

If a device has successfully negotiated a set of features at least once (by accepting the FEATURES_OK device status bit during device initialization), then it SHOULD NOT fail re-negotiation of the same set of features after a device or system reset. Failure to do so would interfere with resuming from suspend and error recovery.

2.2.3 Legacy Interface: A Note on Feature Bits

Transitional Drivers MUST detect Legacy Devices by detecting that the feature bit VIRTIO_F_VERSION_1 is not offered. Transitional devices MUST detect Legacy drivers by detecting that VIRTIO_F_VERSION_1 has not been acknowledged by the driver.

In this case device is used through the legacy interface.

Legacy interface support is OPTIONAL. Thus, both transitional and non-transitional devices and drivers are compliant with this specification.

Requirements pertaining to transitional devices and drivers is contained in sections named ’Legacy Interface’ like this one.

When device is used through the legacy interface, transitional devices and transitional drivers MUST operate according to the requirements documented within these legacy interface sections. Specification text within these sections generally does not apply to non-transitional devices.

2.3 Notifications

The notion of sending a notification (driver to device or device to driver) plays an important role in this specification. The modus operandi of the notifications is transport specific.

There are three types of notifications:

• configuration change notification

• available buffer notification

• used buffer notification.

Configuration change notifications and used buffer notifications are sent by the device, the recipient is the driver. A configuration change notification indicates that the device configuration space has changed; a used buffer notification indicates that a buffer may have been made used on the virtqueue designated by the notification.

Available buffer notifications are sent by the driver, the recipient is the device. This type of notification indicates that a buffer may have been made available on the virtqueue designated by the notification.

The semantics, the transport-specific implementations, and other important aspects of the different notifications are specified in detail in the following chapters.

Most transports implement notifications sent by the device to the driver using interrupts. Therefore, in previous versions of this specification, these notifications were often called interrupts. Some names defined in this specification still retain this interrupt terminology. Occasionally, the term event is used to refer to a notification or a receipt of a notification.

2.4 Device Reset

The driver may want to initiate a device reset at various times; notably, it is required to do so during device initialization and device cleanup.

The mechanism used by the driver to initiate the reset is transport specific.

2.4.1 Device Requirements: Device Reset

A device MUST reinitialize device status to 0 after receiving a reset.

A device MUST NOT send notifications or interact with the queues after indicating completion of the reset by reinitializing device status to 0, until the driver re-initializes the device.

2.4.2 Driver Requirements: Device Reset

The driver SHOULD consider a driver-initiated reset complete when it reads device status as 0.

2.5 Device Configuration Space

Device configuration space is generally used for rarely-changing or initialization-time parameters. Where configuration fields are optional, their existence is indicated by feature bits: Future versions of this specification will likely extend the device configuration space by adding extra fields at the tail. Note: The device configuration space uses the little-endian format for multi-byte fields.

Each transport also provides a generation count for the device configuration space, which will change whenever there is a possibility that two accesses to the device configuration space can see different versions of that space.

2.5.1 Driver Requirements: Device Configuration Space

Drivers MUST NOT assume reads from fields greater than 32 bits wide are atomic, nor are reads from multiple fields: drivers SHOULD read device configuration space fields like so:

u32 before, after; 
 
do { 
 
        before = get_config_generation(device); 
 
        // read config entry/entries. 
 
        after = get_config_generation(device); 
 
} while (after != before);

For optional configuration space fields, the driver MUST check that the corresponding feature is offered before accessing that part of the configuration space. Note: See section 3.1 for details on feature negotiation.

Drivers MUST NOT limit structure size and device configuration space size. Instead, drivers SHOULD only check that device configuration space is large enough to contain the fields necessary for device operation. Note: For example, if the specification states that device configuration space ’includes a single 8-bit field’ drivers should understand this to mean that the device configuration space might also include an arbitrary amount of tail padding, and accept any device configuration space size equal to or greater than the specified 8-bit size.

2.5.2 Device Requirements: Device Configuration Space

The device MUST allow reading of any device-specific configuration field before FEATURES_OK is set by the driver. This includes fields which are conditional on feature bits, as long as those feature bits are offered by the device.

2.5.3 Legacy Interface: A Note on Device Configuration Space endian-ness

Note that for legacy interfaces, device configuration space is generally the guest’s native endian, rather than PCI’s little-endian. The correct endian-ness is documented for each device.

2.5.4 Legacy Interface: Device Configuration Space

Legacy devices did not have a configuration generation field, thus are susceptible to race conditions if configuration is updated. This affects the block capacity (see 5.2.4) and network mac (see 5.1.4) fields; when using the legacy interface, drivers SHOULD read these fields multiple times until two reads generate a consistent result.

2.6 Virtqueues

The mechanism for bulk data transport on virtio devices is pretentiously called a virtqueue. Each device can have zero or more virtqueues³.

A virtio device can have maximum of 65536 virtqueues. Each virtqueue is identified by a virtqueue index. A virtqueue index has a value in the range of 0 to 65535.

Driver makes requests available to device by adding an available buffer to the queue, i.e., adding a buffer describing the request to a virtqueue, and optionally triggering a driver event, i.e., sending an available buffer notification to the device.

Device executes the requests and - when complete - adds a used buffer to the queue, i.e., lets the driver know by marking the buffer as used. Device can then trigger a device event, i.e., send a used buffer notification to the driver.

Device reports the number of bytes it has written to memory for each buffer it uses. This is referred to as “used length”.

Device is not generally required to use buffers in the same order in which they have been made available by the driver.

Some devices always use descriptors in the same order in which they have been made available. These devices can offer the VIRTIO_F_IN_ORDER feature. If negotiated, this knowledge might allow optimizations or simplify driver and/or device code.

Each virtqueue can consist of up to 3 parts:

• Descriptor Area - used for describing buffers

• Driver Area - extra data supplied by driver to the device

• Device Area - extra data supplied by device to driver

• Descriptor Table - for the Descriptor Area

• Available Ring - for the Driver Area

• Used Ring - for the Device Area

Two formats are supported: Split Virtqueues (see 2.7 Split Virtqueues) and Packed Virtqueues (see 2.8 Packed Virtqueues).

Every driver and device supports either the Packed or the Split Virtqueue format, or both.

2.6.1 Virtqueue Reset

When VIRTIO_F_RING_RESET is negotiated, the driver can reset a virtqueue individually. The way to reset the virtqueue is transport specific.

Virtqueue reset is divided into two parts. The driver first resets a queue and can afterwards optionally re-enable it.

2.6.1.1 Virtqueue Reset

2.6.1.1.1 Device Requirements: Virtqueue Reset

The device MUST reset any state of a virtqueue to the default state, including the available state and the used state.

2.6.1.1.2 Driver Requirements: Virtqueue Reset

After the queue has been successfully reset, the driver MAY release any resource associated with that virtqueue.

2.6.1.2 Virtqueue Re-enable

This process is the same as the initialization process of a single queue during the initialization of the entire device.

2.6.1.2.1 Device Requirements: Virtqueue Re-enable

2.6.1.2.2 Driver Requirements: Virtqueue Re-enable

2.7 Split Virtqueues

The split virtqueue format was the only format supported by the version 1.0 (and earlier) of this standard.

The split virtqueue format separates the virtqueue into several parts, where each part is write-able by either the driver or the device, but not both. Multiple parts and/or locations within a part need to be updated when making a buffer available and when marking it as used.

Each queue has a 16-bit queue size parameter, which sets the number of entries and implies the total size of the queue.

Each virtqueue consists of three parts:

• Descriptor Table - occupies the Descriptor Area

• Available Ring - occupies the Driver Area

• Used Ring - occupies the Device Area

where each part is physically-contiguous in guest memory, and has different alignment requirements.

The memory alignment and size requirements, in bytes, of each part of the virtqueue are summarized in the following table:

The Alignment column gives the minimum alignment for each part of the virtqueue.

The Size column gives the total number of bytes for each part of the virtqueue.

Queue Size corresponds to the maximum number of buffers in the virtqueue⁴. Queue Size value is always a power of 2. The maximum Queue Size value is 32768. This value is specified in a bus-specific way.

When the driver wants to send a buffer to the device, it fills in a slot in the descriptor table (or chains several together), and writes the descriptor index into the available ring. It then notifies the device. When the device has finished a buffer, it writes the descriptor index into the used ring, and sends a used buffer notification.

2.7.1 Driver Requirements: Virtqueues

The driver MUST ensure that the physical address of the first byte of each virtqueue part is a multiple of the specified alignment value in the above table.

2.7.2 Legacy Interfaces: A Note on Virtqueue Layout

For Legacy Interfaces, several additional restrictions are placed on the virtqueue layout:

Each virtqueue occupies two or more physically-contiguous pages (usually defined as 4096 bytes, but depending on the transport; henceforth referred to as Queue Align) and consists of three parts:

The bus-specific Queue Size field controls the total number of bytes for the virtqueue. When using the legacy interface, the transitional driver MUST retrieve the Queue Size field from the device and MUST allocate the total number of bytes for the virtqueue according to the following formula (Queue Align given in qalign and Queue Size given in qsz):

#define ALIGN(x) (((x) + qalign) & "qalign) 
 
static inline unsigned virtq_size(unsigned int qsz) 
 
{ 
 
     return ALIGN(sizeof(struct virtq_desc)*qsz + sizeof(u16)*(3 + qsz)) 
 
          + ALIGN(sizeof(u16)*3 + sizeof(struct virtq_used_elem)*qsz); 
 
}

This wastes some space with padding. When using the legacy interface, both transitional devices and drivers MUST use the following virtqueue layout structure to locate elements of the virtqueue:

struct virtq { 
 
        // The actual descriptors (16 bytes each) 
 
        struct virtq_desc desc[ Queue Size ]; 
 
 
 
        // A ring of available descriptor heads with free-running index. 
 
        struct virtq_avail avail; 
 
 
 
        // Padding to the next Queue Align boundary. 
 
        u8 pad[ Padding ]; 
 
 
 
        // A ring of used descriptor heads with free-running index. 
 
        struct virtq_used used; 
 
};

2.7.3 Legacy Interfaces: A Note on Virtqueue Endianness

Note that when using the legacy interface, transitional devices and drivers MUST use the native endian of the guest as the endian of fields and in the virtqueue. This is opposed to little-endian for non-legacy interface as specified by this standard. It is assumed that the host is already aware of the guest endian.

2.7.4 Message Framing

The framing of messages with descriptors is independent of the contents of the buffers. For example, a network transmit buffer consists of a 12 byte header followed by the network packet. This could be most simply placed in the descriptor table as a 12 byte output descriptor followed by a 1514 byte output descriptor, but it could also consist of a single 1526 byte output descriptor in the case where the header and packet are adjacent, or even three or more descriptors (possibly with loss of efficiency in that case).

Note that, some device implementations have large-but-reasonable restrictions on total descriptor size (such as based on IOV_MAX in the host OS). This has not been a problem in practice: little sympathy will be given to drivers which create unreasonably-sized descriptors such as by dividing a network packet into 1500 single-byte descriptors!

2.7.4.1 Device Requirements: Message Framing

The device MUST NOT make assumptions about the particular arrangement of descriptors. The device MAY have a reasonable limit of descriptors it will allow in a chain.

2.7.4.2 Driver Requirements: Message Framing

The driver MUST place any device-writable descriptor elements after any device-readable descriptor elements.

The driver SHOULD NOT use an excessive number of descriptors to describe a buffer.

2.7.4.3 Legacy Interface: Message Framing

Regrettably, initial driver implementations used simple layouts, and devices came to rely on it, despite this specification wording. In addition, the specification for virtio_blk SCSI commands required intuiting field lengths from frame boundaries (see 5.2.6.3 Legacy Interface: Device Operation)

Thus when using the legacy interface, the VIRTIO_F_ANY_LAYOUT feature indicates to both the device and the driver that no assumptions were made about framing. Requirements for transitional drivers when this is not negotiated are included in each device section.

2.7.5 The Virtqueue Descriptor Table

The descriptor table refers to the buffers the driver is using for the device. addr is a physical address, and the buffers can be chained via next. Each descriptor describes a buffer which is read-only for the device (“device-readable”) or write-only for the device (“device-writable”), but a chain of descriptors can contain both device-readable and device-writable buffers.

The actual contents of the memory offered to the device depends on the device type. Most common is to begin the data with a header (containing little-endian fields) for the device to read, and postfix it with a status tailer for the device to write.

struct virtq_desc { 
 
        /* Address (guest-physical). */ 
 
        le64 addr; 
 
        /* Length. */ 
 
        le32 len; 
 
 
 
/* This marks a buffer as continuing via the next field. */ 
 
#define VIRTQ_DESC_F_NEXT   1 
 
/* This marks a buffer as device write-only (otherwise device read-only). */ 
 
#define VIRTQ_DESC_F_WRITE     2 
 
/* This means the buffer contains a list of buffer descriptors. */ 
 
#define VIRTQ_DESC_F_INDIRECT   4 
 
        /* The flags as indicated above. */ 
 
        le16 flags; 
 
        /* Next field if flags & NEXT */ 
 
        le16 next; 
 
};

The number of descriptors in the table is defined by the queue size for this virtqueue: this is the maximum possible descriptor chain length.

If VIRTIO_F_IN_ORDER has been negotiated, driver uses descriptors in ring order: starting from offset 0 in the table, and wrapping around at the end of the table. Note: The legacy [Virtio PCI Draft] referred to this structure as vring_desc, and the constants as VRING_DESC_F_NEXT, etc, but the layout and values were identical.

2.7.5.1 Device Requirements: The Virtqueue Descriptor Table

A device MUST NOT write to a device-readable buffer, and a device SHOULD NOT read a device-writable buffer (it MAY do so for debugging or diagnostic purposes). A device MUST NOT write to any descriptor table entry.

2.7.5.2 Driver Requirements: The Virtqueue Descriptor Table

Drivers MUST NOT add a descriptor chain longer than 2³² bytes in total; this implies that loops in the descriptor chain are forbidden!

If VIRTIO_F_IN_ORDER has been negotiated, and when making a descriptor with VRING_DESC_F_NEXT set in flags at offset x in the table available to the device, driver MUST set next to 0 for the last descriptor in the table (where x = queue_size − 1) and to x + 1 for the rest of the descriptors.

2.7.5.3 Indirect Descriptors

Some devices benefit by concurrently dispatching a large number of large requests. The VIRTIO_F_INDIRECT_DESC feature allows this (see A virtio_queue.h). To increase ring capacity the driver can store a table of indirect descriptors anywhere in memory, and insert a descriptor in main virtqueue (with flags&VIRTQ_DESC_F_INDIRECT on) that refers to memory buffer containing this indirect descriptor table; addr and len refer to the indirect table address and length in bytes, respectively.

The indirect table layout structure looks like this (len is the length of the descriptor that refers to this table, which is a variable, so this code won’t compile):

struct indirect_descriptor_table { 
 
        /* The actual descriptors (16 bytes each) */ 
 
        struct virtq_desc desc[len / 16]; 
 
};

The first indirect descriptor is located at start of the indirect descriptor table (index 0), additional indirect descriptors are chained by next. An indirect descriptor without a valid next (with flags&VIRTQ_DESC_F_NEXT off) signals the end of the descriptor. A single indirect descriptor table can include both device-readable and device-writable descriptors.

If VIRTIO_F_IN_ORDER has been negotiated, indirect descriptors use sequential indices, in-order: index 0 followed by index 1 followed by index 2, etc.

2.7.5.3.1 Driver Requirements: Indirect Descriptors

A driver MUST NOT create a descriptor chain longer than the Queue Size of the device.

A driver MUST NOT set both VIRTQ_DESC_F_INDIRECT and VIRTQ_DESC_F_NEXT in flags.

If VIRTIO_F_IN_ORDER has been negotiated, indirect descriptors MUST appear sequentially, with next taking the value of 1 for the 1st descriptor, 2 for the 2nd one, etc.

2.7.5.3.2 Device Requirements: Indirect Descriptors

The device MUST handle the case of zero or more normal chained descriptors followed by a single descriptor with flags&VIRTQ_DESC_F_INDIRECT. Note: While unusual (most implementations either create a chain solely using non-indirect descriptors, or use a single indirect element), such a layout is valid.

2.7.6 The Virtqueue Available Ring

The available ring has the following layout structure:

struct virtq_avail { 
 
#define VIRTQ_AVAIL_F_NO_INTERRUPT      1 
 
        le16 flags; 
 
        le16 idx; 
 
        le16 ring[ /* Queue Size */ ]; 
 
        le16 used_event; /* Only if VIRTIO_F_EVENT_IDX */ 
 
};

The driver uses the available ring to offer buffers to the device: each ring entry refers to the head of a descriptor chain. It is only written by the driver and read by the device.

idx field indicates where the driver would put the next descriptor entry in the ring (modulo the queue size). This starts at 0, and increases. Note: The legacy [Virtio PCI Draft] referred to this structure as vring_avail, and the constant as VRING_AVAIL_F_NO_INTERRUPT, but the layout and value were identical.

2.7.6.1 Driver Requirements: The Virtqueue Available Ring

A driver MUST NOT decrement the available idx on a virtqueue (ie. there is no way to “unexpose” buffers).

2.7.7 Used Buffer Notification Suppression

If the VIRTIO_F_EVENT_IDX feature bit is not negotiated, the flags field in the available ring offers a crude mechanism for the driver to inform the device that it doesn’t want notifications when buffers are used. Otherwise used_event is a more performant alternative where the driver specifies how far the device can progress before a notification is required.

Neither of these notification suppression methods are reliable, as they are not synchronized with the device, but they serve as useful optimizations.

2.7.7.1 Driver Requirements: Used Buffer Notification Suppression

If the VIRTIO_F_EVENT_IDX feature bit is not negotiated:

• The driver MUST set flags to 0 or 1.

• The driver MAY set flags to 1 to advise the device that notifications are not needed.

Otherwise, if the VIRTIO_F_EVENT_IDX feature bit is negotiated:

• The driver MUST set flags to 0.

• The driver MAY use used_event to advise the device that notifications are unnecessary until the device writes an entry with an index specified by used_event into the used ring (equivalently, until idx in the used ring will reach the value used_event + 1).

The driver MUST handle spurious notifications from the device.

2.7.7.2 Device Requirements: Used Buffer Notification Suppression

If the VIRTIO_F_EVENT_IDX feature bit is not negotiated:

• The device MUST ignore the used_event value.

• After the device writes a descriptor index into the used ring:

  • If flags is 1, the device SHOULD NOT send a notification.

  • If flags is 0, the device MUST send a notification.

Otherwise, if the VIRTIO_F_EVENT_IDX feature bit is negotiated:

• The device MUST ignore the lower bit of flags.

• After the device writes a descriptor index into the used ring:

  • If the idx field in the used ring (which determined where that descriptor index was placed) was equal to used_event, the device MUST send a notification.

  • Otherwise the device SHOULD NOT send a notification.

VIRTIO_F_EVENT_IDX would send a used buffer notification to the driver after the first buffer is used (and again after the 65536th buffer, etc).

2.7.8 The Virtqueue Used Ring

The used ring has the following layout structure:

struct virtq_used { 
 
#define VIRTQ_USED_F_NO_NOTIFY  1 
 
        le16 flags; 
 
        le16 idx; 
 
        struct virtq_used_elem ring[ /* Queue Size */]; 
 
        le16 avail_event; /* Only if VIRTIO_F_EVENT_IDX */ 
 
}; 
 
 
 
/* le32 is used here for ids for padding reasons. */ 
 
struct virtq_used_elem { 
 
        /* Index of start of used descriptor chain. */ 
 
        le32 id; 
 
        /* 
 
         * The number of bytes written into the device writable portion of 
 
         * the buffer described by the descriptor chain. 
 
         */ 
 
        le32 len; 
 
};

The used ring is where the device returns buffers once it is done with them: it is only written to by the device, and read by the driver.

Each entry in the ring is a pair: id indicates the head entry of the descriptor chain describing the buffer (this matches an entry placed in the available ring by the guest earlier), and len the total of bytes written into the buffer. Note: len is particularly useful for drivers using untrusted buffers: if a driver does not know exactly how much has been written by the device, the driver would have to zero the buffer in advance to ensure no data leakage occurs.

For example, a network driver may hand a received buffer directly to an unprivileged userspace application. If the network device has not overwritten the bytes which were in that buffer, this could leak the contents of freed memory from other processes to the application.

idx field indicates where the device would put the next descriptor entry in the ring (modulo the queue size). This starts at 0, and increases. Note: The legacy [Virtio PCI Draft] referred to these structures as vring_used and vring_used_elem, and the constant as VRING_USED_F_NO_NOTIFY, but the layout and value were identical.

2.7.8.1 Legacy Interface: The Virtqueue Used Ring

Historically, many drivers ignored the len value, as a result, many devices set len incorrectly. Thus, when using the legacy interface, it is generally a good idea to ignore the len value in used ring entries if possible. Specific known issues are listed per device type.

2.7.8.2 Device Requirements: The Virtqueue Used Ring

The device MUST set len prior to updating the used idx.

The device MUST write at least len bytes to descriptor, beginning at the first device-writable buffer, prior to updating the used idx.

The device MAY write more than len bytes to descriptor. Note: There are potential error cases where a device might not know what parts of the buffers have been written. This is why len is permitted to be an underestimate: that’s preferable to the driver believing that uninitialized memory has been overwritten when it has not.

2.7.8.3 Driver Requirements: The Virtqueue Used Ring

The driver MUST NOT make assumptions about data in device-writable buffers beyond the first len bytes, and SHOULD ignore this data.

2.7.9 In-order use of descriptors

Some devices always use descriptors in the same order in which they have been made available. These devices can offer the VIRTIO_F_IN_ORDER feature. If negotiated, this knowledge allows devices to notify the use of a batch of buffers to the driver by only writing out a single used ring entry with the id corresponding to the head entry of the descriptor chain describing the last buffer in the batch.

The device then skips forward in the ring according to the size of the batch. Accordingly, it increments the used idx by the size of the batch.

The driver needs to look up the used id and calculate the batch size to be able to advance to where the next used ring entry will be written by the device.

This will result in the used ring entry at an offset matching the first available ring entry in the batch, the used ring entry for the next batch at an offset matching the first available ring entry in the next batch, etc.

The skipped buffers (for which no used ring entry was written) are assumed to have been used (read or written) by the device completely.

2.7.10 Available Buffer Notification Suppression

The device can suppress available buffer notifications in a manner analogous to the way drivers can suppress used buffer notifications as detailed in section 2.7.7. The device manipulates flags or avail_event in the used ring the same way the driver manipulates flags or used_event in the available ring.

2.7.10.1 Driver Requirements: Available Buffer Notification Suppression

The driver MUST initialize flags in the used ring to 0 when allocating the used ring.

If the VIRTIO_F_EVENT_IDX feature bit is not negotiated:

• The driver MUST ignore the avail_event value.

• After the driver writes a descriptor index into the available ring:

  • If flags is 1, the driver SHOULD NOT send a notification.

  • If flags is 0, the driver MUST send a notification.

Otherwise, if the VIRTIO_F_EVENT_IDX feature bit is negotiated:

• The driver MUST ignore the lower bit of flags.

• After the driver writes a descriptor index into the available ring:

  • If the idx field in the available ring (which determined where that descriptor index was placed) was equal to avail_event, the driver MUST send a notification.

  • Otherwise the driver SHOULD NOT send a notification.

2.7.10.2 Device Requirements: Available Buffer Notification Suppression

If the VIRTIO_F_EVENT_IDX feature bit is not negotiated:

• The device MUST set flags to 0 or 1.

• The device MAY set flags to 1 to advise the driver that notifications are not needed.

Otherwise, if the VIRTIO_F_EVENT_IDX feature bit is negotiated:

• The device MUST set flags to 0.

• The device MAY use avail_event to advise the driver that notifications are unnecessary until the driver writes entry with an index specified by avail_event into the available ring (equivalently, until idx in the available ring will reach the value avail_event + 1).

The device MUST handle spurious notifications from the driver.

2.7.11 Helpers for Operating Virtqueues

The Linux Kernel Source code contains the definitions above and helper routines in a more usable form, in include/uapi/linux/virtio_ring.h. This was explicitly licensed by IBM and Red Hat under the (3-clause) BSD license so that it can be freely used by all other projects, and is reproduced (with slight variation) in A virtio_queue.h.

2.7.12 Virtqueue Operation

There are two parts to virtqueue operation: supplying new available buffers to the device, and processing used buffers from the device. Note: As an example, the simplest virtio network device has two virtqueues: the transmit virtqueue and the receive virtqueue. The driver adds outgoing (device-readable) packets to the transmit virtqueue, and then frees them after they are used. Similarly, incoming (device-writable) buffers are added to the receive virtqueue, and processed after they are used.

What follows is the requirements of each of these two parts when using the split virtqueue format in more detail.

2.7.13 Supplying Buffers to The Device

The driver offers buffers to one of the device’s virtqueues as follows:

1.

The driver places the buffer into free descriptor(s) in the descriptor table, chaining as necessary (see 2.7.5 The Virtqueue Descriptor Table).

2.

The driver places the index of the head of the descriptor chain into the next ring entry of the available ring.

3.

Steps 1 and 2 MAY be performed repeatedly if batching is possible.

4.

The driver performs a suitable memory barrier to ensure the device sees the updated descriptor table and available ring before the next step.

5.

The available idx is increased by the number of descriptor chain heads added to the available ring.

6.

The driver performs a suitable memory barrier to ensure that it updates the idx field before checking for notification suppression.

7.

The driver sends an available buffer notification to the device if such notifications are not suppressed.

Note that the above code does not take precautions against the available ring buffer wrapping around: this is not possible since the ring buffer is the same size as the descriptor table, so step (1) will prevent such a condition.

In addition, the maximum queue size is 32768 (the highest power of 2 which fits in 16 bits), so the 16-bit idx value can always distinguish between a full and empty buffer.

What follows is the requirements of each stage in more detail.

2.7.13.1 Placing Buffers Into The Descriptor Table

A buffer consists of zero or more device-readable physically-contiguous elements followed by zero or more physically-contiguous device-writable elements (each has at least one element). This algorithm maps it into the descriptor table to form a descriptor chain:

for each buffer element, b:

1.

Get the next free descriptor table entry, d

2.

Set d.addr to the physical address of the start of b

3.

Set d.len to the length of b.

4.

If b is device-writable, set d.flags to VIRTQ_DESC_F_WRITE, otherwise 0.

5.

If there is a buffer element after this:

(a)

Set d.next to the index of the next free descriptor element.

(b)

Set the VIRTQ_DESC_F_NEXT bit in d.flags.

In practice, d.next is usually used to chain free descriptors, and a separate count kept to check there are enough free descriptors before beginning the mappings.

2.7.13.2 Updating The Available Ring

The descriptor chain head is the first d in the algorithm above, ie. the index of the descriptor table entry referring to the first part of the buffer. A naive driver implementation MAY do the following (with the appropriate conversion to-and-from little-endian assumed):

avail->ring[avail->idx % qsz] = head;

However, in general the driver MAY add many descriptor chains before it updates idx (at which point they become visible to the device), so it is common to keep a counter of how many the driver has added:

avail->ring[(avail->idx + added++) % qsz] = head;

2.7.13.3 Updating idx

idx always increments, and wraps naturally at 65536:

avail->idx += added;

Once available idx is updated by the driver, this exposes the descriptor and its contents. The device MAY access the descriptor chains the driver created and the memory they refer to immediately.

2.7.13.3.1 Driver Requirements: Updating idx

2.7.13.4 Notifying The Device

The actual method of device notification is bus-specific, but generally it can be expensive. So the device MAY suppress such notifications if it doesn’t need them, as detailed in section 2.7.10.

The driver has to be careful to expose the new idx value before checking if notifications are suppressed.

2.7.13.4.1 Driver Requirements: Notifying The Device

2.7.14 Receiving Used Buffers From The Device

Once the device has used buffers referred to by a descriptor (read from or written to them, or parts of both, depending on the nature of the virtqueue and the device), it sends a used buffer notification to the driver as detailed in section 2.7.7. Note:

For optimal performance, a driver MAY disable used buffer notifications while processing the used ring, but beware the problem of missing notifications between emptying the ring and reenabling notifications. This is usually handled by re-checking for more used buffers after notifications are re-enabled:

virtq_disable_used_buffer_notifications(vq); 
 
 
 
for (;;) { 
 
        if (vq->last_seen_used != le16_to_cpu(virtq->used.idx)) { 
 
                virtq_enable_used_buffer_notifications(vq); 
 
                mb(); 
 
 
 
                if (vq->last_seen_used != le16_to_cpu(virtq->used.idx)) 
 
                        break; 
 
 
 
                virtq_disable_used_buffer_notifications(vq); 
 
        } 
 
 
 
        struct virtq_used_elem *e = virtq.used->ring[vq->last_seen_used%vsz]; 
 
        process_buffer(e); 
 
        vq->last_seen_used++; 
 
}

2.8 Packed Virtqueues

Packed virtqueues is an alternative compact virtqueue layout using read-write memory, that is memory that is both read and written by both the device and the driver.

Use of packed virtqueues is negotiated by the VIRTIO_F_RING_PACKED feature bit.

Packed virtqueues support up to 2¹⁵ entries each.

With current transports, virtqueues are located in guest memory allocated by the driver. Each packed virtqueue consists of three parts:

• Descriptor Ring - occupies the Descriptor Area

• Driver Event Suppression - occupies the Driver Area

• Device Event Suppression - occupies the Device Area

Where the Descriptor Ring in turn consists of descriptors, and where each descriptor can contain the following parts:

• Buffer ID

• Element Address

• Element Length

• Flags

A buffer consists of zero or more device-readable physically-contiguous elements followed by zero or more physically-contiguous device-writable elements (each buffer has at least one element).

When the driver wants to send such a buffer to the device, it writes at least one available descriptor describing elements of the buffer into the Descriptor Ring. The descriptor(s) are associated with a buffer by means of a Buffer ID stored within the descriptor.

The driver then notifies the device. When the device has finished processing the buffer, it writes a used device descriptor including the Buffer ID into the Descriptor Ring (overwriting a driver descriptor previously made available), and sends a used event notification.

The Descriptor Ring is used in a circular manner: the driver writes descriptors into the ring in order. After reaching the end of the ring, the next descriptor is placed at the head of the ring. Once the ring is full of driver descriptors, the driver stops sending new requests and waits for the device to start processing descriptors and to write out some used descriptors before making new driver descriptors available.

Similarly, the device reads descriptors from the ring in order and detects that a driver descriptor has been made available. As processing of descriptors is completed, used descriptors are written by the device back into the ring.

Note: after reading driver descriptors and starting their processing in order, the device might complete their processing out of order. Used device descriptors are written in the order in which their processing is complete.

The Device Event Suppression data structure is write-only by the device. It includes information for reducing the number of device events, i.e., sending fewer available buffer notifications to the device.

The Driver Event Suppression data structure is read-only by the device. It includes information for reducing the number of driver events, i.e., sending fewer used buffer notifications to the driver.

2.8.1 Driver and Device Ring Wrap Counters

Each of the driver and the device are expected to maintain, internally, a single-bit ring wrap counter initialized to 1.

The counter maintained by the driver is called the Driver Ring Wrap Counter. The driver changes the value of this counter each time it makes available the last descriptor in the ring (after making the last descriptor available).

The counter maintained by the device is called the Device Ring Wrap Counter. The device changes the value of this counter each time it uses the last descriptor in the ring (after marking the last descriptor used).

It is easy to see that the Driver Ring Wrap Counter in the driver matches the Device Ring Wrap Counter in the device when both are processing the same descriptor, or when all available descriptors have been used.

To mark a descriptor as available and used, both the driver and the device use the following two flags:

#define VIRTQ_DESC_F_AVAIL     (1 << 7) 
 
#define VIRTQ_DESC_F_USED      (1 << 15)

To mark a descriptor as available, the driver sets the VIRTQ_DESC_F_AVAIL bit in Flags to match the internal Driver Ring Wrap Counter. It also sets the VIRTQ_DESC_F_USED bit to match the inverse value (i.e. to not match the internal Driver Ring Wrap Counter).

To mark a descriptor as used, the device sets the VIRTQ_DESC_F_USED bit in Flags to match the internal Device Ring Wrap Counter. It also sets the VIRTQ_DESC_F_AVAIL bit to match the same value.

Thus VIRTQ_DESC_F_AVAIL and VIRTQ_DESC_F_USED bits are different for an available descriptor and equal for a used descriptor.

Note that this observation is mostly useful for sanity-checking as these are necessary but not sufficient conditions - for example, all descriptors are zero-initialized. To detect used and available descriptors it is possible for drivers and devices to keep track of the last observed value of VIRTQ_DESC_F_USED/VIRTQ_DESC_F_AVAIL. Other techniques to detect VIRTQ_DESC_F_AVAIL/VIRTQ_DESC_F_USED bit changes might also be possible.

2.8.2 Polling of available and used descriptors

Writes of device and driver descriptors can generally be reordered, but each side (driver and device) are only required to poll (or test) a single location in memory: the next device descriptor after the one they processed previously, in circular order.

Sometimes the device needs to only write out a single used descriptor after processing a batch of multiple available descriptors. As described in more detail below, this can happen when using descriptor chaining or with in-order use of descriptors. In this case, the device writes out a used descriptor with the buffer id of the last descriptor in the group. After processing the used descriptor, both device and driver then skip forward in the ring the number of the remaining descriptors in the group until processing (reading for the driver and writing for the device) the next used descriptor.

2.8.3 Write Flag

In an available descriptor, the VIRTQ_DESC_F_WRITE bit within Flags is used to mark a descriptor as corresponding to a write-only or read-only element of a buffer.

/* This marks a descriptor as device write-only (otherwise device read-only). */ 
 
#define VIRTQ_DESC_F_WRITE     2

In a used descriptor, this bit is used to specify whether any data has been written by the device into any parts of the buffer.

2.8.4 Element Address and Length

In an available descriptor, Element Address corresponds to the physical address of the buffer element. The length of the element assumed to be physically contiguous is stored in Element Length.

In a used descriptor, Element Address is unused. Element Length specifies the length of the buffer that has been initialized (written to) by the device.

Element Length is reserved for used descriptors without the VIRTQ_DESC_F_WRITE flag, and is ignored by drivers.

2.8.5 Scatter-Gather Support

Some drivers need an ability to supply a list of multiple buffer elements (also known as a scatter/gather list) with a request. Two features support this: descriptor chaining and indirect descriptors.

If neither feature is in use by the driver, each buffer is physically-contiguous, either read-only or write-only and is described completely by a single descriptor.

While unusual (most implementations either create all lists solely using non-indirect descriptors, or always use a single indirect element), if both features have been negotiated, mixing indirect and non-indirect descriptors in a ring is valid, as long as each list only contains descriptors of a given type.

Scatter/gather lists only apply to available descriptors. A single used descriptor corresponds to the whole list.

The device limits the number of descriptors in a list through a transport-specific and/or device-specific value. If not limited, the maximum number of descriptors in a list is the virtqueue size.

2.8.6 Next Flag: Descriptor Chaining

The packed ring format allows the driver to supply a scatter/gather list to the device by using multiple descriptors, and setting the VIRTQ_DESC_F_NEXT bit in Flags for all but the last available descriptor.

/* This marks a buffer as continuing. */ 
 
#define VIRTQ_DESC_F_NEXT   1

Buffer ID is included in the last descriptor in the list.

The driver always makes the first descriptor in the list available after the rest of the list has been written out into the ring. This guarantees that the device will never observe a partial scatter/gather list in the ring.

Note: all flags, including VIRTQ_DESC_F_AVAIL, VIRTQ_DESC_F_USED, VIRTQ_DESC_F_WRITE must be set/cleared correctly in all descriptors in the list, not just the first one.

The device only writes out a single used descriptor for the whole list. It then skips forward according to the number of descriptors in the list. The driver needs to keep track of the size of the list corresponding to each buffer ID, to be able to skip to where the next used descriptor is written by the device.

For example, if descriptors are used in the same order in which they are made available, this will result in the used descriptor overwriting the first available descriptor in the list, the used descriptor for the next list overwriting the first available descriptor in the next list, etc.

VIRTQ_DESC_F_NEXT is reserved in used descriptors, and should be ignored by drivers.

2.8.7 Indirect Flag: Scatter-Gather Support

Some devices benefit by concurrently dispatching a large number of large requests. The VIRTIO_F_INDIRECT_DESC feature allows this. To increase ring capacity the driver can store a (read-only by the device) table of indirect descriptors anywhere in memory, and insert a descriptor in the main virtqueue (with Flags bit VIRTQ_DESC_F_INDIRECT on) that refers to a buffer element containing this indirect descriptor table; addr and len refer to the indirect table address and length in bytes, respectively.

/* This means the element contains a table of descriptors. */ 
 
#define VIRTQ_DESC_F_INDIRECT   4

The indirect table layout structure looks like this (len is the Buffer Length of the descriptor that refers to this table, which is a variable):

struct pvirtq_indirect_descriptor_table { 
 
        /* The actual descriptor structures (struct pvirtq_desc each) */ 
 
        struct pvirtq_desc desc[len / sizeof(struct pvirtq_desc)]; 
 
};

The first descriptor is located at the start of the indirect descriptor table, additional indirect descriptors come immediately afterwards. The VIRTQ_DESC_F_WRITE flags bit is the only valid flag for descriptors in the indirect table. Others are reserved and are ignored by the device. Buffer ID is also reserved and is ignored by the device.

In descriptors with VIRTQ_DESC_F_INDIRECT set VIRTQ_DESC_F_WRITE is reserved and is ignored by the device.

2.8.8 In-order use of descriptors

Some devices always use descriptors in the same order in which they have been made available. These devices can offer the VIRTIO_F_IN_ORDER feature. If negotiated, this knowledge allows devices to notify the use of a batch of buffers to the driver by only writing out a single used descriptor with the Buffer ID corresponding to the last descriptor in the batch.

The device then skips forward in the ring according to the size of the batch. The driver needs to look up the used Buffer ID and calculate the batch size to be able to advance to where the next used descriptor will be written by the device.

This will result in the used descriptor overwriting the first available descriptor in the batch, the used descriptor for the next batch overwriting the first available descriptor in the next batch, etc.

The skipped buffers (for which no used descriptor was written) are assumed to have been used (read or written) by the device completely.

2.8.9 Multi-buffer requests

Some devices combine multiple buffers as part of processing of a single request. These devices always mark the descriptor corresponding to the first buffer in the request used after the rest of the descriptors (corresponding to rest of the buffers) in the request - which follow the first descriptor in ring order - has been marked used and written out into the ring. This guarantees that the driver will never observe a partial request in the ring.

2.8.10 Driver and Device Event Suppression

In many systems used and available buffer notifications involve significant overhead. To mitigate this overhead, each virtqueue includes two identical structures used for controlling notifications between the device and the driver.

The Driver Event Suppression structure is read-only by the device and controls the used buffer notifications sent by the device to the driver.

The Device Event Suppression structure is read-only by the driver and controls the available buffer notifications sent by the driver to the device.

Each of these Event Suppression structures includes the following fields:

Descriptor Ring Change Event Flags

Takes values:

/* Enable events */ 
 
#define RING_EVENT_FLAGS_ENABLE 0x0 
 
/* Disable events */ 
 
#define RING_EVENT_FLAGS_DISABLE 0x1 
 
/* 
 
 * Enable events for a specific descriptor 
 
 * (as specified by Descriptor Ring Change Event Offset/Wrap Counter). 
 
 * Only valid if VIRTIO_F_EVENT_IDX has been negotiated. 
 
 */ 
 
#define RING_EVENT_FLAGS_DESC 0x2 
 
/* The value 0x3 is reserved */

Descriptor Ring Change Event Offset

If Event Flags set to descriptor specific event: offset within the ring (in units of descriptor size). Event will only trigger when this descriptor is made available/used respectively.

Descriptor Ring Change Event Wrap Counter

If Event Flags set to descriptor specific event: offset within the ring (in units of descriptor size). Event will only trigger when Ring Wrap Counter matches this value and a descriptor is made available/used respectively.

After writing out some descriptors, both the device and the driver are expected to consult the relevant structure to find out whether a used respectively an available buffer notification should be sent.

2.8.10.1 Structure Size and Alignment

Each part of the virtqueue is physically-contiguous in guest memory, and has different alignment requirements.

The memory alignment and size requirements, in bytes, of each part of the virtqueue are summarized in the following table:

The Alignment column gives the minimum alignment for each part of the virtqueue.

The Size column gives the total number of bytes for each part of the virtqueue.

Queue Size corresponds to the maximum number of descriptors in the virtqueue⁵. The Queue Size value does not have to be a power of 2.

2.8.11 Driver Requirements: Virtqueues

The driver MUST ensure that the physical address of the first byte of each virtqueue part is a multiple of the specified alignment value in the above table.

2.8.12 Device Requirements: Virtqueues

The device MUST start processing driver descriptors in the order in which they appear in the ring. The device MUST start writing device descriptors into the ring in the order in which they complete. The device MAY reorder descriptor writes once they are started.

2.8.13 The Virtqueue Descriptor Format

The available descriptor refers to the buffers the driver is sending to the device. addr is a physical address, and the descriptor is identified with a buffer using the id field.

struct pvirtq_desc { 
 
        /* Buffer Address. */ 
 
        le64 addr; 
 
        /* Buffer Length. */ 
 
        le32 len; 
 
        /* Buffer ID. */ 
 
        le16 id; 
 
        /* The flags depending on descriptor type. */ 
 
        le16 flags; 
 
};

The descriptor ring is zero-initialized.

2.8.14 Event Suppression Structure Format

The following structure is used to reduce the number of notifications sent between driver and device.

struct pvirtq_event_suppress { 
 
        le16 { 
 
             desc_event_off : 15; /* Descriptor Ring Change Event Offset */ 
 
             desc_event_wrap : 1; /* Descriptor Ring Change Event Wrap Counter */ 
 
        } desc; /* If desc_event_flags set to RING_EVENT_FLAGS_DESC */ 
 
        le16 { 
 
             desc_event_flags : 2, /* Descriptor Ring Change Event Flags */ 
 
             reserved : 14; /* Reserved, set to 0 */ 
 
        } flags; 
 
};

2.8.15 Device Requirements: The Virtqueue Descriptor Table

A device MUST NOT write to a device-readable buffer, and a device SHOULD NOT read a device-writable buffer. A device MUST NOT use a descriptor unless it observes the VIRTQ_DESC_F_AVAIL bit in its flags being changed (e.g. as compared to the initial zero value). A device MUST NOT change a descriptor after changing it’s the VIRTQ_DESC_F_USED bit in its flags.

2.8.16 Driver Requirements: The Virtqueue Descriptor Table

A driver MUST NOT change a descriptor unless it observes the VIRTQ_DESC_F_USED bit in its flags being changed. A driver MUST NOT change a descriptor after changing the VIRTQ_DESC_F_AVAIL bit in its flags. When notifying the device, driver MUST set next_off and next_wrap to match the next descriptor not yet made available to the device. A driver MAY send multiple available buffer notifications without making any new descriptors available to the device.

2.8.17 Driver Requirements: Scatter-Gather Support

A driver MUST NOT create a descriptor list longer than allowed by the device.

A driver MUST NOT create a descriptor list longer than the Queue Size.

This implies that loops in the descriptor list are forbidden!

The driver MUST place any device-writable descriptor elements after any device-readable descriptor elements.

A driver MUST NOT depend on the device to use more descriptors to be able to write out all descriptors in a list. A driver MUST make sure there’s enough space in the ring for the whole list before making the first descriptor in the list available to the device.

A driver MUST NOT make the first descriptor in the list available before all subsequent descriptors comprising the list are made available.

2.8.18 Device Requirements: Scatter-Gather Support

The device MUST use descriptors in a list chained by the VIRTQ_DESC_F_NEXT flag in the same order that they were made available by the driver.

The device MAY limit the number of buffers it will allow in a list.

2.8.19 Driver Requirements: Indirect Descriptors

The driver MUST NOT set the VIRTQ_DESC_F_INDIRECT flag unless the VIRTIO_F_INDIRECT_DESC feature was negotiated. The driver MUST NOT set any flags except DESC_F_WRITE within an indirect descriptor.

A driver MUST NOT create a descriptor chain longer than allowed by the device.

A driver MUST NOT write direct descriptors with VIRTQ_DESC_F_INDIRECT set in a scatter-gather list linked by VIRTQ_DESC_F_NEXT. flags.

2.8.20 Virtqueue Operation

There are two parts to virtqueue operation: supplying new available buffers to the device, and processing used buffers from the device.

What follows is the requirements of each of these two parts when using the packed virtqueue format in more detail.

2.8.21 Supplying Buffers to The Device

The driver offers buffers to one of the device’s virtqueues as follows:

1.

The driver places the buffer into free descriptor(s) in the Descriptor Ring.

2.

The driver performs a suitable memory barrier to ensure that it updates the descriptor(s) before checking for notification suppression.

3.

If notifications are not suppressed, the driver notifies the device of the new available buffers.

What follows are the requirements of each stage in more detail.

2.8.21.1 Placing Available Buffers Into The Descriptor Ring

For each buffer element, b:

1.

Get the next descriptor table entry, d

2.

Get the next free buffer id value

3.

Set d.addr to the physical address of the start of b

4.

Set d.len to the length of b.

5.

Set d.id to the buffer id

6.

Calculate the flags as follows:

(a)

If b is device-writable, set the VIRTQ_DESC_F_WRITE bit to 1, otherwise 0

(b)

Set the VIRTQ_DESC_F_AVAIL bit to the current value of the Driver Ring Wrap Counter

(c)

Set the VIRTQ_DESC_F_USED bit to inverse value

7.

Perform a memory barrier to ensure that the descriptor has been initialized

8.

Set d.flags to the calculated flags value

9.

If d is the last descriptor in the ring, toggle the Driver Ring Wrap Counter

10.

Otherwise, increment d to point at the next descriptor

This makes a single descriptor buffer available. However, in general the driver MAY make use of a batch of descriptors as part of a single request. In that case, it defers updating the descriptor flags for the first descriptor (and the previous memory barrier) until after the rest of the descriptors have been initialized.

Once the descriptor flags field is updated by the driver, this exposes the descriptor and its contents. The device MAY access the descriptor and any following descriptors the driver created and the memory they refer to immediately.

2.8.21.1.1 Driver Requirements: Updating flags

2.8.21.2 Sending Available Buffer Notifications

The actual method of device notification is bus-specific, but generally it can be expensive. So the device MAY suppress such notifications if it doesn’t need them, using the Event Suppression structure comprising the Device Area as detailed in section 2.8.14.

The driver has to be careful to expose the new flags value before checking if notifications are suppressed.

2.8.21.3 Implementation Example

Below is a driver code example. It does not attempt to reduce the number of available buffer notifications, neither does it support the VIRTIO_F_EVENT_IDX feature.

/* Note: vq->avail_wrap_count is initialized to 1 */ 
 
/* Note: vq->sgs is an array same size as the ring */ 
 
 
 
id = alloc_id(vq); 
 
 
 
first = vq->next_avail; 
 
sgs = 0; 
 
for (each buffer element b) { 
 
        sgs++; 
 
 
 
        vq->ids[vq->next_avail] = -1; 
 
        vq->desc[vq->next_avail].address = get_addr(b); 
 
        vq->desc[vq->next_avail].len = get_len(b); 
 
 
 
        avail = vq->avail_wrap_count ? VIRTQ_DESC_F_AVAIL : 0; 
 
        used = !vq->avail_wrap_count ? VIRTQ_DESC_F_USED : 0; 
 
        f = get_flags(b) | avail | used; 
 
        if (b is not the last buffer element) { 
 
                f |= VIRTQ_DESC_F_NEXT; 
 
        } 
 
 
 
        /* Don’t mark the 1st descriptor available until all of them are ready. */ 
 
        if (vq->next_avail == first) { 
 
                flags = f; 
 
        } else { 
 
                vq->desc[vq->next_avail].flags = f; 
 
        } 
 
 
 
        last = vq->next_avail; 
 
 
 
        vq->next_avail++; 
 
 
 
        if (vq->next_avail >= vq->size) { 
 
                vq->next_avail = 0; 
 
                vq->avail_wrap_count ^= 1; 
 
        } 
 
} 
 
vq->sgs[id] = sgs; 
 
/* ID included in the last descriptor in the list */ 
 
vq->desc[last].id = id; 
 
write_memory_barrier(); 
 
vq->desc[first].flags = flags; 
 
 
 
memory_barrier(); 
 
 
 
if (vq->device_event.flags != RING_EVENT_FLAGS_DISABLE) { 
 
        notify_device(vq); 
 
}

2.8.21.3.1 Driver Requirements: Sending Available Buffer Notifications

2.8.22 Receiving Used Buffers From The Device

Once the device has used buffers referred to by a descriptor (read from or written to them, or parts of both, depending on the nature of the virtqueue and the device), it sends a used buffer notification to the driver as detailed in section 2.8.14. Note:

For optimal performance, a driver MAY disable used buffer notifications while processing the used buffers, but beware the problem of missing notifications between emptying the ring and reenabling used buffer notifications. This is usually handled by re-checking for more used buffers after notifications are re-enabled:

/* Note: vq->used_wrap_count is initialized to 1 */ 
 
 
 
vq->driver_event.flags = RING_EVENT_FLAGS_DISABLE; 
 
 
 
for (;;) { 
 
        struct pvirtq_desc *d = vq->desc[vq->next_used]; 
 
 
 
        /* 
 
         * Check that 
 
         * 1. Descriptor has been made available. This check is necessary 
 
         *    if the driver is making new descriptors available in parallel 
 
         *    with this processing of used descriptors (e.g. from another thread). 
 
         *    Note: there are many other ways to check this, e.g. 
 
         *    track the number of outstanding available descriptors or buffers 
 
         *    and check that it’s not 0. 
 
         * 2. Descriptor has been used by the device. 
 
         */ 
 
        flags = d->flags; 
 
        bool avail = flags & VIRTQ_DESC_F_AVAIL; 
 
        bool used = flags & VIRTQ_DESC_F_USED; 
 
        if (avail != vq->used_wrap_count || used != vq->used_wrap_count) { 
 
                vq->driver_event.flags = RING_EVENT_FLAGS_ENABLE; 
 
                memory_barrier(); 
 
 
 
                /* 
 
                 * Re-test in case the driver made more descriptors available in 
 
                 * parallel with the used descriptor processing (e.g. from another 
 
                 * thread) and/or the device used more descriptors before the driver 
 
                 * enabled events. 
 
                 */ 
 
                flags = d->flags; 
 
                bool avail = flags & VIRTQ_DESC_F_AVAIL; 
 
                bool used = flags & VIRTQ_DESC_F_USED; 
 
                if (avail != vq->used_wrap_count || used != vq->used_wrap_count) { 
 
                        break; 
 
                } 
 
 
 
                vq->driver_event.flags = RING_EVENT_FLAGS_DISABLE; 
 
        } 
 
 
 
        read_memory_barrier(); 
 
 
 
        /* skip descriptors until the next buffer */ 
 
        id = d->id; 
 
        assert(id < vq->size); 
 
        sgs = vq->sgs[id]; 
 
        vq->next_used += sgs; 
 
        if (vq->next_used >= vq->size) { 
 
                vq->next_used -= vq->size; 
 
                vq->used_wrap_count ^= 1; 
 
        } 
 
 
 
        free_id(vq, id); 
 
 
 
        process_buffer(d); 
 
}