Basic Bites: I/O

Synchronous, asynchronous, buffered, unbuffered

About Track Summary
I/O operations send or receive data using terminals, files, in-memory strings, sockets, Graphical User Interfaces (GUIs), and devices like keyboards, mice, trackpads, and speakers. This involves one of two rather distinct types of processing: synchronous or asynchronous.
  • Synchronous I/O uses functions that don"t return until the operation completes. Programming language libraries provide read and write functions for that purpose.
  • Asynchronous I/O uses functions that supply a call back function and return immediately. The callback is invoked when the operation completes. Programming language libraries provide async read and async write functions for that purpose.
Synchronous operations interact with the platform API to translate to and from an external environment. Asynchronous operations use either I/O completion ports or Windows messages to send and receive data, using the platform API. API calls are routed to device drivers appropriate for a specified device.
Figure 1. Windows I/O completion ports

I/O Completion Ports

I/O completion ports provide a mechanism for processing multiple asynchronous I/O requests. When sending a request the requesting thread returns without waiting for the I/O process to complete. Asynchronous read or write requests using library methods may use I/O completion ports internally. When an asynchronous I/O request is made, an I/O Request (IOR) object is created and sent to the completion port. The IOR holds a file handle for the specified destination and may also hold a callback reference. Each IOR enqueues an I/O Completion Package (IOCPkg) that gets serviced by a thread pool thread which is responsible for sending the request to an appropriate device driver and await its completion. If a callback is registered it will invoke the callback on completion.

Async Await
.Net provides a Task type that is used with async functions that invoke await and is executed on the current synchronization context. If that is a thread-pool context each task is run on a thread-pool thread. If a non thread-pool synchronization context is used, then when the task blocks it is suspended, enqueued, and another task is dequeued by the same thread. I suspect that the Rust async await functionality works the same way.
Figure 2. Windows Event Processing

Windows Messaging

When external devices are used to send key presses, mouse movements, button clicks, ... each device event results in a device driver creating a message and enqueueing to a raw input queue. The Window manager dequeues each message and routes it to an appropriate window. For example, a mouse button click gets routed to the first window which contains the mouse coordinates in its active area. Keyboard key events by default are routed to the window in focus. Routing means that the Window manager enqueues a filtered version of the message to an appropriate window's message queue. Each window has an event dispatcher that sends the message to an event handler for processing. Note that an application may send messages to a hidden window as a means of internal communication.

Streams

Streams provide buffered I/O operations that collect data from possibly several requests and send that data collection to a device driver as one operation. Each basic read or write request needs to enter the platform kernel to access an appropriate device driver. That takes a significant amount of time compared to user mode processing. Buffering reduces the number of calls into the platform kernel and so improves program throughput. Essentially, streams collect a continuing sequence of event data and, based on an internal threshold, send on all of the data since the last operation as a single new operation. This happens for both input and output streams, that is, data events external to the program - input - and data events generated by the program - output.

Consequences

Each application has the option of sending and receiving either synchronous or asynchronous I/O:
  • Synchronous I/O requires the handling thread to block until completion so operations that may block for a long time, like network communications, will adversely affect program performance if handled on the main thread of execution.
  • Asynchronous I/O allows the requesting thread to return immediately, but increases the overall processing load on machine's cores due to creation of I/O completion object and dispatching to thread pool threads.
Each application has the option of using buffered or unbuffered I/O. This is independent of the choice to use synchronous or asynchronous operations.
  • Unbuffered I/O sends each request to the kernel for processing by a device driver. For infrequent requests this requires no addition processing at the expense of more (infrequent) system calls.
  • Buffered I/O collects a series of I/O requests before entering the kernel, resulting in fewer expensive operations, but incurs the buffering overhead.
For most programs the choice of buffering and asynchrony may or may not improve performance and so should be tested before committing to production code.

References:

  1. Understanding the Windows IO System
  2. Asynchronous I/O: I/O Completion Ports
  3. Part 4 - I/O Completion Ports
  4. async await - stackOverflow   Read answer 4, at bottom
  5. .Net async/await in a single picture
  6. Linux and I/O completion ports?