The Problem of Network Delays

In some situations, network delays can have unintended consequences due to order violations.

In the example above, two independent transactions are issued, a deposit of 100 EUR and a withdrawal of 50 EUR from the same bank account. If we start with an empty bank account, a severe delay of the first message can have the second transaction getting cancelled because at this point the deposit has not been processed yet. This points to a deeper underlying problem. In many applications, we implicitly rely on some ordering principle which a bare communication channel cannot guarantee.

The Struggle for Causality

In a simplified model, we might want to ensure that messages are received in a relative order, meaning that a message sent first is also received first by the other endpoint. We typically call this model First-In-First-Out (FIFO). Protocols like TCP can ensure this within a single TCP session by worst case delaying out-of-band messages and performing reordering under the hood.

Unfortunately, FIFO ordering can only ensure causality between two endpoints, but generally not between more endpoints. As the picture above shows, all messages are compliant with FIFO ordering but the red message is overtaken by the sequence of black messages. From an end-to-end point of view, this case violates causal ordering between the first and the third node. By causality, we mean the capacity of one event influencing the other, which in a generic system for which we have no semantic understanding of cause and action, we have to assume to be a temporal property. Since the network is asynchronous, however, the observation of a message arriving gives us very little indication of when it was sent, especially when the message was forwarded through one or more intermediate nodes.