Before we go anywhere, let us establish the baseline. This is what normal enterprise infrastructure looks like:
Enterprise server + switch — the familiar baseline
Server (2U)
Click any component to understand its roleThis is the baseline we are about to leave behind
Nothing surprising. The CPU is the brain. The NIC moves data between the kernel and the wire. The network is there to connect services, not to compute.
The key characteristic of this world: the network is never your bottleneck. A web server spends most of its time waiting for the next HTTP request to arrive - not waiting for its NIC to drain. A database spends most of its time on disk I/O and CPU query processing - not on network throughput. 25GbE is more than enough headroom for anything a single enterprise server typically needs to send or receive.
This is about to stop being true.