Kernel Bypass performance. Is it OS agnostic ?
Today there is a tendency to use Linux as a platform for HPC or HFT. It was quite the opposite a few years back. It’s a result of at least two things: lack of innovation but also lack of attitude. Looking back there was a short window in which Microsoft was targeting High Performance Computing (HPC) trying to position the Windows OS as a platform. They were late to the game, we saw clusters using commercials off the shelf quickly reaching thousands of compute nodes. But they were all (well… 99.99%) Linux. It was a matter of cost but also open source helped design the platform and there is one crucial factor when scaling out: the network. We don’t want to go into every detail but operating larger number of nodes requires a good management plane, but you also need the network for this to be successful. We know from Amdahl’s law [0] that for scalability of an application it is all about latency. Microsoft’s HPC Pack was first trying to position it’s Winsock Direct as a common platform for various network vendors [2]. However, it would not be able to compete on latency and bandwidth with Vendor solutions like Myricom’s MPICH2-MX for Windows. It was so broken that it died out soon. A new architectural approach introduced Network Direct which allowed for bare metal latencies. How does this reflect today, in light of the topic of this article? The latency performance results, measured as roundtrips divided by two were already showing, that Linux and Windows could come up with the same latencies. The context was a bit different, it was using proprietary protocols and endpoints of the testing needed no interoperability. Our focus here is on Low latency Ethernet with protocols such as UDP or TCP which can talk to any other network running the same protocols.
Kernel bypass in Finance
Quite a few tools exist for trading on Windows. At one point in time Windows was not sexy anymore and more and more people moved on to Linux. We at NEIO Systems [1] believe that the OS should not matter and hence are embracing all worlds which make our customers happy. We also are CPU agnostic. (See our article on DDIO).
However, what we focus on most is tailored solutions, including optimizations in every corner we can get access to. What we see in the picture above is that whether you run on Windows or Linux does not matter. We have taken the effort to have a system dual bootable into Centos or Windows and collected tick to trade results. At NEIO we offer the libfsock stack which allows to run on various networks and OS platforms. What we see is that when doing things right with some advanced engineering you can outperform VENDOR stacks. And also, that latest Windows has the potential to get back some market share. So any reason why the OS does not matter? Well, it’s kernel bypass. So the OS actually has only little impact on the Application to Network performance.
References
[0] Amdahl’s law, https://en.wikipedia.org/wiki/Amdahl%27s_law
[1] NEIO Systems Ltd, http://neio.systems , http://www.fastsockets.com
[2] Myricom announces support for Windows Compute Cluster Server 2003 http://web.archive.org/web/20060209030939/http://www.myri.com:80/
