A dilettante's guide to HFT networking

HFT, of course, is High Frequency Trading, and it's all over the news nowadays, with topics like the Flash Crash, etc. being of great interest.

Now, please understand: I'm not a network engineer, and I'm not in the financial industry, and in general I have no idea what I'm talking about.

But, I found myself being quite interested by modern HFT systems, particular when I read things like this:

They are to trading what Lamborghinis are to cars: smart, sleek, powerful and fast. Their modus operandi is to use the fastest trading tools on the Street, and to spray the market with millions of orders, with cancels immediately behind them. Besides the gobs of short-term liquidity they provide today in equities, their other contribution is the flickering quote.

And things like this:

Computer-assisted trading models need to react to fleeting opportunity windows that may last mere microseconds, opportunities that would never materialize if data-generated signals couldn’t set the appropriate pace.

The new paradigm is about direct-from-venue ultra-low latency data and per symbol/security subscription capabilities delivered in a normalized format.

Well, this certainly sounds good and technical and right up my alley! So I've been trying to learn more, and I thought I would share what I've learned so far, and maybe people will reply with more information and better sources and then I'll learn even more!

There seem to be two levels at which HFT Network Architecture must be considered:

• Firstly, there is the hardware level, where we are primarily concerned with things like latency, bandwidth, and buffering.


• Secondly, there is the network protocol level, where we are primarily concerned with protocol selection and system implementation

Hardware: Switches, co-location, buffering, etc.

A lot of people are very focused on this, and it does seem important. A number of companies are super-focused on this market, because it's such a big deal. For a good starting point, try the Cisco web site at http://www.cisco.com/go/hft. Here there are all sorts of great resources, such as this white paper on High Performance Automated Trading Network Architectures.

As the Cisco white paper observes, the high-level issues of latency, bandwidth, and buffering can all be broken down into lower-level details. For example, consider latency. The paper notes that there are 5 latency contributors:

1. Serialization Delay


2. Propagation Delay


3. Nominal Switch Latency


4. Queueing Latency


5. Retransmission Delay

Of course, if you're interested in this area, you probably already know that "Light takes about 3 usec to traverse 1 km in fiber", but if you're like me, I found the Cisco papers quite readable and informative.

For another take on this, a good resource is the Blade Network Technologies web site (Blade was just bought by IBM last week). For example, you can read about the monster RackSwitch G8124 in the product brief:

The RackSwitch G8124 provides line-rate, high-bandwidth switching, filtering, and traffic queuing without delaying data, and large data-center grade buffers to keep traffic moving.

All ports non-blocking 10 GbE with deterministic latency of fewer than 700 nanoseconds.

480 Gbps non-blocking switching throughput (full duplex).

Wow! My head just spins! This is incredibly high-end stuff.

Network protocols: routing, messaging, etc.

Once you have the hardware in place, properly co-located and terminated into the data center of choice (e.g., the SAVVIS facility in Weehawken, or the BT Radianz facility in Nutley), you'll want to start understanding your choices in network protocols and messaging infrastructure.

I'm not really a hardware guy, so I found myself considerably more interested in the protocol and messaging aspects of the HFT technologies.

For an introduction to network protocols and system implementation, the best resource I've found, so far, is the BATS US Equity/Options Connectivity Manual, although there is a lot of other information online at the BATS website.

As the BATS manual points out, there are two basic messaging systems that you probably need to be thinking about:

1. Market Data Feeds


2. Order Entry

For Market Data Feeds, the manual states:

BATS offers five different types of market data feeds:

• Multicast PITCH


• TCP PITCH


• TCP FAST PITCH


• TOP


• Last Sale

These data feeds are not for the faint-of-heart:

BATS requires that members allocate a minimum of 1 Gb/s per Multicast PITCH Gig-Shaped
feed

Moreover, it is here that the importance of buffering is really brought home. As the BATS manual states:

During spikes in quote updates, members using less than sufficient bandwidth will experience queuing of their market data. Members using the same bandwidth to both receive quotes and transmit orders may expect their orders to be slightly delayed if they have less than sufficient bandwidth. Many members will find these delays unacceptable and should provision their bandwidth to reduce these delays.

Cisco make a similar point:

A microburst is a traffic pattern that causes short-lived congestion in the network. This is often due to a period of high activity that causes network endpoints to send bursts of traffic into the network, for example during high-volatility periods in the market. Benchmarking infrastructure performance during these high-volatility periods is especially important because this is when the automated trading businesses make the most profits.

For Order Entry protocols, it seems that it's all about FIX. So get thee to the FIX website and learn all about it:

FIX is the industry-driven messaging standard that is changing the face of the global financial services sector, as firms use the protocol to transact in an electronic, transparent, cost efficient and timely manner.

A particularly intriguing pointer in the FIX website is the link to the Plain English Business Practices repository at www.sifma.net. For example, here's the Agency Securities Market Business Practices. This is an interesting approach to trying to make this incredibly complex protocol intelligible to people like me.


There's a lot more to learn about HFT networking, but hopefully these pointers and excerpts get you started in the right direction.