C++ for the Real-Time Web

C++? Why on Earth?

C++ seems like one of the least likely languages to write a web application in. C++ is associated with complexity, both with language design and the need to manually manage memory. Programming in C++ unleashes a whole class of bugs associated with memory corruption and memory allocation. This is a trade off for giving more power to the developer along generally being much faster than equivalent programs in languages such as Python or Ruby.

Real-time Web as a Use Case

Making a hard real-time application¹ on the web is a futile task. Unpredictable network latencies make this an impossible task. However firm real-time systems are possible². This means that our application can miss a few deadlines without a total system failure.

Deterministic timing is key when designing real-time systems. Systems do not have to be necessarily high performance as long as deadlines are met. Lack of a garbage collector ensures there are no “pauses” in the execution of the application while the garbage collector runs to reclaim unused memory. This is not to say that it is not possible to create real-time applications in managed languages³ but C++ has the advantage of not needing to worry about garbage collection freezing the program.

When might such an application come up? One instance might be if we have a web service that “guarantees”⁴ a response time. These types of services are typically internal services.

Real-Time Bidders

A good example of an external facing application that has a firm real-time requirement are real-time bidders. These servers bid on ads from ad exchanges which act as an auction house for ad real-estate on the web. Each request represents a potential ad placement. Real-time bidders (RTB) typically must respond within a set deadline (including network transfer time). For example an auction server from OpenX must receive the complete response from a RTB within 125ms. RTBs which violate this deadline repeatedly will have less chances to bid on future auctions, reducing the effectiveness of the RTB. This presents a firm real-time problem because:

• failure to meet the deadline is acceptable but quality of service is degraded – not hard real-time
• after the deadline the result (bid response) is worthless – not soft real-time

Side Note About Scheduling (or Lack Thereof)

One topic that hasn’t been covered is scheduling. Assuming the web application is running on a typical flavour of linux, there is no sense of a real-time scheduler. Depending on the strictness of the real-time constraints it may or may not be necessary to modify the scheduling behaviour of the kernel. Since web application are usually the only main processes living on a single server (or at least a highly dominant one), modifications to scheduling may not be worth the effort versus setting a high priority on the real-time process. These are trade offs I have not run into yet because they usually are only needed under very tight and/or strict real-time deadlines.