Jakob Østergaard Hegelund

Tech stuff of all kinds

HTTP/2: We're doing it wrong

2016-11-18

HTTP/2 is a symptom of a disease; it is a terribly ill conceived deterioration of the otherwise pretty good HTTP/1.1 protocol.

Most really terrible ideas (like a computer in your toaster or a 3D television) usually linger for a while and then die and leave us alone. Of course I was expecting for HTTP/2 to go this route, but yesterday was a shocker:

Looking at integrating with Apple's PSN (Push Notification Service) API, it became apparent they only support HTTP/2 access. Oh dear, I don't yet have an HTTP/2 client library (and frankly I was expecting never to need one).

Let's go over the main features of HTTP/2 one by one. I can start with the only one that is slightly justifiable:

Header compression

Header compression is like Donald Trump. Both are probably very good solutions - to problems that you should not have had in the first place.

Staying on topic here, header compression allows legacy applications that have deteriorated into unmanageable piles of crap and therefore use excessively large headers on all their API requests, to gain some performance advantages by supporting compact and efficient transmission of the bloated crap that is their headers.

I have worked with software development for what is more than a lifetime for some of you who read this - I absolutely understand that you can be in a situation where this is useful. Therefore, I am not against the concept of header compression as such, but I will maintain that it is a solution to a problem what we should strive very hard not to have. There are no good excuses for having bloated headers.

Google with their SPDY extension attempted to add header compression to HTTP/1.1 - frankly I think this would have been a reasonable solution. It would be a hack to support legacy garbage that should really be rewritten - but I absolutely understand the business reasons that could justify improving support for (prolonging the useful life of) legacy garbage applications.

Now, as you can see from the Apple APNS documentation, you can't just use header compression. Apparently there are problems if you use it 'too much', so they warn you to send your headers in very special ways to avoid overflowing header compression tables at their server:

APNs requires the use of HPACK (header compression for HTTP/2), which prevents repeated header keys and values. APNs maintains a small dynamic table for HPACK. To help avoid filling up the APNs HPACK table and necessitating the discarding of table data, encode headers in the following way—especially when sending a large number of streams: ...

This really speaks for itself. So we now have a compression scheme (for no good reason) that is so fragile that we need to pussy-foot around it not to overflow the tables? Really?

Keep-alive

In traditional "old style" networking (you know the kind we use today because it actually works), we would traditionally use TCP to provide us with "connections". On top of the connection layer (possibly with layers in between), we would put our application layer - like HTTP for example. HTTP is a transport protocol that allows us to perform more abstract transactions on top of our reliable connections as provided by the TCP layer.

These days, it's always the OS kernels that actually provide the TCP protocol for us. It was not always like that - applications did once implement their own TCP stacks (or they used libraries that did it for them - but still executed TCP in the application). Why would that be? Why does this code live better in the OS?

The reasons are many. As time went by and networks evolved, a lot of smart people learned a lot of lessons and refined TCP to the point where it is today. Conceptually there's nothing difficult about retransmissions, window sizes, or backing off transmissions in case of congestion. But pulling this off in the real world is difficult. Really really difficult. So difficult in fact, that Linux was the first kernel since BSD (that I know of anyway) to attempt to develop a TCP stack from scratch - everyone else, from Solaris and AIX to Windows and HPUX have refined theirs from the BSD stack.

Despite TCP's best efforts to provide reliable connections, someone have now decided not only that it's not good enough and not worth fixing at the connection layer; but they have even decided that "we" can do better at the application layer. "We" being any application client library and random teenager running cloud services out of his mothers basement. The HTTP/2 PING frame is introduced as a means of checking if the underlying TCP connection is still working; and even Apple encourages it's use for this very purpose.

Really guys? TCP already has keep-alive functionality for this very purpose. And yes, TCP keep-alive is not trouble free, but that is not because of some flaw in TCP, it is because the problem is fundamentally hard in the real world. Moving the problem away from the TCP stack and re-implementing it at the application layer will indisputably make it more expensive traffic wise (TCP layer keep-alive is quite efficiently implemented, something that HTTP/2 fundamentally cannot do because it rides on top of TCP itself). It will also, almost definitely, cause all kinds of traffic problems now that we move the responsibility of sending TCP keep-alive away from the operating system TCP stack (which has decades of refinement in it) into the application layer where history already showed that such functionality doesn't belong (we moved our TCP stacks from the applications to the kernel - remember?).

HTTP/2 ping is a terrible idea. The best we can hope for is that nobody uses it. That boat has sailed already though, as Apple is encouraging the PING frame use for checking the state of TCP connections to their APNS service.

Multiplexing

A wonderful and under-used feature of HTTP/1.1 is that of pipe-lining. You can "stream" any number of requests through your HTTP/1.1 connection without waiting for one request to complete before sending the next. This solves latency problems, allows for servers to concurrently process requests even if you use only a single connection, and it's standard in HTTP/1.1.

People don't get this. And I don't get why. It's super simple. Even big-name applications sometimes use multi-part MIME documents to wrap multiple requests together so they can send them in a single POST - "to solve latency problems" (I kid you not).

Anyway... Some servers apparently do not support pipelining correctly. This is hard for me to believe (having implemented a HTTP/1.1 server that supports it just fine - and that was not hard), but at least that's the word on the street.

For this reason, and apparently for this reason alone, all popular browsers disable HTTP/1.1 pipelining. Yes I know, this is hard to believe but I promise you I'm not making this up.

The solution to this, you ask? If HTTP/1.1 pipelining is mis-implemented, you might think that someone would push to get it fixed. But no... it gets better and as you'll see, I couldn't even make this stuff up if I tried:

HTTP/2 … allows interleaving of request and response messages on the same connection … It also allows prioritization of requests

So... in other words: Basic HTTP/1.1 pipelining is too difficult to implement; therefore we now want to implement a considerably more complex scheme. Really?

In conclusion...

I had honestly thought that HTTP/2 was so obviously ridiculous, and difficult to actually implement reasonably well, that no-one would ever actually employ it in production applications.

It appears that we may not be this lucky. At least Apple is requiring HTTP/2 for their APNS API even though their documentation needs to warn developers about how to work around obvious deficiencies in their implementation (the HPACK restrictions).

Since sanity or even just good taste is obviously not going to save us here, I am left hoping that plain simple laziness will keep a mass migration to the dangerous and broken HTTP/2 from ever happening.

Should we start to see HTTP/2 adoption, I can predict a couple of things:

Actually, these are not even predictions. They are obvious facts that anyone can see. Still, I'm going to pass them as predictions and claim clairvoyance.

Affinity to signedness

2016-10-10

It continues to amaze me when I look at other peoples code, how many good developers think in signed integers. This is a short example of such code and a walk-through of why it is mis-guided.

Signedness in the world of computers

Almost nothing in the world of computers is signed; you cannot send a negative number of bytes, you cannot have a negative amount of memory, you cannot put a negative number of elements in a vector and so on and so forth. We will frequently subtract numbers, but we will usually never have negative results.

For example: I may want to send a buffer using a send routine that does not necessarily send all data at once - subtacting the sent number of bytes from the full number of bytes can never be negative (in other words, the routine can never send more than it was given).

Good code like the STL realizes this and typically uses the size_t type for sizes in general (basically it is used for everything you can count that you can keep in memory). It is of course unsigned (for the aforementioned reasons) and it is as large as it needs to be, for any count of "things you can have in memory".

The problem then arises when a signed-integer-loving developer mixes his int-using code with the better size_t-using code. Good compilers will complain about pitfalls in signed/unsigned arithmetic and the developer is forced to respond with an equal measure of type conversions or casts.

The result will usually be unnecessarily verbose code that is less capable of handling large amounts of data (since the sign bit can no longer be used for actually counting). Granted, the last bit is more a problem on 32-bit systems than it is on 64-bit systems, at least at the time of this writing - but you never know then someone decides to use your code on a 32-bit system. If your code is any good (and often even if it isn't), it will end up on all sorts of systems without your knowledge anyway...

Let's take a look at an example I came across today: This is an extract of a piece of production code that actually works, it's just unnecessarily ugly because the developer had an unhealthy affinity towards signed integers:

int sent_bytes = 0; while (sent_bytes != static_cast<int>(data.size())) { int res = SSL_write(m_ssl, data.data() + sent_bytes, int(data.size()) - sent_bytes); int err = SSL_get_error(m_ssl, res); switch (err) { case SSL_ERROR_NONE: sent_bytes += res; notifyObservers(event_t::conn_tx, res); break;

So we have a variable sent_bytes with a nice readable descriptive name, so far so good. It escapes me how that can be a signed integer - how exactly do we send a negative number of bytes? Right, that is difficult indeed. By making it int rather than size_t, we have either 63 bits instead of 64 bits for the counter (which is probably fine) or 31 bits instead of 32 (which may be a problem if we use large buffers). But regardless of whether the more limited positive range is a problem or not, we don't get anything in return. Nothing is gained by opening up for negative numbers; there is no plausible scenario in which we could ever have sent a negative number of bytes. So we get nothing for something and that's never a good trade. Now I would buy the argument for making this a signed integer if it somehow made the rest of the code more readable - let's proceed with the analysis then.

So in our loop we compare the number of sent bytes to the actual number of bytes we wish to send. Notice how a static_cast<int> is used to stop the compiler from complaining that it is dangerous to compare the two types (as they don't have the same range a conversion is made - and that is not necessarily what you want). Now consider for a moment what this cast accomplishes... It casts the value so that the compiler stops complaining - it does not actually solve the problem the compiler is complaining about. This is like shutting off the fire alarm instead of putting out the fire - it stops the noise but it doesn't really solve the problem. The problem of course would (as of this writing) most likely only occur on 32 bit systems where we would have a 2.5G buffer and after sending the first 2G our sent_bytes would wrap and depending on the rest of the logic we may have more or less luck with our algorithm when sent_bytes is -1.5G (or some other number - depending on the good will of the compiler; read on)

And this is actually what makes it even more difficult for me to understand: Unsigned arithmetic in C++ is valid and well defined - it is integer arithmetic modulo 2^n where n is the bit-size of your integer. In other words, when you add one to your maximum integer value, you're back to zero. There's nothing magical about that, but it is extremely useful. In contrast, the standard does not define behaviour of signed overflow - if you rely on signed integers and they overflow, the behaviour is undefined by the standard. Why would anyone tend towards a type that has less standardized behaviour if it does not provide a benefit? The mind boggles.

We move on to the SSL_write line. Here we must subtract the number of sent bytes from the total size of our buffer so as to compensate for adding the number of sent bytes to the start offset of our data buffer. The developer has chosen an alternative means of getting around the compiler warning this time: Construct a temporary signed integer from the unsigned size of the buffer, then subtract the signed sent_bytes from there. Now, if we were to attempt to send 2.5G of data on a 32-bit platform this code would explode, but the compiler does not tell us about this because we skillfully mislead it by inserting valid constructs that hide our real crimes from the compiler.

Ultimately, we add our signed integer res to our number of sent bytes. Naturally, we expect that res cannot be negative when we use it in this manner - no assertion or otherwise is inserted for this, but that is something one could have done. Or we can trust the library to honour its contract and never return success and set res to a negative value. This concludes the use of sent_bytes, and in no situation do we gain anything from having it be a signed value. How about we consider a fixed version of the code:

size_t sent_bytes = 0; while (sent_bytes != data.size()) { int res = SSL_write(m_ssl, data.data() + sent_bytes, data.size() - sent_bytes); int err = SSL_get_error(m_ssl, res); switch (err) { case SSL_ERROR_NONE: sent_bytes += res; notifyObservers(event_t::conn_tx, res); break;

Note the distinct lack of casts and explicit construction of temporaries to work around type mismatches. Other than that, the only functional difference between the two versions are, that this shorter and more readable version will successfully send 2.5G of data on a 32-bit platform, rather than blowing up unpredictably.

I fear that the affinity for signed integers comes from the old "magic return value" where we use non-negative values for the real return value and assign special meaning to negative magic numbers that callers must remember to check for (like SSL_write in the example earlier). That, of course, is the worst excuse; especially when working in C++ that actually has means to deal with exceptional circumstances where less fortunate C developers might feel compelled to return negative magic numbers.

Old habits die hard I guess. But it troubles me that I see this from young developers - they did not code C back in the '90s (in fact they were born in the '90s). I don't have all the answers - this one is definitely a mystery to me.

Useful return from short-circuit or

2016-10-05

This could have been a bootnote on the next post but time flew and now it will have to be a post on its own. Consider two solutions to the same problem:

// C++ // a(), b() and c() return std::optional std::string pick() { return a().value_or(b().value_or(c().value_or("-default-"))); } ;; LISP ;; (a), (b) and (c) return nil or string (defun pick () (or (a) (b) (c) "-default-"))

My point earlier was that having or return not a boolean, but the actual first non-false argument led to some simple beautiful constructs. I stand by that, but there's another important difference between the two solutions. Consider first the actual sequence of evaluations in the C++ example:

// First, as evaluate the arguments to the value_or functions, we // must call all three functions tmp0 = a(); tmp1 = b(); tmp2 = c(); tmp3 = std::string("-default-"); // Now, with the arguments evaluated, we can "fold" back to get // the result tmp4 = tmp2.value_or(tmp3); tmp5 = tmp1.value_or(tmp4); tmp6 = tmp0.value_or(tmp5); // Our result is in tmp6

Since value_or is a function, it's argument must be evaluated before the function can be called. With this follows that it is necessary to evaluate all three functions before we can decide which result to use - even though all we want is the first value. Compare that to a short-circuit or:

(let ((tmp0 (a))) (if tmp0 tmp0 (let ((tmp1 (b))) (if tmp1 tmp1 (let ((tmp2 (c))) (if tmp2 tmp2 "-default-"))))))

The important point here is, that if (a) evaluates to non-nil, it is returned and nothing else is evaluated. Only if it is nil, do we evaluate (b), and so forth.

So in conclusion; not only is a short-circuit or operator that actually returns the first non-false value useful for elegant programming constructs, it is also the most efficient solution for the solution of this simple problem.

In programming, if you find that your solution to a simple problem is not simple, you are in trouble. The short-circuit or is indeed a simple solution to a simple problem.