API Gateway as an HTTP Service Proxy: Lessons Learned

At work, we’re finishing the implementation of our first API using the Amazon API Gateway for decoupling the API key management, logging, and throttling from the actual backend service. This also marks our first OAuth 2.0 Resource Server, and makes heavier use of Swagger 2.0 for the entire pipeline.  With all these “firsts,” I’d like to share a few notes on our setup.

Perl Futures

I’ve been doing a project with IO::Async and Future, because Amazon::DynamoDB depends on them.

One of the weirdest bits of the API that I’ve run into is that the underlying framework only weakly references Future objects, so if the client doesn’t keep its own reference, then the callbacks will never be invoked. I may not care about “the future object” itself, but I do care about the callbacks I set on it.

I ended up writing a function:

sub hold_future {
    my $f = shift;
    $f->on_ready(sub { undef $f });
    return ();
}

The sub being set does nothing but close over $f in order to create a circular reference. That keeps the Future alive until the callbacks are run. To keep the future from persisting forever, though, it destroys that reference once the future resolves.

One may safely release the future while the callbacks are running, since the resolution machinery is holding onto a strong reference ($self) at that point. Therefore, the future won’t be destroyed until all the callbacks finish.

Now, the rest of my code has clearer purpose:

sub md_delete {
    my ($cb, $id) = @_;
    hold_future(
        dynamo_delete($id)
        ->on_done(sub { $cb->(1) })
        ->on_fail(sub { $cb->(0); log_event("delete $id: $_[0]") })
    );
}

Where dynamo_delete invokes a lot of retry/exponential backoff machinery, and md_delete is a command handler for Memcached::Server.

Having worked with jQuery Deferreds, I find it odd that Future doesn’t provide a separate Future::Promise object. It just hopes that the receiver of the future doesn’t try to call creator-related functions like done.

I also find it odd that the future doesn’t just stick around itself, automatically persisting until the callback phase unless actively canceled by the receiver. Having to manually manage them feels surprisingly alien, even though I’m used to managing file handles and lexical scopes.

Go Slices and the Hash API

I finished my first little program in Go months ago.  One of the mysteries was "Why does Hash.Sum take a byte slice if it doesn't write into it?"

Well, the question is broken.  Let's look at some code that attempts to hash a file's content and save the result into a file-info structure, but which doesn't actually work:

sumbuf := make([]byte, hsh.Size())
hsh.Sum(sumbuf)
fi.hash = sumbuf

What happens?  This allocates a byte slice with both len and cap equal to the output size of the underlying hash, filled with 0x00 bytes.  It then passes into hsh.Sum (which, in this particular case, was crypto/sha512) which copies the internal state, finalizes the hash, then wraps everything up with return append(in, digest[:size]...).

Since sumbuf didn't have any room (make(T, n) is equivalent to make(T, n, n)) the append didn't have anywhere to put it.  So it did what it had to do, allocating a new backing array of sufficient size, then copying both in (aka sumbuf) and the elements from the digest slice into it.  The expanded slice backed by this new array then gets returned... only to be discarded by me.  fi.hash ends up being the slice I allocated containing all zero, which made the program think all files were duplicates.  Oops!

What works?

sumbuf := make([]byte, 0, hsh.Size())
fi.hash = hsh.Sum(sumbuf)

First, the return value must be assigned: the original slice's backing array is shared, but append creates a new slice with a larger length to hold the data.  fmt.Printf's %p will show that fi.hash and sumbuf live at the same address, and they have the same capacity as well, but the len of each differs.

Second, if we don't want append to allocate a fresh array, we need a slice with enough free space after its length to hold the data it wants to copy there.  A slice with nothing in it (yet) and the capacity of the hash length is precisely the smallest thing to fit this description.

Now that we have some working code, let's reflect on the point of passing in a slice to Hash.Sum in the first place.  The goal is to avoid allocating inside Sum, if the caller has space for it.  But Sum already allocates and copies a slice: it needs to finalize the hash for Sum, without disturbing the original state so that writers can still add data.  By working in a temporary buffer on the stack and copying at the end, it doesn't make a discrete allocation on the heap, but it still needs to ask append to copy it.

Why not begin d := append(in, _____) and then work inside the caller's buffer directly?  My guess is that working in a private buffer prevents Sum from leaving partial state visible to the rest of the program.  I don't know if it is, but I would not be surprised if append is atomic from the point of view of running goroutines, and clearly Sum needs to allocate in order to be re-entrant.

PHP unpack()

In Perl, unpack is pretty easy to use.  You unpack something with the format string used to pack it, and you get a list of the values that were packed.  I'm not sure the historical reasoning behind PHP's version of unpack, but they certainly made it as horrible as it could possibly be.

To get Perl-like behavior, the simplest path appears to be:

<?php list(...) = array_values(unpack("vlen/Vcrc", $header)); ?>

Instead of the simple "vV" it would be in Perl, you give each format code a unique name and separate them with slashes.  You have to provide a name and not an index because PHP interprets a number as the repeat count.  There's nowhere to place an index in the unpack format.  Then, array_values() gives you back the items in the order specified in the unpack string, since PHP associative arrays maintain their ordering.  Finally, the field names must be unique, or else unpack will happily overwrite them.

If you try to use "vV" as the format code, there will only be one value unpacked... named "V".  If you try "v/V", there will be second value... at index 1, where it overwrote the first value.

If you're unpacking just one value, you might try to write list($x) = unpack(...) but this won't work—pack inexplicably returns a 1-based array.  PHP will generate a notice that index 0 doesn't exist and assign NULL to $x.

Unintended Consequences (an API Design Rant)

tl;dr

• Your package needs to understand encoding. It can’t just throw away structure and hope for the best.
• If a package is overly simple, it’s likely to be too simple for real-world use; and more likely to be reimplemented with a “slightly less awful” hack because it wasn’t that big in the first place.
• MIME won.  Email packages should understand/generate MIME by default where necessary, and only avoid MIME processing at the caller’s option.  If that.
• Perl’s Unicode implementation adds too much complexity (and it doesn’t help that people don’t agree on terminology).  Now package authors get to write two APIs, one for Unicode and one for octets.

Exposition

Consider the simple program:
use Email::MIME;
$m = Email::MIME->create(
header_str => [
To => join(', ', qw(
  foobar2000@example.com
  spaceball99@other.example.com
  break@domain.example.net
))]);
print $m->as_string;

What happens if you run this on Perl 5.10?  The final email address is “break@domain.example .net”—note the space between “example” and “.net”!

As it turns out, such a thing may actually be legal per RFC 2822: the obs-domain syntax allows for embedded CFWS around the dots of the domain.  However, Amazon SES doesn’t support it, so some email was bouncing with the error message, “Domain contains control or whitespace.”

The rogue space explains the SES failure, but how did it get there in the first place?