Trying F# for static types and functional programming

Between all the ado about monads and rereading a literature review of static vs. dynamic typing on Dan Luu’s blog, I decided to take another swing at both static typing and functional programming in one go. I built a thing with F# because, well, I’m not really sure.  It sounded cool, and F# for fun and profit had a lot of pragmatic articles about it.

Prologue: Hash-Oriented Programming

I think to really understand what I learned, we have to start with what I’ve been doing with my life.  I typically write dynamically typed languages (Python, PHP, ES6; and formerly JavaScript + jQuery, and Perl) with mostly weak type systems.

In PHP, I have a tendency to SELECT * from the database and return the record as an array through a few layers, because as long as we paid for the data transfer, there’s “no point” in restricting the visibility of the results.  (In related news, every DB call is hand-optimized.  I’m not sure anymore that this is the right tradeoff, but it’s a comfortable rule that’s really easy to apply.)

In Python or JavaScript, it’s not too different.  I have a tendency to pass the language’s dictionary-flavor type around, pull what I need at any given point, and pass on the rest.

Basically, hash-oriented programming.

There’s one major problem with this, though.  I cannot see what keys might be in this hash, without tracing through every bit of code lying between a use of the hash, and its creation in a database query.  (Or worse, a .json file, which can’t be documented because JSON doesn’t have comments, but which everyone adores anyway.)

I don’t even know how much a modern IDE could help with this issue, since I write everything in vim by default.

Type-Oriented Programming

I went into F# looking for “how functional programmers handle data soup.”  I thought I was doing it right.  It seemed like I was doing the same thing Clojure encouraged with its “pass around maps” philosophy, but I was getting frustrated with the way I would be able to write code, but not come back and modify it without a lot of reading.  “What is even in this array?” is the basic problem.

(Other people are not necessarily willing to do the reading at all, and would just “shiv into” my architecture, which if not precisely beautiful, was consistent.  Before being ruined.)

What I discovered was that F# encourages defining the data soup.  One doesn’t just move a hash around; one builds records and/or discriminated unions to hold the data.

These definitions then feed into the type inference engine.  Instead of “an array,” the exact type is known, allowing many more invalid operations to be caught at compile time.  On top of that, having an IDE connected adds suggestions for label or case names in the editor.

I still made mistakes, of course.  The fact that the conversion from float back to byte silently takes the remainder of dividing by 256 instead of throwing, in particular, affected almost every conversion.

The Project: BlitCrusher

Once upon a time, I wondered, “what would 8-bit truecolor look like?”  Then, I devised a Python2 program using PIL to quantize an input image, and save the result to a regular PNG, for easy viewing.  I had recently rewritten this code in Python3 and Pillow, and I figured it would be a good match for static types and “lots” of math.

The basic image-processing pipeline is to convert to a destination color space (or use RGB as-is), apply a quantization function to each channel of each pixel, and then (if not RGB) return it to RGB for final output.

In my code, there are a set of quantizers such as levels: int -> float -> float that quantizes a normalized value to some number of levels; a set of colorspace conversion functions that take per-channel quantizer functions, like asHSV: (float -> float) -> (float -> float) -> (float -> float) -> PxRGB -> PxRGB; and finally, a foreachPixel function to take an operator function of PxRGB -> PxRGB and an Image, and produce the result image after applying the operator to every pixel of the image.

Note the design for partial application: asHSV (levels 30) (bits 3) (bits 4) produces an operator function for foreachPixel by partially applying quantizers, which simulates a 12-bit HSV space.

Implementation Experience

Designing types was a major overhead.  Part of it was simple unfamiliarity with the language.  For instance, I didn’t know anything about type aliases, so several things caught me by surprise in my very first version.  I initially had:

type Channel = float
type PxRGB = {R:Channel; G:Channel; B:Channel}
type PxYUV = {Y:Channel; U:Channel; V:Channel}

But because the first one is an alias, floats can be fully interchanged with Channel, and in fact, any binary operator on a float and a Channel would yield a float as far as type inference was concerned.  Also, since it’s public, anyone could create a Channel with out-of-range values.  I could create out-of-range values, and it was a persistent source of bugs!

This sloppy interchangeability of primitives wasn’t really what I signed up for, but it took a lot of design time to refine it to something where the function signatures can actually tell the programmer something useful.  For instance, val toHSV: PxRGB -> PxHSV… and the latter is a type with private structure, to enforce data integrity.

Given that this is my first statically typed project in a long time, after a dozen years of professional experience in dynamically typed languages, we can’t possibly conclude anything about this.

Overall, though, I really liked the experience of working in an IDE with types, and being able to check along the way that my mental model was matching what the type inferencing was saying.  Also, I really liked being able to catch incompatible type errors quickly, in the flow, instead of having them crop up at some later run-time.

Lispers talk about the edit-compile-run cycle being slow, and gaining an advantage from having a REPL at hand: it’s just an edit-run cycle.  With an expressive type system and an IDE, though, it’s reduced to an “edit cycle.” The only real slowness hits when something goes awry that couldn’t be expressed in the type system.

Also, both IDEs I used (MonoDevelop and later, Visual Studio) have an interactive F# window, which is a REPL itself.

After this experience, I might go get an IDE (PHPStorm and/or PyCharm) for my regular languages.

Incidentally, after writing BlitCrusher, one of the next things I did at work was to adapt a hash return value (with user_id and very few other fields) to a full, newly-added UserModel type based on it.  Now, everyone using that in an IDE knows what it contains and what methods can be called on it.  (Also, it has methods…)

Quiet Enlightenment about Option

There are plenty of arguments online about the existence of null values. Mostly, the anti-null camp tends to have the opinion that one should use an “Option” type instead, such as Haskell’s Maybe.

The Codeless Code has the memorable description:

“We can fell a Maybe-tree with a Maybe-ax and always hear a Maybe-sound when it crashes down—even if the sound is Nothing at all, when the ax isn’t real or there’s no tree to fall.” … “It empowers us to code without error[.]”

Great!  Something went wrong, and the program didn’t crash, but here’s the great mystery: what does it tell its user?

That’s a problem I had in Go, really.  I could “elegantly” signal failure by closing the channel, which would stop processing further down the pipeline by ceasing to provide items, but then… the main program wired up a pipeline wih N items and got less than N results.  What does it tell its user?

It actually turns out that Option<'a> is not a complete solution.  It’s fine in the persistence layer; that is, if we allow VARCHAR(32) NULL in the database, then an Option<string> is a reasonable (if unrefined) type in the data structure we build from that.

For processing, the related concept of Result<'a, 'b> is much more useful!  Failure isn’t just None; it’s a place of its own that holds data. BlitCrusher makes use of Result<Image,exn> to represent “you either got a processed image, or an exception thrown.”  And, apparently, my golang problems could have been solved with a channel of results, instead of two channels (one of success, one of err.)

What you see in one language really will make you a better programmer in other, unrelated languages.

Lineage

After implementing nearly all of what currently lives in Bitcore and BlitCrusher, I happened upon my first samples of OCaml code.  It’s nearly the same code, except that the OCaml seems to use more semicolons.

Apparently, the lineage is: ML -> OCaml -> F#

I don’t mean that F# literally forked the OCaml codebase, just that the design and aesthetic of F# looks exactly like “OCaml with C# integration.”  The choice of keywords, the structure of the code, how |> is represented, and everything just look almost identical between the languages.

This implies that I should be learning and using more OCaml if I like F# but want to (or need to) write software for people who can’t stand Microsoft, Mono, and/or the CLR.

Closing Thoughts

I really liked writing F# in an IDE.  IDE hints and feedback really shorten the time between introducing and repairing errors… but language design feeds back into how effective an IDE can hope to be.

If static typing is the price for IDE feedback, and allows for a lower defects-per-hour rate once I reach proficiency in the language, that’s a tradeoff I would take.