radex.io

aboutarchiveetsybooksmastodontwitter

Rethinking JSON in Swift

February 09, 2015
Hey! This is an old post. It reflects my views and the state of the world in 2015, but is likely to be outdated. You’ll find me on Twitter, building Nozbe, and making engineering blueprints.

Parsing JSON has been a hotly debated topic in the Swift community almost since day one. The staticly typed nature of Swift proved somewhat problematic when facing with this problem. In the world of Swift, the structure and types of data are well-defined and known at compile time. When decoding serialized objects, they’re not. At the same time, the new features of Swift enable different approaches and interesting solutions that simply had not been possible in Objective-C.

Anyway, this is a yet another article exploring different solutions to what seems to be a very mundane problem of dealing with JSON.

The problem

Let’s start with defining what we want to do exactly. What we have is a dictionary decoded from JSON:

let json = ["name": "Do this", "completed": false, "position": 10]

This is the type it represents:

struct Task {
    let name: String
    let completed: Bool
    let position: Int
}

And this is what we want to get:

let task = Task(name: "Do this", completed: false, position: 10)

Starting point

I won’t try to start from the first principles. Others have written too much for me to do that. I’m going to start with a technique I’ve been using for a while now:

struct Task {
    static func create(name: String)(completed: Bool)(position: Int) -> Task {
        return Task(name: name, completed: completed, position: position)
    }
}

let task = Task.create
    <%> json["name"]      >>> JSONString
    <*> json["completed"] >>> JSONBool
    <*> json["position"]  >>> JSONInt

⬇︎ Download playground

I won’t go into details to explain how it works exactly — you should read Efficient JSON in Swift with Functional Concepts and Generics first.

But just to recap:

json["name"] >>> JSONString

will get the value for name from json and will try to cast it to String. If they key doesn’t exist or its value isn’t a string, the expression will evaluate to nil. (JSONString is a helper function that optionally casts AnyObject to String.)

<%> and <*> are used to apply consecutive arguments to Task.create. If all of the parameters are valid, you’ll get an instance of Task. If any of the parameters evaluate to nil, the whole chain breaks and evaluates to nil. (Again, to explain the black magic behind this, read the article linked above.)

Type inference

The nice thing about Swift is that in most contexts, types are automagically inferred by the compiler. That means you get the benefits of static typing without all of the burden.

In similar vein, why do we need to write things like >>> JSONString for each argument? After all, the header of Task.create already contains the types of arguments we need to pass there. Let’s try getting rid of this verbosity.

func autocast<T>(x: AnyObject) -> T? {
    return x as? T
}

Are you getting it? This tiny function will take any object you serve it and will try to cast it to whatever type the receiver expects. And if the casting isn’t possible, it will return nil.

And that brings us to this:

let task = Task.create
    <%> json["name"]      >>> autocast
    <*> json["completed"] >>> autocast
    <*> json["position"]  >>> autocast

⬇︎ Download playground

Well, umm… That doesn’t look that much better… Sure, we don’t need to explicitly specify types, but in the end, we’ve only replaced JSONWhatever with autocast.

Alternative to currying

So maybe we’re approaching this wrong. Perhaps instead of playing FP tricks, we should do something different altogether.

I’ll start with <%> and <*>. Once you understand how it works, it’s pretty elegant, but the whole concept of currying and optionally applying arguments is rather foreign to most people getting into Swift.

So let’s analyze why we need that in the first place. We have an initializer that takes a few parameters. And we have a few values that might be valid as arguments for the initializer, but each of those values might also be missing or the wrong type. If all the parameters are valid, we want to pass them to the initializer and get an object. But if any parameter is invalid, we just want to stop the process, and return nil.

Currying the initializer and then attempting to apply each argument separately is one solution. But how about this:

func ensure<A, B, C>(a: A?, b: B?, c: C?) -> (A, B, C)? {
    if let a = a, b = b, c = c {
        return (a, b, c)
    } else {
        return nil
    }
}

This function takes a number of arguments, all of which are nullable. If any of them is nil, it just returns nil. But if all arguments have an actual value, it returns a tuple with the same data — but typed as non-optional. Pretty cool, huh?

Now we can do this:

let task = ensure(
    json["name"]      >>> autocast,
    json["completed"] >>> autocast,
    json["position"]  >>> autocast).map(Task.init_)

⬇︎ Download playground

We pass our arguments to ensure and convert tuple of optionals to an optional tuple. And then we use Optional.map to pass that tuple as parameters to Task’s initializer. Unless it’s nil, in which case we do nothing.

What’s Task.init_? It’s an alias to Task’s actual init, a static function that takes the same arguments as init, and constructs an object with them. It’s a workaround to a current limitation of Swift.

By the way. Notice how damn clever Swift’s inference system is. autocast’s return value is inferred from the context, in this case ensure’s argument types. But those are not explicitly defined in any way either. They are only derived from Task.init_’s signature — which is a function passed as an argument to a method on ensure’s return value! I’m quite amazed it just works.

Single helper approach

You might be thinking this doesn’t really look like an improvement. But with ensure as a building block, we have a clear path to combining extraction from dictionary, casting to the right types and applying to initializer into one step:

func decodeJSON<A, B, C, R>(json: [String: AnyObject],
f: (A, B, C) -> R, a: String, b: String, c: String) -> R? {
    let a = json[a] as? A
    let b = json[b] as? B
    let c = json[c] as? C
    let params = ensure(a, b, c)
    return params.map(f)
}

We’re taking as parameters: a dictionary decoded from JSON, an initializer of the type we want to return, and then keys of the values we’ll pass as initializer’s arguments.

In the function body, we’re extractring values from the dictionary and immediatelly optional-casting them to the types expected by the initializer. Then, we’re putting those values through ensure to get an (A, B, C)? tuple. And finally, we’re using map to optionally apply that tuple to the initializer.

Result:

let task = decodeJSON(json, Task.init_, "name", "completed", "position")

⬇︎ Download playground

This. Is. Awesome.

Flaws

I was really excited when I came up with this a few days ago. What a simple, easy, non-abstract solution. Why bother with currying, application, flat-mapping, casting helpers and crazy custom operators, when Swift allows us to do this? I was going to write a blog post about it immediately…

And then I started to see flaws of this approach.

For one thing, notice that I’ve written ensure and decodeJSON to suit an example where we’re decoding an object with three parameters. But what if we want four parameters? We have to define another variant of both these functions. What if we need ten? Oh boy, brace yourself for some boilerplate code!

Also, I was making an assumption that all parameters are required. Well, what if they’re not? What if your object’s model allows some parameters to be nil? Then this approach falls down completely. Shieeeet…

Improved functional approach

Well. That was fun. Maybe these FP heads know what they’re doing, after all. Let’s go back to this example:

let task = Task.create
    <%> json["name"]      >>> autocast
    <*> json["completed"] >>> autocast
    <*> json["position"]  >>> autocast

The biggest issue here, verbosity-wise, is the autocast call we’re repeating with every parameter. As they say, if custom operators don’t help, you’re not using enough custom operators! OK, no one actually says that. Please proceed with caution. But we’re going to define one anyway:

infix operator <| { associativity left precedence 150 }

func <| <T>(json: [String: AnyObject], key: String) -> T? {
    return json[key] >>> autocast
}

This combines value extraction and auto-casting in one step. And it gives us this:

let task = Task.create
    <%> json <| "name"
    <*> json <| "completed"
    <*> json <| "position"

Not too bad.

Optional parameters

With the improved functional approach, it’s really easy to add support for optional parameters. First, we’ll define this little helper:

func pure<T>(x: T) -> T? {
    return .Some(x)
}

All it does is wraps a value in an optional (don’t worry, it will make more sense in a minute). Now we can define an optional version of the <| operator:

infix operator <|? { associativity left precedence 150 }

func <|? <T>(json: [String: AnyObject], key: String) -> T?? {
    return pure(json[key] >>> autocast)
}

And we can use it like this:

struct Task {
    let name: String
    let completed: Bool
    let position: Int
    let project: String?
}

let task = Task.create
    <%> json <|  "name"
    <*> json <|  "completed"
    <*> json <|  "position"
    <*> json <|? "project"

⬇︎ Download playground

How does it work? Consider what happens when we use <*>. The left-hand side function expects an argument of type A, and the right-hand side value is A?. Unless the value is nil, it will be unwrapped as A and applied to the function. This is the case even if the A expected by the function is actually an optional, some B?. If the right-hand side B? is nil, the chaining is aborted, despite the fact that the function would happily take nil. But if we wrap the rhs value in an optional, making it B??, it’s never going to be nil. The value then is either going to be .Some(.Some(value)) or .Some(nil), but never just nil. <*> will always be satisfied, unwrap the outer optional and pass the rest to the function.

Crazy operators to the max

We could leave it like that. We could. It’s a perfectly respectable technique — it improves upon what we’ve started with, and avoids the pitfalls of the single-helper approach we tried. But we can make it even shorter, so we will. If nothing else, then just as an intellectual exercise.

So the idea is to use operators to apply consecutive arguments, just like we did before, but somehow embed information about the dictionary we’re deserializing at the beginning, so that we don’t have to repeat it later.

func </ <A, B>(decoding: (A -> B, [String: AnyObject])?, key: String)
-> (B, [String: AnyObject])? {
    if let (f, json) = decoding, x = json[key] as? A {
        return (f(x), json)
    } else {
        return nil
    }
}

// Usage:
(Task.create, json) </ "name" </ "completed" </ "position"

Instead of applying actual values with <*>, we’re merely passing the keys to json where the values will be found. And we’re starting not just with a reference to the function, but with a tuple of the initializer, and the dictionary. And at each step, after we apply an argument, we’re passing the dictionary further down the chain.

One small issue is that the result of the whole operation is (Task, [String:AnyObject])?, instead of Task?. Fortunately, with Swift’s overloading rules on our side, we can fix that:

func </ <A, B, C>(decoding: (A -> B -> C, [String: AnyObject])?, key: String) -> (B -> C, [String: AnyObject])? {
    if let (f, json) = decoding, x = json[key] as? A {
        return (f(x), json)
    } else {
        return nil
    }
}

func </ <A, B>(decoding: (A -> B, [String: AnyObject])?, key: String) -> B? {
    if let (f, json) = decoding, x = json[key] as? A {
        return f(x)
    } else {
        return nil
    }
}

I defined the first variant as expecting a A -> B -> C function. This ensures that the function has more than one argument left to be applied. It doesn’t mean it only has two — C could very well expand to D -> E -> F. But it means that after applying an argument, we’ll still have a function. The second variant expects A -> B, and simply returns B?. Unless we’re applying the last argument, the compiler will run the first variant, because it’s more specific. But since after applying the last argument we won’t get a function, the compiler will fall back to the second variant. In other words, it just works.

Adding support for optional arguments is fairly straightforward:

func </? <A, B, C>(decoding: (A? -> B -> C, [String: AnyObject])?, key: String) -> (B -> C, [String: AnyObject])? {
    if let (f, json) = decoding {
        return (f(json[key] as? A), json)
    } else {
        return nil
    }
}

func </? <A, B>(decoding: (A? -> B, [String: AnyObject])?, key: String) -> B? {
    if let (f, json) = decoding {
        return f(json[key] as? A)
    } else {
        return nil
    }
}

// Usage:
let task = (Task.create, json) </ "name" </ "completed" </ "position" </? "project"

⬇︎ Download playground

The verdict

Personally, I think I’m going to stick with this technique:

Task.create
    <%> json <|  "name"
    <*> json <|  "completed"
    <*> json <|  "position"
    <*> json <|? "project"

It might not be a part of Swift’s standard library, but it’s not completely custom and novel, either. I’ve already seen other people on the web doing this, and the technique is inspired by Aeson, a popular Haskell library for dealing with JSON. A big benefit of the approach is that it builds on top of currying, applying and mapping — standard functional techniques that, I think, will also become quite common in Swift. The <| and <|? operators are only very small additions to that.

In contrast, the technique presented in the above section is completely non-standard, with operators that are useless for any other purpose. Yeah, it might be a little shorter, but the benefit isn’t that big.

Also, I’m starting to realize another advantage of the Improved Functional Approach. What if one of the properties of our object is actually a nested structure? Or maybe an enum that you can’t autocast from string. With the FP approach, this isn’t a problem, because we can simply fall back to manual type casting:

Task.create
    <%> json             <|  "name"
    <*> json             <|  "completed"
    <*> json             <|  "position"
    <*> json             <|? "project"
    <*> json["settings"] >>> TaskSettings.decode
    <*> json["status"]   >>> { TaskStatus(rawValue: $0) }

Sure, maybe we could add some special operators for that, but how many more are we going to need? The beauty of the functional technique is that it cuts out most of the verbosity, but still leaves us a lot of flexibility.

The future

I’m curious to see what the future brings in the exciting field of JSON parsing. It’s hard to imagine Swifters of the future continue to consider something so commonplace to be “a pain in Swift”, something to seriously discuss.

One slightly annoying aspect of constructing objects with a curried initializer is that we have to define it ourselves. We have to copy and paste init, change commas to parentheses, and delegate back to init. It’s possible to define a curry function that will automatically transform any function to a curried form, but at the moment, it’s not possible to reference initializers as a function. I expect that to be fixed in the future.

Another improvement we could make would be to standardize the operators. >>>, a.k.a bind or flatMap, is normally >>= in other languages. Unfortunately Swift already defines it as right-shift/assignment operator. It’s technically possible to re-define it, but the compiler will sometimes get confused by that — possibly because of conflicting precedence. <%> is normally <$>, but Swift doesn’t allow dollar sign in operators. <| looks the same as F#’s backwards pipe operator, which passes rhs value as argument to lhs function — a very different operation. Aeson defines <| and <|? as .: and .:? respectively. I think those would work great in Swift too, if only they were allowed.

If there was some metaprogramming macro system in Swift, the single helper approach could be viable. We could effortlessly generate variants of the function for any number of parameters we wanted. Maybe it would be even possible to solve the optional arguments problem, by generating variants for each combination of optional and non-optional parameters of the initializer. (See Andy Matuschak’s gist for more explanation.)

It also seems likely that Swift of the future will have a solution to this problem baked into the standard library. JSON (de)serialization itself got NSJSONSerialization, after all. It only happened after many years of pain, but it did happen. Anyway, I’m not sure what the built-in solution to this would be like. I am skeptical that it would be a highly functional technique like the one above, but we’ll see.

And finally… there’s this other idea I have that could improve, among others, decoding objects from JSON. But that deserves an article on its own, so stay tuned!

Credits and further reading

Published February 09, 2015. Last updated October 05, 2015. Send feedback.