Martin Fowler recently blogged about “Embedded Document” pattern. It’s a very convenient way of dealing with JSON documents in web apps and we are using it for our models in the app we are currently working on.

Closed shell

In this post, we will see how we can abstract away the smaller patterns that occur when using Embedded Document pattern. Here is the code snippet we will be working with (based on code from Fowler’s post):

class Delivery
  def initialize(data)
    @data = data
  end

  def customer
    @data['customer']
  end

  def ship_date
    Date.parse(@data['shipDate'])
  end
end

class Order
  def initialize(data)
    @data = data
  end

  def deliveries
    @data['deliveries'].map{ |d| Delivery.new(d) }
  end

  def quantity_for(a_product)
    item = @data['items'].detect{|i| a_product == i['product']}
    item ? item['quantity'] : 0
  end
end
 
order_hash = JSON.parse(
  '{ "id": 1234,
    "customer": "martin",
    "items": [
      {"product": "talisker", "quantity": 500},
      {"product": "macallan", "quantity": 800},
      {"product": "ledaig",   "quantity": 1100}
    ],
    "deliveries": [
      { "id": 7722,
        "shipDate": "2013-04-19",
        "items": [
          {"product": "talisker", "quantity": 300},
          {"product": "ledaig",   "quantity": 500}
        ]
      },
      { "id": 6533,
        "shipDate": "2013-04-18",
        "items": [
          {"product": "talisker", "quantity": 200},
          {"product": "ledaig",   "quantity": 300},
          {"product": "macallan", "quantity": 300}
        ]
      }
    ]
  }'
)
 
order = Order.new(order_hash)
p order.deliveries
p order.quantity_for('talisker')

We can observe following recurring patterns in above code:

  1. All of these classes need to have a constructor that takes in a data object and stores it in a field.
  2. Many attributes require only a simple look-up in the wrapped hash. (e.g. Delivery#customer)
  3. Some attributes need a look-up in hash followed by application of some transformation. (e.g. Delivery#ship_date)
  4. In some cases, the nested hashes further need to be wrapped in some objects. These nested objects can appear in sequence too. (e.g. Order#deliveries)
  5. Some attributes/methods involve complex operations and are too specific to derive any generalizations from them. (e.g. Order#quantity_for)

Let’s start by defining a class called EmbeddedDocument which provides the base for such classes.

class EmbeddedDocument < Struct.new(:data)
end

This will provide the necessary constructor to a subclass of EmbeddedDocument, thus taking care of #1.

Before we move further, let us conjure up an abstraction named “embedder”. An embedder is nothing but an object with a method named call that takes one of the possible types in JSON document (a number, a string, an array, a hash, or nil) and returns a more useful representation thereof (potentially wrapping it in some subclass of EmbeddedDocument).

Now we can define an identity embedder (something that returns a value as-is, without any transformations) which we will be using for scalar values such as numbers, strings, and nil.

def scalar
  lambda { |x| x }
end

This takes care of #2.

We can define some common transformations in a similar way, and take care of #3. We will define one for date to serve as an example.

def date
  lambda { |x| Date.parse(x) }
end

We will deal with #4 in two steps.

Step 1: We will make a class object associated with every EmbeddedDocument an embedder too. i.e. it will have a call method that takes in parsed data and returns an instance wrapping it.

class EmbeddedDocument < Struct.new(:data)
  def self.call(value)
    self.new(value)
  end
end

Step 2: We will now define a couple of functions that take one embedder and return another one. This is a very powerful technique from functional programming called “combinatory design”. What we are defining below can be referred to as “embedder combinators”.

def sequence_of(e)
  lambda { |array|
    array.map {|x| e.call(x)}
  }
end
 
def defaulted(e, default)
  lambda { |value|
    value.nil? ? default : e.call(value)
  }
end

Let’s define a method key which will allow us to specify a key name and embedder to be used.

class EmbeddedDocument < Struct.new(:data)
  def self.key(name, embedder)
    embedder.call(self.data[name])
  end

  def self.call(value)
    self.new(value)
  end
end

We have all the machinery ready! This is how our original classes will look like with the new utility:

class Delivery < EmbeddedDocument
  def customer
    key('customer', scalar)
  end

  def ship_date
    key('shipDate', date)
  end
end

class Item < EmbeddedDocument
  def product
    key('product', scalar)
  end
 
  def quantity
    key('quantity', scalar)
  end
end

class Order < EmbeddedDocument
  def deliveries
    key('deliveries', sequence_of(Delivery))
  end

  def items
    key('items', sequence_of(Item))
  end

  def quantity_for(a_product)
    item = items.detect{|i| a_product == i.product}
    item ? item.quantity : 0
  end
end

Beautiful, isn’t it? :-)

But this is Ruby. We can do even better! Here are a couple of ways we can improve upon the above solution:

  1. Define a method on EmbeddedDocument’s class that will take only absolute essentials, and define a JSON accessor method for us.
  2. We have many cases above wherein a field access is a simple look-up in the wrapped hash and nothing more. Why not put method_missing to a good use here?

Here’s what our EmbeddedDocument class looks like after above improvements:

class EmbeddedDocument < Struct.new(:data)
  def self.key(name, embedder, accessor_name = nil)
    name = name.to_s
    accessor_name ||= name
    define_method(accessor_name) do
      embedder.call(self.data[name])
    end
  end

  def self.call(value)
    self.new(value)
  end

  def method_missing(sym)
    self.data[sym.to_s]
  end
end

Use:

class Item < EmbeddedDocument
end

class Delivery < EmbeddedDocument
  key :shipDate, date, :ship_date
  key :items, sequence_of(Item)
end

class Order < EmbeddedDocument
  key :deliveries, sequence_of(Delivery)
  key :items, sequence_of(Item)

  def quantity_for(a_product)
    item = items.detect{|i| a_product == i.product}
    item ? item.quantity : 0
  end
end

This is even more concise and beautiful I am sure you would agree. :-)

You might wonder what use is scalar embedder now that we have played the method_missing card. The answer is that it’s still useful, not by itself, but in conjunction with combinators. e.g. defaulted(scalar, 0)

You can find the full code here.

Well that’s it. I hope you find the post interesting and useful. I am quite new to Ruby (as I am sure reflects from my code), so any suggestions, criticism, and other sort of feedback is most welcome.

Note: The app I talked about before is in Scala, not Ruby. I developed this utility first in Scala and then ported it to Ruby. Of course the semantics and abstractions provided by the two languages are quite different, and thus the two implementations have some key differences. This is what the Scala implementation looks like:

import language.implicitConversions
import play.api.libs.json._
import play.api.libs.json.Json.JsValueWrapper
import scala.util.Try

// Lib

trait Embedders {
  type Embedder[+A] = JsValue => A

  def int: Embedder[Int] = _.as[Int]
  def double: Embedder[Double] = _.as[Double]
  def string: Embedder[String] = _.as[String]

  def sequenceOf[A](e: Embedder[A]): Embedder[Seq[A]] =
    _.as[Seq[JsValue]].map(e)

  implicit def factoryToEmbedder[A](factory: JsObject => A): Embedder[A] =
    j => factory(j.as[JsObject])

  def optional[A](e: Embedder[A]): Embedder[Option[A]] =
    j => Try(e(j)).toOption

  def defaulted[A](e: Embedder[A], default: => A): Embedder[A] =
    j => Try(e(j)).getOrElse(default)
}

trait Keys { this: EmbeddedDocument[_] =>
  def key(fieldName: String) = new Key(json \ fieldName)
  def key(fieldNames: String*) = new Key(fieldNames.foldLeft(json: JsValue)(_ \ _))

  class Key(json: JsValue) {
    def as[A](e: Embedder[A]): A = e(json)
  }
}

trait EmbeddedDocument[This <: EmbeddedDocument[This]] extends Embedders with Keys {
  def json: JsObject
  def factory: JsObject => This
  def updated(updates: (String, JsValueWrapper)*): This = factory(json ++ Json.obj(updates: _*))
}

// Use

case class Product(json: JsObject) extends EmbeddedDocument[Product] {
  def factory = Product
  def shots = key("shots") as sequenceOf(Image)
  def background = key("background") as Image
  def masterName = key("masterName") as string
  def id = key("id") as int
}

case class Image(json: JsObject) extends EmbeddedDocument[Image] {
  def factory = Image
}