Making ActiveRecord Models Thin

“Skinny Controller, Fat Model” is a well known best practice in Ruby community. Everybody seems to agree with it and follows it. It’s pretty clear what a skinny controller is. The question is what is a fat model and what should we do if it gets too fat? Even better, what should we do to avoid too fat model? I think many people still confuse Domain Model with ActiveRecord. It’s something more and in this post I will try to explain my new approach to writing Ruby on Rails applications.

Also, I would like to thank Steve Klabnik who triggered the process of writing this post by tweeting this:

We need something better. Persistence and logic are two separate responsibilities that every rails app combines.

I’m really glad more and more people are starting to realize this.

Behavior vs Data

When we say “model” we usually think about ActiveRecord. In Ruby on Rails world this is how we established things. “M” in the MVC means app/models with a bunch of ActiveRecord model files. This is where the domain logic of our applications lives. I think we should stop thinking like that.

Martin Fowler defines Domain Model as:

An object model of the domain that incorporates both behavior and data.

We should remember though that the way your Domain Model behaves and the way your data are persisted are two separate concerns. ActiveRecord objects represent your data. They give you a low level interface to access your data. Yes, low level. If you mix domain specific behavior into ActiveRecord models you will create classes with too many responsibilities. By violating Single Responsibility Principle model code becomes difficult to extend, maintain and test. I have seen it many times, I’m pretty sure you have too.

A few months ago I stumbled upon this quote:

I pull the behavior out of my models into other objects that wrap the models. I prefer to make the AR objects simple wrappers around the db-access stuff in AR.

I have a fairly strict rule that controller actions cannot use AR finders or, in fact, interact with AR at all. AR should be accessed within api methods inside your model, not from the outside.

This describes exactly what I’ve started doing in my recent Rails projects. The outcome of this approach is more than great. I literally left
ActiveRecord models with only validation rules, scopes and before/after hooks. The rest is handled by a separate class hierarchy with domain-specific
functionality. Those clases use ActiveRecord models only for the persistence.

Well Defined API

Something that always bothers me in a typical Rails application is the lack of a well defined model API. Your Domain Model should have an interface to every
action your application should be able to perform. If you have an online shop where a user can buy a product then with a well-written Rails application you
should be able to fire up the console and be able to easily perform this operation. If it’s not so simple then you probably want to think about your model implementation again.

What makes it so hard for us to design and implement a good API for our model? In my opinion it happens because we start with the data instead of behavior. For example if you’re building an online shop, how do you start the design and implementation process? In Rails you probably create migration files to create a db schema. Right? You initially think about the database columns you need to create and validation rules you need to define in the models. After you have all this done you start thinking about the behavior. You add various methods to your ActiveRecord models so they can create new objects, validate and persist them. In the end both data and behavior of your system is mixed together in ActiveRecord models. If you change a column in some table, your system stops working and it’s relatively difficult to fix. Why? Because the domain behavior is tightly coupled with the database schema. Because you started with the db schema and added behavior later.

How about reversing that process and starting with the behavior implemented in separate classes that are not coupled with the database schema? This way you
will define your API at a higher level. What’s more important you will start with an API and you will add the persistence logic later.

Behavior & API

The key difference between using ActiveRecord models and domain model classes is that in case of the latter you clearly specify the behavior. For instance if I want to find a product in my online shop, how do I do that? Well, with ActiveRecord Product model I have plenty of choices. I can #find or #find_by_id
or #where(:id => id).first etc. This is problematic because the same operation can be done in many different ways. Our goal is to create a consistent behavior that is the same in every place of our application.

Let’s use a simplified example of an online shop and focus on one core behavior -
selling a product. Here’s a code spike how it could be modeled:

So, Warehouse can find a product, Customer can find a user, a customer instance can pay for a product and Transaction handles selling a product to a customer. With this ridiculously basic example let’s see how we can write a spec for Transaction:

Running the spec gives you this output:

This way we started off by defining our API, it’s pretty simple to use:

Note that we designed and implemented the behavior and we could easily write a spec that checks if that behavior is correct. What about real data and
persistence?

Behavior + Persistence

To continue with the shop example let’s add ActiveRecord models:

Now let’s use these models in our domain model classes:

This way we hide all the details about our db schema behind objects holding the domain logic behavior of our shop application. If something changes with the
ActiveRecord models you will only need to change the implementation in one place because there’s one way of finding a user and a product and placing an
order.

Testing Benefits

With the approach I described it’s really easy to write solid tests. You can test the behavior in a complete isolation from the db models which results in fast execution of those tests. Most of the logic of your system can be unit-tested without touching the database, this means thousands of test examples running in less than a second. On the other hand when testing ActiveRecord models you are only concerned about validation rules, hooks and finder methods cause there’s no more logic there. It makes the AR tests really clean and easy to maintain.

Feedback?

I understand that it may seem like a heavy approach and Ruby on Rails is all about rapid development and writing less code. However, every project I’ve seen that evolved in something more than a blog written in 15 minutes, sooner or later become a huge mess. I don’t think programmers are guilty here. We’ve been taught to use certain tools and practices and now it’s time to move on, take a step forward.

Still here? Awesome! I would love to get feedback about how you’re dealing with complex logic in your Rails applications – so feel free to comment and let’s start an interesting discussion.

  • http://twitter.com/chipcastle chipcastle

    Fascinating article…  Please give us more, more, more!

  • http://twitter.com/chipcastle chipcastle

    Fascinating article…  Please give us more, more, more!

  • joshuaflanagan

    “I literally left ActiveRecord models with only validation rules, scopes and before/after hooks.”

    I’m wondering why you left them with that much. Validation is business logic, and has nothing to do with persistence, unless you are trying to map to an existing database with constraints out of your control. Before/after hooks are a smell that you don’t have persistence encapsulated. You are forced to use the hooks because you don’t know when some piece of code might load/save your models. If you are encapsulating persistence concerns, as this article suggests, then you should be able to perform whatever additional logic needs to run when you load/save a record within those encapsulating methods.

    • http://solnic.eu/ solnic

      I completely agree with you. Especially on that point about hooks being a code smell. I’ve been moving away from using hooks recently and at some point I probably won’t be using them at all.

      What I’m doing right now is something that you could call a  transition phase. I don’t think that Rails is prepared for a true separation of business logic and persistence concerns. I am experimenting at the moment, trying different kinds of approaches and slowly extracting validations from models too.

  • Fabrizio Regini

    I really enjoyed the post and all the comments. I completely understand the pains described here with fat-too-fat models. In my opinion ActiveRecord is one of the main reasons of Rails’ fortune. I’ve been wondering many times how my programming life would be without it but, you know, let’s think positive :). 

    The point about “where to put validations” is one of the most interesting to me, because it undiscloses what AR is designed to do, that is take care of *a lot* more stuff than just persist data. I like the idea of separating persistence and business logic  but I’m wondering how this is achivable without ‘downgrading’ ActiveRecord to something else.

    Looking at the modules that compose AR we can find hundreds of lines dedicated to features that, for the sake of what we are discussing here, should not be there. Validations and Callbacks already mentioned, but also Serialization, and stuff like Naming and methods that are clearly designed with URLs in mind like to_param.

    It seems that what is coming out from here is that Rails is a fantastic framework for prototyping, a perfect tool for a blogging system, and good for an ecommerce, but you need to change ‘something’ when your app grows up. That’s ok, I kinda agree, ActiveRecord is designed to keep persistence and domain logic in the same place, period. 

    At the same time it would be great to find a solution to this problem that copes with what AR objects are nowadays, maybe the most interesting I found so far is the DCI (http://andrzejonsoftware.blogspot.com/2011/02/dci-and-rails.html) one. We can create mixins that attach additional AR related logic (validations, callbacks etc.). 

    Just to mention, the only thing that Rails ships to deal with AR code bloat is Observers, but I guess they don’t fit much in this discussion.

    • http://solnic.eu/ solnic

      I’m hoping that with a real data mapper implementation all my problems will go away. Everything you mentioned (validations, callbacks, serialization) is something that shouldn’t live close to models responsible for persisting data simply because with complex applications your domain model is becoming less and less similar to the structure of your database. Similarities between how you store data and how it looks like in ‘higher level’ layers of your application will always be there *but* it will never be a 1:1 mapping. Even tiny-little differences can be a PITA to handle in your code and this is exactly why the data mapper pattern was invented so you can have a clean and consistent way of mapping your domain model to a database structure.

      Thanks for the comment!

  • Matthias Zirnstein

    what about dynamic filters like filter result by a date range? is it an aspect of the business logic or part of the orm?
    for me its easy to chain conditional filters with scopes in AR. But its an ORM detail. In my latest code i had to do a decision where to but this filter “domain”. i decided to abstract it out in modules. here’s a little example. https://gist.github.com/1610908 

  • sabereent

    You’re implementing the DataMapper pattern using the ActiveRecord lib. Have you tried using DM lib? What experiences did you have?

    When using AR lib, I prefer to have a method like #persistence(object = Order) than:

    def create_order
    @order = Order.create(:user => customer, :product => product) # you can’t mock the Persistence layer here
    end

    I go with:

    def create_order
    @order = persistence.create(self) # let the persistence deal with the persistence details
    end

  • sabereent

    I use User for business classes (app/business/) and UserPersistence for persistence layer (app/persistence/). 

    Because the term Model refers to both business *and* persistence layers, I don’t have an app/models/ dir (app/models/{business,persistence} could work too). People usually think of a Model as AR object only, whereas business classes are left stranded in the lib/ dir, which is semantically non-sense.

  • http://lucapette.com Luca Pette

    We can fairly say that the size of the project matters a lot in this context. I mean, when you know you’re working on a big project we should totally separate persistence and business logic. I came from the Java world and I hated (a lot) the millions layers of abstractions you have to face within even simple apps.  But there are a lot of situations where the “common” rails style app feels inadequate. And I can say I miss at least the good separation between domain and data from the Java jungle.  For example,  working with big rails apps I ask myself questions like “How much will it hurts if we want to go from Mysql/AR models to a NoSQL solution?” and the answer is always “WAT”. Seriously, the context matters a lot.

    Besides that, I’m sorry I’ve read this article just yesterday for the first time. It’s a must read IMHO.

  • Bharat

    Grails framework which is inspired by Rails and written in Groovy explicitly allows for separating behavior into a separate category called “services” which makes all kinds of sense.  Without it, Rails developers are forced to put this stuff somewhere.  Putting this in Controllers is frowned upon so we have fat models instead.  Question is where do you put this?  I normally put shared logic in the lib directory in a module.  But I will not shy away from putting the business logic in controllers or even helpers (if they have anything to do with the UI piece).  Otherwise the models become the common dumping ground and therefore become fat by osmosis :)

  • tonywok

    Exactly. I think the decision to architect in this way should be driven by requirements.

    It’s certainly possible to go the skinny controller/thick-model route while the requirements are still very CRUD heavy. However, as soon as more complexities are introduced, moving to a three tiered architecture makes sense. And I think the refactors even become obvious as the complexity increases.

  • Blossoming Brotha!!

    I have been saying this for years!!! My experience in Java, .NET showed me that piling a bunch of logic (business logic) in a persistence layer was ABSOLUTELY wrong. I mentioned this as a comment on a blog 2 years ago and all of the Rails Fan Boys literally started cursing at me, calling me racial epithets, etc, called foolish and dumb AND now someone else says the same thing and it’s ok! Only in America!!’

  • Jochen Fromm

    Does every project which evolves in something more than a blog written in 15 minutes sooner or later become a huge mess? Hmmm. I think everything which evolves and grows very strongly tends to be complex, but it is our task to limit the complexity. To build in behavior in ActiveRecord models is a good way to hide the details of the database schema but it does not make them thin.

  • http://twitter.com/glennrob glennrob

    +1

  • Lance Carlson

    One potential issue I see is that we’re coupled to the API of the ORM you’re using. Do you think it would be possible (and is it a bad idea?) to use some sort of dependency injection or abstract out the infrastructure layer you’re calling so that choice of ORM/Database is not a requirement?

    • http://solnic.eu/ solnic

      Do whatever you need to reduce coupling. It really depends on the project and requirements. If it makes sense – do it. Examples I showed are just…examples, they show basic concept of hiding AR stuff behind your own objects so that you can design and implement API of your app that is concise, clean and speaks the language of your domain :)

  • http://danielcadenas.com/ Daniel Cadenas

    I like the way this approach gets closer to DDD ideas.
    One thing I’d change is the name of the persistence models to match those of the domain models but prefixed/suffixed with something, like CustomerPersistence or CustomerRepository for example instead of User.

  • Pingback: Tea-Driven Development :: Hexagonal Rails – Introduction

  • Pingback: Fast Rails Tests, Smoother Development « Sam Serpoosh Blog

  • depy

    Amen! 

    Coming from the Java world I really like ruby and rails because of the simplicity. But I really hated that every rails app mixed domain and persistence. Finally found out that I am not wrong and that this IS a problem… :) 

  • http://blog.shingara.fr shingara

    You don’t put the test of #commit! in your example. In this test, you have only call to ActiveRecords API. So you don’t use mock in this case. This test need to be in “integration mode” with real data. How you do in this case ? Create a new spec file to do this integration test or you put in same spec file ?

    • http://solnic.eu/ solnic

      Please don’t treat examples in this blog post as a real world code that you could use on production :) Please see my reply to Joe in this thread.

      Currently I use repository pattern to hide persistence details. It is doable even with AR from Rails. In my unit tests I mock only my interfaces of repository objects so it’s safe.

      • http://blog.shingara.fr shingara

        Have you some project advice to see usage of this repository pattern and test of it ?

        • http://solnic.eu/ solnic

          Nope :( I’m planning to write about it at some point though

  • http://profile.yahoo.com/N4ANRGESPALBU6UJS2HKCHKA5I Joe

    Excellent post. But I am a bit confused over the use of Product.find_by_id(id) inside Warehouse.find(). That means that the Warehouse model is explicitly depending on ActiveRecord, and further, on the specific AR “model” Product. That can’t be a good thing, can it?

    I’ve been reading about the best way to decouple the persistence from the domain so I can write better (rails) applications. A lot of reading material, videos on patterns exists but not much that I can find as concrete applications of those patterns; esp. with rails. I’m probably about 5% of the way to getting a coherent API for the application domain while separating persistence. So much work to do :|

    • http://solnic.eu/ solnic

      Code examples in this post are not something I would put on production. They just show an idea of hiding AR behind domain objects. In a real world I’d create an database-access interface that would use AR under the hood. The repository pattern is really good for that purpose.

      Separating persistence from domain logic when using AR will always be confusing and awkward. If you want to do that you should either use the repository or data mapper patterns.

      • http://profiles.google.com/warmwaffles Matthew Johnston

        Would you mind providing and example of this?

        • http://solnic.eu/ solnic

          Yeah I’ll eventually blog about repository pattern and AR in rails.

  • Pingback: The Greenfield App Continuum – Ernie Miller

  • BallmerWasRight

    This is all the stuff I hated about .net. the Java and M$ devs do this type of stuff better than anyone else. I’d definitely look at some .Net MVC projects if you enjoy this kind of stuff.

  • http://evan.tiggerpalace.com Evan Light

    I like the general idea but this still bothers me. I would expect the domain model objects to decorate the AR objects. That is, the domain model classes delegate DB lookup to the AR classes and then return the AR objects.

    Would it not be better if the domain model “find” methods were actually factory methods for the domain model class that were used to wrap returned AR objects?

  • http://twitter.com/AnthonyCGreen Anthony Green

    I think it would depend on your application domain. CustomerRecords, ClientFiles, MembersList. Sometimes a synonym for a physical object like Rolodex can be a good choice.

  • fatgeekuk

    hang on, when you build a transaction, your customer is actually a person (ar:b) so how do you get to the customer object to issue the pay method?

  • Perry Smith

    I agree 99%. The 1% where I’m scared is that no one really puts constraints into their databases. AR doesn’t even provide for them. (I have a gem for it.) Pretend databases don’t even implement them. And you can’t really have bulletproof constraints anywhere but down in the DB itself.

    The only thing this implies is that the testing — especially the integration tests — need to convince the database to throw an exception when its constraints are not met (which means adding the constraints to the database in the first place).

    e.g. you can’t charge a customer when you sell a product that you don’t have and you can’t know that until the transaction hits the database as a transaction that includes charging the customer, decreasing the inventory, doing the shipping, and whatever other constraints really need to be done atomically.

    • http://solnic.eu/ solnic

      I’m glad that you mentioned that. We’re currently working on the next major version of DataMapper where db constraints will play a huge role. DM1 already supports some of the constraints and we want to push that idea further.

  • Paul Leader

    Interesting article. I like the fundamental message about working from the outside in. Getting away from thinking about database tables and persistance is an important step that I think all Rails devs go through at some point.

    I also came to the same conclusion a while back about having an API you can use on the console. Nothing annoys me more in my own code when I find I can’t do something relatively straight-forward with a model on the console, without having to write complex AR statements.

    As with all things, the trick is to learn when and where to apply techniques like this. The danger I’ve seen is a lot of relatively inexperienced developers who are prematurely architecting (prefactoring?) their code, splitting trivial models into presenters/decorators/watchers etc, etc without really understanding why they are doing it.