The State of Ruby ORM

We have a lot of different Object-Relational Mapper implementations in Ruby (and a ton and a half of Mongo mappers ;)) and it’s probably a good thing. It seems like the only ORM that really matters right now is ActiveRecord although in “the background” we still have DataMapper and Sequel with growing communities. So I’m wondering…what’s the state of these ORMs? This post is my brain-dump on the subject with a potential to start a disccusion about the future of Ruby ORMs. This is a huge subject but I’ll keep it short.

(beloved) ActiveRecord

I’ve been a rails developer for almost 5 years now and I remember my initial excitement about ActiveRecord just like if it happened yesterday. I remember how blown away I was when I saw ‘an empty’ class in app/models with a ton of functionality added dynamically. I remember how happy I was when I started setting up associations for the first time and seeing that I can easily use them and everything just works OOTB. I remember how awesome it was to use built-in validations – a few lines of simple ruby code and my model is “secured” and invalid data won’t be persisted. Just like that! I also remember what a great feeling it was to use migrations for the first time, oh man there was even a generator which creates files for me, wonderful! Yeah, that was really great and I loved it.

Years were passing and my excitement was slowly disappearing and now it’s almost completely gone. Why? What happened? First of all I’m disappointed with AR3. I thought transition to ARel and the general Rail3 refactor will lead to a better codebase and nicer API. What we have now is like a hybrid of AR2 and something that was promised as AR3 which was not finished in 100%. The general lack of consistency in the API worries me. Too bad because making a major bump in the version number was a great opportunity to clean things up for good. Now we have to wait for 4.0. I’m also still facing various “WTF moments” when I see things like that:

Yeah I know that’s just rude to point things like that, right? On the other hand is this what you would expect from a library which has a version number > 3.0? I’d expect some level of maturity but what I see now is still a bit of a mess. It’s definitely getting better so probably, with this level of support that AR has now, version ~5.0 will be truly great and mature. I’m really looking forward to that.

Through all those years ActiveRecord got a lot of great features though. Lazy queries, attribute serializers, prepared statements just to name a few.
Extracting ActiveModel was also a huge step into the right direction and I’m very happy with it. It’s all great and the improvement is clearly visible.

On the other hand there are still cases where ActiveRecord fails. For instance it still has problems with proper handling of object graph. Which isn’t a great surprise because it’s a tough problem to solve and none of the 3 ORMs has solved it yet.

(bold) DataMapper

With ActiveRecord you get things done, that’s true. Just read the recent post from Xavier Shay “DataMapper Retrospective” where he writes why he likes DataMapper and despite that why he chooses ActiveRecord. He writes “This is my responsible choice at the moment” – I would expect that in many, many cases people pragmaticaly and wisely choose ActiveRecord because the support from community is important and ActiveRecord does have the best support at the moment. When you get yourself into trouble with AR you will likely find help relatively quickly.

After reading Xavier’s post I feel obliged to comment on it. First of all I agree with him and his decisions. DataMapper is trying to solve bigger problems than AR and Sequel. It’s obviously much harder to support any kind of a storage system and it requires a lot of work. It definitely requires more than a couple of years of development and one stable version. DataMapper 1.x series was a great milestone as the API became stable and it’s truly great, just look at this:

DataMapper introduced a lot of fantastic solutions and just recently ActiveRecord started catching up. It doesn’t change the fact that DataMapper 1.x is much harder to use when you have to resort to raw SQL. That’s why I perfectly understand why Xavier prefers AR and Sequel – both ORMs are built specifically to handle RDBMS and they do it in a decent way.

This post would get too big if I started to write more about DataMapper that’s why I’m stopping now. I will write about what’s happening in the DataMapper camp in a separate post where I will give you an overview of our plans for the near future – expect a lot of goodness to say the least.

To quickly sum up – DataMapper is not there either but getting to the point
where we are now was a great experience that’s given us knowledge about how certain problems should
be solved. More about this in a separate post…

(impressive) Sequel

I’ve never been a sequel user. I only tried it out a couple of times in the past and it was really nice. The codebase looks much cleaner than in case of ActiveRecord and the feature set is truly impressive. I’m also absolutely amazed that this project has 0 issues on github – HUGE congratulations to Jeremy Evans. He’s doing a fantastic work and I’m really impressed so you should be.

I remember that Sequel introduced various advanced features before AR did – like lazy data sets (aka lazy execution of queries). You probably quickly forgot or you aren’t aware of the fact that AR was the last Ruby ORM that introduced this feature.

I’ve heard from many people that are familiar with Sequel that it’s simply better than ActiveRecord in every aspect. I would not be surprised if this turns out to be true.

I would love to hear more about your experiences with Sequel so please feel free to leave comments!

The State of ORMs?

Right. So where are we now with our favorite ORMs? In my opinion whether it’s ActiveRecord, DataMapper or Sequel – we’re still “not there yet”. I’ve been looking at other languages lately and it’s pretty clear that in Java, Python and PHP most popular ORMs implement the Data Mapper pattern. I’m not saying that this pattern is better. There’s been an open debate about AR vs DM patterns since for ever and there’s no ultimate answer. The fact is, though, that in most of the cases DM is the preffered one. And ORMs in other languages reflect that. Java with its powerful Hibernate, Python with its SQLAlchemy and PHP with its Doctrine. All those projects are mature, have solid codebases and solve hard problems using well known design patterns.

I really like what I see in those ORMs. They all implement Unit Of Work, Identity Map and a lot of other patterns that I didn’t have time to identify. It’s all there. Damn hard problems solved in a clean way with OOP approach and design patterns that were known for many years. When you open up sources of SQLAlchemy or Doctrine you are able to quickly navigate through those projects – classes are well organized and it’s easy to figure things out.

I understand why people like “the less is more” and “convention over configuration” but personally I would love to see more explicitness in Ruby ORMs. Metaprogramming is a dangerous weapon and we shouldn’t forget about this. You probably think “STFU and switch to Java” now. Well, that’s not gonna happen :)

All of this doesn’t change the fact that I believe we can still do better. By looking at other languages we can find a lot of inspirations but many things can be done in a much better way in Ruby.

I would love to get some feedback from you. Are you happy with your ORM of
choice?

  • Knowshantanu

    I am newbie to Ruby-land, but I like the language very much. I started playing using Ramaze and DataMapper but then migrated to Rails and ActiveRecord after few days. I really liked the DataMapper approach, however their documentation is seriously lagging. I could never figure out how to create a column with FK constraint with non-standard name. And I needed 2-such columns referring to the same column in another table. It was much easier to deal with using ActiveRecord. 

    Blame it on my experience/skill level, but I preferred AR over DataMapper because of the documentation. 

    Here are two more posts related to AR pattern that mainly talk about some other issues mainly related to testing: 
     * http://misko.hevery.com/2009/05/05/the-problem-with-active-record/ 
     * http://misko.hevery.com/2009/06/29/active-record-hard-to-test/

  • Anon

    Whatever happpened to Maglev?

    • jc00ke

      It just hit 1.0. No ORM/ODM needed as it’s not relational or document(al?) just objects. I started a helper library last year called Tesla but haven’t worked on it in a while. It gives you all the niceties of ActiveModel you’d need for forms, etc.

  • Kay

    Perhaps you could use annotate gem?

  • http://twitter.com/shanehanna Shane Hanna

    Sequel already supports DataObjects, it just doesn’t demand you use it which is ideal if you need an faster adapter or something else DataObjects cannot provide. I don’t know if it’s due to the complexity of supporting multiple Ruby implementations or what but DataObjects isn’t as quick as other options which is why I think Sequel giving you the choice is great.

    One of those alternative Sequel options happens to be a small library a workmate and I rolled as a C++ library https://github.com/deepfryed/dbicpp and C extension / ORM https://github.com/shanna/swift to get the performance we needed to continue using Ruby.

    • http://solnic.eu/ solnic

      How about using Virtus in Swift? :)

      • http://twitter.com/shanehanna Shane Hanna

        I did look at Virtus but to be honest it did too many of the things that bugged us about DataMapper so I never got as far as worrying about integration and benchmarking. Off the top of my head we feel the const_missing magic for types, the type casting system and instance variables for attributes with the get/set business don’t really fit what we want Swift to be, simple, explicit and fast.

        • http://solnic.eu/ solnic

          I hear you! I’m planning to extract coercion stuff from Virtus into a separate gem but const_missing is probably here to stay, however I could make it an optional element so you include it only when you want to use it.

          From what you write the only thing that Swift needs are simple attribute accessors without the need for typecasting which is definitely not what Virtus provides.

          • http://twitter.com/shanehanna Shane Hanna

            Yep Swift is an odd duck. For typecasting it assumes you know what your bind values are and have correctly typecast them yourself then leaves it up to the database driver to throw exceptions if you’ve passed the wrong type.

          • http://solnic.eu/ solnic

            Your use-case is yet another reason why I’m thinking about extracting typecasting from Virtus.

  • jc00ke

    I used Sequel once when I was working on a project on Windows that used a SQL Server Express database. I wrote all my queries in SQL first (as that was easiest for this specific project) then translated them to Sequel. It was pretty badass!

  • http://www.waydotnet.com WaYdotNET

    I use mini_record to explicit fields in the model :D

    • http://solnic.eu/ solnic

      oh yeah? I know about this project but I’m very careful with any AR extensions. I’ve had too many troubles using various 3rd party plugins for AR so now I’m trying to limit the dependencies of my projects to the bare minimum.

      I understand you’re happy with mini_record?

  • http://solnic.eu/ solnic

    I’ll address all of these points in a separate post. Thanks for sharing your experiences!

  • http://rosenfeld.heroku.com/ Rodrigo Rosenfeld Rosas

    I was pretty happy with Sequel the first time I used it some years ago for writing a tool for a legacy system. It really was much superior than AR at that time. I’ve been doing very few Ruby programming since then, although I’m still reading news from the Ruby land in a daily basis for several years now. Today I feel that there is no much difference between AR and Sequel from my point of view so I would probably stick to AR. I really prefer the AR pattern over the DM one, but I know this is a personal choice.

    What made me more excited about AR (the Ruby implementation) the first time I’ve seen was that I wouldn’t need to rebuild my own database evolution tool once more, as it has been the case with most systems I worked with before working with Rails ;) The AR migrations rock! Well… almost:

    https://groups.google.com/group/rubyonrails-core/browse_thread/thread/9d181de8bcbd136b

    • http://solnic.eu/ solnic

      I think it should be possible to build an AR-like layer on top of a DM implementation. That would solve the AR vs DM dilemma.

  • http://twitter.com/lailsonbm Lailson Bandeira

    Excellent post! I’ve been wondering about these very things recently. And yep, we can do much, much better than this!

    • http://solnic.eu/ solnic

      Thanks! It’s only a matter of time and things will get better. I’m sure :)

  • mayhap

    I’ve used AR, Sequel and to a much lesser extent DM. Sequel is by far my favorite. The elegance of implementation, the support Jeremy provides, the non-ORM interface makes this a winner package deal. Projects for various reasons don’t always end up being completely ORM suitable. The syntax for Sequel is far easier to work with than Arel, the backend for AR.

  • Yves Senn

    I feel very much the same about AR as you described in the intro. The simplicity and ease of use thrilled me in the beginning. Everything fell into place and made perfect sense. As time moved on I guess my understanding of a solid model layer moved away from implicit to explicit. I want to control my programming model abstract away the database.
    I still think AR is amazing for small to medium applications. In huge projects i miss the power of an ORM like Hibernate, which lets you separate concerns. I’m really looking forward to your posts about your thinkings on DM2.

    • http://solnic.eu/ solnic

      Exactly. This is why we’re so motivated to work on DM2. Lately I’ve stumbled upon this post: http://jonkruger.com/blog/2010/10/19/ruby-on-rails-and-the-single-responsibility-principle/ it’s an interesting discussion in general but there’s one question in the comments of this post which I find really important, a guy wrote:

      “In fact, I do believe that rails are fantastic when you want to build a forms-over-data site. Putting class behavior and data persistence in one place is fine as long as you don’t have much of either. When it grows, however, do you have an option for separating these things?”

      Yeah, that’s a good question…

  • Manu

    We use both and I think DM is cleaner and easier to use. Also as soon as you try non RDBMS, DM is shining in creating easy plugins (simpleDB, S3) and be a nice abstract layer.. AR makes it hard ..

    • http://solnic.eu/ solnic

      Yeah it’s probably even not fair to compare AR with DM cause AR is “just” an RDBMS ORM whereas DM tries to solve a much broader problem which is creating a unified interface to *any* kind of a storage.

      • Guest

        “creating a unified interface to *any* kind of a storage”

        Which to me seems like a fool’s game. Creating a common interface for both relational dbs and nosql solutions is just not practical. Yes, they are both storage mediums, but using them “correctly” requires a completely different approach.

        • http://solnic.eu/ solnic

          This different approach will be provided by separate adapters. This was already partially achieved with DM 1.x and will be highly improved in DM 2.x.

  • Guest

    I started with Sequel and Ruby couple years ago for most of my reporting need, and I haven’t try AR or DM because there’s no need/reason to.   So this is a plus for me already.

    The only issue I had was running MRI Ruby + Sequal (using ADO connector) + MSSQL in Windows.  Jeremy quickly responded, as usual, and suggested to use JDBC + JRuby instead.  While it does solve the problem I had at the time but it introduced another new problem for me (JRuby’s WinOLE is still not working in –1.9 mode).

    Eventually, the problem was solved when TinyTDS became available.

    So another +  for Sequal because of  Jeremy’s dedication to Sequel users.

  • http://twitter.com/fteem Ile Eftimov

    To be honest, I’m not a seasoned Rails developer, but I have worked with AR and DM. What I have to say is that I’m impressed (just as you are) from Sequel. Such nice features. It has the same functionality DM has but using raw SQL is so easy compared to DM. And when I think of Jeremy (the developer) with no errors on github, I don’t know what to say. I’m really impressed with Sequel.I recommend it to everyone in need of a good ORM.

    my 2 cents :)

  • http://www.vadlue.com Alper Akgun

    Started with AR for Rails,  but using DM with Sinatra for 2 years now; I solve all problems finally going deep in DM code. DM has a solid abstraction, less documentation,  less plugins etc.  Having a solid base in SQL, should have considered Sequel earlier..  For Sinatra. Sequel makes more sense I guess.

  • PS

    I find it funny that you point out a weak community support regarding DataMapper compared to AR. I found quite the opposite. It’s really difficult to reach out to AR expert for help. To get help on DM you just have to get on the IRC, ask and probably you solved your problem 15 minutes later.

    • http://solnic.eu/ solnic

      That’s really nice to know! Maybe I wasn’t specific enough – it’s easier to find information about various problems with AR simply because it has many more users than DM. It’s probably true that it’s easier to talk to core DM devs on IRC though :)

  • https://narf.com Jason Thomas

    I chose Sequel because I was working with a badly designed legacy schema that forced a lot of custom SQL including joins. As many have already said, Jeremy is awesome with support and talked me through many issues I was having. One of the drawbacks is the smaller Sequel user base. I was using Oracle and found a bug due to Sequel not supporting the Oracle timezone. AR had this support for weeks but since Jeremy doesn’t have access to Oracle, he is dependent on the community to provide him bug fixes.

  • Jay Godse

    I use AR. I like it for 1-1, 1-many, many-many, and other such stuff. It’s not too useful if you need to do complex queries which include many tables, outer joins, and grouping. The API is not clear. 

  • Geraldo Lopes de Souza

    What I like most in sequel is that when you need sql it has a ruby dsl for constructing clauses
    http://sequel.rubyforge.org/rdoc/files/doc/dataset_basics_rdoc.html
    keeping you distant from sql injection and the same time more close to the db than the Model stuff that sequel has too.

  • http://profiles.google.com/lsiden Larry Siden

    This is a great article.  Thank you!

    One question I have is what’s the difference between the ActiveRecord pattern and the DataMapper pattern? While they are different in theory (I found http://martinfowler.com/eaaCatalog/dataMapper.html and http://goo.gl/iKnA), in my limited experience with DM, it seems to follow the AR pattern in that each class directly maps to a table in the database and each instance of a class maps to a row. Am I missing something?

    I thought about it a little more and I think I just answered my own question. DM has adapters for other storage models like Mongo (non-relational). So it truly is an _adapter_. When it maps to a relational model, then it will have the look and feel of the AR Pattern. Is that right?

  • http://www.wddx.ru Mokevnin Kirill

    doctrine2 on ruby  https://github.com/mokevnin/dogma

  • Jason

    +1 for Sequel when working with existing databases, or when working with a non-Rails DBA.