Tuesday, October 6, 2009

Why Not Roll Your Own Web Framework?

I was actually going to do a long series of "Why TurboGears, Why Not ___?", but have decided not to do so, especially since a couple of paragraphs will summarize my feelings on the others well. After that, though, I'll explain why you should almost never roll your own framework.

So, why not one of the other popular frameworks? Why not Django? Well, I think it can be summarized by reading this article. Seriously, executing code in another module to read the URLs stored in the module? What? If that's not ugly, I don't know what is. And if that's the sort of code to write with Django, I don't want to use it.

As for Ruby on Rails: Plenty of people love it. I've looked at the Ruby syntax, and found that I feel it is unreadable. Add in the idea of scaffolding, which amounts to generated code that I'm not allowed to modify directly (not without potentially having to rebuild the scaffolding later, thereby discarding my changes), and I have to discount Ruby on Rails entirely.

There's plenty of other options, but I'm not going to devote space to them at this time. I'm happy with TurboGears.

For me, the major question for today is simple: Why shouldn't you roll your own web framework?

The answer is simple: Rolling your own web framework may well be interesting, but it is going to take up a large part of your time and effort, and in the end is likely to have fewer features than the other frameworks out there.

Take a look at some of the tasks you'll need to consider:
  • Authentication and Authorization
  • Internationalization
  • Page Caching and Templating
  • Form Validation and Handling
  • Code snippet reuse (a la ToscaWidgets)
  • Java Script library choice and integration
  • Database integration (possibly using an object relational mapper)
And that's just what I could think of inside of a few minutes. Now, after you've got the basics done, you need to look at code organization, maintenance, and optimization. By the time you complete all of this, you're likely to have reinvented someone else's wheel. Furthermore, their wheel is likely to be better than yours, because they've been working on it longer, honing it more, and having a whole community of people help them do so.

Now, consider the developer who is selecting a framework to use: In order to choose between any given set of toolkits, a developer has to sit down, review docs, review requirements, experiment with code, see what fits his style best, see what can be done, etc. The worst part about this involves the amount of time wasted. If, somehow, your custom rolled web framework manages to make the shortlist of the developer who's looking around, then your framework is very likely to waste a significant amount of that developer's time, since it's unlikely to be chosen over one of the other frameworks which have already gone past what you wrote.

The above ignores a few other facts that very relevant:
  • If you code for a living, then any clients of yours which are deployed on your framework are bound to you. This is great for your income, but terrible for your clients' futures. If something happens to you, who else can pick up where you've left off?
  • Any bugs are entirely up to you to resolve. You cannot ask for help, since no one else knows your framework. The only way you manage to change this item is if you somehow make it enough of a success to be noticed. This might be harder to do than writing the code, especially if you're not a marketing expert.
  • Any new features are also entirely up to you to implement. It's all your framework. Until you reach that critical mass, nobody else will help you. And getting to that critical mass produces the classic chicken and egg problem.
None of this means you can't make your own framework. None of it means you won't come up with something better. But it does mean you should think very carefully about what you're doing.

Are you doing it to better the state of the frameworks out there? Has anybody else done anything similar enough? Can your ideas be implemented in someone else's framework? Will your ideas be accepted in that framework when brought to fruition? Would forking an existing project be a better answer than starting from scratch?

These are all questions you must answer. You might find yourself making a new framework, and have very good reasons for doing so, but make sure. Consider contributing to one of the existing ones if at all possible.

Who knows, I might be lucky enough to find you contributing to TurboGears itself. I hope to see you on the IRC channel sometime soon.

3 comments:

metapundit.net said...

>Seriously, executing code in another module to read the URLs stored in the module? What? If that's not ugly, I don't know what is. And if that's the sort of code to write with Django, I don't want to use it.

That's a little FUD-y I gotta say. The article by David Cramer is basically a monkeypatch technique to get other people's apps (not django core) to use an alternate template engine. I mostly work with Django and I never have to write code like this and most people never will. You don't have to if you just want to use Jinja2 in your own django apps either - so pointing this out as damning Django seems a bit of stretch!

Michael Pedersen said...

FUD-dy? Put it this way: That was a random article on Django. I happened to be looking elsewhere, and someone mentioned this article, so I read it.

In other words, from my point of view, it was not something that's a hidden thing. It's open, it's accepted, it might even be expected.

Even if I'm wrong, though, isn't this what WSGI is *for*, to allow us to mount other people's apps inside our own without having to go through the hoops described?

Something is not right when that's something that is found without effort, and the best reply amounts to "I don't have to use this, and most people never will."

And, for me, the worst part about such a comment is that I (invariably) find myself being the person who will have to write just that sort of code. I don't like it, and don't want to write it.

metapundit.net said...

>In other words, from my point of view, it was not something that's a hidden thing. It's open, it's accepted, it might even be expected.

That's fair enough in that I think there ought to be a disclaimer along with David's article.

>Even if I'm wrong, though, isn't this what WSGI is *for*, to allow us to mount other people's apps inside our own without having to go through the hoops described?

Actually what's going on is that David is using Jinja2 as his template library and using a thirdparty django app called django-registration. That's easy enough and requires no weird Django code - a line to add django-registration to included apps and a line in the main url conf file to map a url to whatever urls django-registration exposes would do it.

David wants to supply custom templates for django-registration and it's worth noting that given the default way Django sets up template loading that's easy: global templates are found before app templates so he could copy django-registration templates to his main templates location and modify them. BUT: he want's to use Jinja2 templates instead of the Django templates django-registration expects. He could accomplish this several ways (one way would be to monkeypatch django builtins like render_to_response with versions that use the Jinja2 template engine) but used a technique I didn't really follow to replace some modules and monkeypatch others. This is absolutely not typical code for Django and again: you don't need to use it for your own apps even if you make a non-standard choice of template engines. Therefore it doesn't make sense to damn Django because of this code (analogously: if David was using wsgi to mount a third party Turbogears app he'd be monkeypatching the @expose decorator to force the app to use a different template engine than the one it specified).