Just A Summary : Category typo, everything about typo

The secret of comedy is...

pdcawley@bofh.org.uk (Piers Cawley) — Tue, 08 Jan 2008 11:46:00 -0600

... timing. You either have it or you don’t.

Does this count as good timing?

Finish up some improvements to the way Typo sweeps cached pages
Announce Typo 5.0
Go down with a horrible cough and cold that leaves you exhausted and incapable of hacking
Discover that the ‘improvements’ in Typo’s cache sweeping can, occasionally, cause it to wipe the entire Typo installation directory
Stagger out of bed. Attempt to fix problem
Release Typo 5.0.1
Discover that the fix doesn’t work
Bleargh!
Let your co-maintainer deal with the fall out before releasing Typo 5.0.2 which does fix the cache sweeper
We hope
Recover enough to write a blog entry

Okay folks, Typo 5.0.2 is out and it appears to be working. I’m running it here, and I’ve had no problems so far. I’ve still got the cold, but it’s nowhere near as horrible as it was (went to bed at 5pm on New Year’s Eve, woke up at 11am on New Years Day – first time I’ve missed the turning of the year in ages).

Typo 5 is out - and more on the future

pdcawley@bofh.org.uk (Piers Cawley) — Sun, 30 Dec 2007 09:10:18 -0600

Right, we’ve cut a Typo 5 gem and it’s on rubyforge and heading to various mirrors I hope. Frédéric’s writing the release notification which will be appearing on Typosphere Real Soon Now.

It’s been a surprisingly tricky process – we’re now requiring Rails 2.0.2 because the workings of view_paths have changed in a way which means we can’t quite make themes with Rails 2.0 and 2.0.2 and working with the edge seems like the more sensible proposition. If you’re on the bleeding edge, you should find that you get the right Rails via svn:externals anyway.

Typo futures

Meanwhile, I’ve been playing with stet and I’ve come to the conclusion that, although there’s mileage to be had in a radically slimmed down approach to the way Typo works, I’m better off simply removing the misfeatures from Typo and building from there – there’s a surprising amount of stuff that needs to be done in a competent blogging engine that Typo gets right – starting again would be throwing the baby out with the bathwater I think.

However, this does mean that if you’re following the Typo SVN trunk, you’ll be seeing a reduction in features in the short term. We’ll be copying the current trunk to a 5-0-stable branch before we start with the featurectomies though, so if you’re just after bugfixes, you’ll be better off there.

Multiblogging

We’re aiming to have multiblogging in the next release, but we’re rethinking the how of it. Right now, the ‘Blog’ object adds a bunch of complexity to code that would be much happier simply assuming that it has the database to itself. So we’re going to look at switching to a database per blog approach, that way our core code can pretty much forget about the complexities of multiblogging, and (at least initially) anyone who wants multiblogging can get there by monkeying with configuration files – of course, we intend to add a web based admin interface once things settle down and we know how things are going to work.

Caching

Caching is always a bugbear in any typo installation. Because we want to be installable on the widest possible range of hosts, we can’t rely on the presence of handy tools like ‘memcached’. Also, some of our users are operating under some fairly severe memory and process constraints, so it makes sense to have the webserve serve static files as much as possible. Meanwhile, tools like Evan Weaver’s Interlock are pointing the way towards seriously effective fragment caching. I shall be looking into implementing something that conforms to the interlock interface, but which can use an arbitrary cache backing store for fragments and maintain a full page cache. It’ll be interesting to find out if this is doable…

Atom Publishing Protocol

ActionWebService is going to go away – it’s already in the ousted branch of the rails SVN repository, and including it in Typo to support the various different admin APIs is getting painful. So, we’re going to preempt it. We won’t be getting rid of the various XMLRPC APIs until the pain becomes too great, but we are going to be concentrating on implementing, and strongly favouring, the Atom Publishing Protocol.

Feeds for everything

In particular, we’ll be adding atom feeds for all sorts of administrative data as a means of enabling people to write external tools for, say, spam protection, comment moderation and notification tasks. Right now, there’s a great deal of computation happening on the server side every time someone, say, comments on a post – in the kind of resource limited environments some people are running Typo in, that’s too much work. Switching to a feed + APP approach should help enormously with resource utilization.

Speaking of resources…

Using the server to render article previews is… suboptimal. Expect to see a javascript based preview system akin to the one I use for comments here.

Rails 2.0 and the Future of Typo

pdcawley@bofh.org.uk (Piers Cawley) — Sun, 16 Dec 2007 10:35:00 -0600

So, if you’ve been watching the Typo tree, you’ll see there’s been a fair amount of activity on it since Rails 2.0 got released. There’s a new default theme replacing the rather creaky ‘azure’, and a fair amount of work on getting our code compatible with the current state of Rails. As we work on this, it becomes apparent that Typo’s code is getting horribly brittle. I have said before that there’s been several places where we’ve zigged before Rails zagged, and we’re paying the price for that. It doesn’t help that our test coverage is distinctly ropy either – and I’m probably guiltier than most for letting things get into that state.

So, our goal is to get what we have cleaned up and working with Rails 2 before releasing Typo 5.0. Once that’s done, that line of code will go into maintenance mode – there are still plenty of bugs to fix and documentation to write, but I’m afraid that extending that base is becoming too much of a chore.

Which is why I have a new path in my local svk repository, //stet. I’m using this for experimental development of a new, slimmed down blogging engine that will be, first and foremost, a capable Atom Publishing Protocol host. Things like spam processing will be removed from the core of the application, but we’ll provide a suite of webservice clients that will consume the ‘unmoderated feedback’ webfeed and use APP to either approve or delete the feedback as appropriate.

Theming (at least initially) will probably be confined to Javascript and CSS changes, and I’m even thinking of exposing the sidebars as Atom collections – certainly I expect that, in the first cut, sidebars will be static – if you want content that looks dynamic you’ll have to do it via javascript.

My initial goal is to slim things down as far as I possibly can – I want to build a blogging engine that can cope with the tight memory constraints of shared hosting by off loading much of the heavy lifting to client boxes. After all, I have far more processing capability available to me on the laptop I’m typing this on than the slice of Site5’s hosting infrastructure that’s actually running the blog. By making things small and static, I also hope to wring good performance numbers out of the tool as well – expect aggressive page caching at the very least.

Another important goal is easy migration of Typo databases. I expect to be writing models and controllers from the ground up, but converting the database should just be a matter of running a migration.

Experimental

Of course, stet’s currently very experimental – about the only thing that’s actually written so far are a couple of routing plugins which should help radically simplify our routes.rb (expect an article’s url to change from /articles/2007/12/16/rails-20-and-the-future-of-type to /article/2007/12/16/rails-20-and-the-future-of-typo, but with a redirect in place to cater for the old style urls). I may have grandish plans for the thing, but I could equally discover that I’m off up a blind alley, in which case you can expect me to return to the current typo codebase with a few more lessons learned.

ActiveResource?

I remain unconvinced by ActiveResource as a technology. I agree with the authors of RESTful Webservices – good webservices are joined up. They take full advantage of what could be described as the defining technology of the world wide web, the URL based hyperlink to knit resources together in a discoverable fashion. An ActiveResource based webservice may well be a good HTTP citizen, but it’s still not really ‘webby’ enough for my taste. Which means the Atom Publishing Protocol will remain my friend for most of the things I hope to do with stet. It may be harder to write a good APP server, but I’m convinced that it’s a much better interface for clients, and you should always favour ease of use over ease of implementation. If nothing else, we’re aiming to have more users than developers. Many more.

Typo on Rails 2

pdcawley@bofh.org.uk (Piers Cawley) — Fri, 05 Oct 2007 15:09:04 -0500

Whee! Sat in my home directory is a version of typo that appears to work with Rails 2.0. I ended up giving up on the themer approach which proved to be very hard to get up and running transparently – things kept disappearing off around infinite loops, which is no fun at all, let me tell you.

So, ended up cheating and ended up monkeying with @@view_paths directly, which is almost certainly against the rules, but has the cardinal virtue of working.

I’m sanguine about it because the way themes worked on rails 1.2.x also involved fiddling with undocumented methods which aren’t there in edge rails.

So, it now remains to walk around the thing, kick the tyres, write a few more tests and generally get myself to the point where I’m happy that it all works, and merge my local branch with the main line. I’m going to wait until 2.0.0 proper has been released, but now that I’ve done the work I expect to be merging it sharpish.

My head hurts

pdcawley@bofh.org.uk (Piers Cawley) — Sun, 23 Sep 2007 14:09:24 -0500

During DHH’s keynote at RailsConf Europe it was apparent that there’s a great deal to like in edge rails, so I thought I’d have a crack at getting Typo up on it.

Ow.

I’d expected the pain points to be related to routing, but it seems that the rails routing system is approaching the level of the Excel calculation engine – nobody dares touch it for fear of breaking things, so typo’s custom routes seemed to work quite happily. There were a few things that have been deprecated, pluginized or moved out of the set of modules that’s automatically included when you do a rake rails:freeze:edge, but they were pretty easy to sort – the deprecation messages are a good deal more informative now than they were last time went deprecation squashing. There’s a surprising amount of stuff that’s been removed without any deprecation warnings though, which isn’t very sporting. DHH said there would likely be a 1.2.4 release (possibly a day before 2.0) with a bunch more deprecation warnings covering everything that’s actually going away, so if you’re thinking of moving a maturish app to Rails 2.0 it might make sense to wait for 1.2.4, install that, squash warnings, and move on up to 2.0.

The real pain comes from themes. Typo’s themes rely on Rails internals working in a particular way, but they don’t work like that any more. In theory, the internals appear to be more theme friendly, related to allowing plugins to include views. The problem is, that it’s possible to change Typo’s theme without restarting the server, and the new themish internals don’t expect anything to change until the server’s restarted.

So, I’ve been playing with plugins. The most promising approach appears to be that of the themer plugin, which gets pretty close to doing what we need, and does it in a way that seems like it should work with both 1.2.3 and Edge Rails. It does appear to be making some radically different assumptions about the structure of the themes directory, but the basic framework is good and I should be able to make things work by making our current them object conform to Themer::Base’s interface and duck type my way to the sunny uplands of Edge Rails compatibility.

Which will be nice.

I like the themer approach a lot. Instead of monkeying about in the guts of rails, it monkeys about in front of Rails. It overrides render so that you can pass it a theme/lookup object. If it sees a lookup object, it uses that to rewrite the rest of the render arguments into a form that will render the right thing using the standard implementation of render. In a work project I’ve taken a similar approach to handling polymorphic routes for things like:

map.resources :pictures do |pics|
  pics.resources.comments
end

map.resources :users do |users|
  users.resources.comments
end

I ended up with a to_params method defined on my Comment model, and stuck an extended url_for in front of the default Rails version, which looks something like:

def url_for_with_to_params(*arguments)
  if arguments[0].respond_to?(:to_params)
    with_options(arguments.shift.to_params) do |mapper|
      mapper.url_for_without_to_params(*arguments)
    end
  else
    url_for_without_to_params(*arguments)
  end
end
alias_method_chain :url_for, :to_params

Which is so much neater than the last time I attacked this particular problem (see the acts_as_resource plugin).

One of the nice things about Rails is that, although it’s opinionated and somewhat liberal with the syntactic vinegar for things the core team don’t think is the Right Way, they’re pretty good at leaving the door open for people like me who have other opinions. Both the themer plugin and my as yet unpluginized extension of url_for work by using existing capabilities in new ways and, because those capabilities are documented we can expect them to continue to work over multiple versions of Rails. Plugins that achieve similar effects by monkeying with Rails’s internal interfaces are hostages to fortune. Internal interfaces are free to change at any time, even between point releases, so a plugin can be left high and dry with surprising rapidity. Just ask the Rails Engines folk.

Typo stuff

pdcawley@bofh.org.uk (Piers Cawley) — Fri, 31 Aug 2007 03:07:04 -0500

If you’ve been running on the Typo edge recently, you’ll be all too painfully aware that there have been issues with the cache being flushed at the wrong times and not flushed at all at others. Which is not a happy state of affairs.

However, I’ve recently got back to working on Typo and, after a few warmup refactorings and some missteps, I think caching is working properly now.

There’s lots more work to do (there always is) but I reckon that you could do worse than bump your installation to r1513, which is what I’m running here.

How I learned to stop worrying and love aggressive mockery

pdcawley@bofh.org.uk (Piers Cawley) — Wed, 06 Jun 2007 05:21:00 -0500

There’s something enormously liberating about writing an RSpec description that starts like:

describe ArticlesController, "feeds:" do
  before do
    @the_mock = mock('everything', :null_object => true)
    ActiveRecord::Base.stub!(:find).and_return(@the_mock)
  end

  it "/articles.atom -> atom feed" do
    get :index, :format => 'atom'
    response.should render_template('_atom_feed')
  end

  ...
end

The :null_object flag to rspec’s mock function is remarkably potent.The resulting mock will return itself from any method call that hasn’t got some other expectation set up. When I’m testing that my index methods render the appropriate views for the format, I don’t care that all the various variables have been set up correctly – I’ve already tested that in another description – I just want to get to the point where I’m about to render a template.

What this does is decouple my tests. I can change the way that the index method fetches its stuff from the database and I’m only going to have to change the innards of the specs that test that.

I tend to think of this as being analogous to the object oriented pattern of trying to write your methods at a single level of abstraction. Within a given spec, I should only be setting up expectations that are directly related to what I’m testing.

Dogfood time again

pdcawley@bofh.org.uk (Piers Cawley) — Sat, 02 Jun 2007 03:08:00 -0500

So, I just pushed the first step of what I’m thinking of as the Great Typo Controller Reorganization to the typo repository and updated things here. It’s always scary when I do that – local testing’s all very well, but running on a live site is a different matter.

I’ll be talking about the changes in a later post, there’s a couple of things in the recent change that I’m rather pleased with, as well as a Rails plugin that I hope will be widely useful, but which I need to document.

What the?

pdcawley@bofh.org.uk (Piers Cawley) — Sun, 13 May 2007 03:42:00 -0500

Wow:

$ rake spec
...

156 examples, no failures

$ ./script/spec spec
...

156 examples, 2 failures

$ rake
[unit tests, all pass]
[functional tests, all pass]
[specs...]

156 examples, 2 failures

For extra points, the 2 failures from running the plain rake are not the same as the failures from running ./script/spec spec. And if I run ./script/spec spec after a full rake run, I get a host of extra failures.

I wonder what I’m doing to so comprehensively screw up test isolation.

Ho hum.

Things which are fun

pdcawley@bofh.org.uk (Piers Cawley) — Tue, 08 May 2007 02:12:00 -0500

It’s definitely fun to commit a major rework of something that’s been bugging you to SVN. It’s slightly less fun to check it out in your production server and have it fall over until you remember you to retweak the environment.rb file.

Anyhow, we’re now running on Typo 4.1.1+r1438. Typo 4.1.1 got released last night and, pretty much as soon as it had cleared the gates I checked in a major rework of the Article and Feedback state mechanisms.

What state mechanisms?

When you’re modelling blog posts, and comments, it’s very easy to think of them in terms of simple status flags. An article is published, or its a draft, so we have a #published boolean and we’re done.

Except, what about an article that you want to publish at a specific time? And how do we work out when to send notifications and trackback pings? Should you send the pings again when an article is first published then withdrawn, then published again?

It’s complicated, and you can end up with a rats nest of complex conditional code.

So, you work out that an article can be in one of several states. We went with :new, :draft, :just_published, :published, :publication_pending, :just_withdrawn and :withdrawn, and we used the State Pattern to handle this. All our tangled conditionals were replaced with simple delegations to the state object. There’s still some conditional code, but there’s a great deal less of it now.

Until recently, we implemented the states using Rails’s composed_of helper, but it’s not really suited to the task – you know your code isn’t happy when you find yourself calling a class ContentState::Factory and overriding its new method. It was also really hard to divine the workings of the state machine.

So, during a discussion of Coding without ifs I found myself sketching a way of setting up a state field declaratively and thought to myself “Hey, I could implement that for Typo and make things a good deal clearer…”

So I did.

Now, if you look at the top of article.rb, you’ll find:

  has_state(:state,
            :valid_states  => [:new, :draft,
                               :publication_pending, :just_published, :published,
                               :just_withdrawn, :withdrawn],
            :initial_state =>  :new,
            :handles       => [:withdraw,
                               :post_trigger,
                               :after_save, :send_pings, :send_notifications,
                               :published_at=, :published=, :just_published?])

  include States

And it sets up the delegations (as well as a set of #new?, #draft?, ... predicates based on the names of the valid states) for you. I pulled the States module out into a separate article/states.rb file, but given a simple set of states there’s no reason not to declare them inline.

I’m really pleased with it. I’ll probably do a bit more work on the interface (I want it to be more language like), add a has_strategy declaration (which I intend to use for our feedback spam checking system) and then extract a plugin from it. If you’re interested in doing something similar in your own code right now, the thing you need to pinch is lib/stateful.rb in the Typo distribution, and the only documentation is the source. Check app/models/feedback/states.rb and app/models/article/states.rb for examples of states written to this interface.

Other things which aren't fun

pdcawley@bofh.org.uk (Piers Cawley) — Mon, 07 May 2007 14:29:00 -0500

Last year, you would have been forgiven for thinking that Typo was pretty much dead in the water as an ongoing project. Typosphere was a placeholder, changes were few and far between, the app was a bloated monster. So, people switched, in droves, to Mephisto the new, (and excellent) kid on the block. Heck, even Tobi, the original author of Typo, has switched.

Mephisto’s a great piece of software, and I’m as sure as I can be (without taking a closer look) that its underpinning code is cleaner than typo’s. When you’re developing on a rapidly moving platform like Ruby on Rails, there could be said to be a second mover advantage – the later you start, the more likely Rails is to already do what you want, and the less likely you are to zag where Rails later zigs.

However, Rick and Justin have other projects and demands on their time, and once something is Good Enough, it’s hard to summon up the motivation to make it better until you find something that you want your application to do.

So now, as Typo comes out of hibernation, it seems that Rick and Justin have been getting it in the neck because their project is ‘stalled’. Okay, so I admit, I did smile ruefully to myself when I read that.

Then I read the comments. Wow. Some people have some serious entitlement issues.

It’s been said before, it’ll be said again I’m sure: if you don’t like something about an open source project, “Patches welcome!”. If your patches are repeatedly rejected: it’s open source, fork off. If a project is moribund: offer to help.

Complaints just put people’s backs up.

I think it’s great that there’s choices available if you want a blog built Rails. Different perspectives on a problem space are important – if nothing else, everyone gets to steal from each other, everyone’s apps get better and everybody wins.

Things which aren't fun

pdcawley@bofh.org.uk (Piers Cawley) — Mon, 07 May 2007 09:13:00 -0500

Let’s say you’re running tests against your rails application and a test fails.

“Hmm…” you think, “I wonder what could be causing that, let’s run that test file by itself.”

The test passes.

“Okay… I wonder if it’s the rake test loader, let’s run that test file using that.”

The test passes

“Right, it’s only the third test file on the list that’s failing, let’s try running just the first 3 test files.”

The test fails.

“Okay, so it’s one of the first two test files that’s causing the problem. Let’s remove one and see what happens.”

The test passes.

“Right, because I’m paranoid, let’s try running it with the other test file instead. It should fail.”

The test passes.

“Hmm… the failure appears to be fixture related, let’s try turning of use_transactional_fixtures in test_helper.rb and running the failing set of tests again.”

The test passes.

Houston, we have a problem

Part of our problem is that Typo is so huge and old, and its test suite is somewhat spotty. Something we’re doing in the workings of a couple of our controllers is confusing transactional fixtures and that can’t be good. I suppose my first step is to get all the tests that are still failing with transactional fixtures turned off passing and see if that solves my transactional problem, but I can’t pretend it’s a fun option.

Cunning Typo Sidebar Tricks

pdcawley@bofh.org.uk (Piers Cawley) — Sat, 14 Apr 2007 03:36:16 -0500

If you’re reading this on the website rather than through a feedreader and you look to the left you’ll see either a bunch of links to cited books or an Amazon ‘self optimizing links’ banner (though, if it keeps ‘optimizing’ like it has being I’ll be changing it to something else). This is implemented using a mildly hacked version of the standard typo amazon_sidebar plugin.

“But,” I hear you say, “The sidebar’s over on the right, and Typo can only support one sidebar.”

And you’re right, Typo’s admin interface can only handle configuring one sidebar, and I’m not entirely sure how to make it work with too – hopefully our new maintainer and usability advocate, Frédéric de Villamil has some ideas along those lines. So, how does it work.

Let’s take a look at a fragment of my layouts/default.rhtml file shall we?

<div id="sidebar_lhs">
  <%= render_sidebars \
    AmazonSidebar.new \
      :title => 'Citations', 
      :associate_id => 'justasummary-20', 
      :maxlinks => 10) %>
    </div>

The thing about Sidebar objects is that there’s no real need to store them in the database. It’s done as a convenience so you can manage them through the admin interface, but if you want to use a sidebar widget anywhere in your layout there’s nothing stopping you.

Actually, that’s not quite true, before revision #1407, the render_sidebars helper couldn’t take an argument, so you had to jump through a bunch more hoops, but (as you’ll see if you follow the link), the code change is trivial, if you need the functionality but don’t want to move to an SVN version of typo you can just change that one helper. The change should work in any Typo newer than about 8 months old, and the underlying idea should work with much older version too, you’ll just have to think a bit harder to apply the patch.

Typo Brain Dump

pdcawley@bofh.org.uk (Piers Cawley) — Fri, 13 Apr 2007 12:51:00 -0500

In case anyone’s interested, I’ve written a list of some of the things I’m currently mulling doing with Typo below the fold.

Sorting out routes

I’ve mentioned this before I’m sure, but I’m not entirely happy with the current state of Rails’s routing. I’m not talking about the actual underlying code (which is scary enough), more about what it can do. The problem is that you have to know the specifics of what makes up a route in order to generate one. And you have to know it everywhere you use the route. You can make good enough guesses when you’ve got ‘simple’ routes – I wrote acts_as_resource to do just that, but it doesn’t work for Typo’s routes.

Consider the two sorts of things Typo can provide a feed of: articles and comments. In a perfect world we could use pretty much the same builder templates for everything and generate the routes simply by calling url_for(@thing), which would generate a link to /articles/2007/04/13/typo-brain-dump, or /articles/2007/04/13/typo-brain-dump#comment-212 as appropriate. It’s at its ugliest in the comment case. There you are, innocently generating a link to a comment and you’re required to know that its URL is generated from the broken down publication date of the original article and its ‘permalink’, and you have to remember what the parameters are called. Suddenly you’re writing code like:

    article = comment.article
    url_for \
      :year => article.year, 
      :month => article.month, 
      :article.day => article.day,
      :title => article.permalink,
      :anchor => "comment-#{self.id}"

Even when you hide that behind a helper method it’s all kinds of ugly. We’ve cheated slightly and taken the view that, since we’re modelling webpages here, it makes sense that they would know their URLs¹, so you find a comment’s URL by doing a_comment.permalink_url, which looks like:

    def permalink_url(anchor=:ignored, only_path=true)
      self.article.permalink_url("comment-#{id}")
    end

But that tends to make people who still think that objects are data structures throw a bit of a wobbly.

My current thinking on how to reduce the pain here is to rejig Rails routing to allow it to multilevel keys, so the basic article URL would be specified as:

    /articles/:article[year]/:article[month]/:article[day]/:article[permalink]

In a perfect world, ‘articles’ would be inflected, so an article’s URL would be /article/2007/04/13/typo-brain-dump, but the URL for all the articles posted in April 2007 would be /articles/2007/04/, but I’ll worry about that. For now I can set up specific routes and have done.

I’ve started work on this, but it’s proving tricky… Ho hum, no reason to stop striving I suppose.

Atom Publishing Protocol

The Atom Publishing Protocol is just lovely. Typo should support it. That is all.

Canvas based rendering

I really don’t like RHTML templates. Or HAML ones. Builder’s okay, but it seems to me that Seaside has this dead right. We should be passing a canvas/builder object to a model’s #render_on method and then calling #to_html, #to_xml, #to_json or whatever on the object we passed in. There’s probably a whole series of posts around this idea, starting with how using such a mechanism would allow for some seriously potent filtering and other capabilities and working up to how we’d make theming work (I think it’s possible to support existing themes with some fancy footwork, but I need to experiment).

A question for the lazy web: Is anyone already doing canvas based rendering in Ruby?

Cruft

There’s too much code in Typo. It could stand to receive some serious pruning and refactoring. The problem with this is confidence in the test suite which manages to be large, fairly brittle and inadequate all at once.

And it’s at least as much my fault as it is anyone else’s. I need to go through the test and get some kind of handle on where it needs to be extended to cover what Typo actually does. And then I need to extend it. And fix the test names. And empty all those horrible YAML fixture files. And…

¹ It’s not quite that simple (hah!). Articles and Comments don’t know their URLs, the Blog object is the repository of that knowledge, but they do know which of their attributes are important for building a URL. Comment#permalink_url delegates to Article#permalink_url, which in turn calls Blog#url_for, which mocks up an ActionController and delegates to that.

Cover me; I'm going in!

pdcawley@bofh.org.uk (Piers Cawley) — Thu, 15 Mar 2007 14:06:00 -0500

Mmm… Rails routes, don’t you love them?

Well, not unreservedly, no.

Here’s my current problem: Typo articles have a permalink that looks like /articles/:year/:month/:mday/:permalink, so the permalink for this article is /articles/2007/03/15/cover-me-im-going-in and when someone visits it, the controller gets a params hash which contains keys along the lines of:

{ :year => '2007', :month => '03', :mday => '15',
  :permalink => 'cover-me-im-going-in', 
  # the session, controller, etc }

Which is okay as far as it goes¹, but it’s a complete pain when it comes to generating the url. I have to do:

url_for :year      => @article.year, 
        :month     => @article.month, 
        :mday      => @article.mday, 
        :permalink => @article.permalink

Every value in that hash is dependent on a single object, every key is the same as the method used to find its associated value. I want to be able to write a route that looks like this:

map.article_url \
  '/articles/:article[year]/:article[month]/:article[mday]/:article[permalink]',
    :controller => 'articles', :action => 'show'

article_url @article # generates the route

That’ll do for starters; I’ll worry about making it Resourceful once I’ve got the basic recognizer and generator working.

So, that’s what I shall be working on within acts_as_resource for the next while. Once I’ve got it working, I’ll start using it within Typo and we can get rid of the various model methods we ended up adding to cope with generating our URLs.

Why bother?

The thing to remember is that, while routes like /:resources/:id are okay when you’re prototyping, they’re problematic for production. Opacity is not a virtue. Exposing internal database keys in the URL is a positive vice.

Meanwhile /articles/2007/03/15, /articles/2007/03 and /articles/2007 all have obvious meanings and typo supplies the obvious index pages when you go to look at them.

¹ However, it would be better if it were actually more like: {:article => {:year => '2007', ...}, :controller => ...} because then I can find the article by a simple Article.find(:first, :conditions => params[:article]) and let sanitize_sql do all the work for me.

Getting the Rspec religion

pdcawley@bofh.org.uk (Piers Cawley) — Sun, 26 Nov 2006 02:52:00 -0600

I’ve been eyeing the rspec and rspec on rails packages and thinking I should give them a go.

To my eye at least, something like:

context 'Given a published article' do
  fixtures :contents

  setup { @article = contents(:published_article) }

  specify 'changing content invalidates the cache' do
    @article.body = 'new body'
    @article.invalidates_cache?.should_be true
  end
end

context 'Given an unpublished article' do
  fixtures :contents

  setup { @article = contents(:unpublished_article) }

  specify 'changing content keeps the cache' do
    @article.body = 'new body'
    @article.invalidates_cache?.should_be false
  end
end

reads far more fluently than the equivalent Test::Unit based tests:

class CacheSupportTest < Test::Unit::TestCase
  fixtures :contents

  def test_changing_published_article_invalidates_the_cache
    art = contents(:published_article)
    art.body = 'new body'
    assert art.invalidates_cache?
  end

  def test_changing_unpublished_article_keeps_the_cache
    art = contents(:unpublished_article)
    art.body = 'new body'
    assert ! art.invalidates_cache?
  end
end

So, I installed everything and started to work on a new class in Typo using rspec. Rather annoyingly, this seemed to break the current test suite, so instead of working on my new model class, I set to porting the existing suite.

And, on about my third test suite, I found what I think is a bug in the suite. I’m not sure it’s a bug, because, the way the test is written (by me, I admit it), masks the intent quite dramatically. I’m also finding that the freedom to name specifications and contexts in English rather than method_names_that_go_on_for_ever is forcing me to come up with much more useful descriptions of what I’m testing. I find myself working on making the spec runner output read reasonably well as English, and doing that casts light on what is and isn’t being tested.

I’ve known for a while that Typo’s test suite is, um, spotty, but the porting process is really helping me get familiar with what’s being tested. I’m half tempted to start adding extra specs as I go, and if I could work out how to keep the existing tests working while I did it, I would, but my priority for now is to get to the point where I can check the specs and be confident that the new specs are no worse than the old tests.

Because I’m much more confident that I know what the specs are doing, I’m also confident that it won’t be hard to revisit them to help specify typo’s behaviour better. I’ll just have to give myself the discipline of beginning each coding session with half an hour of fleshing out the specifications before I get back to adding behaviour.

A sketch of declarative ActiveRecord Migrations

pdcawley@bofh.org.uk (Piers Cawley) — Fri, 24 Nov 2006 06:18:00 -0600

Writing migrations can get pretty tedious when you’re being scrupulous about writing both the up and the down side of the migration. Okay, so the Textmate ninjas amongst you can use scarily clever snippets to populate the down migration while you write the up method, but I can’t be the only Mac user who still prefers Emacs. And not everyone gets to run on Macs either.

So, inspired by something Jamis Buck wrote about designing a DSL, I’ve been sketching out a DSL for describing the easy parts of a migration in declarative style. None of this is implemented yet, but I’m pretty sure that it’s relatively simple to implement for a decent Ruby metaprogrammer. I’m brain dumping it here so I can come back to it later, or, you never know, someone might have implemented it by the time I revisit…

The sketch

class AdjustContentTable < ActiveRecord::Migration
  class Content < ActiveRecord::Base
    include ActiveRecord::Migration::Delta

    delta do
      +(:extended, :text)
      -(:excerpt, :text)
      +index :text_filter_id
     end
   end
  end

  callbacks_for(:up) do
    before_any_changes
    Content.before_changes { ... }
    Content.before_additions {...}
    Content.after_additions {...}
    Content.before_removals {...}
    Content.after_removals {...}
    Content.after_changes {...}
    after_any_changes
  end

  callbacks_for(:down) do
    ...
  end
end

Implementation Pointers

By making delta do its work in a block, it’s possible to instance_eval the block using a Delta object that has appropriate implementations of + and -.

Then callbacks_for[:up] puts its block into, say @callbacks[:up] and the block is yielded to to set up the callbacks by the default implementation of up. The default up probably looks something like:

  def self.up
    @callbacks[:up].call
    apply_all_deltas
  end

Of course the fun bit is the implementation of apply_all_deltas – the per class apply_delta is reasonably simple:

  def self.apply_delta(direction)
    callback(:before_changes)
    apply_additions(direction)
    apply_removals(direction)
    callback(:after_changes)
  end

  def self.apply_additions(direction)
    if has_additions?(direction)
      callback(:before_additions)
      eval additions_for(direction).to_ruby
      callback(:after_additions)
    end
  end

  ...

Thinking about it, apply_all_deltas is going to take a certain amount of grovelling about in the namespace to work out what classes need to have their deltas applied, but even with just per class delta declarations, this should be a useful thing to implement.

Braindump ends

Sorry this doesn’t come with a handy dandy Rails plugin with it all implemented; your contributions in this area would be very gratefully received.

Typo on Rails 1.2

pdcawley@bofh.org.uk (Piers Cawley) — Fri, 24 Nov 2006 05:00:00 -0600

Typo users with longish memories will recall the absolute disaster we had when Rails 1.1 shipped and Typo wasn’t even remotely compatible with it. Even edge Typo didn’t work with 1.1. Chaos and confusion, strong words from DHH, all manner of unfun things.

So, this time, we’re not going to let it happen. We hope.

For starters, the current Typo release, 4.0.3 is locked to Rails 1.1.6 – if you install it from the gem it will insist that you have an appropriate Rails gem installed too. If you install it from SVN, it will use the magic of svn:externals to stick the right version of Rails in vendor/rails and, if you’re using SVK, we assume you have enough of a clue to populate vendor/rails appropriately.

What about Edge Typo then?

Edge typo is currently fetching edge rails via svn:externals, but I’ll be switching it to the Rails 1.2 branch in the next couple of changesets, which should mean that, as soon as Rails 1.2 is released, we’ll be able to push out Typo 4.1.0 pretty promptly.

What does Rails 1.2 mean for Typo?

Over time, Typo and Rails have diverged somewhat; in part it’s because we were trying to do some stuff before Rails made it easy, and we didn’t necessarily make the same choices as Rails did, so we’ve got a pretty huge code base that’s doing an awful lot of heavy lifting that could (and should) be left to Rails. One of my goals in the next development cycle is to bring Typo more into line with the Rails way. Some things are sacrosanct of course: Article permalinks are sacrosanct – if an article has the url http://www.bofh.org.uk/articles/2006/11/24/typo-on-rails-1-2 today, then it’s going to continue to have it for as long as you’re running typo and on the same server. And any new articles will have canonical URLs of a similar form.

Pretty much everything else is up for grabs though. We’re going to try and keep well known URLs working but they may be redirects to a more RESTful canonical version, and we’re not about to break your feeds any time soon, but the admin urls will certainly be changing (again we’ll put redirects in for a few releases, but they won’t be supported in perpetuity).

Precalculating in Typo

pdcawley@bofh.org.uk (Piers Cawley) — Mon, 30 Oct 2006 02:09:00 -0600

You may or may not know that Typo is multiblog capable under its hood. There is a blogs table in the database and every single request ends up fetching the blog object by doing a Blog.find_by_base_url query.

We don’t have UI support for multiblogging though, and the vast majority of Typo installations have exactly one blog object. Even when multiblogging comes along, there won’t be that many blogs in any given installation.

The problem is, the instance caching tool we’re currently using only caches objects that are fetched using Model.find(id), so we end up with a basic page hit finding the blog by searching for the base url and then, later, fetching it again using the id.

One option is to fix the caching mechanism.

But that’s too much like hard work. So, in the latest checking on the typo trunk I’ve fixed things so that environment.rb builds a hash which maps from base urls to blog ids. So now, the first thing we do with any incoming request is use the hash to find the id of our blog object and then do Blog.find(id) et voila we’ve fetched our blog and populated the instance cache as well. God is in his heaven, all is right with the world, and we’re doing one less query per page hit.

Getting back to Typo

pdcawley@bofh.org.uk (Piers Cawley) — Sun, 29 Oct 2006 07:19:00 -0600

I’ve been taking an accidental sabbatical from Typo. It started during conference season, then there was a bunch of housekeeping and general home type stuff that needed doing, so I’ve been away from the codeface for a while (and I still haven’t turned my EuroFOO talk into a podcast, or finished scanning and uploading all the EuroOSCON photos…)

However, I am back and coding, and a new typo feature just made it to the top of my todo list: This morning, in a fit of over enthusiasm, I managed to zap the 36 most recent comments on this blog. And, for bonus points, I didn’t have a backup.

So, now I have a backup regime in place. And the next feature I add to Typo is going to be a confirmation box when you attempt to delete any feedback that hasn’t been marked as spam.

Recent changes added to the typo trunk add CachedModel support, which dramatically reduces our query count per page, which is no bad thing either; I have other ideas for getting that count down too.

So, until the Typo Trac is back up, watch this space, I shall try and post here whenever there’s any significant changes on the trunk.

Updates

Google cache is my friend!

Miles Barr reminded me in the comments that, as well as hanging on to all those embarrassing infelicities, Google’s search cache also remembers good things like comments that you foolishly delete.

Sadly, it didn’t remember everything, but it did remember everything, but it did remember a lot, so various articles have their commenty mojo working once more.

Did I really just say ‘commenty mojo working’? It seems I did. Ah well, it’s 6.20am and I’ve been awake since 4, I think I’ll let myself off with a caution.

Oops I did it again

But my backups just saved my arse. Now I really should get on with making sure I don’t do it again.

Reasons to be edgy...

pdcawley@bofh.org.uk (Piers Cawley) — Fri, 29 Sep 2006 02:50:00 -0500

I’ve been busy with a new experimental branch of typo that plays well with edge rails, in part because I wanted to move to a more RESTful URL scheme for Typo.

But then I took a look at some of the things going on in Rails Core and my nipples exploded with delight. For instance, there’s now a ResourceFeeder helper that pretty much magically deals with Atom and RSS feeds for Rails Resources, which would allow us to eliminate a whole pile of ugliness in the Typo source. So that’s a win.

But then, I just discovered that Rails now has ‘real’ around filters which allow you to implement a filter as a block which calls yield. Which means that it’s now easy to write filters that clean up their resources properly, even in the face of exceptions being thrown further down the filter chain; you just catch the exception, do your cleanup and rethrow. Which allows us to eliminate a whole category of possible typo memory leaks and simplify a few things that are currently causing us pain. Result.

Applying design patterns unconsciously

pdcawley@bofh.org.uk (Piers Cawley) — Mon, 21 Aug 2006 07:33:00 -0500

I just realised something about the workings of new style Typo Sidebars: it’s just an application of the Parameter Object design pattern; the render_sidebar helper method takes a Sidebar parameter object and produces a chunk of HTML. The fact that we persist the parameter object using ActiveRecord is almost beside the point – the persistence is more important to the render_sidebars method than anything else.

Old style sidebars couldn’t really be called an application of any design pattern. Maybe that’s what bugged me about them, they lacked the Quality Without a Name. I’m not sure that the new sidebar architecture has the Quality either, but it’s much nearer to having it.

When I was learning to play Go someone told me that it’s easier to remember professional games than those of amateurs because the moves in a professional game are generally more ‘right’ than those of weaker players. To a good go player, the ‘right’ move has an obvious purpose (or, more likely, purposes). Bad moves don’t.

Maybe something similar applies to code. In good code it’s clear what each piece does – maybe it can be seen as an example of a design pattern, maybe not – and the pieces relate well to each other. In bad code, the intent and relationships are muddied.

Do Design Patterns Help?

Continuing the Go analogy, Design patterns can be likened to joseki – ‘standard’ patterns of good play in the opening of the game. It’s useful to know them, but what’s really important, and what takes time, practice and deep study to get good at, is using and adapting them well in the fuseki – the opening and its wider context. You can play a ‘perfect’ joseki in one little corner, but its influence radiates across the board and if you don’t pay attention to what that influence is and use or adapt it well, you’re screwed.

I think the analogy to the use of design patterns in programming is fairly obvious. Don’t you?

Dogfood time

pdcawley@bofh.org.uk (Piers Cawley) — Sun, 20 Aug 2006 05:13:00 -0500

I’ll let you into a secret. You can tell when I’ve done a big svn up on this blog simply by looking at the theme. If it’s all black and white and sans serif, then it’s a racing certainty that I’ve just done an upgrade which has broken my usual custom theme.

So, later today I shall be sitting down with my local theme repository and bringing things up to speed with all of Scott’s changes. Who knows, I might cheat and put back a few helper methods.

Update

Oops, it seems that there was a bug in the Scribbish theme which meant you couldn’t read articles at their permalink…

Cunning tricks with the new typo sidebars

pdcawley@bofh.org.uk (Piers Cawley) — Sun, 20 Aug 2006 04:18:00 -0500

I was going to save this until I’d rejigged my blog to a three column layout and had left everyone mystified by how I’d managed to get sidebars on both sides. But, let’s be honest here, it’s going to be ages before I do that. So, I shall spill the beans preemptively.

Actually, you’ve been able to do this, or something like it for ages, but new style sidebars make it easier.

Imagine you’re editing the layout of your new, three column theme, and you’ve got a left-sidebar div, which you want to populate with a tag cloud and the categories sidebar.

<div id='left-sidebar'>
<%= render_sidebar TagSidebar.new :maximum_tags => 30 %>
<%= render_sidebar CategorySidebar.new :show_empty_categories => false %>
</div>

Then it’s just a matter of styling to get the sidebar in the right place.

This works because the vast majority of sidebars don’t hold any state apart from their settings (Certainly none of the core sidebars do) and their position in the sidebar list. So, if you want to render particular sidebars in your layout, you don’t need to go to the database at all.

Of course, I can conceive of sidebars that will need to manage persistent state, but now I’ve realised that you can manage sidebars like this without having to go to admin/sidebar at all, I shall be working on making things a little bit friendlier. If nothing else, I should probably write a helper that lets you do:

render :sidebar => TagSidebar, :maximum_tags => 15

to make the interface a little more consistent with the rest of rails’ suite of render helpers.

Enjoy.

Update

Now that extended content is working properly again, I’ve moved the details back below the fold.
Dan Milliron has another neat way of doing this.

Victory! For now

pdcawley@bofh.org.uk (Piers Cawley) — Sun, 20 Aug 2006 03:43:20 -0500

Typo Sidebar’s are now controller free!

Which probably means nothing to the vast majority of people reading this. But it makes me very happy.

Until a couple of revisions ago, Typo sidebars were rendered through a convoluted route, which involved n_ + 1 calls to render_component, where _n is the number of items on the sidebar. Calling render_component isn’t something you really want to do even once if you can help it. It’s not as slow as it used to be, but it’s still unpleasant.

So, now a typo sidebar looks pretty much like any other Rails plugin. It lives in vendor/plugins with a model, a view, a test and an only mildly clever init.rb.

For the MVC purists among you, there’s a small amount of shenanigans involved because the model carries information about where to find the sidebar’s template, but I shall plead necessity and have done with it. Besides, the model never makes direct use of the information, it just holds onto it for the controller.

The upshot of this, and Scott’s changes is that Typo’s components directory is now empty. And you can call me Mr Happy!

You don’t have to call me Mr Happy of course, but I reserve the right to be happy anyway.

Expect a quick introduction to writing your own sidebars soon.