Just A Summary : Tag typo, everything about typo

Other things which aren't fun

pdcawley@bofh.org.uk (Piers Cawley) — Mon, 07 May 2007 14:29:00 -0500

Last year, you would have been forgiven for thinking that Typo was pretty much dead in the water as an ongoing project. Typosphere was a placeholder, changes were few and far between, the app was a bloated monster. So, people switched, in droves, to Mephisto the new, (and excellent) kid on the block. Heck, even Tobi, the original author of Typo, has switched.

Mephisto’s a great piece of software, and I’m as sure as I can be (without taking a closer look) that its underpinning code is cleaner than typo’s. When you’re developing on a rapidly moving platform like Ruby on Rails, there could be said to be a second mover advantage – the later you start, the more likely Rails is to already do what you want, and the less likely you are to zag where Rails later zigs.

However, Rick and Justin have other projects and demands on their time, and once something is Good Enough, it’s hard to summon up the motivation to make it better until you find something that you want your application to do.

So now, as Typo comes out of hibernation, it seems that Rick and Justin have been getting it in the neck because their project is ‘stalled’. Okay, so I admit, I did smile ruefully to myself when I read that.

Then I read the comments. Wow. Some people have some serious entitlement issues.

It’s been said before, it’ll be said again I’m sure: if you don’t like something about an open source project, “Patches welcome!”. If your patches are repeatedly rejected: it’s open source, fork off. If a project is moribund: offer to help.

Complaints just put people’s backs up.

I think it’s great that there’s choices available if you want a blog built Rails. Different perspectives on a problem space are important – if nothing else, everyone gets to steal from each other, everyone’s apps get better and everybody wins.

A sketch of declarative ActiveRecord Migrations

pdcawley@bofh.org.uk (Piers Cawley) — Fri, 24 Nov 2006 06:18:00 -0600

Writing migrations can get pretty tedious when you’re being scrupulous about writing both the up and the down side of the migration. Okay, so the Textmate ninjas amongst you can use scarily clever snippets to populate the down migration while you write the up method, but I can’t be the only Mac user who still prefers Emacs. And not everyone gets to run on Macs either.

So, inspired by something Jamis Buck wrote about designing a DSL, I’ve been sketching out a DSL for describing the easy parts of a migration in declarative style. None of this is implemented yet, but I’m pretty sure that it’s relatively simple to implement for a decent Ruby metaprogrammer. I’m brain dumping it here so I can come back to it later, or, you never know, someone might have implemented it by the time I revisit…

The sketch

class AdjustContentTable < ActiveRecord::Migration
  class Content < ActiveRecord::Base
    include ActiveRecord::Migration::Delta

    delta do
      +(:extended, :text)
      -(:excerpt, :text)
      +index :text_filter_id
     end
   end
  end

  callbacks_for(:up) do
    before_any_changes
    Content.before_changes { ... }
    Content.before_additions {...}
    Content.after_additions {...}
    Content.before_removals {...}
    Content.after_removals {...}
    Content.after_changes {...}
    after_any_changes
  end

  callbacks_for(:down) do
    ...
  end
end

Implementation Pointers

By making delta do its work in a block, it’s possible to instance_eval the block using a Delta object that has appropriate implementations of + and -.

Then callbacks_for[:up] puts its block into, say @callbacks[:up] and the block is yielded to to set up the callbacks by the default implementation of up. The default up probably looks something like:

  def self.up
    @callbacks[:up].call
    apply_all_deltas
  end

Of course the fun bit is the implementation of apply_all_deltas – the per class apply_delta is reasonably simple:

  def self.apply_delta(direction)
    callback(:before_changes)
    apply_additions(direction)
    apply_removals(direction)
    callback(:after_changes)
  end

  def self.apply_additions(direction)
    if has_additions?(direction)
      callback(:before_additions)
      eval additions_for(direction).to_ruby
      callback(:after_additions)
    end
  end

  ...

Thinking about it, apply_all_deltas is going to take a certain amount of grovelling about in the namespace to work out what classes need to have their deltas applied, but even with just per class delta declarations, this should be a useful thing to implement.

Braindump ends

Sorry this doesn’t come with a handy dandy Rails plugin with it all implemented; your contributions in this area would be very gratefully received.

Getting back to Typo

pdcawley@bofh.org.uk (Piers Cawley) — Sun, 29 Oct 2006 07:19:00 -0600

I’ve been taking an accidental sabbatical from Typo. It started during conference season, then there was a bunch of housekeeping and general home type stuff that needed doing, so I’ve been away from the codeface for a while (and I still haven’t turned my EuroFOO talk into a podcast, or finished scanning and uploading all the EuroOSCON photos…)

However, I am back and coding, and a new typo feature just made it to the top of my todo list: This morning, in a fit of over enthusiasm, I managed to zap the 36 most recent comments on this blog. And, for bonus points, I didn’t have a backup.

So, now I have a backup regime in place. And the next feature I add to Typo is going to be a confirmation box when you attempt to delete any feedback that hasn’t been marked as spam.

Recent changes added to the typo trunk add CachedModel support, which dramatically reduces our query count per page, which is no bad thing either; I have other ideas for getting that count down too.

So, until the Typo Trac is back up, watch this space, I shall try and post here whenever there’s any significant changes on the trunk.

Updates

Google cache is my friend!

Miles Barr reminded me in the comments that, as well as hanging on to all those embarrassing infelicities, Google’s search cache also remembers good things like comments that you foolishly delete.

Sadly, it didn’t remember everything, but it did remember everything, but it did remember a lot, so various articles have their commenty mojo working once more.

Did I really just say ‘commenty mojo working’? It seems I did. Ah well, it’s 6.20am and I’ve been awake since 4, I think I’ll let myself off with a caution.

Oops I did it again

But my backups just saved my arse. Now I really should get on with making sure I don’t do it again.

Other viewpoints, and why they are important

pdcawley@bofh.org.uk (Piers Cawley) — Wed, 16 Aug 2006 02:50:00 -0500

Many thanks to Paul Ingles who has just written a fantastic article about writing a Typo sidebar in a test driven way.

For too long we’ve been assuming that Typo sidebars are, in fact, detestable (“If it’s not testable, it’s detestable” – someone at the Sydney XP group, by way of Martin Fowler). Paul neatly demonstrates that sidebars are perfectly testable, we just haven’t been testing them. Bad us.

So, I’m rolling back the changes in my current sidebar rejig and starting again by writing some tests for a few of the core sidebars, and then I can start my refactoring without quite so much of The Fear.

So, once again, thanks Paul. It’s really great to occasionally see the curious outsider’s view of what we’re working on – it makes us question our assumptions and often shows us better ways.

Scratching an Itch

pdcawley@bofh.org.uk (Piers Cawley) — Sun, 09 Apr 2006 03:29:00 -0500

Some time ago, around the time of the Typo themes contest, someone, I think it may have been Scott Laird, wrote a nifty little sparklines textfilter.

It’s a really cunning piece of code, the problem is that, to make it work, Scott had to make some dramatic changes to the way textfilters work which, in turn, meant we had to change the way we got at the htmlized versions of our content. Before sparklines, you could do content.body_html in your view and it would just work, generating the html as required. After the change you had to do content.html(@controller, :body), which I’m sure you’ll agree is rather gruesome. So we wrapped it up in a helper method, letting you do: article_html(@article, :body), which is only slightly less gruesome.

At a stroke, we managed to break almost all of the themes entered in the contest, the eventual winner included. Ho hum.

To be honest, screwing up the winning theme’s almost beside the point, I’ve disliked the new interface for getting at html content ever since we introduced it, but I’ve liked cunning textfilters more, so we’ve bitten the bullet and accepted the fact that Typo 4.0 will have a different theme API from the last release.

Well… I say accepted. I’ve been looking for a way out of it in a desultory fashion ever since we introduced it. And today I cracked it. I knew – despite a few howls of protest – that introducing a Blog class was the right thing to do even if we never take the step to full multiblog capabiliity. Today, it paid off…

It turns out that the blog object and its class is the ideal place to hang both static and dynamic context information. The static context includes the usual blog wide settings (and eventually things like the sidebar configuration).

Dynamic context is currently just the active controller. I had been toying with having the controller set a $current_controller global, but that becomes a massive pain in the bum once you start rendering components. However, if you tell the blog about the current controller it can keep a stack of them. Every time you render a component, the blog pushes the component’s controller onto the stack and pops it off again once the sub request has completed.

So: the blog knows about the current controller; the content knows about the blog, which means that we can write:

class Content
  def body_html
    html(blog.controller, :body)
  end
end

And all your views can use the same API as they did back in the days of Typo 2.

The real body_html is more sophisticated, of course, and in the absence of a controller it falls back on the cached value.

I am a happy camper this morning.

BTW. If you’ve still got problems porting an old theme, it’s probably because your theme renders sidebars ‘wrong’. Check in your themes/<theme>/layouts/default.rhtml and look for the line that looks like

<%= render_component(:controller => 'sidebars/sidebar', :action => 'display_plugins') %>

and replace it with:

<%= render_sidebars %>

The other big change in the way themes are rendered (and again, its’ sidebar related) is in the <head> section of the layout. Here’s what the default theme header looks like nowadays:

<head>
  <title><%=h page_title %></title>
  <%= page_header %>
  <%= stylesheet_link_tag "/stylesheets/theme/azure", :media => 'all' %>
  <%= stylesheet_link_tag "/stylesheets/user-styles", :media => 'all' %>
  <%= stylesheet_link_tag "/stylesheets/theme/print", :media => 'print' %>
</head>

You should change yours to look similar (you pretty much must include that <%= page_header %> part.

I hope this helps.

And... relax...

pdcawley@bofh.org.uk (Piers Cawley) — Fri, 31 Mar 2006 14:38:00 -0600

Rails 1.1 got released on Sunday.

Lots of hosting services upgraded.

Typo, both the ‘stable but old’ and the bleading edge versions, didn’t get on with Rails 1.1. For quite spectacular values of ‘not getting on’. So, we ran around like headless chickens for a bit, gradually refining the incantations needed to get your existing typo installation working again – rake freeze_edge VERSION=3303 does the trick if you’re still having problems – and then Scott, Kevin and I spent the best part of a day nailing down the issues.

It turns out that there was rather a lot of ‘cargo cult’ code in older versions of Typo. Cargo cult code is code that isn’t there because it works, but because it reassures the programmer into thinking it will work. Rails 1.0 was far more forgiving of our cargo cult (excessive use of require_dependency, an undocumented method that doesn’t quite do what we thought it did) than 1.1.

So, with that done, and a few more bugs squashed and we’re almost ready for a release (future distributions will come with rails baked into the tarball – but not if you get it from the repository), both of 2.6.0 with a frozen rails distribution and of the forthcoming 4.0.0. As I write their are two issues outstanding on the 4.0.0 milestone, the trickiest of which is Scott’s plan to make typo installation substantially less user hostile. However, if you’re happy to deal with the slightly tricksy installation, there’s never really been a better time to jump onto the Trunk. Grab a copy and start blogging. You know you want to.

If you test it, I will patch

pdcawley@bofh.org.uk (Piers Cawley) — Sun, 12 Mar 2006 13:17:00 -0600

I’ve been working on Typo this weekend, mostly going through open tickets and deciding whether to apply patches.

There’s plenty of potentially good patches in the queue, but too many of them give me The Fear.

They don’t have tests!

I’m the first to admit that I’ve added untested code to my projects (and I’ve been bitten by it too), but I’m more sanguine about that risk because, when it goes wrong, I can often rack my brains and pull back some memory of what my intent was. Knowing my intent helps me write meaningful tests and fix the issue. However, I can’t remember the intent of your code. The only thing I have to go on is what you wrote. If you’ve written tests, I can tell what you expect your code to do, which is far more important than being able to divine what your code actually does. Given a choice between code without tests and tests without code, I’ll take the tests every time.

So, if you’ve got a typo patch on the trac and you want it to get accepted, you must write tests for it. If you’re implementing something that’s supposed to be conforming to a standard, then please reference those standards in your tests.

Out of box experience

pdcawley@bofh.org.uk (Piers Cawley) — Fri, 09 Dec 2005 16:38:00 -0600

Have you ever bought hardware from Apple?

Lovely wasn’t it?

I don’t mean the kit was necessarily lovely—Apple have build their fair share of clunkers in the past. I’m talking about the experience of getting your iPod, PowerBook, PowerMac or whatever out of its box for the first time. It’s obvious that Apple have spent a good deal of time on making that out of box experience as satisfying as possible. A box that you will probably only use once seems to have had as much care lavished on it as the thing it contains.

Maybe Sony have got better at this sort of thing, but I remember when I bought a MiniDisc player a few years ago. Where my iPod came in a great looking box that opened in a satisfying way which made it easy to get at the goodies it contained, the MiniDisc came in just another box and was awkward to get at to boot.

I felt, and still feel, good about my iPod. I haven’t a clue where my MiniDisc Recorder has got to.

So, why is this post in the Typo category?

Well…

Typo’s out of box experience sucks. From the point of view of someone who just wants to get their blogging engine up and running, it’s about as friendly as a cut snake.

I know this because, in the last few days I have spent hours on the #typo IRC channel, walking folks through a set of simple steps to get to a working typo installation. Well, I thought they were simple—apparently they aren’t. Which is why it’s great to talk to real users even if it can be dispiriting sometimes.

So, we need to work to improve typo’s out of box experience. Here’s my current plan:

The user unpacks the typo distribution outside the html tree somewhere.
At a shell prompt, she types rake configure
A webrick server kicks off and the user is prompted to point their browser at http://localhost/3000 or some such ‘private’ url.
Typo walks her through a straightforward sequence of steps which configure the real typo installation.
At the end of the process, assuming it’s possible, the user will have appropriate symbolic links set up, a database.yml that makes sense and (ideally) a running blog engine.

Now all that remains is to implement it.

Calling all Typo theme hackers

pdcawley@bofh.org.uk (Piers Cawley) — Thu, 08 Dec 2005 04:51:00 -0600

As I write, Typo Themes Contest is in that nervous interlude that comes between the last day for entries and the judges’ announcement of fabulous prizes (I haven’t totted up the value of the prize fund, but there’s a Powerbook and an iBook for first and second place) for the winners.

So, if you’re one of the entrants busily chewing your fingernails down to the quick as you anxiously await the news of your shiny new powerbook, I have a request:

Please take the time to write a post mortem of your experience of writing your theme. What was good? What wasn’t? What really pissed you off about Typo? What documentation did you wish was available?

When you’ve written it, post it somewhere and either comment or send a trackback ping to this message. I promise that I’ll read every such postmortem I hear about. Feedback on this stuff is important to making sure that Typo theme engine is easy to use and well documented in the future—I’m a coder not a designer; what’s intuitive to me isn’t necessarily intuitive to you and vice versa. And where interfaces are concerned, user expectation usually trumps developer expectation.

Bwah hah ha ha!

pdcawley@bofh.org.uk (Piers Cawley) — Mon, 05 Dec 2005 22:11:00 -0600

This evening, I acquired a commit bit for the Typo project. I’ve been chucking the occasional big patch and ideas at Scott Laird, the most active of the current maintainers, but he’s been busy doing all sorts of stuff, and he just started working at Google (go Scott!) so he doesn’t have quite as much copious free time as before.

So, this evening I asked for, and received, my very own commit bit. Which means I can make changes to Typo without needing to funnel them through a maintainer—I’m now part of the funnel.

Of course, responsible programming means I’ll still run big changes by Scott and the other maintainers, but it’s also useful to be able to make the kind of small changes that sometimes need to be made without it blocking on a busy man.

Wish me luck.

Testing Your Assumptions

pdcawley@bofh.org.uk (Piers Cawley) — Sun, 04 Dec 2005 09:37:00 -0600

One of the joys of writing applications using Ruby on Rails is the way the framework is constantly evolving better ways of doing stuff. It’s one of the dangers too.

Each release of Rails brings new and groovy features to the table, so it’s nice to stay close to the edge. However, when you do that, a change in the framework can bring your whole app crashing down because a key assumption you made turns out not to be true any more.

This is especially common when the assumed behaviour is undocumented, or a surprising consequence of behaviour that is documented.

When this happens, your test suite can have failures everywhere; it’s hard to work out precisely what the underlying problem is.

So, what to do?

Easy. Write tests that express your assumptions. It’s always good to make assumptions explicit. It’s better still to make them executable. Whenever you use something that’s a little bit surprising or hard to find in the docs, or when you do something that works around a bug, write a test.

Then, when the framework changes and tests are failing all over the place, test your assumptions. Hopefully there’ll be one or two failing tests there that point you to a solution to your problem. If there aren’t, then at least you know what’s not causing the problem. And, when you do find the underlying problem, you can add the new assumption to your tests for later.

I’ve just been working on Typo and found (rather usefully as it happens) that, if you add counter caching to a model, the model that holds the cache field will call all its save/update hooks whenever the cache field is updated.

Which is a little odd; it could be argued that simply adding what should be an ‘invisible’ cache field to a model shouldn’t go changing its behaviour in this way. So, I’ve created an assumptions_test.rb file and written the behaviour up as a test. Now, if David Heinemeier Hansson decides that this is undesirable behaviour and changes it, we’ll be able to see what’s going on.

The idea (as is so often the case) comes from Kent Beck. In this case, from Test Driven Development. One more book by Beck that deserves to be on every programmer’s bookshelf.

Testing Is Good

pdcawley@bofh.org.uk (Piers Cawley) — Wed, 05 Oct 2005 04:01:00 -0500

If you’ve hung out with extreme programmers, the Perl 6 crew, a large subsection of the Perl community, or, well, almost anyone who’s given it any thought, you’ll know that having a good test suite is essential.

But how good is good enough?

Why am I asking?

Recently, Typo, the ruby based blogging engine that this site runs on, underwent a strutural change that involved replacing 4 different tables in the database with a single table, and migrating all articles, comments, trackbacks and pages into this new table.

The migration was easy to write. Typo’s built on top of Rails, and Rails has tools for doing database migrations. It’s hard to test a migration though, so the first ‘big’ migration that moves articles into the new contents table just raises an exception saying “BACK STUFF UP before uncomment this line in the code and rerun the migration.”

Making that backup is a really good idea because, if you’ve turned on spam protection, the next migration accidentally throws away all comments on sufficiently old articles.

Oops.

When a comment is saved, it first ensures that the article it’s attached to isn’t too old (more than 14 days old by default). If it is, the comment doesn’t get saved. This is fine when you first attach a comment to an article, but less fine when you’re, say, adding a guid field to the article or, as in the case of the migration, moving the comment from one database table to another.

So, I’m adding a test or two to the test suite to ensure that we can modify comments on old articles. Once I’ve got that passing, we can be reasonably confident that, next time we do a migration that targets comments, we can be confident that all the comments will get modified appropriately, which can’t be bad.

What have I learned?

Because the change was a refactoring which adds no new functionality to Typo, I hadn’t written any new tests, just made sure that all the existing ones were passing. But no test suite is complete (unless you are testing a “Hello, world” program I suppose), so the issue that causes us (and anyone else who migrated to edge typo with spam protection turned on) pain wasn’t caught by the tests.

I don’t blame whoever wrote the tests in the first place for missing this particular case; there’ve been occasions when I’ve wanted to edit an old comment to correct a URL or something and have bounced off the spam protection, but it never occurred to me that this might be a problem for migrations until I saw the bug report.

What I have been forcibly reminded of is that no test suite is complete. So, when a bug comes in, you turn it into a test and add it to the test suite, then you should never introduce the same bug again.

Nothing new there of course. Surely everyone already knows this rule of testing. But there’s nothing quite like actually doing it again, and having a good deal of confidence that adding the test will save you from an entire class of problems in the future to help you internalize it.

I think that, before I do much more work on Typo, I’ll be taking a closer look at what is and isn’t tested in its test suite.

Thinking aloud about Typo

pdcawley@bofh.org.uk (Piers Cawley) — Fri, 23 Sep 2005 02:41:00 -0500

Unless you’re interested in the internals of the Typo blogging engine and a possible rejigging of it, don’t bother reading the rest of this.

Typo Content Models

I’ve been attempting to follow the DRY Principle and rejigging Typo so that there’s fewer places with similar code to handle the cached html associated with a content object. Also, I’ve been thinking of how to build a more intelligent cache sweeper—at the moment, if you change a single article, every page in the cache is removed, which isn’t ideal. To that end I wrote a ContentModel mixin that adds a new class method:

class Article < ActiveRecord::Base
  include ContentModel

  content_fields :body, :extended
  ...

Declaring a content_field sets up a setter method that would notify any object observers that the content has changed and reset the associated _html attribute.

Which is all very well, but I think the time might have come to make a Content superclass. The various typo content models already have quite a few fields in common, and even Article, the ‘biggest’ of them doesn’t have too many unique fields, so using Single Table Inheritance a completely insane idea.

What does that buy us?

Conceptually, it buys us a neater model, albeit at the expense of a slightly scruffier table structure (Rails can’t do multi-table inheritance yet).

A unified ID scheme for our content models means that any putative table mapping content objects to currently cached files is going to be substantially simpler.

Threading. Add parent_id column to the table and, because of the uniform ID structure, a comment gets to have a parent that is either an Article, another Comment or (possibly) a Trackback.

How could it hurt?

The database table’s a bit scruffier.

Writing the migrations could be fun.

We’d have to make sure that the old articles/read/id URLs continued to work—Adding an old_id column is one way forward, but it strikes me that a better way to do it would be to extend the articles table until it had all the fields required then rename it appropriately and then copy all the other content objects into it.

Conclusions

Well, I think it’s worth doing. But it’s my idea, so I would say that wouldn’t I. If you’re a Typo person and you’re reading this, what do you think?

Version Control

pdcawley@bofh.org.uk (Piers Cawley) — Sun, 04 Sep 2005 17:52:00 -0500

The trouble with version control systems is that, if you’re not careful, it’s almost as bad as having no version control at all. I’m currently juggling about 4 active branches of typo: the local development branch, the production branch and a couple of branches that isolate some of the local changes so I can make clean patches for the typo developers.

So far, I’ve managed to upload about 5 patches, and I’m still not sure if the patches that are up on the Typo tracker are sane. And back porting the clean new system’s been a complete nightmare.

Still, it could be worse. I could be still trying to understand the inner workings of Moveable Type.