One of the less satisfying sessions/panels at OSCON this year was the ‘Perl Certification’ Panel, in which a panel of various luminaries, moderated by Tim Maher of the Seattle Perl Users’ Group spoke inconclusively about whether there was a need for Perl Certification. It seems to me that, if you’re going to have a panel on this kind of topic, you must make sure that your moderator is impartial (but not uninterested). Tim Maher, for all his many virtues, was not impartial.

I arrived late to the panel, so I could be forming a bad impression, but it seemed to me the panel had the cart before the horse, and any similar panel will also have such an ungainly configuration, and here’s why:

The important thing about certification is not the certificate but the standards on which it is based.

My wife has been involved in the business of producing standards for trainers, health and safety professionals and others here in the UK; these standards are used as the basis of high level vocational qualifications for such professionals, and they seem to be successful. The way the process works in the UK is as follows:

1. Employers, professionals and people with experience writing standards meet and thrash out the competencies required in their field. At this stage, the role of the standards developers is purely to describe competence.
2. Once consensus is reached, the standards are published. You could, theoretically stop there, and have a useful set of guidelines for trainers, professionals and employers.
3. An awarding body may then go on to develop qualifications based on these standards. This includes developing ways of assessing candidate’s competence through any or all of
   • Observation of practical work
   • Written work
   • Verbal questioning and discussion
4. The qualification is published.
5. Trainers develop training programs and materials aimed at bringing candidates up to the standards required.

In order to qualify there is no requirement placed on the candidate to undergo training, they can simply pay an assessment fee and if they can show competence they qualify. This assessment may well be a relatively expensive process; competency cannot usually (ever?) be assessed with a simple paper examination. Passing a written exam simply proves you can answer questions, but says nothing at all about your ability to do anything else.

Also, trainers/assessors must themselves be competent (and they must also be competent assessors). Under the NVQ system there’s a whole series of checks and balances involving external assessors and supporting materials. In essence though, the whole thing is straightforward, a candidate goes to a competent assessor, demonstrates their competence and receives either a qualification or guidance on what areas they need to develop further. That is all. The whole edifice stands or falls on the quality and integrity of this assessments, but that was always the case, it’s just made explicit with standards based qualifications.

One of the things that slowed the take up of standards based vocational qualifications in the UK has been the inertia of the training organisations. Trainers like to sell training packages, but the qualifications emphasise assessment—Training requirements may arise where more competence is required. Trainers need to rethink their priorities; when assessment is what counts, trainers should become assessors too and start making money from the assessment process as well as from any training requirements that arise. Many UK trainers have now made this jump.

I doubt very much that we’ll ever get a single standard in this area. But that’s fine, I’m sure that the various software tribes will be able to agree on a core set of standards, then the Crafty Camp can go on to develop standards for competence in, say, CRC Manipulation, Estimation, Test Infection, Pair Programming etc. Meanwhile the Software Engineering Collective can develop standards in UML Reading, Formal Analysis Techniques, Bogging the Project Down in Use Case Writing… (can you tell which side of that particular divide I’m on?)

Of course, if the Agile camp had standards, they might encourage the development of standards for customers. I can just imagine the pre-sale conversation:

“Ah, you have a level III Extreme Customer NVQ, that carries a 10% increase in velocity over a project with an inexperienced customer”

“And I see your team has an average Level IV in Refactoring and Test Writing, but your Estimation could use some work, let’s see if we can’t all improve while we work on this”

“Okay, so what are the stories?”

…

“So, three months to start with, fortnightly iterations, the standard XP contract?”

“Yeah, let’s get started”