Just a Summary

Restaurant Review: Slice

2025-10-21T17:39:00Z

The last time I was in New York

Which may well turn out to be my Last Time in New York, for both environmental and political reasons.

I was on my way to OSCON in Portland, OR but I took the opportunity to spend a couple of days with my much missed friend David (dha) Adler – founder of the New York Perl Mongers, Doctor Who fan, terrible movie aficionado, comics fan, fountain pen collector and all around delightful nerd.

For lunch that first day, he took me to his favourite New York restaurant, Shopsin’s in a corner of Essex Market. The menu was bewilderingly long and seemed to make absolutely no sense, but once I’d finally decided (almost by shutting my eyes and pointing at random) what I was having, the food was magnificent. Flavour combinations of sweet and savoury that I would never have thought of. Ludicrously large portions. The very best kind of brusque service. And a queue halfway around the market by the time we’d finished to let us know exactly how special the place was. We ate lunch there every day of my layover and everything I was similarly bonkers and delicious. The pastrami on rye from Katz’s deli that evening was pretty damned overwhelming too, but that was a one and done. Shopsin’s was a place I could have happily eaten my lunch at for the rest of time.

Which brings me to Slice, which sits in an unassuming shed on a back road halfway between Gweek and Falmouth in Cornwall. Another Perl monger friend recommended it to us as a great place for brunch, so we paid a6 visit when we were on holiday a couple of weeks back.

As soon as I saw the menu—stupidly long, full of sweet/savoury flavour collisions, remarkably inexpensive—I thought, “Hmm… this reminds me of somewhere.” I ordered a “Morio II”, Slice’s take on a steak sandwich made on a sourdough flatbread packed with steak, caramelized onions, cheese, hash browns and really good salad greens, and the first bite was a seriously emotional experience. It was nothing like anything I ate in New York, but I was back there in the corner booth by the pass, talking shite with a much-missed friend and love, love loving the food. Everything about that meal was about as close to perfect as you can get. The Morio II was amazing. Every bite a brilliant collision of deliciousness. Gill’s “Full Hash Up” was a more conventional take on the full English breakfast

she swapped the tomatoes and beans for spiced house rosti at the suggestion of our server, Lily. A very good call indeed. Thanks Lily.

which I confess I helped her to finish and which was very good indeed. And, oh god, the huge charred marshmallow and hot chocolate I had to finish up? Stunning.

We didn’t manage to eat there every day after that, but we did get back once and the food was just as good and imaginative. My “The King”—waffles with steak, maple syrup, butter and deep fried crispy poached eggs—was good, but not to my taste.

Except for those deep fried crispy poached eggs! Wow!

Gill’s poached eggs on sourdough toast with green oil was everything you could want, and was quickly polished off without my assistance (sadly).

Both delightful experiences, with that lovely Cornish casual service that’s commonplace on the peninsula: Friendly, passionate about the food and helpful. As typically Cornish as Luke’s service at Shopsin’s was stereotypical New York service.

I really can’t recommend the place highly enough. If you have fond memories of Shopsin’s and can get to the middle of nowhere in Cornwall, then you should definitely give this place a try. And if you have no idea what I’m talking about, but you like generous plates of enormously tasty surprising combinations of food, then get your arse down there too. You won’t regret it.

And if you have the time, go and sit on the harbour wall in Porthleven with an Ann’s

Not Philps’, good and close as they are

Pasty and admire the fine selection of gorgeous dogs taking the air with their owners while you eat. It’s a great tradition of ours that more people should share.

Talking to the Wayback Machine

2025-10-02T14:14:00Z

Way back in 2016, I migrated this site from its Publify

Publify is the Rails based blogging engine that started out as Typo, which I ended up maintaining for a while before handing it off to Frédéric de Villamil, who is still on the current maintenance team. Go him!

incarnation to a static site generated from markdown files by Hugo. In that migration, I fucked up and truncated a buttload of posts and didn’t realise what I’d done until long after (about a week ago now) I had misplaced the database that the site had originally been generated from.

Oops.

However, the Internet is still a marvellous place, blessed with useful sites like The Wayback Machine, which lets the interested reader browse historic versions of web pages. Which means, provided a page got noticed by archive.org’s crawlers, I can fetch a page from back before I fucked up and, with a little bit of massaging, turn it into something that Hugo can understand and get the whole article back again.

I can even recover the comments, which I had deliberately left out of the initial import, thinking “I’ll get around to importing those as well one day!” Thanks to my ADHD, that never quite happened. Unless you count the current activity, of course.

“Provided” is doing a lot of work there, and some posts definitely got missed, but something is better that nothing.

I’ve reached the point in my recovery process that I’ve started to hate the ad hoc way I was grabbing stuff from the archive. I want to only grab an archived page if the archived version is from before I buggered things up. So I’ve written some Emacs lisp. Obviously.

Here’s what I’d like to write.

(with-wayback-page-from-before url 20160618
 (web-tidy-buffer)
 (fixup-escaped-typo:code-blocks)
 (convert-to-org-mode)
 (restrict-to-article-and-comments)
 (fixup-comments)
 (org-string-nw-p
 (buffer-substring-no-properties (point-min) (point-max))))

The idea being that I fetch the archived version of the post into a temporary buffer where I can run it through HTML Tidy and a few extr HTML cleanup steps

An ever expanding list of cleanup steps. Every time I have to tidy something up by hand for the second time, I add something to the cleanup pipeline.

before converting it to Org format with Pandoc, continuing the cleanup in org mode (I prefer Emacs org mode tooling to its HTML tooling) and returning a nice clean string to insert into the org capture buffer.

I plan to return to the cleanup in future articles, but we’re just concerned with with-wayback-page-from-before for now.

If you want to see the full code (along with way more stuff), you’ll find it in my dotemacs repo on Github. I’m exceedingly unlikely to turn this into a full wayback.el package, but you’re more than welcome to use it as a starting point.

Here’s what that looks like:

(defmacro with-wayback-page-from-before (url date &rest body)
 (declare (indent 2) (debug t))
 (let ((capture-url (make-symbol "capture-url")))
 `(when-let* ((,capture-url (wayback-get-capture-before ,url ,date)))
 (with-temp-buffer
 (request ,capture-url
 :sync t
 :success (cl-function
 (lambda (&key data &allow-other-keys)
 (insert data))))
 ,@body))))

It’s a macro that uses wayback-get-capture-before to find the URL of the most recent capture of our target url before the given date, fetches it into a temporary buffer and executes the body of the macro. A common Emacs pattern.

do M-x describe-function and type with- and check out the completions to see just how common

The real meat lies in wayback-get-capture-before, which uses the Wayback CDX Server API to discover the capture url we’re interested in. There are other, simpler to use Wayback machine APIs, but they only let us find the closest capture to our date of interest, and we want to find the most recent capture that’s strictly before our date and that requires the CDX API. I’ve been a little lazy and used the request package to do the web request stuff because I prefer its API to the native url-retrieve in vanilla Emacs.

(defvar wayback-cdx-endpoint "https://web.archive.org/cdx/search/cdx"
 "The endpoint for the Wayback Machine's CDX server.")

(defvar wayback-cdx-json-parser
 (apply-partially 'json-parse-buffer :array-type 'list)
 "Parser for json data returned from the CDX server.")

(defun wayback-get-capture-before (url date)
 "Use the CDX applet to find any version of URL captured before DATE string.
Returns nil if there's no such capture"
 (let ((capture-url nil))
 (request wayback-cdx-endpoint
 :params `((url . ,url)
 (to . ,(if (or (numberp date)
 (stringp date))
 date
 (format-time-string "%Y%m%d%H%M%S" date)))
 (collapse . digest)
 (output . json)
 (fl . "timestamp,original")
 (limit . -1))
 :parser wayback-cdx-json-parser
 :sync t
 :success (cl-function
 (lambda (&key data &allow-other-keys)
 (setq capture-url
 (pcase (cadr data)
 (`() nil)
 (`(,timestamp ,target-url)
 (s-lex-format "https://web.archive.org/web/${timestamp}/${target-url}")))))))
 capture-url))

The CDX API’s JSON response format is derived from a CSV style text file. We’re only really interested in the timestamp and the “original” url that our target url resolved to, so we set (fl . "timestamp,original") in the request parameters and limit the results to the most recent one ((limit . -1)) before ((to . ...)) the date we’re interested in. That gives us:

[["timestamp", "original"],
 ["20250212175727", "https://bofh.org.uk/2016/06/19/static-migration/"]

You can tell it comes from something CSV like, can’t you?

The JSON response gets parsed into a Lisp list and we extract the interesting bits using a pcase statement

(pcase (cadr data)
 (`() nil)
 (`(,timestamp ,target-url)
 (s-lex-format ...)))

which passes an empty list through, or grabs the timestamp and target-url from the second entry in the results list and uses s-lex-format to generate a wayback machine URL. Easy.

This has the makings of a more general package, but that’s very much a back burner project. It does what I need, and does it well enough that I can consider this yak shaved and get on with the job of recovering my truncated blog posts. I’ll continue on my way of not releasing anything that anyone might want me to support.

Making use of Webmentions

2025-09-09T20:49:00Z

When last we left off we had worked out how to grab all the mentions of this site that Webmentions.io knew about and now we want to write that out to the data/ directory in a format that’s easy to deal with in Hugo.

If you’ve read my earlier note, you’ll know that I’ve been evolving the data schema towards something that’s easy for Hugo to deal with and reasonably comprehensible for me too.

As things currently stand, I’ve settled on dropping all the mentions in a single file, data/mentions/all.json

I’d rather use data/mentions.json, but Hugo’s data system doesn’t seem to pick that up, so I’ll live with the slightly more clunky option.

which is structured along these lines:

{
 "/note/7/": {
 "like-of": [
 {
 "type": "entry",
 "author": {
 "type": "card",
 "name": "Daniel Kelly Music",
 "photo": "https://avatars.webmention.io/fsn1.your-objectstorage.com/7aa4815cbc1f993c0e2a6df03280dc168f5ad07fecd0313ddfc27eeb02e0b437.png",
 "url": "https://aus.social/@yasslad"
 },
 "url": "https://mendeddrum.org/@pdcawley/115160370354067277#favorited-by-109307830089461078",
 "published": null,
 "wm-received": "2025-09-07T01:31:46Z",
 "wm-id": 1936817,
 "wm-source": "https://brid.gy/like/mastodon/@pdcawley@mendeddrum.org/115160370354067277/109307830089461078",
 "wm-target": "https://bofh.org.uk/note/7/",
 "wm-protocol": "webmention",
 "like-of": "https://bofh.org.uk/note/7/",
 "wm-property": "like-of",
 "wm-private": false
 },
 {
 "type": "entry",
 "author": {
 "type": "card",
 "name": "Jess Robinson",
 "photo": "https://avatars.webmention.io/fsn1.your-objectstorage.com/a11ee0a58e54873140bf1f1965900378d866fce3fafae79e116b979bea3a8773.jpg",
 "url": "https://fosstodon.org/@castaway"
 },
 "url": "https://mendeddrum.org/@pdcawley/115160370354067277#favorited-by-109562941096076318",
 "published": null,
 "wm-received": "2025-09-07T07:19:01Z",
 "wm-id": 1936873,
 "wm-source": "https://brid.gy/like/mastodon/@pdcawley@mendeddrum.org/115160370354067277/109562941096076318",
 "wm-target": "https://bofh.org.uk/note/7/",
 "wm-protocol": "webmention",
 "like-of": "https://bofh.org.uk/note/7/",
 "wm-property": "like-of",
 "wm-private": false
 }
 ],
 "in-reply-to": [],
 "mention-of": [],
 "repost-of": [],
 "other": []
 }
}

Just imagine that JSON object has a bunch more paths as keys referencing similar objects keyed by mention type.

As it stands, wm--fetch-all is returning a flat sequence of webmention objects that we want to process into a more structured object,

JSON/Javascript calls them objects, old Perl heads like me think of them as hashes, and they’re “Hash Tables” in Emacs Lisp. I’ll be calling them hashes from now on.

in other words we want to fold (or “reduce” in Emacs Lisp terminology) the sequence into a hash. And I know just the function for that. Let’s see what describe-function has to say about seq-reduce:

seq-reduce is a byte-compiled function defined in seq.el.gz.

*Signature*
(seq-reduce FUNCTION SEQUENCE INITIAL-VALUE)

*Documentation*

Reduce the function FUNCTION across SEQUENCE, starting
with INITIAL-VALUE.

Return the result of calling FUNCTION with INITIAL-VALUE
and the first element of SEQUENCE, then calling FUNCTION
with that result and the second element of SEQUENCE,
then with that result and the third element of SEQUENCE,
etc. FUNCTION will be called with INITIAL-VALUE (and then
the accumulated value) as the first argument, and the
elements from SEQUENCE as the second argument.

If SEQUENCE is empty, return INITIAL-VALUE and FUNCTION
is not called.

This does not modify SEQUENCE.

So we can write

(seq-reduce #'wm--add-mention-to-hash-table
 (wm--fetch-all)
 (make-hash-table :test 'equal))

and that will handle the business of iterating over the sequence of mentions for us, and all we have to do is write wm--add-mention-to-hash-table to populate the hash we made with (make-hash-table :test 'equal)

We need to use that :test ’equal part because json-insert wants a hash with strings as keys and the default hash returned by (make-hash-table) compares keys using eql which might or might not work when comparing strings. Not a problem which equal has.

one mention at a time, and return the modified hash (You and I both know that it’s the same old hash mutated, but let’s pretend it isn’t, eh?).

What does that function look like? Here’s what I wrote:

(defun wm--add-mention-to-hash-table (acc mention)
 "Helps reduce a list of mentions into a two level hash."
 (require 'dash)
 (let* ((path (--> mention
 (gethash "wm-target" it)
 (url-generic-parse-url it)
 (url-path-and-query it)
 (car it)))
 (mentions-hash (or (gethash path acc nil)
 (wm-new-mentions-hash)))
 (mention-type (gethash "wm-property" mention))
 (mentions (or (gethash mention-type mentions-hash)
 (progn
 (setq mention-type "other")
 (gethash mention-type mentions-hash))))
 (new-mentions (if (seq-contains mentions mention)
 mentions
 (vconcat mentions (list mention)))))
 (puthash mention-type new-mentions mentions-hash)
 (puthash path mentions-hash acc)
 acc))

We grab the path from the "wm-target" key, which is actually a URL rather than a simple path

We could just use the URL, and that would work fine on this site, but not when I’m running on localhost. The path will always match with .RelPermalink, but the host part of .Permalink is different in development than in production.

so, rather than writing

(car
 (url-path-and-query
 (url-generic-parse-url
 (gethash "wm-target" mention))))

we’ll thread mention through that series of transformations using dash.el’s threading macro, -->.

We use the path to grab mentions-hash from the acc-umulating hash and, if there isn’t already one there, we grab an empty, but structured hash using wm-new-mentions-hash, which looks like this:

(defun wm-new-mentions-hash ()
 "Make a new empty hash to hold categorised webmention data."
 (copy-hash-table
 #s(hash-table
 test equal
 data ("like-of" [] "in-reply-to" []
 "mention-of" [] "repost-of" []
 "other" []))))

Now we look up "wm-property" in mention, and use that to grab its associated vector of mentions. Well, we would, but there’s a small wrinkle.

We’re only currently interested in four kinds of mention, but Webmention.io doesn’t know that. We could throw the extras away, but what if we became interested in bookmark-of mentions or whatever somewhere down the road. So let’s collect them under the other key. Which is where this hacky section of our let* form comes in:

(mention-type
 (gethash "wm-property" mention))
(mentions
 (or (gethash mention-type mentions-hash)
 (progn
 (setq mention-type "other")
 (gethash mention-type mentions-hash))))

What’s going on here then?

First, we make a guess at the mention-type we’re going to file the current mention under by grabbing the "wm-property" and use that value to lookup the mention type in mentions-hash. If it’s one of the four types we’re interested in, that will be a vector, which is truth-y, otherwise we get nil, which is false-y so we change the mention type to “other” and grab that vector from the mention hash.

We now know the key path we’re going to store our mentions in, and we have the current vector of mentions associated with it. So, if we already know (seq-contains mentions mention) about the current mention, we reuse that, otherwise we make a new vector with the current mention added to it.

That done, it’s a simple matter of putting the new mentions vector into our mentions hash, putting the mentions hash into our accumulating hash and returning that.

With that done, it’s a simple matter of opening data/mentions/all.json, erasing the buffer, calling (json-insert (seq-reduce ...)) to update the data and saving it. Here’s the code which does exactly that.

(defun wm-unflatten-mentions (mentions-vec)
 (seq-reduce 'wm--add-mention-to-hash-table mentions-vec
 (make-hash-table :test 'equal)))

(defun wm-fetch-mentions ()
 "Fetch the webmentions of `wm-domain'."
 (interactive)
 (save-current-buffer
 (let ((all-entries (wm--fetch-all)))
 (with-temp-file (expand-file-name "all.json" wm-data-dir)
 (erase-buffer)
 (json-insert (wm-unflatten-mentions all-entries))))))

Over in the Hugo partial that renders the bit of the page immediately after this, we can get at the data like this:

{{- $all_mentions := index site.Data.mentions.all .RelPermalink -}}
{{ $likes := index $all_mentions "like-of" | default slice -}}
{{ $reposts := index $all_mentions "repost-of" | default slice -}}
{{ $replies := index $all_mentions "in-reply-to" | default slice -}}
{{ $mentions := index $all_mentions "mention-of" | default slice -}}
<footer class="metaline">
 <ul class="response-summary">
 <li>{{ $likes | len }} {{ if eq 1 (len $likes) }}like{{ else }}likes{{ end }}</li>
 <li>{{ $reposts | len }} {{ if eq 1 (len $reposts) }}repost{{ else }}reposts{{ end }}</li>
 <li>{{ $replies | len }} {{ if eq 1 (len $replies) }}reply{{ else }}replies{{ end }}</li>
 <li>{{ $mentions | len }} {{ if eq 1 (len $mentions) }}mention{{ else }}mentions{{ end }}</li>
 </ul>
</footer>
...

I’ll leave the rest as an exercise for the interested reader. However, I will note that the Webmention.io API includes the option to pass in a since argument, so it wouldn’t be hard to write

(seq-reduce
 'wm--add-mention-to-hash-table
 (wm-fetch-mentions-since wm-last-checked)
 (wm-parse-mentions-file
 (expand-file-name "mentions/all.json"
 wm-data-dir)))

without having to change our reducing function at all.

Separation of concerns, baby! Separation of concerns!

Fetching webmentions again. With Emacs this time!

2025-09-07T17:17:00Z

You might have noticed, if you’re a regular visitor that webmentions have started showing up on the site again. I turned them off a while ago,

I turned off the home server that was handling the web hook calls from Webmention.io, planning to quickly move it and spin it up again. Ask me how that’s going.

but Aaron Parecki’s invaluable Webmention.io service has still been gathering them for me, so I’ve turned them back on. But in the mean time, I mislaid the code I was using to populate the necessary Hugo data files from Webmention. Exploratory code ahoy.

Start by faking it

I’m heavily indebted to Brian Wisti for his post, Using the Webmention.io API as the starting point to my explorations, but since I can’t be doing with Python, I used emacs.

I started with my very minor fork of restclient

# Grab the most recent 5 webmentions of bofh.org.uk
GET https://webmention.io/api/mentions.jf2?domain=bofh.org.uk&sort-dir=down&per-page=5&token=:token

Which produces the following JSON data:

Disclose this for the full wall of JSON

Don’t say you weren’t warned!

{
 "type": "feed",
 "name": "Webmentions",
 "children": [
 {
 "type": "entry",
 "author": {
 "type": "card",
 "name": "Mike Spencer",
 "photo": "https://avatars.webmention.io/fsn1.your-objectstorage.com/3ba5a0fdc660c44995fef428a601770fd0fe2619fc1f8c7e70ec7e9e1da66d4b.jpg",
 "url": "https://mastodon.scot/@mikerspencer"
 },
 "url": "https://mendeddrum.org/@pdcawley/115162152891409560#favorited-by-109365771998686190",
 "published": null,
 "wm-received": "2025-09-07T09:35:22Z",
 "wm-id": 1936897,
 "wm-source": "https://brid.gy/like/mastodon/@pdcawley@mendeddrum.org/115162152891409560/109365771998686190",
 "wm-target": "https://bofh.org.uk/note/8/",
 "wm-protocol": "webmention",
 "like-of": "https://bofh.org.uk/note/8/",
 "wm-property": "like-of",
 "wm-private": false
 },
 {
 "type": "entry",
 "author": {
 "type": "card",
 "name": "Jess Robinson",
 "photo": "https://avatars.webmention.io/fsn1.your-objectstorage.com/a11ee0a58e54873140bf1f1965900378d866fce3fafae79e116b979bea3a8773.jpg",
 "url": "https://fosstodon.org/@castaway"
 },
 "url": "https://mendeddrum.org/@pdcawley/115160370354067277#favorited-by-109562941096076318",
 "published": null,
 "wm-received": "2025-09-07T07:19:01Z",
 "wm-id": 1936873,
 "wm-source": "https://brid.gy/like/mastodon/@pdcawley@mendeddrum.org/115160370354067277/109562941096076318",
 "wm-target": "https://bofh.org.uk/note/7/",
 "wm-protocol": "webmention",
 "like-of": "https://bofh.org.uk/note/7/",
 "wm-property": "like-of",
 "wm-private": false
 },
 {
 "type": "entry",
 "author": {
 "type": "card",
 "name": "Daniel Kelly Music",
 "photo": "https://avatars.webmention.io/fsn1.your-objectstorage.com/7aa4815cbc1f993c0e2a6df03280dc168f5ad07fecd0313ddfc27eeb02e0b437.png",
 "url": "https://aus.social/@yasslad"
 },
 "url": "https://mendeddrum.org/@pdcawley/115160370354067277#favorited-by-109307830089461078",
 "published": null,
 "wm-received": "2025-09-07T01:31:46Z",
 "wm-id": 1936817,
 "wm-source": "https://brid.gy/like/mastodon/@pdcawley@mendeddrum.org/115160370354067277/109307830089461078",
 "wm-target": "https://bofh.org.uk/note/7/",
 "wm-protocol": "webmention",
 "like-of": "https://bofh.org.uk/note/7/",
 "wm-property": "like-of",
 "wm-private": false
 },
 {
 "type": "entry",
 "author": {
 "type": "card",
 "name": "Nick Anderson",
 "photo": "https://avatars.webmention.io/fsn1.your-objectstorage.com/856b210b0770a292a4f9e76699c84aa3833f9252716ad3cab58b50a7b57205ee.jpg",
 "url": "https://fosstodon.org/@nickanderson"
 },
 "url": "https://mendeddrum.org/@pdcawley/115107421781297473#favorited-by-109475479621313511",
 "published": null,
 "wm-received": "2025-08-28T21:48:52Z",
 "wm-id": 1934308,
 "wm-source": "https://brid.gy/like/mastodon/@pdcawley@mendeddrum.org/115107421781297473/109475479621313511",
 "wm-target": "https://bofh.org.uk/note/6/",
 "wm-protocol": "webmention",
 "like-of": "https://bofh.org.uk/note/6/",
 "wm-property": "like-of",
 "wm-private": false
 },
 {
 "type": "entry",
 "author": {
 "type": "card",
 "name": "Nick Anderson",
 "photo": "https://avatars.webmention.io/fsn1.your-objectstorage.com/856b210b0770a292a4f9e76699c84aa3833f9252716ad3cab58b50a7b57205ee.jpg",
 "url": "https://fosstodon.org/@nickanderson"
 },
 "url": "https://fosstodon.org/@nickanderson/115108589417986943",
 "published": "2025-08-28T21:48:05+00:00",
 "wm-received": "2025-08-28T21:48:51Z",
 "wm-id": 1934307,
 "wm-source": "https://brid.gy/comment/mastodon/@pdcawley@mendeddrum.org/115107421781297473/115108589456284711",
 "wm-target": "https://bofh.org.uk/note/6/",
 "wm-protocol": "webmention",
 "content": {
 "html": "<p><span class=\"h-card\"><a href=\"https://mendeddrum.org/@pdcawley\" class=\"u-url\">@<span>pdcawley</span></a></span> I was gonna say, that looks like a source block within a source block which means there's another src block to make it render. Nice</p>",
 "text": "@pdcawley I was gonna say, that looks like a source block within a source block which means there's another src block to make it render. Nice"
 },
 "in-reply-to": "https://bofh.org.uk/note/6/",
 "wm-property": "in-reply-to",
 "wm-private": false
 }
 ]
}

The essential shape is something like this:

{ "type": "feed",
 "name": "Webmentions",
 "children": [
 { "wm-target": "https://bofh.org.uk/note/8/",
 "wm-property": "like-of",
 ... },
 { "wm-target": "https://bofh.org.uk/note/8/",
 "wm-property": "in-reply-to",
 ... },
 ... }]}

Restclient is great for interactively exploring a RESTful API, but it’s not so great for slicing and dicing the data in a Emacs-y way. I could sit down and learn jq again, but I know Lisp, dammit, so after a frustrating hour or so trying to wrap my head around the default url-retrieve interfaces, I went and grabbed the emacs-request package instead because I found its API more comprehensible.

Because request is a function where restclient is more like an application running in Emacs, it’s way more useful for automating things. Here’s more or less the same request as above done in Lisp.

(let (response)
 (request
 "https://webmention.io/api/mentions.jf2"
 :params `(("domain" . "bofh.org.uk")
 ("token" . ,wm-api-token)
 ("per-page" . "5")
 ("sort-dir" . "down"))
 :parser 'json-parse-buffer
 :sync t
 :success (cl-function
 (lambda (&key data &allow-other-keys)
 (setq response data))))
 response)

The code is obviously fiddlier, but it’s also programmable and, because we set an arbitrary :parser function, it’s trivial to convert the returned JSON into a native Emacs lisp hash table

It’s not hard to generate an old school alist either, but that was annoyingly hard to serialise back to JSON, so I went with the default types because they seem to just work.

which looks a bit like this:

A wall of Lisp

#s(hash-table test equal data
 ("type" "feed" "name" "Webmentions" "children"
 [#s(hash-table test equal data
 ("type" "entry" "author"
 #s(hash-table test equal data
 ("type" "card" "name"
 "Mike Spencer" "photo"
 "https://avatars.webmention.io/fsn1.your-objectstorage.com/3ba5a0fdc660c44995fef428a601770fd0fe2619fc1f8c7e70ec7e9e1da66d4b.jpg"
 "url"
 "https://mastodon.scot/@mikerspencer"))
 "url"
 "https://mendeddrum.org/@pdcawley/115162152891409560#favorited-by-109365771998686190"
 "published" :null "wm-received"
 "2025-09-07T09:35:22Z" "wm-id" 1936897
 "wm-source"
 "https://brid.gy/like/mastodon/@pdcawley@mendeddrum.org/115162152891409560/109365771998686190"
 "wm-target" "https://bofh.org.uk/note/8/"
 "wm-protocol" "webmention" "like-of"
 "https://bofh.org.uk/note/8/" "wm-property"
 "like-of" "wm-private" :false))
 #s(hash-table test equal data
 ("type" "entry" "author"
 #s(hash-table test equal data
 ("type" "card" "name"
 "Jess Robinson" "photo"
 "https://avatars.webmention.io/fsn1.your-objectstorage.com/a11ee0a58e54873140bf1f1965900378d866fce3fafae79e116b979bea3a8773.jpg"
 "url"
 "https://fosstodon.org/@castaway"))
 "url"
 "https://mendeddrum.org/@pdcawley/115160370354067277#favorited-by-109562941096076318"
 "published" :null "wm-received"
 "2025-09-07T07:19:01Z" "wm-id" 1936873
 "wm-source"
 "https://brid.gy/like/mastodon/@pdcawley@mendeddrum.org/115160370354067277/109562941096076318"
 "wm-target" "https://bofh.org.uk/note/7/"
 "wm-protocol" "webmention" "like-of"
 "https://bofh.org.uk/note/7/" "wm-property"
 "like-of" "wm-private" :false))
 #s(hash-table test equal data
 ("type" "entry" "author"
 #s(hash-table test equal data
 ("type" "card" "name"
 "Daniel Kelly Music" "photo"
 "https://avatars.webmention.io/fsn1.your-objectstorage.com/7aa4815cbc1f993c0e2a6df03280dc168f5ad07fecd0313ddfc27eeb02e0b437.png"
 "url"
 "https://aus.social/@yasslad"))
 "url"
 "https://mendeddrum.org/@pdcawley/115160370354067277#favorited-by-109307830089461078"
 "published" :null "wm-received"
 "2025-09-07T01:31:46Z" "wm-id" 1936817
 "wm-source"
 "https://brid.gy/like/mastodon/@pdcawley@mendeddrum.org/115160370354067277/109307830089461078"
 "wm-target" "https://bofh.org.uk/note/7/"
 "wm-protocol" "webmention" "like-of"
 "https://bofh.org.uk/note/7/" "wm-property"
 "like-of" "wm-private" :false))
 #s(hash-table test equal data
 ("type" "entry" "author"
 #s(hash-table test equal data
 ("type" "card" "name"
 "Nick Anderson" "photo"
 "https://avatars.webmention.io/fsn1.your-objectstorage.com/856b210b0770a292a4f9e76699c84aa3833f9252716ad3cab58b50a7b57205ee.jpg"
 "url"
 "https://fosstodon.org/@nickanderson"))
 "url"
 "https://mendeddrum.org/@pdcawley/115107421781297473#favorited-by-109475479621313511"
 "published" :null "wm-received"
 "2025-08-28T21:48:52Z" "wm-id" 1934308
 "wm-source"
 "https://brid.gy/like/mastodon/@pdcawley@mendeddrum.org/115107421781297473/109475479621313511"
 "wm-target" "https://bofh.org.uk/note/6/"
 "wm-protocol" "webmention" "like-of"
 "https://bofh.org.uk/note/6/" "wm-property"
 "like-of" "wm-private" :false))
 #s(hash-table test equal data
 ("type" "entry" "author"
 #s(hash-table test equal data
 ("type" "card" "name"
 "Nick Anderson" "photo"
 "https://avatars.webmention.io/fsn1.your-objectstorage.com/856b210b0770a292a4f9e76699c84aa3833f9252716ad3cab58b50a7b57205ee.jpg"
 "url"
 "https://fosstodon.org/@nickanderson"))
 "url"
 "https://fosstodon.org/@nickanderson/115108589417986943"
 "published" "2025-08-28T21:48:05+00:00"
 "wm-received" "2025-08-28T21:48:51Z" "wm-id"
 1934307 "wm-source"
 "https://brid.gy/comment/mastodon/@pdcawley@mendeddrum.org/115107421781297473/115108589456284711"
 "wm-target" "https://bofh.org.uk/note/6/"
 "wm-protocol" "webmention" "content"
 #s(hash-table test equal data
 ("html"
 "<p><span class=\"h-card\"><a href=\"https://mendeddrum.org/@pdcawley\" class=\"u-url\">@<span>pdcawley</span></a></span> I was gonna say, that looks like a source block within a source block which means there's another src block to make it render. Nice</p>"
 "text"
 "@pdcawley I was gonna say, that looks like a source block within a source block which means there's another src block to make it render. Nice"))
 "in-reply-to" "https://bofh.org.uk/note/6/"
 "wm-property" "in-reply-to" "wm-private"
 :false))]))

Verbose as hell, but something we can work with. Here’s a simplified alist representation which might be a little easier to understand.

'((type . "feed")
 (name . "Webmentions")
 (children
 . [((wm-property . "like-of")
 (wm-target . "https://bofh.org.uk/note/8/")
 ...)
 ((wm-property . "in-reply-to")
 (wm-target . "https://bofh.org.uk/note/8/")
 ...)]))

The interesting stuff lives under the "children" key, which we can get with (gethash "children" data).

To get all the webmentions for our domain, the Webmention API allows for pagination. We can ask for pages of, say 100 entries and if we get 100 entries back, append the result to our running collection of entries and request the next page. Once we get a result with fewer than 100 entries, we know we’re done and we can massage the data into a shape that Hugo can cope with

Then turn what we learn into a commands

Now we know what the data coming from Webmention.io looks like, and how we can page through it, let’s write a function, wm--fetch-all to do that for us.

The (while more? ...) loop keeps requesting more data until it gets a short response, at which point more? becomes false and we return the accumulated all-entries vector

And relax

We now have a handy list of all the webmentions relating to our site. The next step is to massage it into a data structure that will suit Hugo and export it as JSON files in the site’s data directory. Which is a topic for another blog post, I think. If only because I’m reasonably sure that the data structure I’m currently using isn’t great.

I’ll hack it about a bit and report back.

Fiddling with structural templates in Org-mode

2025-08-27T21:59:00Z

I’ve been dealing with some enduring niggles in my literate Emacs configuration, and have just landed Commit dbc316b, and I thought I’d write about it here because it’s an example of what keeps me on Emacs more than thirty five years after I first started using it.

Back when I started, Emacs was merely a text editor that felt easier to use than vi, but these days, it’s all that and more. I don’t live in Emacs quite as much as I used to; I don’t hang on USENET or IRC any more, and I check my email as little as possible, so I’ve not got around to configuring Emacs for that again.

Back in the day, it all went through the behemoth that is GNUS, but I lost that version of my .emacs file several computers and ISPs ago.

These days, what I like about Emacs is it’s malleability. It’s not uniquely malleable, either. Vim and Neovim diehards will no doubt have tales to tell about their setups, but Emacs is my editor. There are others like it, but this is mine.

I’ve been seduced by the Literate Programming idea of maintaining my ~/.emacs

Emacs used to keep its config in a file in your home directory called .emacs, then it moved to ~/.emacs.d/init.el, which made it a little easier to break a big config out into multiple files. These days, Emacs looks first in ${XDG_CONFIG_HOME}/emacs, which defaults to ~/.config/emacs and that’s where my config files live today.

file in a single org-mode file, which gets ’tangled’ into early-init.el and init.el, which are the files that Emacs actually loads.

One feature of Org mode’s literate programming support that I rather like and take advantage of is the ability to write things out of order and assemble them correctly using noweb. The idea is that src blocks can be named, and then referenced from other source blocks. For instance, you might have:

#+begin_src emacs-lisp

(dolist (template `(("t" "Task with annotation" entry
 (file ,pdc/org-inbox-file)
 "* %?\n:PROPERTIES:\n:created: %U\n:END:\n\n%i\n\n~ %a"
 :prepend t)

 ))
 (add-to-list 'org-capture-templates template t
 (lambda (a b) (equal (car a) (car b)))))

#+end_src

which will eventually be tangled as a dolist that adds all your capture templates to org-capture-templates.

That <<capture-templates>> is what makes the magic happen. When we tangle README.org, the exporter gathers up the contents of any source blocks associated with the tag and replaces <<capture-templates>> with them

It applies some heuristics as well, which is why that dolist isn’t indented exactly how it would be without the noweb reference, but you can read the docs to find out more.

So, elsewhere in my README.org, I can keep capture templates related to a particular app close to the rest of the configuration of that app, where it makes sense to me. For example, in the section where I configure my blogging tools, I do this:

#+begin_src emacs-lisp :tangle nil :noweb-ref capture-templates

("b" "bofh.org.uk post" entry
 (file+headline ,(pdc-site-posts-file "bofh") "Posts")
 (function +org-hugo-new-subtree-post-capture-template))

#+end_src

Notice the header arguments. :tangle nil tells the exporter not to simply write the code out at the current position, and :noweb-ref capture-templates tells the exporter to instead write it out wherever it sees <<capture-templates>> in a source block.

We have to be careful to ensure that the including code gets run when and where all the values variables and functions used in the included fragment are in scope, which can be fiddly, but it’s definitely doable.

Which brings me to what I actually want to write about! I don’t type all that #+begin_src … stuff out by hand every time. Because doing that is simply asking for errors. Instead, I take advantage of an org feature called structure templates, so I have:

#+begin_src emacs-lisp :tangle nil :noweb-ref org-structure-templates

(add-to-list 'org-structure-template-alist )
'("el" . "src emacs-lisp")
'("ett" . "src emacs-lisp :tangle nil :noweb-ref")

#+end_src

And, until slightly before I wrote this, I’d type <el at the beginning of a line, hit TAB and org would expand that to

#+begin_src emacs-lisp

#+end_src

And <ett would expand to

#+begin_src emacs-lisp :tangle nil :noweb-ref

#+end_src

Then I’d fill in the correct :noweb-ref and the appropriate code.

It was fine.

Well, it was better than typing it all by hand, but as soon as I’d expanded a template, I’d immediately hit C-c ' to edit the code block in a separate buffer that was in the correct mode to edit Emacs Lisp (or whatever language the source block was for).

Eventually, I got annoyed enough by the repetition to work out how to make that happen automatically. Normally, I’d expect to add a function to a hook variable somewhere, but that doesn’t quite work here. Time to break out the Swiss Army Knife that is advice-add.

Here’s my first take:

(defun +org-insert-structure-template/after-advice (&rest _)
 (when (derived-mode-p 'org-mode)
 (org-edit-special)))
(advice-add 'tempo-insert-template :after #'+org-insert-structure-template/after-advice)
(advice-add 'org-insert-structure-template #'org-edit-special)

We’re adding advice to two different functions here, because there are two different mechanisms for inserting a structure template, either via the <foo expansion, or by calling M-x org-insert-structure-template, both of which I make use of on different occasions.

This version lasted a while. It did 90% of what I wanted after all. Indeed, for most of my structure templates, it does 100% of what I want.

But there’s always that one case, isn’t there? Take a look at that ett template from earlier. Notice that it’s missing the value to assign to :noweb_ref. In a perfect world, we should fill that in before we start editing the code. Or rely on remembering to do it after the fact. Because we always do that, don’t we?

So, this morning, that chunk of code looked a little like this.

I’ve had to reconstruct the code from memory, I’m afraid because it never made it into the git repo. You can call me sloppy if you like, but this is personal code written on my own time, so you can fuck off.

(defun +org-insert-structure-template/after-advice (&rest _)
 (when (derived-mode-p 'org-mode)
 (let ((datum (org-element-begin datum)))
 (save-excursion
 (goto-char (org-element-begin datum))
 (when (re-search-forward
 "\\(:\\S-+\\)\\s-*$" (pos-eol) t)
 (let ((key (match-string-no-properties 1)))
 (end-of-line)
 (unless (looking-back "\\s-" 1)
 (insert " "))
 (insert (read-from-minibuffer (format "%s: " key)))))))
 (org-edit-special)))

(advice-add 'tempo-insert-template
 :after #'+org-insert-structure-template/after-advice)
(advice-add 'org-insert-structure-template
 :after #'+org-insert-structure-template/after-advice)

Because an org file is Just A Text File™, we could have written this using Emacs’ basic buffer editing commands, but we’ll take advantage of some of org and org-babel’s helper functions to make life a little easier and (hopefully) to help me understand what I’m doing and why I’m doing it when I come back to the code later. Named behaviour is great

Especially in a self-documenting editor like Emacs. If I’m not sure what a particular function in this code does, the documentation, or even the source code, is always a couple of keypresses away.

for helping make code more understandable.

What we do here is save our place in the buffer, jump back to the very beginning of the source block and look at the header arguments. If they end with a property name (:like-this), then we deduce that more information is needed, so we use read-from-minibuffer to ask for it, add the answer to the end of the header arguments, jump back to wherever the template originally left us (by exiting the save-excursion block) and call org-edit-special to start editing the file in a dedicated buffer.

Job jobbed, no?

Well… kinda. See, we’re filling in the value of :noweb-ref using an error prone free text value, rather than presenting a list of known noweb references to choose from. In the case where the unset parameter is :noweb-ref, we really want to use completing-read. A quick trawl through the existing code didn’t find a function to do what we want, nor did a web search. An org file’s Just A Text File though, and org-babel-noweb-wrap returns a regular expression that will match a noweb reference in the current file

<<reference>> is the default form for a noweb ref, but the delimiters are overridable. At one point, I was using «reference» rather than the defaults, so it’s handy to have a function that deals with that for us.

so we could save our place, jump to the beginning of the file and find every match for that regular expression and use that to build a completing-read candidates list. But <<foo>> is only a noweb reference if it’s in a source block, so we could end up with a bunch of false positives. We want to search through every source block, ignoring the rest of the file. Surely there’s already something in existence to let us do that since it’s the sort of thing that happens during the process of tangling a file.

A quick M-x describe-function org babel src block yields a bunch of interesting functions, including the promising sounding org-babel-src-blocks. The documentation reads:

Signature
(org-babel-map-src-blocks FILE &rest BODY)

Documentation
Evaluate BODY forms on each source-block in FILE.

…

It goes on to explain that the body is evaluated with some useful variables set. The one we’re interested in is body, which is a “string holding the body of the code block”. Sorted.

With that, and other helper functions, we can write:

(defun +org-babel-noweb-refs ()
 "Find all the noweb refs in the current buffer."
 (require 's)
 (require 'dash)
 (let ((match-exp (org-babel-noweb-wrap))
 result)
 (org-babel-map-src-blocks nil
 (let ((plain-body (substring-no-properties body)))
 (setq result
 (-concat
 result
 (-map (-partial #'s-replace "(.*)\\'" "")
 (-map #'second
 (s-match-strings-all
 match-exp
 plain-body)))))))
 (-sort #'string< result)))

It’s a bit unsubtle, but it’s quick enough and accurate enough for my purposes. Then we can rewrite the relevant bit of our advice function along these lines:

(if (re-search-forward "\\(:\\S-+\\)\\s-*$" (pos-eol) t)
 (let* (arg (match-string-no-properties 1))
 (value
 (cond ((string= arg ":noweb-ref")
 (completing-read ":noweb-ref: "
 (+org-babel-noweb-refs)))
 (t (read-from-minibuffer (concat arg ": "))))))
 (end-of-line)
 (unless (looking-back "\\s-" 1)
 (insert " "))
 (insert-value))

Note the use of cond here even though we could use a single if. I’m making it easier to special case behaviour for header arguments other than :noweb-ref. I’m probably not gonna need it, but it’s easy enough to be kind to the future me who does need it.

I’m sure your editor of preference can do something like this. If it can’t, then why on earth do you put up with it? I know Emacs though, and it it’s taken me longer to write about this whole process of eliminating a bump in my road than it did to implement the necessary functions and advice in the first place. I didn’t even have to restart Emacs, and it remained usable throughout the process

The code was actually written within a source block

but it got smoother and smoother with every step.

I’m not the first person to point out how powerful Just A Text File can be, especially if you’ve also got a huge pile of functions at your disposal to manipulate that file in useful ways and with mode specific semantics. Provided, of course, you’ve got decent tools to search through that pile. Emacs is just such a pile of useful functions and is a great tool for sifting through it. Give it a go, why don’t you.

Take the time to look at how you use your editor. I’m sure some point of friction will come to mind. Then work out how to write some code that will make things a bit smoother. You’re not looking for perfect here, you’re looking for better. The remaining roughness will no doubt niggle at you enough for you to take another pass at sanding it down one day, but for now luxuriate in the fact you’ve made life a little better for yourself.

I’d suggest that, if you keep doing that as part of your practice, you’ll eventually have your tools working exactly how you want them to. But I’ve been an Emacs user since 1988 and I’ve yet to reach that point. But it’s about the journey, not the destination, isn’t it?

A Guest's Guide to Zoom Song Swaps

2024-03-04T08:51:00Z

This is intended as a quick guide for anyone who comes on as a Friday night Song Swap guest, but anyone who’s thinking of using Zoom (or doing their own streams) to share music with folk might find it useful, so I’m making it public.

Publicity material

I’m absolutely terrible at marketing, but it will definitely help me do at least some if you can provide me with some or all of the following:

A photo of you that you don’t hate

If you’ve got a standard headshot or publicity photo, that’s great. I’ll use that to make the YouTube preview card.

If you don’t, here’s a few tips for taking your own. These are basic guidelines, not rules. You can ignore all of them and still get a great photo, but if you follow them, you should at least get a decent one.

Your phone is fine

Seriously, I would have killed for something the quality of even my iPhone SE back when I first started doing digital photography.

Try to enjoy yourself

If you don’t like having your photo taken, it can show and you’ll end up with another photo you don’t like, so try this. For the duration of the shoot, pretend you’re playing the part of someone who actually likes having their photo taken. Weirdly, that will almost certainly make the shoot go faster because you’ll get a good shot nice and early and you can stop pretending.

Get a friend to take the photo

It’s just easier. You can use a tripod or a selfie stick, but another human being’s your best choice. If you’ve got a friend who’s an experienced photographer, then for heaven’s sake ask them and ignore the rest of this list – be guided by them. Or book a session with a professional if you have the time, money and desire.

Use a big light

Big light sources are the most forgiving and flattering ones. I like a nice big north facing window, or bright overcast days. It’s not exciting or dramatic light, but that’s fine.

It’s not a firing squad

You know the typical passport photo shot? The one where you look like you’ve been lined up against a wall, standing square to the camera and looking nervous. Don’t do that.

Here’s a pretty foolproof lighting/posing idea:

Stand facing the biggest window you can find at a time when the sun isn’t shining directly through it.
Turn about 45° to the left or right
Look across at your friend with the camera who’s standing at 90° to the window light.
Your friend will, of course, take an otherwise fabulous photograph of you with your eyes closed, so repeat the last step until they manage a shot you like.
CHECK YOUR BACKGROUND! Make sure there’s no dead flowers apparently growing out of your ears, an embarrasing cat licking its arse, or a pile of washing in the shot.

Ideally you’ll have a nice smooth out of focus thing going on there (portrait mode, if your phone has it can help), but if you can’t manage that, shoot for something that’s not too busy. You can’t go far wrong with a bookshelf, but stand as far away from it as you can so it’s not tack sharp and distracting.

Portrait or square format, please

Bear in mind that I’m going to be putting your photo into a square frame in the YouTube title card, so a wide image probably won’t work too well.

Some links

Got a website? Bandcamp? YouTube channel? Tell me about them and I’ll link to them in the event description and during the show.

A short biography

When I make the YouTube event for a song swap, I write something about my guests. I’ll write something anyway, but it really helps if I’ve got something from you to base it on. After all, I know what I think about you, and I like you or I’d not have asked you to be my guest, but unless you tell me, I don’t know what you think is important about yourself. A short bio can really help there.

[Optional] Some talking points

A song swap isn’t some kind of forensic interview process; it’s supposed to be an informal chat interspersed with songs. If there’s something you really want to talk about, please let me know. The same goes if there’s anything you want me to steer clear of. It’s fine if the conversation gets dark – folk music’s full of murders and misogyny after all – but it’s really not fine if it gets uncomfortable for you.

Tech

Zoom isn’t my favourite piece of software and its default settings are definitely slanted towards business meetings rather than helping musicians sound good. The most important thing you can do in your Zoom settings is to turn on ‘Original Sound For Musicians’, which should show up in the top left of your screen. It’s off by default, so click it and turn it on. If it doesn’t show, you’ll need to dig into Zoom’s audio settings and check the box to make it available.

There’s a catch: turning on original sound disables all of Zoom’s audio processing, including the echo cancellation magic, so it’s really best if you can use a pair of headphones or earbuds rather than speakers to hear me. Or you can just remember to turn original sound on when you start singing and off again when we start chatting. Headphones are easier though.

If your laptop or phone’s built in camera and microphone are all you have, don’t worry, we can work with that. They’re not the best, but they are optimised for someone sitting within reach of the keyboard and making human noises from their mouth hole. You’ll look and sound fine.

If you’ve got money to spend and you want to look and sound better on stream then I have a few suggestions, but this is an area where there’s no right answers, so do shop around and talk to anyone you know who you reckon looks good on your Zoom calls.

Prioritize your spending on gear. In general (and especially for musicians) sound is more important than lighting, which is more important than camera quality. A pin sharp, beautifully lit video of a muddy sounding performer is much harder to watch than a blurry, crystal clear sounding performer singing in a murky cave.

Some audio suggestions

If you’re an unaccompanied singer, this is pretty easy because we don’t have any problems getting your sound balanced and into Zoom. If you’re an instrumentalist, or a band, things get a little more fun, so let’s break it down based on the number of mics you’ll need.

Bear in mind that there are entire books written on this subject; I’m barely scratching the surface here. If you’re serious about getting good sound for streaming and/or recording, it’s worth doing your own research. If all else fails, for your upcoming song swap, buy some of the gear recommended here from a reputable mail order site and rely on the 30 day, no questions asked returns policy they all have because that’s a legal requirement in the UK. But MAKE SURE that when you know what you really want, you buy from the same supplier.

One mic wonders

If the sound you make in the room is the sound you’re happy with (unaccompanied singers, acoustic guitarists, acoustic bands) then it’s just a matter of choosing between a USB mic and an XLR mic with an audio interface. Unless you’re in a particularly noisy environment, I’d recommend some kind of condenser mic. The Blue Yeti has been the standard starter USB mic for years and you could do far worse than do that yourself. However, I would definitely recommend going down the slightly more expensive XLR mic and audio interface route as it’s significantly more flexible and upgradeable.

You’ll want a cardioid pattern large diaphragm condenser microphone (I love my Aston Spirit which looks and sounds great, but it’s a multi-pattern mic and nearly £300 new). Brands like Rode, Aston and SE Electronics make great mics, and honestly, pretty much anything that comes up on an Amazon search for ’large diaphragm condenser microphone XLR’ will still sound better than your laptop or webcam’s microphone. You’ll use a balanced XLR cable to connect that to your interface (a cheap cable’s fine, more expensive ones with Neutrik brand connectors and the like might prove more durable and/or reassuring.)

With a single mic setup you only need a really basic audio interface. Something like the Focusrite Scarlett Solo or 2i2 or any cheap class compliant USB audio interface will do the trick so long has it has phantom power available. Just plug your interface in, connect the mic, turn on 48v/phantom power and tweak the gain until, at your loudest you’re not quite going into the red on the meters, select the interface in Zoom, turn on direct monitoring and you’re good.

Fun with multiple inputs

If you’re a band, or a guitarist where you want to adjust the balance between your voice and your guitar, you’re going to need something a little more sophisticated. How much more sophisticated is up to you, of course and I’m a little out of my depth here, but I’ve got a few suggestions anyway. Don’t hesitate to chat to any live sound engineers of your acquaintance – buy the sound engineer at your local open mic a drink and quiz them, for instance.

The thing to remember here is that Zoom is pretty crap when it comes to audio handling – it doesn’t know anything about pro audio gear, it just expects to receive a mono mix on the first channel of the audio device you select (or a stereo mix on channels 1&2 if you turn stereo on in the audio settings), so you’ll need to do the mixing yourself, either with a standalone mixer, the facilities of your audio interface or some other software on your computer.

I’m going to ignore the software option, but investigate software like Loopback on the Mac and Voicemeeter on Windows. Using your audio interface can be more or less easy, depending on what capabilities the interface has, of course. Modern interfaces are generally more capable in this area. I’ve not actually tried it, but something like the Zoom AMS-24 looks like it would be ideal for a guitarist (put it in ‘streaming’ mode and turn off loopback).

Once you get past a couple of input channels, you’re going to have to go down the mixer route. You can either get a dumb mixer with at least as many inputs as you have instruments and mics, sort out your stereo mix and feed that into a simple two channel audio interface, which will make Zoom happy. That’s fine if you’re primarily interested in live performance, but if you want to do any recording, you’d be better off with a mixer that can also work as a multi-channel audio interface so you can record your vocals and instruments in separate tracks. Something like the Roland GigCasters or the RodeCaster II, for instance. I think Mackie do something in this space too, but I’ve not used any myself, but I do know there are plenty of options.

If your guitar’s got a built in mic or pickup, just plug it into your mixer with a balanced cable, otherwise, you’ll need to mic it up too. There’s whole books written on mic placement, but it’s generally accepted that a good starting point for micing an acoustic guitar is to point the mic at the place where the neck meets the body rather than at the sound hole, maybe a foot or so away. Again, if you know anyone with any audio engineering experience, then talk to them not me.

Once you’ve got your mixer, you’ll need mics. Generally you’ll be close miking things to allow you to mix the different sound sources (if you get lots of bleed between mics, then you have fewer options when it comes to mixing). In theory you use a condenser mic for all these things, just put them close to your mouth or mic, turn the gain down and rely on the inverse square law to give you some separation. In practice, it’s more common to grab a dynamic microphone or two and use them. The canonical mic for the job is probably the Shure SM58, which is built like a tank and looks exactly like you think a stage microphone should look. The SM57 is well regarded as an instrument mic too, and there’s plenty of knock-offs of both. I have an SE Electronics V7, that I use for open mics which sounds pretty good too.

It really helps to have someone else fiddling with the knobs to get your sound dialled in, because they’re not hearing the sound in your head. With decent headphones on, they’re primarily hearing the sound that’s going through the mixer, so they can have a better chance of getting pre-amp gain dialed in, EQing your voice an instruments so they don’t overlap too much and balancing your levels nicely. If at all possible, get some help here.

EQing is a dark art that I’m only vaguely aware of, and mostly don’t have to worry about anyway as an unaccompanied singer. I mostly just leave things flat and hope. There’s plenty of advice to be found on YouTube or sites like Sound On Sound, so I recommend investigating those.

Lighting Suggestions

Again, there are entire books on this, and there’s no end of gear you can buy if really fall down the rabbit hole. Elly Lucas made a great ‘Visual Content Level Up Tutorial’ that’s a great starting point.

Seriously. Just watch that. I was going to write more, but she covers pretty everything I was going to say.

Cameras

If at all possible, don’t use the webcam in your laptop. Investigate ways to use your phone as a webcam. Certainly that’s possible if you’re in Apple World – Zoom can treat your iPhone as a webcam and the camera in your phone is substantially better than almost any webcam you can find. If you’ve got a mirrorless or dSLR camera, more recent ones often have software that lets you use them as a webcam – check your manufacturers website. If they can’t be used directly as a webcam, check whether they have what’s referred to as a ‘clean HDMI’ output and look at getting a cheap and cheerful usb HDMI capture card (or spend rather more on something like the Elgato Camlink. I went with the cheap and cheerful option and it’s fine).

Stick your camera or phone on a good solid tripod, wobbly cameras are really distracting. And level is great. You don’t want people wondering why the things on your shelves aren’t sliding off. Plain backgrounds make this less critical.

If you’re shooting for a solo video, you probably want to frame things so you’re slightly off centre in the frame. However, it makes life much easier for me setting up the Song Swap if you frame yourself in the middle of the shot. One of these days I might fix things so that’s not necessary, but for now centred is best.

Set dressing

Plain walls are great. If you can arrange to hang something like a duvet behind you, that acts as quick and dirty audio treatment of your room. I have a duvet and a rug hanging behind me as a background and it definitely helps with the sound. Getting some distance between you and your background helps blur it a little and make it less distracting.

The advice on making your own headshot applies here as well.

What to expect

The aim of a song swap is to have a good old natter and sing a bunch of songs and to enjoy ourselves while we do it. It’s not an interview, if only because I’m a terrible gobshite with a tendency to go off on tangents. Don’t hesitate to tell me to shut up. I try to rein myself in, but every time I watch a show back I think “Yeah, you could have shut up a bit more there Piers.”

If you’re the sort who likes a set list, I’d suggest planning to cover as much material as you’d get through in a forty minute folk club set with maybe a couple of encore pieces. Nobody’s run out of material yet.

It’s a song swap tradition that the first question I ask a new guest is to ask them to talk about their first encounter with Martin Carthy, whether in person or recorded. Chris Manners memorably described first hearing Martin singing The Bedmaking on the John Peel show playing a guitar ‘hard enough to drive rivets through concrete walls’ and rearranging his entire world. My first encounter was at the Soles and ‘Eels folk club in Northampton. Everyone had hyped the gig up beforehand like it was the Second Coming and I bloody hated it. Martin was great, as usual, I just wasn’t in the right place to realise that. We’re not about gatekeeping here though, so if you don’t know who I’m talking about, let me know and I’ll probably ask you how you came to discover you were a musician.

Payment

The show is supported by people chucking money in the virtual hat at https://ko-fi.com/pdcawley. On the following Monday, I tot up the total takings and send you half through the magic of Paypal.

Panning Overdubs in Loopy Pro

2024-01-13T18:03:00Z

It’s common practice in music production that, when you’re layering your vocals with harmonies or doubles, you pan everything but the lead vocal around the stereo field. That gives a sense of physical separation between the voices and it feels more realistic – or artificial, if that’s what you’re going for.

With loopers though, you’ll more usually have your overdubs all in the same place, which is fine at the local open mic, where the front of house PA is often mono anyway, but for streamers like me, it feels a little limiting, so I’ve been working on a way to get that effect.

This is still a work in progress, but enough people expressed an interest when I showed this off in a “How Do You Loop?” chat with John Paul Music UK here.

Quickstart

If you don’t want the gory details of how the pandomiser works, here’s how to add it to your Loopy Pro and get going quickly:

Set up miRACK in your project

Install miRACK from the App store. It’s a full-featured modular synthesizer simulator, and it’s capable of far more and weirder things than we’re making it do here. Check out the examples.
Grab the pandomizer bundle from my website and unzip with the Files app on your iDevice. Copy pandomiser.mrk to your miRACK folder so the plugin can find it.
Open the mixer, tap the ⊕ button and choose ‘Add MIDI’ and choose the miRack ‘MIDI FX’ option
Open the miRACK interface and tap on the four squares icon in the top left, choose ‘Open Existing’ and load pandomiser.mrk.

Set Up Loopy’s Audio Routing

I’m a pure vocal looper, so I’m going to assume you have a single audio input for your vocals that’s routed to all Loopy’s colours. If you’ve got seperate inputs for your voice and other instruments and not everything is routed to all the colours, you might have to add multiple panning busses, but what follows should give you enough information to get started. This setup also assumes that you don’t want to affect the panning of your live audio, only where it gets placed in your loops and oneshots.

Add a send to your audio input, choose ’new bus’ when prompted, then long press on the send and choose ‘After All Effects’
Tap ‘Destinations’ in the new bus and turn off the hardware output channels. Tap the ‘all’ button to turn on all the Color Destinations.
Tap ‘Destinations’ on your audio input and ’none’ to turn off all the Color Destinations

Now your input audio is going to your loops via this new bus, which means we can mess with its balance setting and it won’t affect the live sound going to your speakers. We can do anything here, and it won’t be heard until the clip it’s recorded on gets played out. Use this power responsibly. All we’re going to do here is mess with the ‘balance’ setting.

Let’s go.

Choose ‘Control Settings’ from the hamburger menu in the top right of Loopy’s screen, then choose ‘Default’ from the Current Project setting.
Tap ‘Add New Binding’ and choose ‘Adjust Parameter’ from the Bus Actions section. Choose your new bus as the target, and make ‘Balance’ the parameter.
Change Controller Input Start to 1%. This avoids the problem that 0-127 doesn’t have a central value. If you don’t change this, then the bus’s balance will get reset to 1 and, if you’re anything like me, it will drive you up the wall.

Test everything is working by opening the mixer and bringing up the keyboard in the miRACK window then holding down C4. You should see your panning bus’s balance jumping to a new random value, then resetting to the centre each time you press and release C4.

Basic Pandomiser use

Here’s one simple way of ensuring that all your overdubs are placed at different points in the stereo field using a couple of follow actions.

Open Clip Settings from main menu, scroll down the settings sheet and choose Follow Actions
Add an action to Begin Overdub, choose Send MIDI Message with your miRACK as the target. Send a note, C4 on channel 1 with velocity (Value in the Loopy UI) 127 (those all the defaults for sending a note, by the way).
Go back to Follow Actions and tap the ‘Reorder’ button, now press and hold the follow action you just added and drag the new action down to Finish Overdub.
Open the new action and drag the Value slider down to zero.

Now, whenever you overdub a clip, as soon as your overdubbing starts, Loopy will ‘hold down’ C4 and the pandomiser will set the recording balance to a new random value. Once overdubbing stops, Loopy will ‘release’ the key and the balance will return to the centre.

Complicating things

In my Loopy setup, I don’t want to change the balance of every overdub of every clip. There might be a couple of clips that will be overdubbed for each verse of the song, and I want the pan to remain the same for each overdub associated with the verse. So I can set the first clip to send the noteOn message when I start overdubbing and set the second to send a noteOff when I finish.¹ You might only want to mess with the panning for certain colours, or even for specific clips, or you might want to set up widgets to record with or without messing with the panning. All you have to do is send C4 to miRACK when you want to randomise the balance, and release C4 when you want it reset to the centre. Have fun.

How does it all work?

A modular synthesizer like miRACK thinks in terms of voltages, and those voltages can mean different things. We use a midi trigger input module that we’ve configured to send +10V whenever C4 is held and we connect that to the Gate input of a Sample & Hold (S&H) module, which we’re using as a source of randomness. Whenever the modules Gate voltage goes high, the module ‘samples’ the voltage at its input, and sets its output voltage to the same until the next time it detects a rising edge. If there’s nothing connected to the input though, it samples an internal noise generator and outputs that voltage. I’ve configured the module to so that its noise source is a white noise generator a range of ±5V, and we can think of that as ranging from a hard left pan at -5V through the centre at 0V and on to a hard right pan at +5V.

The output of the S&H module is now jumping to a new random value every time we press C4, but we really want to output 0V when C4 isn’t pressed, so we feed its output into the input of another staple of a modular setup, a Voltage Controlled Amplifier (VCA). We’ve set this VCA up at unity gain, which means that, when it sees 10V at its Control Voltage (CV) input, it outputs, it sends 100% of its input voltage to its output, and when it sees 0V CV, it sends 0% of its input. So, if we connect the C4 trigger to the VCA’s CV input, we’ve turned it into a gate – whenever C4 is held, the VCA sends the random voltage from S&H, otherwise it sends 0V.

Now we just need to convert that into something MIDI understands, an unsigned value between 0 and 127. The MIDI CC Output module can do some of this for us, but it’s expecting a voltage in the range 0–10V, and right now we’ve got something in the range of ±5V. So we feed the signal from the VCA into the A input of a CONST ADD MULT module, set the constant to 5V, and feed the associated A+B output to one of our CC outputs that we’ve configured to send CC8 to our host app. Job jobbed!

Next Steps

I think the next version of this will allow me to sequence balance for the first few steps, so it might go: hard left, hard right, mid left, mid right, then random pans until the system is reset. I have ideas about how to implement this too, but if you beat me too it, please let me know!

I’ll also be making a YouTube video walking you through it (think of this as a draft script for that video).

I don’t actually do that, because I have a slightly more complex setup that supports multiple song structures with the same set of clips. I’ll explain all that later. ↩︎

Week ending 2023-09-24

2023-09-24T10:47:00Z

I’m not quite sure where the time went these last couple of weeks, but here we are at another Sunday. We’ve spent quite a bit of the time watching a new family of kittens that have taken to playing on the flat roof of our garage. They’re almost obscenely cute scraps of black and white fur and just delightful to watch.

Sunday

Back in 2012, we were living in Cornwall and used to go to the regular Farmers’ Markets in Mullion and Helston. One week, at Mullion, a new trader showed up selling home made bean to bar chocolate under the name of Chocolarder. I got chatting with Mike, the guy who made the stuff, and bought a few bars and some of his sea salt caramel truffles. If we weren’t actually his first customer, we were damned close.

Every month, he showed up with plain looking bars of amazing chocolate. One time he’d bought a bunch of rose petals (apparently, they can be had quite reasonably after Valentine’s Day because there’s something of a glut), dessicated them, ground them fine and added them to the chocolate. Bloody delicious!

I told Mike about some milk chocolate with sea salt that I’d tried and really loved and suggested he do something similar. He’d have a go he said. Months later (after many experiments, apparently) there he was with some bars of salted milk chocolate so I bought as many as I had the cash for and loved every mouthful. He didn’t make them again though. Except, today, with a bit of birthday money burning a hole in my pocket, I thought “I’d love some Chocolarder chocolate, it’s been an age” and what did they have? You guessed it: Cornish sea salt milk chocolate. So that’s a chunk of birthday money spent.

Yes, it’s a lot of money for a bar of chocolate, but believe me, it’s amazing stuff and Mike is as committed to ethical and sustainable manufacture as anyone I’ve ever met. We’ve visited the factory a couple of times and I remember the time we visited and he was more excited about showing off his new, plastic free packaging as he was about the chocolate. He buys direct from cacao farmers and has been known to get his beans shipped by sail rather than container ship.

Week ending 2023-09-03

2023-09-03T13:47:00Z

Another quiet week. Streamed tired on Friday night, so quite a bit of pilot error with the looper, but folk still seemed to enjoy it.

Scrooby show was lovely on Saturday. Nice to catch up with a few folk I’ve not seen in quite a while and the weather was flat out gorgeous. I took some modelling balloons, planning to do a few balloon animals and hats, had something like a 60% burst rate with the Sempertex 260Ss that were all I could get hold of at short notice. Won’t be using those again. Qualatex all the way, I think.

Week ending 2023-08-27

2023-08-27T12:32:00Z

A bit of a mixed week, mood wise. Capped by a great day’s singing and chatting in Peterborough yesterday.

It’s great to get out of the house sometimes. Gill coped really well by herself too – I’m a full time carer, but it’s definitely good to know that I can have the odd day off without it completely buggering things up.

Monday

Folk FOMO

Whitby Folk Festival FOMO is real. But also, there’s a COVID spike going on, and crowded rooms full of unmasked singers aren’t the safest of environments, so I think I’ll comfort myself in the knowledge that at least I won’t be likely to bring an infection home with me.

Blog fiddling

I swear I’m going to wrap my head around the workings of the way to optionally build a custom formatted Date tree using org-capture, but for now I’ve just tweaked the template I use to add

#+hugo: more

to the heading for the week. This means that, when Hugo’s rendering the index page, the week’s notes will be represented by a summary which links to the extended per-day notes for the week.

Furthermore, I’ve added a new capture to let me capture a weeks’ summary. I think I’ll probably end up wrapping that in a summarize-week command that will show me the wider weeknote context while I write the note, then mark the week as DONE, then bring up magit so I can commit and push the changes. But maybe not for a while yet, on the “fake it until you can’t stand not to automate it.” principle.

Thursday

So, I have the ADHD thing of putting a thing down and completely forgetting where I put it, or even its very existence. Object permanence is clearly not a thing with me.

Or I thought it was.

We have a house guest right now, and she has this habit of trying to help by tidying putting stuff in ‘sensible’ places. So I’ll find the squash in amongst the bottles of oil, vinegar and sauces in a completely other part of the kitchen; or Gill’s socks will show up in an admittedly convenient, but surprising, new place after I’ve given up looking for them in the place I usually put them, and the airer I dry them on, and in the washing machine and laundry basket. Maybe it’s on the floor between any of those places… Grrr.

It turns out I’m kind of comfortable with not quite knowing where a thing is, but I am absolutely viscerally infuriated by finding said thing in the wrong fucking place, somewhere I would never ever ever in a month of Sundays deliberately put it.

And don’t… don’t get me started on the utter utter utter wrongness of using the lids of things as shelves. I will end up foaming at the mouth and shouting. Ask me how I know.

It’s all the more distressing because I hate getting angry about stuff, especially objectively trivial stuff like this, and so the rate spirals. Bah!

Saturday

Had a great day at Mill Con 2 down in Peterborough. I still think filk music as a genre is a bit weird, but there’s no denying that the people who make it are lovely people, and it’s hard to beat singing in company for lifting your mood.

Mike Whitaker was kind enough to give me a concert spot at a week’s notice too, so I did a forty minute set with a couple of songs with Loopy Pro in what was only the second time I’ve used the gear in any setting but my ‘streaming studio.’ It went well, but there’s still a way larger profusion of wires than I’m happy with, and I definitely want to assemble some kind of all in one pedal setup if I’m going to be taking the gear out of the house more often.

Week ending 2023-08-20

2023-08-21T10:17:00Z

A quiet week. Had to cancel Friday night’s singing session with Emily – initially because Emily’s still recovering from COVID, but on the night itself, my diabetes meds decided to give me hellacious indigestion. Ozempic/semaglutide might well do wonders for my HBA1c readings, but it can’t half mess with my guts as well.

Week ending 2023-08-13

2023-08-14T09:02:00Z

Oops, missed a week; seems I didn’t have anything interesting to say, or I was too busy doing stuff to write about it. Probably the former.

Not a bad week, this week. My step daughter and her family called in on their way back from holiday on Friday night and we spent a pleasant evening with them and a few Cawleys who were knocking about, sat outside the Wool Market. Mostly good food, but apparently the Greek place isn’t that good. Rustic Pizza is still good though.

Friday

Most of the proceeds of my Magic: The Gathering cards will be spent on repairing my grandfather’s recliner. But… I wouldn’t be me if I didn’t spend some of it on something gamelike. So I bought myself a ‘GameDad’. In my case, an Anbernic RG 353VS and it’s a hell of a thing. Not much bigger than an old school Game Boy (and cheaper! Not just in real terms, but the Game Boy launched at $89.99 and I got mine for $87.99), but with a large, bright colour screen and enough grunt to play SNES and PlayStation games at full tilt. Apparently, you can even make it play Nintendo 64 stuff, but not necessarily at full speed.

I don’t really care about emulating consoles I never owned though. I want to play Manic Miner, Tempest, Dig Dug, Galaxians and the other games that gladly ate my pocket money, ten pence at a time, down the local arcade (the building’s still there in all it’s new brutalist concrete glory, but the arcade where I boggled at I, Robot and thrilled to the exploits of the masters of Defender and Robotron is long gone).

So, in search of that heady thrill and those unmistakable sound effects, I’ve been frequenting archive.org’s library of delights and installing a few of my old favourites.

The first to get seriously played was my old favourite Tempest – Atari’s miracle of colour vector graphics where you controlled a spiky yellow thing running around the top of a blue tube shooting the terrifying geometric shapes that were climbing up towards you with deadly intent. When I first started playing it, I’d hold the fire button down and spin madly round the top of the tube and die all too quickly. But it was such fun I’d just shove another coin in the slot until my money was all gone. Then, in an arcade in Whitby, I watched someone playing the game in an entirely new way and my mind was blown. The walls weren’t blue! The colours were different and there were new, scary shapes. This guy wasn’t spinning around, and he wasn’t just holding down the fire button either.

Tempest was unusual for the time in that it had autofire. If you held down the fire button on most games of the era, you’d fire one shot, then nothing would happen. But in Tempest, you’d autofire bursts of eight missiles, then a slight pause and the cycle would repeat. And it was the slight pause that would kill you. The Whitby guy had sussed that out and was mashing the fire button at a measured speed that kept up a constant stream of evenly spaced bullets that were far more likely to save you when a Flipper had reached the top of the tube and was making its way towards you; they could only kill you if they in the same space as you and were vulnerable to your shots while they were flipping that last step towards you. If you were simply relying on autofire, you could bet that that flip would happen during the short pause between bursts.

Whitby guy had also worked out that the larger the angle a flipper had to flip through, the more chance you had of killing it before it killed you, so for lots of levels, it was just a matter of finding the safest place and staying there. There are a couple of levels where you were only ‘safe’ from flippers coming from one side. Those were the levels that killed you unless you got good at moving from place of safety to place of safety.

I watched intently and, when I returned to my home arcade, suddenly the top three scores – the ones that got burned into non volatile memory – on the arcade’s machine belonged to PDC. I could reliably reach the red levels and even the next, yellow, set.

I can’t do that on the Game Dad. Not yet at least. I’m old enough and RSI’d enough, that the thought of bashing the fire button at 8 Hertz just gives me the shivers.

But! Modern emulators have all sorts of convenience functions, surely I could configure something that would emulate the steady rate of fire that my youthful fingers were capable of. And maybe I could do something about the incredibly sensitive controls, where even the lightest touch of the analog stick would see me moving two or more segments when what I really wanted was a surgical one step move.¹

I turns out that I could. But, frustratingly, not via the very slick UI. I had to edit text files! I had to make new text files. And because popping the Micro SD card out of the Game Dad and into a card reader, editing a file, putting it back in the GD, testing it and then having to fiddle with the text file again is… less than ideal, I did it by logging into my handheld games console via SSH from my iPad, editing the file and just restarting the game!

You can’t do that with an original Game Boy can you? The damned thing’s running Linux. I’m at once annoyed that I had to log in to it and fiddle with text files and astonished that I could even do that.

It’s not the nostalgia that’s making me feel old, it’s my assumptions about what’s capable of what kind of computing.

Real Tempest machines have a gorgeously weighted ‘spinner’ – a free spinning rotary encoder with enough resolution that there still multiple ‘steps’ per tube segment. It meant that your blaster had a lovely fluid movement with very direct control. Basically, Tempest is pretty damned close to perfect on its original hardware and if you ever get the chance to play on a well maintained machine, you should grab it! ↩︎

Week ending 2023-07-30

2023-07-30T22:48:00Z

Three weeks on the trot. Definitely calling that a win.

Also, Good Omens 2 is a delight. Still enough of Terry’s character hanging around it, and the new writers help it not feel too Neil-y.

Wednesday

After a bit of fiddling, I’ve worked out how to add helpers to the Emacs `C-x 8` keymap, so now I have shortcuts for typing ‘λ’, ‘🙂’ and various other characters that I type more or less frequently. Beats the crap out of doing `C-x 8 <ret>` and then typing out the name of the character I’m looking for.

In case you’re interested, here’s the code:

 (general-define-key
 :keymaps 'iso-transl-ctl-x-8-map
 ". ," "…"
 ": )" "🙂"
 ": D" "😀"
 "; )" "😉"
 "\\" "λ"
 "a ^" "↑"
 "a u" "↑"
 "a v" "↓"
 "a d" "↓"
 "a |" "↕")

If you’re not using `general`, but you’ve got `use-package` installed, you can do something similar with `bind-keys`:


 (bind-keys
 :map 'iso-transl-ctl-x-8-map
 (". ," . "…")
 (": )" . "🙂")
 (": D" . "😀")
 (":|" . "😐")
 ("; )" . "😉")
 ("\\" . "λ")
 ("a ^" . "↑")
 ("a u" . "↑")
 ("a v" . "↓")
 ("a d" . "↓")
 ("a |" . "↕"))

You can no doubt use define-key as well, but I find `general` or `bind-keys` to be much nicer to work with. The latter has the advantage that it’s included in Emacs as part of `use-package` and plays nice with `which-key`, so I might go and redo my key bindings and get rid of `general`, nice as it is, since the real selling point of that library is how easy it is to bind stuff in `evil-mode` states.

Sunday

I still miss Twitch Sings. It’s how I started streaming—long before the Friday night Song Swaps and folk streams. I’d be happily belting out Lady Gaga’s Bad Romance, hamming it up to You Spin Me Round or giving it my best Johnny Cash

Not a particularly good impression. I can’t get that low!

on Hurt. It was just huge fun and a great way to make friends on Twitch.

Twitch ended up pulling the plug because it was a free app and… well, free apps and sync rights really don’t play well together.

You’ll still find people doing Karaoke on Twitch though, many of them the same faces I met back in Twitch Sings days. This morning, I woke up early and spotted some friends Karaoke-ing it up on a Discord, so I pulled on pyjamas and went and joined ’em for a few songs. These days, I just use Loopy Pro rather than searching YouTube for backing tracks. It’s great fun though, and definitely makes for a more enjoyable way to spend the occasional hour or so of early morning insomnia.

Singing in company, even virtual company is still the best thing you can do in public with your clothes on. I encourage you all to sing more. What’s the worst that could happen?

Week ending 2023-07-23

2023-07-23T21:37:00Z

Small victory of the week: Actually got off my arse and did something about selling off my old Magic the Gathering cards. For my next trick, I hope to do the same with my collection of [mostly card] magic books.

Tuesday

Made a capture template for adding a week note. Support functions are currently not the prettiest, and don’t deal with a bunch of corner cases, but they seem to work for my case, so I’ll leave ’em be for the time being. I plan to write it up in a longer post, and that will no doubt tweak my coder pride enough to make things suck a little less.

Oh god, once I start fiddling with my Emacs configuration, it’s impossible to stop!

Wednesday

Nipped over to Mum and Dad’s for lunch at Zini’s, and to borrow dad’s drills for my on going cigar box MIDI controller project. Managed to get eight holes accurately placed enough that I only had to drill 7.8mm holes for the M7 threaded rotary encoders I’d soldered to my stripboard. I’m calling that a win! Next trick, get the microcontroller wired up and appropriate software written.

Also discussed making PID controller I promised to make dad for his heat treatment setup a while back. A Pi Pico and one of its mini displays looks like it should do the job nicely. The plan is to make an extension cable with an SSR as a separate bit of kit, then control that from the prototype controller. Once they’re working as separate parts, we can work out how to bring it all into one container. I shall wuss out of making the kind of thing I saw in a commercial radio controlled plug, which powered the control circuit with a very simple capacitor based power supply, with the slightly worrying wrinkle that the controller’s 0V line was floating at around 5V below mains Vmax. Clever, sure, but scarier than I’m prepared to work with.

Friday

Holy crap, but old Magic the Gathering cards are getting horrifically pricy. According to the buy list of the shop I just took my cards in to, I should be expecting about £400 for just four of my cards. And probably another couple of hundred for the two dual lands (assuming they’re not from the Unlimited set, in which case they’re worth a lot more). All being well that’s covered the cost of getting my grandfather’s old recliner reupholstered and fixed.

If I could be arsed with it, I could probably get a lot more by selling direct on eBay, but I was already losing the will to live just sorting things out to take in to the shop.

Do not ask me about the Tabernacle at Pendrell Vale and Black Lotus that I sold far too early, because that might make me grumpy.

Week ending 2023-07-16

2023-07-16T18:32:00Z

Small victory of the week: I’m starting to get on top of the washing

Tuesday

Overwhelm had left us with a huge pile of washing to do, filling the sink, the draining board, and various work surfaces and I kept putting off tackling it because my brainweasels just saw the sheer amount of work involved and shut down. No fun. Anyway, Gill grabbed her perching stool and set to and before I knew it there was a full dishwasher burbling away, an empty sink, a full draining board and the beginnings of a system to keep it that way. The goal is to keep the sink empty and the draining board full. Before I cook, I clear the draining board. Any pans I use go in the sink and the next time I make tea, I wash up what’s in the sink and anything that’s still not been done of the mahoosive pile, until there’s a full draining board again. Next time I’m brewing up, I can put the dry stuff away and, if I have the spoons, chip away at some more of the pile (though, post-COVID, I rarely have the spoons for much – I can only stand for so long).

It’s not perfect, but it gets stuff done, and I’m calling that a win.

Wednesday

Emily came over for rehearsal and Carcassonne. It was mostly Carcassonne, if I’m honest, but The Housewife’s Lament is starting to seem like we’ve learned it, as is We’ll Sit Upon The Gate, though that one feels like it could use another verse. The Mary Ellen Carter is pretty damned solid too, which is good.

We set a new record city score in Carcassonne too, managing to share a 105 point city. We even managed my first ever draw in the second game.

Friday

A solo stream this week. Again with the Overwhelm getting in the way of getting more guests booked, but I’m starting to fill the diary again, which is good. I’ve got Alex Cumming as my guest in a couple of weeks, and Helen Edwards, Talis Kimberley, Emily, and a folk legend who will remain nameless for the time being lined up for August and September.

Loopy Pro was mostly rock solid. The one-shot overdub whine cropped up once, and there was a hard crash to the home screen at one point, but a restart was fast and clean. Perils of running beta versions of sofware, I guess.

Saturday

I toddled over for the morning and afternoon sessions at the Bradfield Traditional Music Weekend – spent a happy few hours singing Americana (I sang Cabin in Glory and We’re Gonna Camp a Little While in the Wilderness which seemed to go down well) in the first session, then there was a lovely, ballad heavy, song session. I sang Tamlyn about as well as I’ve ever sung it, and it went down really well. I was knocked out by a cracking version of The Famous Flower of Serving Men in particular, but the whole session was great.

Sunday

Dim sum at the China Palace for lunch with Dougal & Liz, Matt and Jo and a few of the kids. Excellent as always. We may have overordered

I’m off to the closing session of BTMW later. I suspect it will be as good as the Saturday sessions.

Impermanence matters

2023-06-06T10:43:00Z

Back when lockdown started, for all we were both classed as ‘critically vulnerable’, we did pretty well. We had each other for company, the house is paid for, we have some private outdoor space, Doncaster Market remained open and I’d got a 25kg bag of flour. Life was… tolerable.

Except, as any musician will tell you, making music needs company. At least it does if you catch the magic that happens when a group of people are together making something beautiful. Which is why I started singing folk songs to the internet¹ every Friday night.² At first, I just sang into the dark to the two or three people who showed up in the text chat

Thank you all of you! You were awesome.

but I was still missing harmonies. So I worked out how to use Logic for live sound and did on the fly multitracking of chorus harmonies and that was great. Recently, I’ve switched to using Loopy Pro on my iPad³ and it’s been great – I can sing harmonies on far more songs now and I’ve started experimenting with fancier arrangements too.⁴

A key realisation came after a late night conversation with Squeezy John [Spiers] on Twitter about how a YouTube archive of performances can become a millstone around an artist’s neck. After all, if someone can catch a performance online from a couple of years ago, why would they pay to watch you doing it again? I don’t think most people think this way, but the life of a professional musician is marginal enough that it doesn’t take many to have an effect. So I started making the catch-up videos private after a week.

Or whenever I got round to it, if I’m honest

It was incredibly liberating. I stopped worrying too much about repeating myself every week – I always open with the same song, I usually close with one of about three songs, there’s a couple of staples that people seem to miss if I don’t sing them and if I fuck something up, it’s gone in a week and I can concentrate on doing better next time. My audience seem to be fine with it. If I do something particularly well, or something funny happens like my cat bringing the backdrop down on me

https://youtu.be/kbeFRIqSUZg

then I have the original video files, so I can edit and upload a high quality clip that I’m happy to leave available forever.

And it’s closer to the experience of music in person, too. Live music is a magical transient thing that lasts as long as the song and then it’s gone. Recordings are souvenirs, they can be delightful, but they simply can’t be the same as being in the moment.

It’s gone. I might sing it again next week, but just as you can’t cross the same river twice, that performance will be different. And that’s brilliant.

I’ll tell you what is permanent: the repeating Friday night entry in my diary that reads ‘Folk Stream.’ It doesn’t matter whether I’m streaming for 2 people and no tips or a busy room and £100+ in Ko-fi’s hat, singing to the internet never fails to lift my mood. It’s like I’ve made an appointment with joy.

Catch me every Friday night at https://youtube.com/pierscawley/live from 8pm UK Time. ↩︎
I tried a few of the Zoom singarounds that sprang up, but for reasons I can’t quite put my finger on, I found them far more stressful than just singing to the camer and interacting with an audience in text chat. The audio only Clubhouse ballad sessions that I started were way less stressy for me too. ↩︎
And a fancy dual-USB audio interface that means I can capture Loopy’s audio really cleanly. Check out the iConnectivity AUDIO4c (affiliate link) if that’s a thing you might need. There are other dual-USB audio interfaces that are probably at least as good, but this is the one I’ve got experience of using. ↩︎
Check out https://youtu.be/hWPmADRfPFQ for an example. ↩︎

More Mastodon fiddling

2022-11-20T22:32:00Z

There’s a certain frustrating joy in fiddling with the details of a thing so as to improve the formatting of the new thing you’ve just added to your site, and discovering that a side effect of the change is that a couple of niggles that you’d not quite got to the bottom of on the site itself are fixed in passing.

I think I’ve got the crossposting to Mastodon looking less awful, and I know I’ve got my front page looking better. Isn’t that lovely?

Migrating to Mastodon

2022-11-20T13:59:00Z

Twitter, eh? Elon bloody Musk!

In the happier timeline, Elton John bought Twitter and it became even more fabulous with every passing day. In the far more depressing timeline we find ourselves living in, Elon seems to be determined to tank the company and fuck the community.

So I’ve buggered off to Mastodon. At the time of writing, you’ll find me at @pdcawley@mendeddrum.org and, you might even be reading a version of this via Mastodon rather than directly on the site.

General purpose computing is the best!

2022-05-12T13:21:00Z

Sometimes folk ask why there’s a Raspberry Pi in the rack case that holds my streaming rig and I admit that the primary reason is that I had one kicking about the place, and the rack unit that holds my M1 Mac Mini is designed to hold a couple of Pis in the space that isn’t holding the Mac, so I might as well attach it. I had the feeling it would come in handy.

It turns out, I was right. It runs OBS to handle motion graphics, and Companion to control the ATEM Mini from a streamdeck. In the future I plan to use it as a router so the gear in the rack can live on its own private network with the Pi handling the connection to the outside world either via a network cable or (hopefully not) wifi or a tethered phone.

The latest “Oh, of course! It can do that too!” job it’s taken on is pretending to be an Apple TV, so I can mirror my iPad’s screen to it without mucking up the Mac’s display.

Once again, I’m reminded that computers can be anything you can program them to be… sometimes all at once. The Pi 4 is a mindbogglingly capable bit of kit. I’ve bought USB hubs

USB Hubs that don’t work!

that cost more and do less than my Pi.

There’s a Pi 3 plugged in somewhere

I think I know where it is, but I’m not entirely sure

that runs Pi Hole and helps eliminate intrusive ads from my web experience. It has sufficient spare power that, once I get around to it, I plan to hook up a cheap workhorse laser printer to it and configure it to work as a print server, which should let me retire the crappy unreliable bubblejet printer that only prints when you stand over it with a big stick and is always demanding the blood of innocents

The printer calls it ink, but have you seen the price of that stuff?

before it’ll deign to even try to print something.

About the only thing wrong with them (at the moment) is that they’re almost permanently out of stock everywhere. I watch rpilocator and it’s clear that people do get stock, but they sell out almost immediately. A chum who works for Raspberry Pi says that they are shipping lots and lots of boards, but the demand stays sky high. Which is a nice problem to have for them, I suppose.

Evolving my streaming setup

2022-05-10T17:10:00Z

Back when I first started streaming on the internet, I used a Logitech webcam and some lights I had picked up for doing product photography and such for the Loafery and some audio gear I had because, well, recording stuff is just fascinating. It was okay, but even with decent lighting and audio, the webcam was frustrating to control (basically, there was no control), so I picked up a cheap capture card from Amazon and drafted my Nikon D810

Absolutely not the camera I would recommend if you’re going out to buy a camera for streaming – it’s primary virtue being that it’s the camera I already owned. Get something mirrorless if you’re heading down this road

as my webcam and the appearance of my streams improved enormously. This worked fine with my slightly aging PC and Twitch Sings.

Then Twitch pulled the plug on Sings and I started doing my regular Friday night folk streams. At first this involved running OBS and Logic Pro on my aging MacBook Pro and it was sort of okay, until I started having guests and doing Song Swaps – the MacBook simply wasn’t up to the job of running Logic, OBS and Zoom, so for a while I had some unholy lashup, with Zoom and OBS running on the PC, my Mac running Logic, with patch leads between the Mac and PC audio interfaces so I could hear my guests and vice versa. It wasn’t pretty.

What saved it was adding an ATEM Mini Pro ISO from BlackMagic. Now I could hand off all the capturing, streaming and recording duties to that box and run Logic and Zoom on the Mac and everything just worked (with an utter rats nest of cables on my desk). The ATEM takes HDMI input from up to four sources and lets me switch what gets sent to the stream between them. There’s also a monitor preview output that can be switched independently between those four sources, as well as multiview and stream previews. Of course, I quickly used up all four sources, and I don’t even have a second camera! The way things are arranged by default is that I have a camera feed, my mac’s primary and secondary displays, and the output from a Raspberry Pi. Mac screen 2 is where the zoom window lives, and is connected to the ATEM via an HDMI splitter with the second output feeding a 7-inch field monitor that sits on a teleprompter setup so I can look guests in the eye.

Best purchase ever! I primarily use it with Zoom or to display the YouTube chat when I’m doing solo streams

The primary Mac screen is where I set up streams and drive Logic from. It feeds into the ATEM mostly so I can see it in the preview window or full size when I need to without having to have a another screen, but if I ever get a second camera, I’ll break mac screen 1 out to its own monitor and dedicate input 2 to that secondary camera. Input 4 is a Raspberry Pi that I use for motion graphics. This used to be a copy of chromium running in kiosk mode to display the Ko-Fi stream widget which displays donations as they happen, but I’ve recently managed to compile OBS studio with a working browser plugin and I’m using that instead now, which should allow me to add more overlays later. The only catch with that is that I’ve not yet managed to get Companion to compile,

Companion’s a brilliant app for controlling AV gear via a web interface or Stream Deck. I know it works on the Pi because they provide OS images for it.

so I think I’ll have to move that onto the Mac.

This all worked fine until recently. You see, the thing I love about unaccompanied singing is singing harmonies. And harmonies don’t work unless the timing is properly tight. That means I can’t sing harmonies with Zoom guests because the speed of light

and video and audio compression and decompression

screws things up completely. I got around this by harmonising with myself – either by using a Looper plugin in Logic¹ or just by using Logic’s multi-tracking² to record multiple layers of a song’s chorus. It all works, and works well, but I could never work out how to record harmonies for shanty type refrains.

Consider:

Oh the rain it rains all day long
  Bold Reilly oh, bold Reilly!
And them Northern winds they blow so strong
  Bold Reilly oh’s gone away

Chorus:
  Goodbye my sweetheart, goodbye my dear-o
    Bold Reilly oh, bold Reilly
  Goodbye my darling, goodbye my dear-oh
    Bold Reilly oh’s gone away

Ideally, I want to be layering up harmonies on the Bold Reilly oh, bold Reilly! and Bold Reilly oh’s gone away lines within each verse, as well as on the chorus (and potentially on any chorus repeats too). It should be possible to set up Logic’s live looping feature to enable this sort of thing, but I could never work out how, and I wasn’t ready to switch to Ableton Live (where I couldn’t be sure I knew how to make it work either). So I stuck to just harmonising on the chorus and wishing there was a better way.

Enter Loopy Pro!

I’ll write a full review of Loopy Pro one of these days, but suffice it to say, there is now a better way. Loopy Pro is the long awaited successor to Loopy HD, a software looper for iOS. It’s been a very long time coming, but by god, does it deliver! As well as being a looper, it’s a great replacement for MainStage and (at least for live use) Logic Pro, and it runs on even pretty ancient iPhones and iPads. It’s predecessor was remarkably capable, but was ‘just’ a looper with a configurable number of loops available. Loopy Pro is fully customisable, supporting any number of loops, one shot samples, beat slicing, AUv3 hosting, mix busses, control widgets, faders and dials. There’s a deep system of actions and ‘follow actions’ that allow you to customize its behaviour as well as its appearance, and the audio routing capabilities could embarrass some far more expensive DAW software. It’s astonishingly capable.

Figure 1: Loopy Pro configured for singing ‘shanty’ structured chorus songs

I’ve been fiddling with it since it was released, and now have it set up to allow me to sing shanty style songs with harmonies on the refrain as well as stuff like this:

https://youtu.be/hWPmADRfPFQ

so now I’m running Friday night stream audio from the iPad, leaving the Mac to run Zoom. This has all been made much easier since I added a new audio interface: the iConnectivity AUDIO4c interface is a remarkable bit of kit. Uniquely, as far as I can tell, it can be used as an audio interface simultaneously by my mac and my iPad, and can route audio from the iPad to the mac and vice versa. It’s got four inputs and six outputs, which is two more than existing interface. That means I can have my guest and me fed to the ATEM mini on separate channels, as well as giving myself a different headphone mix. And it’s only one rack unit high!

Focusrite “small” interfaces, which are what I’ve used up to now are all about 1.5U high unless you buy the versions with 8 preamps – lovely bits of kit, but rather more than I either need or can afford.

The remaining bit of the puzzle is to reliably capture the iPad display. I’m looking for, and so far failing to find, a powered USB-C hub that has a bulletproof iPad HDMI connection and gigabit ethernet, which is much more reliable when it comes to remote control of the ATEM. It’s a frustrating search. Everything I’ve found so far has intermittent HDMI dropouts, which would be annoying enough if it weren’t for another ‘feature’ of iOS and iPadOS.

It works like this: on iDevices, you don’t have the option to choose which audio interface you want to use, instead the OS autoselects whichever interface was plugged in most recently. Which would be fine (sort of) if it weren’t for the fact that an HDMI connection to an audio capable device is treated as a new audio interface. So… if the HDMI connection was reliable, getting prepped for a stream would just involve unplugging the audio interface from the hub, connecting the HDMI and reconnecting to the AUDIO4c. But then the HDMI drops and comes back, and suddenly it is the most recently connected interface and the stream’s audio is buggered.

If you know of an iPad friendly USB-C hub that has rock solid HDMI, then I’d love to hear about it because my search is getting really frustrating. Right now I’m working around it by disconnecting the audio, screen sharing to the Mac and reconnecting the audio, but even with a dual screen mac set up, screen sharing takes over both screens

Well… it takes over one screen and fades the other to black. Thanks Apple!

and the whole thing has a bunch of latency that you don’t really want – it’s surprising how little latency will start making audio software in particular seem seriously out of sync. I can correct for that when editing, but not so much on a live stream.

But… once I have reliable HDMI from the iPad I run into another problem. Where do I plug it in? Suddenly four inputs on the ATEM aren’t really enough.

Time to start saving up for an ATEM Mini Extreme ISO, which is the extra-wide version of the ATEM Mini Pro ISO, complete with 8 HDMI inputs, 2 HDMI outputs, 2 USB-C connections, a 3.5mm audio jack output (so I can monitor the audio going to the stream rather than just looking at the VU meters on the multiview display) and something called a ‘Super Source’, which would definitely simplify the business of setting up the split screen view when I have a Zoom guest.

Because of restrictions in the way the ATEM Mini Pro ISO works, I have to sit to one side of the shot because I can only adjust the scale, crop and position of one video source at a time. The Super Source deals with all that.

The extreme version appears to fix pretty much everything that I find slightly annoying about the Mini, which is brilliant, but could be brillianter. Ah well, a boy can dream.

If you’re interested in seeing what all this technology ends up looking like on stream, then I stream at 8pm UK time every Friday night on my YouTube channel. Maybe pop by, and if you like what you see and hear, don’t forget to like and subscribe.

I used the delightfully named Augustus Loop from Expert Sleepers combined with a Lua script I wrote to make it behave more or less the way I wanted it to. Kind of fiddly to set up, but repaid the effort. There’s still a few things that AL can do that I can’t do with Loopy Pro, but as I write those are due in the next big Loopy patch. ↩︎
Logic isn’t really set up to do live sound, but MainStage, which is, can’t do multi track recording and playback and I couldn’t work out how to configure its looper to emulate that. So I just used Logic. ↩︎