Fixing the irregular

Our word “fix” comes from fixus: unwavering; immovable; constant; fixed/fastened. Well, the scottbot irregular has been slowly breaking for years, finally broke last week, and it was time to fix it.

Broken how?

A combination of human error (my own), accruing chaos, the complexities of WordPress, and the awful-but-cheap hosting solution that is As many noticed, the site’s been slowing down, interactive elements like my photo gallery stopped working, and by last week, pages would go dark for hours at a time. By this week, bluehost no longer allowed me ftp or cpanel access. So yesterday I took my business to, the hands-down best (and friendliest) hosting service for personal and small-scale academic websites.

Quoth the Server "404"
Quoth the Server “404”

I still haven’t figured out what finally did it in, but with so many moving parts, it seemed better to start fresh than repair the beast. I’m currently working on a jekyll static website; this new wordpress blog you’re reading now is an interim solution. However, I couldn’t just cut my losses and start over, since I’ve put a lot of my soul into the 100+ blog posts & pages I’ve written here since 2009.

More importantly, my site has been cited in dozens of articles, and appears on the syllabus of hundreds of courses, DH and otherwise. If I delete the content, I’m destroying part of the scholarly record, and potentially ruining the morning of professors who assign my blog posts as reading, only to find out at the last minute that it no longer exists.

Here lies the problem. Because I no longer had back-end access to my website, I could not download my content through the usual channels. Because of the peculiarities of my various WordPress customizations, not worth detailing here, I could not use a plugin to export my site and its contents.

Since I wanted the form of my site preserved for the scholarly record, the only solution I could come up with was to crawl my entire site, externally, and download static html versions of each page on as it used to exist.

the old scottbot irregular
the old scottbot irregular

Fixed how?

This is where the double-meaning of fix, described above, comes into play. I wanted the site functioning again, not broken, but I also wanted to preserve the old website as it existed at the URLs everyone has already linked to. For example, a bunch of syllab(uses|i)  link to to direct their students to my networks demystified posts. I wanted to make sure that URL would continue to point to the version of the site they intended to link to, while also migrating the old content into a new system that I’d be able to update more fluidly. Thankfully the old directory for the site, /HIAL/ (the site used to be called History Is A Lab), made that easier: the new version of the irregular would reside on, and the archive would remain on

This apparently isn’t trivial. The first step was to use wget (explained and taught by Ian Milligan on the Programming Historian) to download a static version of the entire original irregular. After fiddling with the wget parameters and redownloading my site a few times, I ended up with a mostly-complete mirror of all the old content. Then I uploaded the entire mirror to my new host in the /HIAL/ directory. Yay!


The only catch was that old dynamic page URLs, like, were saved by wget as static html pages, like The solution, Dave Lester helped me figure out last night, was to edit the .htaccess file to make people linking to & visiting HIAL/?tag=networks-demystified automatically redirect to HIAL/index.html@tag=networks-demystified.html

The .htaccess file sits on your server, quietly directing traffic to the various places it should go. In my case, I needed to use regular expressions (remember that thing Shawn, Ian, and I taught in The Historian’s Macroscope?) to redirect all traffic pointing to HIAL/?[anything] to HIAL/index.html@[anything]. An hour or so of learning how .htaccess worked resulted in:

RewriteEngine On
RewriteBase /HIAL/
RewriteCond %{QUERY_STRING} ^(.*)$
RewriteRule ^$ index.html@%1.html? [L,R=301]

which, after some false starts, seems to work. The old site is now fixed, as in constant; secured; unwavering, at The new irregular, at, is now fixed, as in functional, dynamic. It will continue to evolve and change.

the scottbot irregular is dead. Long live the scottbot irregular!

A quick note on blog sustainability

[edit: I’ve been told the word I’m looking for is actually preservation, not sustainability. Whoops.]

Sustainability’s a tricky word. I don’t mean whether the scottbot irregular is carbon neutral, or whether it’ll make me enough money to see me through retirement. This post is about whether scholarly blog posts will last beyond their author’s ability or willingness to sustain them technically and financially.

A colleague approached me at a conference last week, telling me she loved one of my blog posts, had assigned it to her students, and then had freaked out when my blog went down and she didn’t have a backup of the post. She framed it as being her fault, for not thinking to back up the material.

Of course, it wasn’t her fault that my site was down. As a grad student trying to save some money, I use the dirt-cheap bluehost for hosting my site. It goes down a lot. At this point, now that I’m blogging more seriously, I know I should probably migrate to a more serious hosting solution, but I just haven’t found the time, money, or inclination to do so.

This is not a new issue by any means, but my colleague’s comment brought it home to me for the first time. A lot has already been written on this subject by archivists, I know, but I’m not directly familiar with any of the literature. As someone who’s attempting to seriously engage with the scholarly community via my blog (excepting the occasional Yoda picture), I’m only now realizing how much of the responsibility of sustainability in these situations lies with the content creator, rather than with an institution or library or publishing house. If I finally decide to drop everything and run away with the circus (it sometimes seems like the more financially prudent option in this academic job market), *poof* the bulk of my public academic writings go the way of Keyser Söze.

So now I’m going to you for advice. If we’re aiming to make blogs good enough to cite, to make them countable units in the scholarly economy that can be traded in for things like hiring and tenure, to make them lasting contributions to the development of knowledge, what are the best practices for ensuring their sustainability? I feel like I haven’t been treating this bluehost-hosted blog with the proper respect it needs, if the goal of academic respectability is to be achieved. Do I self-archive every blogpost in my institution’s dspace? Does the academic community need to have a closer partnership with something like to ensure content persistence?

The Historian’s Macroscope

Whelp, it appears the cat’s out of the bag. Shawn Graham, Ian Milligan, and I have signed our ICP contract and will shortly begin the process of writing The Historian’s Macroscope, a book introducing the process and rationale of digital history to a broad audience. The book will be a further experiment in live-writing: as we have drafts of the text, they will go online immediately for comments and feedback. The publishers have graciously agreed to allow us to keep the live-written portion online after the book goes on sale, and though what remains online will not be the final copy-edited and typeset version, we (both authors and publishers) feel this is a good compromise to prevent the cannibalization of book sales while still keeping much of the content open and available for those who cannot afford the book or are looking for a taste before they purchase it. Thankfully, this plan also fits well with my various pledges to help make a more open scholarly world.

Microscope / Telescope / Macroscope [via The Macroscope by Joël de Rosnay]
Microscope / Telescope / Macroscope [via The Macroscope by Joël de Rosnay]
We’re announcing the project several months earlier than we’d initially intended. In light of the American Historical Association’s recent statement endorsing the six year embargo of dissertations on the unsupported claim that it will help career development, we wanted to share our own story to offset the AHA’s narrative. Shawn, Ian, and I have already worked together on a successful open access chapter in The Programming Historian, and have all worked separately releasing public material on our respective blogs. It was largely because of our open material that we were approached to write this book, and indeed much of the material we’ve already posted online will be integrated into the final publication. It would be an understatement to say our publisher’s liaison Alice jumped at this opportunity to experiment with a semi-open publication.

The disadvantage to announcing so early is that we don’t have any content to tease you with. Stay-tuned, though. By September, we hope to have some preliminary content up, and we’d love to read your thoughts and comments; especially from those not already aligned with the DH world.

Zotpress is so cool.

So, you may have noticed this site has been overhauled over the past few days. The old WP theme really wasn’t doing it for me, so I decided to switch to the Great and Glorious Suffusion theme, which is more customizable than barrel of monkeys. The switch to the new theme opened up all sorts of real-estate for new content, and a brief look around the #DH blogosphere landed me on Zotpress.

Do you guys use Zotero? You should use Zotero. It’s a fantastic citation management program that snuggles up nice and close to your browser and turns it into a super research machine.

Dear Zotero, I ♥ you.

Anyway, Zotpress is a WordPress plugin that allows you to put the power of Zotero into your blog. Want to reference stuff? Easy! Want to make a list of most recently read items? Cake! (See the right side of this blog for that particular feature.) This is one of those plugins that I never thought I needed, but now that I have it I cannot imagine blogging efficiently without it.

For your reading pleasure, below is a list of some of super cool articles, courtesy of Zotpress:

[zotpress item=”34ABEHCE,D9SRGW5H,H52588XW,EF3KZ27G,HK6XQ3CI,42WF9AT7,SH7RT4P5″]