Fixing the irregular

Our word “fix” comes from fixus: unwavering; immovable; constant; fixed/fastened. Well, the scottbot irregular has been slowly breaking for years, finally broke last week, and it was time to fix it.

Broken how?

A combination of human error (my own), accruing chaos, the complexities of WordPress, and the awful-but-cheap hosting solution that is As many noticed, the site’s been slowing down, interactive elements like my photo gallery stopped working, and by last week, pages would go dark for hours at a time. By this week, bluehost no longer allowed me ftp or cpanel access. So yesterday I took my business to, the hands-down best (and friendliest) hosting service for personal and small-scale academic websites.

Quoth the Server "404"
Quoth the Server “404”

I still haven’t figured out what finally did it in, but with so many moving parts, it seemed better to start fresh than repair the beast. I’m currently working on a jekyll static website; this new wordpress blog you’re reading now is an interim solution. However, I couldn’t just cut my losses and start over, since I’ve put a lot of my soul into the 100+ blog posts & pages I’ve written here since 2009.

More importantly, my site has been cited in dozens of articles, and appears on the syllabus of hundreds of courses, DH and otherwise. If I delete the content, I’m destroying part of the scholarly record, and potentially ruining the morning of professors who assign my blog posts as reading, only to find out at the last minute that it no longer exists.

Here lies the problem. Because I no longer had back-end access to my website, I could not download my content through the usual channels. Because of the peculiarities of my various WordPress customizations, not worth detailing here, I could not use a plugin to export my site and its contents.

Since I wanted the form of my site preserved for the scholarly record, the only solution I could come up with was to crawl my entire site, externally, and download static html versions of each page on as it used to exist.

the old scottbot irregular
the old scottbot irregular

Fixed how?

This is where the double-meaning of fix, described above, comes into play. I wanted the site functioning again, not broken, but I also wanted to preserve the old website as it existed at the URLs everyone has already linked to. For example, a bunch of syllab(uses|i)  link to to direct their students to my networks demystified posts. I wanted to make sure that URL would continue to point to the version of the site they intended to link to, while also migrating the old content into a new system that I’d be able to update more fluidly. Thankfully the old directory for the site, /HIAL/ (the site used to be called History Is A Lab), made that easier: the new version of the irregular would reside on, and the archive would remain on

This apparently isn’t trivial. The first step was to use wget (explained and taught by Ian Milligan on the Programming Historian) to download a static version of the entire original irregular. After fiddling with the wget parameters and redownloading my site a few times, I ended up with a mostly-complete mirror of all the old content. Then I uploaded the entire mirror to my new host in the /HIAL/ directory. Yay!


The only catch was that old dynamic page URLs, like, were saved by wget as static html pages, like The solution, Dave Lester helped me figure out last night, was to edit the .htaccess file to make people linking to & visiting HIAL/?tag=networks-demystified automatically redirect to HIAL/index.html@tag=networks-demystified.html

The .htaccess file sits on your server, quietly directing traffic to the various places it should go. In my case, I needed to use regular expressions (remember that thing Shawn, Ian, and I taught in The Historian’s Macroscope?) to redirect all traffic pointing to HIAL/?[anything] to HIAL/index.html@[anything]. An hour or so of learning how .htaccess worked resulted in:

RewriteEngine On
RewriteBase /HIAL/
RewriteCond %{QUERY_STRING} ^(.*)$
RewriteRule ^$ index.html@%1.html? [L,R=301]

which, after some false starts, seems to work. The old site is now fixed, as in constant; secured; unwavering, at The new irregular, at, is now fixed, as in functional, dynamic. It will continue to evolve and change.

the scottbot irregular is dead. Long live the scottbot irregular!

A quick note on blog sustainability

[edit: I’ve been told the word I’m looking for is actually preservation, not sustainability. Whoops.]

Sustainability’s a tricky word. I don’t mean whether the scottbot irregular is carbon neutral, or whether it’ll make me enough money to see me through retirement. This post is about whether scholarly blog posts will last beyond their author’s ability or willingness to sustain them technically and financially.

A colleague approached me at a conference last week, telling me she loved one of my blog posts, had assigned it to her students, and then had freaked out when my blog went down and she didn’t have a backup of the post. She framed it as being her fault, for not thinking to back up the material.

Of course, it wasn’t her fault that my site was down. As a grad student trying to save some money, I use the dirt-cheap bluehost for hosting my site. It goes down a lot. At this point, now that I’m blogging more seriously, I know I should probably migrate to a more serious hosting solution, but I just haven’t found the time, money, or inclination to do so.

This is not a new issue by any means, but my colleague’s comment brought it home to me for the first time. A lot has already been written on this subject by archivists, I know, but I’m not directly familiar with any of the literature. As someone who’s attempting to seriously engage with the scholarly community via my blog (excepting the occasional Yoda picture), I’m only now realizing how much of the responsibility of sustainability in these situations lies with the content creator, rather than with an institution or library or publishing house. If I finally decide to drop everything and run away with the circus (it sometimes seems like the more financially prudent option in this academic job market), *poof* the bulk of my public academic writings go the way of Keyser Söze.

So now I’m going to you for advice. If we’re aiming to make blogs good enough to cite, to make them countable units in the scholarly economy that can be traded in for things like hiring and tenure, to make them lasting contributions to the development of knowledge, what are the best practices for ensuring their sustainability? I feel like I haven’t been treating this bluehost-hosted blog with the proper respect it needs, if the goal of academic respectability is to be achieved. Do I self-archive every blogpost in my institution’s dspace? Does the academic community need to have a closer partnership with something like to ensure content persistence?

Hire me.

I just added a new static page to the Irregular asking you to hire me for short consulting jobs. It’s not that I need the money (although it certainly wouldn’t hurt), but rather I enjoy being part of new DH projects and love traveling to new places. I’ve done a bunch of consulting work in the past; my favorites were helping out at Early Modern Letters Online at Oxford University, and leading a team to design and implement a virtual research environment for the CKCC in the Netherlands. I’ve also helped plan and worked on DH startup grants, Digging Into Data projects, and a handful of other projects you can find on my CV. In an effort to get more experience in various types of projects before I finish my Ph.D., as well as to travel more, I’m offering my experience to help plan digital humanities projects, create criteria for hiring DH faculty, design curricula, etc. If you want more information or know anyone who would like assistance in anything from computational methodologies to event planning, follow this link!

DH gun for hire. via.

I redesigned the irregular

And I’d like your opinion on whether or not I should keep this design or revert to the old one. Comments are welcome, as are suggestions to fix the front-page excerpt size which seems to be stuck at 55 words, even after adding the appropriate line to functions.php. Click through for the poll.

The theme has been (slightly) modified from Montezuma and includes all sorts of new and exciting widgets which add very little content to the site but still make me happy.

Currated syllabi

Those who follow me on twitter already know about my new curated course syllabi page, but here’s an announcement for the RSS crowd (I’m looking at you, Elijah). Basically, there are a bunch of DH courses out there (and great lists of syllabi at CUNY, the DH Working Group on Zotero, and the DH Education Group on Zotero), but I know of no resource that specifically focuses on the computational / algorithmic / quantitative humanities syllabi, so I decided to put one together. You can find it here, as well as in the pages menu up top, at this point including 19 courses taught over 24 semesters, and I welcome suggestions of more.

via xkcd


Welcome to the scottbot irregular. As the title suggests, this blog will (I think) remain irregular, both in content and in timing. It will probably be host to news and musings about new scientific discoveries I find sexy or alarming, discussions of exciting happenings in the world of history of science, information science, or digital humanities, and meta-discussions or critiques of scientific methodologies and computational humanities methodologies. Also my preliminary research. Also whatever else I feel like. As for the irregular timing, if you want to keep up with the blog, it’d probably be best to just subscribe via RSS.