The lesson is not what you think

Instructure, business continuity, and enterprise IT

I had hoped to avoid saying anything about the Instructure ransomware event. It’s getting so much press at the moment, and honestly I haven’t reached out to any contacts to even try to find out what really happened. I’m assuming people smarter and more connected than I am are putting pen to paper and I’m sure some point soon we’ll learn the nitty gritty of what happened. Was this a serious case of sophisticated hacking? A self-inflicted wound? God help me if it turns out to be “some contractor’s laptop” or “a rogue developer’s unprotected account.”

I do, however, want to take a moment for a long overdue self-reference. In one of my first national publications, I think it was for the Chronicle, I referred to our campus provided services as “dinosauric”. Mysteriously the editor didn’t try to make me change it. This came to mind as I was reading a short Atlantic piece on the Canvas incident from the perspective of a faculty member. While the article itself is a fun read, the real meat and potatoes are the comments. For me, the money shot is this comment (italics are mine).

I taught at a college that used Canvasas's [sic] competitor Blackboard. It was so clunky, especially in grading and rubrics. For most of Blackboard's functions, I set up a parallel system on Google Docs and through gmail. That saved me tons of headaches. It also made it easy to deal with a courseware outage.

On one hand, this can be seen as a testament to human ingenuity; on the other, it’s a sign of how dinosauric our enterprise systems can be. This workaround - or some flavor of it - will be all too familiar to anyone who’s worked in higher education, either academically or administratively¹. It’s also just as likely to reflect the rule of thumb that people would prefer to use the familiar rather than train themselves to use the unfamiliar. I wouldn’t at all be surprised if the parallel system described is far more labor intensive than the designed system it’s replacing. Human nature being what it is.

Browsing the comments one sees a wide spectrum of beliefs on display: the attitudes range from smug “tech-skepticism” to pragmatic defense of progress, all tied together by a shared resentment toward clunky enterprise software. Some comments focus on the lack of agency students and faculty felt when the system went down. That “the system” is the law - without it only failure remains. Others point to the “management vs. learning” divide, convinced that a learning management system (essentially an administrative front end to a course) has been forced on them as a proxy for the good old days of faculty to student connection².

A few comments go down the path of inevitability and incentives. Since institutions are stuck with these terrible systems, providers have little incentive to improve them. My own take is that while this may be broadly true in a monopoly context, I think the real challenge is deeper than this. And of course, a common thread through many of the comments is pragmatism vs. nostalgia. Many long for the past of small to modest classes, classrooms with little technology, and only the strength of faculty personality animating teaching. Others recognize that while these modern systems are flawed, they’ve permitted the delivery of courses to millions who would otherwise be unable to attend college.

There are a number of takeaways from this incident and the comments to the Atlantic piece. First, I think it’s a beautiful encapsulation of how enterprise IT is viewed by faculty, who are the very engine of the institution. Granted, this isn’t comprehensive data captured by rigorous surveys, interviews, or analysis. It’s a modest number of Atlantic readers who bothered to post a comment. But it does sound and feel awfully familiar to someone who’s spent their career in academia.

Nor is it clear that a great solution exists. Every school wrestles with these hopeless and hopelessly expensive systems - ERP and educational - and none of them are really any good. The delta between tech as experienced IRL and at work just keeps burning user goodwill, despite the obvious improvements these systems have made over the years. I can bank, plan a trip, buy tickets, book hotels, order food, and date with my thumb, but filing an expense report or checking the results of an exam tests anyone’s patience.

In a previous role, the CIO had promised the senior administration that a new ERP would be better and cheaper than sliced bread. Since they trusted me, I was quietly invited for a conversation about the matter. I told them the truth: the new system would be hated, more difficult for staff than the finely honed existing system, but that this would lead to a workforce refresh as the older staff retired and were replaced. It would cost 2-4x what they were told - but, the market simply didn’t offer many alternatives. Just different flavors of ‘meh’. I also said it was necessary since the old system was rapidly becoming unsupportable. In the market of higher ed ERPs, we weren't buying a solution; we were buying a decade of slightly more modern debt. I suspect this paragraph is as true now, and in the future, as it was then.

So you can see that enterprise IT shops are between a rock and a hard place. This is truly the deeper challenge I alluded to earlier. No one is going to hand you the tens or hundreds of millions it costs for these systems with that as the system précis. Be that as it may, it’s essential that IT providers understand how their offerings are viewed and give a damn about it. Woven throughout the comments are a number of ideas that simply aren’t true and should be subject to a deliberate and sustained effort to correct. The beliefs and attitudes are the lived experience of the faculty and cannot be dismissed - but the narratives shaping them can be changed. I recognize that many in IT will see only the naïvete in the comments. But I would argue that to do so is to ignore a valuable source of feedback from an essential community.

The next takeaway brings us back to the incident itself, or more precisely, our dependency on providers who inevitably will suffer incidents. Everyone is treating this as a security event, when the lesson to learn here is one of business continuity. In an earlier piece, I talked about how we should interrogate our governance bodies to lead them to express assumptions and risk tolerances - to operate at the wisdom layer of governance. In response to this event, I would hope schools are asking questions like what are we structurally blind to? What does this say about our procurement model? What risk posture are we choosing?

Obviously, for the first, many institutions were blind to their total dependence on Canvas. Where else does that dependency lie? As to our procurement model, it is the persistence of these tools despite near universal hatred that needs questioning. In terms of our risk posture, are we trading resilience for the convenience of centralized “management”?

---

It’s simply too easy to tsk tsk at Instructure. While it may yet prove that they deserve some scorn, I would hope institutions use this to look at their own reflection in the mirror of this event. Outages don’t create failure - they reveal where the real system already exists. Institutions should recognize that risk posture is an emergent property of procurement and dependency choices seemingly unrelated to cybersecurity. We have co-authored the chaos we now are experiencing.

1. I also wonder if institutional fragility is being masked by user ingenuity - and what precisely is the scope of that issue?

2. Over the years I’ve taught plenty of courses with hundreds of students - I would have loved a management tool to help with the logistics of those.