To my mind this is one of the most blindingly obvious policy changes needed in Washington – on par with other desperately needed changes like campaign finance reform. Rather than re-explain why this is true, I’ll reuse this brief video I made for Open Education Week several years ago. In the video I explain why all publicly funded educational resources should be openly licensed. The video uses research articles as its primary example, but the logic applies to videos, textbooks, curricula, simulations, and all other educational materials created with public funds.

I was very happy to see the Open Education Group’s Review Project included in the letter’s list of evidence for why this policy change should be made. It’s good to know the work we’re doing is making a difference.

Observation 1: We Need Dramatically More OER, and Current Approaches Won’t Get Us There

A growing number of individual faculty, as well as groups of faculty comprising entire degree programs, are choosing to abandon increasingly expensive textbooks (e.g., see this article on the $400 textbook) for open educational resources (OER). This strategy is extraordinarily effective at eliminating barriers to access and success for students in many of the highest enrolling courses, like introductory and general education courses, business courses, and mathematics courses.

However, faculty that want to use OER sometimes find that insufficient resources are available for the specific courses they teach. The problem exists for both popular community college degree programs, like criminal justice or nursing, and upper division courses in universities. Given that universities can have over 2000 courses in their catalogs, and there are sufficient OER available for something like 100 courses, there are still around 2000 courses worth of OER that need to be created – and constantly updated – before we can realize the vision of ubiquitous OER adoption across all of higher education (recognizing the narrow but important limits on replaceability I have outlined previously).

Foundations and other institutions funded the initial wave of OER production which, over the past decade or so, has brought the field of open educational resources to where we currently are. While this pattern will likely continue at some level, the funding and production models that successfully kickstarted the OER movement cannot possibly scale to provide an additional 2000 courses worth of material. This inability to provide enough OER to meet faculty and student demand is a critical obstacle to achieving ubiquitous adoption of OER across all of higher education. You can’t adopt what isn’t there.

(You may be tempted to say that the inability of the market to respond to clear signals of demand for OER proves that there is something fundamentally broken with the economics of OER. If OER “worked right,” “the market” would obviously “deploy capital” and solve the “demand problem,” you might think to yourself. If you had a thought like this one, please pause for a few moments and familiarize yourself with the brilliant work of Yochai Benkler before continuing.)

Observation 2: Replacing Disposable Assignments with Renewable Assignments is Awesome

The defining characteristic of a disposable assignment is the tacit understanding that as soon as the faculty member returns the graded assignment to the student, the student will promptly throw it away. Aside from their pedagogical benefit – which faculty are notoriously poor at helping students understand – disposable assignments add no value to the world. Given its immutable destiny for the garbage can, students see little value in investing themselves in these assignments. And faculty dislike grading these assignments almost as much as students dislike doing them, and for the same reasons.

However, assignments don’t have to be a seemingly pointless endeavor that drive both students and faculty to complain on Facebook about completing them and grading them. There are excellent pieces of student homework that have undeniably made the world a better place. Take Murder, Madness and Mayhem, Project Management for Instructional Designers, and Emerging Perspectives on Learning, Teaching, and Technology as examples. In each of these cases, students created new material or revised existing material, ensuring that the final product was thorough, thoughtful, well documented, and well suited to the needs of students studying specific topics. Three of the articles written for Murder, Madness and Mayhem achieved Featured Article status on Wikipedia and have already been viewed by hundreds of thousands of readers with hundreds of thousands more to come. Project Management for Instructional Designers and Emerging Perspectives on Learning, Teaching, and Technology have been adopted at several universities.

None of these assignments will ever see the bottom of a garbage can. And the value they add to the world increases exponentially because they are all openly licensed. In addition to being viewed and used by countless people, they will also be extended, revised, and improved by future students and others. As a contrast to “disposable” assignments, it seems appropriate to call these renewable assignments. These renewable assignments result in meaningful, valuable artifacts that enable future meaningful, valuable work. Students tell me that they invest significantly more time and effort in these assignments and enjoy doing them more. And as a faculty member I can definitely say that I find grading these assignments to be much more rewarding.

Of Birds and Stones

There appears to be a fascinating opportunity here to kill multiple birds with a single stone. Could a move to renewable assignments solve the problem with scaling OER production?

First, let’s see what kind of capacity we’d be talking about. How much time are students collectively spending doing disposable assignments?

• In 2012 there were over 20 million students enrolled in US institutions of higher education. 13M of these were full-time and 7M were part-time.
• If the full-time students are taking 12 credit hours per semester and the part-time students are taking six, then the average student is taking about 10 credits per term, or 20 credits per year.
• Students are often told to expect to spend two hours outside of class for every hour they spend in class.
• Much of the time students spend outside of class is reading for class, studying for tests, etc. Let’s be conservative and estimate that only 5% of their outside of class time is spent doing what we would call disposable assignments.
• What does that give us? 20M students x 20 credits per year x 2 hours outside of class per credit x 5% of that time spent on disposable assignments = 40 million hours spent by students on disposable assignments. Every year. Year after year.

It’s a bit like Shirky’s notion of cognitive surplus. These are 40 million hours that students are already spending every year producing artifacts specified by faculty. Faculty simply need to tweak the specification to which students are working. And if converting some proportion of those disposable assignments into renewable assignments would provide the benefits I’ve listed above and others I’ve discussed before, it would be worth doing even outside the context of the OER supply problem. Providing a solution to the OER supply problem is a happy secondary benefit that just makes it that much more worth doing. The idea of instructional materials written by students for students is incredibly appealing to me.

Ok, there’s clearly enough student production capacity already in place. What would faculty have to do to make this work?

Faculty will have to be thoughtful about creating and providing a Table of Contents-like framework in which the renewable assignments can be completed and combined into a textbook replacement. Benkler covers this territory in some detail, though there will be important differences when contributors are students as opposed to the traditional volunteer contributors of open source software Wikipedia. In general this should make the task easier instead of harder.

Renewable assignments also imply a shift in faculty thinking from “grading” to “editing.” For each individual assignment, each individual student is creating an artifact that provides a unique, student-centric view of a topic. This artifact will be learned from and then extended and improved upon by future students. Faculty editorial feedback and direction encourages students to make this work as good as it can be – hopefully in many cases good enough to be provided to students in place of a section or chapter of a textbook.

Thinking about the whole collection of assignments, the faculty member is essentially editing a volume of contributed pieces. Some of them might be shorter essays and others might be book chapters. Some might be videos or original songs or printable card games. Faculty will of course need to assign a grade to the artifact at some point, but my experience has been that when students are engaged in this kind of task almost all of the work ends up being A work.

Finally, faculty need some basic level of facility with open licenses and technology to make this work. They will need to be able to explain to students what the Creative Commons Attribution License is, what obligations it places on users, what rights it extends to users, and why students should openly license their work. And they will need to be able to help students who agree to publish their work online under an open license actually do so.

These requirements of framework creation, editing, explaining open licenses, and providing technology support will be beyond many faculty unless they receive some targeted professional development. This means that, while the total amount of time students currently spend on disposable assignments may be around 40 million hours, we can’t actually reclaim all of them. We probably can’t even reclaim most of them. Still, even a tiny percentage of them would yield 500,000 or a million hours year after year focused on producing and improving OER – for students, by students. (The connections to Von Hippel’s work on democratizing innovation have interesting application here as well.)

Next Steps

I need to think about this more, but that’s the point of most of my writing – thinking out loud. The majority of the logistics for making this work are either outlined by Benkler or things we’ve already learned through previous experiments like Project Management for Instructional Designers. There will be messaging and professional development challenges. These will be real challenges but they don’t appear, a priori, to be insurmountable. And we don’t actually need to reach every faculty member initially – we only need to reach a sufficient number. Something like 5 faculty per course (out of all the faculty in the country teaching that course) would likely be enough to create an initial openly licensed textbook replacement in 2 years. Then others could begin adopting and extending that baseline collection. Network effects then ensue.

Thoughts? What am I missing?

I run dozens of websites – everything from opencontent.org to davidwiley.org to openedconference.org to the veritable reusability.org. Several of the websites I run are 15 or 20 years old. For over a decade now I’m been hosting all my websites at BlueHost. But a few weeks ago I decided to throw in with the awesomeness Tim and Jim are making over at Reclaim Hosting and move all my web hosting there.

I could not possibly be happier with my choice. The people, support, and tools at Reclaim Hosting are ridiculously wonderful. They made the migration of all my sites completely painless. If you host websites, I strongly encourage you to make the move to Reclaim Hosting. Get terrific support for your website and support the great things these fine folks are doing.

There are many reasons to be deeply concerned about this fully automated future. One of the things that concerns me most about this vision of “optimized” instruction is its potential to completely undermine learners’ development of metacognitive skills and deprive them of meaningful opportunities to learn how to learn.

Like every other skill – from playing the piano to factoring polynomials to reasoning about the likely causes of historical events – learning how to learn requires practice. Learners need opportunities to plan out their own learning and select their own study strategies and learning resources. Learners need opportunities to monitor and evaluate the effectiveness of the strategies and resources they’ve selected in support of their own learning. Learners need to experience – and reflect on – a range of successes and failures in regulating their own learning in order to understand what works for them, and how they should approach the next learning task they encounter in school or life.

Some adaptive systems are designed specifically to take control of these metacognitive processes away from learners. These systems make decisions on behalf of the learner, monitoring what does and doesn’t appear to be working and updating their internal models and strategies. The processes by which these decisions are made are hidden from the learner, and are likely trade secret black boxes into which no reviewer can ever peer. At the end of the current reading, or video, or simulation, the system will present the learner with a “Next” button that hides all the complexity and richness of the underlying decision, and simply presents the “optimal” next learning activity to the learner.

Without meaningful opportunities to develop metacognitive skills, there is no reason to believe that learners will develop these skills. (Have you ever spontaneously developed the ability to speak Korean without practicing it?) A fully adaptive system likely never provides learners with the opportunity to answer questions like “What should I study next?”, “How should I study it?”, “Should I read this or watch that?”, or “Should I do a few more practice exercises?” It goes without saying that the ability to learn quickly and effectively is possibly the single most important skill a person living in the modern world can have. In this context, any potential short-term benefits of adaptive instruction seem like a poor trade.

Instead of designing technologies that make choices for students, we have an important opportunity to design technologies that explicitly support students as they learn to make their own choices effectively. Such technologies must respect learner agency, leaving key choices in their hands – even at the risk of learners making some suboptimal choices. (I should say that fully automated recommendations, like “Consider viewing this supplementary video,” fit within this framework to the degree that they respect learner agency.)

Of utmost importance, these new systems must provide learners with simple ways to reflect on the choices they make about their learning and the results of those choices. I believe that providing this kind of feedback, together with opportunities to reflect on what it means, will be a hallmark of future educational technologies that support radical improvements in learning.

As we work to move entire degree programs from commercial textbooks to open educational resources, we must answer this critical question – can all commercial materials be replaced with open educational resources? The answer to this question is no, but perhaps not for the reason you suspect.

The primary object of study in a college course is often a model, principle, theory, equation, causal relationship, or other idea. While one particular way of explaining an idea can be copyrighted, an idea itself cannot be copyrighted. This feature of copyright law means that, despite how anyone else chooses to leverage their copyright in an explanation of the second law of thermodynamics or the cause of the Civil War, you and I are forever free to create our own explanation of these and other ideas. When we exercise that right and explain an idea in our own words (and perhaps other media), we then hold the copyright to our explanations of those ideas. And as the copyright holders we are free to openly license our explanations, thereby creating OER alternatives to the All Rights Reserved explanations published and controlled by commercial publishers.

Sometimes, however, the object of study is not an idea, but is a specific creative work like a painting by Pollock or a score by Shostakovich. Unlike ideas, creative works are copyrightable. And no matter how much we want to save students money and increase pedagogical flexibility by using OER, it simply isn’t possible for you or I to create an openly licensed equivalent of Catcher in the Rye. This impossibility can also arise when we want students to read a seminal research article rather than someone else’s summary of its key ideas and findings.

In summary, in circumstances where (1) the primary object of study is not an idea, but is a specific creative work which is still under copyright and (2) the copyright holder has chosen not to publish the work under an open license, it is literally impossible to replace all the commercial content in that course with OER.

Whenever it is impossible to replace commercial materials with OER for these reasons, we find ourselves in a situation where commercial materials must be used in the course. If openly licensed materials cannot be used, is there perhaps a way to have students use commercial materials for free (legally, of course)? Asking this question can lead directly to a fruitful collaboration with your campus library. The library may be able to purchase or license copies of these commercial materials and make them available to students for free. In fact, they may have already purchased or licensed copies which are just waiting to be used. The library is a trusted, capable, and unfortunately often overlooked potential partner for closing the access gap to commercial materials. (Your library may also be curating OER you don’t know about – libraries are actually leading the charge toward OER on some campuses.)

Why Use OER at All – Why Not Just Use Library Resources?

Recognizing that the library may already have purchased or licensed copies of materials that could be provided to students for free raises the question: why not just use library resources instead of making the move to OER? The answer to this question combines thinking about cost and permissions.

1. Commercial textbooks are (a) extremely expensive and (b) are published under an All Rights Reserved model that restricts what faculty and students can do with them.
2. Library resources are (a) available to students for free (though they likely pay a fee which supports the library’s acquisitions budget), but (b) are also published under an All Rights Reserved model that restricts what faculty and students can do with them.
3. Open educational resources are (a) available to students for free (though they may pay a technology support fee), and (b) are available to students and faculty under open licenses which provide them perpetual, irrevocable permissions to retain, reuse, revise, remix, and redistribute them.

In tabular form:

From the perspectives of affordability and pedagogical flexibility, when faculty make course materials adoption decisions they should chose OER first, all other things being equal. When it is impossible to choose OER, faculty should choose library resources. Students should be required to individually purchase commercial materials only when OER and library resources are impossible to use.

The Danger Zone: A Third Category Replaceability

Finally, it is important to recognize a middle category between situations where faculty can adopt OER and situations where it is impossible to replace the object of study with an openly licensed equivalent. The three categories of replaceability are:

1. An open replacement exists: OER already exist that can be adopted in place of commercial materials.
2. An open replacement could exist, but doesn’t yet. No adoptable OER exist, but this problem could be solved through the appropriate investment of time and resources.
3. An open replacement is impossible. No OER exist because it is impossible to create an openly licensed replacement of the principal object of study.

While it is critically important for us to recognize that a range of circumstances exist in which an open replacement is impossible, it is equally important that we recognize that “impossible” is not the same as “really hard” or “really expensive.” Situations in which it would be really hard or really expensive to create an OER replacement belong in the “could exist, but doesn’t yet” category. As a community, we can and should be working systematically toward moving items from the “doesn’t exist yet” category into the “an open replacement exists” category. This process begins by making a special effort to resist the temptation to throw our hands up and retreat back to commercial materials when we realize one of our learning outcomes is in the “not yet” zone.

Audrey’s constant refrain that “no one seems to remember our history” was made for her again this week when McGraw-Hill and Microsoft announced a new project based around – I kid you not – learning objects. Reading this news created in me an irresistible urge to join Audrey in reminding the field of its not-at-all-distant and yet already-forgotten history regarding the Reusability Paradox.

Learning objects are meant to be aggregated into a wide range of larger instructional structures. Over a decade ago I worked through the problems implied by this statement in excruciating detail. The tl;dr is that any learning object needs to fit into the aggregation in which you want to reuse it. The degree of fit is purely a function of two contexts: (1) the internal context of a specific learning object and (2) the external context created by the juxtaposition of the other learning objects in the aggregation. Unfortunately, it turns out that the amount of internal context of any learning object is directly correlated with its educational efficacy, while that same amount of internal context is inversely correlated with the number of aggregations the learning object “fits” into. This is the famous Reusability Paradox – the pedagogical effectiveness of a learning object and its potential for reuse are completely at odds with one another. (While the Reusability Paradox was singlehandedly sufficient to quash the realization of the learning objects ambition – I spent over five years of my life on this stuff – there are also numerous other flaws underlying the model as it is traditionally conceived. See sections 2 and 3 in that link, but be warned – it contains Vygotsky, Leontiev, Wertsch, Friere, and M. Night Shyamalan references.)

The Reusability Paradox typically leads designers of learning objects to attempt to “strike a balance” between effectiveness and reusability. This generally results in materials that are neither particularly effective NOR particularly reusable across contexts. No one wants to trade efficacy for reusability (or for lower cost, or for anything else – as the recent Babson survey showed, faculty want proven efficacy more than anything else). And yet we do this all the time without really realizing it. Instead of targeting a specific audience and a specific context, almost all teaching materials adopt their own version of Wikipedia’s Neutral Point of View. Educational materials – and learning objects specifically – try to be just generic enough so as to not be offensive to anyone. They lack what Giant Robot Dinosaur calls a Minimum Viable Personality.

For example, take teaching materials about the Ruby programming language. Here are the first three results that come up for me after searching for Ruby tutorials – Ruby in 20 Minutes, My First Ruby Program, and Ruby Quick Reference Guide. They’re each so bland and generic as to be almost indistinguishable from each other. Contrast these resources with Why’s Poignant Guide to Ruby. You can actually distinguish this resource from the others. It was obviously written with a specific audience in mind – and they love it. But the internal context created in Why’s Poignant Guide – the cartoon foxes, Blix the cat, etc. – are SIMULTANEOUSLY what makes it awesome for a specific audience and what prevents it from being reused more broadly.

So what are we to do? We have three choices – we can either (1) create highly decontextualized resources that can be reused broadly but teach very little, (2) we can build highly contextualized resources that teach effectively in a single setting but are very difficult to reuse elsewhere, or (3) we can shoot for the mediocre middle.

There’s actually a fourth choice. The Reusability Paradox is only a paradox as long as your thinking about educational materials is caught in the ambient copyright trap. “Everyone knows” you’re not allowed to make changes to textbooks, learning objects, videos, and other educational media, and so the learning objects model is built partly in response to that “reality.” But the Reusability Paradox only arises when “reuse” means “reuse exactly as is.” According to this pervasive view, learning objects can never be altered after they’re created – so the author has to make a trade-off between effectiveness and reusability and the rest of us have to live their choice.

The way to escape from the Reusability Paradox is simply by using an open license. If I publish my educational materials using an open license, I can produce something deeply contextualized and highly effective for my local context AND give you permission to revise and remix it until it is equally effective to reuse in your own local context. Poof! The paradox disappears. I’ve produced something with a strong internal context which you have permission to make fit into other external contexts.

This brings us full circle back to the Remix Hypothesis. Learning objects that are published using open licenses – also known as open educational resources – eliminate the Reusability Paradox. However, making something possible is not the same as actually doing it. OER make it possible for us to contextualize our resources and customize our pedagogies to support more effective learning, but they don’t do the work for us. We have to take advantage of the 5R permissions and actually do the work of contextualizing and customizing our open educational resources and open pedagogies. Thus, the Remix Hypothesis states that changes in student outcomes that occur in conjunction with OER adoption will correlate positively with faculty revising and remixing activities.

Which brings me back to the announcement of the McGraw Hill – Microsoft partnership. The press release reads, in part:

A key component of McGraw-Hill Education’s relationship with Microsoft is the ability of educators to develop compound learning objects through Office Mix, a media-rich extension of Microsoft PowerPoint that is free to the education community, and combine them with McGraw-Hill Education content and technology…. Compound learning objects will serve as the basis for all of [McGraw-Hill]’s K-12 products going forward starting next year, with its higher education portfolio soon to follow.

From these and other statements in the release, it sounds like a core goal of the partnership is to get teachers to make their own learning objects in Powerpoint and then upload them into McGraw-Hill’s platform, to be used side-by-side with MH’s own learning object-ized content. This is an “adaptive and analytics” platform, consistent with MH’s goal of “becoming the world’s foremost learning science company.” At the end of the day, this is a technical partnership that focuses on platform – Microsoft’s Office Mix platform and McGraw-Hill’s (unnamed in the release) adaptive and analytics platform.

This emphasis on platform belies a belief that innovations in platform can solve the problems that have beset earlier learning objects initiatives. For 15 years now organization after organization has made the mistake of thinking that the reason past learning objects initiatives have failed is that the platforms supporting revise / remix were too hard for faculty to use (or were broken in some other way). How many times have we heard someone exclaim, “Our breakthrough platform will finally make it possible!”? While ease of use of platform will certainly have a role to play in making a learning objects initiative successful, the platform issue is not the fundamental issue that must be resolved – it is a secondary issue. The Reusability Paradox is the primary issue to overcome, and this can only be done by means of the 5R permissions granted by open licenses. If the Reusability Paradox is not resolved there’s very little need for a platform – no one wants to reuse decontextualized resources that don’t teach effectively, it’s very difficult to reuse highly contextualized resources that do, and there’s no real difference between all the content that shoots for the middle (so there’s no point substituting one learning object for another).

Oh how I wish the field could remember that.

My problems with the phrase “high quality” are two-fold: (1) how the phrase gets equated with a single authoring process to the exclusion of all other authoring processes, and (2) how that usage distracts us from the efficacy conversation.

I do not have any issues with the traditional authoring process itself. As I wrote yesterday, “I fully believe that resources created through the ‘traditional process’ can effectively supporting learning.” Not only do I not have an issue with the traditional process itself, I don’t have an issue with the people or organizations who employ the traditional process – in fact I count some of them as good friends.

To be clear, my first issue is with the way “high quality” is often equated with the traditional process and that process only. According to this usage, if you don’t follow the traditional authoring process it is literally impossible for you to create “high quality” materials. This restrictive usage serves to lock out alternative processes from competing in the marketplace. The usage makes it impossible, for example, for commons-based peer production models to result in “high quality” resources (because “high quality” was previously defined as “adhering to the traditional process”). That’s just plain wrong. We need to create more openness to alternative authoring models in the minds of faculty. The traditional process can create effective learning materials, but so can other processes. These alternative processes likely share some common functions with the traditional process – like quality reviews – but the specifics of how those functions are performed can differ significantly across processes (e.g., focused review by a smaller number of experts versus ongoing review by a much larger number of peers) and still result in effective educational materials.

My second issue is the way this misuse of “high quality” omits any consideration of results, and tends to distract us from the efficacy conversation we need to be having. As this series of posts demonstrates, “high quality” can mean many things. But for the sake of our students, the one thing we most desperately need it to mean is “effective.” It’s quite simple – in order to be considered “high quality” educational materials must be effective. We need to help faculty understand that the phrase “high quality” is often disconnected from measures of efficacy when used by publishers to describe textbooks and other instructional materials, and we need them to help bring effectiveness back into the conversation.

Let me give an example to clarify exactly what I’m calling out and exactly what I’m not.

Take OpenStax as an example. These guys are doing God’s own work, creating a catalog of open textbooks for high enrolling courses using the most open of all the Creative Commons licenses. They seem to use a fairly traditional authoring process, and that’s absolutely fine. Again, the process that any person or organization uses is entirely outside the scope of this critique.

What is absolutely relevant to the critique is this – I’ve never heard OpenStax badmouth OER that were created using other processes solely because they were created using alternative processes. Even though they appear to use a fairly traditional process internally, they don’t try to perpetuate the “high quality if and only if traditional process” myth. As you would expect from leaders in the open educational resources community, they’re open minded about the possibility of other authoring processes resulting in effective materials.

The second point to make in the context of my critique of the way “high quality” gets misused is that OpenStax actually cares about efficacy. In fact right now there’s a huge banner on their homepage inviting people to participate in an efficacy study being conducted by a highly respected third party (the OER Research Hub). OpenStax aren’t trying to distract people from the efficacy conversation; on the contrary, they’re dedicating homepage real estate to trying to get people more involved in the conversation.

I provide the OpenStax example in order to clarify that (1) I’m not criticizing the traditional authoring process itself, and (2) just because an organization uses a more traditional authoring process doesn’t mean that they’re making the mistakes that I think the field at large needs to stop making. My critique of “high quality” is simply that it is being defined far too narrowly – unnecessarily excluding alternate authoring processes and omitting effectiveness. Perhaps rather than needing to stop using the phrase “high quality” we instead need to engage in a campaign to “redefine high quality” so that the phrase emphasizes efficacy and is more accepting of diversity in authoring processes.

Photo by Shira Golding, CC BY NC.

The problem with the phrase “high quality” as used by traditional publishers is that it puts process over outcome. If publishers were basketball players, they would say, “When I shoot free throws, I align my toes with the foul line and square my shoulders to the basket. I slow my breathing and count to 5. I dribble three times, exhale once more, and then shoot, making sure to keep my elbow in and fully extend my arm.” Honestly, who cares? What you really want to know about a basketball player is whether or not he makes his foul shots. You aren’t going to draft him based on his free throw shooting process – you’re going to draft him based on his free throw shooting percentage. If the player you’re vetting shoots underhanded but makes over 90% of his foul shots, you’re going to draft him. The same is true with a salesperson – you don’t care about her sales process, you care about the number of sales she closes. Or with a baseball player – you don’t care about his batting process, you care about his batting average. Or with a network engineer, you don’t care about her specific troubleshooting process, you care about whether your employees can reach the internet or not.

So why, why, why, would we accept a publisher telling us that “high quality” is a function of process and not a function of results? Publishers want “high quality” to mean educational resources that are “written by experts, copyedited by professionals, reviewed by peers, laid out by graphic designers, and provided in multiple formats,” with literally no reference to results. What we need to know is how much do students who use the resources learn? But as a community, faculty largely accept publishers’ claim that “process = high quality” and don’t ask for outcomes or results data as part of our textbook or other materials adoption process.

And this notion is absolutely critical for the field of open education to understand: it is clearly in publishers’ best interest to focus faculty on process rather than outcome. By (1) equating “high quality” with process rather than results, and then (2) creating extremely complex authoring processes they proclaim to be “the industry standard,” publishers are attempting to create a barrier to entry for other would-be creators of educational resources (like many OER authors). “Oh, you can’t afford to replicate our elaborate publication process? That’s too bad, because our process is synonymous with high quality. Ergo, your materials are low quality.” And see? There’s literally no appeal to results in this argument, only slavish devotion to process. It’s a blatant attempt by publishers at keeping fresh competition – including OER – at bay.

In this bizarro world where results don’t matter, resources that produce better learning results than content produced using the traditional process are described as low quality. Huh?!? Encouraging people to talk about results instead of process – encouraging them to avoid nebulous phrases like “high quality” in favor of words like results, outcomes, or efficacy – is about taking back the conversation from publishers and focusing it where it belongs.

Now, I fully believe that resources created through the “traditional process” can effectively supporting learning. But there are two things I don’t believe:

1. That conformance to the traditional process guarantees that every resource created that way will effectively support learning, and
2. That the traditional process is the only process that can result in resources that effectively support learning.

There has to be a recognition by faculty – if not an admission by publishers – that alternate development processes can result in highly effective educational materials. But currently there’s not. It feels a bit like we’re trapped in 2005, still arguing over whether or not the Wikipedia authoring process can create writing as accurate as the Encyclopedia Britannica process. We settled this argument ten years ago. What are we still arguing about?

Postscript. In the comments on my first “Stop Saying High Quality” post, one commenter asked for a concrete example of OER significantly outperforming commercial materials. His comment makes the point of the argument while seeming to completely miss it – “Oh yea? Show us proof we should stop saying high quality!” To oblige the commenter, you can see an example of a college abandoning a Pearson textbook and MyMathLab bundle in favor of OER and the open source MyOpenMath practice system published in Educause Review. (Spoiler alert: Pass rates rise from 48% to 60% from Spring 2011 to Spring 2013.)

After some reflection I think the reason the phrase gets my goat is that “high quality” sounds like it’s dealing with a core issue, while actually dodging the core issue. The phrase is sneaky and deceptive. (Now I don’t mean that the people who were using it were trying to be deceptive; they weren’t. But the phrase itself tends to blind people.) And by “core issue” I mean this – the core issue in determining the quality of any educational resource is the degree to which it supports learning. But confusingly, that’s not what people mean when they say that a textbook or other educational resource is “high quality.”

It’s very easy to demonstrate that “the degree to which it supports learning” is the only characteristic of an educational resource that matters. If an educational resource is written by experts, copyedited by professionals, reviewed by peers, laid out by graphic designers, contains beautiful imagery, and is provided in multiple formats, but fails to support learning, is it appropriate for us to call it “high quality”? No. No, no, no. A thousand times no. Despite this fact, which is intuitively obvious, when people say “high quality” they actually mean all these things (author credentials, review by faculty, copyediting, etc.) except effectiveness. In the world of textbooks and other educational materials, “high quality” describes the authoring and editorial process, and is literally unrelated to whether or not the educational resource supports learning.

In this way, saying “high quality” obscures the issue we should care about most. Instead of letting people and companies off the hook by checking boxes during the pre-publication process, we should care about whether or not a particular resource supports learning for each of our particular students. Seen this way, the true desideratum of educational materials is “effective.” I really don’t care what the pre-publication processes was like as long as my students are learning (unless the process was unethical in some way).

So please – let’s stop saying “high quality.” We don’t want “high quality” educational materials – we want “effective” educational materials. In the future, when you catch yourself saying “high quality,” stop and correct yourself. When you hear others say “high quality,” take that teachable moment to help them understand that the phrase is a ruse. If we can change this one element of the education conversation, we’ll have done something powerful.

(And don’t forget – when materials are so expensive that students can’t afford them, they are perfectly ineffective.)

However, there are other studies of OER adoption that show large positive changes in student outcomes. (For sake of completeness, we are unaware of any studies showing large negative impacts of OER adoption on student outcomes.) Of these studies we can say that OER save students significant amounts of money while actually improving their learning outcomes. Clearly, supporting more learning at lower cost is also a solid “win” for OER.

Seeing two solid wins for OER, and no clear losses, reassures those of us pursuing this line of work that we are on the right track. However, as this irregularity in research results persists – some showing essentially no change in student outcomes while others show large improvements in students outcomes – you have to begin asking yourself, “Self, what is going on here? Why is there such a large difference in the results of many of these studies?” For the past several months I have been pondering this question and speaking to some of the faculty who taught the courses reported in the studies. While I don’t have any research to report on the issue today, I have developed an initial working understanding of the discrepancy. Until I think of something more descriptive I’ll call it “The Remix Hypothesis.”

In it’s simplest form, The Remix Hypothesis states that changes in students outcomes occurring in conjunction with OER adoption correlate positively with faculty remixing activities. Specifically, I hypothesize relationships between (at least) three levels of remix activity by faculty who adopt OER and changes in student outcomes, based on what I’ve seen in the research to date.

Level 0 – Replace
At this level faculty engage in no remixing whatsoever. They simply adopt OER (most often an open textbook) in place of a commercial textbook and preserve other aspects of the course as they taught it previously. I hypothesize no changes in student outcomes when faculty Replace – except possibly in one special case. In the case of students who are particularly financially disadvantaged, where faculty were previously assigning very expensive textbooks, there may be a small positive effect attributable to the increased percentage of students who can access the core instructional materials of the course.

Level 1 – Realign
At this level faculty remix their open course materials. In my work to date, this has most often involved faculty stripping a course’s content down to its bare learning outcomes, and then selecting the OER from multiple sources that they feel will best support student learning of specific course learning outcomes. I hypothesize small to modest positive changes in student outcomes when faculty Realign.

Level 2 – Rethink
At this level faculty remix both course materials and pedagogy. In conjunction with the Realign activities described above, faculty create or select new learning activities and assessments – possibly inviting students to co-create and openly share them – often leveraging the unique pedagogical possibilities provided by the 5R permissions of OER. (This is what I refer to as open pedagogy.) I hypothesize modest to large positive changes in student outcomes when faculty Rethink.

Remixing and revising at the Rethink level will be significantly more effective if faculty make those decisions after gathering, analyzing, and reflecting on empirical data about their courses. Helping them ground these decisions in their own data is extremely important. I’ve personally seen several cases where faculty’s own memories about what is and isn’t working in their own classes are contradicted by data in their own gradebooks. It’s an amazing process to watch faculty struggle to understand the conflict between their intuitions and their data. We need to support faculty in this reflection process every time they teach the course – supporting ongoing, continuous improvement. Rethink is ideally an iterative process by which pedagogy and supporting materials come into increasing harmony, supporting deeper and deeper student learning.

Very roughly, we might say in terms of the 5Rs that at Level 0 faculty take advantage of their permissions to Retain, Reuse, and Redistribute educational materials. At Level 1 they add Remix. At Level 2 they add Revise, and expand from open materials into open pedagogy. Strictly speaking this characterization isn’t completely accurate, but I think it provides a good approximation to get our thinking going.

The three levels build on one another. First, a faculty member decides to Replace her textbook with OER. A logical next step a semester or three later is to Realign, making more sophisticated choices about which specific OER to use to support specific student learning outcomes. Finally, as faculty become more familiar with the benefits the 5Rs provide them as teachers, they may begin to grasp the potential benefits the 5Rs can provide to learners. This will lead them to Rethink their assignments and assessments so that they maximize the learning-related benefits of openness to their students.

As I said above, I don’t have empirical data from a specifically designed study to corroborate The Remix Hypothesis yet, but I hope to either validate or disprove it empirically in the next few years in collaboration with my awesome partners in the Open Education Group.