by Royce Kimmons, George Veletsianos, and Torrey Trust
Published onNov 26, 2024
Judicious AI Use to Improve Existing OER
·
As the practical value and ethics of using generative artificial intelligence (AI) in education receive focused attention, open educational resources (OER) provide a mechanism toward an AI future that is more bright and hopeful than the current trajectory. The ecological argument against AI particularly highlights existential threats to humanity; as the training and use of these tools may drastically and disproportionately deplete access to water and power and lead to social decision-making that values resources over people (Crawford, 2024). However, such concerns may lose some of their urgency if we shift to more judicious use of these tools (cf., “applied AI” vs. “innovative AI”, Dong et al., 2024). In this paper, we will discuss what we mean by “Judicious AI Use,” ground our discussion in our own experiences and aspirations in developing and growing an open publishing platform (EdTech Books), and explore how open education can serve as a space for more sustainable and equitable uses of AI to achieve socially valuable goals.
The Problem of Infinitely Scaling AI Use
EdTech Books is a free open publishing platform available at https://edtechbooks.org, and in 2018, a book author first shared a book they had made on Facebook. The resulting traffic, however, crashed our server. The reason for this was that accessibility was a key concern for us, and in our initial development, we tried to make our content available in as many formats as possible, including PDFs. However, for PDFs to remain current they must be rebuilt whenever content changes are made to the source HTML. The original solution, we thought, was simple: rather than relying upon a static PDF, the server should rebuild the PDF for each user and deliver a fresh download file for each click. What we failed to take into account, however, was that many PDFs on the site would be over 1,000 pages long with hundreds of images, requiring a strong web server several minutes to generate. With increased usage, the server had to generate 100,000 or more content pages simultaneously. The (now obvious) result was a full system crash and loss of service.
In that moment of crisis, we considered two options: either (a) increase hardware capabilities to meet this astronomical demand or (b) rethink our PDF generation strategy. Though the first option could be theoretically possible given infinite cloud server scalability, the cost of doing so would have been unsustainable. The simple and obvious solution, therefore, was to rebuild PDFs on a periodic schedule, store the result, and serve these static files via inexpensive file sharing services. This solution allowed us to keep PDFs up-to-date while also drastically reducing processing time, power consumption, and cost.
This type of solution is not novel. Imagine the processing power necessary if every time you entered a term like “lightbulb” into Google’s search bar the search engine had to go through over 1 billion sites, scrape their data, parse meaning from them, and build your results on-the-fly. Search engines could not operate on this model, because the amount of time and energy required to do this would simply be untenable. Instead, search engines only actually scrape sites periodically, but they cache data in a variety of ways and at multiple levels of abstraction to facilitate faster indexing and more efficient recovery of stored results.
Conversely, the current explosion of generative AI is in large part attributable to the advent of the chatbot, which operates on an assumption that every interaction with a user is unique and merits full semantic treatment via natural language processing. If you ask ChatGPT to “define lightbulb” and I ask “what is a lightbulb,” it treats each of these queries as if they are wholly unrelated, novel tasks. It then traverses billions of token relationships to provide a definition that does not need a generative solution. As a result, defining “lightbulb” via ChatGPT requires enough energy to run an actual 60-watt lightbulb for 3 minutes—nearly ten-times the resource cost of a Google search (EPRI, 2024) and hundreds of times the cost of accessing a static resource like Wikipedia. For this kind of task, the cost of generative AI is not defensible even once, let alone incurring the cost each time a new user wants the same information. And yet, a recent survey of more than 1,000 teens and young adults found that the most common use of generative AI was simply retrieving information (Common Sense Media, Hopelab, and the Center for Digital Thriving, 2024).
Collectively, we ought to be exploring how to judiciously use power-heavy AI processes in ways that do not unduly waste resources for performing mundane or repetitive tasks. In the realm of OER, we already have access to millions of pages worth of static content that are educationally valuable. Imagine if each of these resources came coupled with lifelike audiobooks, a summary of key points, embedded learning checks, dictionaries of key terms, translations into popular languages, robust accessibility features, and so forth. If created with AI and stored, such supports could be made available as static OER themselves (like the PDF example above), thereby eliminating the cost and environmental impacts of one-off AI processing.
Judicious AI Use
Judicious AI use in open education, then, involves using AI to improve OER in ways that are valuable to multiple people and then caching or storing these improvements for others to benefit from in a low-cost, sustainable, and scalable manner. Such an approach differs from common AI use that allows learners to interact directly with an AI (Figure 1) by mediating learners’ AI interactions through a set of predetermined impactful prompts and then caching responses as OER for future retrieval (Figure 2).
Figure 1: Typical (Non-Judicious) AI Use
Figure 2: Judicious AI Use
This means (a) identifying impactful improvements to existing OER that can be accomplished with AI, (b) using AI to enact and store those improvements in a finite (typically one-time) manner, and (c) efficiently serving those improved materials to learners at scale as OER. For instance, rather than having thousands of learners rely on their own AI tools to generate a summary of a textbook chapter, why not use an AI tool to generate a summary once and provide it to all learners? Instead of having learners use AI to define every term they do not understand, why not define key terms once, store definitions as static content, and embed these as popups? And why not follow this same pattern for translations and other AI use cases? This approach leverages many of the immense possibilities of AI while also capitalizing on the sustainability features of scalable web technologies and the ethical moorings of openness.
Providing cached AI responses as OER is also a more equitable approach than using restrictive or pay-as-you-go AI services layered on top of OER. This method ensures that all learners receive the same educational tools and resources without requiring paid individual subscriptions for features like AI-generated comprehension questions. Additionally, it avoids relying on learners’ varying literacy levels to prompt AI effectively (c.f., Chander & Sunder, 2004); instead, platform developers can create optimized prompts aligned with learning science principles, producing reliable, reusable learning aids accessible to all.
Furthermore, as AI-generated learning supports scale, their per-user cost approaches zero, making this solution highly sustainable and globally accessible. For example, translating a chapter via AI might cost 1 USD initially, but as more learners access it, the individual cost drops to a fraction of a cent. This impressive ROI makes it a compelling investment for governments and philanthropists supporting OER, as each dollar spent benefits an increasing number of learners at minimal additional cost, providing nearly an infinite ROI.
Learner interaction data can also be used to measure the impact of these low-cost AI-generated supports on learning, providing an ROI metric to guide scaling decisions. For example, if AI adds key points to chapters in an open book, tracking learner outcomes can show whether this improves learning, supporting similar efforts across resources. Similarly, usage data from AI-generated text-to-speech articles can inform whether to extend this feature to other materials. Testing of such features could also be automated using modern approaches, such as split- or A/B-testing (Kimmons, 2021), which would allow us to experimentally study the effects of minor improvements and supplements in situ for informing overall design efforts.
Judicious AI in Practice: Examples in Our Work
In our own EdTech Books platform, we are engaged in an ongoing process of enacting judicious AI use with highly-trafficked content while also attempting to determine the relative benefit (and therefore justifiability) of each. Some examples of active, in-progress, and future uses that we find promising include the following:
High-Quality Text-to-Speech (Active)
Using deep learning models via Coqui TTS, users can request audio for book chapters or journal articles. If the audio exists, the user’s device retrieves it from Amazon S3; if not, it is added to the generation queue, and the user is notified. This ensures only requested audio is generated while efficiently serving existing files to all interested users.
Glossaries (Active)
Content authors can use Llama3 to generate definitions for terms, which are stored as assets for review and adjustment. Authors can then reference these definitions via pop-ups. In future versions, readers will also be able to access definitions of common terms from a static dictionary and request new ones, which authors can approve for Llama3 to define.
Peer Review (In Progress)
Peer review ensures scholarly quality, but maintaining it in recent years has become challenging due to factors like heavy faculty workloads, adjunctification of the professoriate, and more electronic outlets (Flaherty, 2022). To aid editors in creating quality OER and assessing content readiness for review, we are testing a custom GPT (built on ChatGPT 4o-mini) that uses the venue’s scope and aims along with a provided rubric and direct instructions to conduct a pre-peer-review of the open content. Early results indicate that AI-generated reviews match or exceed human reviews in quality.
Abstracts, Summaries, and Keywords (In Progress)
Abstracts, summaries, and keywords are important for search engine optimization and to assist readability, but authors often do not create these simple aids themselves. We are in the process of building pipelines via Llama3 to automatically generate such supporting content of all chapters and articles, which are then stored as assets to the content item for the reader to access. The author will then have the ability to verify and edit generated text to ensure accuracy.
Translations (Future)
We plan to use Llama3 for translation similarly to our text-to-speech feature. Users can select from supported languages, including English, Spanish, French, German, Italian, Portuguese, Dutch, Russian, Chinese, Japanese, and Korean. If a translation exists, it is served from the database; if not, it enters the generation queue, and the user is notified. Content authors can then edit translations for accuracy.
Textual Complexity Leveling (Future)
We plan to use Llama3 for readability leveling, similar to our text-to-speech and translation features. Users will see the content’s Flesch-Kincaid reading ease score and can select an alternate level (“Primary (K-6),” “Secondary (7-12),” “Post-Secondary,” or “Original”). If a simplified version exists, it will be served from the database; otherwise, the request enters a generation queue, and the author can edit it for accuracy once generated.
Learning Checks (Future)
We plan to use Llama3 to scan chapter contents, identify key points, and insert multiple-choice learning checks to assess comprehension. Authors can edit these checks for accuracy and view results in a dashboard, allowing evaluation without ongoing AI reliance.
Accessibility Improvements (Future)
We plan to use an appropriate AI tool to automatically scan through the content of every chapter and to provide accessibility improvement suggestions to the author, such as alternative attributes on image tags, language leveling, etc.
Presentation Materials (Future)
We also plan to use an appropriate AI tool to automatically generate Google Slides presentations of content for any article or chapter, identifying key points, quotes, and illustrations.
Requirements and Limitations
Making such judicious AI uses practicable requires some significant forethought and technical development. These are not trivial considerations, and though relying on learners to use AI individually may reduce demands on OER developers and creators, doing so is inequitable and environmentally unsustainable due to duplicated efforts and overreliance on generative AI processes when they are not needed. To be judicious, developers first need to identify potentially impactful AI uses that are useful to large groups of learners, such as speakers of a language other than the OER’s original language. And second, developers need to build intermediary web-based systems to manage the logic, storage, delivery, and prompting of caching systems as they interface with appropriate AI tools.
Additionally, an emphasis on judicious AI use precludes the pursuit of AI to do some things, such as providing fully individualized learning experiences. However, judicious use can still provide valuable differentiation without being fully individualized. For instance, if a 7th grader accesses a textbook, a judicious approach might only provide a 6-8th grade option in an attempt to only have four different versions (K-5, 6-8, 9-12, 13-16) rather than 17 or more (one each for K-16 grades). In such cases, however, we should consider how essential it is to have a 6th grade, 7th grade, and 8th grade version of the same content when a single version to meet all three reading levels might be reasonable, and coming to terms with the reality that ecological resources are finite requires consideration of how much granular individualization (and therefore AI generation) is actually needed.
This approach also enables a unique, human-controlled quality assurance mechanism to AI use by allowing the OER creator to revise and refine AI responses through managing, correcting, and improving the cached content. This helps address various problems of AI use, such as hallucinations, errors, and misinformation. For instance, though an AI might define the term “phenomenology” for a learner reading an education research book, the creator of the resource would be able to refine the definition for accuracy and contextual alignment to the field of study, thereby using the AI as a starting point and leveraging the expertise of the human creator as a refiner and quality-checker of the resulting OER cache.
Concluding Thoughts
Judiciously using AI to improve existing OER provides a more sustainable and equitable path forward than typical, one-off uses of generative AI by both reducing the ecological dangers associated with these tools and directly promoting equitable social good through AI-generated learning supports as editable OER themselves. This approach also breaks away from technocentric and hyperbolic views of AI that treat it as a panacea for all of our educational problems (e.g., “every learning experience can be improved with a chatbot”) to provide a more realistic and tenable use of AI in education that is complementary to our recognized ways of doing things. Such an approach allows us to aggregate marginal gains from a variety of improvements to address the complexities of learner needs, to realize meaningful differentiation, and to provide higher-quality learning experiences to all.
References
Chander, A., & Sunder, M. (2004). The romance of the public domain. California Law Review, 92(5), 1331-1373.
Dong, M., Wang, G., & Han, X. (2024). Impacts of artificial intelligence on carbon emissions in China: in terms of artificial intelligence type and regional differences. Sustainable Cities and Society, 113, 105682.024
Creative Commons Attribution 4.0 International License (CC-BY 4.0)
