Social Annotation Analysis
Travis Dolence
AI and Social Annotation Software
Artificial intelligence can be used to examine social annotation within Pressbooks by analyzing patterns in student engagement, highlighting key themes or questions that emerge from collaborative comments. This can help educators identify areas of confusion, interest, or critical thinking in real time. By leveraging AI tools alongside social annotation, instructors can gain deeper insights into how learners are interacting with the text and with one another.
Example: Using Copilot to Interpret Hypothesis Social Annotation
Goal
Evaluate Copilot as a tool for analyzing social annotations on Open Educational Resources (OER). Instructors could use Copilot to identify themes, assess tone, or spot areas needing clarification in student annotations of an OER textbook or materials hosted in Pressbooks.
What I did
Hypothesis is a social annotation software that allows users to collaboratively annotate digital texts. This can be especially useful for educational purposes, as it enables students and instructors to highlight, comment on, and discuss specific sections of a text in a shared online space. By integrating social annotations into their study materials, learners can foster deeper engagement and understanding through interactive dialogue and peer feedback.
Hypothesis is integrated into Pressbooks, allowing educators to use it with a wide range of open source textbooks.
Step 1: In this example, two individuals utilized Hypothesis to create a series of sample annotations within an open-source textbook available through the Pressbooks Directory to simulate student responses. This process necessitated the use of the Hypothesis browser extension.
Step 2: We aimed to determine whether a faculty member could utilize Copilot to review these annotations. This was achieved through two methods:
- Using Copilot within the MS Edge browser to view annotations within the Hypothesis browser.
- Export the annotations as .txt and employ Copilot in Word, as well and uploaded into CoPilot 365.
We used chapter one of this book for our example: https://minnstate.pressbooks.pub/appropriation/
Results
There was occasional ambiguity regarding the content being addressed by Copilot. Specifically, it was unclear whether Copilot was referencing the entire text, the quoted section, or the student comment. This issue appeared more prevalent when using the browser integration with Copilot. However, when comments were downloaded as a .txt file, the AI seemed to perform better in this regard. Additionally, instructing the AI to specifically consider comments proved beneficial, reflecting the language used in the downloaded file. The improvement can be observed in the responses to the final prompt asked in each series: “Can you identify any questions or topics in the comments that could or should be expanded upon or clarified by an instructor?”
While using a larger corpus of annotations would yield better results, the advantages of utilizing Copilot for examining annotations are evident, especially when dealing with extensive text over a substantial volume.
Considerations
Potential issues
- Hypothesis requires users to create an account and download a web browser extension. This process works efficiently in Chrome, the browser for which the extension was designed. In the Microsoft browser Edge, this process was also possible but required multiple attempts to function correctly. The issues were mainly related to logging into the extension, but it eventually worked after several attempts.
- During the attempt, Copilot encountered difficulties with the Rich Text HTML file, which appear to have been temporary issues that may be resolved upon retrying at a later time.
- Given the potential amount of student information in comments, consider using an AI product that complies with privacy standards.
Best Tools for the Job
- Comments in AI can be queried using the integrated browser or by downloading an exported file in formats such as txt, rich text HTML, or csv. These files can also be uploaded to Copilot 365 or opened in applications like Word. Initially, it appears that working with a downloaded file using the language of the file structure may be more effective than using the browser. However, further testing is required to confirm this.
About the author
Travis Dolence is the Digital Initiatives Librarian at Minnesota State Moorhead. Travis’s work includes providing liaison services to faculty, collection development, and working on digital initiatives such as the Library’s makerspace and institutional repository.
