Category: Quantified Data

{This post continues my series of reflections on quantified student feedback regarding the use of digital pedagogy in the classroom. Be sure to check out Part 1 and Part 2, as well.}

My previous posts on digital pedagogy tended to emphasize the helpfulness of resources drawn from the digital humanities in lower-level undergraduate coursework. And while it might seem more obvious to deploy such resources in a large undergraduate survey, I’m confident that digital tools can still help us get the job done in senior-level courses, as well.

Examples of such resources that reward more sustained engagement would include:

(1) Podcasts: While podcasts might strike some as primarily entertainment media, there are a number of series out there that aim to bridge any supposed gap between the ivory tower and the public square. One of the best, in my view, is Peter Adamson’s History of Philosophy Without Any Gaps podcast, which breaks much of ancient and medieval intellectual history down into bite-size, twenty-minute chunks. Adamson manages to capture the best of both worlds, crafting episodes that are as listenable as they are edifying.

(2) Apps: One of the most labour-intensive yet rewarding applications in the field of the Digital Humanities is the creation of apps tailored to certain historical figures or texts. Villanova University has recently developed (and continues to develop) an app devoted to the Confessions of Augustine of Hippo, which happens to be a text near and dear to my own heart. Having used this app in a classroom setting already, I can attest to its efficacy. It really succeeds in its mission of making Augustine’s masterwork more accessible for students living about sixteen-hundred years after the Saint’s death.

In the end, it’s all about calibration. It’s all about fitting the amount and usage of digital resources to the level of the course, as well as to its theme and desired learning outcomes. If those three factors are kept in mind, digital pedagogy can work at almost any stage of undergraduate education.

To back up my claim that digital pedagogy can work at higher levels of undergraduate learning as well, please see the following data drawn from a fourth-year seminar on theories of time in antiquity. This course included 13 students, all of whom decided to participate in the feedback questionnaire. Again for reference purposes, here are the questions to which the charts supply quantified responses:

1. Overall, I found completing the digital humanities assignments to be:
   1. Educational: 13/13 = 100%
   2. Confusing: 0/13 = 0%
   3. Pointless: 0/13 = 0%
   4. None of the Above: 0/13 = 0%
2. Before the digital humanities assignments, my sense of the digital humanities was:
   1. Quite Good, Actually: 1/13 = 8%
   2. Limited & Incomplete: 10/13 = 77%
   3. Non-Existent: 2/13 = 15%
3. These assignments improved my grasp of what “digital humanities” means:
   1. True: 12/13 = 92%
   2. False: 1/13 = 8%
4. The difficulty-level of the digital humanities assignments was:
   1. Too Hard: 1/13 = 8%
   2. Too Easy: 1/13 = 8%
   3. Just Right: 11/13 = 84%
5. If these kinds of assignments required deeper engagement with digital resources, that would:
   1. Be More Fun: 4/13 = 31%
   2. Be Too Burdensome: 4/13 = 31%
   3. Not Change Anything: 5/13 = 38%
6. What is your attitude about the idea of the “digital humanities” generally?
   1. It Was a Refreshing Change: 8/13 = 62%
   2. It Sounded Easier But I Didn’t Really Get Much Out of It: 2/13 = 15%
   3. I Would Prefer to Just Write a One-Page Reading Response: 3/13 = 23%
7. Which digital humanities assignment did you find most useful?
   8. Podcast Review: 8.5/13 = 65%
   9. App Reflection: 4.5/13 = 35%
   10. Neither Were Useful: 0/13 = 0%
8. How did listening to a podcast episode compare to an in-class lecture on the same material?
   1. Listening to a Podcast is Preferable: 1/13 = 8%
   2. Listening to an In-Class Lecture is Preferable: 4.5/13 = 35%
   3. Both Are About Equally Effective: 7.5/13 = 57%
9. What did access to the Confessions app change about your approach to the text itself?
   1. The App’s Resources Helped Me Understand Augustine Better: 10/13 = 77%
   2. App Was Cool But Didn’t Help Me Understand Augustine Better: 3/13 = 23%
   3. The App In No Way Aided My Understanding of Augustine: 0/13 = 0%

And because, once again, numbers never tell the whole story, here’s a selection of student comments regarding the use of digital resources in History 476:

If you could pass any advice on to Dr. Peter Adamson, proprietor of the History of Philosophy podcast, what would it be?

• “If possible. I would love to get an outline or actual script of his podcast, since I usually have to listen to the podcast multiple times (which is a good thing). But I still hope there is something I can actually read.”
• “Make his voice less soothing.”
• “I enjoyed the emphasis on the history aspect, but there wasn’t quite as much info on the actual philosophy I was trying to focus on.”
• “Make it more user-friendly to find subjects. Have a stronger search engine.”
• “I found the podcast to be very enlightening. Having some more in-depth discussions and info would be great!”
• “Keep doing what you’re doing! I loved it.”
• “Keep doing what you’re doing. I found the podcast to be both interesting and useful.”
• “Make them longer, since some sections seem too oriented toward surveying topics that merit more depth.”
• “Subtitles, please! Sometimes I cannot hear the words clearly, especially during the interview episodes.”

If you could pass any advice on to the team at Villanova University that made the Confessions app, what would it be?

• “It helped my understanding of the text, but the app needs a little work.”
• “Break up the long book audio clip segments.”
• “Emphasize external sources less.”
• “Involve more reflections and commentaries (possibly longer ones).”
• “The commentaries in the app gave me just a taste, but not enough.”
• “I found the app to be very useful and didn’t have many problems with it.”
• “As somebody not entirely familiar with sections of the Bible, abbreviated citations were initially confusing.”
• “I learned to make a pretty good cake from it!”
• “The app is well made and helped me quite a bit. I would suggest having discussion boards where app users can have discussions with the app creators.”
• “Work out the small bugs to make the whole process smoother.”
• “Great job! Thank you for the hard work. My only advice has to do with the audio format. A different interface might be easier. Other than that, it’s fantastic.”
• “The app was useful, but also expensive. It needs work.”
• “The difference between our translation [Hammond] and the app’s translation was too big. I understand why, but it made it hard to use in conjunction with our text.”
• “There should be a better way of finding quotes.”
• “Keep doing what you’re doing. I found the app to be very helpful.”
• “Reformatting the footnotes would be appreciated, as would an index, specifically for the interpretive articles.”
• “Make it easier to search for keywords while reading through the chapters.”Do you have any ideas about other digital resources that you’d like to use when completing assignments in a Humanities context?

Is there anything else you’d like to recommend to further the work that the digital humanities can accomplish in the classroom?

• “Visual clips.”
• “No, but I do think it’s a good idea to do assignments like this. It was very interesting to work with podcasts.”
• “More podcasts would be helpful, not as an assignment but as an ‘extra’ for topics in class.”
• “I personally use YouTube when I don’t understand a concept (in all of my courses). This may be useful to other students if credible videos are available.”
• “I would like the addition of more audio books.”
• “Creating a podcast or other digital project such as a wiki or website would be fun.”
• “Video resources would be cool.”
• “Perhaps some form of video content or supplementary lecture material from other speakers or authorities or YouTube or online lectures.”

In a rather pleasant turn of events, the powers that be at Humanities Commons have seen fit to feature my post on quantified student feedback to digital pedagogy!

You can find it on the Humanities Commons homepage, alongside some really excellent exemplars of what this platform is capable of, not least of which is Dr. Meredith Warren’s site (one of the best I’ve seen on here).

Hopefully we can all keep the momentum up as the summer carries on…

Following up on my earlier post on quantifying student feedback in the digital humanities, here are some rough-and-ready charts that aim to visualize some of the feedback data I’ve collected regarding the efficacy of digital pedagogy in the classroom. In general, it tends to demonstrate that the properly calibrated use of digital resources can see real results in a humanities setting.

This dataset communicates feedback findings drawn from a roughly 40-student session of History 101 (“Foundations of the Modern World up to 1500 CE:” a modest topic!).  22 of the 40 students responded to the questionnaire. For reference, here are the seven questions to which the charts supply quantified responses:

1. Overall, I found completing the digital assignments to be:
   1. Educational: 22/22 = 100%
   2. Confusing: 0/0 = 0%
   3. Pointless: 0/0 = 0%
   4. None of the Above: 0/0 = 0%
2. Before the digital assignments, my sense of the digital humanities was:
   1. Quite Good, Actually: 5/22 = 23%
   2. Limited & Incomplete: 8/22 = 36%
   3. Non-Existent: 9/22 = 41%
3. These assignments improved my grasp of what “digital humanities” means:
   1. True: 22/22 = 100%
   2. False: 0/22 = 0%
4. The difficulty-level of the digital humanities assignments was:
   1. Too Hard: 0/22 = 0%
   2. Too Easy: 8/22 = 36%
   3. Just Right: 14/22 = 64%
5. If these kinds of assignments required deeper engagement with digital resources, that would:
   1. Be More Fun: 12/22 = 54%
   2. Be Too Burdensome: 5/22 = 23%
   3. Not Change Anything: 5/22 = 23%
6. What is your attitude about the idea of “digital assignments” generally?
   19. It Was a Refreshing Change: 19.5/22 = 88%
   20. It Sounded Easier But I Didn’t Really Get Much Out of It: 5/22 = 7%
   21. I Would Prefer to Just Write a One-Page Reading Response: 1/22 = 5%
7. Which digital assignment did you find most useful?
   4. Digital Mapping (DA #1): 4.5/22 = 20%
   5. Timeline Creation (DA #2): 5/22 = 71%
   6. Textual Analysis (DA #3): 2/22= 9%
   7. None Were Useful: 0/0 = 0%

Because numbers never tell the whole story, here’s a selection of student comments regarding the use of digital resources in History 101:

• “No a million times over to short written assignments!”
• “Timeline creation was most useful, followed by digital mapping and then textual analysis.”
• “DH does make history a little more engaging.”
• “It would be interesting to see other types of digital assignments, as well.”
• “Some assignments were a bit confusing, but after a bit of time it was fine.”
• “I think DH helps us to understand history much more easily.”
• “I think it would be helpful to have a little more direction in where to find historical sources.”
• “I enjoyed the assignments. However, it would be nice to have one written assignment. It would help with greater understanding.”
• “I just really enjoyed the textual analysis! It was interesting. J”
• “It was interesting in the sense that it enlightened me to a subsection of history I have not heard about before. However, the assignments were very easy, specifically the textual analysis. The map and timeline were good.”
• “The digital assignments were educational to a certain extent.”
• “I do like reading responses as a way to explore the material. However, the digital assignments were very beneficial and enjoyable. I use mapping and timelines a lot for studying, so this really helps me expand my toolbox.”
• “The mapping and timeline assignments were helpful for understanding how the things we were studying related to each other. The textual analysis assignment seemed less helpful for that purpose, and it didn’t seem to relate.”
• “The timeline and mapping assignments helped cement my knowledge of the material. However, the textual analysis really didn’t help with my understanding even though it was interesting.”
• “I felt that if the assignments were a little more comprehensive I would’ve enjoyed them more. Since I thought they were super-easy, I sort of blew them off and then did the bare minimum at the end. I would’ve liked it to be harder. I loved them though!”
• “I loved the mixture of assignments in this class. They did seem a bit easy or a lot of it was plug and play. But it was nice not having to write sixteen billion papers for this class.”
• “Thanks for teaching this class! You made it fun and enjoyable. Keep up the good work.”
• “I think you should drop Voyant [textual analysis tool], because it is not educational and doesn’t improve understanding of the course material.”
• “I thought the digital assignments were interesting. I enjoyed them and I did learn some things. They helped me remember things about historical events and places.”
• “The mapping and timeline assignments were useful. They put things into perspective. I didn’t get much out of the textual analysis assignment.”
• “I think the mapping and timeline assignments were very helpful for the course. They helped me study and visualize. The textual analysis assignment, meanwhile, didn’t do much to increase knowledge of the course. It was interesting to do, just did not have the educational aspect that the mapping and timeline assignments did.”
• “I would have liked to do maybe one or two more different ones for a little more engagement.”

For now, this remains mostly a snapshot of raw data, with a few bare-bones visualizations thrown in. But, when these numbers are combined with comparable datasets from other courses, I’m cautiously optimistic that more sophisticated and usable conclusions will result. Stay tuned for Part 3!

One of the first questions that comes to mind when one hears the troublesome phrase “Digital Humanities” is an exceedingly basic one: “What does Digital Humanities *mean,* anyway?”

Sometimes, this question is asked in a spirit of honest humility and with a desire to learn more: “What does Digital Humanities mean? What kinds of tools and objects and practices count as DH? What can I learn from DH methods?”

At other times, the question seems to be asked with a bit more of an edge to it: “What even *is* DH, anyway? Does the term mean anything at all? Are we simply dealing with empty buzzwords here? What ever happened to good ol’-fashioned book-learnin’?” (And so on…)

In my experience, most students tend to pose the question in the former sort of way, while some of my fellow instructors prefer to opt for the latter. This is not a hard and fast rule, of course. There are a few students who could be described as hardline anti-DH-ers, while we all have a number of colleagues who either hold a genuine interest in DH or are actively engaged in doing DH work. But, in my admittedly anecdotal experience, it does seem like students–especially younger undergraduates–can be more open to exploring the potential of what DH could mean for them.

If we want to provide straightforward, concrete examples of what great DH work can look like, we could simply point toward huge collaborative efforts like Orbis and Pleiades, or toward specific scholars who have established themselves as preeminent pioneers of DH in their respective fields. In the world of classics and late antiquity, for example, I tend to point students in the direction of scholars like Dr. Sarah Bond or Dr. Jennifer Barry, both of whom run DH workshops for the North American Patristics Society.

When it comes to my meagre corner of the academic world, however, I usually focus in more closely on Digital Pedagogy. Here the operative question becomes: how can we best go about using digital resources to augment students’ own attempts to engage with the historical cultures they’re studying? (In my case, once again, the most relevant historical epoch is antiquity, although I also teach courses in medieval and early modern history.)

If that’s our operative question, the next step should be to ask students themselves what they think about the digital tools we use in class. It’s one thing to experiment with digital pedagogies in a course; it’s another to gather instructor-side data about new assignment-models; but it’s yet another thing to actually go out there and listen to what students have to say about the whole experiment in all of its unwieldy facets.

To that end, I’ve spent the first year of my position here at MacEwan compiling student-side feedback about a wide range of digitally tinged assignments. In a first-year world-history survey, we talked timelines and textual analysis; in a second-year medieval Europe course, we made multiple mock-up maps of the Mediterranean world in the Middle Ages; and in a fourth-year seminar, we worked to incorporate both podcasts and apps into our discussion of some challenging material from antiquity. In every case, students voluntarily submitted feedback info that I was then able to preserve as data and, in many cases, visualize for popular consumption.

Part of my goal with this site is to get some of my data on digital pedagogy out there, so that from its rudimentary roots something more substantive might grow. There are any number of other scholars and instructors out there who have delved deeper into these issues than have I, but I’m hopeful that offering up my feedback as open data will be a not-entirely-pointless exercise.

To that end: in future posts, I plan to share some of the data I’ve collected, as well as the contextual information needed to make sense of the raw numbers. As I continue to experiment with digital pedagogy in the months and years to come, I’ll aim to tweak my approach both to assignment-design and data-collection, so that a fuller picture of the capability of these new resources will begin to come into view.