In this blog post, I’m sharing reflections on my first remote teaching experience of a grad seminar course: Technologies to Optimize Human Learning. The course syllabus is here: https://docs.google.com/document/d/1mA5cUk57EvpYvImkPRDRLQS_8ST2Q8ehFvZl88QEDcU/edit?usp=sharing
There are 20 students in the class, with a breakdown of 3 undergraduate, 11 masters, and 6 Ph.D. students. Different from seminars with only doctoral students, I tried adding more structures and hands-on assignments into it.
1. Learning objectives of paper-reading seminar courses
I’ve taken lots of seminar courses that the major learning activities were reading papers. I’ve taken it for granted as the “standard” pedagogy of seminar courses until I’m teaching my own. What are the learning objectives of assigning papers to students to read? What do we want students to get out of reading the papers? Here’s a list of learning objectives I can think of:
a) Know the facts including methods/findings/contributions presented in the paper
b) Be able to critically assess the contributions of the paper
c) Know the structures of academic writing (e.g., problem statement, research questions, related work, methods, results, etc.)
d) Know the style of academic writing (e.g., how to report statistics, how to describe participants’ quotes, etc.)
e) As a step in legitimate peripheral participation, getting to know the authors of the papers and becoming a part of the research community.
f) Disposition-wise: grow interest in the field and may be willing to join research projects afterwards.
g) Be able to make and deliver presentations in public (this is added post-hoc given the students are indeed exercising public speaking.)
Please comment with other learning objectives I’m missing here :)
I listed the above objectives in the order of importance they were to me. I consider #a as the most critical learning objective, since all of the subsequent discussions about the papers, and the critical evaluations of the papers need a basis that the students know the facts presented in the papers.
2. Designing quizzes to support active reading of papers
I then reflect on what activities I give students that would support the learning objective of a) Know the facts including methods/findings/contributions presented in the paper. They were assigned two papers to read before each class meeting [though reading can be passive, students don’t get feedback, and it’s quite often that students do not read the papers], they were required to post a discussion post in the forum before class [still, students don’t get feedback on discussion posts, and the discussion is actually not interactive at all], and during class, one student is responsible for presenting one paper [it’s an active learning experience for the presenter, but for the rest of the class, it’s still passive], the instructor summarizes the key points from the papers [it’s still passive for the class].
I then started designing a quiz for each of the assigned papers to help students engage in active reading experience, and get feedback on the content of the paper. I give students unlimited attempts on the quiz, so that they can see the corrective feedback after each attempt and make multiple attempts.
An example quiz question looks like this:
The hope is that even if student don’t read the paper, they’ll get a basic understanding of what the paper does through the quiz, so that they can engage in in-class discussions about the paper. The quiz serves as an instructional activity rather than an assessment activity.
3. What worked/What I’ll continue doing
Quizzes/active reading
This was discussed in detail in the previous section, and I think is especially important in remote learning settings. When we’re ~2/3 into the class, I can tell that the students are exhausted, and I’m also exhausted. On the one hand, it’s hard to gather student attention in class, on the other hand, when students don’t have their cameras on, I don’t get any feedback on whether my lecture notes get across. I begin to realize, if there’s something I really want the students to know, I’d better design a quiz question out of it, just emphasizing in lectures doesn’t work. The goal is not to test students (since they have unlimited attempts), rather the goal is to give them an opportunity to actively process the information I want to convey.
Intermediate checkup points on final projects
I had one project checkup point during the semester, where students were asked to submit a 1–2 page summary on their final project ideas. If they have multiple ideas, they can describe them and I’ll give feedback. While this is hard to scale, and then not applicable for bigger classes. For my class, I found it very helpful for me to provide guidance on student projects (at an early stage) so that they don’t spend more efforts on an unpromising direction. And they were also given a chance to incorporate the feedback. Having more of these checkup points and having them earlier could be more helpful.
Hands-on sessions/worked examples for assignments
For the second and third assignments (build a cognitive tutor, and develop a coding manual), we had a hands-on session where students start working on the assignment following some instructions/tutorials. The students liked having this scaffold and a small worked example before they get to their assignments. Again, creating the tutorials and assignments for these hands-on sessions is time consuming. We need better technologies/AI to facilitate this instructional design process.
Role-playing paper reading
I came across this article on Twitter and tried this method in my class. It worked pretty well, the students were more likely to contribute based on their assigned roles compared to a completely open-ended discussion.
4. What and how I’ll improve next time
More polling in class
I often release the reading quizzes at the beginning of the class meeting, and give students 10 minutes to work on them. They have unlimited attempts and can work on the quizzes afterwards as well. I’ll try next time to space the quiz questions in the lectures and do more polling during lectures to promote active learning. For the last two class sessions, I released the quizzes two days earlier, so that students can choose to do the quiz instead of reading the whole paper. I’ll make this an option for future classes as well.
Reduce student presentation time/Provide more guidance on presentations
For each class meeting, there are two students each presenting one paper and leading the discussion. The benefit of this approach is that students get to exercise public speaking and presentation skills, and for the presenting student they’ll get an in-depth understanding of the paper. However, I also realize as the class proceeds that some of these student-led presentations were not highlighting the most critical arguments in the paper, sometimes they were long and may not be good instructional activities for the rest of the class. These are another form of passive lecturing (led by students). The only benefit is perhaps it saves my time in class preparation as I crowdsource lectures :D. In the future, I may consider providing more structures/scaffolds for student-led presentations, e.g., giving question prompts, templates, and strictly control the time on presentations, to minimize the duration of passive lecturing.
More wrap-up/summary sessions
I had a few wrap-up sessions at the end of each module, many students said they like the wrap-up sessions in which we summarize the papers and discussions we had in the previous 3 or 4 weeks. There is this trade-off between breadth and depth, on the one hand, I wanted to cover as many topics as possible, and on the other hand, I wanted the students to be able to internalize the important messages from some topics. We had the first two modules focused on depth (intelligent tutoring systems, and conversational agents) which span several weeks. And for the rest of the class, we covered one topic each class meeting. The next time, I’ll cut some content off, and insert more wrap-up/summary sessions to help the students connect the different topics we discussed and synthesize them. Instead of treating each paper as a discrete project, these synthesis sessions may help students build a concept map about the literature in the field of learning technologies.
Summary
One major reflection is that although I’m well aware that the transmission model is wrong, it’s hard to completely avoid the mindset. The transmission model refers to when teachers are making instructional designs/choices, we tend to think about what we want to cover in classes (e.g., I really need to includes these in my slides so that the students will know), though including something in slides never means students will learn anything. The correct way is to think about what students can learn from the activities we design (e.g., students need to do this to be able to learn how cognitive tutors work.). The transmission mindset often comes when there’s time constraint in teaching preparation or for reasons such as teaching isn’t the highest priority in our job. Thinking about what I want to cover is easier, whereas thinking about how students will learn the best is much more challenging. Even when we know active learning is the way to go, there are practical constraints on implementing them. There is much work to do in the space of learning technologies, especially AI-augmented instructional design to make the process of instructional decision-making, design and content authoring more accurate, efficient, scalable and can actually result in better learning outcomes.
