Online formative assessment coupled with synchronous online learning: Insight from an Indian medical college
Objectives:During the coronavirus disease-19 pandemic, majority of the institutions have started distance education. Assessments are also being conducted online. Our question was about the interest of students in assessing their classroom learning by an online quiz. The aim of this study was to observe students’ participation pattern in online anonymous formative assessment immediately after synchronous 1-h online class.
Material and Methods:We designed online quizzes with five questions related to the preceding class. In the last quarter of the 1-h class, we shared the quiz with the students. A total of 20 such classes were conducted. We recorded anonymous data on attendance, participation, time of participation, and obtained marks. The data were expressed in mean, standard deviation (SD), and percentage. Chi-square test, t-test, and ANOVA were used according to the data.
Results:Among 100 1st-year medical students, average attendance in online classes (62.1 ± 13.5) was lower than the face-to-face 1-h lecture class (80.35 ± 13.01, t-test P < 0.001). Average 55.48% (34.45 ± 7.13) of the attendee participated in the online formative assessment. Approximately, students took 4¼ min to answer the online quiz (minimum 45, median 204, maximum 988, mean 255.76, and SD 154.96 sec). The quiz score was high among the students with 46.73% of the quiz participants scoring full marks.
Conclusion:Nearly half of the students attending online classes opted for an anonymous, optional, and online are the characteristics of the self-assessment. The online quiz is a quick method of formative assessment requiring only few minutes. Further, research should be conducted to find ways to increase participation among the students.
On March 11, 2020, the World Health Organization declared the coronavirus disease-2019 (COVID-19) a global pandemic. The whole world is searching for an effective treatment for the disease, and the protocols are evolving with continuous research finding. To limit the disease, social distancing and nation-wide lockdown are being followed in India.[2-4] The maintenance of at least six-feet distances among students in medical colleges is practically difficult. Hence, medical institutions across the globe have closed for face-to-face sessions of teaching and learning activity. The institutions have no choice but to gradually adopt distance educational tools.[6-8] As the classes are being conducted online, the assessment is also being conducted online. However, at this point in time, many institutions are conducting only the compulsory assessments (e.g., summative assessment at the end of a semester that is counted as an internal assessment and is necessary for university examination). However, formative assessment is a great tool, and online formative assessment is also a promising method of self-assessment that may enhance learning.
Online formative assessment has the advantage of providing feedback immediately after the test. The answer keys and explanation of the answer can be shown with the result of the test. Moreover, this feedback may help the students to improve themselves in the summative assessment. Poor performer in the formative assessment should take timely measures to improve themselves as the online quizzes are a predictor for summative assessment.[10,11]
Although the online assessment was started a long back, its application in Indian medical institutions is relatively new. No previous study has ascertained the pattern of participation of medical students in an online formative assessment coupled with online classes. With this background, the aim of this study was to observe the pattern of participation of 1st-year medical students in a voluntary, anonymous, and online formative assessment of learning from 1-h synchronous learning sessions.
MATERIAL AND METHODS
Study type, settings, and ethics
This was an observational study conducted in a government-run medical college in the eastern part of India. We analyzed anonymous data gathered during the conduct of teaching-learning activity. The data were collected during June 05, 2020, to August 24, 2020.
Conduct of online class
We are conducting multiple 1-h online classes with the help of conference/meeting software. For the class to start, the teachers share the meeting identifier before starting of the class in a WhatsApp group of all the students.
Formative assessment quiz
We designed quizzes on Google forms with five questions. The questions were set according to the objectives of the online class. There were two first-order questions, two second/third order questions, and one clinical or image-based question. To make the quiz fully anonymous, we did not collect the email address or any other personal identifiers (e.g., roll numbers or names) of the students. When the form is submitted by a student, the score of the quiz along with feedback is shown for wrong as well as correct answers. The rationality of adding feedback on the correct answer was to reinforce the already gained knowledge.
The aim, benefits, and disadvantages of online formative assessment were described in detail to the students in the very first class (June 05, 2020) coupled with the quiz. The students were informed that participation in the formative assessment is voluntary. They were assured that the quiz is fully anonymous and the teacher cannot access any individual student. We designed the 1-h class so that at least 10 min can be used for the assessment. The weblink of the quiz was shared in the WhatsApp group (where we share the meeting identifier) approximately in the last 10–15 min of the 1-h online class. A total of 20 human physiology classes were conducted with a quiz. Data regarding the initial attendance in the meeting and attendance at the end of the class were noted. This was done as some students may leave the class before the end. The attendance at the end of the class was taken as the final attendance for analysis.
Data and statistical analysis
We obtained fully anonymous data from the Google forms responses. In this sheet, the quiz responses are time stamped. Hence, we can observe the time taken by the students to complete the quiz by comparing the time when we shared the link of the quiz. As the quiz had 5 questions of 1 mark each, the maximum mark was 5, and minimum mark was 0. The distribution of students in different marks was tabulated from the summary on the Google forms. Individual question-wise correct answers were also obtained from the summary. As we conducted 20 classes with various number of attendees, we calculated the average percentage of students for categorizing them according to the score and correct answers. The average percentage from x% of a, y% of b, and z% of c is calculated as:
The data are presented in mean, median, range, standard deviation (SD), and percentage. Binomial test, t-test, ANOVA, and Spearman’s correlation were used according to the data. We used Microsoft Excel® 2010 for database and GraphPad Prism 6.01 for statistical analysis. P < 0.05 was considered statistically significant.
Among the 20 classes conducted for 100 students, attendance at the starting of the class was 69.35 ± 12.33 and attendance at the end of the teaching (i.e., final attendance) was 62.1 ± 13.5. The attendance in online classes was lower than the attendance of face-to-face 1-h lectures in preceding 20 classes [Table 1].
|Participation in online quiz||On time||30.6||7.15||29.5||19–47||<0.0001‡|
SD: Standard deviation. *Offline attendance was of 20 1-h offline lectures conducted before closure of the institution due to COVID-19 pandemic. †Paired t-test between attendance in 20 offline and final attendance of 20 online classes. ‡Binomial test between students answering on time and later
The average participation of attendee in the formative assessment was 34.45 ± 7.13. On average, 55.48% (two-tailed P of Binomial test = 0.37) of the final attendee participated in the formative assessment. The trend of initial attendance, final attendance, and participation in quiz is shown in Figure 1. Spearman’s rank correlation coefficient (rs) between the final attendance and participation was 0.64 (95% CI: 0.26 to 0.85, P = 0.003), which indicates a significant positive correlation between the final attendance and the number of participation. Hence, the participants are not constant and depend on the attendance in that class.
The participants took an average 4¼ minimum to answer the online quiz containing five questions (min 45, median 204, maximum 988, mean 255.76, and SD 154.96 sec; ANOVA P = 0.21) [Figure 2].
On an average, 46.73% students scored full marks followed by 28.01%, 15.97%, 6.68%, 1.74%, and 0.87% students scoring 80%, 60%, 40%, 20%, and 0, respectively. There was statistically significant difference in mark-wise distribution of students (ANOVA P < 0.0001) [Figure 3a].
The score in first-order, second/third-order, and clinical/ image-based questions was similar (ANOVA P = 0.15), as shown in Figure 3b. On average, students correctly answered Q1 84.47%, Q2 82.87%, Q3 79.83%, Q4 72.71%, and Q5 83.16% (Q1 and Q2 were first order, Q3 and Q4 were second/ third order, and Q5 was a clinical/image-based question).
About half of the attendees were interested to participate in the online quiz. We used this quiz as a mode of self-assessment immediately after the online class. As it was created on Google form and the links were shared on the WhatsApp groups, it could be accessed on a smartphone as well as on personal computers (through WhatsApp desktop software application). Despite being an anonymous self-assessment where the students are not judged or compared to other students, the participation was nearly constantly half of the attendee in 20 sessions [Figure 2]. This was our experience from a government-run medical college from the Eastern part of India. The response rate may change across the country and colleges. Although the medical students are interested in online assessments, the “optional” nature of our assessment may not make almost half of the students interested. However, the predictors for participation and non-participation may be explored in future study. Medical teachers may explore the ways to increase participation of students as e-learning and e-assessment would be supplementary teaching-learning activity in near future in India.
Attendance and interest
Till date, medical students are staying home to implement the currently available measure (i.e., social distancing) to limit the spread of coronavirus.[16,17] Attendance in an online class from home is less than the offline classes. Even some of the students leave the online class after joining the meeting. The underlying reason may be the students like a more structured learning environment which is best provided by offline classes. In addition, the attendance is merely a number, and whether students on the other end are watching the class or not is uncertain. Indirectly, the participation in the quiz may indicate that the students were interested in the class. However, that is nearly one-third (30.6±7.15) of the total students (100). The class attendance (face-to-face) is positively correlated with academic performance. If this finding is extended to online class attendance too, medical teachers need to find ways to increase the interest of students.
Students’ scores in the formative assessment were high, with 46.73% securing full marks. Although correct answers to the second/third order questions were lower than other questions, the difference was not statistically significant. The reason behind it may be the participation of limited interested students (i.e., highly adaptive students) who might be attentive in the class and had learned the topic better.
Participation along timeline
We kept a minimum of 10 min for the quiz so that the next class is not hampered from the participation in our subject. We observed that majority of the students participated within the stipulated time. A few students also participated later. However, for assessing oneself on classroom learning, immediate formative assessment with feedback is the most accepted method. The reason behind late participation may be multifactorial. The student may try to read the topic from the book and then wanted to test her/his learning. It may also be the case that the student who did not attend the class later takes part in the quiz.
Although the students took on average 4¼ min to complete the quiz, some students may deviate from the mean. Hence, additional time may be allowed for the online quiz. In the extra time, students get opportunity to think about or read the feedback (i.e., answer key and explanation given along with the result of the quiz). Hence, with our experience, we suggest that a minimum 8–10 min (approximately 15% of 1-h lecture) may be invested for formative assessment.
Limitation of the study
For making the quiz truly anonymous, we did not collect the email address. Hence, if a student submitted multiple responses, it was beyond our detection. However, as the student gets her/his score and answer feedback, and the student knows that it is just for assessing her/his learning and not connected to academic progress, we presume that chances of multiple submissions are very less. This study is based on a pilot project with only 20 classes conducted in a single subject by a teacher.
Online formative assessment can be conducted at the end of an online class when other methods are not feasible. We found that half of the first-year medical students who attend the online classes take part in an anonymous online quiz. Those who participated in the quiz showed a high score in the assessment. In future, research should be conducted to identify the ways to make more students interested in online formative assessment.
Declaration of patient consentPatient’s consent not required as there are no patients in this study.
Financial support and sponsorshipNil.
Conflicts of interestThere are no conflicts of interest.
- Eur J Dent Educ. ;2020COVID-19 is a challenge for dental education-a commentary.
- Sciencing. Available from: https://www.sciencing.com/calculate-averages-percentages-6437936.html [Last accessed on 2020 Sep 09]How to Calculate Averages of Percentages.
- MATERIAL AND METHODS
- Study type, settings, and ethics
- Conduct of online class
- Formative assessment quiz
- Data and statistical analysis
- Attendance and interest
- Participation along timeline
- Time factor
- Limitation of the study