This was our main training session on the system where we had a trainer from Speedwell onsite for the day to run through all aspects of the system with us, from initial configuration to creating questions and exams. We will also be deploying the Safe Exam Browser as part of this project.
Their quiz tool is Respondus 4, which was described as a legacy product, and it did look old. It was demonstrated running on a Windows 7 machine which is sufficiently old now that when I see Windows 7 I wonder why, does it not work on 10? Despite that, Respondus integrates with a number of VLEs and mirrors the available quiz questions types and settings which are available there. Importing and exporting from text files and Word documents was demonstrated and it seemed to work pretty well, though questions and answers have to be in exactly the right format to be recognised. I’m not sure why we would use this over using the quiz tool directly in Canvas though, and it doesn’t give us something that can replace the EDPAC system.
That comes instead from their LockDown Browser product, the one we were interested in. This allows you to set quizzes that can only be taken through LockDown Browser, a stripped down web browser which only allows access to the VLE and once the quiz begins blocks students from opening any other applications or webpages. I was a little concerned about accessibility as it relies on user’s own screen reading software and blocks certain keyboard shortcuts. Nevertheless, it seems to be popular in UK HE so it can’t be too bad.
And then there was the weird one, Monitor, which they tried to sell alongside LockDown Browser. Monitor is designed to be used for remote invigilation, and does so by recording from students’ webcams. On starting up Monitor students have to take a photo and show their university ID for verification purposes, and then Monitor will record them through the duration of the quiz and flag up any ‘unusual’ practices if detected, e.g. going away from the computer or someone else coming into the picture, which then have to be reviewed by a tutor. Recordings are stored online for up to five years on Amazon’s web services. I didn’t quite get a clear answer on whether or not they have access to a data centre in the UK / EU. Is it just me or does this all sound a bit creepy? I also didn’t get a clear answer on whether or not any UK / EU customers were using Monitor. They bundle 200 free licenses of Monitor with LockDown Browser, so there was a fudged ‘yes’, leaving open the possibility that although institutions have Monitor they aren’t using it. Bizarrely they have a completely different pricing model for LockDown Browser and Monitor, and then there are the technical problems. All of the webcam recording and playback functionality uses Flash which Adobe are finally killing in 2020. I asked about their plans on migrating to another solution and they couldn’t answer that either, saying it was all down to Amazon.
We’ll never get Monitor. I can’t imagine any UK university using it. We may get LockDown Browser. The third system demonstration we’ve had as part of this project is Speedwell, but I missed that one as I had another meeting. Other solutions are also under investigation.
We’re looking at options for a secure eAssessment system that would be able to replace our archaic EDPAC forms, and ExamSoft were the first company to provide a demonstration and discussion for us this morning. For those who have no idea what I’m talking about, EDPAC forms are the old pink sheets that you complete by penciling in a cross in the correct answer box (and it does have to be a pencil of the correct weight too!) Those forms are then scanned by a machine we dub the bacon slicer and then we spend hours correcting all the mistakes and typing the comments manually. Everything about it is awful, and we’ve wanted to get rid for years, but there are pockets of use where people are wedded to this system and won’t switch to using MCQs in the VLE. So it is for them that we are looking for a new solution.
ExamSoft’s big selling point is that it can be used on student’s own devices, computers or iPads, which their software can completely lock down for the duration of the exam. This means that we could still get hundreds of students in one secure location all taking the same exam at the same time, one of the arguments in favour of EDPAC. Otherwise, ExamSoft is a fairly standard MCQ system. Questions can be tagged according to the subject or taxonomy of your choice, it can export and import from most other similar systems, integrates with Canvas, etc. I was a little concerned about what seemed to be the limited number of question types – I didn’t see drop-down or calculated questions for example – and I have doubts about how successful it could be as a bring-your-own-device solution for us.
It’s one thing for students to willingly have and use their personal devices to complement their studies, but if we as an institution require them to provide their own kit in order to take exams we’re opening up issues of responsibility as well as imposing an additional financial burden. If someone is bringing in their laptop and it is broken or stolen on the way for example, is that on us? Our insurance? Then there is the issue of technical support, both with the ExamSoft software itself, and logistical considerations such as ensuring that we have sufficient power sockets for the inevitable dead batteries (we don’t) and that our wireless network is robust enough to handle hundreds of simultaneous connections in a small area (it isn’t). Providing our own equipment via something like a laptop safe could offer a solution to some of these problems.
A webinar demonstration of The University of Nottingham’s eAssessment solution, Rogō, created in-house and published under the GPL open source license. Rogō was developed in response to their dissatisfaction with commercial quiz tools and has evolved into a mature and comprehensive solution with support for over a dozen question types and different kinds of presentation, including self-assessment, summative and survey.
We were all pretty impressed with Rogō and, pending the outcome of our VLE review, it is something that we will look at again if we find ourselves in need of a separate quiz tool.
Being open source software, anyone who is interested can download Rogō from Nottingham’s dedicated website and install it onto their own LAMP server.
It took a while due to some leave and then illness, but today I finally took and passed my CSM exam following my training in February. I scored 30 out of 35, or 85.7%, on my first attempt. The pass mark is 24 out of 35 so I’m pretty happy with that result.