Press "Enter" to skip to content

Tag: Evaluation

Should I Be Researching?

Published 19th May 2025

An excellent question, posed by the HeLF folks, to which the only possible answer is a resounding ‘yes’. But that would make for a very short webinar, so we discussed the issues around this too. Obviously a very interesting session for me, as I have been trying to push my career in this direction over the past few years, as you can probably tell, and the work I’ve been doing on Studiosity has afforded me an excellent opportunity to do so.

We had a good discussion on the nature of research and the differences between research and evaluation. The latter, generally, being something which is done for internal purposes and audiences only, while research is likely of wider interest and therefore there is value in sharing via relevant publications. Within our community, however, there may be barriers which prevent, or make it difficult for professional services staff to publish. One colleague mentioned a publication, not named to protect the guilty, which charged for publication, but gave steep discounts to academic contracted staff, but none if you happened to have ‘professional services’ on your contract.

We also talked a lot about ethics committees, which again can be hard to access, with another colleague reporting that they weren’t even allowed to submit something to an ethics panel, while at another institution professional service staff were kicked out of their ethics board because it was felt to be having a negative impact on their REF submission.

That all sounds rather bleak, but there are solutions to these problems. Some people reported having nominal 0.2 academic contracts to get over institutional barriers, while others are running their own internal ethics boards. It was a very good discussion this morning, and something which is going to become a series, so I will be learning and writing more on this.

Relevant related reading: Defining the Scholarship of Teaching and Learning, by Ann M. Gansemer-Topf, Laila I. McCloud, and John M. Braxton.

Leave a Comment

PG Cert AP: Day 6

Published 8th December 2016

The final day of the first semester was a little unusual. The morning was given over to a review of the assignments for this module which are to complete the UKPSF form, critique a learning session, analyse a learning theory, and write a report on the experience of peer observation, comparing the experience of being the observer and the observee. Drafts are due at the end of semester 2, with final versions by September. All well and good, and all covered in the module guide. This session didn’t add anything, and yet we did literally spend the entire morning debating it. Strange things happen when you have academics as students.

The afternoon session was more useful. First there was a short presentation on evaluation in general, why and how to do it, followed by an introduction to nominal group technique. A definition of evaluation was given as ‘assessing the process and practice of a prior learning strategy or event by feedback and trying to make objective summaries of an often subjective interpretation.’ This was followed by a discussion on the different types of evaluation – student, staff, data, and self – and the difference between quality assurance, which is backwards looking and tends to be about accountability, and quality enhancement, which is about how to improve and develop your programme or module.

With quality enhancement in mind, nominal group technique was then introduced followed by actually using it to evaluate this first semester. As a group, and with the programme leader absent, we drew up two lists of ten to twelve points of things that are going well, and things which we think need to be improved. These were written on a board in no particular order, then individually we had ten votes, or points, with which to rank what we thought were the most important points. So for example, if you thought that ‘over-assessment’ and ‘use of VLE’ were the two most important things that needed to be improved upon, then you could give each one five votes. The programme leader was then invited back in and the votes were added up to show what we collectively ranked as the most important things for improvement, and what we felt was going well. The outcome of this evaluation will be actively used in the development of the programme for the second semester.

Leave a Comment

Blackboard Roadmap 2016

Published 3rd February 2016

bb_learn_ultra

Ah! Blackboard, my old friend, it’s been a while. Since we’re officially in the market for a new VLE now, the team dipped into one of Blackboard’s roadmap webinars to see what’s new.

The first thing I noted was that the webinar was delivered using the classic version of Blackboard Collaborate, not Ultra, which I later learned is because Ultra is currently limited to 100 participants. They are working on upgrading this to 250, with 1,000 users under consideration for further in the future. This was the first time I had seen Ultra in any detail and it looks so much nicer and smoother than classic Collaborate. It’s entirely browser based, with no downloads or plugins required, which is great as this is still the major stumbling block for collaborative and conferencing tools. There are also still some features of classic that haven’t made it into Ultra yet, including Breakout Groups and Polling, though they are on the roadmap. For this reason most institutions are running with both version at the moment, a possible source of confusion.

Next up was the core product, Learn, which still looks very much like it did when I left Northumbria which was on 9.1 with Service Pack 14 at that time. Blackboard showed some screenshots of how it is going to evolve as they integrate the design language from Ultra (naughty screenshot above), which is also going to include significant improvements on responsive design in many areas. Under the hood they are upgrading the JDK to version 8 and introducing support for SQL Server 2014.

In spite of the efforts to improve Learn with responsive design, they are also still supporting and developing a number of iOS and Android mobile apps. The old Blackboard Mobile Learn app looks to have been abandoned now – no updates to the iOS version since September 2014 – but they are still supporting it for existing users. This has been superseded by the new Blackboard Student app, with a Blackboard Instructor app under development for staff. As a stopgap they have released Blackboard Grader for staff which allows you to grade and provide feedback on assignments.

Finally there was discussion of the various models of provision. Though self and managed hosting are still available, they are actively encouraging people to migrate to their new SaaS and continuous release model.

Leave a Comment