By Ryan Locke (Lecturer, Computer Arts) and Greg Bremner, (Lecturer, Economics) Abertay University
In this blog entry we report on a presentation given to the Quality Assurance Agency for Higher Education (QAA) by Ryan Locke and Greg Bremner in March 2018 for its event entitled ‘Focus On: Assessment and Feedback.’ This forms part of a project, developed in collaboration with the Scottish Higher Education Enhancement Committee (SHEEC) and the QAA, which comprises 3 strands of work.
- Use of technology to support assessment and feedback practice
- Consistent implementation of assessment and feedback policy and practice
- Students associations working in partnership to improve feedback on assessment.
Our presentation to the QAA, entitled ‘What can Online Gaming Communities Teach Higher Education about Engagement & Feedback?’ builds on ‘The Future of Teaching and Learning at Abertay – a Vision’ which outlines the Teaching and Learning Enhancement (TLE) strategy as it plays into the broader Abertay Strategic Plan, 2015-2020.
As a matter of course, Ryan delivers feedback to art students in the form of video, recorded on Open Broadcaster Software (OBS), and uploaded to a private ‘YouTube’ channel for the student to see, thus replacing the Blackboard comment insertion platform that is widely used in the University. Greg deploys video feedback for students who submit large volumes of text and diagrams; dissertations and PhD chapters, for example. The QAA presentation afforded the opportunity to demonstrate our use of video technology in two diverse subject areas; art and economics.
Benefits to Students
The benefit that accrues to students who receive video feedback is well settled in the academic literature that underpinned our presentation (See Carless, 2006, for an extensive survey). Put briefly, Crook et al (2012) demonstrated that the use of video feedback resolves many of the common problems of feedback in relation to the quality of student engagement. Students find videos easy to access, the content to be clear, and, importantly, video improves their motivation to engage. Part of this improvement in motivation might be because, where feedback is typed, students believe that all written comments are equally important. Video feedback provides students with both verbal and visual cues which highlight the relative importance of each comment (Robinson et al, 2015). Experiments with audio feedback, reported by Lunt & Curran (2010), showed that students are at least 10 times more likely to open audio files compared to collecting written feedback. The voice is able to convey more than the written word: the tone, nuance, expression, informal language, sincerity and the evidence of personal, dedicated input from the academic provides additional layers of meaning and feedback to the listener (Rust, 2001).
Complementing the benefits to students is an element of value that accrues to those educators who provide video feedback. Carless (2006) and MacLellan (2001) documented the differing perceptions of traditional feedback held by tutors and students; tutors believed their written feedback to be more useful than students believed it to be. The idea that we are reflective practitioners demands precisely this sort of information as a mechanism for reflection and improving and updating the way in which we operate as educators. Technology affords an opportunity to deliver feedback in a more impactful way, thus facilitating reflective practice and providing a component of education that is, not only grounded in academic literature, but is also as up-to-date as technically possible.
It would be a reasonable assumption on the part of a potential student, when considering their UCAS options, that Abertay’s reputation in the world of computer games education will, by extension, underpin a better overall technology-driven higher education experience than that which they might expect elsewhere. The idea of the ‘interactive’ university, where technology is the fundamental component underlying the promotion of student engagement, sits comfortably on the spectrum between ‘teflon’ and ‘flipped’ models of future universities and video feedback, insofar as it is pedagogically expedient, can facilitate the expectations of students who were schooled in a more technology-driven pre-university environment than many of those from whom they receive feedback.
Providing feedback via video is not an end in itself. Reflective higher education practitioners, who can harness modern technology to the benefit of their students, are now a necessity that is expected of all technologically adept universities.
The tech edge
Our QAA presentation demonstrated the ease with which video feedback can be deployed; easily accessible hardware and software allows educators to explore, innovate and implement new practices with few barriers. We are accustomed to using mobile smart technology and wherever we are. We carry internet access, high-quality cameras, microphones and other accessories with us, which make documenting and recording easier than ever. Complimenting these technical advantages are numerous free-to-use services that make uploading and sharing instant and easy. Abertay staff already use the ‘record audio’ feature on the Turnitin iPad app.
A crucial further enhancement is the rapid rise of streaming technology and services. ‘Youtube Live’, ‘Facebook Live’, and ‘Twitch’ could be considered the ‘Big 3’, but take a moment to consider ‘Twitch’. Hobbyists can broadcast their craft via ‘Live Streams’ from home, complete with features that don’t just allow viewers to watch; they can chat, follow, donate, ask questions of the Streamer and the community. Today’s students are of a generation who, as a matter of course, access self-made live broadcasters who use ‘YouTube’ or ‘Twitch’ to ‘stream’ – this is largely done for fun, but a significant portion of these channels, or ‘streamers’, are dedicating their live broadcasts or recordings to educating people. Indeed, some of your own students may be live broadcasters, ‘YouTubers’ or ‘Twitchers’ themselves, with access to vast audiences.
While we have access to a free video library on ‘YouTube’, the really interesting part may lie beyond its platform, for users to teach through demonstration; the interactive, social and community building aspects of Live Streaming services are coming to the fore and are there to be harnessed by educators.
Fig 2. How do you make scrambled eggs? Jamie Oliver has explained it to nearly 10 million viewers.
A Question: We would like you to ask yourself: Have I ever used YouTube.com (or similar) to learn how to do something?
Fig.3. A ‘Streamer’ paints live online via ‘Twitch.com’. There are approx. 500 live viewers.
The live stream format has influenced our recorded video feedback format. The video that students receive as feedback is designed to include a visual of their submission, this might be a piece of art work (see Fig.4) or a PhD chapter, which forms the main body of the screen. This is complimented by a video and audio recording of the feedback in real-time. The feedback video comes with a sense of conversation, as explained earlier, allowing for tone of voice and facial expression to be noted by the student. Upon completion the video is uploaded and a link sent to the individual student, who can watch and re-watch their video feedback at their convenience on any device. The streaming facilities and associated social and community features also pose other avenues worthy of exploration and pedagogic enquiry.
Changes in established practices, especially changes that involve technology, can arouse a range of reactions; among them, trepidation. However, Abertay’s TLE team will support and facilitate staff in the learning and deployment phase. We (Ryan and Greg) are seeking to expand the discussion on the use, implementation, possibilities and limits of video technology in assessment and feedback. Consequently, we will host open-invite coffee mornings in the near future where academics can ask questions, make suggestions or just listen. These will be arranged through the TLE office and if you are interested, please email firstname.lastname@example.org.
Meantime, we want to know if there are other disciplines or scholastic activity where video recording technology can have an impact that enhances the student experience.
Do you have ideas? We would like to know as we grow and explore this exciting area at Abertay. Please email us directly at email@example.com and firstname.lastname@example.org.
We look forward to hearing from you,
Ryan and Greg.
 Paper prepared by Dr Alastair Robertson, Director of Teaching and Learning Enhancement and available (to Abertay staff only)here: https://intranet.abertay.ac.uk/download.php?f=SMG%20meeting%20note%20and%20papers%202018%2003%2019.pdf
 ‘Differing Perceptions in the Feedback Process’, Carless. D. Studies in Higher Education Vol. 31, No. 2, April 2006, pp. 219–233
 ‘The Use of Video Technology for Providing Feedback to Students: Can it Enhance the Feedback Experience for Staff and Students?’ Crook. A, Alice Mauchline. A, Maw. S, Lawson. C, Drinkwater. R, Lundqvist. K, Orsmond, P, Gomez. S, Park. J, Computers & Education 58 (2012) 386–396
 ‘The Benefits of Delivering Formative Feedback via Video-Casts’ S.J. Robinson, L. Centifanti, G. Brewer, and L. Holyoak. UCLan Journal of Pedagogic Research, Volume 6(2015)
 ‘Are you listening please?’ The advantages of electronic audio feedback compared to written feedback.’ Lunt, T, Curran, J (2010) Assessment & Evaluation in Higher Education 35(7): 759–69.
 ‘A Briefing on the Assessment of Large Groups.’ Rust. C. (2001) LTSN Generic Centre A Briefing on Assessment of Large Groups November 2001
 ‘Assessment for Learning: The Differing Perceptions of Tutors and Students’, MacLellan. E. Assessment & Evaluation in Higher Education, Vol. 26, No. 4, 2001
By Dr Janet Horrocks
Lecturer, School of Science, Engineering and Technology
Day one of the SEDA meeting and I felt a bit like a fish out of water (appropriate as the meeting was held in a waterside venue). The majority of delegates were educational developers from institutions across the UK and further afield. I was there to talk about some of the recent developments at Abertay but acutely aware that as a lecturer in Biomedical Science I was rather short on eduspeak. Fortunately we were scheduled to run in the first of 6 parallel sessions so after coffee on Day 1 I was able to relax and make the most of the event.
One of my main interests this year has been team and group work. Last year I ran two new modules with significant elements of team and group work and I am doing the same this semester. The sessions on Evaluating Individual Contributions to Group Work: How useful is Peer Assessment and Preparing Students for Group Work seemed ideal. The first session gave a good example of using a detailed rubric for peer assessment however (as is often the way) with meetings the discussions on our own practice was illuminating and the idea of one delegate: asking students to write their own peer marking criteria struck a chord and I am using this approach this semester. What better way for students to “buy in” to the grading criteria than to develop it! Interestingly every group I have used this for have included attendance element in the peer assessment criteria. The second session was investigated ways of preparing students for team work. As a consequence I decided to add in a team building activity into the first week of my elective module. Identifying a suitable activity was a problem as I wanted an activity that would not allow a single group member to dominate proceedings. We decided on a fun exercise which involved students putting up tents while blindfolded (with one team-member acting as a sighted guide but not able to handle any of the tent). At the end of the exercise the students had begun to develop a team identity (and had had a lot of fun).
The keynote on the second day introduced the idea of SCALE UP (Student-Centred Active Learning Environment with Upside-down Pedagogies). This is an approach developed by a Physics Professor at North Carolina State University and involves students working in groups on short problem –solving and enquiry-based activities. SCALE UP uses a combination of flipped learning, on the spot feedback during tasks and “students as teachers”. In a session the lectures might give feedback on progress, student teams might be working in small groups, teams will be presenting their work to each other or the whole group and the lecturer might summarise the output or give a short whole-class talk. A crucial part of this approach was appropriate learning spaces, more specifically round tables that are big enough to work on but not so big that communication is difficult, access to lap tops and mirroring technology so that output could easily be shared. The approach has been most enthusiastically adopted by around 200 academics at Nottingham Trent University (the speaker’s home institution) with classes of up to 100 (run with several staff).
An ongoing theme for the meeting was the “Death of the Lecture” in its formal sense. I felt unable to admit I was going back to Abertay to deliver … a lecture to a room of 50 students all sitting in rows. It was interesting to see how different institutions were preparing for the lecture’s demise by developing flexible learning spaces. Some of the plans for converting tiered lecture theatres into interactive spaces looked quite exciting (although only useful if the lectures could manage the steps) and one institution was building a new campus with no formal lecture theatres. Could we do the same at Abertay?? I enjoy using interactive learning approaches but still feel the lecture has its place in my own teaching.
So in answer to my original question do meetings influence practice – yes they do and I will update you on the student written peer marking rubrics.
A lighthearted and reflective account by Professor Carl Schaschke
Head of the School of Science, Engineering and Technology, Abertay University
Whilst in the midst of a seemingly light-hearted conversation at a party recently, I found myself being caught off-guard by a somewhat over-inquisitive guest: “So, what is it that you actually do?” With canapé in hand, a more convincing answer had been expected than the informal end-of-year setting merited.
As Head of School I am responsible for providing effective leadership, management and development within the framework of the University’s mission and strategic direction. Reporting to the Vice-Principal, I am the representative in internal University and external business networking, and hold responsibility for developing the School’s stakeholder relationships that lead to the successful development and progress of the School. I have a key role in implementing University policy and in progressing the University’s long term aims and objectives particularly in institutional strategic planning and decision-making.
While it says everything and, according to my questioner, nothing, it was the use of the word “actually” that had caught me off-guard.
So, what is it that I actually do?
Perhaps it is worth reflecting on my career motivations that has led me to becoming Head of School? Graduating in chemical engineering, I joined the nuclear industry with a clearly structured career ahead of me. Over time, however, I had steadily begun to consider the up-and-coming biotech industry to be a more attractive proposition than the nuclear industry that had generously supported my undergraduate studies with a scholarship. Parting company to return to university to pursue a PhD in biochemical engineering to gain the necessary qualification, it was nearing the end of my research studies that I found myself increasingly caught up on the excitement my research and informing others. I had not, up to that point, considered a future as an academic in preference to my goal as an industrialist? Yet, when a timely opportunity presented itself, I soon found myself on a very different and unexpected career route.
A move to another university followed with subsequent career promotions based largely on my growing research profile. While I had been focused on my own work, I increasingly began to appreciate that the operations of the University lay beyond that of my own department. Joining Senate through a competitive peer-nominated process gave me exposure to the wider work of the University. But it was my motivation to become head of one of the largest academic departments of chemical engineering in the UK that was based on an ambition to raise significantly the Department’s profile through the Research Assessment Exercise (RAE): the forerunner to REF. The Department had been conspicuous by its absence in previous league tables that I felt had unhelpfully held it back in achieving its potential.
Appointed as Head, the University’s Senior Management endorsed my ambitious plans that included an RAE submission, the appointment of new academic and support staff along with significant investment in new laboratories, equipment and learning spaces. Presenting a clear vision within the realms of reality, I worked with key people across the University, transforming the Department from one of being a largely teaching-focused to being a high-profile research-led and industrially-facing grouping with the intention of raising the external profile attracting the most promising of applicants for its courses, the most outstanding of academics and support for its research through industry, research councils and other funding bodies.
With a focus on ensuring industrial relevance based on my early career experience, I introduced new academic programmes with new features such as industrial placements, bursaries, prizes, seminars and site visits. I was programme leader of the newly introduced flagship MEng programme whilst also being Head with a near full lecturing load. I felt most comfortable operating an open-door policy and made myself available and visible to both staff and students. The effect was a steady increase in student numbers, student satisfaction (NSS), increased league table positions as well as RAE outcome. I subsequently became involved in the mentoring other Heads of Departments. I also engaged in being mentored for my own professional development, which ultimately led to my successful move to Abertay.
With the mandate to raise the visibility of the newly created School of Science, Engineering and Technology at Abertay, my aim from the outset was to ensure students are at the focus of my actions and who remain fully supported. I began at Abertay by carrying out a full assessment of the School and an assessment of the impact of delivery of programmes within the available resources. Notable changes that followed included re-profiling the School and Divisions, the appointment of new academic staff, supporting promotion opportunities of staff, the introduction of new undergraduate programmes and termination, suspension and rebranding of others. Curriculum review and reform followed along with the development of new laboratories and procurement of equipment along with forming new college partnerships, introducing a new approach to health and safety and a new approach to business continuity and recovery based on my earlier experience following a major fire that had destroyed an entire engineering building.
While having begun my career as an industrialist, I continue with my industry links largely through my professional organisations which leads to influencing policy and strategies such as with the government. Yet it is the direct engagement with students themselves that lies behind the motivations of my academic rather than industrial career. A shortlisted finalist of a Daily Telegraph science writing competition and a secondment to BBC TV’s Tomorrow’s World which included producing science stories including, memorably, a mouse with a human ear on its back, gave me confidence in my abilities to communicate science to an even wider audience. I later become a commissioned science writer for a magazine and then editor of a several of scientific journals. Having amassed a wide range of experience and observed the modes of study use by many students over the years, I took the initiative to widen the impact of academic practice from teaching and journal publications to the authoring of academic books aimed at supporting teaching for an international audience. Now with five books published, the premise of each has been a recognition that students frequently experience difficulty in identifying the necessary and relevant information to solve science and engineering problems or who may be hesitant in applying theories. This is often due to an incomplete understanding of the principles or due to a lack of confidence as the result of unfamiliarity with the subject matter. Having won several teaching awards and received correspondence from graduates and readers with new ideas as well as appreciation of the help my books have provided in their studies, it remains deeply satisfying to have had an input into the future careers of the next generation of scientists and engineers.
So, what is it that I actually do?
By John Babraj, Lecturer in the Division of Sport and Exercise, Abertay University
In the previous blog, we looked at how to create a research environment within the curricula. This looked at different strategies and expectations to create a cohesive framework. In this blog, I want to explore how we can utilise research as an assessment tool. This closes the circle with the students becoming active researchers and allows them the opportunity to explore topics in real depth.
The final part of embedding research into the curriculum is to utilise research as an assessment tool. Whilst the assessment itself comes in the form of a written paper, there is a lot of work to be done in the module to get the students ready to submit. However, it does provide an opportunity to package lecture, tutorial, practical and assessment as a complete platform. One of the advantages of this approach is that it embeds research methods firmly within an applied setting, moving it away from theoretical concepts to a process that is required to enable them to answer the question. For the students, this means that they have to take ownership of the assessment – they have to do more than just write an essay, as there is data collection, data analysis and the need to develop an understanding of the results. The greater ownership a student takes of the research process is generally reflected in module achievement.
I have used this approach successfully at stage 8, 9 and 10 and have found it is an approach that can engage and enthuse the students. This approach can be used to get the students to demonstrate understanding of key concepts at stage 8 or to explore real world issues and novel approaches to overcome them at stage 10. However, the approach to the assessment needs to change to reflect the capabilities of the students. If we think about the research process, it can be broken down into 5 key stages: research question/topic area; study design; data collection; data analysis and report writing. Each of these areas can form part of the overall assessment strategy, but there are issues to consider.
- Research question/ Topic area
For this to work it requires that the students take complete ownership of the assessment. They are setting out the research area that they wish to look at and this requires a high degree of knowledge by the students. If this happens without guidance then it can lead to a disconnect between assessment and module. Therefore, you need to support the students towards relevant topic areas or limit the options of what can be researched. In practice, I have found that it works better to give the students the topic area, due to the time constraints of a module and the different interest of students making it difficult to reach consensus on a topic.
- Study Design
This requires them to come up with ideas on how to answer the research topic. This helps to develop problem solving skills and critical thinking within the students. This works well when information finding is embedded into the tutorial sessions and allows the students to find relevant material with guidance. It can be used to create real life situations that a student could encounter in their future career. For example, in stage 9 I ask the students to choose five tests to carry out on an athlete of their choice. This requires them to explore the key physiological requirements for that sport and decide what the best way to test it is. It allows the students to critically evaluate why we test and link testing to performance. It is also a situation they could be faced with if they went on to work in sports science support.
- Data collection
This helps to develop practical skills that are needed in employment and helps to build confidence in the students as independent researchers. It requires time within the module to build up their confidence so that they can be left unsupported to do data collection. It is important to make clear from the outset that they have to do data collection and that time in the laboratory should be used to make sure they are confident with every test. I have found that this approach stops students being disengaged in laboratory sessions and encourages them to ask questions and to think about what is being done.
- Data analysis
Development of analytical skills is essential for student growth. In the research scenario, they have generated numbers and they now have to make sense of them. It requires them to use SPSS and determine how to analyse the data. In my experience, students are terrified of statistics and this approach allows them to understand why they are doing them. It is important that the statistical analysis is appropriate for the stage, with early years relying on simple statistics such as T-tests, means and standard deviations whilst later years will be relying on repeated measures ANOVA approaches. It may be that you need to provide support either through tutorials or through blackboard resources to enable the students to carry out the analysis. It also allows you the opportunity to move beyond p values and start getting the students to think about how much of an impact would any change have in the real world.
- Written assessment
This is the part that is assessed but you can see there has already been a lot of work put in before the student gets to write the paper. I use a journal paper format but do not follow any particular journal instructions. It would be possible to follow a journal submission requirements but I have chosen not to do so. The assessment is typically broken down into four parts, introduction, methods, results, discussion, with different weightings for each section. I prefer to have larger weightings for the introduction and discussion sections (about 70% of the final mark) as this is where the students show their understanding. In the introduction they have the opportunity to show their wider reading on the research area and introduce why the study is being done. In the discussion they have to analyse the data and put it into context, demonstrating an understanding of what the results mean and why they have occurred.
Over the two blogs, I have hopefully given you food for thought on how it is possible to develop a cohesive research based curricula. From experience, these approaches develop more enquiring students who actively seek out new knowledge. Which at the end of the day is what we are trying to develop. The key take home messages from the blogs have to be
- A research environment is not complicated but requires you as an academic to support the students to ensure they develop as researchers. This could mean thinking of how to use tutorial or practical sessions in a different way (guided learning).
- It takes research methods away from a theoretical subject and places it at the centre of the curricula, encouraging engagement from students in a structured way.
- Research is an excellent assessment strategy that can encourage the students to perform better and become more enquiring. However you as the academic need to tailor the research to the students knowledge base.
By John Babraj, Lecturer in the Division of Sport and Exercise, Abertay University
We continually hear about embedding research into the curriculum as a way to enhance the student experience. This sounds like a simple goal that all academics would seek to embrace fully, given the dual role of the academic as an educator and researcher. However, once you scratch the surface, this simple goal can turn into a minefield; like most simple problems it has layers of complexity that make it difficult to solve in a meaningful way. You may already feel that you are embedding research into the curriculum as you utilise academic journals in preparing your lecture materials. When I started lecturing this was my approach and I would have stated that I was successfully embedding research into my teaching. However, is this view not too shallow; have I really achieved anything other than deliver material that will form the basis of updated academic textbooks in 18 months? So if this is not embedding research, then what is?
Developing a culture around research
This is really at the core of how I have evolved my practice of embedding research in the curricula. The extent of the research culture can vary across modules but at its centre is the idea that the students are junior researchers. This idea is not so strange if we consider where we as lecturers have come from. At some point in time, somebody has had the belief in us that we have the ability to question and generate knowledge. For me this should be at the core of what the research culture in the undergraduate curriculum is. In order to do this there are a number of ways I have utilised to develop the classroom research culture.
It is important to make students aware of the expectations you have around their involvement with the research environment. Across all modules I have delivered since starting at Abertay I have made the students aware that all references for assessments need to come from primary research material. This is regardless of level and I find with support during tutorials in earlier years you can build the students abilities in finding appropriate material. Within 3rd year there can be an influx of direct entrant students and it is important that they get support whilst tutorial tasks can still challenge continuing students. In order to do this there needs to be computer facilities for tutorials and I set real world problems around sports performance whose answer fits around what is being taught but requires the student to find the research material. This allows me to check sites being visited by the students, guiding them through literature searching. This seems obvious but literature searching is a core research skill that we sometimes wrongly assume students should just be able to do. It also leads to an opportunity to discuss strengths and weaknesses of research design in an informal setting. This can help the students see the importance of research to real world issues.
There is no such thing as a stupid question. As an educator it is our job to encourage students to question what we are telling them. This is difficult to do. Students are often reluctant to answer direct questions or to ask questions in group settings. It is also possible that lecturers can feel uncomfortable being challenged in this way. However, I find this approach stimulating and students can bring questions which are unexpected and challenge your narrative. This can happen in lectures, tutorials or practical’s depending on how comfortable the students feel. It is important that you deal with the questions honestly and on occasion this can be to say I don’t know but I will get back to you or simply current research knowledge hasn’t managed to answer that (which can then lead to a discussion of why we haven’t answered the problem). I find that the creation of this active learning environment is a more effective way to deliver physiology teaching, creating students with more content knowledge and greater self-efﬁcacy.
The core skills are study design, methods and analysis. How we teach these skills is a major issue for all lecturers. In the UK we view these as important requirement for achieving honours but research methods can be dull when taken out of real world situations and students can fail to see the relevance. When these skills are embedded within modules, I find students engage fully and develop a better understanding of the underlying research skills.
In my experience this approach leads to a greater engagement of students with each module. They feel that they are being exposed to the most up to date knowledge and the material being delivered is innovative. Students are surprisingly interested in staff research and the enhanced engagement can lead to undergraduate dissertations generating research outputs. I published research from a number of undergraduate students (BMC Endocr Disord. 2009 Jan 28;9:3; Appl Physiol Nutr Metab. 2012 Oct;37(5):976-81; J Strength Cond Res. 2016 Oct;30(10):2761-6, Sport Sci Health (2016). doi:10.1007/s11332-016-0313-x), some of which have been cited over 100 times. The students that produce publishable data sets have typically moved on to postgraduate study, reflecting their desire to continue to understand the research areas they are exposed to.
This approach improves employability skills of the students. They develop critical analysis skills, as they seek to deconstruct information and analyse why. They also develop attention to detail, through doing their project or being exposed to data collection methods in class, they realise the requirements for accuracy within research. Together these are two transferrable skills that enhance employability of students.