FIGURES AND DATA
KILDARE WICKLOW EDU
Created on September 4, 2021
Over 30 million people build interactive content in Genially.
Check out what others have designed:
NORMANDY 1944
Horizontal infographics
BEYONCÉ
Horizontal infographics
DEMOCRATIC CANDIDATES NOV DEBATE
Horizontal infographics
ONE MINUTE ON THE INTERNET
Horizontal infographics
SITTING BULL
Horizontal infographics
RUGBY WORLD CUP 2019
Horizontal infographics
GRETA THUNBERG
Horizontal infographics
Transcript
110 survey responses from FET educators about quality assured assessment processes
72%
Agreed that QA processes strengthen the principles of assessment
Click for overview of survey themes
Open & collegial, leading to authentically enhanced quality
Managerialist, bureaucratic and illogical
Assessors are concerned about assessment reliability and the consistency of awards and standardsAssessors need the opportunity to benchmark
33%
This increases to 41% at QQI Levels 5 & 6
Educators’ Experiences of Quality Assured Assessment in the Further Education Sector:Insights from a practice-based innovation, using ePortfolios for Assessment
Rachael Doherty, MA in Educational Practice
of assessors fully compile their learners' assessment portfolios
Click for overview of focus group themes
A way forward with ePortfolios KWETB ePortfolio Project Focus Group
with assessors having experience being an authenticator having positive effect
Assessors are overwhelmingly positive about the use of technology in asessment
to
Additional QA support is required for new educators
PROMOTION OF ASSESSOR ROLE
PROFESSIONAL LEARNING AND DEVELOPMENT
PROCEDURES TO SUPPORT INTERNAL MODERATION
Implications for Practice
Implications for policy
Survey respondants had vastly different experiences of QA, from:
OVERVIEW OF FINDINGS
Quality assurance
ASSESSMENT
TECHNOLOGY
EducatorsEXPERIENCES
QQI (2018) have themselves determined that: “..designing and implementing summative assessment procedures and tasks that are: a) valid and reliable for the purposes of certification; b) nationally consistent; c) fair and suitable for use in competitive allocation of places in higher education; may be, realistically, we suspect, beyond the unaided capabilities of many providers under the current arrangements.”
Opportunities for assessors to take on the role of authenticator as part of an open and collegial quality process were identified as valuable experiences that enhance quality. In order to explore whether answers to the below statement was different based on whether the respondents had authentication experience, responses were grouped based on this variable. A Fishers exact test found that the frequencies were significantly different (p = .046). Self reporting of assessor confidence in their strategy for quality assuring assessment is significantly different on the basis of authentication experience (p<0.05, Fishers exact test)
The administrative burden associated with compiling and organising every learners assessment portfolio is not recognised by management or in policy/procedure as part of an assessors role. Also, this reduces learners autonomy associated with constructing their showcase portfolio of assessment pieces.
Assessors enrolled in ePortfolio project felt it had benefits for student learner centred assessment, learner autonomy, organisation and QA. It also supported staff to develop greater confidence and clarity about the assessors role in QA. All three of the key survey themes were supported by the opinions of all focus group participants, supporting the triangulation of the study findings (Caillaud & Flick, 2017). Issues of excessive administrative burdens that lack clear purpose were discussed. The use of ePortfolios was described as going some way to reducing this burden. Again, comments by participants illustrated that their complaints are not simply about the amount of work the current system demands, it is that because what is being asked of them feels senseless. Instead, they want an enhanced quality system that is more authentic. The following response highlights this: "Where's the focus on the actual teaching and learning? Hopefully the ePortfolio changes this, because since we've started using it, there's less emphasis on the tick box exercises…It just makes sense. And it's not to be getting out of any work, because like we've all just mentioned, these tick box exercises are just a waste of time." Inauthenticity in the process comes up several times. The following participant names a key issue as a perceived lack of trust in educators in the sector, stating: "We were constantly giving (students) feedback in person, in real time. There's no trust in the tutor that you are providing feedback. It's meaningless writing on a form that you have given a student feedback, because that may or may not be true. It proves nothing." Trust and accountability are inextricably linked, and recent research featuring the higher education sector describe how bureaucratic accountability can burden educators with administrative reporting, which does not contribute to any improvements to quality (Ehren et al., 2019). Only when there is open and genuine conversation about the quality of assessment can there be a meaningful discussion about change. For this to occur, FE tutors must be trusted to fulfil more of their core roles without surface level monitoring and to be active participants in a deep and meaningful conversation about quality. The ePortfolio project promoted open discussion about assessment processes among educators. Some of the participants found this very valuable. One participant also felt more confident about their role in the process when using the new system. "I've gotten one ePortfolio ready for IV and I feel a great sense of confidence. Whereas usually, I'm worried because I could be missing a whole section. I love the organisation of it, and I'm really enjoying it and the community of practice is just brilliant and I hope it keeps going." Opportunities such as this voluntary pilot allow for professional development, peer learning and a genuine improvement in quality, with a supportive framework for educators. While professional development has traditionally been delivered by external agencies, research and experience has demonstrated that professional learning is often best located “in house” where educators can focus on problems and improvements in areas of relevant practice, while offering support and opportunities to learn from each other (Hara, 2009). Other studies have also acknowledged the potential of similar “technology in education projects” for staff, as being an important spark for professional learning (Ambler et al., 2014). The knowledge and expertise held by professionals within an organisation, who have the added understanding of local policies, practices and student needs cannot be bought (Lieberman & Miller, 2011). The focus group conversation also dealt extensively with the issues of assessor confidence regarding reliability and consistency of standards in the sector. The group felt that a move to ePortfolios from the paper-based system provided multiple opportunities that were previously impractical. These included the sharing of exemplar ePortfolios for various disciplines and to represent a range of grades and assessment review meetings, similar to those held at both second and higher-level institutions that enable educators to collaboratively reach consistency in their judgments of student work (National Council for Curriculum and Assessment, 2019; Quality and Qualifications Ireland, 2013). Previously, this idea would have been very challenging, with small education centres and colleges often having only one tutor per academic discipline, making in house cross moderation virtually impossible. Looking further afield, moderation across colleges or providers was highly impractical, with moving and transporting boxes of student assessment folders logistically difficult and problematic from a security point of view. One idea put forward by a participant and strongly endorsed by all others was "if we compiled fictitious ePortfolios with different pieces of work, and then got a small focus group to all grade them using the same marking schemes and see what the results are.” This suggestion was thought to be the most constructive and collegial starting point, in a sector where there is little precedence for second marking and for assessors whom initially may feel vulnerable about cross moderation activities. Finally, from the focus group discussion it seemed that the move to ePortfolios provided benefits for students in managing and having autonomy over their learning. For example, they were able to see their marks and map their overall progress. One respondent noted that “students are able to go back into the ePortfolio and go, ‘oh, yeah, I'm on task for that or I have to up my game for that’”. It was identified by several participants that the portfolio structure was a support to students allowing “…students to see the empty pages and where they need to fill in and what they're missing. They were able to tell me exactly what they were missing, which is unreal”. It also supported the continuity of assessment and gave students a greater opportunity to learn from the process with one participant remarking: “I've been able to correct as I go, and then I've put all their feedback up, they've got it straightaway and then when they come to their next assignment, they're ready.” These comments highlight that the ePortfolio model may elevate the value of the assessment as and for learning, allowing greater opportunity for students to use ongoing assessment and feedback as opportunities to improve and to monitor their own learning, with a greater understanding of assessment criteria (Earl, 2007). These improvements in student experience are important examples of quality enhancement in assessment. Not only do they align with the principles of Universal Design, but they suggest that the ePortfolio model has the potential to strengthen the constructivist, learner-centred process of assessment that it was intended to support (Wolf, 1998). If portfolios are to remain as a primary feature of assessment and authentication in the sector, this is an area that warrants further exploration and analysis in the near future.
Online submission systems were described as being a support to students as well as for educators: “Less stress for students printing out assignments. All assignments will always be submitted online from now on, no hard copies.” This sentiment is mirrored in the findings of a recent survey by QQI, examining how COVID impacted teaching, learning and assessment in the Irish Further Education sector, which found that educators preferred the online tools for provision of feedback and submission of assessments and would like to retain these practices (QQI, 2020).
“Due to Covid, the glass ceiling has been broken" “Technology in the classroom has really come to life due to COVID. The advancements have been extraordinary over the past year and have definitely benefitted both teacher and student.” “The use of electronic portfolios will help efficiency, quality, uniformity throughout the organisation”. Other participants expressed concern about the number of new digital assessment options being presented to staff, such as Moodle, Microsoft Teams and ePortfolios and a lack of consistency in approaches. It is suggested that “one system across the ETB sector should be introduced”.
- Knowing the standard with ePortfolios
- Proposed new ePortfolio model for Further Education
Survey quotations "Doing IV myself gave me more insight into general standards and approaches to measuring learning across multiple modules taught by different staff. The process of having my folders go through IV was less enlightening, as it is all done behind closed doors and feedback is only given if there is an error in the addition of numeric grades or a signature missing." “I think seeing a variety of work as an IV is a positive process, it opens up conversations, allows the sharing of ideas and helps colleagues iron out minor issues” “Internal verification can be a positive learning tool in picking up good practices and seeing how other colleagues interpret assessment"
Survey quotations "The QA processes need to be fully understood by management to ensure they are rolled out correctly. They should not become a labour-intensive process that at times does not make logical sense, it’s purpose needs to be clear and not just additional paperwork for the sake of it." “Lack of appreciation for deep learning. Focus seems to be mainly on yellow tape and paperwork.” “Just a lot of unnecessary duplication of marking sheets which are required...too much paperwork to submit” "Up to now we were told we had to have three written feedbacks for each student per module - this felt very artificial and very much like a paper exercise to satisfy the external examiner or prove that we were giving feedback. We were giving feedback anyway. Documenting it on a specific form was time consuming, especially if the class was large."
A clear subtheme from the survey, in the area of assessor confidence, was that new staff in particular require additional support, as stated very clearly by multiple participants, with one stating “I think colleges need to give new teachers more support in relation to how it works in their particular institution. In my case it's left to each individual teacher there's no real guidance or clarity.” Many survey responses captured some difficulties and challenges that were faced by new and inexperienced staff, who often reached their first assessment period unprepared and lacking in confidence. One participant recalled that in their first assessment experience they were “…left virtually alone without help or support to complete assessment and I made so many mistakes and interpreted things incorrectly. Training and support would be vital.” Another concurred that their first experience “…was very difficult” and they felt “that a lot of emphasis needs to be given to helping new teachers through assessment periods and peer mentoring would be an invaluable help in this area.” In order to determine whether levels of assessor confidence was affected by respondents length of service, respondents were grouped based on whether they had up to or greater than ten years service. A Fisher’s exact tests found results were significantly different (p = .031) in response to the statement below:
Survey quotations “There is a distinct lack of quality control and comparison of learner's work and grades within centres never mind between centres or nationally” “Prior to working as an EA” they had “no opportunity in FE to see work from another teacher of the same subject.” “I believe that more transparency and sharing of materials across ETBs would vastly improve the overall QA. Simply relying on EAs for this is not enough in my opinion”. “Double marking of a percentage of assessments within the college may be beneficial to QA”
Educators’ Experiences of Quality Assured Assessment in the Further Education Sector:Insights from a practice-based innovation, using ePortfolios for Assessment
Rachael Doherty25th August 2021
Introduction
Quality assurance
ASSESSMENT
TECHNOLOGY
EducatorsEXPERIENCES
Background and Rationale
Assessment
Authentication
Quality Assuring Assessment
Portfolios support learner-centred model of education Portfolios support intensive recording and oversight in system ePortfolio Project
Survey: Main Findings
"Behind Closed Doors" Issues with Defending QA with Administrative AccountabilityAdministrative Burden -”Ticking the Boxes” Benefits of an “Enhancing Quality” Culture
Title hereLorem dor
LOREM IPSUM DOLOR Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
survey of assessorsMAIN Themes
Assessor Confidence Support for New TeachersConcerns about Reliability - Knowing the Standard
* p < 0.05
Technology Enhances Assessment Processes for Assessors and Students Increased Accessibility and Efficiency Use of ePortfolios for Assessment
Focus Group: Main Findings
Enhancing Quality Assurance
LOREM IPSUM DOLOR Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
EpORTFOLIOFOCUS GROUPMAIN THEMES
Developing Confidence in Reliable Standards
LOREM IPSUM DOLOR Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
Reimagining Student Centred Assessment Processes
LOREM IPSUM DOLOR Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
Implications for Practice
Knowing the Standard- Quality Assuring Fair and Consistent Assessment Assessors meeting together Sharing of graded exemplars Double marking of samples Discussion of queries and borderline grades
Implications for Policy
PROMOTION OF ASSESSOR ROLE
PROFESSIONAL LEARNING AND DEVELOPMENT
PROCEDURES TO SUPPORT INTERNAL MODERATION
Future Plans
CONTINUOUS IMPROVEMENT
STUDENT EXPERIENCE
INTERNAL MODERATION
ePortfolioof Assessment
"Behind Closed Doors" Issues with Defending QA with Administrative AccountabilityAdministrative Burden -”Ticking the Boxes” Benefits of an “Enhancing Quality” Culture
LOREM IPSUM DOLOR Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
Title hereLorem dor
LOREM IPSUM DOLOR Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
survey of assessorsMAIN Themes
Assessor Confidence Support for New TeachersConcerns about Reliability - Knowing the Standard
LOREM IPSUM DOLOR Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
Technology Enhances Assessment Processes for Assessors and Students Increased Accessibility and Efficiency Use of ePortfolios for Assessment
LOREM IPSUM DOLOR Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
Enhancing Quality Assurance
LOREM IPSUM DOLOR Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
Title hereLorem dor
LOREM IPSUM DOLOR Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
EpORTFOLIOFOCUS GROUPMAIN THEMES
Developing Confidence in Reliable Standards
LOREM IPSUM DOLOR Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
Reimagining Student Centred Assessment Processes
LOREM IPSUM DOLOR Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
Who Compiles Learners' Portfolios of Assessment?
Confidence that assessment strategy is fully compliant with QA process
Confidence that assessment strategy is fully compliant with QA process
Confidence that assessment strategy is fully compliant with QA process