Incorporating AI in Learning Assessments: A Guided Pathway

Can students use generative AI tools in an assessment activity?

Do you think using AI will have significant pedagogical implications and can enhance learning outcomes?​

Yes

No

Consider exploring these options when designing assessed activities

Should there be any restrictions on the students’ use of AI tools?​

Yes

No

Yes

No

What AI tools to use

How AI tools are used

Consider the following principles

This flowchart guides educators in integrating AI tools into assessment activities. It begins with a decision on AI usage, then splits into two routes: one explores the pedagogical impact and appropriate use of AI tools, while the other offers a range of traditional assessment techniques. The chart emphasizes ethical considerations, academic integrity, and equal tool access throughout.

Ensure that these changes are managed effectively

Will the use of AI tools significantly change the student experience?

Yes

No

Incorporating AI in Learning Assessments: A Guided Pathway (test)

Consider the following principles

Ensure that these changes are managed effectively

Will the use of AI tools significantly change the student experience?

If the objective is to measure students' learning outcomes in a subject discipline, such as their competencies, knowledge or skills, without the use of generative AI tools, consider using the following (the list is not exhaustive):

1. Face-to-face, closed-book assessments
2. Contextualized project-based or problem-based assessments, community projects
3. Reflection papers with an oral defence
4. Authentic (real-world tasks) assessments
5. Complex tasks investigated over a sustained period
6. Written or oral synchronous assessment
7. Incorporating some parts of assessments in class

Consider which tools can be used. Examples:

• text generative AI tools
• image generative AI tools
• code generative AI tools

You should also consider the following:

1. What is the extent of AI tools' usage that should be allowed in the assessment? This could have two outcomes:
   • Limited Usage: If the AI tool is to be used only for certain tasks, specify these tasks clearly to avoid any confusion.
   • Extensive Usage: If the AI tool can be used extensively, provide clear guidelines on what that entails.
2. Should the usage of AI tools by students be monitored? Consider ways to implement this, such as through software that tracks tool usage or requiring students to document their process.
3. Should there be an approval process for the AI tools that students want to use? If yes, provide a clear process for students to propose and justify their choice of tools and have them approved by the teacher or a committee.
4. Should students demonstrate proficiency in using the AI tool? If yes, suggest methods for students to show they are capable of using the tool effectively, such as a pre-assessment task.
5. Are there data privacy and security considerations? If yes, provide guidelines on how to ensure data privacy and security when using the AI tools.

Consider how tools can be used. Examples:

• AI in the planning stages of the assessment task
• AI as a core part of the task
• AI for self-testing
• AI as copyediting tool

You should also consider the following:

1. What is the extent of AI tools' usage that should be allowed in the assessment? This could have two outcomes:
   • Limited Usage: If the AI tool is to be used only for certain tasks, specify these tasks clearly to avoid any confusion.
   • Extensive Usage: If the AI tool can be used extensively, provide clear guidelines on what that entails.
2. Should the usage of AI tools by students be monitored? Consider ways to implement this, such as through software that tracks tool usage or requiring students to document their process.
3. Should there be an approval process for the AI tools that students want to use? If yes, provide a clear process for students to propose and justify their choice of tools and have them approved by the teacher or a committee.
4. Should students demonstrate proficiency in using the AI tool? If yes, suggest methods for students to show they are capable of using the tool effectively, such as a pre-assessment task.
5. Are there data privacy and security considerations? If yes, provide guidelines on how to ensure data privacy and security when using the AI tools.

Here are some suggestions for steps that could be taken:

1. Provide Adequate Training: Ensure that both educators and students are adequately trained on how to use the AI tools effectively. This could involve hands-on workshops, online tutorials, or even a dedicated 'getting started' session at the start of the course.
2. Offer Additional Support: Recognize that some students may struggle more than others with the new technology. Offer additional support for these students, such as one-on-one tutoring, drop-in help sessions, or online resources.
3. Modify Course Structure: Consider modifying the course structure to accommodate the use of the AI tools. This could involve changing the timing of certain activities, reallocating grading weight, or adjusting the pacing of the course.
4. Solicit Regular Feedback: Since this is a significant change, it's important to regularly check in with students to see how they are adapting. Solicit feedback through surveys, informal conversations, or dedicated feedback sessions. Use this feedback to make ongoing adjustments to the course.
5. Establish Clear Communication Channels: With new technology, there can often be unexpected issues or questions. Establish clear channels for students to ask questions, report issues, or discuss their experiences with the AI tools.
6. Promote Peer Learning: Encourage students to learn from each other. This could involve setting up study groups, online forums, or peer tutoring sessions where students can share their experiences and tips for using the AI tools.
7. Highlight the Benefits and Real-world Applications: To motivate students and help them see the value in learning to use AI tools, highlight the benefits and real-world applications of these tools. Show examples of how they are used in the industry or in research, and discuss the skills students are gaining by learning to use them.
8. Assess Student Adaptation: Along with regular academic assessments, include an evaluation of how well students are adapting to the use of AI tools. This could involve a self-reflection component or a practical task that specifically assesses tool usage.

• Ensure students have equitable access to tools
• Communicate what learning outcome will be measured by the assessment activity
• Communicate if and how the use of AI tools will be considered in the evaluation of student’s performance in the assessment activity
• Provide AND communicate guidelines on ethical implications and possible bias in AI tools
• Provide AND communicate academic integrity policies
• Provide exemplars for students

Here are some suggestions for steps that could be taken:

1. Provide Adequate Training: Ensure that both educators and students are adequately trained on how to use the AI tools effectively. This could involve hands-on workshops, online tutorials, or even a dedicated 'getting started' session at the start of the course.
2. Offer Additional Support: Recognize that some students may struggle more than others with the new technology. Offer additional support for these students, such as one-on-one tutoring, drop-in help sessions, or online resources.
3. Modify Course Structure: Consider modifying the course structure to accommodate the use of the AI tools. This could involve changing the timing of certain activities, reallocating grading weight, or adjusting the pacing of the course.
4. Solicit Regular Feedback: Since this is a significant change, it's important to regularly check in with students to see how they are adapting. Solicit feedback through surveys, informal conversations, or dedicated feedback sessions. Use this feedback to make ongoing adjustments to the course.
5. Establish Clear Communication Channels: With new technology, there can often be unexpected issues or questions. Establish clear channels for students to ask questions, report issues, or discuss their experiences with the AI tools.
6. Promote Peer Learning: Encourage students to learn from each other. This could involve setting up study groups, online forums, or peer tutoring sessions where students can share their experiences and tips for using the AI tools.
7. Highlight the Benefits and Real-world Applications: To motivate students and help them see the value in learning to use AI tools, highlight the benefits and real-world applications of these tools. Show examples of how they are used in the industry or in research, and discuss the skills students are gaining by learning to use them.
8. Assess Student Adaptation: Along with regular academic assessments, include an evaluation of how well students are adapting to the use of AI tools. This could involve a self-reflection component or a practical task that specifically assesses tool usage.

• Ensure students have equitable access to tools
• Communicate what learning outcome will be measured by the assessment activity
• Communicate if and how the use of AI tools will be considered in the evaluation of student’s performance in the assessment activity
• Provide AND communicate guidelines on ethical implications and possible bias in AI tools
• Provide AND communicate academic integrity policies
• Provide exemplars for students