The Challenge of Assessment

Traditionally, professors have graded the end results rather than a student’s process. AI is disrupting this method because it provides the end results without a student having to take the expected process. Professors are now having to recon with the idea that although the end results are accurate, the student’s lack of investment does not help achieve learning outcomes.

Approach to Rethinking Assessment

Completely reworking all assessments may sound like the solution but it is time-consuming, if not unrealistic, to implement. Instead, Professor Gulya recommends the following a calm approach, using patches and overhauls. Tasks to rework assessments can be divided into two ‘buckets’: patches and overhauls.

A patch is a quick fix to possible vulnerabilities in your assessment. If some deadlines do not allow for a full re-evaluation or if you are teaching multiple classes, a good approach would be to GEM.

G - Ground the assessment in another lesson or module. If an assessment has been compromised in the age of AI, moving the assessment to another module or assignment type can make students form connections between subjects. For example, a written assignment can be turned into an open class discussion.

E - Embedded AI in the activity. A great example of embedding AI was Professor Gulya’s creating an AI chatbot. Using ChatGPT, he created a custom GPT that would ask students very specific questions from the perspective of a young child. The assignment was to talk about a movie of interest and answer each question the bot asked. Students would then receive insightful feedback from the bot. In creating it, Professor Gulya included very specific boundaries to ensure the GPT would not directly tell students the answers.

M - Multimedia. Assessments like discussion boards can shift from being text-only to engaging students with videos if they are comfortable.

An overhaul is a completely new approach to an assessment. Overhauls are hard to manage and time-consuming so it is recommended to do 1-2 per semester. When considering an assessment overhaul, some big questions are: • How can I prioritize in-demand skills? • What do I want my classes to look like? • How can I create a better learning space?

The overhaul process might involve:

Alternative grading Because students are incentivized by grades and know that professors are often more focused on results, they are likely to cut corners to achieve better grades. By de-emphasizing grades and grounding the assessment in the process, students can focus on learning and less on the product.

Transparency Students should have the opportunity to disclose AI use and professors should provide clarity on what levels of AI use are permitted within the course. A usage scale can help students understand the ethical component of AI.

AI usage scale by Leon Furze

Metacognitive Exercises A process-based grading system rewards creativity and reflection. Students need the opportunity to reflect on their assessment to promote true learning.

For more information about GEM or rethinking assessment, Professor Gulya has provided the QR code