How to Set Up Autograding in GitHub Classroom? (with Canvas)

How many repo will be created?

The naming scheme is very confusing when you first use GitHub Classroom. I want to be as clear as possible here 😬.

There are exactly 3 repositories involved:

1. {template repo}
2. {your classroom name}-{assignment title}-{template repo name}
3. {assignment title}-{github user name}

Suppose you use the following values:

• Template repo name: hw-starter
• Classroom name: isdn3000e
• Assignment title: hw1-cpp-basics
• Student’s GitHub username: XingxinHE

Then the 3 repos will be:

1. hw-starter
2. isdn3000e-hw1-cpp-basics-hw-starter
3. hw1-cpp-basics-XingxinHE

What are the relationships among these 3 repos?

Ideally, we just want to create the assignment repo. However, the Github classroom requires that an assignment is built from a template repo. That’s why we must create hw-starter first.

Once you create the assignment in the dashboard, the assignment repo (isdn3000e-hw1-cpp-basics-hw-starter) is automatically created by the GitHub bot. It squashes all the commit history from hw-starter and initializes this new repo.

For example, if hw-starter has 6 commits, they will be combined(squashed) into a single initial commit in the assignment repo.

Once a student accepts the invite link, Github automatically creates a repository inside your organization for them. This repo (hw1-cpp-basics-XingxinHE) is a fork of the assignment repo (isdn3000e-hw1-cpp-basics-hw-starter), NOT the template repo (hw-starter). This is very important!

Can students see other repos in your org?

The short answer is no. After the student accepts the assignment, their repository (hw1-cpp-basics-XingxinHE) is created inside your organization. The student receives an email inviting them to be an outside collaborator for their repository only. They cannot see the template repo, the assignment repo, or any other students’ repositories.

Autograding system

At first, the autograding options in the UI can be confusing. What do these tabs mean?

The simple answer is that the “GitHub Preset” in the UI is just a helper tool that generates a .yaml file for GitHub Actions. Since that’s all it does, it’s much better to just learn the syntax of the autograding .yaml file yourself.

The autograding system relies on three specific Github Actions:

• autograding-python-grader
• autograding-command-grader
• autograding-io-grader

These corresponds to the three grading methods provided by Github Classroom. I’ll use the autograding-command-grader as an example.

name: Autograding Tests
'on':
- push
- repository_dispatch
permissions:
  checks: write
  actions: read
  contents: read
jobs:
  run-autograding-tests:
    runs-on: ubuntu-latest
    if: github.actor != 'github-classroom[bot]'
    steps:
    - name: Checkout code
      uses: actions/checkout@v4
    - name: 'Task 1'
      id: task-1
      uses: classroom-resources/autograding-command-grader@v1
      with:
        test-name: 'Task 1'
        setup-command: "./scripts/build_for_grade.sh"
        command: "./scripts/grade_task1.sh"
        timeout: 10
        max-score: 40
    - name: 'Task 2'
      id: task-2
      uses: classroom-resources/autograding-command-grader@v1
      with:
        test-name: 'Task 2'
        setup-command: "./scripts/build_for_grade.sh"
        command: "./scripts/grade_task2.sh"
        timeout: 10
        max-score: 60
    - name: Autograding Reporter
      uses: classroom-resources/autograding-grading-reporter@v1
      env:
        TASK-1_RESULTS: "${{steps.task-1.outputs.result}}"
        TASK-2_RESULTS: "${{steps.task-2.outputs.result}}"
      with:
        runners: task-1,task-2

• The setup-command acts as your build command (i.e., whatever needs to run before the test_.
• The command is the actual grading command. Because this is a command-grader, the exit code determine whether the test passes or fails. If the script returns 0, the test passes and the student gets the points✅.

If this is a C++ assignment, your build_for_grade.sh might look like this:

#!/usr/bin/env bash
set -euo pipefail
 
cmake -B build
cmake --build build

And if you are using GoogleTest, your grade_task1.sh could look like this:

#!/usr/bin/env bash
set -euo pipefail
 
./build/test_assignment --gtest_color=yes --gtest_filter='*YourTestName/*'

Get the client ID

Write an email to the Canvas team. Here is an email template you can use:

Dear Canvas Support Team,

I am writing to request the Client ID required to integrate GitHub Classroom with the course, {your course name}.


We are using GitHub Classroom to manage coding assignments and would like to link it to Canvas to sync our student roster. According to the GitHub documentation, this integration requires a Client ID that was generated when HKUST registered its Canvas instance with GitHub.

https://docs.github.com/en/education/manage-coursework-with-github-classroom/teach-with-github-classroom/connect-a-learning-management-system-course-to-a-classroom#linking-a-canvas-course-with-a-classroom

I am wondering if it is possible to have the Client ID for the approved GitHub Classroom LTI tool?

Sincerely thanks,

{your name}
Teaching Assistant of {your course name}

Add the app to your course

Once you receive the Client ID, go to your course page in Canvas. Navigate to “Settings” > “Apps” > ”+ App”.

Fill the Client ID provided by your Canvas support team.

Register your learning management system

Navigate to register-lms . Fill in the following information:

• LMS Type: Canvas
• Issuer Identifier: https://canvas.instructure.com
• Domain: https://canvas.{your university domain}
• Client ID: {your client id, e.g. 2694000xxxx}
• OIDC Authorization end-point: https://canvas.{your university domain}/api/lti/authorize_redirect
• OAuth 2.0 Token Retrieval URL: https://canvas.{your university domain}/login/oauth2/token
• Key Set URL: https://canvas.{your university domain}/api/lti/security/jwks

If the client id is valid and the access is configured correctly, your Canvas course homepage should now feature a GitHub Classroom button.

From there, you can click the link, sync your roster, and add your students!