Assignment dashboard info is unreliable

GitHub Classroom has been a big help for my class on Python programming for Linguists, but I’m finding myself using it only for assignment distribution and autograding, and I’m shifting to do most tasks offline. In particular, the assignment dashboard info is frequently incorrect in multiple ways, and since it is misleading it is perhaps best to not display it unless it can be more reliable.

  1. The “Not submitted” text never changes even after a student pushes commits before a deadline, but this has been reported already.

  2. Sometimes a student’s submission reports that tests have been run, I see the point totals, and a button to view tests, but other times it looks as though no tests have been run when in fact they have:

  1. Even if it reports that the tests have been run, sometimes it doesn’t report the results of the most recent push. Clicking on the Actions tab of the student’s repository shows a later run.

  2. Related to the above, the number of commits is often incorrect, and this seems independent of reporting the most recent test results. In the image below, the starter repository has 1 commit, so any commits by the student would show 2 and up. The first student has not committed anything. The last student has pushed commits. Both show “1 commit”, but the latter shows the tests passing.

ghc2

  1. A student recently had an issue accepting the assignment (see here). The repo was created but they were not made an Outside Collaborator. I went to the repo settings and invited them again, and they accepted. The dashboard still says that the student “has not accepted the assignment”, but I can click on their username and see it.

All these symptoms seem like there’s some trigger to update the dashboard info that is not reliably firing. May I suggest an additional poll-based trigger (e.g., a cron job) or a manual trigger to refresh all the info?

While we’re on the subject, it would be great if the dashboard could show the time of the latest autograding action. Since the commit time can be spoofed by the student it is unreliable for determining how late an assignment is. My current solution is to navigate to each student repo’s Actions tab and check the time that the autograding action ran.

3 Likes