How to anti cheating for student's repo?

Hi everyone, I am Edison from China. We are building an open-source course in Golang which about (distributed database system) on Github and trying to use Github Classroom to manage the works of participants. Inspired by the work of Omar Shaikh, We use Travis-CI to check the correctness of the student’s work. But here come questions:

Here is the basic workflow:

  • Participant’s repo will be managed in the same organization, and Travis-CI will trigger a job for each repo on push.
  • After job complete, by using Travis-CI Webhook and other API, our service can grab the job’s log and calculate the score by parsing it, and store in a database.
  • We send an email to notify the owner of the repo that his score is stored.

But after rethinking this workflow, we find some weakness.

  • For each replica of the student’s repo, all test cases will be delivered. They can remove hard test cases to pass the CI. this is cheating. Is there any way to prevent this?
  • We think maybe we need to fetch another codebase to run the test cases. For each Travis-CI Job, we fetch our complete codebase first, then copy the student’s file for which need to be filled into it, then run the test cases in our codebase rather than student one. But .travis-ci.yaml is also delivered to students, they can modify it to prevent the fetch of the codebase.

We want to know, is there anything we are missing by using Github Classroom and Travis-CI? Do you have any idea about how to prevent the cheating methods mentioned above? Any comment is preciated.

If you can share your workflow, it’s much more helpful for us.

Hi @JinLingChristopher

This is actually a vulnerability yes :+1:

Whenever the test suite is delivered to the user’s repo along with the codebase then a maliciuous user may manipulate the test, at his own risk of being caught anyway :smirk: (we’re talking about students who can be serioulsy punished for cheating).

To get around this, one possibility is what you suggest. We’re doing right this with our grading system where we run tests on a dedicated server that verifies students’ solutions downloading the test suite from a proper endpoint.

To any rate, I aim to explore the opportunity to use GitHub Actions in this respect (in place of Travis CI). Apparently, the user needs to be admin to manipulate the test recipe located in the repo. This way, we would simply avoid giving students admin rights and we’re done!

Interestingly, consider that GitHub Eduction has been developing its own auto-grading system (still in beta) based on GitHub Actions.

1 Like

I’m afraid I was wrong about that.
From this table, it stems that anyone with write permission can edit GH workflows :worried:

© 2017 GitHub, Inc.
with by
GitHub Education