Hi everyone, I am Edison from China. We are building an open-source course in Golang which about (distributed database system) on Github and trying to use Github Classroom to manage the works of participants. Inspired by the work of Omar Shaikh, We use Travis-CI to check the correctness of the student’s work. But here come questions:
Here is the basic workflow:
- Participant’s repo will be managed in the same organization, and Travis-CI will trigger a job for each repo on push.
- After job complete, by using Travis-CI Webhook and other API, our service can grab the job’s log and calculate the score by parsing it, and store in a database.
- We send an email to notify the owner of the repo that his score is stored.
But after rethinking this workflow, we find some weakness.
- For each replica of the student’s repo, all test cases will be delivered. They can remove hard test cases to pass the CI. this is cheating. Is there any way to prevent this?
- We think maybe we need to fetch another codebase to run the test cases. For each Travis-CI Job, we fetch our complete codebase first, then copy the student’s file for which need to be filled into it, then run the test cases in our codebase rather than student one. But .travis-ci.yaml is also delivered to students, they can modify it to prevent the fetch of the codebase.
We want to know, is there anything we are missing by using Github Classroom and Travis-CI? Do you have any idea about how to prevent the cheating methods mentioned above? Any comment is preciated.
If you can share your workflow, it’s much more helpful for us.