Keep tests for autograder secret

kurmasz · April 29, 2020, 12:42pm

What are the best ways to keep “secret” tests secret when using autograding with GitHub Classroom?
In other words, what do you all do to keep students from being able to see the tests that are run by the autograder workflow?

I have two main areas of concern: (1) I don’t want the students to see the source code for the tests, and (2) I don’t want the students to be able to ascertain the test inputs by putting print statements in their code. Ideally, (3) students wouldn’t be able to run my tests locally; but, that is a lesser concern at the moment.

My primary concern at the moment is Java. But, in the fall, I’ll also need a solution for C.

I know that there are some new features coming. I’m just wondering what techniques you all have developed in the meantime.

mhadaniya · May 7, 2020, 5:04pm

I don’t keep the tests in secret to my students. I explained to them where tests are, how to run and encourage them to write their own tests.

Until now, I don’t have to worry about it. As we know, not all students do the homework fairly, so I got some repos changing the tests, but are really few. In the revision, I do the comments, so, even the test pass, it will not be graded.