Private Test Suites

For the purposes of grading, I would like to have a private test suite for each assignment. I would like to run this suite via Travis CI each time the student commits. The idea is that the students should be able to see how many tests they are passing/failing but not the actual contents of the tests (to encourage them to write their own tests). Is anything like this possible using GitHub Classroom?

3 Likes

Hi @pzp1997 ,
A couple of fellow teachers seem to make good use of Travis CI alongside the private repositories that github provides.

Check out this post, or even this discussion. Both should be able to give you a few more insights :wink:

Hi @pzp1997,

we are facing a similar issue to the one you found.

We have some unit tests that we give to the students with their homework, but we would also like to have some (private) tests for grading purposes (that the students shall not know).

If your tests where to be in a public repo, then you can simply clone it in the .travis.yml script (but any student could get to them):

If your tests are in a private repo, then it seems that you need to define it as a submodule of the “homework” repo, but then it is not private anymore to the students. Here you can find some clues to solve GitHub dependencies in Travis CI: https://docs.travis-ci.com/user/installing-dependencies/ but I guess that none of them solve your (and mine) privacy requirement.

If you find a solution, please let me know.

Best,

Jesus

@mozzadrella Any ideas? This feature is a must have for us and it sounds like it might be for others as well.

@pzp1997 we :ear: you. Configuring folders with private tests is on our roadmap (theme 2).

4 Likes

Please @mozzadrella keep us informed on that!!

1 Like

I did this in my class the past two semesters.

The trick was that in the student’s repository, I have a placeholder “graded tests” file. Then as part of the Travis CI build script, I download the actual graded tests file from my web server, and overwrite the placeholder file. To make sure that students cannot download the graded tests file themselves, I setup the .htaccess file on my web server to only accept IP addresses from the Travis CI list (here).

Once the graded test file downloads, it can compile and run the unit tests as normal (this was in a C++ project, but it should work for any such project).

4 Likes

Nice trick @chalonverse indeed… I could not figure it out

My apologies for reviving an old thread. @chalonverse this seems like a great approach, but how do prevent the students from, e.g., just doing cat downloaded-tests in the build script referenced by their .yml file?

Well, since that there would be a history of that change on GitHub, hopefully that would be enough of a deterrent from students trying to do that.

Thanks, that makes sense.

© 2017 GitHub, Inc.
with by
GitHub Education