Continuous integration for each submission

(Andreas Mueller) #21

Coming back to this, it seems there is still no solution that allows for using the students’ CI budget to be used, and I won’t be able to use (a sufficient amount) of CI with classroom, right?
I thought that I could avoid using classroom by allowing students to fork and PR, but it looks like it’s not possible to fork a public repo into a private repo easily. I guess forking a private one into a private one might work?

(Andreas Mueller) #22

This time around Travis is not giving me 5 concurrent instances (which they had provided before as a courtesy) and that makes github classroom basically unusable. I had them run trivial tests with pytest (like checking that 1 / 2 is .5 in Python3) and some students waited 10 hours for a single run. The build-matrix was of size 3 and I think each run took only 10 seconds (but I didn’t find a way to enable auto-cancellation for all repositories).

(Dan Wallach) #23

The test suite in my class grows each week, ultimately taking 1-3 minutes to run, depending on the vagaries of Travis. (The same test suite runs in about 6 seconds on my local machine.)

With 160 students, we saw Travis backups of 3-4 hours around deadlines, given only a single concurrent instance. Not great, but tolerable. I cautioned my students to run their tests locally, and that Travis isn’t meant for real-time feedback.