Improving autograde tests + ordering

I’m using GitHub education assignments and autograding for my programming assignments. On the whole, it’s a wonderful service. It gives the students near instant feedback and saves time on manual grading tasks.
I end up creating 50+ tests in each assignment, some of which end up building on previous tests. For instance, test 1 might produce output that test 2 uses as input.
I get into trouble when I want to make corrections in the test set, insert tests in the flow, or remove tests. There’s no way to reorder the tests in the autograding.
Can I make this into a feature request somehow? Anybody have ideas how to create a more flexible test regimen?

Tell us more. What programming language and test framework are you using? Are you running unit tests or system tests?

Hello Zack,
I am using C for my assignments. The tests are primarily end-to-end system tests. Input/output tests dominate and in order to create complicated testing scenarios, I can’t use a single input/output test to accomplish it. I use the autograding test framework provided by GitHub classroom - the add tests feature that you can choose when creating a new Classroom assignment.
The tests end up being GitHub Actions that are autorun every time a student pushes or whenever I update the assignment test suite. The tests are stored in a yml file, but I never directly edit the yml file. I just use the Classroom ‘edit assignment’ screen.

I suspect the best solution would be for you to just write the Actions yourself.

For my assignments (which are digital logic and assembly), I take an old .github directory from a previous project and modify the .json and .yml files until they do what they want. I don’t use any of the Classroom-provided functionality.

Unfortunately, my knowledge of GitHub actions is quite limited, so I don’t think I would be much help offering specifics for your situation.

Feel free to check out one of my recent assignments: cis351-functions-lab/.github at master · kurmasz-assignments/cis351-functions-lab · GitHub

© 2017 GitHub, Inc.
with by
GitHub Education