Automatic Grading Script


(Ahmet Arslan) #1

Hello GitHub Education Community,

I am teaching two Java programming courses at the Department of Computer Engineering, Anadolu University, Eskisehir, Turkey.

I came across GitHub classroom system (cool stuff by the way) and started to distribute and collect programming assignments for both of the courses. Furthermore, the system inspired/encouraged me to introduce an automatic homework grading system. The classes are crowded (122+87=209) and it is easy to resolve the individual homework repository names by supplying a repository prefix in the form of $CourseCode-$hw#

Basically, for a given GitHub username, the automatic grading script does the followings:

  • resolves/constructs the repository URL ($CourseCode-$hw#-$GitHubUserName)
  • checks if the URL exists (not all students accept the invitation and do their homework)
  • clone corresponding homework repository
  • build the repository using Apache Maven (I usually supply valid pom.xml and configure necessary maven plugins in the starter code)
  • run the program with certain inputs and save the result in a text file
  • compare the actual result file with expected results by using grep (in a similar way to JUnit tests, but the students cannot see the test variables and their expected results. So, students cannot shortcut)

I am inclined to use a started code, with a readme.md file explaining the homework, and a valid pom.xml file to save/isolate students from burden/complexity of Maven. I also include some JUnit tests, (not for grading) but for demonstrating how the program should work. I publish the grades on a Web server, as well as the dump (stdout and stderr) of the grading script so that students can have some clue why they get zero etc. For example, when mvn package fails for some reason, it can be deciphered from the dump file. I allow a few iterations to fix mistakes for encourage purposes.

I have given 4 assignments, three of which have public starter code:

I am including corresponding evaluation scripts to share my experiences with the community. Please feel free to use starter codes or scripts, which are licensed under the Apache 2.0 license.

I really appreciate your feedbacks and comments.

Thanks,
Ahmet


:wave: Teachers! Introduce yourself here
:wave: Teachers! Introduce yourself here
(Scott Sanicki) #2

Hi Ahmet,

Thanks for sharing this! I’ve moved your post from the introductions thread to it’s own topic for improved visibility. Feel free to make another post to say hello to the community.

Cheers,
Scott


(Michiel Cuijpers) #3

Hi Ahmet,

About Higher Grades for Better Code we have created this post. In your case you could use this as bonus points for non-functional quality aspects. Hope this helps in your course.


best regards from the BetterCodeHub team.
Michiel