Grading interface?


(Andreas Mueller) #1

It would be great if github classroom would have some support for grading. Looking at Submission and Grading Workflow (Redux) it seems that people are doing per-commit comments. That seems like a really bad idea, since students rarely make sensible commits. Reviewing the whole repo (as a PR) would make more sense (which you would get if you wouldn’t be using github classroom). But if you have jupyter notebooks you’ll look at json, which is not great.

I think using the github interface is not a good match for reviewing assignments anyway. You want the reviews be split up by sub-task so different graders can grade them. And you want a rubric, and an easy way to provide canned answers.
I looked at gradescope, but unfortunately their free tier doesn’t allow programming assignments. Has anyone tried any other solution (or has experience with gradescope)?

Will github classroom provide support for grading at some point?

I looked at https://github.com/Submitty/Submitty which seems promising, but no idea how to integrate it with github classroom.


(Vanessa) #2

@amueller the short answer is maybe. We don’t want to duplicate the LMS (which is designed for FERPA compliance). So as far as where grades live, no.

But as far as plugging in with LTI I’d like to see that feature implemented.

As always, you can check with the Classroom roadmap: https://github.com/education/classroom/blob/master/ROADMAP.md


(Andreas Mueller) #3

Thanks. I’m already using a LMS, and I imagine most schools have their own that they are tied to. I can still use something like gradescope, which is just for grading assignments.

I’m not entirely sure what the role of LTI is. The output only needs to be ids (that are already on the roster) followed by points per assignment in a csv file. That’s enough integration for me. The LMS can handle that.

What I want is a more friendly interface for the TAs to look at the code for grading, and provide feedback. Have you looked at how gradescope works?


(Andreas Mueller) #4

Looking at the roadmap, I’m not sure what the themes mean.
Everything I’d want out of the platform is part of “I want to manage and grade my course assignments easily.”.

For me, coding assignments are:

  1. hand out a starter
  2. update the starter periodically
  3. provide automatic feedback
  4. students submit the homework at a deadline
  5. grade the submission (partially automated?)
  6. possibly provide solutions

Last year, we had 1, and 2 is available via pull-request. 3 is sort-of available, but at much reduces resources because of organizing everything as an organization. Now it looks like you added 4. The next logical step for me would be 5.


(Kari Bancroft) #5

This is definitely not built into github classroom, but one thing that we’ve done is created a custom solution using the GitHub API to be able to utilize a standard rubric for each project and add the feedback with the rubric as a pull request comment once the project is graded.

We maintain a feedback/rubric template in every project repository along with our project requirements. The templates have a standardized naming convention, so within our tool, we pull down that markdown file and populate it with the feedback. When we submit the feedback in our tool (which uses the API) it will add it as a comment to the student’s pull request and track that feedback was provided within our custom tool. This is not at all automated and fully requires feedback providers to review the code in the pull request.


(Matt Price) #6

@kariabancroft do you have code for that tool somewhere we can look at? sounds neat.


(Casiano Rodriguez-Leon) #7

Thanks @mozzadrella and @amueller for arising the question.

I agree that a grading interface is more the job of the Learning Content Management System.
However I believe that Classroom can assists us a lot introducing a slight redesign of the actions taken when the new assignment button is clicked. The idea is to add a new field create teacher’s eval repo .

If this field create teacher’s eval repo is filled Classroom will:

  1. Create a new empty repo assignment-name-eval inside the organization
  2. As soon as a student accepts the assignment Classroom will git submodule add the just created student repo to the global assignment-name-eval repo as a submodule

Advantages I see:

  1. A simple git clone assignment-name-eval clones all the students work.
  2. A simple git push --recurse-submodules=on-demand or git submodule foreach git push origin master will send the teacher changes to the students
  3. A simple git submodule foreach command executes command in all the students repos (For example git submodule foreach run-unit-tests)