How to give positive feedback by mail after "autograding"?

Note: I have asked this before on SO: https://stackoverflow.com/questions/61363648

We are experimenting and evaluating GitHub Classroom and GitHub Actions for “autograding”.

In our assignment students get points if unit tests are passed. So the autograding.json is comparably simple running mvn test -Dtest=testClass#testmethod.

In the default setup I am unhappy with the following points:

  • GitHub (Actions) sends you a mail if the build (i.e. test) fails, but you do not receive a mail, if the test succeeds. It would be very nice if students get a feedback when they pass the autograding (i.e. unit test). How can this be done?

  • GitHub (Actions) does not provide any details in the mail, you have to visit the actions page. It would be nicer if you could supply the number of points and maybe some one-line explanation of the error. How can this be done?

To illustrate what I would like: it would be nice if the JSON allows to specify a feedback message upon success and a feedback message upon failure and the GitHub Action mail should contain this text (if that is not possible, there should be a nicely formatted web-page displaying this text). Unfortunately I did not find any documentation on the JSON. Is there any documentation of the autograde.json?

I am also interested in other options.

Here is an example for the autograde.json currently used by us:

{
  "tests": [
    {
      "name": "Unit Tests - Testing your Application: DoubleVector - BasicFunctionality",
      "setup": "",
      "run": "mvn -q -B -Dtest=DoubleVectorFromArrayTest#testBasicFunctionality test",
      "input": "",
      "output": "",
      "comparison": "included",
      "timeout": 10,
      "points": 0.5
    },
    {
      "name": "Unit Tests - Testing your Application: DoubleVector - Accuracy",
      "setup": "",
      "run": "mvn -q -B -Dtest=DoubleVectorFromArrayTest#testAccuracy test",
      "input": "",
      "output": "",
      "comparison": "included",
      "timeout": 10,
      "points": 0.5
    },
    {
      "name": "Unit Tests - Testing your Application: QuadraticEuation",
      "setup": "",
      "run": "mvn -q -B -Dtest=QuadraticEuationFromCoefficientsTest test",
      "input": "",
      "output": "",
      "comparison": "included",
      "timeout": 10,
      "points": 1
    }
  ]
}

Hey @cfries!

Thanks for the feedback!

We unfortunately can’t modify the GitHub Actions emails, those are all handled by the folks working on that product. There is some configuration here that might improve your experience though: https://github.com/settings/notifications. Notably, there’s a setting under GitHub Actions to send emails for runs that didn’t fail.

We’re considering having our own GitHub Classroom mailers that tell relevant information such as when your students pass tests, accept assignments, etc. These would have more relevant context than the out-of-the-box GitHub Actions emails. I’ve noted your feedback and hopefully we’ll get to implementing this soon!

Thanks!

Nathaniel
GitHub Classroom

Thanks Nathaniel for your reply. With respect to the GitHub Notifications. If I understand this correctly, this modifies the notifications for the current account (here: my account) and not for the students (forking the template repo and doing the assignment). Right?

It it possible to configure the notification behaviour via a YAML file in the assignment repo? I did not find that information? Where is the place to ask these questions (since they are more related to GitHub Actions)?

@cfries That is correct, yes, your students will have to manage their own email notification preferences.

It isn’t possible to configure notification behavior via a YAML at the moment. The best place to bring your questions for GitHub Actions would probably be https://support.github.com/. There’s also the community forum (like this one, but for GitHub.com) here: https://github.community/

© 2017 GitHub, Inc.
with by
GitHub Education