Using JS unit tests to give student feedback & facilitate grading?


(Matt Price) #1

Hi Everyone,
After a couple of years of thinking about it, I am moving my first course to github classroom this year! The course is called “Digital History” and includes a few very small introductory coding assignments. My students are mostly undergraduate history majors at a canadian university. Most of them have pretty low confidence and base skill levels when it comes to any STEM field, so the technical aspects of the coursework are intended to foster confidence. Last year’s website is here, and the initial assignment is on Github here.

I’d like to add tests to this assignment and use them in two ways:

  • For my own grading purposes, I’d like to hook the tests up to Travic CI so I have a good sense of how the class is doing before I sit down to grade assignments. I might use this to semi-automate the marking process, too (though I’m not sure what the most efficient workflow is for this)
  • However, I’d also like the students to be able to check their own progress and receive useful feedback about what’s going wrong in their code. I guess I would have to set up a test harness html page and point them to that page somehow.

I’d like to make this whole process as frictionless as possible both for myself and for the students. In past years I have given a lot of direct support to students, but this year I am teaching a larger class and have a heavy courseload, so I won’t be able to give as much individualized support.

That was a long roundabout way of saying: what test frameworks and solutions have other people come up with So far I have been experimenting with mocha and chai, and they work fine for testing standaline Javascript code. However, I am a bit stuck getting them to run tests of a local HTML file with embedded Javascript. For instane, I would like to do some simple things like:

  • test to see if a particular link has been added to index.html (the very first step in the student assignment is to add a link to a profile page)
  • test whether a script run in index.html changes the content of a <td class="someclass"></td> from Some Name to <a href="https://en.Wikipedia.org/wiki/Some Name">Some Name</a>.

To do either of these things, I need to get access to the DOM in a local web page (which by default would be served off of a file:// URL, to an http[s]:// one) and in the second case the test has to wait for some Javascript to load before inspecting the DOM. I’v ebeen trying to figure out how to od this in mocha but have not had any luck.

Finally, if at all possible I would like to have the same set of tests running in Travis, as run in a web page where the students can inspect the test results themselves. This would ensure that the students and I always see the same results.

Does anyone have any experience with this kind of thing? I could really use the help! thank you!


(Dan Wallach) #2

Sounds like you’re looking for a “headless browser”, which allows you to write tests that can run “in the cloud”, asserting properties of the DOM. See, for example: https://blog.logrocket.com/introduction-to-headless-browser-testing-44b82310b27c


(Joel Ross) #3

I’ve spend the quarter creating automated tests for assignments in my Web Development Course. You can see examples of exercises and their test suites at https://github.com/info343b-a17 in each exercise repo–for example, exercise01 includes a simple “make an HTML page” exercise along with tests to verify it.

As mentioned above, in order to test HTML code I’m using a virtual DOM library (specifically cheerio); this lets you take the file’s HTML content, load it into the DOM, and then query it just like you would with jQuery.

While I’m using Jest as a test runner (since my course covers React), another runner like mocha would work just as well. As long as your repo includes a .travis config and a package.json with a test script and dependencies, pushing the code to GitHub will cause the tests in that repo to run perfectly fine on Travis.

Note that my course has students running tests directly on the command-line (since we’re teaching them CLI and npm and such). For a GUI-based version of testing, you might check out https://github.com/thomasjbradley/markbot (which was linked from this forum a while back), which could be easier both for your students and for you.


(Matt Price) #4

@joelwross, this is very helpful, thanksk. Markbot is neat but a little hard for me to figure out – it’s not obvious to me how to customize the tests. I have been writing CLI tests with mocha & jsdom, and then rewriting those tests using mocha in the browser, so students can get feedback directly from a local web page rather than having to wait for travis to compile (I understand from @thomasjbradley that slow travis becomes an issue close to submission times). So it’s a pretty similar approach ot yours, but not as far along yet. It’s great to have your examples!

@thomasjbradley actually replied to an earlier thread of mine, from a year ago – I’d forgotten! – so it was nice to revisit his comments.

I’ll try to remember to check in here as I go along. Appreciate everyone’s help.


(Matt Price) #5

(also @joelwross your assignments are great and sort of ripe for the stealing…).


(Joel Ross) #6

Feel free! Glad to share if it helps :slight_smile:


(Steve Mattingly) #7

As it happens, I was just working on something like this for one of my courses next semester. What I have so far is much simpler than some of the other helpful replies, but it meets some criteria that were important to me, and maybe to you, too.

  1. It is completely client-side, and runs in an ordinary browser with no additional installs or set up.
  2. It shows test results that students can easily view, and easily copy/paste into a markdown file to submit via GitHub. (I want them to give me evidence that their work is correct.)
  3. It does not assume that the JavaScript under test is HTML-DOM related. (I am teaching some introductory classes using game programming in JS with a game engine. The students are writing object-oriented code but not (yet) learning anything about HTML or DOM.)

This was created within the last week or two, so it has not been thoroughly tested. I literally just put an example/demo onto GitHub at https://github.com/smattingly/client-side-js-tests.


(Matt Price) #8

Thanks @smattingly, and sorry for the longish delay. Your work, like that of @joelwross and @thomasjbradley, has been very helpful to me. I ended up writing mocha unit tests and relying on npm scripts to run them (I tried for quite a while to get the tests to run inside a web page, but it was so much harder to get that to work that I gave up).

The tests are written in mocha with the chai assertion library, and I use the mochawesome test reporter to generate a pretty html output… They rely on a combination of cheerio and jsdom to do DOM testing. The code quality is still ironically pretty low; I had to learn a lot in order to be able to write the tests at all!.

I’m a little anxious about the ability of my stuents to install node and NPM, so I’ll see how that goes this semester. In the future I would love to have an electron app that was maybe a lightweight version of @thomasjbradley’s Markbot: I guess the idea would be that I’d load all the NPM dependencies into the app, and have a set of spec files that only contained the tests. As you can see, there’s a lot of repetitive code in my asignments that I’m not looking forward to maintaining in the future.

My first 3 assignments are now finished (phew), and I expect the rest of then to go a lot quicker. They’re online here: https://github.com/DigitalHistory . I’m in the process of converting the website from wordpress to Hugo (not as awful as it sounds, since the source files are all written in emacs org-mode). But last year’s syllabus &c is linked to above somewhere if you’re interested in seeing the course.

Thanks everyone!


(Thomas J Bradley) #9

@titaniumbones Node & NPM installation and maintenance on student computers (and Terminal use) was my major reason for creating Markbot too. I’m sorry we couldn’t make Markbot work as you’d hoped—I think it’s almost too tailored to what and how I teach.


(Matt Price) #10

(@thomasjbradley I meant to respond to that and say, I don’t think the problem is with your code at all, just that I was barely able to each myself unit testing in December – electron app programming wasn’t gonna be in the cards!)