Module 3 Exercise 3: Assessing collaboration


(Vanessa) #1

3.4

Exercise: Assess a live project

  1. Find an active open source project.
  2. Point to 2 pieces of evidence that the project is active.
  3. These pieces of evidence can be:
  • How quickly pull requests are reviewed
  • Data in Insights
  • Number of forks or stars
  • Some other metric :slight_smile:

Reflect on how you might use these insights to assess collaboration in group work.


(Clara) #2


Is active:

  • plenty of stars and forks
  • recent commits
  • insights is busy
  • PRs reviewed, documentation, tests, pretty code coverage reports

Takeaways for student code: for me, the insights page - particularly commits and contributors - is very helpful for a high-level view of who did what.

For students, encouraging students to write meaningful commit messages helps me give them the credit that’s due. Discussion in PRs and issues also very useful for assessing group work.

I will never deal with grading group projects without GitHub again :slight_smile:


(François Roland) #3

When looking at GitHub Classroom project, we see plenty of forks and stars.
We can also see in Insights that there are new commits consistently.

On the other hand, if we dive into the contributor list, there is clearly a single person maintaining the project during its whole lifetime and then other people giving a hand from time to time.


(Richard) #4

Ok, so I chose Atom (had to be a GitHub product, right? :smiley: ):
The most obvious evidence is from the Pulse overview for the past week, 13 pull requests closed, 45 issues closed. Decent numbers.


The second piece of evidence is from the Contributors view, but not the contributors themselves, but rather the commit group which shows the consistency of commits over an almost seven year period.


The first of these is likely to be more useful in student work than the second on a small scale, but for me the mist useful part of using git(GitHub) for student group work is to show that they are working consistently thought out the project lifetime and not just cramming in the work at the last minute.


(Mike Crabb) #5

The project that I looked at was ORCiD Github Repo and is the open source backend of the ORCiD ID project.

The first sign of evidence that this repo is active is the huge number of commits that they have…17,243

On top of this, the commit history shows a good level of commits over the last few years.


and the contributions tab shows a lot of activity by different users (I’ve only included the top row in the screenshot below.

Reflection on Use in Group Work

I tried something like this last year with a web development module that I was running where students had to submit a screenshot of the contributions page at the end of the module and then explain why it looked the way it did. This forced students to think about the roles that they were carrying out and gave a good indication for groups where all students were contributing equally and also groups where one student was doing all of the work.


(Omal Perera) #6

I found facebook’s react-native as an active open source project.

Let’s analyze some of its evidence, one by one.

Stars, Forks

Commit frequency, Pull Requests & Issues

(2017-mar-05 to 2018-feb-25)

For the past month

Releases


(James ODonnell) #7

This screenshot of the “insights” from the last month indicate that the biopython project is an active undertaking by a small group of specialists. In the last month there were 8 issues resolved and an additional 10 new issues submitted. There were 6 active contributors. The fact that old issues are being resolved on roughly the same scale that new problems would suggest that problems are being resolved as they are found.

The over-all level of activity over the long haul has been high, with lot of contributors


(Randy Johnson) #8

I chose to look at https://github.com/tidyverse/tidyverse.

The repository has 16 contributors and was last updated about 1 month ago. When looking at the Contributors portion of the Insights tab, it looks like there have been periods of active development followed by periods of less activity – probably because the package is fairly stable at this point. There has been one major contributor (the repository owner) and 15 other minor contributors (looks like their contributions were all via pull requests). There are 4 open pull requests and 23 closed pull requests – all but 3 of the pull requests were accepted.

It looks like there are a lot of good metrics for tracking who contributes to the project. It might also highlight groups that aren’t communicating well (e.g. if there are a lot of pull requests that are rejected because someone else in the group had already done the work).


(Nasseef Abukamail) #9

I chose one of Atom’s repositories. It had many contributors. It’s great to see who contributes more often and who doesn’t. This is going to be helpful in classroom setting.


(Derek Land) #10

I go for https://vaex.io/ ( https://github.com/maartenbreddels/vaex )

evidence:

  1. recent commits and releases
  2. active discussion in issue section

Tricky thing to look at just the numbers. Is a commit with 1000 lines better than 1000 commits with one line of code?
For collaboration as part of a grade I prefer to not only judge on statistics but also on feedback from the students themselves.


(Vanessa) #11

Hi @nasseef what in particular did you notice about the activity in this project that made it successful with regards to collaboration?


(Vanessa) #12

:sparkles: :100: :tada:


(Vanessa) #13

@froland you will see @Kadaaran doing a lot of heavy lifting in that repository, in addition to new faces soon :sparkles:


(Benjamin Soltoff) #14

I chose to examine broom. From the insights tab I can see there are several active issues and pull requests. There are also hundreds of stars and forks, indicating the project is active and frequently utilized (not surprising, as it is a major package in the tidyverse).

While the majority of the commits are written by a handful of authors, there are also at least 20 authors with only 1 or 2 commits. These probably come from pull requests for minor changes to the codebase, indicating project managers are open to incorporating improvements from a wide userbase.

This could be useful to assess collaboration in a class group setting, perhaps by examining frequency of commits from different students, activity responding to issues and pull requests. Of course, this also depends on how frequently students stop and commit file changes. One commit could be extremely minor or containing major changes to the codebase depending on if the student follows best practices. It would still be better to examine the underlying codebase and commits/PRs independently for each student (if this is feasible for the class size).


(Evan Misshula) #15

I really love the Codeworld project. It teaches functional programming through middle school math. Chris Young has most of the commits and it seems like the only people who contribute a large number of commits do it during the summer. It does have 732 stars and 104 forks.

I want to get myself on the list of contributors myself. :slight_smile:


(Miren Berasategi) #16

I chose the pandoc project (a universal markdown converter) which I happen to use a lot :slight_smile:

Evidence that it is active:


Last commit 19mins ago.


Huge, increasing and consistent amount of contributions.


Activity for last week in terms of pull requests and merges is, well, quite active.

I can see how this allows to get an overall view of how students would be working and contributing to group repositories, but I still do not have a clear idea on how to use this for grading…


(Paola Corrales) #17

I chose one of the R-Ladies repositories: https://github.com/rladies/starter-kit

This repository contains the guidelines to setup a new r-ladies chapter. It is not a code development project in itself but it needs the contribution of many people.

At first sight you can see that have many contributors (33!) and the latest commit was 5 hours ago.

In the last year, the contributions are very constant and are increasing in the last weeks:

This info would be useful to see how the groups are working and who isn’t participating via the contributions graph. But it’s true what you says above, you can commit 1 line or 1000 lines and are count in the same way. So, It’s important to look the individual contributions in detail.


(Ricardo Martin Marcucci) #18

I choose https://github.com/photonstorm/phaser repository. This is a HTML Game Engine.
From Insights, you can see that the project is in active development with 25 merged pull requests in the last week and 30 closed issues.

From timeline, you can see that it has been active since the beginning, with a peak in the last months.

The resume about how much commits a student has made and how much lines has added is very useful to know which students participate and contribute to the team work.


(Vitor Rios) #19

@mozzadrella, the thread title says exercise 3.3, but this thread is for exercise 3.4

I chose RStudio, and IDE for the R language. The project is quite active, with 4 authors merging 20 pull requests in the last week. The numbers of oforks and watchers also show high community engangement


(Tess Homan) #20

Hi,

I chose scikit-image because I wanted to see what was out there on GitHub related to image processing. I found it because it has many many stars, and then realised that it is still used a lot. The number of commits is not as high as when the project was started in 2010, or during the big adjustments in 2013 and 2015, but there is a steady amount of code being added of deleted in-between these peaks, up to today. So I guess bugs and small changes are still being made.
For the group work I will check that every person has a constant stream of commits, and if this is not the case, ask if they actually worked together in “real life” :slight_smile:
I will also try to force them to work on the project steadily and not just work all night before deadlines.