MeerCat, an IDEAS-ECP project

MeerCat houses a set of tools to support users, developers, and managers of ECP code repositories on GitHub.

It is one component of the larger UO-IDEAS project (GremCat).

MeerCat's Focus is on a GitHub Pull Request

As a repo changes over time, there is a need for diligence that a change will not cause a misalignment with documentation, tests or the general code base. Typically this falls on a conversation between a Pull Request (PR) author and reviewer to hash out potential problems. Also, typically, there are Continuous Integration (CI) tools to support the PR process including automated running of tests (unit and regression) and code quality checkers. MeerCat is a complement to these CI tools as they relate to Pull Requests. It monitors GitHub events and goes into action when it sees the creation of a Pull Request.

MeerCat Pull Request Support

MeerCat has grown ground-up, talking to developers involved with ECP projects about their needs. While we are interested in supporting both PR authors and reviewers, it is reviewers that we have tended to focus on. As one might expect, their ideal is to see a final PR that has been well designed and thought out by the author. Thus, MeerCat's rationale is that by supporting the author, we meet our end goal of supporting the reviewer. Before we list the author tools we provide, there is a big caveat: we have found that most ECP projects have their own unique requirements and workflow: there is no one size that fits all. Hence, we allow a project to turn on or off features of MeerCat as they see the need.

• Code documentation support

  If a project uses formatted comments in code to help both document the code but to also auto-generate user documentation, MeerCat can help. It can scan all the files of a PR, looking for either missing or incomplete formatted comments or comments that are misaligned with the code. This currently includes formats such as Doxygen, Robodoc, Numpy, Google and Doc++. MeerCat provides an editor that is smart about these formats and can help an author edit their files to bring code documentation into alignment with both the code and project requirements.

• Project documentation support

  Typically a project maintains (manually) types of overview and user guides in markdown-style documents. While MeerCat does not currently attempt to automatically find misalignment from a code change to one of these documents, it may be able to remind an author that a changed code file appearing in a PR could likely require a change to a documentation file, and that file does not appear in the PR. In essence, MeerCat keeps a mapping between certain code files and certain document files that are linked.

• Code quality support

  MeerCat has linters that will identify common code issues. MeerCat will highlight those pieces of code that have the linter warnings in its own editor, and allow a developer to make changes right there in Meercat.

• Project testing support

  In some cases, a change to code will lead to a problem with existing tests. The most obvious symptom is that one or more test cases fail given the new change. The problem is that it may take hours to see failure. MeerCat may be able to point the PR author to testing files that may need to change to avoid failure.

  In some cases, particularly when new features are added, it may be that new test cases are needed as well; while current test cases cause no problem they also do not cover the new feature. MeerCat may be able to alert the user to test files that need to be augmented to account for new features.

  One caveat to this MeerCat feature is that many (most) projects have their own idiosyncratic way of writing and organizing tests. In some simple cases, e.g., Pytest, Meercat can search a repo for relevant test files. In other projects, e.g., Flash-X, a test-to-file mapping is created during testing and can be cached to supply MeerCat with what it needs. From our perspective, we would like a project to supply MeerCat with file-to-test API that we can call to help identify relevant test files. But the form of this API, and whether it is even possible, is still under exploration.

• Pull Request Recommender

  Our conjecture is that some files typically appear together in a PR. One example is a config file and a documentation file that provides guidance on using configurations. MeerCat looks for missing siblings in a PR, e.g., a change to a config file but no change to documentation. It will recommend considering missing files that appear closely related (by prior history).

• Pull Request Discussion

  For a Pull Request that involves a significant change to the project, e.g., adding new modules, changing a core implementation approach, a robust discussion may be needed. MeerCat can help identify other developers on the project team, other than the author, reviewer and those with a mention, that may have useful input. It does so in an active fashion by supplying the names of all the developers who have worked on files in the PR in the past. The author may select one or more people from this list with an invite to join the discussion.

  MeerCat also provides a passive feature that allows any developer to "subscribe" to changes to one or more files. If any of those files appear in a future PR, the developer is notified by email. They can then choose whether to join the discussion.

MeerCat is unobtrusive

The features of MeerCat are made available through the (this) MeerCat website. You have the option of getting a brief feedback comment on GitHub from MeerCat, when a PR is created, which provides the link to the full analysis. It is up to author and reviewer to choose to use this analysis or not. In particular, unlike some CI tools, MeerCat makes no attempt to block or fail a PR.

MeerCat's analysis can be applied to any PR, open or closed, from the MeerCat website.

If you are interested

If you are interested in adding your project to MeerCat, please contact the project lead, Stephen Fickas (stephenfickas@gmail.com).