Developer Guide

Whole Tale is an NSF-funded collaboration but also structured as an open source project. External contributions are encouraged. Whether you are part of the core team or a new contributor, we’re very happy to have your help with the project. Please feel free to ask questions or suggest changes to the this Developer Guide.

Issue management

The core team uses ZenHub for issue management across the collaboration.

During weekly development calls, issues are prioritized, clarified, and assigned to release milestones.

Defining “done”

What does it mean for an issue or task to be “done”?

  • Code complete
  • Unit tests complete and passing
  • Manual tests defined and passing
  • Documentation updated
  • PR reviewed and merged

Code management

Best practices:

  • Never commit code to master. Always use a fork or feature branch and create a Pull Request for your work.
  • Name your branch for the purpose of the change. For example feat-add-foo.
  • Always include clear commit messages
  • Organize each commit to represent one logical set of changes. For example, separate out code formatting as one commit and functional changes as another.
  • Reference individual issues in commits
  • Prefer rebasing over merging from master
  • Learn to use rebase to squash commits – organize commits for ease of review.
  • Never merge your own PR if not approved by at least one person. If reviews aren’t happening in a timely matter, escalate them to the team.
  • Merging a PR means that the work has been tested, reviewed, and documented.

Testing

Every PR must include either a unit test or manual test scenario. PRs will not be be merged unless tests run successfully.

Manual test cases will be added to the test plan template.

For the Whole Tale API, we leverage Girder’s automated testing framework.

Tests are run via CircleCI. Tests will fail with < 82% coverage.

Repositories and components

The project has the following repositories:

Core services:
  • dashboard - Whole Tale dashboard
  • girder_wholetale - Girder plugin providing basic Whole Tale functionality.
  • girder_wt_data_manager - Girder plugin for external data management.
  • wt_sils - Girder plugin providing Smart Image Lookup Service.
  • gwvolman - Girder Worker plugin responsible for spawning Instances and mounting GirderFS on compute nodes
Images:

Setting up for local development

The entire WT platform stack can be deployed locally or on a VM using the development deployment process.

It is also possible to deploy a multi-node installation on OpenStack using the Terraform deployment process.