.. _developers: ************************************** Developers' Guide ************************************** .. contents:: Guidelines for contributing ================================== When preparing contributions, please follow the guidelines in :ref:`contribution`. Also: * If the planned changes are substantial or will be backward-incompatible, it's best to discuss them on the `claw-dev Google group `_ before starting. * Make sure all tests pass and all the built-in examples run correctly. * Be verbose and detailed in your commit messages and your pull request. * It may be wise to have one of the maintainers look at your changes before they are complete (especially if the changes will necessitate modifications of tests and/or examples). * If your changes are not backward-compatible, your pull request should include instructions for users to update their own application codes. Reporting and fixing bugs ------------------------- If you find a bug, post an issue with as much explanation as possible on the appropriate issue tracker (for instance, the PyClaw issue tracker is at https://github.com/clawpack/pyclaw/issues. If you're looking for something useful to do, try tackling one of the issues listed there. Developer communication ----------------------- Developer communication takes place on the google group at http://groups.google.com/group/claw-dev/, and (increasingly) within the issue trackers on Github. .. _setup_dev: Installation instructions for developers ======================================== Cloning the most recent code from Github ---------------------------------------- You can create a read-only development version of Clawpack via:: git clone git://github.com/clawpack/clawpack.git cd clawpack python setup.py git-dev This downloads the following clawpack modules as subrepositories checked out at specific commits (as opposed to the tip of a branch). * ``_ (Python code, some of which is needed also for Fortran version) * ``_ (Utility functions, Makefile.common used in multiple repositories) * ``_ (Classic single-grid code) * ``_ (AMR version of Fortran code) * ``_ (Riemann solvers) * ``_ (Python graphics and visualization tools) * ``_ (GeoClaw) This should give a snapshot of the repositories that work well together. (Note that there are many inter-dependencies between code in the repositories and checking out a different commit in one repository may break things in a different repository.) If you want to also install the PyClaw Python components, you can then do:: python setup.py install If you plan to work on the Python parts of Clawpack as a developer, you may instead wish to do:: pip install -e . The advantage of this is that when you edit Python code in your clawpack directly, it will immediately take effect, without the need to install again. However, the (potential) danger of this approach is that the path to your clawpack directory will be stored in the file site-packages/easy-install.pth and prepended to your PYTHONPATH whenever you run Python. This path will take precedence over any manually added paths, unless you delete the .pth file. If you want to use the Fortran versions in `classic`, `amrclaw`, `geoclaw`, etc., you need to set environment variables and proceed as described at :ref:`setenv`. Updating to the latest development version ------------------------------------------ The repositories will each be checked out to a specific commit and will probably be in a detached-head state. You will need to checkout `master` in each repository to see the current head of the master branch. You should never commit to `master`, only to a feature branch, so the `master` branch should always reflect what's in the main *clawpack* repository. You can update it to reflect any changes via:: git checkout master git fetch origin git merge origin/master or simply:: git pull origin master:master Remember that you need to do this in each repository before running anything to make sure everything is up to date with *master*. .. _dev_remote: Adding your fork as a remote ---------------------------- If you plan to make changes and issue pull requests to one or more repositories, you will need to do the following steps for each such repository: #. Go to ``_ and fork the repository to your own Github account. (Click on the repository name and then the *Fork* button at the top of the screen.) #. Add a *remote* pointing to your repository. For example, if you have forked the `amrclaw` repository to account `username`, you would do:: cd amrclaw git remote add username git@github.com:username/amrclaw.git You should push only to this remote, not to `origin`, e.g.:: git push username You might also want to clone some or all of the following repositories: * ``_ (documentation) * ``_ (To collect applications) * ``_ (Regression tests) * ``_ (Previous versions, 4.6) These are not brought over by cloning the top `clawpack` super-repository. You can get one of these in read-only mode by doing, e.g.:: git clone git://github.com/clawpack/doc.git Then go through the above steps to add your own fork as a remote if you plan to modify code and issue pull requests. Modifying code ============== Before making changes, make sure *master* is up to date:: git checkout master git pull Then create a new branch based on `master` for any new commits:: git checkout -b new_feature master Now make changes, add and commit them, and then push to your own fork:: # make some changes # git add the modified files git commit -m "describe the changes" git push username new_feature If you want these changes pulled into *master*, you can issue a pull request from the github page for your fork of this repository (make sure to select the correct branch of your repository). **Note:** If you accidentally commit to `master` rather than creating a feature branch first, you can easily recover:: git checkout -b new_feature will create a new branch based on the current state and history (including your commits to `master`) and you can just continue adding additional commits. The only problem is your `master` branch no longer agrees with the history on Github and you want to throw away the commits you made to `master`. The easiest way to do this is just to make sure you're on a different branch, e.g., :: git checkout new_feature and then:: git branch -D master git checkout -b master origin/master This deletes your local branch named `master` and recreates a branch with the same name based on `origin/master`, which is what you want. .. _developers_pr: Issuing a pull request ---------------------- Before issuing a pull request, you should make sure you have not broken anything: #. Make sure you are up to date with *master*:: git checkout master git pull If this does not say "Already up-to-date" then you might want to rebase your modified code onto the updated master. With your feature branch checked out, you can see what newer commits have been added to *master* via:: git checkout new_feature git log HEAD..master If your new feature can be added on to the updated master, you can rebase:: git rebase master which gives a cleaner history than merging the branches. #. Run the appropriate regression tests. If you have modified code in pyclaw or riemann, then you should run the pyclaw tests. First, if you have modified any Fortran code, you need to recompile:: cd clawpack/ pip install -e . Then run the tests:: cd pyclaw nosetests If any tests fail, you should fix them before issuing a pull request. To issue a pull request (PR), go to the Github page for your fork of the repository in question, select the branch from which you want the pull request to originate, and then click the *Pull Request* button. .. _test_pr: Testing a pull request -------------------------- To test out someone else's pull request, follow these instructions: For example, if you want to try out a pull request coming from a branch named *bug-fix* from user *rjleveque* to the *master* branch of the *amrclaw* repository, you would do:: cd $CLAW/amrclaw # (and make sure you don't have uncommitted changes) git checkout master git pull # to make sure you are up to date git checkout -b rjleveque-bug-fix master git pull https://github.com/rjleveque/amrclaw.git bug-fix This puts you on a new branch of your own repository named *rjleveque-bug-fix* that has the proposed changes pulled into it. Once you are done testing, you can get rid of this branch via:: git checkout master git branch -D rjleveque-bug-fix .. _toplevel_pr: Top-level pull requests ----------------------- The top level *clawpack* repository keeps track of what versions of the subrepositories work well together. If you make pull requests in two different repositories that are linked, say to both *pyclaw* and *riemann*, then you should also push these changes to the top-level *clawpack* repository and issue a PR for this change:: cd $CLAW # top-level clawpack repository git checkout master git pull git checkout -b pyclaw-riemann-changes git add pyclaw riemann git commit -m "Cross-update pyclaw and riemann." git push username pyclaw-riemann-changes Git workflow ------------ See :ref:`git-resources` for useful links. Catching errors with Pyflakes and Pylint =========================================== Pyflakes and Pylint are Python packages designed to help you catch errors or poor coding practices. To run pylint on the whole PyClaw package, do:: cd $PYCLAW pylint -d C pyclaw The `-d` option suppresses a lot of style warnings, since PyClaw doesn't generally conform to PEP8. To run pylint on just one module, use something like:: pylint -d C pyclaw.state Since pylint output can be long, it's helpful to write it to an html file and open that in a web browser:: pylint -d C pyclaw.state -f html > pylint.html Pyflakes is similar to pylint but aims only to catch errors. If you use Vim, there is a nice extension package `pyflakes.vim `_ that will catch errors as you code and underline them in red. Checking test coverage ======================== You can use nose to see how much of the code is covered by the current suite of tests and track progress if you add more tests :: nosetests --with-coverage --cover-package=pyclaw --cover-html This creates a set of html files in `./cover`, showing exactly which lines of code have been tested. Trouble-Shooting Tips ===================== If you are having trouble installing or building Clawpack try out some of the following tips: - Check to see if you have set the environment variable `FFLAGS` which may be overriding flags that need to be set. This is especially important to check when building the PyClaw fortran libraries as a number of flags must be set for the Python bindings and will override the defaults.