====== Test-driven development, debugging, and profiling ======

Lecturer: Pietro Berkes, <pietro.berkes_AT_ googlemail _DOT_ com>

{{:software_carpentry_cheatsheets_v1_6.pdf|Download the cheat sheets}} with a summary of the tools and methods discussed in the class!

===== Course material =====

Slides: {{:software_carpentry.pdf|class slides}}

Exercises: 

  * {{:carpentry_exercises.pdf|carpentry_exercises.pdf (exercise sheet)}}
  * {{:carpentry_exercises.zip|carpentry_exercises.zip (source code)}}
  * {{:carpentry_solutions.zip|Exercise solutions}} (available on Tuesday afternoon)

The Live Demo TDD code, implementing the k-means algorithm: {{:k_mean.zip|k_mean.zip}}

Outline of the lecture (2h):

  - The agile development cycle, scientific-style
  - Unit testing
    * unittest
    * testing with numpy arrays
    * what to test and how for scientific programming
  - Debugging
    * pdb
  - Profiling
    * timeit
    * cProfile
  - Coverage
  - Continuous integrations


Many live demos, music and dances!

===== Good programming practices =====

==== Software carpentry in general ====

Two comprehensive software carpentry courses:

[[http://software-carpentry.org/|sofware-carpentry.org]]

[[http://itb.biologie.hu-berlin.de/~zito/teaching/SC/|Tiziano's course]] at the Bernstein center in Berlin

The classic book about software carpentry: [[http://pragprog.com/titles/tpp/the-pragmatic-programmer | The pragmatic programmer]]

==== Coding standards ====

[[http://www.python.org/dev/peps/pep-0008/|PEP8]], the official Python style guide

[[http://www.logilab.org/857 | pylint ]], tool to check coding standards

[[http://pypi.python.org/pypi/pyflakes | pyflakes]], passive code checker (only errors, not style)

[[http://pypi.python.org/pypi/pep8/0.6.1 | pep8]], script to check PEP8 coding standard
===== Test suites =====

[[http://docs.python.org/library/unittest.html|unittest]], the standard Python test framework

[[https://nose.readthedocs.org/en/latest|nose]], an alternative framework that simplifies writing tests and allows for extension and customization of test experience. There is a useful option for running a test coverage analysis: ''nosetests --with-coverage''

[[http://pytest.org/latest/|py.test]], a less common alternative

[[http://docs.python.org/library/doctest.html | doctest]], write tests inside of docstrings

[[http://www.voidspace.org.uk/python/mock | mock]], the most popular tool for creating mock objects for testing.
===== Debugging =====

[[http://docs.python.org/library/pdb.html|pdb]], the standard python debugger

[[http://winpdb.org/download/|winpdb]], a graphical interface for ''pdb''. It's
platform-independent despite its name.

[[http://www.gnu.org/software/ddd/|DDD]] (DataDisplayDebugger), graphical general-purpose debugger.

Python IDEs often have a built-in debugger.

===== Profiling =====

[[http://docs.python.org/library/profile.html|cProfile]], the batteries included
 Python profiler

[[http://docs.python.org/library/timeit.html|timeit]], built-in Python module to
 measure the execution time of small code parts

Tools to visualize profiling results:

[[http://www.vrplumber.com/programming/runsnakerun/|RunSnakeRun]], a wxPython-based visualization of profiling results -- very intuitive!

[[http://pycallgraph.slowchop.com/|PyCallGraph]]

[[http://code.google.com/p/jrfonseca/wiki/Gprof2Dot|GProf2Dot]]

[[http://kcachegrind.sourceforge.net/html/Home.html|KCacheGrind]]