How to develop for Fail2Ban
***************************

Fail2Ban uses GIT (http://git-scm.com/) distributed source control.
This gives each developer their own complete copy of the entire
repository. Developers can add and switch branches and commit changes
when ever they want and then ask a maintainer to merge their changes.

Fail2Ban uses GitHub (https://github.com/fail2ban/fail2ban) to manage
access to the Git repository. GitHub provides free hosting for open-
source projects as well as a web-based Git repository browser and an
issue tracker.

If you are familiar with Python and you have a bug fix or a feature
that you would like to add to Fail2Ban, the best way to do so it to
use the GitHub Pull Request feature. You can find more details on the
Fail2Ban wiki (http://www.fail2ban.org/wiki/index.php/Get_Involved)


Pull Requests
=============

When submitting pull requests on GitHub we ask you to:

* Clearly describe the problem you're solving;

* Don't introduce regressions that will make it hard for systems
  administrators to update;

* If adding a major feature rebase your changes on master and get to a
  single commit;

* Include test cases (see below);

* Include sample logs (if relevant);

* Include a change to the relevant section of the ChangeLog; and

* Include yourself in THANKS if not already there.

If you are developing filters see the FILTERS file for documentation.


Code Testing
============

Existing tests can be run by executing *bin/fail2ban-testcases*. It
has options like --log-level that will probably be useful.  Run *bin
/fail2ban-testcases --help* for the full list of options.

Test cases should cover all usual cases, all exception cases and all
inside / outside boundary conditions.

Test cases should cover all branches. The coverage tool will help
identify missing branches. Also see
http://nedbatchelder.com/code/coverage/branch.html for more details.

Install the package python-coverage to visualise your test coverage.
Run the following (note: on Debian-based systems, the script is called
*python-coverage*):

   coverage run bin/fail2ban-testcases
   coverage report

Optionally:
   coverage html

And then browse htmlcov/index.html and see how much coverage your test
cases exert over the code base. Full coverage is a good thing however
it may not be complete. Try to ensure tests cover as many independent
paths through the code.

Manual Execution. To run in a development environment do:

   ./fail2ban-client -c config/ -s /tmp/f2b.sock -i start

some quick commands:

   status
   add test pyinotify
   status test
   set test addaction iptables
   set test actionban iptables echo <ip> <cidr> >> /tmp/ban
   set test actionunban iptables echo <ip> <cidr> >> /tmp/unban
   get test actionban iptables
   get test actionunban iptables
   set test banip 192.168.2.2
   status test


Testing with vagrant
--------------------

Testing can now be done inside a vagrant VM.  Vagrantfile provided in
source code repository established two VMs:

* VM "secure" which can be used for testing fail2ban code.

* VM "attacker" which can be used to perform attack against our
  "secure" VM.

Both VMs are sharing the 192.168.200/24 network. If you are using this
network take a look into the Vagrantfile and change the IP.


Coding Standards
================


Style
-----

Please use tabs for now. Keep to 80 columns, at least for readable
text.


Tests
-----

Add tests. They should test all the code you add in a meaning way.


Coverage
--------

Test coverage should always increase as you add code.

You may use "# pragma: no cover" in the code for branches of code that
support older versions on python. For all other uses of "pragma: no
cover" or "pragma: no branch" document the reason why its not covered.
"I haven't written a test case" isn't a sufficient reason.


pyflakes
--------

pyflakes can be used to find unused imports, and unused, undefined and
redefined variables. pyflakes should be run in any python code,
including python based actions:

   pyflakes bin/ config/ fail2ban/


Documentation
-------------

Ensure this documentation is up to date after changes. Also ensure
that the man pages still are accurate. Ensure that there is sufficient
documentation for your new features to be used.


Bugs
----

Remove them and don't add any more.


Git
---

Use the following tags in your commit messages:

* 'BF:' for bug fixes

* 'DOC:' for documentation fixes

* 'ENH:' for enhancements

* 'TST:' for commits concerning tests only (thus not touching the main
  code-base)

Multiple tags could be joined with +, e.g. "BF+TST:".

Use the text "closes #333"/"resolves #333 "/"fixes #333" where 333
represents an issue that is closed. Other text and details in link
below. See: https://help.github.com/articles/closing-issues-via-
commit-messages

If merge resulted in conflicts, clarify what changes were done to
corresponding files in the 'Conflicts:' section of the merge commit
message.  See e.g.
https://github.com/fail2ban/fail2ban/commit/f5a8a8ac


Adding Actions
--------------

If you add an action.d/>>*<<.conf file also add a example in
config/jail.conf with enabled=false and maxretry=5 for ssh.


Design
======

Fail2Ban was initially developed with Python 2.3 (IIRC). It should
still be compatible with Python 2.4 and such compatibility assurance
makes code ... old-fashioned in many places (RF-Note).  In 0.7 the
design went through major re-factoring into client/server, a-thread-
per-jail design which made it a bit difficult to follow. Below you can
find a sketchy description of the main components of the system to
orient yourself better.


server/
-------

Core classes hierarchy (feel welcome to draw a better/more complete
one):

   ->   inheritance
   +    delegation
   *    storage of multiple instances

   RF-Note   just a note which might be useful to address while doing RF

   JailThread -> Filter -> FileFilter -> {FilterPoll, FilterPyinotify, ...}
                 |         * FileContainer
                 + FailManager
                 + DateDetector
                             + Jail (provided in __init__) which contains this Filter
                   (used for passing tickets from FailManager to Jail's __queue)
   Server
     + Jails
        * Jail
          + Filter  (in __filter)
          * tickets (in __queue)
          + Actions (in __action)
            * Action
            + BanManager


failmanager.py
~~~~~~~~~~~~~~

FailManager

   Keeps track of failures, recorded as 'tickets'.  All operations are
   done via acquiring a lock

FailManagerEmpty(Exception)

   raised by FailManager.toBan after reaching the list of tickets (RF-
   Note: asks to become a generator ;) )


filter.py
~~~~~~~~~

Filter(JailThread)

   Wraps (non-threaded) FailManager (and proxies to it quite a bit),
   and provides all primary logic for processing new lines, what IPs
   to ignore, etc

   .failManager  [FailManager] .dateDetector [DateDetector]
   .__failRegex  [list] .__ignoreRegex [list]

      Contains regular expressions for failures and ignores

   .__findTime   [numeric]
      Used in *processLineAndAdd* to skip old lines

FileFilter(Filter):

   Files-aware Filter

   .__logPath [list]
      keeps the tracked files (added 1-by-1 using addLogPath) stored
      as FileContainer's

   .getFailures
      actually just returns True

         if managed to open and get lines (until empty)

      False
         if failed to open or absent container matching the filename

FileContainer

   Adapter for a file to deal with log rotation.

   .open,.close,.readline
      RF-Note: readline returns "" with handler absent... shouldn't it
      be None?

   .__pos
      Keeps the position pointer


ipdns.py
~~~~~~~~

DNSUtils

   Utility class for DNS handling

IPAddr

   Object-class for IP address handling


filter*.py
~~~~~~~~~~

Implementations of FileFilter's for specific backends.  Derived
classes should provide an implementation of *run* and usually override
*addLogPath*, *delLogPath* methods.  In run() method they all one way
or another provide

   try:
      while True:
         ticket = self.failManager.toBan()
         self.jail.putFailTicket(ticket)

   except FailManagerEmpty:
      self.failManager.cleanup(MyTime.time())

thus channelling "ban tickets" from their failManager to the
corresponding jail.


action.py
~~~~~~~~~

Takes care about executing start/check/ban/unban/stop commands
