The Lino Team uses the TravisCI service for doing it. Our home page there is https://travis-ci.org/lino-framework
It happens regularily that some build fails there, and we have a sticky ticket (#269) just for this case: to analyze and repair these failures.
The following files are important
Most projects have a
.travis.ymlfile which specifies what TravisCI should do after each commit.
Some notes which might be useful
beautifulsoup4, html5lib, reportlab and pisa are actually needed only when you want to run the test suite, not for normal operation. Despite this they must be specified in install_requires, not in tests_require, because the doctests are run in the environment specified by install_requires.