Continuous Integration

Overview

Teaching: 30 min
Exercises: 60 min

Questions

• How can we run tests on our repos automatically?

Objectives

• Explain the purpose of continuous integration.

• Explain how to setup continuous integration.

• Explain .

• Explain the use of code blocks in episodes.

A episode covering the basics of continuous integration.

Continuous integration (CI) automatically runs builds your codes and run your tests on a variety of different platforms. Typically this may be run when new code is committed or before merging experimental code into the main repository. CI is useful for catching bugs before they reach your end users, and for testing on platforms that are not available to every developer.

Most CI software and services have webhooks for integration github.

Travis-CI and other services

We will be using TravisCI. The service is free for open source projects, and integrates with GitHub. Some other options you might choose for continuous integration include CircleCI or Azure Pipelines.

Using Travis is essentially getting access to an empty machine (with a specified OS installed). In order to test our project using Travis, we have to set up the correct environment and tell Travis what to do.

Setup of CI

This is where the .travis.yml file to tell comes in. Fortunately, CookieCutter has created this file for us, and set it up to run our tests already using pytest. If you wanted, you could go to travis-ci.org right now, enable it on this repository, have Travis automatically execute your tests every time you push to the repo, and never have to worry about what Travis is doing (since the CookieCutter has very nicely set it up for us). However, we will go over this file in detail.

So far in this workshop, we have been working in the molssi_devops directory and the tests directory. We will now consider the .travis.yml and some other files which are in the devtools directory.

First, open the .travis.yml file. You will see something like the following at the top of the file.

language: python

# Run jobs on container-based infrastructure, can be overridden per job

matrix:
  include:
    # Extra includes for OSX since python language is not available by default on OSX
    - os: osx
      language: generic
      env: PYTHON_VER=3.6
    - os: osx
      language: generic
      env: PYTHON_VER=3.7


    - os: linux
      language: generic  # No need to set Python version since its conda
      env: PYTHON_VER=3.6
    - os: linux
      language: generic
      env: PYTHON_VER=3.7

This is the first section of our file. It tells Travis which operating systems and Python versions we want to test on. Travis will execute our tests on each of these environments. CookieCutter has set up for us to test on OSX using Python 3.6, OSX using Python 3.7 and then two different Linux systems (also varying Python versions).

Moving on to the next part of the file, the part of the travis.yml file given below sets up the testing environment. This will be execute for each OS and Python version specified above.

The first section sets up the environment.

before_install:
    # Additional info about the build
  - uname -a
  - df -h
  - ulimit -a

    # Install the Python environment
  - source devtools/travis-ci/before_install.sh
  - python -V

The section of this script which really does something is the line source devtools/travis-ci/before_install.sh. This script installs installs miniconda in the environment where it is executed and configures conda so that packages (dependencies) can be installed.

After this section, Travis has an environment running a particular OS, and version of Python, and miniconda installed.

The next section of the .travis.yml file gives instructions for creating an environment using conda, and installing our package.

install:

    # Create test environment for package
  - python devtools/scripts/create_conda_env.py -n=test -p=$PYTHON_VER devtools/conda-envs/test_env.yaml
    # Activate the test environment
  - conda activate test
    # Build and install package
  - python setup.py develop --no-deps

The first line (starting with - python devtools/scripts), creates a Python environment named test using the files create_conda_env.py and test_env.yaml. The file test_env.yaml in particular is important to us. This tells Travis what packages we need in our testing environment.

Open devtools/conda-envs/test_env.yaml

name: test
channels:
dependencies:
    # Base depends
  - python
  - pip

    # Testing
  - pytest
  - pytest-cov

    # Pip-only installs
  - pip:
    - codecov

We can see here, that this file specifies that pytest and pytest-cov are dependencies for this environment. If our package used another dependency (numpy, for example), we would add that dependency here for it to be available to Travis.

After this, our testing environment is set-up.

The following line actually executes the test.

script:
  - pytest -v --cov=molssi_devops_2019 molssi_devops_2019/tests/

This command should look very similar to what we executed in our previous lesson on testing. However, here we have added an extra command to include code coverage.

Configuring Travis

Here, we have used Travis in a very specific way to run pytest. To do this, we first prepared the environment by installing miniconda, and creating a test environment with the required packages installed.

The way the CookieCutter has done this is one way to do it, however, note that there are many ways you could have achieved the same effect. Similarly, you can use Travis to do more than just run pytest.

Using travis-ci.org

• Next, you should navigate to travis-ci.org and hook it to this repository. Demonstrate a push and watch the tests run automatically

Describe code coverage

- Explain why it is important
- Explain what it is *not*
- Explain how to enable it

Key Points

• Continuous integration services can be used to automatically test your code.