Pytest

1. Intro
2. Markers
- 2.1. Skip Test on Some Versions
- 2.2. Parametrized
3. Fixtures
4. Builtin Fixtures
5. Mocks
6. Asycnio
- 6.1. pytest-asyncio
- 6.2. asynctest

1 Intro

1.1 Test Discovery

Test files should be named =test_<something>.py or <something>_test.py.
Test methods and functions should be named test_<something>.
Test classes should be named Test<Something>.

1.2 Useful Options

--collect-only
-v, --verbose
-x, --exitfirst: useful when debugging a problem
-s, --capture=method: method should be one of fd|sys|no. -s is shortcut for --capture=no
–lf, --last-failed: only run last failing tests, or use –ff, –failed-first
-l, --showlocals: local variables and their values are displayed with tracebacks for failing tests
--tb=style: useful styles short, line, and no
--durations=N: show N slowest setup/test durations (N=0 for all), helpful to speed up testing

1.3 Plugin Options

--pdb
--cov, --cov-report=html

2 Markers

Markers are one of the best ways to mark a subset of your test functions so that they can be run together.

import pytest

@pytest.mark.somemark
def test_func():
    pass

pytest -v -m 'some mark'

2.1 Skip Test on Some Versions

@pytest.mark.skipif(tasks.__version__ < '0.2.0',
		    reason='not supported until version 0.2.0')

2.2 Parametrized

Parametrized testing is a way to send multiple sets of data through the same test and have pytest report if any of the sets failed. @pytest.mark.parametrize

@pytest.mark.parametrize('arg1,arg2', [(1, 'arg2_value1'), (2, 'arg2_value2')])

3 Fixtures

3.1 Tracing Fixture Execution

--setup-show

3.2 Data Fixture

@pytest.fixture(name='show_on_result', scope='session')
def a_tuple():
    return (1, 2, 3)

3.3 Fixtures for Resource

@pytest.fixture()
def resource():
    # Setup
    yield
    # Teardown

3.4 Specifying Fixtures with `usefixtures`

@pytest.mark.usefixtures('fixture1', 'fixture2')

3.5 Fixture Scope

function(default), class, module, session

3.5.1 Class Scope

need to use usefixtures

# class scope
@pytest.fixture(scope='class')
def class_scope():
    """A class scope fixture."""

@pytest.mark.usefixtures('class_scope')
class TestClass:
    def test_method(self):
	# use class_scope fixture
	pass

3.6 Autouse Fixture

@pytest.fixture(autouse=True)

3.7 Renaming Fixture

@pytest.fixture(name='simple')
Use --fixtures option to list all the fixtures available for the test

3.8 Parametrizing Fixtures

@pytest.fixture(params=tasks_to_try, ids=task_ids)
With parametrized fixtures, every test function that uses that fixture will be called multiple times.
Use ids to specify fixture identities.

4 Builtin Fixtures

4.1 `tmpdir` & `tmpdir_factory`

We get session scope temporary directories and files from the tmpdir_factory fixture, and function scope directories and files from the tmpdir fixture.

a_dir = tmpdir_factory.mktemp('mydir')
a_file = a_dir.join('something.txt')
a_sub_dir = a_dir.mkdir('anything')
another_file = a_sub_dir.join('something_else.txt')
a_file.write('contents may settle during shipping')
assert a_file.read() == 'contents may settle during shipping'

4.2 request

Used with fixture parametrization request.param

4.3 pytestconfig

Adding command-line options via pytest_addoption should be done via plugins or in the conftest.py file at the top of your project directory structure.

            def pytest_addoption(parser):
		parser.addoption("--myopt", action="store_true",
				 help="some boolean option")
		parser.addoption("--foo", action="store", default="bar",
				 help="foo: bar or baz")

Using options

def test_option(pytestconfig):
    print('"foo" set to:', pytestconfig.getoption('foo'))
    print('"myopt" set to:', pytestconfig.getoption('myopt'))

4.4 cache

Storing information about one test session and retrieving it in the next. examples: --lf, --ff

command line options: --cache-show, --cache-clear
cache.get, cache.set
By convention, key names start with the name of your application or plugin, followed by a /,

and continuing to separate sections of the key name with /’s. The value you store can be anything that is convertible to json, since that’s how it’s represented in the .cache directory.

cache is function scope fixture, using request.config.cache in any other scopes.

4.5 capsys

out, err = capsys.readouterr()

4.6 monkeypatch

setattr, delattr: Set/Delete an attribute.
setitem, delitem: Set/Delete a dictionary entry.
setenv, delenv: Set/Delete an environmental variable.
syspath_prepend: Prepend path to sys.path
chdir: Change the current working directory.
examples:

def test_def_prefs_change_home(tmpdir, monkeypatch):
    monkeypatch.setenv('HOME', tmpdir.mkdir('home'))
    cheese.write_default_cheese_preferences()
    expected = cheese._default_prefs
    actual = cheese.read_cheese_preferences()
    assert expected == actual

fake_home_dir = tmpdir.mkdir('home')
monkeypatch.setattr(cheese.os.path, 'expanduser',
		    (lambda x: x.replace('~', str(fake_home_dir))))

monkeypatch fixture functions can be in conjunction with unittest.mock to temporarily replace attributes with mock objects

4.7 doctest_namespace

4.8 recwarn

Examine warnings generated by code under test

5 Mocks

mocker.patch
mocker.patch.object

6 Asycnio

6.1 `pytest-asyncio`

@pytest.fixture can decorate coroutines or async generators
custom event loop
@pytest.mark.asyncio

6.2 `asynctest`

6.2.1 Mock

asynctest.Mock(object)
asynctest.create_autospec(class/func): to create mock objects
side_effect can be a function, an exception object or class or any iterable object.
Putting it all together

import asynctest
class TestCacheWithMagicMethods(asynctest.TestCase):
    async def test_one_user_added_to_cache(self):
	user = StubClient.User(1, "a.dmin")

	AsyncClientMock = asynctest.create_autospec(AsyncClient)

	transaction = asynctest.MagicMock()
	transaction.__aenter__.side_effect = AsyncClientMock

	cursor = asynctest.MagicMock()
	cursor.__aiter__.return_value = [user]

	client = AsyncClientMock()
	client.new_transaction.return_value = transaction
	client.get_users_cursor.return_value = cursor

	cache = {}

	# The user has been added to the cache
	nb_added = await cache_users_with_cursor(client, cache)

	self.assertEqual(nb_added, 1)
	self.assertEqual(cache[1], user)

	# The user was already there
	nb_added = await cache_users_with_cursor(client, cache)
	self.assertEqual(nb_added, 0)
	self.assertEqual(cache[1], user)

6.2.2 Patching

Patching is especially useful when one need a mock, but can't pass it as a parameter of the function to be tested.

When an object is hard to mock, it sometimes shows a limitation in the design: a coupling that is too tight

with asynctest.patch("logging.debug") as debug_mock:
    await cache_users_async(client, cache)
debug_mock.assert_called()

# or
@asynctest.patch("logging.error")
@asynctest.patch("logging.debug")
async def test_with_decorator(self, debug_mock, error_mock):
    ...