Testing is hard, and it can be made harder when we come across edge cases we did not anticipate. With traditional testing methods, these edge cases can be hard to debug. Enter property-based testing, your new fuzzy friend who is a master at weeding out these oddball cases.

What is property-based testing?

Traditional unit tests involve specifying inputs for which the output is known and checking if the function behaves as expected. While this is effective for testing known cases, it may not expose edge cases or unexpected behavior patterns.

Traditional testing

Traditional testing typically involves writing explicit test cases for specific known inputs. For example, to test a palindrome function, you might write:

def test_palindrome():
    assert is_palindrome("radar") == True
    assert is_palindrome("hello") == False
    assert is_palindrome("noon") == True

def is_palindrome(s):
    return s == s[::-1]

test_palindrome()

In this approach, each test case manually checks for a specific expected result based on a predetermined input.

However, property-based testing flips this approach by allowing you to specify the logical properties your code should adhere to, and then automatically generating tests that try to prove these properties wrong. This is similar to saying, “no matter what the input is, the output should meet certain criteria.”

We call this fuzzy testing, where instead of using explicit values, we give them randomness to allow us to expose edge cases we might not have considered.

Setting up hypothesis

Hypotheses is a library that can generate random inputs for our objects, and make sure that the object behaves as expected with the given values.

It includes a wide array of value generation strategies, which can massively reduce the amount of test code we have to write, as it can effectively cover more cases.

To get started with Hypothesis in your Python project, you first need to install the library. You can install Hypothesis using pip:

pip install hypothesis

or by using Poetry:

poetry add hypothesis

Basic example

Let’s illustrate how Hypothesis works with a simple function that checks if the reversal of a string is the same as the original string (a palindrome check):

from hypothesis import given
from hypothesis.strategies import text

def is_palindrome(s):
		if len(s) < 2:
			return True
    return s == s[::-1]

@given(text())
def test_palindrome(s):
		if len(s) < 2:
			assert is_palindrome(s) is True, f"Expected {s} to be a palindrome"
		else:
	    assert is_palindrome(s) == (s == s[::-1]), f"Expected {s} to be a palindrome"

test_palindrome()

Here Hypothesis generates a number of strings to test our function, meaning it will test both our positive and negative cases.

By using this simple decorator, we have been able to test both cases we have in this example, in very few lines of code.

Advanced usage

Hypothesis offers a variety of strategies to customize how data is generated. For instance, you can specify constraints for numbers, such as ranges, or customize text generation to include specific characters.

Here’s a more complex example, testing a sorting function:

from hypothesis import given
from hypothesis.strategies import lists, integers

def sort_numbers(numbers):
    return sorted(numbers)

@given(lists(integers()))
def test_sorting(numbers):
    sorted_numbers = sort_numbers(numbers)
    for i in range(1, len(sorted_numbers)):
		    prev, curr = sorted_numbers[i-1], sorted_numbers[i]
        assert prev <= curr, f"Expected {curr} to be greater than {prev}"

test_sorting()

This test automatically verifies that the output list is sorted correctly, no matter what list of integers the Hypothesis library generates.

Tips for effective property-based testing

1. Define clear properties: The effectiveness of property-based testing hinges on the clarity and correctness of the properties you define.
2. Combine with traditional unit tests: Use property-based tests in conjunction with traditional unit tests to cover both expected and unexpected cases.
3. Start small: Begin with simple properties and gradually introduce more complex scenarios as you become more familiar with the library’s features.
4. Use coverage tools: Integrating coverage tools can help identify parts of your codebase that are not being adequately tested by your properties.

Final thoughts

Property-based testing with the Hypothesis library offers a robust supplement to traditional testing methods in Python. By automatically generating test cases that challenge your assumptions, it can uncover hidden bugs and improve the quality of your code. Whether you’re testing simple functions or complex systems, Hypothesis can be a valuable tool in your testing arsenal.

With practice, you can harness the full power of property-based testing to build more reliable, bug-resistant software.