Programming Testing: Seams

Seams are places where you can alter the behaviour of a program when it is under test.
When designing a program for testing, you should provide seams to allow it to be tested.

Seams allow you to:

• substitute a file/method/object for another
• override a method/object
• type params, package vars with interfaces and pass/swap with a fake

Application Design

When writing your application, think about areas that:

• have side effects (ex. filesystem operations)
• need to expose testing-points (ex. logged messages)
• handle behaviours (ex. renderers)

Consider defining interfaces for these, and variations so they can be tested/isolated.

Seam Types

PreProcessor Seams

In C/C++ and other languages with macros,
you can use an #ifdef to inject or replace test methods.

c example

TODO:

untested

In the real file, we include testmacros.h,
which in our tests will define functions locally.

// main.c

#include <stdio.h>
#include "libusers.h"
#include "testmacros.h"  // <-- when testing, define 'create_db_user' locally

int main(int argc, char *argv[]) {
    int id = 100;
    create_db_user("foo@example.com", "foo");  // <-- this will use our macro defined func
}

In our macro, we define a function that fakes creating a user in the database.

// testmacros.h

#ifdef TESTING

struct User {
    int  id;
    char *email;
    char *name;
}

User users[5] = {};

#define create_db_user(email, name) \
    { \
        struct User user; \
        user.id = 123; \
        user.email = email; \
        user.name = name; \
        users[0] = user; \
    }

#endif

Linker/Import Seams

You can alter the path code is sourced from to substitute in entirely new files.

CLASSPATH=test/foo:${CLASSPATH}    # java
GOPATH=test/foo:${GOPATH}          # go
PYTHONPATH=test/foo:${PYTHONPATH}  # python

Package/Module Seams

Reassign a package variable for a single testrun.

go example

This package defines OsCreate, a function that is used throghout the codebase.

// internal/fs/fs.go
package fs

import "os"

var OsCreate = os.Create // exported function, we can override

We call fs.OsCreate here

// foo.go
package main

import "foo.com/x/foo/internal/fs"

func Foo() {
    _, err := fs.OsCreate("/var/tmp/foo.txt")
    if err != nil {
        panic(err)
    }
}

In our testfile, we swap OsCreate with one that returns an error

// foo_test.go
package main

import (
    "errors"
    "foo.com/x/foo/internal/fs"
)

func TestFoo(t *testing.T) {
    var ExpectedError = errors.New("Expected")
    fs.OsCreate = func(path string) (*os.File, error) {
        return nil, ExpectedError
    }

    foo.Foo()  // <-- when foo calls 'fs.OsCreate', it will return expected error
}

Object Seams

Dependency Injection

Don't get information, and use it in one method.
Get info, then Pass information into method in separate steps.

go example

Bad

func getUser() User {
    return User{First: "luke", Last: "skywalker"}
}

func FullName() string {
    user := getUser()
    return fmt.Sprint(user.First, user.Last)
}

func Main() {
    user = getUser()
    FullName(user.First, user.Last)
}

Good

func getUser() User {
    return User{First: "luke", Last: "skywalker"}
}

func FullName(first, last string) string {
    return fmt.Sprint(first, last)
}

func Main() {
    user = getUser()
    FullName(user.First, user.Last)
}

Interface Seams

Inversion of Control

Params or Attributes typed as interfaces rather than concretions lets you sub in a fake.
You can also do this by passing lambdas.

go example

Define an interface

// internal/interfaces/user.go
type UserInterface interface {
    Rename(n string) error
}

Real code implements interface

// user.go

type User struct {}

func (this *User) Rename(n string) error {
    // ...
}

Fake also impelements interface

type FakeUser struct {
    RenameError error
}

func (this *FakeUser) Rename(n string) error {
    return this.RenameError
}

Foo accepts UserInterface, rather than concrete User.

// foo.go

func Foo(user UserInterface) error {
    return user.Rename("vader")
}

Within test we'll pass the fake, which we can manipulate to return an error

// foo_test.go

var ExpectedError = errors.New("Expected")

func TestFoo(t *testing.T) {
    user := FakeUser{RenameError: ExpectedError}
    foo.Foo(user)
}

System Interface

Rather than creating an interface for every object-type,
it might make sense to define an entire system (ex. all db calls).
Then you could combine this with a Package Seam.

For example, golang afero defines an interface for all os/io operations.
One concretion performs real operations, another records them in-memory without changing filesystem.