Writing tests

What you’ll build

Tests for two small banking functions, Withdraw, which throws when there isn’t enough money, and Interest. You’ll write passing tests, read a failing one’s report, and skip one. Then two short programs show how fb test reports a test that panics and one that leaks.

A first test

A test is a top-level function marked @Test that takes nothing and gives nothing back, though it can throw. It goes in the same file as the code it tests, where it sees that file’s Private declarations too. Import std.test brings in Expect, which checks a condition.

Import std.core
Import std.test

Function Interest(balance: Int, percent: Int): Int => balance * percent / 100

@Test
Function InterestIsAPercentage()
    Expect(Interest(1000, 5) == 50)
    Expect(Interest(0, 5) == 0)
End Function

Print(Interest(1000, 5))

     Running 1 test
test main.InterestIsAPercentage ... ok

test result: ok. 1 passed; 0 failed; 0 skipped; 0 filtered out; finished in 0.02s

fb test builds the file’s tests, or a package’s, and runs them:

fb test main.fb

The tests run in parallel. Those that use what only the main thread can, like a Global, run on the main thread, one at a time. fb build and fb run leave the tests out, so they cost the program nothing, and run its top-level code as usual.

A failing test

When an Expect fails, the test fails, and its report shows the condition as it’s written, where it is, and each side’s value. The test goes on after a failed Expect, so one run reports every check that fails.

Import std.core
Import std.test

Function Interest(balance: Int, percent: Int): Int => balance * percent / 100

@Test
Function InterestRoundsUp()
    Expect(Interest(150, 3) == 5)
End Function

     Running 1 test
test main.InterestRoundsUp ... FAILED

failures:

---- main.InterestRoundsUp ----
expect failed: Interest(150, 3) == 5 at main.fb:8:5
  left: 4
  right: 5

test result: FAILED. 0 passed; 1 failed; 0 skipped; 0 filtered out; finished in 0.02s

150 * 3 / 100 is 4 in integer arithmetic, which the report’s left: 4 shows. fb test exits with 1 when any test fails, so a script or CI job stops there.

Errors

A test that calls a function that throws either declares Throws and uses Try, so an error it didn’t expect fails it, or catches the error to check it. ExpectThrows<E>(x) runs x, and gives the error if it throws an E, or fails the test and gives Null if it doesn’t.

Import std.core
Import std.test

Class NotEnough Extends Error
End Class

Function Withdraw(balance: Int, amount: Int): Int Throws NotEnough
    If amount > balance Then Throw NotEnough("balance is {balance}, asked for {amount}")
    Return balance - amount
End Function

@Test
Function WithdrawTakesTheAmount() Throws NotEnough
    Expect(Try Withdraw(100, 30) == 70)
End Function

@Test
Function WithdrawRefusesTooMuch()
    Local error = ExpectThrows<NotEnough>(Withdraw(10, 30))
    Expect(error?.Message == "balance is 10, asked for 30")
End Function

@Test
Function LargeAmounts()
    Skip("needs Int64 balances")
End Function

     Running 3 tests
test main.LargeAmounts ... skipped: needs Int64 balances
test main.WithdrawRefusesTooMuch ... ok
test main.WithdrawTakesTheAmount ... ok

test result: ok. 2 passed; 0 failed; 1 skipped; 0 filtered out; finished in 0.02s

Skip(reason) ends a test and reports it as skipped. Fail(message) fails one outright, for a case an Expect doesn’t fit.

Panics and leaks

A test also fails if it panics, as on an index out of range or a failed Assert. fb test starts the program again after a panic, so the rest of the tests still run, and the panic’s stack trace follows the report:

Import std.core
Import std.test

@Test
Function IndexesInRange()
    Local items = [1, 2, 3]
    Expect(items[3] == 3)
End Function

@Test
Function StillRuns()
    Expect(1 + 1 == 2)
End Function

     Running 2 tests
test main.IndexesInRange ... FAILED
test main.StillRuns ... ok

failures:

---- main.IndexesInRange ----
panic: index 3 is out of range for 3 elements at main.fb:7:12

test result: FAILED. 1 passed; 1 failed; 0 skipped; 0 filtered out; finished in 0.04s
panic: index 3 is out of range for 3 elements at main.fb:7:12
    at IndexesInRange (main.fb:7)

A test fails, too, when a debug build finds that it leaked objects, and the report of the leak names the cycle of references that kept them:

Import std.core
Import std.test

Class Node
    Field Next: Node?
End Class

@Test
Function LinksTwoNodes()
    Local a = Node()
    Local b = Node()
    a.Next = b
    b.Next = a
    Expect(a.Next != Null)
End Function

     Running 1 test
test main.LinksTwoNodes ... FAILED

failures:

---- main.LinksTwoNodes ----
leak: 2 Node objects were never released

test result: FAILED. 0 passed; 1 failed; 0 skipped; 0 filtered out; finished in 0.02s
leak: 2 Node objects were never released
leak: a cycle of 2 objects: Node.Next -> Node.Next -> the first Node

The two Nodes refer to each other, so neither is ever released. A Weak reference for one of the links, Field Next: Weak<Node>, would fix it.

Choosing tests, and testing from outside

--filter runs only the tests whose names have its text in them:

fb test main.fb --filter Withdraw

A test’s name is its module’s and its function’s, as main.WithdrawRefusesTooMuch. In a package, fb test runs the tests in every file, and the files of its tests/ folder, which test the package from outside, through its Public names only, as another package would use it.

What you learned

• A test is a @Test function, beside the code it tests, which builds leave out.
• Expect checks a condition and reports each side’s value when it fails; the test goes on.
• ExpectThrows<E> checks that code throws an E, and gives the error.
• A panic or a leak fails a test, and the other tests still run.
• fb test runs a file’s or package’s tests in parallel, and --filter picks some.