Julia v1.5 Testing: Best Practices

Do you think running tests are too slow? Cannot figure out how to do Test Driven Development (TDD) in Julia? Here is a guide.

  • Regular testing that you do with some frequency while developing.
  • Rapid iteration testing for when you run tests really frequently. E.g. because you follow TDD (Test Driven Development). In this case individual tests you are adding need to run fast, because they are run a lot.

Infrequent and Correct Testing

Say you are developing a package named Foobar which you want to test.

$ cd Foobar
$ julia
julia> ]
(@v1.5) pkg>
(@v1.5) pkg> activate .
(Foobar) pkg>
(Foobar) pkg> test
Testing Foobar
Status `/private/var/folders/qb/zc__4nlj4_z4w24t_trvx1cm0000gn/T/jl_K7IPCe/Project.toml`
[9a3f8284] Random
[9e88b42a] Serialization
[6462fe0b] Sockets
[8dfed614] Test
Test Summary: | Pass Total
All Tests | 57 57
Testing Foobar tests passed
(Foobar) pkg> test

Regular Testing

In your daily workflow, you will do more regular running of all the tests. Then the pkg> test approach is suboptimal as it is pointless to check state of dependent packages multiple times a day. They are unlikely to change. In this case the sensible way to test is:

shell> julia test/runtests.jl
Test Summary: | Pass Total
All Tests | 57 57
@testset "All Foobar tests" begin

include("foo-tests.jl")
include("bar-tests.jl")
include("baz-tests.jl")

end
tests = ["foo", "bar", "baz"]
if !isempty(ARGS)
tests = ARGS # Set list to same as command line args
end

@testset "All Foobar tests" begin

for t in tests
include("$t-tests.jl")
end

end
$ julia test/runtests.jl foo baz

Testing Environment

An important side note: When you run tests this way, you are actually using the v1.5 Julia environment and not the one setup for your tests. But you can use the --project switch to specify the environment to use:

$ julia --project=test test/runtests.jl

Rapid Iteration Testing

To be able to do anything rapidly in Julia you want to avoid having the Julia JIT compile the same stuff over and over again. Hence you want to stay within your fully initialized REPL environment for as long as possible.

julia> include("test/runtests.jl")
Test Summary: | Pass Total
All Tests | 57 57

Single Tests

However to move even faster we want to be able to easily run just one single test. To make this practical we need to make some practical changes to our code.

include("setup.jl")

@testset "All Foobar tests" begin

include("foo-tests.jl")
include("bar-tests.jl")
include("baz-tests.jl")

end
@testset "Without operands" begin
@test disassemble(901) == "INP"
@test disassemble(902) == "OUT"
@test disassemble(000) == "HLT"
end

@testset "With operands" begin
@test disassemble(105) == "ADD 5"
@test disassemble(112) == "ADD 12"
@test disassemble(243) == "SUB 43"
@test disassemble(399) == "STA 99"
@test disassemble(510) == "LDA 10"
end
julia> include("test/setup.jl")

Geek dad, living in Oslo, Norway with passion for UX, Julia programming, science, teaching, reading and writing.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store