My New Workflow with Julia 1.0

A practical guide to how you can work with Julia in a practical and effective manner.

So there has been a lot of changes with this first release of the Julia programming language and it is sometimes hard to distill the most essential information from the rather dense documentation.

But here is how I am presently working.

Setting Up Your System

I like to be able to write julia at the terminal whenever I want, so make sure it is in my PATH by creating a symbolic linke:

$ ln -s /Applications/Development/Julia-0.7.app/Contents/Resources/julia/bin/julia /usr/local/bin/julia

If you are not on a Mac, you’ll setup this different. E.g. on my linux box at work, I got Julia installed under /opt/julia-0.7, so I create a link to the opt/bin directory.

Startup File

Configuration of your startup file has changed position since Julia 0.6. This file contains Julia code which is automatically run each time you open Julia. You put initialization code there you get tired of writing every time you start a Julia session.

$ mkdir  -p .julia/config
$ touch .julia/config/startup.jl

In my startup.jl file I've put the line:

push!(LOAD_PATH, "$(homedir())/Development/Julia")

Because I typically write my Julia packages as subdirectories of ~/Development/Julia. E.g. my package for reading the Apple plist format is under ~/Development/Julia/Plists.

Oh My REPL

To get access to syntax highlighting and rainbow colored parenthesis (makes finding matching parenthesis much easier) I install the OhMyREPL package.

$ julia

Then I hit the ] key to jump into package mode in the Julia REPL.

(v0.7) pkg>

Then just tell Julia to add the OhMyREPL package:

(v0.7) pkg> add OhMyREPL

Pay attention to the first part that says (v0.7), because that tells you which environment you adding the package to, more on that later.

Since I love OhMyREPL so much I add it to my startup.jl file. So my whole file looks like this:

push!(LOAD_PATH, "$(homedir())/Development/Julia")
using OhMyREPL

Revise Package

There is one more package I strongly advice you to install Revise:

(v0.7) pkg> add Revise

This package allows Julia to monitor changes in your code, and update the REPL environment to reflect that. Since we no longer have the workspace() command available this is needed. It isn't perfect but most of the time it means a much smoother workflow than I was used to before.

Writing Code and Package Management

You need to pay a bit more attention to the package management system now than before when writing code.

I’ve started writing most of my code inside a package. This has several advantages that I will touch upon.

When writing code for reading plist formatted files I would create it by going into package mode and write:

(v0.7) pkg> generate PLists

But lets just create a throw away package called Foobar to demonstrate the system.

(v0.7) pkg> generate Foobar

Next we jump into shell mode to go into the newly create directory containing Foobar package.

shell> cd Foobar

We do this because we want to add packages we depend on to it. Initially the the package will contain a project configuration file:

shell> cat Project.toml
authors = ["Erik Engheim <erik.engheim@earth.com>"]
name = "Foobar"
uuid = "641dae7e-b2eb-11e8-36b1-0d425f221c6d"
version = "0.1.0"

When we issue package commands, they always affect a Project.toml and Manifest.toml file in the currently active directory. That represents your environment.

So if we want to add packages to the environment our Foobar package is in, we need to make the Foobar directory the active one:

(v0.7) pkg> activate .

This works because the shell happens to be at ~/Development/Julia/Foobar, if we had still been in Julia/, we would have written: activate Foobar.

(Foobar) pkg> add TerminalMenus

This adds the TerminalMenus package as a dependency for our Foobar package. You can see this reflected in the project file under [deps]:

shell> cat Project.toml
name = "Foobar"
uuid = "641dae7e-b2eb-11e8-36b1-0d425f221c6d"
authors = ["Erik Engheim <erik.engheim@mac.com>"]
version = "0.1.0"

[deps]
TerminalMenus = "dc548174-15c3-5faf-af27-7997cfbde655"

And because we have added packages to the environment, we get a new file Manifest.toml, which contains more details. It gives the full list of all packages used directly by TerminalMenus:

shell> cat Manifest.toml
[[TerminalMenus]]
deps = ["Compat", "REPL", "Test"]
git-tree-sha1 = "9ae6ed0c94eee4d898e049820942af21daf15efc"
uuid = "dc548174-15c3-5faf-af27-7997cfbde655"
version = "0.1.0"

[[Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"

[[DelimitedFiles]]
deps = ["Mmap"]
uuid = "8bb1440f-4735-579b-a4ab-409b98df4dab"
...

Keep in mind that I could have made any directory active with the package system. There is nothing magical about Julia packages. I could pick any random directory say Spam and write activate Spam. You would not see any change initially. But if you start adding packages:

(Spam) pkg> add TerminalMenus

Then Spam directory would get Project.toml and Manifest.toml files.

What is the Point of the Environments?

So we can go around and create project and manifest files all over the place but what purpose does it serve?

Like the RVM and pyenv systems in Ruby and Python, this files affect what Julia will do when you try to load a package:

julia> using TerminalMenus

This will fail, if you have not added the TerminalMenus package to the environment which is currently in, or more concretely if there are no Project.toml and Manifest.toml files containing info about the TerminalMenus package.

The beauty of this is that you can sort of create your own little isolated sandboxes. Every directory with these files forming an environment sees a different reality. They know a different set of packages and package version numbers.

Practical Usage

When doing my regular development I start Julia from the directory where I have my code in (my package).

I would start by loading Revise, because then I can change the code of my functions and that will be reflected in the REPL as soon as the file is saved. No need for reload of Julia.

julia> using Revise

I make sure I do this while still in the Julia (v0.7) environment, the default. If you are not there, you can always jump back by writing activate with no argument:

julia> activate

The reason I have to do it there is because in say my Foobar environment, Julia would not know about Revise unless I specifically installed it. But we don't want our packages to contain dependencies to package we just use to aid our development.

Afterwards I make the Foobar environment active, so that further using statements will be based on that environment.

(v0.7) pkg> activate .
julia> using Foobar

However since Foobar parent directory ~/Development/Julia, is in LOAD_PATH you don't have to do that. Julia will look in all LOAD_PATH for packages when you issue a using SomePackage.

Working in Multiple Locations on a Package

Just check in all the file in a package in a git repository and push it to e.g. github. Convention is to give is a .jl suffix to clearly identify it as a Julia package.

If you have not registered your package, you can still easily add it to another computer. E.g. when I want to work on my PLists package on another computer. I clone it from github:

$ git clone git@github.com:ordovician/PLists.jl.git PLists

Then I go into the directory, activate it’s environment so that when I run instantiate Julia will pull the dependent packages as described in the environment of the package (what Julia calls a project environment):

$ cd PLists
$ julia
pkg> activate .
pkg> instantiate

The dependent packages are described in the Project.toml and Manifest.toml files inside the package. So let me clarify again what confused me a bit at first. Any directory you put these files you are essentially creating a separate Julia environment. A Julia package thus happens to have its own environment. So a package always has an environment in Julia 1.0, but not all environments are in packages.

Working on Multiple Dependent Packages

One of my packages, QtUIParser depends on another package PLists, and neither are registered. I also work on them in parallel at times, because I may need to extend PLists with new features I need inQtUIParser.

This process I must admit is a bit confusing to me. I’ve gotten it working myself, but I am not sure what the intended ideal way of doing this is.

To make it so that PLists would get download when I run instantiate on another computer on the QtUIParser package, I need to add it in a way that lets Julia find it anywhere.

$ cd QtUIParser
$ julia
pkg> activate .
pkg> add https://github.com/ordovician/PLists.jl

By using the full URL of the dependent package, Julia can find it. Had it been a registered package, Julia would not need me to specify the full path, because it could lookup the path in the registry.

An alternative approach is to download the package manually using git. Then you would add it to your project by specifying a local path:

$ git clone git@github.com:ordovician/PLists.jl.git SomeLocalPath/PLists
$ cd QtUIParser
$ julia
pkg> activate .
pkg> add SomeLocalPath/PLists

The downside of this approach is that, then you have created an environment which depends dependent packages being located in the same relative path on all computers installing QtUIParser.

A problem with working on packages dependent on each other is that while you may modify the code of the packages locally, other packages using that package won’t know about the changes.

The package dev command works similar to add but instead of locking the package to the state it was when you added it, the environment using the package will continuously use the latests changes to the package.

pkg> dev PLists

or

pkg> dev https://github.com/ordovician/PLists.jl