Image for post
Image for post
Julia Read-Evaluate-Print Loop (REPL) Session

REPL Based Development Explained to C++/Java and C# Developers

If you use one of the mainstream statically typed languages such as C++, Java or C#, your are probably used to using a large sophisticated IDE. It might seem exceedingly primitive how many people work with dynamic languages such as Python, Ruby or Julia just using the command line and a text editor.

I will try to explain the benefits of this sort of development with some practical examples to contrast it with an IDE.

Your Language is Also Your Development Tool

With statically typed languages, the language and the tools you use to develop code, are two completely separate things. If you program in C++, you don’t use C++ code to aid in that development. Rather you use the IDE to lookup usage of functions, type hierarchies, which methods exist for a given type etc. To configure your build system, add libraries etc you use your IDE not C++. It probably sounds odd that I even mention this because to a Java or C++ developer this is self evident.

However for many dynamic language developer this is fundamentally different. Most of these tasks which the IDE does is instead handled by the language itself. I primarily use Julia, so I will use that as an example.

A C++ developer would download third party software, and then they would have to use the IDE to add it to their project. In the IDE they would configure compilation settings etc. When I run the Julia REPL however, I do this all within Julia itself. At the Julia prompt I’ll write:

julia> Pkg.add("LightXML")

Which will cause the LightML package to be downloaded and installed. If I want to use the package in my code I write in my source code:

using LightXML

The language is used in all sort of aspects of package development. The IDE user will likely click through some kind of wizard to decide what sort of project to create, write the name of project and get a template generated. In Julia if I want to create a package named Foobar with an MIT license I write at the Julia REPL:

julia> PkgDev.generate("Foobar", "MIT")

Should I at any point want to run the test of a package including my own, I write:

julia> Pkg.test("Foobar")

I might seem slow to have to write out these full commands to do tasks you do repeatedly. Opening some panel and clicking a buttons might seem faster. However most REPL environments have lots of facilities to cut down on typing by doing command completion, utilizing history of previously issued commands etc.

For instance I can simply write:

julia> Pkg.

and hit the up or down arrow keys to cycle through all previous commands I have issued, which starts with Pkg.. Alternatively I can use tab to get suggestions for functions existing under the Pkg module.

Alternatively if I don’t remember the beginning of my command but just want to cycle through all commands with the word “Foobar,” inside it I can hit Ctrl+R and Ctrl+S to cycle back and forth between statements containing “Foobar”.

Most IDEs today have functionality which allows you to either:

  • jump to the definition of a function or type
  • get documentation about a function or type
  • find out about subtypes or subclasses
  • help you find methods available for an object

To get all this functionality, the creators or your IDE has to specifically build it.

Contrast this with a dynamic language using a REPL. You can do most of these things without depending on somebody building a large IDE for it. When you load code into a REPL, a dynamic language will create a data structure for all the parsed code, which gives information about what module it belongs to, which source code file it comes from etc.

This makes it very easy to add functions and macros to the language itself to explore the code in various ways. E.g. in Julia I can write:

julia> @edit length("hello")

This will open my preferred editor (can configure this) at the location where the function length() is defined. Not only that but it will look at the type of the argument and pick the length() function handling strings rather than the one handling arrays e.g.

Or I could use the @less macro instead to see the code right in my REPL:

julia> @less length("hello")

The interesting thing here is that @edit is a macro in the standard library, using a function called edit() which is around 100 lines of code. So it is not a major undertaking to write this function if it did not already exist.

The Julia REPL is written in Julia and can easily be extended with Julia code. It supports different modes. The one I have been showing you gives you a julia> prompt. However you can add any number of other modes. In fact Julia already comes with two other modes built in. If I hit the question mark symbol, my prompt changes to help> I can use this to get help about any function.

help?> println
search: println print_with_color print print_shortest sprint @printf isprint @sprintf

println(io::IO, xs...)

Print (using print) xs followed by a newline. If io is not supplied, prints to STDOUT.

The nice thing about this is that it works with any documentation I write as well.

julia> "add two numbers" my_add(a::Number, b::Number) = a + b
my_add

help?> my_add
search:

add two numbers

As you can see all I have to do is to add a string in front of the function definition. It then becomes immediately available in the help system.

IDEs usually have various inspectors to show you information about types, such as their fields, subclasses, superclasses etc. In fact in the IDE I usually use for C++ development, Qt Creator, much of this functionality is so slow I don’t use it unless I really need to.

Again in a dynamic language such as Julia, I can just use simple functions to solve this problem. Say a library defines a type like this:

struct Point
x::Int
y::Int
end

I want to know the fields the type has. I can then write:

julia> fieldnames(Point)
2-element Array{Symbol,1}:
:x
:y

I could look at subtypes and super type easily:

julia> supertype(Int64)
Signed

julia> subtypes(Signed)
5-element Array{Union{DataType, UnionAll},1}:
Int128
Int16
Int32
Int64
Int8

A key thing to realize about this approach to working with code is that since these are just functions in the very same language that you are developing in and already familiar with, it is trivial to extend these. We can reuse the subtype() function to create more elaborate functionality. E.g. here is a function from the Julia wiki book.

julia> function showtypetree(T, level=0)
println("\t" ^ level, T)
for t in subtypes(T)
if t != Any
showtypetree(t, level+1)
end
end
end

I can use this to show a tree of subtypes.

julia> showtypetree(Integer)
Integer
BigInt
Bool
Signed
Int128
Int16
Int32
Int64
Int8
Unsigned
UInt128
UInt16
UInt32
UInt64
UInt8

Method completion is one of the killer features of an IDE, which I think I see mostly often mentioned by static typing fans as a reason for working with an IDE and a statically typed language. I can agree that this is in fact an advantage of statically typed languages.

However we are not completely lost at sea with a dynamically typed language either. This is where the significance of REPL based development really comes into play. If you write code primarily in a simple text editor you will be somewhat limited. However in my style of development I jump between the text editor and REPL a lot. I load code into my REPL frequently either by simply quitting the REPL and restarting it with:

$ julia -i my-source-code.jl

Or I simply load a file in the REPL with:

julia> include("my-source-code.jl")

In fact there are quite a number of ways of doing this. One could also indicate that a whole module should be reloaded as well.

This means that as I am developing using my REPL, most of my code is live. Julia has parsed it and stores a representation of it in memory, which means it can be queried at runtime in various ways. E.g. I might want to know all functions which take an argument of type Kelvin:

julia> methodswith(Kelvin)
9-element Array{Method,1}:
gas_mass(P::Real, V::Real, T::Airship.Kelvin, molecular_mass::Real) in Airship at gaslaws.jl:33
gas_moles(P::Real, V::Real, T::Airship.Kelvin) in Airship at gaslaws.jl:26
gas_volume(n::Real, T::Airship.Kelvin, P::Real) in Airship at gaslaws.jl:20
*(t::Airship.Kelvin, c::Airship.GasConstant) in Airship at gaslaws.jl:8
*(c::Airship.GasConstant, t::Airship.Kelvin) in Airship at gaslaws.jl:9
+(x::Airship.Kelvin, y::Airship.Kelvin) in Airship at temperature.jl:41
-(x::Airship.Kelvin, y::Airship.Kelvin) in Airship at temperature.jl:42
convert(::Type{Airship.Celsius}, kelvin::Airship.Kelvin) in Airship at temperature.jl:21
convert(::Type{Airship.Fahrenheit}, kelvin::Airship.Kelvin) in Airship at temperature.jl:24

Of course you might prefer to ask this of an object the way an IDE does. I can write a function quickly to do this:

julia> completions(obj) = methodswith(typeof(obj))

All sorts of utility functions like this can be stored in a Julia startup file so you can use it with any project. Now if I have a variable containing a temperature like this:

julia> temp = Celsius(4)
Airship.Celsius(4.0)

I can quickly check which functions are available for this object with:

julia> completions(temp)
7-element Array{Method,1}:
gas_mass_CO2(P, V, celsius::Airship.Celsius) in Airship at densities.jl:30
gas_mass_N2(P, V, celsius::Airship.Celsius) in Airship at densities.jl:31
gas_mass_O2(P, V, celsius::Airship.Celsius) in Airship at densities.jl:32
+(x::Airship.Celsius, y::Airship.Celsius) in Airship at temperature.jl:38
-(x::Airship.Celsius, y::Airship.Celsius) in Airship at temperature.jl:39
convert(::Type{Airship.Kelvin}, celsius::Airship.Celsius) in Airship at temperature.jl:19
convert(::Type{Airship.Fahrenheit}, celsius::Airship.Celsius) in Airship at temperature.jl:23

This is actually more sophisticated than what you get in a typical IDE, as you can find all functions where any of the argument are of the given type, not just the first one.

And of course in the REPL itself we already have completion based on types, modules and functions live in the REPL environment. If I start writing e.g. split and hit tab, this happens:

julia> split
split splitdir splitdrive splitext

This also works for completion on types and fields of types. So say I defined this rectangle class:

julia> struct Rect
x::Int
y::Int
width::Int
height::Int
end

julia> r = Rect(10, 10, 40, 40)
Rect(10, 10, 40, 40)

julia> r.
height width x y

Debugging

When I began using dynamic languages I had used statically typed languages in an IDE for a while. I was used to sophisticated debuggers where I could step through code one line at a time and have a window showing me a list of variables and letting me inspect them.

The seeming lack of debuggers in most dynamic languages seemed confusing to me at first. Of course this situation varies greatly and it has been in dynamic languages such as Smalltalk and Racket where I have been most impressed by the debugging even though it looks very plain.

Smalltalk usually comes with an IDE although a simple looking one by the standard of the static typing crowd. Racket has a simple IDE called DrRacket.

Usually these allow you to step through code. The beauty of these however is that once you stop at a line, you are not limited to the functionality the IDE providers gave you to inspect and modify code. With dynamic languages, code is a living organism, so you can inspect any object you want using the code and conventions you are used to. You can add or modify any function you like in the middle of debugging. Changing state of a variable to see if it will make the code run properly is just a simply variable assignment like any other.

This is the benefit of not having a clearly separate compile time and runtime as with most static languages. Your program code is easily modifiable at runtime in a dynamic language.

For Julia you got a package Gallium.jl which simply extends the Julia REPL and gives another mode which allows debugging functionality. However using this debugger is just like installing and using any other package.

julia> Pkg.add("Gallium")
julia> using Gallium

As I’ve shown you before functionality is provided by simply adding regular Julia functions and macros. If I want to step into a function and see what it does I simply use the @enter macro:

julia> @enter join(["hello", "world"], " ")

This will jump into the execution of the join() function and allow you to step through.

However I don’t normally use debuggers much with dynamic languages. With REPL based development you learn to develop with a different philosophy. I write very short functions which are functional in nature. That means I am continuously trying out and testing functions as I write them. This is very easy to do in a REPL environment. I can paste or write subsets of a function easily into to REPL to try it out.

A code segment you suspect isn’t working the way it should in C/C++ is something I would simply factor out to a separate function in Julia and test right away in the REPL. You typically catch problems at the function interface in other words.

Extending Your Tools

You can think of your REPL as your IDE. Unlike a traditional IDE, it is extensible with simple functions written in the very same language you program in.

Say I want to switch between camel case and snake case. I write a simple Julia function which does this by reading the clipboard and writing back to the clipboard. For instance to turn snake case like this hello_world to camel case like this HelloWorld, I can do this in a single line.

camel_case(s::AbstractString) =  join(capitalize.(split(s, "_")))

Then I define another version which reads the clipboard and writes to it afterwards.

function camel_case()
s = camel_case(clipboard())
clipboard(s)
s
end

So with a couple of lines I’ve added another feature to my “IDE”. I write lots of little functions like this for all sorts of tasks I do while developing. E.g. you can add functions for the typical operations you do with git.

The problem with doing this in a regular IDE, is that functions can’t be called as they are. You have to write a lot of glue code to interface with the rest of the IDE and transform the input of the user. Just marshaling the input and output from the camel_case() function in a regular IDE would take many more lines of code that the whole core of the function.

I must admit I was also quite late to this line of thinking. Before I used Julia, I was used to writing small scripts in Python or Go, which I invoked from the shell. The problem with this approach is that I had to write lots of code to simply parse the arguments at the shell. Also at a regular Unix shell nothing is particularly easy to compose in sophisticated ways since there are no objects, only plain text.

Here is an example of earlier an earlier Julia script I created for generating PNG files of different sizes for iOS development.

#!/usr/bin/env julia

if isempty(ARGS)
println("Usage: svg2icons.jl pixelsize svgfiles")
println("Example: svg2icons.jl 25 foo.svg bar.svg")
exit()
end

ssize = ARGS[1]
if !all(isdigit, ssize)
println("Error: First argument '$ssize' needs to be pixel width and height.")
exit()
end
size = parse(Int, ssize)
for image in ARGS[2:end]
range = rsearch(image, ".svg")
output = image[1:(first(range) - 1)]
run(`svg $image $output.png $size:$size`)
for x in 2:3
dim = size*x
run(`svg $image $output@$(x)x.png $dim:$dim`)
end
end

Julia showed me with its approach to e.g. package installation, development and publishing, that you don’t really need tools invoked from the shell. Just use Julia itself as the shell and call functions there directly. Then you avoid all the tedious glue code and parsing.

So this is my new way of writing shell tools. I just stick lots of useful Julia functions in a source code file and then I just load the Julia REPL whenever I want to use any of that functionality.

Here is an example of the same functionality exposed as a function rather than a script. It is almost half the code because I don’t have to process inputs. Julia will take care of making sure only arguments of the right type are passed to the function.

"`svg2icons(size, svgfiles)` create .png files with multiples of `size`"
function svg2icons(size::Integer, svgfiles::Vector{T}) where T<:AbstractString
for image in svgfiles
range = rsearch(image, ".svg")
output = image[1:(first(range) - 1)]
run(`svg $image $output.png $size:$size`)
for x in 2:3
dim = size*x
run(`svg $image $output@$(x)x.png $dim:$dim`)
end
end
end

I still use the regular shell for switching directories, running programs, text search and starting programs, but for pretty much anything more sophisticated than that I drop into the Julia REPL.

Written by

Geek dad, living in Oslo, Norway with passion for UX, Julia programming, science, teaching, reading and writing.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store