Functions in Mathematics and Programming

Explaining mathematical notation for functions using programming concepts

Although mathematics inspired programming, many people will find it easier to understand programming syntax than mathematical notation.

So I just got the idea to demystify mathematical notation by showing analogies from different programming languages.

Look at this simple math function.

f(x) = x + 3

We can express this function in different programming languages. In python we would write:

def f(x):
return x + 3

In my favorite language, Julia you could write this in a number of ways, the long form:

function f(x)
x + 3
end

Or you could use shorthand and write it as:

f(x) = x + 3

If you tried to write this in a more low-level language such as C, we get a problem:

int f(int x) {
return x + 3
}

Specifying Type Using Set Theory

The problem is that we have to specify type, which our math example did not bother with. This does not mean mathematical notation does not allow you to express argument types, quite the contrary. Look at this example:

f(x) = x + 3 where x ∈ ℤ

It defines f as a function which takes an argument x which must belong to the set of all integers . is notation from set theory, meaning "is element of" or "in". In mathematics we use the symbols ℕ, ℤ, ℚ, ℝ to denote the natural numbers, integers, rational numbers and real numbers.

So we are using set theory notation to indicate what type our arguments are.

Most programming languages are not very flexible in how they deal with numbers. They deal with numbers in very concrete ways. We must e.g. specify whether our function takes arguments which are 8, 16, 32 or 64 bit integers. A mathematician will not specify bit size as that is not relevant to mathematical expressions.

However Julia, which was designed for numerical work can deal with numbers in a more mathematical sense.

f(x :: Integer) = x + 3

This says that the argument x is an integer. In Julia this is an abstract type, unlike C, where integer is short for 64 bit or 32 bit integer depending on architecture.

Julia has abstract and concrete numbers, defined in terms of set theory. So Integer is the union of all concrete integer types such as Int32, UInt32, Int64 etc. Thus the C example written in Julia syntax would have to be written as:

f(x :: Int) = x + 3

Which is short hand for:

f(x :: Int64) = x + 3

If you want to write a function where the argument may be a real number, meaning an integer or floating point number, then you write:

f(x :: Real) = x + 3

In mathematics we have the full flexibility of set theory to specify valid argument in detail.

This first example is nonsensical since it is the set union of integers and real numbers.

f(x) = x + 3 where x ∈ ℤ ∪ ℝ

However reals include all integers, but I just wanted to give you an example using the union of two sets.

Here is a more specific example. We limit f to only be defined when xis one of the numbers 1, 2, 3, or 4.

f(x) = x + 3 where x ∈ {1, 2, 3, 4}

Most programming language don’t have anything near this sort of flexibility to specify number types.

Julia however does at least allow us to take the union of two sets of numbers. Here we are saying x must be a value within the union of all integer and floating point numbers.

g(x :: Union{Integer, AbstractFloat} ) = x + 3

Arrays

In most programming languages you can specify arrays using syntax similar to what you find below:

A = [2, 4, 6]
A = [2 4 6; 8 10 12]

The first example is of a vector (one dimensional array) and the second example is of a matrix in Julia (two dimensional array). Mathematical notation is similar:

Image for post
Image for post

However the notation used for accessing individual elements is different. In Julia we would access a 1D and 2D array like this:

i = 2         # Second row
j = 3 # Third column(img)
a = A[i] # i'th element in 1D array
a = A[i, j] # Element at i'th row and j'th column

In this case mathematical notation is quite different.

Image for post
Image for post

However functions are used in a lot of cases in mathematics where a programmer would use an array or range.

You could look at i as an index into an array, or perhaps more accurately as a dictionary mapping the i value to the function value. We could write this in numerous ways.

Image for post
Image for post

But this would be the more common and compact way.

Image for post
Image for post

In theory we could also indirectly define such a relationships in this way:

Image for post
Image for post

While not common it illustrates how mathematics is declarative. It describes patterns or relationships. Programming languages in contrast are imperative, they explain how to do something.

Anonymous Functions

You could add up all the values of a function in this way.

Image for post
Image for post

In Julia I could accomplish the same by writing:

b = sum(A, [1, 2, 3])

Technically it is not entirely the same since I am using an array which is ordered, while mathematically I have expressed a set. So strictly speaking I would have to write the Julia code like:

sum(A, Set([1, 2, 3]))

As a programmer you may be familiar with closures, lambdas or anonymous functions. Basically we can express the function inline without giving it a name:

sum(i->2i, [1, 2, 3])

I can accomplish the same using mathematical notation with the “maps to” arrow ↦.

Image for post
Image for post

However this is uncommon to do in mathematical notation, since it will usually be obvious which variable actually varies. This way however it is explicit.

Higher Order Functions

What we just saw was an example of a higher-order function. This is a concept from functional programming. It means functions which take other functions as arguments or return new functions, rather than just taking values and returning values.

sum was an example of a higher-order function. More common examples are map, reduce and filter.

Here are some examples of using these in Julia:

julia> f(x) = x^2
f (generic function with 1 method)

map takes a list of values and maps them to another list of values using a supplied function.

julia> map(f, [2, 4, 6])
3-element Array{Int64,1}:
4
16
36
julia> map(x->x^2, [2, 4, 6])
3-element Array{Int64,1}:
4
16
36

reduce reduces a list of values into just one value by applying a function that takes two arguments in succession.

julia> reduce(+, [5, 3])
8
julia> reduce(*, [5, 3])
15

Let me explain with an example how it works. Say we reduce a list of numbers using the function f like this:

reduce(f, [1, 2, 3, 4])

This translates into:

f(f(f(1, 2), 3), 4)

These functions take functions and return values. But we could also return functions.

function adder(x :: Number)
return function(y :: Number)
x + y
end
end

This is just a very explicit form of writing it, but we can write in more compact as:

adder(x) = y -> x + y

You can use this in Julia like this:

julia> ten_adder = adder(10)julia> ten_adder(2)
12
julia> ten_adder(5)
15

With these examples it is easier to explain the mathematical concept such as derivatives, integrals and transforms.

Function Derivatives as Higher Order Functions

You where probably introduced to derivation and integration in high school, as part of Calculus. As a quick refresher: The derivative of some function f(x) is another function f'(x) which instead of giving the value of the function at x gives the slope of the function at x.

Image for post
Image for post

Above is a simple example of a function with its corresponding derivative. If you think about it, differentiation, which is what we call finding the derivative of a function, is really just calling a higher order function.

There are some alternative ways of writing the derivative of a function, which may help hammer home this point. You can also view differentiation as using the operator D on a function.

Image for post
Image for post

To be consistent with how we write integral transforms which I will discuss further down, I could invent my own notation.

Image for post
Image for post

This should logically speaking work, but I haven’t seen anybody use that. I am just writing it this way to get across that D returns a new function and the (x) actually refers to the argument to this new function, not the argument to f itself. This programming notation may help you get the idea:

julia> f(x) = x^2julia> f(1)
1
julia> df = D(f)julia> df(1)
2
julia> df(3)
6

Usually in software we handle differentiation in a numerical way, so one would write something like this to find the derivative of f for x = 3:

julia> derivative(f, 3)
3

Thus we could define the D operator like this:

function D(f::Function) 
function(x)
derivative(f, x)
end
end

Integrals as Higher Order Functions

We can think of integration in a similar manner. As a quick recap, integration is to find the area underneath a function.

Image for post
Image for post
Integrals are define on a range. E.g. the area under f(x) from a to b.

We will usually write the integral of a function from a to b like this:

Image for post
Image for post

This is very close to what the higher order function sum does in Julia. Of course it is not exactly the same because sum cannot deal with infinitesimal (incalculably small) values of dx. But we can se that it gives somewhat similar values. E.g. if f(x) = 2x then the integral is We can test that this is roughly the case.

julia> f(x) = 2xjulia> 10^2  # 
100
julia> sum(f, 0:10)
110

It doesn’t have to be perfect as we are just trying to build an intuition about how this works. The integral is a higher order function because it takes a function as one of its arguments. sum is a semi-crappy approximation of an integral. It can also be used as a higher order function, since its first argument can be a function.

Integrals and derivatives are fairly straight forward in this context. It should help you relate to the integral transforms that I cover next. They are also higher order functions but a bit more complex to wrap your head around.

Integral Transforms

The best known transforms used in mathematics are of a type called integral transforms. Examples of these are the Fourier transform, Laplace transform and convolution (the latter I am not certain of).

We can express an integral reform using mathematical notation like this:

Image for post
Image for post
  • T the transform.
  • f function being transformed
  • u argument of function transform evaluates to (function returned by transform in programming speak).
  • K kernel function. Most integral transforms are defined by the kernel function they use.
  • t dependent variable, also called dummy variable. It is the variable that assumes the values in the range t₁ to t₂

While not necessary, you could make it more explicit which variable is the dependent one:

Image for post
Image for post

Let us look at the long and short version of expressing something comparable to this in code. First the long version:

function T(f::Function)
function(u)
sum(t1:t2) do t
K(t, u) * f(t) * Δt
end
end
end

We can express this in a more compressed form:

T(f) = u -> sum(t -> K(t, u)*f(t)*Δt, t1:t2)

Both examples clearly show that a mathematical Transform is essentially just a higher-order function. It takes some function f(t) and returns another function of u.

Interpretation of Integral Transforms

You can use transforms on functions which take 1 or more arguments. We can use functions taking single arguments to represent a signal such as a radio wave, a sound signal or just about anything.

In this regard you can think of transforms as taking a signal and transforming it to another signal. Since a signal can be thought of as a function, transforming a signal is the same as taking one function as an argument and returning another function.

Image for post
Image for post

The picture above shows a Fourier transform. The input function is the signal in red. We can think about the signal as being composed of multiple simple sinus shaped signals, shown in cyan.

A Fourier transform decomposes the red signal into its constituents and give us the function with the blue spikes, one spike for each frequency

Functions taking two arguments could be used to represent an image. The arguments would represent the x, y coordinates. The value of the function would be the light intensity or color at that coordinate. Here is a simple and somewhat contrived example of how such a function could be defined.

Image for post
Image for post

Naturally real functions will not be defined like this.

Laplace Transform

I will not explain what the purpose of the Laplace transform is here, but instead shows you some different possible notation for it as well as give examples of how you would translate this into code.

Image for post
Image for post

The transform is denoted with the letter , taking a function f of t as argument and return a function F of s.

In Julia code this could be expressed roughly as:

ℒ(f) = s -> sum(t -> f(t) * e^(-s*t) * Δt, 0:∞)

Or in the longer form:

function ℒ(f::Function)
function(s)
sum(0:∞) do t
f(t) * e^(-s*t) * Δt
end
end
end

This is of course different from the mathematical definition, because in code we cannot deal with continuous definition. We are dealing with a discrete version. Instead of an infinitely small dt we have to deal with some sensibly picked Δt.

Nor can we deal with an infinitely large range such as 0:∞, but have to pick some practical approximation. The point of showing this code is just to help you understand the principle.

If we look at our previous definition of an integral transform, you can deduce that the kernel function, K in the Laplace transform is:

Image for post
Image for post

Fourier Transform

Let us look at some possible ways of defining the Fourier transform using mathematical notation and code.

Image for post
Image for post

You can probably tell that our kernel K in this case is:

Image for post
Image for post

We can then define the Fourier transform in code as:

𝓕(f) = ξ -> sum(x -> f(x) * e^(-2π*i*xξ) * Δx, -∞:∞)

Or in the longer form:

function 𝓕(f::Function)
function(ξ)
sum(0:∞) do x
f(x) * e^(-2π*i*xξ) * Δx
end
end
end

Final Remarks

I hope this article helped you to read mathematical notation and get a glimpse of how you can translate mathematics into code.

Mathematics has the luxury of expressing theoretical perfect worlds which we cannot deal with in code directly. In code we need to find some discrete numerical representation of the problem. We would have to make approximations to the real mathematical concept.

We can e.g. approximate integrals as summing over a range of values with some discrete stepping value. Usually an efficient and accurate code version to implement these transforms will be far more complex that the naive implementations you could create directly from the definitions.

Written by

Geek dad, living in Oslo, Norway with passion for UX, Julia programming, science, teaching, reading and writing.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store