The relevance of speed here is that you don't have to realy on a second language such as C++, which profoundly affects how you can create a machine learning library.
The higher performance of Julia allows Flux to work in a very different fashion from TensorFlow.
TensorFlow is based on organizing a graph of nodes made in C++. With Flux you simply write plain code. The code IS your graph.
I know that sounds a bit weird, so here is a more detailed explanation: https://medium.com/@Jernfrost/flux-vs-tensorflow-misconceptions-2737a8b464fb