A Guide to Different Web Technologies and How They Are Related

A desktop developer trying to make sense of the huge Web ecosystem

14 min readFeb 17, 2019

I am trying to get into web development, and searching for the best tools and approaches based on the style of working that appeals to me. It is challenging given the myriad of tools and frameworks. So here is my overview and impressions thus far.

Relationships Between HTML, CSS, HTTP, REST, JavaScript and JSON

A web browser such as Explorer, Firefox or Chrome will draw a web page based on a description found in a file written in HTML format. HTML like Markdown and Latex only describes structure and semantics of a document. It does not describe the looks. CSS files is what described what different elements in an HTML document are supposed to look like.

HTTP in contrast is a communication protocol, built on top of TCP/IP. It is what a Web browser uses to actually et hold of HTML and CSS files. HTTP defines things such as URLs. URLs define a sort of path to a resource on the internet. With these URLs HTTP associates a set of operations such as GET, POST, UPDATE and DELETE. When you write a URL in your web browser’s address field, you actually get the browser to perform a GET request. When you fill out a form on a web page and press the “submit” button, it will usually cause a POST to be performed. This collects your input and sends to the server.

REST means exposing a service to the outside world in terms of the vocabulary of the HTTP protocol. That sounds a bit abstract, so let me clarify: If you got a program running on a server, with a bunch of functionality you want other people to be able to use. There are lots of ways of making that accessible from the outside. Way back they used CORBA, which was a way of making it look like you had objects on the network, which you could call methods on. You would have references to objects, and you could send these references as arguments. It was a binary protocol. Later we got things like XML-RPC and Soap, which tried to make method calls and object references be represented as XML text. In essence it was a verbose version of CORBA.

REST was a rejection of this complexity, and people saying: Instead of accessing functionality in terms of objects with method calls, why don’t we just use the principles already found in the HTTP protocol? Try to represent functionality in terms of URLs which you do GET, POST, UPDATE and DELETE on.

JavaScript is a simple script language which was made to be able to offer some interactivity to HTML pages. Due to the ubiquity of JavaScript it became popular to represent data in JavaScript syntax, thus JSON was born. JSON is just JavaScript syntax for strings, arrays, numbers and dictionaries. Syntax for loops, if statements and functions has been excluded.

Differences Between REST, WebSockets and Unix Sockets for Communications

For an old-school programmer such as myself it was not obvious what things like WebSockets where, since it brought the associating to regular Unix sockets. In the Unix world sockets, are sort of like files. You can open and close them like files, but they don’t represent storage on a hard disk but rather a network connection. Like a file, you can read and write data to a socket. That data gets received on socket opened at the other end of the connection. So a client and a server can communicate with each other by both opening a socket which get connected to each other.

Sockets exist at the TCP/IP layer. But you can also create e.g. UDP sockets rather than TCP sockets. WebSockets in contrast are not really sockets at all, but just a protocol built on top of HTTP. The difference is that typical HTTP is setup so that a web browser creates a regular Unix socket connection to fetch a web page or other resource. As soon as it is received, it closes down the socket connection. Thus regular HTTP is not well suited for continuous communication back and forth. You can thing of WebSockets as a persistent communication channel built on regular sockets, which server and client can use to send data back and forth between each other.

WebSockets can thus be used as an alternative or supplement to REST. You may use REST to access functionality on the server and then later setup a WebSocket connection to get continuous updates or events from the server.

Communication Protocols: AJAX vs WebSockets

To make matters more complicated, WebSockets is not the only way for a webpage to get data outside of the regular HTTP request. There is also the older technology called AJAX. My understanding is that this is for basically doing regular HTTP requests, but without loading a whole new page. So you can request extra information on a page after the page is loaded.

However it is request/response based. Connections are closed down just like regular HTTP. You cannot use it keep an open connection like WebSockets, where e.g. a server can push data when some event happens without the browser requesting it.

Templating Engines: Assemble Web Pages by Combining Reusable Parts

As a desktop developer I am used to creating new classes and assembling objects to create ever larger reusable parts, which I can combine to even bigger parts.

For instance I can write a special class for a fancy button, or a sophisticated table. Even a whole color picking dialog could be a separate class.

On the web, creating reusable parts works quite different. The web is fundamentally about finding ways of effectively producing text documents (HTML).

In regular programing you can create new classes, composed of existing classes. However in HTML you cannot create new tags composed of existing tags. An HTML page must be made of only the HTML tags provided by the designers of HTML.

The solution has been to create text templating engines. What developers will do is to add their own special syntax to HTML which the browser doesn’t understand. This syntax is just for making placeholders where the templating engine can fill in data to create a final result which is valid HTML, which a browser can understand.

Here is an example from the Mustache template engine. The code below is not fully valid HTML, because it has mixed in parts surrounded by {{ }}.

<h1>{{header}}</h1>
{{#bug}}
{{/bug}}

{{#items}}
  {{#first}}
    <li><strong>{{name}}</strong></li>
  {{/first}}
  {{#link}}
    <li><a href="{{url}}">{{name}}</a></li>
  {{/link}}
{{/items}}

{{#empty}}
  <p>The list is empty.</p>
{{/empty}}

The template engine takes this template as input as well as data in the form of a JSON dictionary:

{
  "header": "Colors",
  "items": [
      {"name": "red", "first": true, "url": "#Red"},
      {"name": "green", "link": true, "url": "#Green"},
      {"name": "blue", "link": true, "url": "#Blue"}
  ],
  "empty": false
}

When this is run, it will spit out valid HTML which you can send to the web browser:

<h1>Colors</h1>
<li><strong>red</strong></li>
<li><a href="#Green">green</a></li>
<li><a href="#Blue">blue</a></li>

Templating can happen both on the server, and get sent to the web browser client as well as on the client itself utilizing JavaScript. So there are both JavaScript and server side templating engines.

That way you could put a color dialog, table or more complex component in one place, by just putting template placeholder which the templating engine will replace with pure HTML.

JavaScript Libraries and Frameworks: jQuery, React, Angular and Vue.js

HTML is a subset of XML, but where particular tags are given specific meaning relevant to web pages. When you parse an XML document you usually get a DOM tree (Document Object Model tree). It is a tree structure with nodes describing the HTML page. This is given to the web browser together with CSS to visualize the web page.

However you are not limited to creating an HTML page to create such a tree. You can also create one from scratch of manipulate the one created by the HTML page using JavaScript. Through a built in JavaScript API, you can alter the DOM, adding, removing or moving nodes, to change the visual appearance of you web page.

jQuery originally arose as a third party JavaScript library. It gave a way to locate and manipulate nodes in the DOM tree in a simpler manner than the built in API allowed. It also simplified other things such as AJAX calls. However in modern web browsers it is less needed so you hear less talk of it. The reason is that the built in JavaScript API for manipulating the DOM, has adopted a lot of jQuery functions.

Here is an example of locating a tag with the id myID.

// jQuery
$("#myID")
 
// Modern Javascript
document.querySelector("#myID")

Here is an example of achieving an animation on your webpage. This involved moving the element (tag) with the id myID 200 pixels to the left.

// jQuery
$("#myID").animate({left: '200px'})
 
// Modern plain CSS
#myID { animation: myMove 1s;}
@keyframes myMove {
    from {left: 0px;}
    to {left: 200px;}
}

Extensive examples of why you don’t need jQuery anymore in modern JavaScript here.

jQuery is sometimes contrasted with JavaScript libraries/frameworks: React, Angular and Vue.js. However while there is overlap in functionality, the latter exists at a higher abstraction level.

These can be thought of more like what a desktop developer would call a GUI library such as Qt, Swing or Cocoa. However because this is the web, things work a bit different as I’ve explained in the section about templating engines. They are meant for what people refer to as single page web applications. Less interactive web pages may not need these frameworks, as more functionality is done on the server.

These libraries/frameworks are bundled with their own templating engines to define reusable GUI parts. Templating engines however don’t add behavior to the reusable parts. We need JavaScript and CSS to get behavior. These libraries/frameworks allow us to bundle together logical units consisting of HTML, CSS and JavaScript describing one UI component.

They typically implement a Model-View-Controller pattern so that there is a way of connecting the UI elements you see with an underlying data model. Several UI elements can connect to the same data model so we can synchronize the data shown in several UIs.

React is made by Facebook and Angular is made by Google. That means both are really big and popular. Vue.js is smaller and less known (not having a big corporate backer), however it seems to have been designed based on what has been learned from both React and Angular, picking up good ideas from both.

I started my foray into web development trying to learn Angular but that seems like it may be premature. It is a lot to chew over. Angular unlike React and Vue is considered a framework. It is the whole kitchen-sink. It provides everything you need to make an interactive web app. With React and Vue, you need to add your own libraries for non-GUI things.

Both are just UI libraries which makes them quicker to learn. Conceptually Vue.js looks more similar to the approach taken by Angular. React has invented a sort of templating language on steroids called JSX, which mixes JavaScript and HTML into one language.

Vue and Angular instead extends HTML with what they call directives. This is basically attributes which don’t exist in regular HTML tags. These are used to control the actions of the templating engine.

My present idea is thus to pursue Vue.js rather than Angular as it seems like a smaller learning curve.

How to Select Nodes in The DOM: XPath vs CSS Selectors

When working with a document object model tree, we want effective ways of locating one or more nodes. Perhaps we want to alter their properties, remove them or add something to them.

CSS uses CSS selector syntax to pick one or more nodes to apply a style to. XPath in contrast is a more generic system, not tied specifically to styles. XPath is older and seems to be going out of fashion, partly because it is harder to work with an often slower. XPath however offers more flexibility. With XPath you can move both up and down a tree, while CSS selectors is just for moving down.

XPath example: Selects the first book element that is the child of the bookstore element.

/bookstore/book[1]

Here is the equivalent using CSS selectors:

bookstore > p:nth-child(1)

JavaScript on the Desktop and Server: Node.js and Electron

A lot of people got tired of writing in JavaScript on the webpage (front-end) and Java, Python or Ruby on the server side (back-end). They wanted the same language both places and thus Node.js was invented. It is basically JavaScript running outside the confines of the browser.

This gave new opportunities for making desktop applications using JavaScript and thus Electron was born. What they did was to strip the core out of the Chrome browser used for rendering HTML and paired it with Node.js.

As a desktop developer the best way for me to compare this with technology I know of is to compare this to a game engine, such as Unity3D. In a game engine you typically got a C++ application running which does the heavy lifting of rendering 3D graphics, checking collisions etc. However specifying the layout of a game level, the character behavior etc is better done in a flexible script language. Hence the engine can run scripts to control its behavior. I think of Electron the same way. It is sort of like a higher performance Engine, which can read scripts written in Node.js compatible JavaScript. What I mean by that is that it has access to similar libraries and have the same legal syntax. Since Node.js can access more functionality on the platform, it offers more flexibility in making applications than pure web. JavaScript in the browser is essentially sandboxed and cannot easily access files on the native platform.

Transpilers and WebAssembly: Alternative Programming Languages For the Web

If you are a JavaScript hater like me, a significant barrier to the Web is the usage of JavaScript. Fortunately the language is evolving and getting better, but it has for long been a rather kludgy language.

Evidently I was not the only one having issues with JavaScript, which has created a whole cottage industry of people and companies making so called transpilers. I am not sure why that word is used as one could equally well call it code generators or compilers.

Basically a JavaScript transpiler is a compiler who’s output is not machine code, but JavaScript code. That way you can write a web application in an entirely different language than JavaScript and just compile it to JavaScript, which is the only language the browser understands (until recently that is).

A major flaw of JavaScript is that it is very weakly typed. That was fine for what JavaScript was designed for. It was designed to just have 20–50 lines of code on a web page. However when applications start having tens of thousands of lines of code, this weak typing becomes a problem.

One of the early transpilers was thus TypeScript, which is just JavaScript with added type information. The transpiler could then verify that you were not mismatching types.

Python Clone?

Other transpilers tried to make nicer syntax, such as CoffeScript, which seems a bit Python inspired. It uses whitespace more actively to indicate code blocks.

Haskell Clones?

Another language which has inspired creators of Transpilers is Haskell. Haskell is a very strictly and statically typed functional language. That means more bugs are caught at compile time at the expense of a more complex type system. Both Elm and PureScript have been inspired by Haskell.

Elm seems to be the pragmatic approach, while PureScript seems like a more dedicated effort to create a Haskell clone. I suspect that those who are willing to invest in learning either of these languages will see large benefits with building larger web applications. The strict typing will help avoid many bugs.

The problem is that getting into strong functional programming takes time.

LISP/Clojure Clone?

Another popular but dynamically typed functional language is LISP/Scheme which inspired the creation of Clojure running on the Java platform. Clojure inspired the creation of ClojureScript, which compiles to JavaScript. The toolchain used with ClojureScript and libraries are essentially the same as those used by Clojure.

ClojureScript seems to me a bit hampered by its reliance on the large Java ecosystem. Others may see it as a benefit. The positive potential as I see it, is that being a LISP makes it very well suited for designing domains specific languages. Creating abstractions for expressing CSS and HTML should thus be more elegant in this language than the alternatives.

Java Clone?

However for a lot of developers Haskell and Clojure are rather alien languages. Is there something looking more mainstream? Dart developed by Google does in fact look like such a language. It looks like a fairly standard statically typed OOP language which users of C#, C++ and Java would be familiar with.

As a big fan of Google’s Go, I was hoping for something with the same kind of simplicity and ease of use. Dart does not seem to be like that at all. In fact it is rather complex, and makes me think more of C++ than Go. If you want to get into web development quickly while avoiding JavaScript, Dart does not seem to be the solution.

It does look very well designed IMHO, however it is not something you learn quickly. And if want a better Java/C#/C++ then frankly I think Swift is a much better choice. Of course Swift does not run in the browser.

Multiple Targets: Nim and Haxe

Lesser known alternatives are Nim and Haxe. These are not purely JavaScript transpilers as these are designed as multipurpose languages which can compile to native code as well.

Nim can be used as a systems language for instance. To me Nim seems like a sort of mix between Swift, Kotlin, Dart and Python. Basically a modern statically typed language with nice expressive syntax. Nim has an enthusiastic following and seems like a very well designed language.

The problem is that Nim does not have big company backing by the likes of Facebook, Google, Apple or even Mozilla. What seems interesting to me personally is that it is supposed to have a well designed meta programming facilities. As a LISP and Julia lover, it is something I really like having in a language.

Haxe, I frankly don’t know well. It is often compared with Nim, and has more corporate backing and is perhaps more geared towards the web. However one never seems to see as much enthusiasm for it as Nim.

High Performance System Languages: WebAssembly

Ultimately one should not have to compile down to javascript to run another language in the browser. That is what WebAssembly tries to solve. It specifies a sort of assembly language for the web browser. Several web browser already support it.

Web assembly can be written in many different formats. One of the formats uses what is called s-expressions, which is also how LISP programs are written. Here is an example of WebAssembly in such a format.

(module
  (import "math" "exp" (func $exp (param f64) (result f64)))
  (func (export "doubleExp") (param $0 f64) (result f64)
    (f64.mul
      (call $exp
        (get_local $0)
      )
      (f64.const 2)
    )
  )
)

A benefit of this format is that it can very easily be manipulated, especially by a LISP program. However this is just a text format. Normally it will be stored in compact binary format. However this format can easily be converted easily to different text formats such as this s-expression format.

Web assembly allows us to compile C++ code to assembly which can run in a web browser. This is useful for e.g. games.

At the moment the utility of WebAssembly is limited because it does not support garbage collection and cannot manipulate the DOM tree. However these features are supposed to be added later. That means for regular web programming it is not yet a full replacement for JavaScript.

It means we cannot compile garbage collected languages such as Java, Go, Python and Ruby to Web Assembly.

Final Remarks

This is a writeup to the best of my knowledge of what role the different web technologies serve in the whole ecosystem. I may very well have made some errors.