Building a New Internet From the Ground Up – Part 4: Network

After introducing the ThreeFold Grid in part one and diving deep into storage and compute, we’re back with part 4 to look at the networking aspects of the ThreeFold Grid.

Computer networking is a rather complex phenomenon with a thankfully intuitive outcome: you send messages from one place to another. We all understand that our digital devices become a lot less interesting when they are not connected to the Internet. However, networking is not just Internet, and when it comes to building a new Internet, it’s important to understand what that means.

The existing Internet can be defined as a global set of networks and protocols that enable us to do things like load web pages and use apps that rely on external data. It depends on lots of underlying infrastructure, including fiber optic cables running along the ocean floor, our home routers and their connection to an Internet service provider (ISP), lots of cell service towers, and an increasing number of satellites. All of that makes the Internet possible, and it’s still required for a new Internet to function.

ThreeFold uses and builds new protocols to create a network that is interoperable with the existing Internet. These include private overlay networks, the Planetary Network based on Yggdrasil, and various interfaces to the public Internet. Let’s see how they work.

🔗Private overlay networks

Every deployment on the ThreeFold Grid belongs to a network, which is one of the Grid primitives. These networks allow data to be transmitted between workloads in a private and secure manner. They are called overlay networks because they pass traffic over some underlying network. That could be within a single node or between nodes on infrastructure provided by a farmer. The Grid uses software called WireGuard for this purpose, a simple, fast, and secure option among its alternatives.

By means of encryption, the traffic on a private overlay network is unreadable to any other device on that network. Likewise, the workloads within the private network have no visibility of the underlying network. Individuals who deploy workloads can access their networks over a secure tunnel, assuming that one of the nodes in the network is reachable from the public Internet. This is suitable for workload administration and some private applications. For public services and peer-to-peer communication, the ThreeFold Grid provides other solutions.

🔗Planetary Network

The Planetary Network is an implementation of the Yggdrasil Network, which is a next generation approach to network routing. It is a peer-to-peer solution where all traffic is encrypted and takes the shortest path to its destination. These features make it highly compatible with the ethos and goals of the ThreeFold Grid: privacy, security, and efficiency. The design also allows nodes, workloads, and users to communicate freely without relying on centralized intermediaries.

Many ThreeFold Farmers connect their nodes to a home router which blocks all inbound traffic by default. This is a sensible feature that doesn’t interfere with most Internet usage (enthusiasts who want to host their own website or a gaming server might bypass this, for example). However, nodes on the Grid need to be reachable, especially in order to receive workload definitions from deployers.

The Planetary Network solves this by accepting inbound traffic through an outbound connection to a publicly accessible peer. You can think of this like making a call to an operator and then waiting for another call to be routed to you on that line. Even if your phone is set to block all incoming calls, someone can still reach you if they can reach the operator.

This also provides a way for workloads to communicate with one another and be reachable by users. Anyone can enable the Planetary Network on their own device to access sites and services in a very secure and private way. While the ThreeFold Grid is designed to be fully interoperable with the existing Internet, the Planetary Network provides an alternative which can sit side by side in a seamless way.

🔗Reliable Message Bus

Using the Planetary Network, nodes and Grid users can send messages over the Reliable Message Bus. Notably, it is used for the delivery of workload definitions as mentioned earlier. Zero-OS supports some other RMB messages as well, for doing things like checking available capacity on a node in real time. RMB is a general purpose protocol that can be used by developers in their applications. It queries the ThreeFold Blockchain as a kind of address book for the Planetary Network, enabling nodes and users to find each other in a decentralized way.

🔗Public IPs and Web Gateways

Finally, let’s see how the Grid can offer services over the public Internet using public IP addresses and web gateways. Everytime we type a domain name into our browser, like threefold.io, it is resolved to an IP address. The browser then forms a connection with that address to retrieve the site we’re looking for. Reviving a theme from earlier, that IP must be publicly reachable.

🔗Public IPs

We’re in the midst of a shift from the older IPv4 standard to the newer IPv6 standard. While IPv6 has plenty of addresses for every device on the planet and many more, it hasn’t been universally adopted yet, and IPv4 addresses are in short supply. You likely have one assigned to your router at home, and these days, even getting a second one from your ISP can be a difficult and expensive proposition.

The ThreeFold Grid allows farmers who own blocks of IPv4 addresses to rent them out to deployers. These addresses get attached to individual workloads which are then reachable over the public Internet. A deployer could then assign their own domain name to that IP and host a website there. This is a very flexible way to make Grid deployments publicly available, but it comes at a relatively high cost.

🔗Web Gateways

Web gateways are another way to make workloads on the ThreeFold Grid available over the public Internet. In this case, the farmer may provide a domain name that points to one of their nodes which is publicly reachable. Deployers are then able to reserve a subdomain, like explorer.threefold.io is a subdomain of threefold.io, that routes traffic to their workload. It is also possible for a deployer to point their own domain to the gateway node which then routes traffic to the appropriate workloads.

We glossed over a bit in the earlier description of domain name to IP address resolution. In fact, multiple domains can point to a single IP address and this is common practice among existing web hosts. Adding similar functionality to the Grid helps to ease the demand for IPv4 addresses and is a lower cost solution for deployers than renting a dedicated IP. The web gateway also serves to enhance security and reliability of Grid-hosted services.

The gateway provides a connection from the public Internet to the secure private overlay networks that all workloads use. However, this is not a standard network connection but instead a network socket, which provides security through separation. This is something like passing along the contents of mail pieces without their envelopes. Web gateways also enable redundancy, as multiple gateways can point to the same workload, and multiple workloads can serve the same set of gateways.

When we say the ThreeFold Grid scales limitlessly, this is one of the ways that’s true. Many nodes, many gateways, many applications, and many users can all fit into this model. Deployers can start with a single virtual machine and a single public IP like in the example I’ll use to wrap up this piece below, then ramp up to a globally distributed and highly redundant system as their requirements grow.

🔗Send, receive, complete

Now we’ve seen how the ThreeFold Grid handles networking. From private and secure communication between workloads, to next generation peer-to-peer communication and full compatibility with the existing Internet, these offerings are comprehensive. Not only that, but they can be pretty darn fun to play with too.

In my own recent experience, I deployed a virtual machine on the Grid with a public IP and hosted a simple website with a map of nodes on the Grid that I’d been working on. I was able to quickly share it with some colleagues and members of the community. It went from a project I was tinkering with on my own computer to a live site on the Internet in a matter of minutes, thanks especially to the networking features of the ThreeFold Grid.

For the final piece of this series, we’ll see how everything comes together to turn computer code into a meaningful digital experience. Stay tuned!