First introduced by Scalingo in January 2019, SAND is an autonomous service managing overlay networks that integrates with Docker. SAND (Scalingo Awesome Network Daemon) is designed for web-scale companies like Scalingo, managing thousands of containers every day. SAND features a simple API to create private overlay networks based on VXLAN to link containers together. One of the main advantages of this solution is that SAND is agnostic of the underlying container technology. It’s one of the cornerstones of the Software-Defined Networking (SDN) infrastructure of Scalingo’s platform.
At Scalingo, SAND is a critical piece of infrastructure powering some parts of container networks. It is currently used to run hundreds database clusters.
We are excited today, to announce the open sourcing of SAND. By allowing others in the software-defined networks community to leverage unified schedulers and workload co-location. Scalingo will open the door for more ways to manage networks. Moreover, open sourcing SAND will enable greater industry collaboration and open up the software to feedback and contributions from industry engineers, independent developers, and academics across the world.
Features in the current release
Scalingo has been running SAND for about half a year in production and it has been running and scaling very well. It aims at providing a way to manage private overlay networks. It is written in Go. The API lets you CRUD a private network and adds any entity (called endpoint) to this network.
The initial release also provides a Docker driver to easily integrate with this container engine.
Benefits of using SAND
Private networks created with SAND are based on the VXLAN (Virtual Extensible LAN) technology. VXLAN isolates different networks just like VLAN would, but encapsulate the layer 2 Ethernet frames into layer 4 UDP datagrams. This protocol makes VLAN more scalable.
While Docker provides libnetwork which allows to define an overlay network, it is strictly bound to Docker. On the contrary SAND is completely agnostic of the underlying container technology and lets you use something else than Docker.
The SAND network daemon should be installed on all the hosts which will have containers running in one of the overlay networks. The network configuration is stored on the key value store etcd.
Each created overlay network can use its own IP range, but there is no problem
using the same range for different networks as they are completely isolated
from each other. By default each overlay network will get IP addresses in
Creating a network is a no-op operation where a unique VXLAN ID is allocated and where the network configuration is stored on etcd.
When a first endpoint is added to a network, the service will create a
dedicated network namespace containing the network VXLAN interface on the
server which is adding the endpoint. A pair of virtual Ethernet interfaces
veth) will link the targeted namespace (the container network) and the
overlay namespace. All the veth interfaces are linked to the VXLAN with a
bridge interface. As a consequence, there is no impact on root namespace of the
server running SAND, everything is handled in dedicated namespaces.
When an endpoint is added or removed, all the other hosts having at least one endpoint in the same network are adding routes to the newly created endpoint modifying ARP and FDB tables of the VXLAN interface, allowing routing between the different members of the overlay network.
How to use it
Standalone Command Line Interface
SAND provides a command line interface allowing operators to manipulate and overview the overlay networks and their members:
# On any member of the cluster $ sand-agent-cli network-create New network created: * id=ed80c475-782d-4507-bd80-631b476d9ecc name=net-sc-ed80c475-782d-4507-bd80-631b476d9ecc type=overlay ip-range=10.0.0.0/24, vni=13 # On the node which is running the container to link $ sand-agent-cli endpoint-create --network ed80c475-782d-4507-bd80-631b476d9ecc --ns /var/run/docker/netns/73d3c79736d3 New endpoint created: * [ACTIVE] ID=6ffc3990-b212-47f2-a5d1-2afe7526e548 networkID=ed80c475-782d-4507-bd80-631b476d9ecc hostname=dev.172.17.0.1.xip.st-sc.fr IP=10.0.0.2/24 NS=/var/run/docker/netns/73d3c79736d3 # The SAND overlay network is now accessible from the targeted namespace $ nsenter --net=/var/run/docker/netns/73d3c79736d3 ip addr 483: sand0@if484: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default link/ether 02:84:0a:00:00:02 brd ff:ff:ff:ff:ff:ff link-netns sc-ns-ed80c475-782d-4507-bd80-631b476d9ecc inet 10.0.0.2/24 brd 10.0.0.255 scope global sand0
Create an endpoint for each entity which has to join the overlay network, they will be able to communicate with each other instantly.
It is also possible to integrate SAND with Docker using Docker libnetwork
The initial Network resource should be created using SAND API/CLI.
$ sand-agent-cli network-create New network created: * id=ed80c475-782d-4507-bd80-631b476d9ecc name=net-sc-ed80c475-782d-4507-bd80-631b476d9ecc type=overlay ip-range=10.0.0.0/24, vni=13 # Integrate the SAND network with Docker networks $ docker network create \ --driver sand --opt sand-id=ed80c475-782d-4507-bd80-631b476d9ecc \ --ipam-opt sand --ipam-opt sand-id=ed80c475-782d-4507-bd80-631b476d9ecc \ my-overlay-network # Use the docker network when creating a container, the SAND network will be setup $ docker run --network my-overlay-network -it ubuntu:latest bash root@feb4eb485d57:/# ip addr 487: sand0@if488: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default link/ether 02:84:0a:00:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 10.0.0.2/24 brd 10.0.0.255 scope global sand0
Trying out SAND
With SAND we developed the underlying technology to build private networks automatically via an API aka Software Defined Networking. Currently we’re using it to isolate database clusters from our Database as a Service offering. In the future we envision many fun applications like Private Spaces: all your apps and databases in the same private network.