After a year of heads -down development to port Docker's open source project from Linux to Windows Server 2012, I finally have some time to describe some of the technical challenges we faced. But first, a quick introduction to containers.
Containers are described as the natural replacement for virtual machines, since they are smaller in footprint and faster than virtual machines. Technically, you can think of them as multiple copies of the same libraries (DLLs on Windows) running on the core operating system. Each container provides an isolated system to a user to run their services.
For us, the challenge was that the Windows Server 2012 kernel does not support the constructs needed for containers. Our product, Windocks, uses Windows Job objects as the basis for a container, so there are plenty of limitations. Our goal for WinDocks is to support the development and use of trusted software by an organization, on a dedicated VM or bare metal host. This goal appears to be shared by Microsoft and AWS, both who deliver container services with VMs for assured security.
Now to the challenges in porting the docker engine....
Docker is written in Go and the building of the docker engine requires docker itself ! Programmers will appreciate the infinite recursion irony here since on Windows, we had no Docker binaries to begin with! So, we did things the hard way using Go and C++ compilers on Windows.
The code that creates the containers (Job Objects and more) is in C# and our second challenge was the communication between the Go docker engine and the container code in C#. Luckily, this proved easy by using an intermediate layer written in C++. As you can see, our use of programming languages seemed to grow by the day. First, Golang, then C#, then C++!
Another challenge we faced was that the docker engine creates goroutines (threads) for each request. Our design had to be conservative enough to prevent thread related bugs but at the same time had to be aggressive enough to speed execution of parallel docker commands from multiple users. While it was tempting to put global locks on each docker command, we resisted the temptation! We have a locking design which uses container ids, the nature of the docker command and more to decide on the granularity of the lock. We tested our locking by slamming the Windocks server on a 4 core machine with 10+ docker requests simultaneously and found no issues (I hate to say no bugs - call me pessimistic!)
Perhaps our hardest challenges were around SQL Server where we implemented the ability to add database copies into the container or mount remote databases (with or without cloning). While, SQL Server was never really designed for mdf / ndf file manipulation, we have managed to make the adding of database files or mounting of remote database files work reliably in Windocks. We were able to create a SQL Server container with a clone of a 750 GB database in 45 seconds, see this video.
If our SQL Server challenges were hard, I think the value of this is paramount to Windocks, Microsoft and the devops community. I have a strong bias towards SQL Server and consider it one of the finest database servers around, particularly for its ability to do high performance bulk inserts and updates. With Windocks, Microsoft can open its doors to a much higher volume of SQL Server usage because of the ease of getting it started for multiple developers and testers.
It has been a fascinating technology challenge for us to port the docker engine to Windows Server and we are quite excited to be the first ones to market with a commercially available solution on Windows that supports many of the docker commands.