Continuous Integration with .NET and SQL Server Containers on AWS

Continuous Integration (CI) is a popular topic these days, and the need for better tooling and process is evident in industry surveys. Unisphere Research recently published “The State and Adoption of DevOps,” which shows that over 80% of the market is testing databases twice monthly or less!

WinDocks is a port of Docker’s open source to Windows for .NET and SQL Server continuous integration support. WinDocks supports Windows 8, Windows 10, Windows Server 2012, and Windows Server 2016, with full stack support for .NET and all editions of SQL Server 2008 onward. A pending update will add support for Nginx, Node.js, a “scratch” container image, and a Jenkins plugin.

WinDocks has a growing technical community based on a free WinDocks Community Edition. The Community is largely focused on CI, with a heavy emphasis on SQL Server. We’ve been fortunate to have the opportunity to support a team at WireStorm, and a growing relationship with AWS. This article summarizes Wirestorm’s use of containers for Continuous Integration.

Environments with Latest Code and Data in Seconds

Wirestorm faces common challenges of supporting a geographically distributed development team. In this project they used Angular, .NET, and SQL Server, with the team located in Romania and a couple of US locations.

Each frontend developer required an environment that includes IIS and SQL Server, usually supported by a local virtual machine. Creating and sustaining these environments was an exercise, and each front-end developer needed Visual Studio to support the .NET interaction with the Angular front-end. Copying and maintaining databases with the distributed team was always challenging. In the end, building the environment with updated code and data could take several hours.

Wirestorm implemented their CI solution on AWS, with a single Windows Server 2012 with SQL Server 2014 standard. Each developer is provided isolated but identical environment consisting of the latest .NET app in one container, integrated with a running SQL Server container. A 200 GB database was housed on a NetApp ONTAP VM, which is used to generate snapshots and database clones, which are mounted to the SQL Server containers. Using mounted db clones saves time involved in copying the data into each container, and provides a writable copy of the data.

The .NET and SQL Server containers are defined by Dockerfiles, which specify the base image, container names, port assignments, paths to added or mounted databases, Git repos for pulling code, and SQL Server scripts. The dockerfile also includes the NetApp commands for generating the database snapshot and clones. Incorporating storage system support into Dockerfiles allows a DBA or DevOps engineer to support a dev or test team with self-service environments. Finally, a PowerShell script automates the creation of the complete environment.

A step-by-step view of the process includes:

1) Snapshots of the production databases are made, each being a writable copy. In this example, the snapshots are cloned and mounted to each SQL Server container. Smaller databases can also be copied and run in the SQL Server container.
2) A PowerShell script runs the >Docker build on the SQL Server dockerfile. The dockerfile calls NetApp storage to generate a clone of the target databases. Each is mounted to the SQL Server container.
3) The SQL Server container is started and build-time scripts are run (for data masking, and other needs).
4) The PowerShell script parses the SQL Server container port and credentials to be used in the associated .NET web.config file.
5) Powershell runs the Docker build of the .NET dockerfile. The dockerfile includes a Git pull from the github repos. The web.config file is updated with the SQL Server container port and credentials, and saved. The dockerfile can also specify container ports and naming.
6) The .NET container is started. The integrated environment includes a .NET application based on the latest code and a paired SQL Server container using the most recent DB snapshots.
7) At this point automated Unit Tests are run against the environment.

A shared VM easily supported up to 20 simultaneous running environments, each provisioned in just 100 seconds. The front-end and back-end teams can replicate bugs quickly and validate tests by simply choosing the appropriate branch in the build script. Bugs are resolved quickly and easily.

Do More, Faster, and Spend Less!

Wirestorm used two VMs compared to previous projects involving twenty or more. Team efficiency was dramatically improved, with complete environments provisioned in 100 seconds, and maintenance of a complex development environment being handled by Wirestorm’s DevOps engineer.

Another WinDocks customer uses a bare metal host with 8 cores and 96 GB of RAM, to support twenty SQL Server containers for dev and test. Each container includes over twenty databases running in the container (not mounted). Prior to using containers, each VM took over an hour to provision. Using WinDocks the provisioning time is down to a couple of minutes. As with Wirestorm and other users, the result is a 20x reduction in VMs, a 5:1 reduction in CPU cores, and a dramatic financial savings in MS license costs.

WinDocks “scratch” container for Nginx, Node.js and a Jenkins Plugin

The WinDocks team is excited to be releasing a significant update. WinDocks 1.04 will install as an upgrade on previous releases, and adds support for a new empty container (similar to Docker’s “scratch” image on Linux), along with dockerfiles for support of Nginx and Node.js. Dockerfiles include scripts to automate the port assignment at run time, providing effective image support for these environments. A new Jenkins Build plugin is also available.

Explore the use of .NET and SQL Server Containers

Join thousands worldwide who have downloaded the WinDocks Community Edition, and explore the new world of containers with your own copy at WinDocks Community https://www.windocks.com/community-docker-windows The Community Edition includes sample dockerfiles and PowerShell scripts that replicate the workflow described in this article.

WinDocks is also available in preconfigured VMs available through Amazon’s AWS Marketplace, check it out at: Amazon AWS Marketplace.