Sunday, October 28, 2012

This is Automation Sparta - Multiple Deployment Repositories

This post is the 2nd in a set of posts outlining Automation Sparta - a No Configuration setup for Web Development.

Many months ago I read about multiple deployment repositories using Git in this post:

Since then I've taken it a step further with Mercurial. In this post I'm going talk generally about separate repositories for each environment in your Project Universe (Dev, Staging, Live etc), how they can help you automate deployment, and get yourself of some deployment trouble.

When we start a project we normally have a single repository. In Subversion you may develop in the trunk repository. Many people deploy from the trunk. However it is often considered better practice to have a deployment branch. Having a deployment branch means that you merge into this branch when you're ready to deploy. Deployments happen only from this branch. This saves you having to change configurations to deploy from different branches. When the code has been made Live, the trunk must be merged into so that you have a reference to the current state of the Live environment. Note that in Mercurial, we can "clone" - branching is slightly different. 

This workflow can sometimes be a problem and is prone to human error. For example, what if a developer forgets to merge to the trunk? Now you've lost your reference to what is Live. What does the next developer do when they need to fix a bug? Also, how do you deal with a situation where a feature is waiting to be pushed live, but a bug fix must go up first?

Taking deployment repositories a step further...

Having separate deployment repositories means that your Live repository is always going to reference what is Live. Moreover, you know exactly what is on Dev, and Staging etc.

Some people think that multiple repositories is overkill and a lot of extra work. But having separate deployment repositories also means that you can now automate deployment to your different environments. All you need to do is set up a build trigger on your Continuous Integration server. For example, when a change in the Dev repository is detected, build and deploy. Automating your deployment is extremely powerful.

If it still sounds a bit overwhelming, consider these user stories:
"As a developer I want to drop my new code in a deployment box, so that when it gets dropped in there, an automatic build is triggered, and deployment to the development server occurs". 

"As a developer I want to drop my new code in a deployment box, so that when it gets dropped in there, an automatic build is triggered, and deployment to the staging server occurs". 

So think of these deployment repositories NOT as repositories, but as boxes where your code goes to get deployed. The fact that these deployment boxes are actually full featured code repositories gives you much more power and more options. For example, Developer 1 has features sitting on Dev and Staging waiting to be deployed. Developer 2 has fixed some bugs but doesn't know about Developer 1's changes. When Developer 2 tries to push to the deployment box (repository) for deployment to the development server it will fail. This is a positive thing because Developer 2 now knows that there was actually other work being done, and that they should communicate with Developer 1. 

To deploy to the development server, Developer 2 will simply strip Developer 1's changes on the deployment repository before pushing the bug fix. Developer 2 will also encounter this on Staging, but not on Live because the features were not pushed Live yet. For Developer 1, they will encounter a similar issue when instructed to push the changes Live. Developer 1 now knows that they must merge the bug fix into the feature set before trying to deploy again. And of course going through a proper testing a deploy process to ensure stability of the merged bug fixes. Mercurial and Git make merging automatic (or at least very easy compared to SVN) as much more information is kept for each change set.

We've talked about deployment repositories, but where is it that you are doing your work? Ie. where is your working repository? It doesn't matter how you structure your working/feature/maintenance/main repositories. They can be anywhere you want. All you need to do is put your code into the deployment box for where you want to deploy to.

Hopefully you've been convinced that multiple deployment boxes (repositories) is much more powerful and actually makes your life easier when you couple it with automated build and deployment from a Continuous Integration server.

My controversial statement of the day: I'm sure you can do something similar using TFS and Subverion. But personally I think that if you're not using Mercurial or Git then you're living in the dark ages of version control. And if your company doesnt want to move to a distributed version control system you might consider changing companies, because they're probably holding you back as a developer.


Damiaan said...

I agree, but in my experience i fiund that html/css people don't understand the concept of merging on dvcs's like git and mercurial offer. How can we teach them?
Maybe second remark, is using a dvcs in a 2 member team not slightly overkill in your opinion?
Kind regards

Anthony said...

To answer your first question... experience is the only thing that can teach people about dvcs. My first delve into mercurial from svn was a bit painful. The local commit and server push/pull seemed cumbersome. But I knew deep down that it was the right way to go. Now that I'm used to it, I see SVN and TFS as archaic as Windows 95.

To answer your 2nd question... I use Mercurial for my own projects. I'm a 1 man team. Any time you have 2 or more people committing to the same repo is where you see the real power of Mercurial. In SVN you avoid mergers at all cost. In Mercurial you just click merge and most of the time it does it for you. The reason being that Mercurial stores a hell of a lot more information about each local commit. Enough for it to intelligently merge changesets without human intervention.

Miami Vending Machines said...

Some people think that multiple repositories is overkill and a lot of extra work.
Miami Vending Machines

Anthony said...

Miami Vending Machines (if that is your real name),

If you have to create multiple repositories BY HAND, then yes it is a lot of work.

If it is automated, as part of your development process then it is actually less work than creating a single repository by hand.

And remember, these extra deployment repositories are not meant to be used as repositories. These are deployment BOXES. Your code gets dropped into a staging BOX so that it is automatically deployed to the staging server. The fact that it is actually a repository means nothing. It could be a folder in a shared drive. But a repository gives you more powerful features than a file system folder.

Multiple deployment BOXES also solves some significant deployment issues (as mentioned above).

More info here:

And automating it all allows you do create and deploy a website to multiple servers in literally 10mins. See this presentation:

Shalini said...

Superb. I really enjoyed very much with this article here. Really it is an amazing article I had ever read. I hope it will help a lot for all. Thank you so much for this amazing posts and please keep update like this excellent article.thank you for sharing such a great blog with us. expecting for your.
Digital Marketing Company in India
seo company in india

Post a Comment