The challenge of test or staging websites in a Digital Agency

Digital Agency website development is usually fast-paced and high pressure. Every effort is made to ensure zero down time for your client’s websites and zero loss of data. But in this environment mistakes can and do happen. One of the main reasons for this is the complexity involved in maintaining proper staging websites. I want to explore some of the challenges and the amazing solutions that are now available for Digital Agencies.

Any Digital Agency development environment has to try to satisfy 3 important elements:

Time

Time is money. If you take too long to develop something (longer than the amount of hours traffic assigned) you are costing the company money. As a result developers often put in hacked together, sticky tape solutions, which are likely to break the next time you do something on the site.
~

Data storage

A Digital Agency is all about getting leads for their clients. You absolutely cannot EVER lose this lead data.

Testing & live websites

Nothing gets changed on a client’s website unless they have viewed it on a test website & approved it. So you must run a test website for every live website you manage. The idea being that the test site is a copy of the live site where you can make & test changes. Then move those changes to the live site. But this is where the most problems are likely to occur. Let me explain.

Setting up staging websites

To start, you simply copy your live website into a test environment, where you are going to do the development. At this point in time you have a perfect copy of your live site.

 

So now you start working on this test site, making changes, trying things. And eventually, you manage to get it right, just on deadline. So you quickly copy your test site to your live site. The problem here is that while you were developing on the test site, the live site was still gathering lead data. And now, when you copy your test site database across to the live site, you overwrite that lead data. Oops! Or perhaps in the process of copying the files across, you miss a couple of vital ones. Next thing you know the client’s live site is down. Big oops!

What’s more, because of the time deadline, you did not go back and clean up the sloppy code you tried which did not work. You were careful & did not copy this to the live site (whew), but the next person who comes to do some dev work on this test site won’t understand why their code is not working. And so on, until the test site is no longer anything like the live site.

Your staging website is a perfect copy
Your staging website perfect copy after a little while

The only way to avoid this is to start with a fresh copy of the live site every time but this is time consuming…

Moving websites is not particularly difficult. It is a complex process and there are lots of places where you can make a mistake, that is true. But we are not talking about moving websites here. What we are talking about here is version control and merging data. The trick with managing a successful testing and live website environment is the merging of changes and being about to undo those when needed. So how do we achieve this flawlessly?

Let’s break it down into what we need:

Staging website: Ideally, every time you start to do any further development on a website, you should first copy the live website so you know you are working on the right files and data. But how can an agency explain this development cost. Every time you want to add a new blog post, you first need to copy the whole website? Well, yes, in an ideal world. But it can take 1 or 2 hours to do this, depending on the website. For the client this means that the dev time to add a new post, 1 hour, is actually doubled because you first need to make a copy. So the only possible solution here is to be able to copy your live website into a staging environment within a few minutes. And it must be a simple process with little room for error.

Version control: You need to keep a record of changes and be able to undo these changes if needs be. This is particularly difficult in an agency where multiple different people will work on the same development project. We need to track all changes, made by all people, on all projects. Sounds easy right?

Merging of data & backups: After making changes in the staging environment and getting client sign-off, you need to be able to merge your changes back into the live website, without losing any of the data on your live website. And we need to ensure that we make constant backups of our live website in the event of some big failure, meltdown ,hack etc…

The solutions

1. Using Gitbucket to manage version control

Gitbucket is a fantastic open-source project. It allows you to keep an online repository of any digital files and it will track all changes to those files. Basically, if you want to work on something, you first do a pull request to get the files. You use those files to setup your WordPress website on your local machine (using XAMP or something similar). Then you make your changes and finally commit those back to the repository. Your bucket is super clever and will keep track of all changes. It will even track changes made by various people (each doing their own pulls & commits). And when you commit, it will merge the changes.

Here is a great tutorial on exactly how to setup a staging website and use Gitbucket to track the changes. If you read that tutorial you will notice a few things. There are lots of steps in the process and some are quite technical. This means training is required and the potential for error is greater. It is also quite a time consuming process, pushing changes to a test site first, then copying those same changes to the live site, all this on top of the development time you already used making the changes.

So my 2 big issues here is that most new staff members need to be trained to use this system. And it is still a time consuming process.

I did find an interesting open-source project but at this point in time, it is not ready to be used in an agency environment. It is called Versionpress and it simplifies this process quite a bit! But I also found some even better options.

2. Dedicated WordPress hosting

There are 2 dedicated WordPress hosting companies that tick all the boxes (except one) for me. They are Flywheel and WPengine. Their pricing and services are pretty similar so you can do your own comparison and decide which is best for you. For me it is important that they both offer the following services:

  • Easy staging websites and changes merging: In a few clicks you can create a copy of your live site in a staging area. This site is automatically marked as “no-index” so Google won’t crawl it and you can even password lock it. You can get FTP access to it to make file changes. And, most importantly, when you are finished making changes, you can choose how and what to merge back into your live site. Each step is logged and you can undo at any point in time.
  • WordPress updates & security: They will keep your website up to date and they keep it as secure as possible. They do constant malware scans and automated backups every night!
  • Support: With their Enterprise/Agency accounts, they offer 24/7 ticket support and a dedicated account manager

They both offer a lot more than just this but those are the important factors for me. This kind of hosting allows a Digital Agency to provide the kind of service their clients expect, within the time frames it needs to be done.

If I had to choose one over the other, I would probably go with Flywheel. Mostly because they offer a couple of nice extras like “demo sites”.

The only box it does not tick for me is server location. If you are based in South Africa (like me) and your clients are also here, and their clients/website users are also in South Africa, then it makes sense to host in South Africa. It is just quicker! At the time of writing WPengine offers servers in the US and UK, while Flywheel offers servers in the US, Europe & Asia.

[As someone from SA who has hosted in Germany for years I can tell you that the connection breakdowns are getting less and less frequent, while the connection speeds are steadily increasing. I believe that in the near future this point of server location will not actually matter.]

Get Flywheel WordPress hosting

WP Engine hosting

3. Dedicated WordPress Staging or Testing websites

WP Stagecoach: no more cowboy codingWPStagecoach is another alternative that provides dedicated WordPress staging sites. It does not offer ALL the benefits of dedicated WordPress hosting as mentioned above, but it is also significantly cheaper!

It’s fairly simple. First you sign-up with them and download their plugin. Install this plugin on your website and in it’s settings, add the API key you get from your Stagecoach account (they have extensive documentation to guide you). That’s it, now you’re ready to ‘ride the stagecoach’.

This plugin allows you to create a staging website (on their hosting) with a name you choose. They will give you the URL and FTP details to access it. From there you work on it like any other testing site. When you are done, simply send the link to your client for approval. Then you go back to the Stagecoach plugin, on your live website and tell it to check for changes on the staging site. If it finds changes it will ask you what to do with them. You can merge them back into the live site to ensure no data is lost or even overwrite the live site (be careful with this). Most importantly, if anything goes wrong, you can undo easily.

And finally, at the end of the process, you are prompted to delete your staging site. This will ensure that the next person who works on the staging site starts with a clean copy of the live site.

What they do not do is backups, but most good hosting providers do this for you anyway. And you get many WordPress plugins that will do this too. It also does not do updates but given how easy it is to create a test site, it is now a quick process to setup a test site, do the updates, check that nothing is broken and pull those updates through to your live site. So while you still have to do your own maintenance, it is now a much simpler task.

In closing

Digital Agencies are expected to produce a very high level of work under enormous time constraints. Their development processes need to be simple to remain foolproof and cost effective. Dedicated WordPress hosting or staging does come at a premium cost but can the modern agency afford to take a chance anymore? I say it makes sense to pay a little extra up front, to ensure that your project are done in time and without any mistakes such as the loss of data. Fixing mistakes or trying to recover lost data is going to work out more expensive in the long run. I therefore think it is imperative that any Digital Agency makes use of the services I have recommended here.

I have tried to offer a range of choices to suit all situations (obviously focused on WordPress which is my preference). If anyone has any other suggestions, please feel free to comment below.