Test Environment considerations

One of the things that I have always grappled with is the lack of emphasis placed on test environments. Although several books pay lip service to the topic, there has been nothing concrete, nothing tangible that provides good patterns to use when creating and utilizing test environments ( i would love to be proved incorrect however) Mostly, the test environment verbiage usually speaks to the test environment(s) as some monolithic entity without really speaking to the nuances and the thought process behind creation, build out and usage.  You see, there are actually different stages in the software delivery cycle devoted to testing and the ‘QA’ or testing group typically owns one or more of the stages in that process.
There is movement towards a more team focus on quality, which i am definitely an advocate of, but typically it is still incumbent upon QA to provide independent test verification services and that usually means loosely ‘owning’ a particular testing stage(s). As the industry moves more and more towards Agile methodologies and Service Oriented Architectures there is an increasing need to have multiple software projects concurrently queued up in the pipeline with the possibility of service integration occurring once or more in the cycle before deployment. When rapid delivery is the objective, this can create a burden on your test environment, processes, cause scheduling nightmares and expose other gaps or inefficiencies earlier in the life-cycle.

Some of the questions that typically come to my mind when addressing testing environments are:

  • What types of testing needs to be conducted? i.e. Performance, Replication, Load Balancing, Redundancy, Functional, Integration, etc…
  • How many test stages are there or should there be and what are the triggers for build promotion? In other words, how does software move through the Delivery process?
  • How many sub-environments / ecosystems are needed within this particular environment(s) or test region?
  • What are the bottlenecks in the current environment or process and how can it be made more efficient?
  • Are there opportunities for sharing test environment resources?

In addition to the questions above, I’ve seen multiple variables come into play when discussing test environments, for example

  • UAT, Staging, Readiness, QA, Systems Integration, Alpha/Beta, Performance
  • Average number of active projects in progress
  • Regression testing of existing products or services

I’m currently in the process of expanding on our test infrastructure and process and may be able to speak how we have addressed some of the questions/challenges in the near future.  In the meantime, what’s worked for you?

Advertisements
  1. #1 by QASymphony (@QASymphony) on December 6, 2011 - 4:17 pm

    Hey Stevin,
    You’re exploration into the different environments and stages of testing and QA is a trend that interests me greatly. The trend is constantly changing with emphasis on quality, but the push for a final product coming from so may sub-projects coming from different directions can get exhausting. It seems that most people, according to uTest, have been simplifying this by using the popular tools like HP and Jira.

    I’m a part of QASymphony and we just launched our first product qTrace. We’re working to help testers with our automatic generator for manual tests (i.e. helping the QA workload with less documenting, and more testing). If you could use your expertise and excellent observations, like above, to download our product and give some feedback it would be greatly appreciated.

    Thanks,
    Sarah
    http://www.qasymphony.com/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: