Introduction
I would like to explain how testing on clean PC will help you to overcome the issues that may arise due to presence of additional software or improper settings on the Test PC.
Many of the test engineers perform testing on PC that have lot of additional programs installed like messengers, browser toolbars and products that your potential clients or customers may not be using. Also, a test engineer involved in functional testing may install different versions of the AUT during the testing life cycle. Most of the install or uninstall programs prompt the user whether to uninstall a shared file or dll during the uninstall process. However, the fact is that most of the users ignore this message due to lack of knowledge or awareness. Also, the install program written for your software may fail to remove some of the registry entries.
I have across situations where the client was unable to use the software due to the absence of some files in his machine during the starting of my career. After a little bit of research we came to know that the PC was used for testing had third party software, which had the missing file. But for our bad luck, the customer did not have the same software installed.
Test Environment
Many companies have adopted a phase in their testing lifecycle called installation testing or ignition testing where the software is tested in a clean environment. However, this is normally carried out after regression or system testing. A clean environment refers to a fresh PC loaded with the required OS and with the software(s) your customers will be using.
It will be a good idea to figure out the actual environment used by the client during the requirements phase. This applies to both client PC’s, Application and database servers. In addition, some of the settings of your PC like regional settings and browser settings also play a major role in identifying issues or defects. It is a good practice to maintain a checklist for the PC that you will be using for your testing and configure the same. The usage of test automation tools has increased in the recent past. Many tools available in the market have issues identifying GUI objects inside the AUT if one change the settings like display properties in the OS. Therefore, it is important to maintain the same settings across all the PC’s used for testing.
Also, consider an AUT that has to be tested on three different OS (Win 98,XP and 2000) with IE 6,IE 7 and Netscape 7. If the customer that you are working for requires a weekly build then the time that one will be allocated for testing will be limited. In this case, one has to have nine fresh PC’s with OS and browsers to deliver the product after testing. Normally the process of setting up a PC with OS and configuring the same with the checklist takes a minimum of 3 hours time. The time required to setup the PC’s each iteration of testing will be close to 27 person hours. One should not forget the overhead of maintaining the nine PC’s. So, this approach is not feasible for performing testing on a clean PC during the time constraint.
Ghosting
So how do we overcome this problem? Most of the branded PC’s in the market is sold with the OS installation in today’s market. Have you thought how PC manufacturing companies install OS in the PC’s that they are selling? Any Windows OS installation takes a minimum of 1.5 hours. So a company that manufactures 500 PC’s a day has to spent 750 person hours to install an OS. Most of the companies will not be ready to spent time and valuable resources for this.
I got an opportunity to visit one of the hardware-manufacturing units in India a few years back. The company had a mechanism to install OS on a new PC within 7-8 minutes. They were using ghosting software called Drive Image Pro for this. All the PC ‘s manufactured in a department required almost the same display or sound card drivers. A PC will be loaded with the OS and the required drivers and a ghost image of the hard disk is taken. The same images are restored to all the PC’s later within minutes. The same concept can be adopted for setting up PC’s for testing.
There are many tools available in the market for this purpose like Norton Ghost, G4U, and G4L etc. Install the required OS and software’s and capture an image of the same. On has to make sure that your image is void of your AUT. You may have to maintain multiple copies of images based on the browser versions or OS that one is going to test. Restore the image, install your AUT and use them whenever you do your testing.
But this approach has its own limitations too. I have listed down some of them below.
One has to maintain different copies of images based on the OS, browser etc.
All the images have to be updated once a patch or service pack is released for your OS.
One should have sufficient privileges to rename PC’s if your try to use the same image for all test pc’s in a networked environment.
Most of the ghosting tools operate from a bootable CD or startup floppy and one may require physical access to the PC.
System Restore
Many of Microsoft OS likes ME and XP is having a feature called System restore. The System Restore feature enables administrators to restore their computers to a previous state without losing personal data files (e.g. Word documents, graphic files, e-mail). System Restore actively monitors system file changes and some application file changes to record or store previous versions before the changes occurred. Nevertheless, this method also has its own limitations.
System Restore monitors only a core set of specified system and application file types (e.g. .exe, .dll etc).
System Restore does not monitor changes to or recover users' personal data files such as documents, graphics, e-mail, and so on.
System data contained in System Restore's restore points are available to restore to for only a limited period.
System Restore does not completely uninstall any program if restoring to a point prior to the program installation.
You can find more information regarding system restore in the Microsoft site.
Virtual Machine Technology
VM technology helps you to run multiple OS environments on a single server or physical hardware system. Current technology like the MS Virtual Server 2005 R2 has added a layer on top of the OS in the physical environment. In addition, one can allocate virtualized hardware resources to Virtual Machines.
For example consider a high end desktop PC with 4 GB RAM and 100 GB HDD. One can install the host operating system with 1 GB RAM and 40 GB HDD space. After this, one can go ahead and create six VM ‘s with each VM having 512 MB RAM and 10 GB HDD space. . The technology also can package an instance of an OS and the configure applications into a VHD file. These files can be rapidly deployed on to your test environment very easily. One has to maximize test hardware to reduce costs and to cover more additional test scenarios. This approach is easy to use and has gained popularity fast. You can find more information on this in the site Microsoft Site. Also, keep in mind that the one should have separate licenses for the VM’s installed on Host OS. Some of the limitations are given below: -
If the Virtual Server is down due to hardware issues all the VM’s installed on the Host will be unavailable.
In addition, one has to invest money to purchase high end desktop PC’s.
This approach can be used only for non-production purposes.
Conclusion:
In this article, I have explained the advantages of using a clean PC for testing. I then went on and explained the different approaches that can be used to setup a Clean PC. One should evaluate the various options explained before implementing the same in their organization.
I would like to explain how testing on clean PC will help you to overcome the issues that may arise due to presence of additional software or improper settings on the Test PC.
Many of the test engineers perform testing on PC that have lot of additional programs installed like messengers, browser toolbars and products that your potential clients or customers may not be using. Also, a test engineer involved in functional testing may install different versions of the AUT during the testing life cycle. Most of the install or uninstall programs prompt the user whether to uninstall a shared file or dll during the uninstall process. However, the fact is that most of the users ignore this message due to lack of knowledge or awareness. Also, the install program written for your software may fail to remove some of the registry entries.
I have across situations where the client was unable to use the software due to the absence of some files in his machine during the starting of my career. After a little bit of research we came to know that the PC was used for testing had third party software, which had the missing file. But for our bad luck, the customer did not have the same software installed.
Test Environment
Many companies have adopted a phase in their testing lifecycle called installation testing or ignition testing where the software is tested in a clean environment. However, this is normally carried out after regression or system testing. A clean environment refers to a fresh PC loaded with the required OS and with the software(s) your customers will be using.
It will be a good idea to figure out the actual environment used by the client during the requirements phase. This applies to both client PC’s, Application and database servers. In addition, some of the settings of your PC like regional settings and browser settings also play a major role in identifying issues or defects. It is a good practice to maintain a checklist for the PC that you will be using for your testing and configure the same. The usage of test automation tools has increased in the recent past. Many tools available in the market have issues identifying GUI objects inside the AUT if one change the settings like display properties in the OS. Therefore, it is important to maintain the same settings across all the PC’s used for testing.
Also, consider an AUT that has to be tested on three different OS (Win 98,XP and 2000) with IE 6,IE 7 and Netscape 7. If the customer that you are working for requires a weekly build then the time that one will be allocated for testing will be limited. In this case, one has to have nine fresh PC’s with OS and browsers to deliver the product after testing. Normally the process of setting up a PC with OS and configuring the same with the checklist takes a minimum of 3 hours time. The time required to setup the PC’s each iteration of testing will be close to 27 person hours. One should not forget the overhead of maintaining the nine PC’s. So, this approach is not feasible for performing testing on a clean PC during the time constraint.
Ghosting
So how do we overcome this problem? Most of the branded PC’s in the market is sold with the OS installation in today’s market. Have you thought how PC manufacturing companies install OS in the PC’s that they are selling? Any Windows OS installation takes a minimum of 1.5 hours. So a company that manufactures 500 PC’s a day has to spent 750 person hours to install an OS. Most of the companies will not be ready to spent time and valuable resources for this.
I got an opportunity to visit one of the hardware-manufacturing units in India a few years back. The company had a mechanism to install OS on a new PC within 7-8 minutes. They were using ghosting software called Drive Image Pro for this. All the PC ‘s manufactured in a department required almost the same display or sound card drivers. A PC will be loaded with the OS and the required drivers and a ghost image of the hard disk is taken. The same images are restored to all the PC’s later within minutes. The same concept can be adopted for setting up PC’s for testing.
There are many tools available in the market for this purpose like Norton Ghost, G4U, and G4L etc. Install the required OS and software’s and capture an image of the same. On has to make sure that your image is void of your AUT. You may have to maintain multiple copies of images based on the browser versions or OS that one is going to test. Restore the image, install your AUT and use them whenever you do your testing.
But this approach has its own limitations too. I have listed down some of them below.
One has to maintain different copies of images based on the OS, browser etc.
All the images have to be updated once a patch or service pack is released for your OS.
One should have sufficient privileges to rename PC’s if your try to use the same image for all test pc’s in a networked environment.
Most of the ghosting tools operate from a bootable CD or startup floppy and one may require physical access to the PC.
System Restore
Many of Microsoft OS likes ME and XP is having a feature called System restore. The System Restore feature enables administrators to restore their computers to a previous state without losing personal data files (e.g. Word documents, graphic files, e-mail). System Restore actively monitors system file changes and some application file changes to record or store previous versions before the changes occurred. Nevertheless, this method also has its own limitations.
System Restore monitors only a core set of specified system and application file types (e.g. .exe, .dll etc).
System Restore does not monitor changes to or recover users' personal data files such as documents, graphics, e-mail, and so on.
System data contained in System Restore's restore points are available to restore to for only a limited period.
System Restore does not completely uninstall any program if restoring to a point prior to the program installation.
You can find more information regarding system restore in the Microsoft site.
Virtual Machine Technology
VM technology helps you to run multiple OS environments on a single server or physical hardware system. Current technology like the MS Virtual Server 2005 R2 has added a layer on top of the OS in the physical environment. In addition, one can allocate virtualized hardware resources to Virtual Machines.
For example consider a high end desktop PC with 4 GB RAM and 100 GB HDD. One can install the host operating system with 1 GB RAM and 40 GB HDD space. After this, one can go ahead and create six VM ‘s with each VM having 512 MB RAM and 10 GB HDD space. . The technology also can package an instance of an OS and the configure applications into a VHD file. These files can be rapidly deployed on to your test environment very easily. One has to maximize test hardware to reduce costs and to cover more additional test scenarios. This approach is easy to use and has gained popularity fast. You can find more information on this in the site Microsoft Site. Also, keep in mind that the one should have separate licenses for the VM’s installed on Host OS. Some of the limitations are given below: -
If the Virtual Server is down due to hardware issues all the VM’s installed on the Host will be unavailable.
In addition, one has to invest money to purchase high end desktop PC’s.
This approach can be used only for non-production purposes.
Conclusion:
In this article, I have explained the advantages of using a clean PC for testing. I then went on and explained the different approaches that can be used to setup a Clean PC. One should evaluate the various options explained before implementing the same in their organization.
Good article,keep bloggering.
ReplyDelete