With VS2010 ALM you can very easily take a manual test case and turn it in an automated test case. But this isn’t without investment. There are still some additional things to do which take time and money before we get the benefit of automated execution of a test case.
These are some very basic test automation investment levels. Just to keep in mind when working with Microsoft Test Manager and CodedUI.
VS2010 ALM has several automation capabilities. The most noticeable is CodedUI, C# code generated from a MTM action recording or recorded within VS2010 another automation capability is the Fast Forward functionality in Microsoft Test Runner, Fast Forward till a validation step for manual validation. This Fast Forward functionality is also valuable in the shared steps and for test data iterations.
The investment levels explained.
Zero time investment: completely no investment in any kind of automation, no Fast Forward no CodedUI. Although the FF is recorded for every test it isn’t used. MTM is used in this situation not for its automation capabilities, but more for its test case management, bug filling and reporting capabilities.
Which test cases don’t need any kind of automation depends. The most logic reason would be that the test case only is going to be executed once. the other reason can be that automating the test case would be a to big investment and to complex, and there are more reasons.
This post is a useful source “effort-estimation-for-test-automation” to make your decision to automate or not, other good reading is this pdf “TEST AUTOMATION EFFORT ESTIMATION - Best practices”.
Little time investment: executing the same test case on multiple environments, or test cases executed several times during the sprint are candidates for automation, but sometimes the investment of making it CodedUI is too big. Making use of the Microsoft Test Runner’s Fast Forward capabilities will speed up test execution. but, getting that additional benefit of it you have to tune the action recording to make the Fast Forward functionality in the Test Runner more smooth and less error prone.
When executing a manual test case, you never do it correct the first time. You have to search a bit, maybe make a wrong step, go back in the application etc… execute a test script complete correct with the optimal amount of clicks the first time is hard. And when you do click everything correct the first time the action recording will collect a bit too much information, in some situation (see image, opening a browser window with Bing as homepage and than move your mouse around to browse for the website under test). Cleaning the action recording will make the FF more efficient and less error prone. this cleaning of the action recording needs to be done during the execution of the test case in Microsoft Test Runner.
Note: The little investment of cleaning the action recording, also is for the shared step and test cases with test data iteration. Test case which use test data iterations are candidate for a bit more investment for a clean action recording, shared steps are for sure candidates.
Some more investment with Basic CodedUI functionality: Although it is very easy to create CodedUI C# files for already run manual test cases by using their action recordings (hopeful a clean action recording from the little investment step). It is still a bigger in time investment to create them. Beside the time it takes to create the CodedUI C# it also asks an investment in test infrastructure to execute them, see this post “Running Automated Tests on Physical Environments, the different flavors…”.
See image, you also have to create the assertion for the automated manual test case. 1) the manual validation in the Test Case. 2) Add the assertion by using the UI Test Builder 3) drag the crosshair on the application under test and select the property to validate 4) the assertion is added to the CodedUI test.
The main idea for this investment level is that only the basic generated code is used (not customized) and an assertion is manual added to it.
Serious investment in time with advanced CodedUI, customization of the CodedUI and UIMap: Customizing the generated CodedUI is a real trap. You can make the most sophisticated UI tests in the world. Resulting in C# code which is even more complicated as the functionality it tests with probably even as many or more bugs, who tests the tests trap. So, this investment should be done careful, but sometimes it’s a worthy investment. For example for test cases which are going to be run the whole life of the system, or maybe are boring and have many steps. (see the links in the zero investment level.)
Customizations of this level can be started from the ‘UIMap builder’ (see image), change the search properties, move steps out of the code generation and customize them, add fancy test data iteration to the steps, etc etc…
How to optimize your CodedUI for maintainability so the investment will be its money worth for a longer time is a topic for an other post