Test Automation Investment Levels for VS2010, MTM and CodedUI

With VS2010 ALM you can very easily take a manual test case and turn it in an automated test case. But this isn’t without investment. There are still some additional things to do which take time and money before we get the benefit of automated execution of a test case.

Picture2

These are some very basic test automation investment levels. Just to keep in mind when working with Microsoft Test Manager and CodedUI.

VS2010 ALM has several automation capabilities. The most noticeable is CodedUI, C# code generated from a MTM action recording or recorded within VS2010 another automation capability is the Fast Forward functionality in Microsoft Test Runner, Fast Forward till a validation step for manual validation. This Fast Forward functionality is also valuable in the shared steps and for test data iterations.

The investment levels explained.

0Zero time investment: completely no investment in any kind of automation, no Fast Forward no CodedUI. Although the FF is recorded for every test it isn’t used. MTM is used in this situation not for its automation capabilities, but more for its test case management, bug filling and reporting capabilities.

Which test cases don’t need any kind of automation depends. The most logic reason would be that the test case only is going to be executed once. the other reason can be that automating the test case would be a to big investment and to complex, and there are more reasons.

This post is a useful source “effort-estimation-for-test-automation” to make your decision to automate or not, other good reading is this pdf “TEST AUTOMATION EFFORT ESTIMATION - Best practices”.

1Little time investment: executing the same test case on multiple environments, or test cases executed several times during the sprint are candidates for automation, but sometimes the investment of making it CodedUI is too big. Making use of the Microsoft Test Runner’s Fast Forward capabilities will speed up test execution. but, getting that additional benefit of it you have to tune the action recording to make the Fast Forward functionality in the Test Runner more smooth and less error prone.

When executing a manual test case, you never do it correct the first time. You have to search a bit, maybe make a wrong step, go back in the application etc… execute a test script complete correct with the optimal amount of clicks the first time is hard. And when you do click everything correct the first time the action recording will collect a bit too much information, in some situation (see image, opening a browser window with Bing as homepage and than move your mouse around to browse for the website under test). Cleaning the action recording will make the FF more efficient and less error prone. this cleaning of the action recording needs to be done during the execution of the test case in Microsoft Test Runner.

12

Note: The little investment of cleaning the action recording, also is for the shared step and test cases with test data iteration. Test case which use test data iterations are candidate for a bit more investment for a clean action recording, shared steps are for sure candidates.

2 Some more investment with Basic CodedUI functionality: Although it is very easy to create CodedUI C# files for already run manual test cases by using their action recordings (hopeful a clean action recording from the little investment step). It is still a bigger in time investment to create them. Beside the time it takes to create the CodedUI C# it also asks an investment in test infrastructure to execute them, see this post “Running Automated Tests on Physical Environments, the different flavors…”.

13

See image, you also have to create the assertion for the automated manual test case. 1) the manual validation in the Test Case. 2) Add the assertion by using the UI Test Builder 3) drag the crosshair on the application under test and select the property to validate 4) the assertion is added to the CodedUI test.

The main idea for this investment level is that only the basic generated code is used (not customized) and an assertion is manual added to it.

3 Serious investment in time with advanced CodedUI, customization of the CodedUI and UIMap: Customizing the generated CodedUI is a real trap. You can make the most sophisticated UI tests in the world. Resulting in C# code which is even more complicated as the functionality it tests with probably even as many or more bugs, who tests the tests trap. So, this investment should be done careful, but sometimes it’s a worthy investment. For example for test cases which are going to be run the whole life of the system, or maybe are boring and have many steps. (see the links in the zero investment level.)

Customizations of this level can be started from the ‘UIMap builder’ (see image), change the search properties, move steps out of the code generation and customize them, add fancy test data iteration to the steps, etc etc…

clip_image004

How to optimize your CodedUI for maintainability so the investment will be its money worth for a longer time is a topic for an other post

Comments (41) -

Real   superb  visual appeal on this  site, I'd  value it 10 10.

It’s truly a nice and helpful piece of info. I’m glad that you shared this helpful info with us. Please stay us up to date like this. Thank you for sharing.

Great post of commenting and how to very well written and all that link adore you just poured out, great post, if this was a forum you would have a sticky on this.

Just came across your site and have to say I'm delighted I found it, bookmarked can't wait for more

Thanks for this. I have a website in a similar niche and this is the third time that I have been back to check out your site. I gave you a little holler to my friends on FB, hope that's cool.

Did the first of many VFX breakdowns. I pull a Top Gear at one point with the line "And then - disaster!"

This is certainly my initial stop by and I really like what I'm seeing. Your weblog is so much fun to look over, quite compelling as well as informative. I'll undoubtedly recommend it to my friends. Nevertheless, I did have some problem with the commenting. It kept giving me an problem whenever I clicked on publish comment. I hope, that can be fixed. Many thanks

Ich möchte dass Sie wissen, dass mir dieser Blog sehr Freude bereitet hat. Ich habe sie bereits in meine Bookmarks geladen und werde sie auch meinen Bekannten geben. Sollten Sie meine Netzauftritte über VDD / Verdopple Deine Dates anschauen, werde Ich euch sehr dankbar sein und mich erkenntlich zeigen. Lesen Sie mehr über VDD / Verdopple Deine Dates von dem Profi David D., dem Pua und Rumkriegen Charakter schlechthin.

Bing has awesome interface, especially the new html5 interface in chrome and IE9 is cool..

Nice. Thanks for posting this. Not sure that I completely understand it, but thanks for taking the time to write this out.

This post is awesome..i've been reading tons of crap posts from other blogs, but shows you have a more educated reader base.

Indeed an article worth reading for. Great choice of topic and the content are brief and concrete. It really help my research.

Interesting post. I'll have to keep it in mind, because its difficult to find such tremendous info also It'll obviously be challenging, so I can use all the help I can get..

This post is very good, rich tourist resources, a wide choice with beautiful environment. By the way I know somewebsites is very well. Such as Hurry up and look at it, more surprise waiting for you!

I must say that overall I am really taken with this site.You obviously know what you are talking about as you write with passion. If only I had your writing ability I have bookmarked your website and look forward to additional updates.

This blog is going to be impressive resource. Thanks a lot for a collection of good tips. I look forward to reading more in the future. Keep up the excellent work!

Microsoft has announced that Khalid Elhasumi, country manager for Microsoft Libya, has been held in custody by authorities in Tripoli since the evening of Saturday, March 19th.

If I had $20,000, what do you believe the highest profitable method is to invest it?

The beauty of organized chaos, I love it! this is totally awesome, looks like a very disciplined army preparing for battle. Wink

I dont know what to say. This post is fantastic. Thats not really a really huge statement, but its all I could come up with after reading this. You know so much about this subject. So much so that you made me want to learn more about it. Your blog is my stepping stone, my friend. Thanks for the heads up on this subject.

I loved what youve done here. The design is elegant, your stuff classy. Yet, youve got an edginess to what youre offering here. Ill definitely come back for more if you keep this up. Dont lose hope if not too many people see your vision, know youve got a fan right here who values what youve got to say and the way youve presented yourself. good on you!

Its like you read my mind! You seem to know so much about this, like you wrote the book in it or something. I think that you could do with some pics to drive the message home a bit, but other than that, this is great blog. A great read. Ill definitely be back.

Incredibly beneficial article. I’ve discovered your web site by Google and I am genuinely happy about the information you present with your posts. Anyway keep in the great hard work!

In search of attempting on the internet for one high-quality read pertaining to the following topic . Scouting in Yahoo I lastly discovered this blog post. Reading this info I’m just glad to mention that I’ve got a really good feeling I came across the very things I was ready for. I’ll ensure to remember this blog and go here consistently.

JTPRATT – great idea. Finding a store that relates to your blog is something that I do every now and again with my digicam blogs (photography stores) and even for ProBlogger (news stands).

Throughout this grand scheme of things you secure  a B+ just for effort and hard work. Exactly where you lost me was on your facts. As they say, the devil is in the details... And it couldn't be more correct here. Having said that, let me say to you just what did give good results. Your text is certainly quite engaging and that is possibly why I am taking an effort to opine. I do not make it a regular habit of doing that. 2nd, even though I can certainly see a leaps in reason you make, I am not necessarily convinced of just how you seem to unite your points which inturn help to make your final result. For the moment I will, no doubt yield to your position however wish in the foreseeable future you actually link the dots much better.

No cash down mortgages have become widely accepted and used as of late and rightfully so.

Interesting post.  Would you maybe be interested in some guest article writing?  I have a website that focuses on <a href="http://www.customcanopy.info">custom tent</a>.

I'm imprerssed at how you have a good hold of the matter. Do you mind if I quote a few of your articles as long as I provide credit and sources back to your site? My Facebook blog is in the exact same niche as yours and my visitors would certainly benefit from some of the information you present on your site. Please let me know if this a problem with you. Thanks!

To many times they say the same thing. Thanx for making this blog a bit diffrent.

Hi. Many thanks for your post. This post, C l e m e n s | Test Automation Investment Levels for VS2010, MTM and CodedUI is amazing and has helped me so much.

I'm not in a position to view this web site properly on firefox I think there is a drawback

HA! I WAS RIGHT, one of my classmates was saying that this wasnt possible, I did some research online to prove it to him and found your post, now I have proof, thanks!

Sie dürfen davon überzeugt sein, dass mir diese Seite sehr gefallen hat. Ich habe dieser Blog bereits in meine Bookmarkings abgespeichert und werde sie auch meinen Kollegen zeigen. Sollten Sie meine WebsiteWebangebote über Verdopple Deine Dates Ebook visitieren, werde Ich euch sehr dankbar sein. Sehen Sie mehr über Verdopple Deine Dates von David Deangelo, dem Pick up artist und Aufreiß Fachmann schlechthin.

Write more, thats all I have to say. Literally, it seems as though you relied on the video to make your point. You obviously know what youre talking about, why throw away your intelligence on just posting videos to your weblog when you could be giving us something informative to read?

I agree or in it comments above good post I must add

Awesome...thanks for posting this. Didn't completely understand all of it (my english isn't perfect, yet), but it was enjoyable to read anyway.

I built a website in a seperate but related niche. Thanks for giving me some ideas on design and layout. Cheers.

Thanks for this. I have a website in a similar niche and this is the third time that I have been back to check out your site. I gave you a little holler to my friends on FB, hope that's cool.

Add comment