Microsoft Test Manager Test Case Specification what kind of action to use

A test step within a test case in Test Manager (MTM) can be a normal action step or a validation action step, and both of them can contain additional information (attachments).
See image three steps:

  1. normal action step
  2. action step with validation
  3. normal action step with attachment (validation action step can also contain an attachment)

image

The question is: what kind of action steps do I need to use for my test case…

Most often, actually almost always, I see test cases with only validation action steps in MTM. Something like this:

image

You're not alone with only validation action steps, see the test case in these articles and posts, most of them write these kind of test cases

The reasons why people use the validation action that much has two reasons: first it’s nice and clear for the tester who executes the test… do an action and validate the action, show the tester who executes the test knows the state of the application. The other reason is, because it’s there... didn’t think of it just used it. I have to say the the columns “action” and “expected result” make you think that you have to fill both, ever action has a result… (sounds like mathematics :)

Why not

imageThere are a few bad things about having every step as an validation step, or having a lot of validation steps.

First, it’s very hard to understand for the tester who executes the test what the real test is (see image right side). Example above, is the test to validate the open browser action? or that Bing loads well? not really.. the true test is my blog should be in the top one position… you can write this down in the summary, or use a descriptive title, but still it’s confusing… 

The second ‘not so good thing’ about having every test step a validation step is more tool related. MTM is a tool which helps the manual tester by executing his test tasks. With test runner (see image right side) when executing a test case it can Fast Forward several steps till your application is in a state where you have to manual validate it. 
This Fast Forward functionality, this helping the manual tester by MTM, is gone when every  step is a manual validation step. Again the real validation is: my blog should be in the top one position. Having only that as a validation step, I can fast forward to the Click search button and validate the result.

image

Fast Forward works, but if every step is a validation step… you have to manually set the steps to pass… [or the whole test case at once].

image

With just one, well chosen, validation test step this works much more comfortable.

Another drawback of using that much validation steps you get when the test case is going to be automated with CodedUI. When generating CodedUI tests from both test cases (see above, one with parameters and a well chosen validation step and one test case with only validation steps) they are almost identical, the only difference is the parameters for the search text.

image

The differences are in the assertions, because for the first one with all the validation steps I , as an developer who doesn’t understand testing, will create for every validation step an corresponding assertion method to automate the visual validation of the manual tester. This will result in four assertion methods for the first test case against just one assertion method for the other test case.

So, instead off randomly making use of validation steps… think about it up front how MTM is helping the manual testers and the collaboration with the developer and don’t ruin this powerful functionality… when you do want to add some additional information about the state of the  application after a certain step, use the attachment capability.

This is how the ‘well designed with MTM in mind’ test case looks like… 
image

We could use a shared step for the browser and Bing thing, and use a parameter for the search engine, and configure the test plan test environments for different browsers… but first lets start with writing clean and understandable steps.

Add comment