@ClemensReijnen

Recent posts

Tags

Categories

Navigation

Pages

Archive

Blogroll

    Disclaimer

    The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.

    09. Getting Testing Done in the Sprint – Test Tasks on the Board

    It seems obvious that you need to have activities in the sprint for your testing activities. At least when you have something in your Definition Of Done what covers testing, like:

    • Test Complete
    • Integration tested done
    • Functional testing done
    • Regression testing done
    • Performance testing done
    • Acceptance testing done

    See: http://www.scrumalliance.org/articles/106-definition-of-done-a-reference. When there is totally no risk or business value in the system you are creating it can be that you don’t have to test it. (read: Risk and Business driven Tests), for all other systems… you need test tasks on the scrum board. I’ve seen multiple teams working with a scrum board which didn’t had any test activities, testing was a completely separate activity. So, this topic isn’t that strange.

    image

    These test tasks cover all the test activities the team need to execute to create a BPI conform the definition of done. this will be activities like:

    • Create test cases based on this and this test design technique for this PBI.
    • Setup the test infrastructure for load testing
    • Create test data for regression testing.
    • Automate this and this test cases.

    You name it, there are a lot of test tasks which need to be executed for a PBI to get Done. Often people get scared when they see all the test activities that need to be done, and want to minimize them. Bad job, this is a main reason why you can’t finish testing in a sprint.

    A good thing, people (Sogeti) already have been thinking about test activities and they have written a book about it: “TMap Next”.
    An even better thing, I create a TMap 4 TFS app with all the test activities ready to publish to you TFS Scrum board.

    For every phase in the test lifecycle

    a details page with test tasks

    with a detailed description of this task.

    5 2 3

    clip_image002

    Watch on YouTube

    Read the details

    When you don’t have a Windows Phone, you also can use the TMap for VS2010 process template, this one also contains a predefined set of testing tasks. Having predefined tasks isn’t really good, but it is still better as no testing tasks.

    Past Tips:
    01. Getting Testing Done in the Sprint - The Team and Activities
    02. Getting Testing Done in the Sprint – Regression Test Sets
    03. Getting Testing Done in the Sprint – Test Automation
    04. Getting Testing Done in the Sprint – Undone Backlog Item
    05. Getting Testing Done in the Sprint – No Double, Triple Testing
    06. Getting Testing Done in the Sprint – PBI Implementation Sequence
    07. Getting Testing Done in the Sprint – Risk and Business driven Tests
    08. Getting Testing Done in the Sprint – Write Logical Acceptance Tests

    Next Tips:
    10. Getting Testing Done in the Sprint – Done
    11. Getting Testing Done in the Sprint - The Customer Test Team
    12. Getting Testing Done in the Sprint – The Test Branch

    TMap 4 TFS WP7 App – Connect 2 TFS video explanation

    For all the people who don’t own a windows phone and want to see how the TMap for TFS App works… see this video.

    Posted: May 17 2012, 07:41 by ClemensReijnen | Comments (1) RSS comment feed |
    Filed under: ALM | MTM | TMap | VS2010 | VS11 | Testing

    TMap for TFS Windows Phone 7 App available in the marketplace.

    TMap (Test Management Approach) is a method for structured testing of software. An essential part of the method is the Life Cycle.

    SNAGHTML8b95af8

     

    image      image

    The TMap Life Cycle app describes the different phases of the TMap Life Cycle and has the capability to create TMap Test Tasks in your Team Foundation Server project.

    The TMap app also offers various downloads for checklists and templates supporting the TMap process. As well as the possibility to download books and papers.

    promo

     


    TMap for TFS menu’s

    The TMap for TFS App has four main menu’s; TMap, TFS, Sogeti and Books.

    1

    TMap is the entry point to the TMap Test Lifecycle. In this menu you can find the different Test Phases, from creation of the plan to the completion and preserving of the testing effort. Every phase has a page with exists out of five sections. Aim, discuses the why if the phase. The activities section shows the TMap test activities and has the capability to upload these to your TFS project, so they become part of your project task list. When you tap on an test activity item it show a more detailed description of this activity. Operation, describes how to run this phase and the products list gives you some guidelines which product should be realized during this phase. The toolbox section has a collection of tools and practices which you can use to execute this phase.

    Under the TFS menu you can make the settings and connection to your TFS project for uploading TMap test tasks. See section below for details.

    The menu Sogeti and Books provide some additional information.

    21 41 61 books

     

     


    Plug in

    TMap Test Tasks in Team Foundation Server Projects

    41

     

    The connection with TFS

     

    image51 Collection.
     
    The connection with TFS uses the OData service for TFS.
    When you want to use this for your own TFS server you need to install and configure the OData service for it, see the docs in the download
    You also can use it for your codeplex project. For these login settings see: CodePlex OData API.
    For TFS 11 Preview OData isn’t available.

    The TMap for TFS app saves your settings and will move to the next screen when connected.


    image

    projects


    Team Projects.

    Select or search for a Team Project where you are working on.
     

    image

    55


    Work Item.

    Make the default work item settings.
    Type the name for the work item type you want to create. For Codeplex projects this is ‘Work Item’ for Scrum projects this will be ‘Task’.
    Give the initial state of this work item type. For Codeplex this will be ‘Proposed’, for Scrum it will be ‘New’.
    Finaly set the area and iteration path of the work items. 








    Finally the created work items will appear in TFS and will be accessible by all Visual Studio client tools, Visual Studio, Microsoft Test Manager, Excel, SharePoint and by Web Access.
    5-6-2012 9-01-18 AM

    08. Getting Testing Done in the Sprint – Write Logical Acceptance Tests

    During release planning meeting. Capture acceptance criteria and immediately add the as logical test cases to the PBI. It will help the team to understand, clarify the discussion and more important for this topic, it helps testers be involved, and be important at the early stages of the software cycle.

    Within VS11 | TFS11 this is very easy to accomplish:

    1. add the PBI to the backlog.image
    2. add logical test cases, from the backlog item work item. and only add the test case title.
      image
      image
    3. Start planning and execute the sprint
    4. open Microsoft Test Manager, add a test plan for the current sprint.
      image
    5. Add the Backlog items as Requirement Suites to the plan and see the test cases listed in the suite, ready to add test steps.
      image

     

    This is a very small tip, but is very useful to get testing involved in a sprint (see tip 01).

    Past Tips:
    01. Getting Testing Done in the Sprint - The Team and Activities
    02. Getting Testing Done in the Sprint – Regression Test Sets
    03. Getting Testing Done in the Sprint – Test Automation
    04. Getting Testing Done in the Sprint – Undone Backlog Item
    05. Getting Testing Done in the Sprint – No Double, Triple Testing
    06. Getting Testing Done in the Sprint – PBI Implementation Sequence
    07. Getting Testing Done in the Sprint – Risk and Business driven Tests

    Next Tips:
    09. Getting Testing Done in the Sprint – Test Tasks on the Board
    10. Getting Testing Done in the Sprint – Done
    11. Getting Testing Done in the Sprint - The Customer Test Team
    12. Getting Testing Done in the Sprint – The Test Branch

    07. Getting Testing Done in the Sprint – Risk and Business driven Tests

    I really like the mindset “no risk, no test”. So, when there isn’t any business risk, there aren't any tests and is it easy to fit testing in a sprint. More realistic do a good risk analysis on your product backlog items before start writing thousands of tests. Also in scrum is risk an important attribute.

    The release plan establishes the goal of the release, the highest priority Product Backlog, the major risks, and the overall features and functionality that the release will contain.

    Products are built iteratively using Scrum, wherein each Sprint creates an increment of the product, starting with the most valuable and riskiest.

    Product Backlog items have the attributes of a description, priority, and estimate. Priority is driven by risk, value, and necessity. There are many techniques for assessing these attributes.

    from the scrum guide

    Within the TMap test approach product risk analysis is an important technique. Determine risk analyses are part of the proposed activities in the Master Test Plan of TMap: ‘Analysing the product risks’. It not only supports the Product Owner to make the right decisions it also the Team benefits in a later stage, this information is invaluable while defining the right test case design techniques for the Product Backlog Item.

    “The focus in product risk analysis is on the product risks, i.e. what is the risk to the organization if the product does not have the expected quality? ”
    www.TMap.net

    Having full product risk analysis for every Product Backlog Item during the Release Planning meeting is slightly overdone, but the major risks should be found. Determine product risks at this stage will also provide input for the Definition of Done list.

    3

    TMap Product Risk Analyses

    Within the Visual Studio Scrum 1.0 Process Template Product Backlog Items are written down in the Work Item Type ‘Product Backlog Item’. This Work Item Type hasn’t got a specific field for risk classifications. Adding a risk field is preferable (TFS Powertools makes this an easy task) so you can query on this property, or you can make the risk analyses a more product generic property.

    image

    TFS Scrum Product Backlog Item

    The flow with risk analyzing, classification, discussion and test design with the Product Owner can look like the diagram below.

    image

    But again most important for fitting testing in a sprint, know the risks use test design techniques to cover the risk and only write useful test cases.

    Post partly taken from previous posts:

     

    Past Tips:
    01. Getting Testing Done in the Sprint - The Team and Activities
    02. Getting Testing Done in the Sprint – Regression Test Sets
    03. Getting Testing Done in the Sprint – Test Automation
    04. Getting Testing Done in the Sprint – Undone Backlog Item
    05. Getting Testing Done in the Sprint – No Double, Triple Testing
    06. Getting Testing Done in the Sprint – PBI Implementation Sequence

    Next Tips:
    08. Getting Testing Done in the Sprint – Write Logical Acceptance Tests
    09. Getting Testing Done in the Sprint – Test Tasks on the Board
    10. Getting Testing Done in the Sprint – Done
    11. Getting Testing Done in the Sprint - The Customer Test Team
    12. Getting Testing Done in the Sprint – The Test Branch

    Create a TFS11 on Azure account with Clemens and get a 3 minute quick start explanation … #techdaysnl

    image

    TechDays in Den Haag, Netherlands

    TFS11 Service is the brand new Team Foundation Service on Azure, it makes it very simple to have your own Team Foundation Server. But, there are some new concepts in TFS11.

    In a 3 minute face to face session with me, we create a TFS11 on Azure account and you get an explanation of these new concepts so you can start using it and be productive immediately.

    WP_000214

    Look for this laptop at the Sogeti boot or ATE area at the TechDays in Den Haag, Netherlands.

    Thursday only.

    Decks: TFService Basic and Advanced

    Decks I used Yesterday for a TFService presentation. Normally it is a demo only presentation, but a scheduled maintenance made me use them Sad smile (see http://blogs.msdn.com/b/tfservice/ for the schedule).  Lucky me I have them, now you have hem too…

    First presentation (second in the series, first session was about scrum 101) is some general usages of TFService and how it supports an agile way of working.

     

    Second deck (fourth in the sessions series, third one was about how we use it in real projects) is about how we solved some usages challenges like multiple backlogs, bug backlog, testing etc.

     

    Some more reading:

    1. http://www.clemensreijnen.nl/post/2011/10/09/Teams-in-VS11.aspx
    2. http://www.clemensreijnen.nl/post/2011/10/10/Teams-in-Visual-Studio-11-feature-teams-and-backlogs.aspx
    3. http://www.clemensreijnen.nl/post/2011/10/12/Teams-in-Visual-Studio-11-CTP-the-Bug-Backlog.aspx
    4. Teams and TFS Groups in TFS11, the backlog, board and security settings.

    06. Getting Testing Done in the Sprint – PBI Implementation Sequence

    A challenge in getting testing done in a sprint is the fact that the software (PBI) isn’t ready to be tested till implementation/ coding is done. 

    How To.
    Work on completing each item in the sprint backlog and finish one item after another, see task boards below.

    BAD -- Not able to start testing task board, every team member works on a different pbi. Testing can only start at the end of the sprint.

    image

    GOOD -- Ready to start testing task board, team members work together on a pbi one after another till it’s done. Testing can start early.

    image

    It is a good practice, but not often followed. it really helps getting testing done.

     

    Past Tips:
    01. Getting Testing Done in the Sprint - The Team and Activities
    02. Getting Testing Done in the Sprint – Regression Test Sets
    03. Getting Testing Done in the Sprint – Test Automation
    04. Getting Testing Done in the Sprint – Undone Backlog Item
    05. Getting Testing Done in the Sprint – No Double, Triple Testing

    Next Tips: 
    07. Getting Testing Done in the Sprint – Risk and Business driven Tests
    08. Getting Testing Done in the Sprint – Write Logical Acceptance Tests
    09. Getting Testing Done in the Sprint – Test Tasks on the Board
    10. Getting Testing Done in the Sprint – Done
    11. Getting Testing Done in the Sprint - The Customer Test Team
    12. Getting Testing Done in the Sprint – The Test Branch

    05. Getting Testing Done in the Sprint – No Double, Triple Testing

    One way of getting testing done in a sprint is making it [testing] as efficient as possible, meanwhile this also saves money. One way of getting more efficient is not doing things double or triple. An interesting thing with testing, we do test double or triple. We test it during unit, during unit integration, during functional and during acceptance testing, this can be done better, we could save a lot of time. I’ve seen teams which also automated all these types of tests, testing the same thing... not really efficient, they didn’t manage to get it done in a sprint.

    Each phase of the development process has its own test goals and depth of test coverage. The depth particularly depends upon the risks: higher probability of damage requires better test coverage. These test goals are especially aimed at discovering errors that can be found in that phase (“as early as possible”).

    • during unit testing the internal logic of the unit is tested,
    • the unit integration test needs to show that units ”understand” each other.
    • the system test needs to prove/demonstrate that the system complies with the functional requirements that have been agreed to,
    • finally, the acceptance test considers how the system fits into the environment in which it will operate.

    Test goals and (depth of) test coverage per phase are made compatible with each other using a Master Test Plan (see TMap® www.tmap.net).

    clip_image002

    Every test must be able to rely on previous tests, only then will the complexity of errors be contained to errors that were introduced during the last step. Setting up a learning cycle makes it possible to correct shortcomings, if any, in the testing process chain.

    Each phase of the development process and its corresponding test goal introduces a specific role with specific abilities and tasks. These roles are supported by specific ways of testing, often visualized in the Agile Test Quadrants.

    image

    http://lisacrispin.com/wordpress/2011/11/08/using-the-agile-testing-quadrants/

    How the quadrants work

    Tests on the left-hand quadrants help the team know what code to write, and know when they are done writing it. Tests on the right hand side help the team learn more about the code they’re written, and this learning often translates into new user stories and tests that feed back to the left-hand quadrants. Business stakeholders define quality criteria for the top two quadrants, while the bottom two quadrants relate more to internal quality and criteria.
    from:
    http://readytohelpfortesting.blogspot.com/2012/01/ttesting-quadrants.html

    How To.

    All these types should work together, and as already mentioned a Master Test Plan (see TMap® www.tmap.net) can help. What also can help; is getting the team together and discus it [what to test where]. This is even more important when the first tip isn’t in place (01. Getting Testing Done in the Sprint - The Team and Activities) or when there is a Customer Test Team (will be explained in tip: 11. Getting Testing Done in the Sprint - The Customer Test Team).

    I like the example: where it is very hard to create unit tests for a specific piece off functionality, like login screens. To create unit test you have to use a mocking framework that mimics the user database or AD. This takes some time implement and maintain, while the functional testers will use the login screen (and test it) every time they run a test. A nice piece of functionality to have a good discussion with the tester where to test and automate this.
    An other example is a really world scenario I’ve seen; this is where the developers created some great unit tests for input validation, the functional system testers created test scripts to tests the input validation and the acceptance tester tested the input validations... really a scenario where there wasn’t any communication between the testers and people didn’t trust each other (see 01. Getting Testing Done in the Sprint - The Team and Activities)

    More reading: the Test Automation Pyramid http://blog.goneopen.com/2010/08/test-automation-pyramid-review/

    Tool Support:

    All test types are supported by VS2010.

    • The development tester:
      This tester knows the inner workings of the system and makes use of test-frameworks to automate the tests that he wants to perform. This testing role, has been supported for a while now by technical aids as unit test frameworks in Visual Studio, and is primarily done by developers. VS2010 eases the life of the development tester by intelligent support of Test Driven Development.
    • The system tester:
      The system tester tests the system for completeness, whether requirements have been met. These are not only functional requirements, but also quality requirements like performance and security (see agile test quadrant).
      System testers use a lot of scripts to automate their activities, he/she would like to perform the same tests using various infrastructures and configurations. VS2010 has had Load- and Web testing for quite a while in the Test Edition and with the introduction of VS2010 the creation of test scripts is simplified by introduction of the ‘Coded UI’ framework.
    • The acceptance tester:
      The acceptance tester, also called user tester or black box tester, truly is a tester who performs a test of the application and knows nothing about the internal workings of the application. He/she constructs test cases using functional requirements, using various testing techniques.
      Acceptance testers, who often still use Excel sheets to perform their tasks, are abundantly well supported in Visual Studio 2010. Test cases are registered in TFS and he/she has access to Microsoft Test Manager on optimal support of, usually, manual testing activities.

    There isn’t any support to see if pieces of functionality are tested twice. But with unit test coverage and data diagnostic adapters for functional testing there are some capabilities in place to extract the information from the system. Maybe time to create a custom data diagnostic adapter, visualize it and compare it with the unit test coverage.

    Past Tips:
    01. Getting Testing Done in the Sprint - The Team and Activities
    02. Getting Testing Done in the Sprint – Regression Test Sets
    03. Getting Testing Done in the Sprint – Test Automation
    04. Getting Testing Done in the Sprint – Undone Backlog Item

    Next Tips:
    06. Getting Testing Done in the Sprint – PBI Implementation Sequence
    07. Getting Testing Done in the Sprint – Risk and Business driven Tests
    08. Getting Testing Done in the Sprint – Write Logical Acceptance Tests
    09. Getting Testing Done in the Sprint – Test Tasks on the Board
    10. Getting Testing Done in the Sprint – Done
    11. Getting Testing Done in the Sprint - The Customer Test Team
    12. Getting Testing Done in the Sprint – The Test Branch

    04. Getting Testing Done in the Sprint – Undone Backlog Item

    A tip straight from the previous Scrum Guide (The guide updated and the undone TIP is removed, read The 2011 Scrum Guide – A Lot of Changes! ). But, the TIP still works, although it isn’t an ‘official’ tip anymore…

    Undone End to End Testing

    End to end testing, is not an agile specific testing practice, it’s a common sense testing practice. Create test cases to detected defects anywhere in the chain. This type of testing has been growing the couple of years enormous due to the rise of Service Oriented Architectures and more recent Cloud Computing.  as the TMap end-To-End Testing books describesketentesten%20EN_tcm9-57123

    “End-to-end testing is a form of testing in which the (business) process is tracked through an interconnected series of systems, which may be based in different organizations, aiming to answer the following questions:
    - Does the IT process support the business process adequately? - Are the process and systems integrated with each other in the right way?”
    TMap.net End-To-End Testing

    End-to-End testing is seen as a challenges in Agile environments, with the most important challenge that systems under development aren’t ready for End-to-End testing. Not every service is finished to test the complete end-to-end business process. Stubs are a solution, but can these be made in the current sprint and are stubs good enough to say something about the quality of the system. And, when the system are ready and a change in a service is needed, how and can we re-run all the end-to-end tests, is there time enough.

    All these challenges with end-to-end testing in agile projects can be summarized in two topics:

    • Not Ready Systems in a Sprint

    • Not Enough Time in a Sprint

    tipScrum solves this problem, this problem of not able to be compliant on the definition of done at sprint finish, by defining ‘Undone work’. the scrum guide mentions this in a TIP and in the final thoughts of the scrum guide

    Some organizations are incapable of building a complete increment within one Sprint. They may not yet have the automated testing infrastructure to complete all of the testing. In this case, two categories are created for each increment: the “done” work and the “undone” work. The “undone” work is the portion of each increment that will have to be completed at a later time. The Product Owner knows exactly what he or she is inspecting at the end of the Sprint because the increment meets the definition of “done” and the Product Owner understands the definition. “Undone” work is added to a Product Backlog item named “undone work” so it accumulates and correctly reflects on the Release Burndown graph. This technique creates transparency in progress toward a release. The inspect and adapt in the Sprint Review is as accurate as this transparency.

    Within the Visual Studio 2010 Scrum process template this means that you have to link or create tasks to a specific ‘Undone Work’ product backlog item. this ‘Undone Work’ product backlog item accumulates the undone work regarding to End-to-End Testing (and others) over time and can be monitored just as other backlog items.

    http://www.slideshare.net/xwarzee/path-to-agility-ken-schwaber 

    image

     

     

    Tool Support

    In TFS with the Scrum process template we can create Backlog Items very easy.

    image

    So, creating an undone backlog item isn’t that hard, the challenge is take it with you every sprint. for example in my undone backlog item I’ve got some undone tasks from sprint 1 and some from sprint 4. The iteration paths for these tasks still point to the sprint 1 and 4 sprints.

    image

    This will result in a wrong sprint backlog, because the tasks who point to other sprints as the current sprint won’t appear in the sprint backlog. So, when moving this Undone backlog item to another sprint because not all tasks are done, you also have to change the iteration path for the corresponding tasks.

    image 

    Just keep in mind TFS doesn't update the hierarchy, you have to do it manually (or use Excel, my favorite).

    Past Tips:
    01. Getting Testing Done in the Sprint - The Team and Activities
    02. Getting Testing Done in the Sprint – Regression Test Sets
    03. Getting Testing Done in the Sprint – Test Automation

    Next Tips:
    05. Getting Testing Done in the Sprint – No Double, Triple Testing
    06. Getting Testing Done in the Sprint – PBI Implementation Sequence
    07. Getting Testing Done in the Sprint – Risk and Business driven Tests
    08. Getting Testing Done in the Sprint – Write Logical Acceptance Tests
    09. Getting Testing Done in the Sprint – Test Tasks on the Board
    10. Getting Testing Done in the Sprint – Done
    11. Getting Testing Done in the Sprint - The Customer Test Team
    12. Getting Testing Done in the Sprint – The Test Branch