@ClemensReijnen

Recent posts

Tags

Categories

Navigation

Pages

Archive

Blogroll

    Disclaimer

    The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.

    11. Getting Testing Done in the Sprint - The Customer Test Team

    I don’t mean the acceptance test team, although many think and organize it as if it is the same.

    On a test conference a few weeks ago I heard the story of moving testing out of the sprint, or adding a specific test sprint after a development sprint. Both ‘worksrrounds’ I dislike, you lose the flexibility and feedback loop. It was something like the ‘IndependentParallelTesting’ story, also not my favorite solution.

    During the conference I posted a tweet, and got replies from Sander and Marcel. Lucky me I’m not alone Emoticon met brede lach (I like the last comment from Sander, the process just should support and embrace it, see: http://www.smartusecase.com.)

    image

    image

    The most clear characteristic of the customer test team is that they aren’t part of the scrum team. You could call them early adopters, power users or the customer test team. Often they execute real life scenario’s, play around in the application or when they are early adopters they do some real work with it.

    Beside the question, “do we really need them” is the challenge with these disconnected test teams is, how to get them and their information in a sprint. Threat them as operational findings is a way. But doesn’t solve the challenge of getting their usages data, their findings and reports back on our backlog and in a sprint.

    A nice explanation can be found in this video from ‘Groeten uit Delft’ where Rob van Lanen (www.agilestudio.nl) explains how they work together with operations.

     

    Beside the solution of giving the ‘disconnected’ customer test team a copy of Microsoft Test Manager 2012 and let them execute their test cases in a ‘structured’ exploratory way. http://msdn.microsoft.com/en-us/library/hh191621(v=vs.110).aspx
    Microsoft Test Manager - start exploratory test

    A nice work around to get the findings and knowledge from the customer test team in your teams project, is create a separate backlog for them (read this old post – will update it soon).

    Two other interesting technologies which can help teams with this scenario (findings found outside the team) are:

    1. PreEmptive Analytics for TFS.
      PreEmptive Analytics for TFS CE offers a free, lightweight way to integrate feedback-driven development processes into your development workflow. Your applications will automatically send back exception report data to your PreEmptive Analytics endpoint service as errors occur during their execution


      http://blogs.msdn.com/b/bharry/archive/2012/04/11/preemptive-analytics-in-visual-studio-and-tfs-11.aspx
    2. Collecting IntelliTrace Data Everywhere
      The collector saves IntelliTrace data as a recording that contains details about exceptions, threads, Web requests, modules, and other system information that is collected while your application runs. In most sections of the recording, you can select one of these items and start debugging with IntelliTrace in Visual Studio Ultimate.

     

    Past Tips:
    01. Getting Testing Done in the Sprint - The Team and Activities
    02. Getting Testing Done in the Sprint – Regression Test Sets
    03. Getting Testing Done in the Sprint – Test Automation
    04. Getting Testing Done in the Sprint – Undone Backlog Item
    05. Getting Testing Done in the Sprint – No Double, Triple Testing
    06. Getting Testing Done in the Sprint – PBI Implementation Sequence
    07. Getting Testing Done in the Sprint – Risk and Business driven Tests
    08. Getting Testing Done in the Sprint – Write Logical Acceptance Tests
    09. Getting Testing Done in the Sprint – Test Tasks on the Board
    10. Getting Testing Done in the Sprint – the Definition of Done

    Next Tips:
    12. Getting Testing Done in the Sprint – The Test Branch

     

    Posted: Jul 10 2012, 08:02 by ClemensReijnen | Comments (3) RSS comment feed |
    Filed under: ALM | Agile | VS2012

    Sogeti Windows 8 Metro Application Demo

    cross post from: Darren W Baker

    This video is a compilation of Windows 8 Metro Applications that Sogeti consultants are working on world wide. Sogeti was asked to put together a small video to be played at WPC 2012 during one of the Keynotes, and we are all very excited to see our hard work on the big screen.<<read more>>

    Posted: Jul 07 2012, 02:20 by ClemensReijnen | Comments (1) RSS comment feed |
    Filed under: Sogeti | win8

    Test Automation Day Deck - Agile ALM - Getting testing done in a sprint

    Below my http://www.testautomationday.com/ presentation (demo’s will follow next week on my YouTube channel).

    Agile teams find it hard to get the testing effort in sync with the other development activities. Not only development tests are executed during sprints, all other kind of testing activities are part of done. This session will give guidance how Microsoft Visual Studio ALM tools can support agile teams. How to run sprints and get testing done in a sprint.

     

     

    10. Getting Testing Done in the Sprint – the Definition of Done

    Are we done? When this is unclear the team will do too less or too much. Both are bad. The Definition of Done helps to define when you are done, the same as a master test plan helps to define the test levels. A good definition of done specific for your team and project will definitely help knowing when you're ready and to get testing done in a sprint. Mainly because you know when you're done.

    Making a good definition of done is a trivial task and books are written about it.
    A good starting point are these posts:

    One thing I really like to keep in mind when defining a Done List are:

    • Risk Analysis, and together with this the Master Test plan. I should be clear what the risks are and what the impact is on when you are done.
      You can think of Done ListItems, which help understand the level of testing to be done, like:
      • All High Risk BPI’s have a test coverage of ..% and no open bugs
      • All high prio test cases are automated
      • Low Risk, Low Business PBI’s may have 10% open low impact bugs.  
    • Test Levels, balance the unit, functional and acceptance testing and define where what is tested. Definition of Done items like; all unit tests are green, all functional testing is done or all written acceptance tests are executed, are unclear. How many tests should we write to be done? Code Coverage is one for unit testing, but how do you know when your done specifying functional tests and acceptance tests. So, writing down in the Done list that testing is done isn’t enough it also should have a statement about when test writing is Done.
    • When are we done creating the Regression Set and when are we done with Test Automation

    So, when testing is important and you don’t want to create a random amount of test cases for each PBI, should mention it in the Done List, and for sure make the validation automated with a build system.

    image

    Past Tips:
    01. Getting Testing Done in the Sprint - The Team and Activities
    02. Getting Testing Done in the Sprint – Regression Test Sets
    03. Getting Testing Done in the Sprint – Test Automation
    04. Getting Testing Done in the Sprint – Undone Backlog Item
    05. Getting Testing Done in the Sprint – No Double, Triple Testing
    06. Getting Testing Done in the Sprint – PBI Implementation Sequence
    07. Getting Testing Done in the Sprint – Risk and Business driven Tests
    08. Getting Testing Done in the Sprint – Write Logical Acceptance Tests
    09. Getting Testing Done in the Sprint – Test Tasks on the Board

    Next Tips:
    11. Getting Testing Done in the Sprint - The Customer Test Team
    12. Getting Testing Done in the Sprint – The Test Branch

    09. Getting Testing Done in the Sprint – Test Tasks on the Board

    It seems obvious that you need to have activities in the sprint for your testing activities. At least when you have something in your Definition Of Done what covers testing, like:

    • Test Complete
    • Integration tested done
    • Functional testing done
    • Regression testing done
    • Performance testing done
    • Acceptance testing done

    See: http://www.scrumalliance.org/articles/106-definition-of-done-a-reference. When there is totally no risk or business value in the system you are creating it can be that you don’t have to test it. (read: Risk and Business driven Tests), for all other systems… you need test tasks on the scrum board. I’ve seen multiple teams working with a scrum board which didn’t had any test activities, testing was a completely separate activity. So, this topic isn’t that strange.

    image

    These test tasks cover all the test activities the team need to execute to create a BPI conform the definition of done. this will be activities like:

    • Create test cases based on this and this test design technique for this PBI.
    • Setup the test infrastructure for load testing
    • Create test data for regression testing.
    • Automate this and this test cases.

    You name it, there are a lot of test tasks which need to be executed for a PBI to get Done. Often people get scared when they see all the test activities that need to be done, and want to minimize them. Bad job, this is a main reason why you can’t finish testing in a sprint.

    A good thing, people (Sogeti) already have been thinking about test activities and they have written a book about it: “TMap Next”.
    An even better thing, I create a TMap 4 TFS app with all the test activities ready to publish to you TFS Scrum board.

    For every phase in the test lifecycle

    a details page with test tasks

    with a detailed description of this task.

    5 2 3

    clip_image002

    Watch on YouTube

    Read the details

    When you don’t have a Windows Phone, you also can use the TMap for VS2010 process template, this one also contains a predefined set of testing tasks. Having predefined tasks isn’t really good, but it is still better as no testing tasks.

    Past Tips:
    01. Getting Testing Done in the Sprint - The Team and Activities
    02. Getting Testing Done in the Sprint – Regression Test Sets
    03. Getting Testing Done in the Sprint – Test Automation
    04. Getting Testing Done in the Sprint – Undone Backlog Item
    05. Getting Testing Done in the Sprint – No Double, Triple Testing
    06. Getting Testing Done in the Sprint – PBI Implementation Sequence
    07. Getting Testing Done in the Sprint – Risk and Business driven Tests
    08. Getting Testing Done in the Sprint – Write Logical Acceptance Tests

    Next Tips:
    10. Getting Testing Done in the Sprint – Done
    11. Getting Testing Done in the Sprint - The Customer Test Team
    12. Getting Testing Done in the Sprint – The Test Branch

    TMap 4 TFS WP7 App – Connect 2 TFS video explanation

    For all the people who don’t own a windows phone and want to see how the TMap for TFS App works… see this video.

    Posted: May 17 2012, 07:41 by ClemensReijnen | Comments (1) RSS comment feed |
    Filed under: ALM | MTM | TMap | VS2010 | VS11 | Testing

    TMap for TFS Windows Phone 7 App available in the marketplace.

    TMap (Test Management Approach) is a method for structured testing of software. An essential part of the method is the Life Cycle.

    SNAGHTML8b95af8

     

    image      image

    The TMap Life Cycle app describes the different phases of the TMap Life Cycle and has the capability to create TMap Test Tasks in your Team Foundation Server project.

    The TMap app also offers various downloads for checklists and templates supporting the TMap process. As well as the possibility to download books and papers.

    promo

     


    TMap for TFS menu’s

    The TMap for TFS App has four main menu’s; TMap, TFS, Sogeti and Books.

    1

    TMap is the entry point to the TMap Test Lifecycle. In this menu you can find the different Test Phases, from creation of the plan to the completion and preserving of the testing effort. Every phase has a page with exists out of five sections. Aim, discuses the why if the phase. The activities section shows the TMap test activities and has the capability to upload these to your TFS project, so they become part of your project task list. When you tap on an test activity item it show a more detailed description of this activity. Operation, describes how to run this phase and the products list gives you some guidelines which product should be realized during this phase. The toolbox section has a collection of tools and practices which you can use to execute this phase.

    Under the TFS menu you can make the settings and connection to your TFS project for uploading TMap test tasks. See section below for details.

    The menu Sogeti and Books provide some additional information.

    21 41 61 books

     

     


    Plug in

    TMap Test Tasks in Team Foundation Server Projects

    41

     

    The connection with TFS

     

    image51 Collection.
     
    The connection with TFS uses the OData service for TFS.
    When you want to use this for your own TFS server you need to install and configure the OData service for it, see the docs in the download
    You also can use it for your codeplex project. For these login settings see: CodePlex OData API.
    For TFS 11 Preview OData isn’t available.

    The TMap for TFS app saves your settings and will move to the next screen when connected.


    image

    projects


    Team Projects.

    Select or search for a Team Project where you are working on.
     

    image

    55


    Work Item.

    Make the default work item settings.
    Type the name for the work item type you want to create. For Codeplex projects this is ‘Work Item’ for Scrum projects this will be ‘Task’.
    Give the initial state of this work item type. For Codeplex this will be ‘Proposed’, for Scrum it will be ‘New’.
    Finaly set the area and iteration path of the work items. 








    Finally the created work items will appear in TFS and will be accessible by all Visual Studio client tools, Visual Studio, Microsoft Test Manager, Excel, SharePoint and by Web Access.
    5-6-2012 9-01-18 AM

    08. Getting Testing Done in the Sprint – Write Logical Acceptance Tests

    During release planning meeting. Capture acceptance criteria and immediately add the as logical test cases to the PBI. It will help the team to understand, clarify the discussion and more important for this topic, it helps testers be involved, and be important at the early stages of the software cycle.

    Within VS11 | TFS11 this is very easy to accomplish:

    1. add the PBI to the backlog.image
    2. add logical test cases, from the backlog item work item. and only add the test case title.
      image
      image
    3. Start planning and execute the sprint
    4. open Microsoft Test Manager, add a test plan for the current sprint.
      image
    5. Add the Backlog items as Requirement Suites to the plan and see the test cases listed in the suite, ready to add test steps.
      image

     

    This is a very small tip, but is very useful to get testing involved in a sprint (see tip 01).

    Past Tips:
    01. Getting Testing Done in the Sprint - The Team and Activities
    02. Getting Testing Done in the Sprint – Regression Test Sets
    03. Getting Testing Done in the Sprint – Test Automation
    04. Getting Testing Done in the Sprint – Undone Backlog Item
    05. Getting Testing Done in the Sprint – No Double, Triple Testing
    06. Getting Testing Done in the Sprint – PBI Implementation Sequence
    07. Getting Testing Done in the Sprint – Risk and Business driven Tests

    Next Tips:
    09. Getting Testing Done in the Sprint – Test Tasks on the Board
    10. Getting Testing Done in the Sprint – Done
    11. Getting Testing Done in the Sprint - The Customer Test Team
    12. Getting Testing Done in the Sprint – The Test Branch

    07. Getting Testing Done in the Sprint – Risk and Business driven Tests

    I really like the mindset “no risk, no test”. So, when there isn’t any business risk, there aren't any tests and is it easy to fit testing in a sprint. More realistic do a good risk analysis on your product backlog items before start writing thousands of tests. Also in scrum is risk an important attribute.

    The release plan establishes the goal of the release, the highest priority Product Backlog, the major risks, and the overall features and functionality that the release will contain.

    Products are built iteratively using Scrum, wherein each Sprint creates an increment of the product, starting with the most valuable and riskiest.

    Product Backlog items have the attributes of a description, priority, and estimate. Priority is driven by risk, value, and necessity. There are many techniques for assessing these attributes.

    from the scrum guide

    Within the TMap test approach product risk analysis is an important technique. Determine risk analyses are part of the proposed activities in the Master Test Plan of TMap: ‘Analysing the product risks’. It not only supports the Product Owner to make the right decisions it also the Team benefits in a later stage, this information is invaluable while defining the right test case design techniques for the Product Backlog Item.

    “The focus in product risk analysis is on the product risks, i.e. what is the risk to the organization if the product does not have the expected quality? ”
    www.TMap.net

    Having full product risk analysis for every Product Backlog Item during the Release Planning meeting is slightly overdone, but the major risks should be found. Determine product risks at this stage will also provide input for the Definition of Done list.

    3

    TMap Product Risk Analyses

    Within the Visual Studio Scrum 1.0 Process Template Product Backlog Items are written down in the Work Item Type ‘Product Backlog Item’. This Work Item Type hasn’t got a specific field for risk classifications. Adding a risk field is preferable (TFS Powertools makes this an easy task) so you can query on this property, or you can make the risk analyses a more product generic property.

    image

    TFS Scrum Product Backlog Item

    The flow with risk analyzing, classification, discussion and test design with the Product Owner can look like the diagram below.

    image

    But again most important for fitting testing in a sprint, know the risks use test design techniques to cover the risk and only write useful test cases.

    Post partly taken from previous posts:

     

    Past Tips:
    01. Getting Testing Done in the Sprint - The Team and Activities
    02. Getting Testing Done in the Sprint – Regression Test Sets
    03. Getting Testing Done in the Sprint – Test Automation
    04. Getting Testing Done in the Sprint – Undone Backlog Item
    05. Getting Testing Done in the Sprint – No Double, Triple Testing
    06. Getting Testing Done in the Sprint – PBI Implementation Sequence

    Next Tips:
    08. Getting Testing Done in the Sprint – Write Logical Acceptance Tests
    09. Getting Testing Done in the Sprint – Test Tasks on the Board
    10. Getting Testing Done in the Sprint – Done
    11. Getting Testing Done in the Sprint - The Customer Test Team
    12. Getting Testing Done in the Sprint – The Test Branch

    Create a TFS11 on Azure account with Clemens and get a 3 minute quick start explanation … #techdaysnl

    image

    TechDays in Den Haag, Netherlands

    TFS11 Service is the brand new Team Foundation Service on Azure, it makes it very simple to have your own Team Foundation Server. But, there are some new concepts in TFS11.

    In a 3 minute face to face session with me, we create a TFS11 on Azure account and you get an explanation of these new concepts so you can start using it and be productive immediately.

    WP_000214

    Look for this laptop at the Sogeti boot or ATE area at the TechDays in Den Haag, Netherlands.

    Thursday only.