10. Getting Testing Done in the Sprint – the Definition of Done
Are we done? When this is unclear the team will do too less or too much. Both are bad. The Definition of Done helps to define when you are done, the same as a master test plan helps to define the test levels. A good definition of done specific for your team and project will definitely help knowing when you're ready and to get testing done in a sprint. Mainly because you know when you're done.
Making a good definition of done is a trivial task and books are written about it.
A good starting point are these posts:
One thing I really like to keep in mind when defining a Done List are:
- Risk Analysis, and together with this the Master Test plan. I should be clear what the risks are and what the impact is on when you are done.
You can think of Done ListItems, which help understand the level of testing to be done, like: - All High Risk BPI’s have a test coverage of ..% and no open bugs
- All high prio test cases are automated
- Low Risk, Low Business PBI’s may have 10% open low impact bugs.
- Test Levels, balance the unit, functional and acceptance testing and define where what is tested. Definition of Done items like; all unit tests are green, all functional testing is done or all written acceptance tests are executed, are unclear. How many tests should we write to be done? Code Coverage is one for unit testing, but how do you know when your done specifying functional tests and acceptance tests. So, writing down in the Done list that testing is done isn’t enough it also should have a statement about when test writing is Done.
- When are we done creating the Regression Set and when are we done with Test Automation
So, when testing is important and you don’t want to create a random amount of test cases for each PBI, should mention it in the Done List, and for sure make the validation automated with a build system.
Past Tips:
01. Getting Testing Done in the Sprint - The Team and Activities
02. Getting Testing Done in the Sprint – Regression Test Sets
03. Getting Testing Done in the Sprint – Test Automation
04. Getting Testing Done in the Sprint – Undone Backlog Item
05. Getting Testing Done in the Sprint – No Double, Triple Testing
06. Getting Testing Done in the Sprint – PBI Implementation Sequence
07. Getting Testing Done in the Sprint – Risk and Business driven Tests
08. Getting Testing Done in the Sprint – Write Logical Acceptance Tests
09. Getting Testing Done in the Sprint – Test Tasks on the Board
Next Tips:
11. Getting Testing Done in the Sprint - The Customer Test Team
12. Getting Testing Done in the Sprint – The Test Branch
…
…
09. Getting Testing Done in the Sprint – Test Tasks on the Board
It seems obvious that you need to have activities in the sprint for your testing activities. At least when you have something in your Definition Of Done what covers testing, like:
- Test Complete
- Integration tested done
- Functional testing done
- Regression testing done
- Performance testing done
- Acceptance testing done
See: http://www.scrumalliance.org/articles/106-definition-of-done-a-reference. When there is totally no risk or business value in the system you are creating it can be that you don’t have to test it. (read: Risk and Business driven Tests), for all other systems… you need test tasks on the scrum board. I’ve seen multiple teams working with a scrum board which didn’t had any test activities, testing was a completely separate activity. So, this topic isn’t that strange.
These test tasks cover all the test activities the team need to execute to create a BPI conform the definition of done. this will be activities like:
- Create test cases based on this and this test design technique for this PBI.
- Setup the test infrastructure for load testing
- Create test data for regression testing.
- Automate this and this test cases.
You name it, there are a lot of test tasks which need to be executed for a PBI to get Done. Often people get scared when they see all the test activities that need to be done, and want to minimize them. Bad job, this is a main reason why you can’t finish testing in a sprint.
A good thing, people (Sogeti) already have been thinking about test activities and they have written a book about it: “TMap Next”.
An even better thing, I create a TMap 4 TFS app with all the test activities ready to publish to you TFS Scrum board.
| For every phase in the test lifecycle | a details page with test tasks | with a detailed description of this task. |
 |  |  |
Watch on YouTube
Read the details
When you don’t have a Windows Phone, you also can use the TMap for VS2010 process template, this one also contains a predefined set of testing tasks. Having predefined tasks isn’t really good, but it is still better as no testing tasks.
Past Tips:
01. Getting Testing Done in the Sprint - The Team and Activities
02. Getting Testing Done in the Sprint – Regression Test Sets
03. Getting Testing Done in the Sprint – Test Automation
04. Getting Testing Done in the Sprint – Undone Backlog Item
05. Getting Testing Done in the Sprint – No Double, Triple Testing
06. Getting Testing Done in the Sprint – PBI Implementation Sequence
07. Getting Testing Done in the Sprint – Risk and Business driven Tests
08. Getting Testing Done in the Sprint – Write Logical Acceptance Tests
Next Tips:
10. Getting Testing Done in the Sprint – Done
11. Getting Testing Done in the Sprint - The Customer Test Team
12. Getting Testing Done in the Sprint – The Test Branch
…
…
TMap 4 TFS WP7 App – Connect 2 TFS video explanation
For all the people who don’t own a windows phone and want to see how the TMap for TFS App works… see this video.
TMap for TFS Windows Phone 7 App available in the marketplace.
TMap (Test Management Approach) is a method for structured testing of software. An essential part of the method is the Life Cycle.
The TMap Life Cycle app describes the different phases of the TMap Life Cycle and has the capability to create TMap Test Tasks in your Team Foundation Server project.
The TMap app also offers various downloads for checklists and templates supporting the TMap process. As well as the possibility to download books and papers.
TMap for TFS menu’s
The TMap for TFS App has four main menu’s; TMap, TFS, Sogeti and Books.
TMap is the entry point to the TMap Test Lifecycle. In this menu you can find the different Test Phases, from creation of the plan to the completion and preserving of the testing effort. Every phase has a page with exists out of five sections. Aim, discuses the why if the phase. The activities section shows the TMap test activities and has the capability to upload these to your TFS project, so they become part of your project task list. When you tap on an test activity item it show a more detailed description of this activity. Operation, describes how to run this phase and the products list gives you some guidelines which product should be realized during this phase. The toolbox section has a collection of tools and practices which you can use to execute this phase.
Under the TFS menu you can make the settings and connection to your TFS project for uploading TMap test tasks. See section below for details.
The menu Sogeti and Books provide some additional information.
TMap Test Tasks in Team Foundation Server Projects
The connection with TFS
  | Collection.
The connection with TFS uses the OData service for TFS.
When you want to use this for your own TFS server you need to install and configure the OData service for it, see the docs in the download.
You also can use it for your codeplex project. For these login settings see: CodePlex OData API.
For TFS 11 Preview OData isn’t available.
The TMap for TFS app saves your settings and will move to the next screen when connected.
|
|
Team Projects.
Select or search for a Team Project where you are working on.
|
 |
Work Item.
Make the default work item settings.
Type the name for the work item type you want to create. For Codeplex projects this is ‘Work Item’ for Scrum projects this will be ‘Task’.
Give the initial state of this work item type. For Codeplex this will be ‘Proposed’, for Scrum it will be ‘New’.
Finaly set the area and iteration path of the work items.
|
|
Finally the created work items will appear in TFS and will be accessible by all Visual Studio client tools, Visual Studio, Microsoft Test Manager, Excel, SharePoint and by Web Access. |
Create a TFS11 on Azure account with Clemens and get a 3 minute quick start explanation … #techdaysnl
TechDays in Den Haag, Netherlands
TFS11 Service is the brand new Team Foundation Service on Azure, it makes it very simple to have your own Team Foundation Server. But, there are some new concepts in TFS11.
In a 3 minute face to face session with me, we create a TFS11 on Azure account and you get an explanation of these new concepts so you can start using it and be productive immediately.
Look for this laptop at the Sogeti boot or ATE area at the TechDays in Den Haag, Netherlands.
Thursday only.
Decks: TFService Basic and Advanced
Decks I used Yesterday for a TFService presentation. Normally it is a demo only presentation, but a scheduled maintenance made me use them 
(see http://blogs.msdn.com/b/tfservice/ for the schedule). Lucky me I have them, now you have hem too…
First presentation (second in the series, first session was about scrum 101) is some general usages of TFService and how it supports an agile way of working.
Second deck (fourth in the sessions series, third one was about how we use it in real projects) is about how we solved some usages challenges like multiple backlogs, bug backlog, testing etc.
Some more reading:
- http://www.clemensreijnen.nl/post/2011/10/09/Teams-in-VS11.aspx
- http://www.clemensreijnen.nl/post/2011/10/10/Teams-in-Visual-Studio-11-feature-teams-and-backlogs.aspx
- http://www.clemensreijnen.nl/post/2011/10/12/Teams-in-Visual-Studio-11-CTP-the-Bug-Backlog.aspx
- Teams and TFS Groups in TFS11, the backlog, board and security settings.
Teams and TFS Groups in TFS11, the backlog, board and security settings.
Back to the ‘teams’ topic, it is new in TFS11 … (see below the previous posts about this topic)
- http://www.clemensreijnen.nl/post/2011/10/09/Teams-in-VS11.aspx
- http://www.clemensreijnen.nl/post/2011/10/10/Teams-in-Visual-Studio-11-feature-teams-and-backlogs.aspx
- http://www.clemensreijnen.nl/post/2011/10/12/Teams-in-Visual-Studio-11-CTP-the-Bug-Backlog.aspx
Beside using the new team concept in TFS11, you also just can do it the ‘old’ way. Put all your team members in the different TFS Groups (readers, contributors, … ) and give them access to the team project. Just as you did the past decade. But, you have to understand what happens when you do it. You also can mix things, create teams and only add TFS groups to it, or give people specific rights within a team… actually you can make it a complete chaos. In this post some usages scenarios. (at the end of this post a final conclusion, maybe you want to read that one first.)
Scenario 1: A team project with no teams and only team members in TFSgroups configuration.
When you create a team project, you get by default a team with the team project name, prefixed with ‘team’. This default team has one member, the creator of the team project. Which as you can see I deleted.
When you browse to the administration page, you will notice that you can’t delete the default project. but, with no members it can’t do any harm.
Next, you can start adding members to the different TFS groups. For example I added several people to the contributors group.
Following step, make security settings for the different groups... for example for the iterations
or the areas. You can find it at the same places as you can in VS2010.
Interesting is that the security settings for the source repository only can be set in Visual Studio.
When you have a team project without a team (beside the default team), you do have a backlog, with a board.
Everything works as expected, only the capacity planning seems to be broke. The ‘only members in tfs groups’ project gets a bit more complicated when you start working with area’s. For example add two sub-areas and mark one of them as current.
Good to know when you mark the parent area as the current one you can set a child area as default. When adding backlog items by using the backlog it will automatically set the area property.
Doing this and adding PBI’s and backlogs to the this area. Will result in a new backlog and new task board, helping the team members (which are only in TFS groups) to focus on this specific area only. Set the area setting to the parent area, including all sub area.. will result in a combined backlog.
The only thing what stays the same when changing the areas for your team project are the Team favorites (we don’t have a team :) and the queries behind it don’t have a current area path clause. You could add it yourself.
Conclusion
It is possible to create projects without using the new Team concept. Things look a bit strange, we have team favorites but no team members for example. But the main functionality works. It only gets a bit complicated when you start using areas and make different people responsible for different areas. You have to swap between the admin page and the home screen when you want to see a different backlog / functional area.
Scenario 2: teams only
I created two additional teams for the two different areas, added the team members to the default team and added the default team to Team 1 and 2.
All teams are in the contributors TFS group, resulting in the fact that all team members have the same security settings.
Having you teams setup in this way, gives them the easy capability to swap between the teams. The backlogs are up to date and the team capacity planning works. One annoying thing, the team favorites, the work item queries, are the same for the whole project so I have to create queries for every team and add them as team favorite.
Scenario 3: teams with tfs groups
Another scenario, for playing with security, teams and TFS groups is… adding the team members to TFS groups and add these groups to the teams as members. Now, you have members in your team with different security settings. Doesn’t feel really scrum, but it works.
Only the TFS Groups Readers and Contributors in the the default team and Team 1 and 2 have only the default team as team member.
Scenario 4 and 5 Teams with team members with specific security settings.
The last scenario, is where you add only users to your team and set explicit security settings per team members.
I won’t recommend this, you really can’t administer this.
Final Conclusion.
So, after this small journey across the different possibilities of configuring security settings in TFS11, I must say… Be careful, you can create chaos which you aren’t able to administer anymore, and think up front about your areas and team structure, it influence the usage and capabilities of the nice agile tools.
- Scenario 1: No Teams only TFS groups with Team members.
You can use the backlog and task board, the capacity planning doesn’t work (yet?). It gets complicated with multiple areas and multiple backlogs. For example when you want to use a bug backlog, don’t use this scenario go for number two or three.
- Scenario 2: No TFS groups only Teams with Team members
This feels like the only real agile solution, all team members have the same security settings and the usages of multiple backlogs is easy.
- Scenario 3: Teams with TFS Groups with specific security settings
When you want to have team members with different security settings, this is the way to go. Areas and multiple backlogs are fully functional including the capacity planning.
- Scenario 4: Teams with Team members with specific security settings
Forget this one… you will go crazy over time
- Scenario 5: Teams with Team members with specific security settings and with TFS Groups with specific security settings.
You not only will be crazy you probably will have to go to the hospital when you try to maintain this scenario.
For now… number three is my favorite, number two is more pure… one is ok, four and five not ok.
TFS11 Scrum board update remaining work without opening the work item…
Very nice MSFT added the capability to update the remaining hours without having to open the work item. and even more nice it has some intelligence behind the drop down, because it populates the dropdown list with values in range.
TFS11 on Azure January 2012 update – request feedback
The latest update on TFS ( see: Team Foundation Service Planned Maintenance Tues Jan 17th ), fixed an issue with builds, they look stable again. And my Azure build service executes them happy. Now its time to expand this scenario with test controllers and test agents for automatic test execution and test environment provisioning.
But, I want to point to the new menu item ‘request feedback’ … it is the feature for getting early feedback about your product, for example during the sprint review.
It is a really nice feature supported with a tool similar as the Test Professional, Test Runner.
More explained with a walkthrough for the dev preview can be found in Brian Kellers dev11 VHD and the handson lab: Building the Right Software - Generating Storyboards and Collecting Stakeholder Feedback with Visual Studio 11 (docx).
How it works, I can start asking feedback … about a feature I just created. For example in this screen I’m asking if Brad the Product Owner can look at the website for typos.
It generates an email for Brad, asking to start the feedback session.
Now this is where TFS Azure isn’t complete, it can’t send the mail and the feedback tool URL points to the MSDN subscriptions download page. But, when you look at the Brian Keller hands on lab you know what happens.
there isn’t a feedback work item type..
A very promising feature, special in distributed teams and with TFS on Azure we can ask any one for feedback.
So, get your product owners informed… they will get involved in dev11.
Teams in Visual Studio 11, the basics.
The next version of Visual Studio is supporting Teams. Members in a team work together on a project or feature, they are multidisciplinary and deliver working software in sprints. From the online help:
By defining a group of people who work on your team projects, you can more easily organize, track, and facilitate the work that those people perform. This organizational unit is called a team, and you create a team for each team project. Members of teams can work together to complete a list of work items, called a backlog, that you define or that members of the team collectively define for themselves. Each backlog represents the planned or proposed work that your team wants to accomplish. For additional structure, you can create and use areas to help categorize the work items that your team will perform. By creating iterations, you can also define the schedule against which you want your team to track and perform the work.
https://tfspreview.com/_content/TeamHelp.htm
It is a bit more complex:
- A team project can have zero or more teams.
- A team contains one or more team project members (an empty is not possible)
- A team project member doesn’t need to be in a team
- A team project has release / sprints (with date, which is cool)
To set the area and iteration for a project, select the team project and select ‘administration’ on the right. Team project name is bold. Area and Iteration are also visible in the team administration which can get confusing. You can only set the data for a sprint, and add areas in the team project admin screen.
- A team project has areas defined
(see iteration for details)
- A team project is in a sprint (date restriction)
- A team is responsible for an area
- Multiple teams can be responsible for the same area (I won’t recommend doing this)
Team 1 and Team 2 responsible for the same area, so they have the same backlog.
and the same sprint backlog, but with different capacity planning.
and the teams have the same same task board, but… a team can move other teams tasks (with the right security settings)
(again… don’t do this)
- A team follows the team projects sprint cycle.
All team are in the same sprint (look at the sprint backlog in the images). When a team isn’t selected for a sprint, the sprint burndown graph in the sprint backlog isn’t available.
- A backlog contains product backlog items for a teams area. also when it is assigned to someone else.
- The task board contains tasks for an area for the current sprint.
- The task board contains team members with tasks for an area for the current sprint.
(see Brian Harry’s post: http://blogs.msdn.com/b/bharry/archive/2011/06/14/agile-project-management-in-visual-studio-alm-v-next.aspx)
- The backlog visualizes the capacity planning for the team responsible for the area.
(see Brian Harry’s post: http://blogs.msdn.com/b/bharry/archive/2011/06/14/agile-project-management-in-visual-studio-alm-v-next.aspx)
- You can assign a task to a team project member which isn’t in a team. <— don’t do this, it will make a mess.
(only team project member Clemens-C is in Team 3)
not only the task board gets confusing also the capacity planning. see screenshots: Multiple teams can be responsible for the same area.
So, you can do a lot with the board and the backlog for team management, but you also can make a mess out of it. When you don’t understand it anymore, you can fall back on the work item queries. But at that moment you really know you’re project is in danger.
