Testing in one week sprints, hierarchy backlog items and test coverage.
One week sprints are great and bring big benefits to you team and product.
The artifacts that are in your system will be; 
the backlog with nice small backlog. That small that they can be realized in the one week sprint. your sprint will contain a 
test plan for the sprint, with related product backlog items. Probably there also will be some 
additional test plans for other concerns/ risks. The 
regression test plan finally will contain a collection of test cases gathered from the sprint test plans and the additional test plans. And the customer will use a 
acceptance test plan for there work.
This works great. And, because the PBI’s 
with their corresponding test cases 
are that small the team will have the continuous discussion if the needed validation should be implemented with unit test technology/ automated test or, if it would be easier/ faster/ better/ higher ROI to specify and execute the test as a manual test. Work around testing in this way makes it a real team effort which makes it easier to get testing done in the sprint.
How should we test this PBI?
The test cases during a sprint are that small that also the 
regression set will become a collection of very small validations of the system. After several ‘one week’ sprints the bigger scenario / end to end test cases are missing. A collection of small validations made the team uncomfortable about the quality of the regression and unit test case set.
For example: when making several settings the system behaves in a different way. In our situation is was the backend connection with either SharePoint 2007, 2010, 2013 or O365. Making this setting and saving this setting was one PBI. The different behavior according to this setting where other multiple PBI’s. And the overall collection of settings (all different PBI’s) made the system behaves different. We needed an Activity Diagram to explain its behavior.
The decision to make use of Parent PBI’s is easy made (we called them Features and prefixed them with this name).
Related to these ‘feature’ PBI’s a feature test plan can be created, which will cover the full scenario and all path coverage's. in this way giving the team the comfort that all paths and scenarios are covered.
Finally ending up with a bit more complex organization, but with a better test coverage.
Feature PBI’s broken down in child PBI’s, which are partly implemented in a sprint. A nice additional benefit is that the team focuses on Feature PBI after Feature PBI, implementing them one after another. 
Activity diagrams support the scenario, and when needed use case or other UML diagrams. The overall scenario (activity diagram) is related to the Feature PBI and the different actions can be related to the child PBI’s. 
Testers can use their Test Design Techniques when analyzing the UML diagram for test coverage and test cases. And the 
regression set will be a mix of Sprint Test plan test cases and Feature Test plan test case.
Everything perfect. But the bit more complex organization of the backlog has its challenges. Not all PBI’s relate to a Feature PBI and often a PBI’s influences / relates to multiple Feature PBI’s. The one activity diagram per feature is fake and a theoretically example, the real world is more challenging. See also Martin’s post You can’t stack rank hierarchical work items?.
Although it will give you some extra backlog grooming activities, still Features PBI’s can bring benefit to the system. Don’t try to make it too perfect and definitely have testing knowledge in place for the coverage then it is a good thing to give it a your bigger systems.
6 Testing with Visual Studio 2012 Agile TMap _ The QUALITY A-Z Roadshow
The QUALITY A-Z Roadshow:
A full day overview, with demo’s and real world experiences of all the testing capabilities of Microsoft Visual Studio and needed testing practices.
1: Keynote: The Value of Quality
2: Test Planning
3: Test Specification and Execution
4: Test Controlling and Reporting
5: Test Infrastructure and Virtualization
6: Operations Integration
Feel free to ping me if you want to have these sessions delivered at your company.
10. Getting Testing Done in the Sprint – the Definition of Done
Are we done? When this is unclear the team will do too less or too much. Both are bad. The Definition of Done helps to define when you are done, the same as a master test plan helps to define the test levels. A good definition of done specific for your team and project will definitely help knowing when you're ready and to get testing done in a sprint. Mainly because you know when you're done.
Making a good definition of done is a trivial task and books are written about it.
A good starting point are these posts:
One thing I really like to keep in mind when defining a Done List are:
- Risk Analysis, and together with this the Master Test plan. I should be clear what the risks are and what the impact is on when you are done.
You can think of Done ListItems, which help understand the level of testing to be done, like: - All High Risk BPI’s have a test coverage of ..% and no open bugs
- All high prio test cases are automated
- Low Risk, Low Business PBI’s may have 10% open low impact bugs.
- Test Levels, balance the unit, functional and acceptance testing and define where what is tested. Definition of Done items like; all unit tests are green, all functional testing is done or all written acceptance tests are executed, are unclear. How many tests should we write to be done? Code Coverage is one for unit testing, but how do you know when your done specifying functional tests and acceptance tests. So, writing down in the Done list that testing is done isn’t enough it also should have a statement about when test writing is Done.
- When are we done creating the Regression Set and when are we done with Test Automation
So, when testing is important and you don’t want to create a random amount of test cases for each PBI, should mention it in the Done List, and for sure make the validation automated with a build system.
Past Tips:
01. Getting Testing Done in the Sprint - The Team and Activities
02. Getting Testing Done in the Sprint – Regression Test Sets
03. Getting Testing Done in the Sprint – Test Automation
04. Getting Testing Done in the Sprint – Undone Backlog Item
05. Getting Testing Done in the Sprint – No Double, Triple Testing
06. Getting Testing Done in the Sprint – PBI Implementation Sequence
07. Getting Testing Done in the Sprint – Risk and Business driven Tests
08. Getting Testing Done in the Sprint – Write Logical Acceptance Tests
09. Getting Testing Done in the Sprint – Test Tasks on the Board
Next Tips:
11. Getting Testing Done in the Sprint - The Customer Test Team
12. Getting Testing Done in the Sprint – The Test Branch
…
…
08. Getting Testing Done in the Sprint – Write Logical Acceptance Tests
During release planning meeting. Capture acceptance criteria and immediately add the as logical test cases to the PBI. It will help the team to understand, clarify the discussion and more important for this topic, it helps testers be involved, and be important at the early stages of the software cycle.
Within VS11 | TFS11 this is very easy to accomplish:
- add the PBI to the backlog.
- add logical test cases, from the backlog item work item. and only add the test case title.
- Start planning and execute the sprint
- open Microsoft Test Manager, add a test plan for the current sprint.
- Add the Backlog items as Requirement Suites to the plan and see the test cases listed in the suite, ready to add test steps.
This is a very small tip, but is very useful to get testing involved in a sprint (see tip 01).
Past Tips:
01. Getting Testing Done in the Sprint - The Team and Activities
02. Getting Testing Done in the Sprint – Regression Test Sets
03. Getting Testing Done in the Sprint – Test Automation
04. Getting Testing Done in the Sprint – Undone Backlog Item
05. Getting Testing Done in the Sprint – No Double, Triple Testing
06. Getting Testing Done in the Sprint – PBI Implementation Sequence
07. Getting Testing Done in the Sprint – Risk and Business driven Tests
Next Tips:
09. Getting Testing Done in the Sprint – Test Tasks on the Board
10. Getting Testing Done in the Sprint – Done
11. Getting Testing Done in the Sprint - The Customer Test Team
12. Getting Testing Done in the Sprint – The Test Branch
…
…
07. Getting Testing Done in the Sprint – Risk and Business driven Tests
I really like the mindset “no risk, no test”. So, when there isn’t any business risk, there aren't any tests and is it easy to fit testing in a sprint. More realistic do a good risk analysis on your product backlog items before start writing thousands of tests. Also in scrum is risk an important attribute.
The release plan establishes the goal of the release, the highest priority Product Backlog, the major risks, and the overall features and functionality that the release will contain.
Products are built iteratively using Scrum, wherein each Sprint creates an increment of the product, starting with the most valuable and riskiest.
Product Backlog items have the attributes of a description, priority, and estimate. Priority is driven by risk, value, and necessity. There are many techniques for assessing these attributes.
from the scrum guide
Within the TMap test approach product risk analysis is an important technique. Determine risk analyses are part of the proposed activities in the Master Test Plan of TMap: ‘Analysing the product risks’. It not only supports the Product Owner to make the right decisions it also the Team benefits in a later stage, this information is invaluable while defining the right test case design techniques for the Product Backlog Item.
“The focus in product risk analysis is on the product risks, i.e. what is the risk to the organization if the product does not have the expected quality? ”
www.TMap.net
Having full product risk analysis for every Product Backlog Item during the Release Planning meeting is slightly overdone, but the major risks should be found. Determine product risks at this stage will also provide input for the Definition of Done list.
TMap Product Risk Analyses
Within the Visual Studio Scrum 1.0 Process Template Product Backlog Items are written down in the Work Item Type ‘Product Backlog Item’. This Work Item Type hasn’t got a specific field for risk classifications. Adding a risk field is preferable (TFS Powertools makes this an easy task) so you can query on this property, or you can make the risk analyses a more product generic property.
TFS Scrum Product Backlog Item
The flow with risk analyzing, classification, discussion and test design with the Product Owner can look like the diagram below.
But again most important for fitting testing in a sprint, know the risks use test design techniques to cover the risk and only write useful test cases.
Post partly taken from previous posts:
Past Tips:
01. Getting Testing Done in the Sprint - The Team and Activities
02. Getting Testing Done in the Sprint – Regression Test Sets
03. Getting Testing Done in the Sprint – Test Automation
04. Getting Testing Done in the Sprint – Undone Backlog Item
05. Getting Testing Done in the Sprint – No Double, Triple Testing
06. Getting Testing Done in the Sprint – PBI Implementation Sequence
Next Tips:
08. Getting Testing Done in the Sprint – Write Logical Acceptance Tests
09. Getting Testing Done in the Sprint – Test Tasks on the Board
10. Getting Testing Done in the Sprint – Done
11. Getting Testing Done in the Sprint - The Customer Test Team
12. Getting Testing Done in the Sprint – The Test Branch
…
…
06. Getting Testing Done in the Sprint – PBI Implementation Sequence
A challenge in getting testing done in a sprint is the fact that the software (PBI) isn’t ready to be tested till implementation/ coding is done.
How To.
Work on completing each item in the sprint backlog and finish one item after another, see task boards below.
BAD -- Not able to start testing task board, every team member works on a different pbi. Testing can only start at the end of the sprint.
GOOD -- Ready to start testing task board, team members work together on a pbi one after another till it’s done. Testing can start early.
It is a good practice, but not often followed. it really helps getting testing done.
Past Tips:
01. Getting Testing Done in the Sprint - The Team and Activities
02. Getting Testing Done in the Sprint – Regression Test Sets
03. Getting Testing Done in the Sprint – Test Automation
04. Getting Testing Done in the Sprint – Undone Backlog Item
05. Getting Testing Done in the Sprint – No Double, Triple Testing
Next Tips:
07. Getting Testing Done in the Sprint – Risk and Business driven Tests
08. Getting Testing Done in the Sprint – Write Logical Acceptance Tests
09. Getting Testing Done in the Sprint – Test Tasks on the Board
10. Getting Testing Done in the Sprint – Done
11. Getting Testing Done in the Sprint - The Customer Test Team
12. Getting Testing Done in the Sprint – The Test Branch
…
…
04. Getting Testing Done in the Sprint – Undone Backlog Item
A tip straight from the previous Scrum Guide (The guide updated and the undone TIP is removed, read The 2011 Scrum Guide – A Lot of Changes! ). But, the TIP still works, although it isn’t an ‘official’ tip anymore…
End to end testing, is not an agile specific testing practice, it’s a common sense testing practice. Create test cases to detected defects anywhere in the chain. This type of testing has been growing the couple of years enormous due to the rise of Service Oriented Architectures and more recent Cloud Computing. as the TMap end-To-End Testing books describes
“End-to-end testing is a form of testing in which the (business) process is tracked through an interconnected series of systems, which may be based in different organizations, aiming to answer the following questions:
- Does the IT process support the business process adequately? - Are the process and systems integrated with each other in the right way?”
TMap.net End-To-End Testing
End-to-End testing is seen as a challenges in Agile environments, with the most important challenge that systems under development aren’t ready for End-to-End testing. Not every service is finished to test the complete end-to-end business process. Stubs are a solution, but can these be made in the current sprint and are stubs good enough to say something about the quality of the system. And, when the system are ready and a change in a service is needed, how and can we re-run all the end-to-end tests, is there time enough.
All these challenges with end-to-end testing in agile projects can be summarized in two topics:
Scrum solves this problem, this problem of not able to be compliant on the definition of done at sprint finish, by defining ‘Undone work’. the scrum guide mentions this in a TIP and in the final thoughts of the scrum guide
Some organizations are incapable of building a complete increment within one Sprint. They may not yet have the automated testing infrastructure to complete all of the testing. In this case, two categories are created for each increment: the “done” work and the “undone” work. The “undone” work is the portion of each increment that will have to be completed at a later time. The Product Owner knows exactly what he or she is inspecting at the end of the Sprint because the increment meets the definition of “done” and the Product Owner understands the definition. “Undone” work is added to a Product Backlog item named “undone work” so it accumulates and correctly reflects on the Release Burndown graph. This technique creates transparency in progress toward a release. The inspect and adapt in the Sprint Review is as accurate as this transparency.
Within the Visual Studio 2010 Scrum process template this means that you have to link or create tasks to a specific ‘Undone Work’ product backlog item. this ‘Undone Work’ product backlog item accumulates the undone work regarding to End-to-End Testing (and others) over time and can be monitored just as other backlog items.
http://www.slideshare.net/xwarzee/path-to-agility-ken-schwaber
Tool Support
In TFS with the Scrum process template we can create Backlog Items very easy.
So, creating an undone backlog item isn’t that hard, the challenge is take it with you every sprint. for example in my undone backlog item I’ve got some undone tasks from sprint 1 and some from sprint 4. The iteration paths for these tasks still point to the sprint 1 and 4 sprints.
This will result in a wrong sprint backlog, because the tasks who point to other sprints as the current sprint won’t appear in the sprint backlog. So, when moving this Undone backlog item to another sprint because not all tasks are done, you also have to change the iteration path for the corresponding tasks.
Just keep in mind TFS doesn't update the hierarchy, you have to do it manually (or use Excel, my favorite).
Past Tips:
01. Getting Testing Done in the Sprint - The Team and Activities
02. Getting Testing Done in the Sprint – Regression Test Sets
03. Getting Testing Done in the Sprint – Test Automation
Next Tips:
05. Getting Testing Done in the Sprint – No Double, Triple Testing
06. Getting Testing Done in the Sprint – PBI Implementation Sequence
07. Getting Testing Done in the Sprint – Risk and Business driven Tests
08. Getting Testing Done in the Sprint – Write Logical Acceptance Tests
09. Getting Testing Done in the Sprint – Test Tasks on the Board
10. Getting Testing Done in the Sprint – Done
11. Getting Testing Done in the Sprint - The Customer Test Team
12. Getting Testing Done in the Sprint – The Test Branch
…
…
TFS11 Scrum board update remaining work without opening the work item…
Very nice MSFT added the capability to update the remaining hours without having to open the work item. and even more nice it has some intelligence behind the drop down, because it populates the dropdown list with values in range.
TFS11 on Azure January 2012 update – request feedback
The latest update on TFS ( see: Team Foundation Service Planned Maintenance Tues Jan 17th ), fixed an issue with builds, they look stable again. And my Azure build service executes them happy. Now its time to expand this scenario with test controllers and test agents for automatic test execution and test environment provisioning.
But, I want to point to the new menu item ‘request feedback’ … it is the feature for getting early feedback about your product, for example during the sprint review.
It is a really nice feature supported with a tool similar as the Test Professional, Test Runner.
More explained with a walkthrough for the dev preview can be found in Brian Kellers dev11 VHD and the handson lab: Building the Right Software - Generating Storyboards and Collecting Stakeholder Feedback with Visual Studio 11 (docx).
How it works, I can start asking feedback … about a feature I just created. For example in this screen I’m asking if Brad the Product Owner can look at the website for typos.
It generates an email for Brad, asking to start the feedback session.
Now this is where TFS Azure isn’t complete, it can’t send the mail and the feedback tool URL points to the MSDN subscriptions download page. But, when you look at the Brian Keller hands on lab you know what happens.
there isn’t a feedback work item type..
A very promising feature, special in distributed teams and with TFS on Azure we can ask any one for feedback.
So, get your product owners informed… they will get involved in dev11.