Software Performance and Load Testing / Application Optimization

Testing the Testing Strategy

Formerly the site of Gray Test Consulting, I am now using the site for sharing information about software testing best practices

Part 3-Proper Scenarios and Use Cases

As of this point, we should have well defined goals, objectives and success criteria (part 1) and we should have an idea of how we want to build our data (part 2). Now we want to lay out the basic scenarios and use cases. Before we start this, I want to give you my definitions for “Scenario” and “Use Case”. I have two versions for each term. The first applies to how we work with them in our test planning and the second applies to how they are used inside Visual Studio.

Term Scope Definition

Scenario

 Real World (test plan) – A Scenario here defines a grouping of different tasks that a user may perform on the system. I often define a separate scenario for each type of user role that will be part of the test (a school application might define teachers and students as two different roles. Each role will get a separate scenario in my test plan).
Scenario Visual Studio (load test) – A logical grouping of tests and test settings in a Visual Studio Load Test that tells the test rig how to generate load.
Use Case Real World This is a certain task that a user will perform using the application.
Use Case Visual Studio (web test) – The specific commands in a web or unit test that mimic the steps a user performs when executing a task (Real World Use Case).

Example 1 – “School’s Back In Session”

For this example, I will use a web site that a school uses to manage the classes offered. I will focus on two user roles; TEACHERS and STUDENTS. This will allow me to define two different “scenarios” in my test plan. Then, for each scenario, I will look at the work that a user from each role might do. The work will be broken down into specific tasks, and each task will become a “use case”:

  • Teacher:
    • Login (I define this separately because I know it will be a test I want to perform so I get it out on the radar right away)
    • Enter grades for tests already taken
    • Submit updated homework assignments to public portal
  • Student:
    • Login
    • Look up test scores for current semester
    • Download the latest homework assignments from the public portal

Example 2 – “Are You Ready For Some Football?”

My apologies up front to all of the readers who know soccer as “football”. I love soccer, but in this case, I am partial to what Americans call football and am referring to NCAAF and the NFL. Anyway, it’s time to hit the gridiron and I want to make sure my site can handle the load. So for this example, I will focus on a site that hosts both fantasy football and real football information, sales and game updates.

  • Fantasy Football Fan:
    • Draft players onto the team
    • Check the stats for the players throughout the season
  • Real Football Fan:
    • Buy tickets for the game
    • Follow the game in real time on the site.

Building the Test Plan Items

Now that we have our scenarios and use cases, we can incorporate them into our test plan. As we fill in this information, keep in mind that the data I am providing here has NOTHING to do with Visual Studio or any other testing tool. I am merely laying out the expected real world behavior that I wish to mimic when testing.

Example 1 – Our GOAL for this testing (just to make things easy to follow) is to ensure that the site will be able to handle the expected load and perform within the expected criteria (see the Introduction post for more info if needed).

Scenario Use Case # users # times per user per week Notes
Teachers Login 150 5
Teachers Enter grades 150 5 Assumes teacher has 5 classes, therefore, updates 1 test set per class per week.
Teachers Update homework 150 25 Assumes teacher has 5 classes, therefore, updates all homework each day.
Students Login 3000 5
Students Check grades 3000 4 Assumes student has 4 classes.
Students Get new homework 3000 20 Assumes student has 4 classes.

Of course, your table will be much more detailed and may contain caveats or other requirements here, but the idea is that we defined our use cases and numbers based on “expected load” and using “real world” examples.

Example 2 – Our GOAL for this testing is to determine the breaking point of the application, therefore we use a slightly different table to define our scenarios and use cases. Since we are finding a breaking point, we do not know what the final real world user load will be. However, we should be able to at least have a ratio of the amount of users and actions relative to all of the other use cases involved.

Scenario Use Case # users relative to other use cases # times per user per week Notes
Fantasy Draft 100 0.1 Draft only happens once per season, so this number is way low
Fantasy Check Stats 100 20
Real Buy Tickets 1 1
Real Follow Game 2000 180 This assumes that the users update their page every minute of a 3 hour game.

In the above table, the “# users” column represents a ratio (or multiplier) that I will use to increase load. If I decide that there will be 50 people buying tickets, then I will have 50 x 2000 (or 100,000 people following the game online). Also, we might want to be able to change the load for the “Real” scenario without changing the load for the “Fantasy” scenario. If that is the case, then we should call that out in our test plan. This is another example of “getting all of the necessary information up front.”

NOTE: I could easily split the “Real” scenario into two scenarios if I wanted to, but I chose to make it simpler. It doesn’t affect the way I will approach building the harness.

Conclusion

At this point, we should now have enough data to start building the test harness and start modeling the loads and profiles. We will cover that in the next post.