“Test and Attack” is a phrase that describes a methodology for using load tests to tune environments and optimize applications in a short period of time. The idea is to have experts from all the different areas of development and operations working side by side executing load tests, analyzing the results, implementing quick changes/fixes and repeating the steps until a favorable outcome is achieved from the load tests.
The diagram below shows the four parts required for a successful engagement. The Plan and Prepare sections are done prior to the testing effort. The “Execute” phase is the part that requires all of the members be present (or at a very minimum be on an open video-conference). Here we will run a test, the experts will collect logs, traces, data, etc. Then they will analyze the results and (AS A TEAM) decide on a single change to make. We will make the change directly in the testing environment and rerun the test to compare the results. We can then determine if we want to keep the change or toss it and try the next item. We repeat this process until we see diminishing ROI or we run out of allocated time for the testing.
The final phase, “Report” is also critical because this is where all of the changes that were discovered get reported back to management and implemented in the production code.
Thus, the QA teams are testing WITH the development teams, not FOR the development teams. This allows for much faster and more accurate tuning/optimization efforts.
Testing the Testing Strategy 