Wednesday, December 21, 2011

Testing the New Client/Server Arrangement

After the brief delay caused by my hard drive failure, the code is now in place for the local versions of the client and server.  The first round of testing has produced a pretty intimidating to-do list. Here's what went down:

The Procedure

I established a testing procedure that I believed would involve a sample of the affected functionality and uncover any regression issues. This is not a comprehensive test plan, but it did involve a good representative sample of client/server interaction.
  1. Select Single Player from the main menu.
  2. Log in as myself (Michael).
  3. Select a team color.
  4. Recruit a team captain.
  5. Send a chat message from the Lobby.
  6. Recruit another character.
  7. Upgrade a character.
  8. Signal ready.
  9. Move all characters through two or three rounds in the game.
  10. Cycle through the team and character info displays.

The Results

I went through those same steps four times in a row without modifying the code. The test results follow.

Step Test 1 Test 2 Test 3 Test 4
1 Passed Passed Passed Passed
2 Passed Passed Passed Passed
3 Passed Passed Failed (Server registered me as a spectator.) Passed
4 Passed Passed Unable to proceed. Passed
5 Failed (Engine defect.) Failed (Engine defect.) Failed (Engine defect.)
6 Passed Failed (Character list had disappeared.) Passed
7 Passed Unable to proceed. Passed
8 Passed Passed
9 Passed Failed (My characters worked but one AI got duplicated.)
10 Passed Passed


Conclusion

There is more work to be done!  I'm going to tackle one defect at a time and stick with these same ten steps until they work reliably.  Then I'm going to elaborate with a more detailed test plan.

No comments:

Post a Comment