Test Plan Development for Game Testing

Hi!
Here we go with one my part of the text found here.
As I said before, this text is not mine and if you know who is the author, please, let me know so I can give the RIGHT CREDITS to him/her/it.
Enjoy the reading!
There are two types of test cases; one is testing the functionality of the product; commonly known in the software industry as Positive Testing. The other one is testing the durability and the level of tolerance resistance of the game; known as Negative Testing (it is also known as Stress Testing or Load Testing). “Negative” test cases are created with “breaking the game” in mind, so we will test for certain odd situations and determine if the game knows how to respond, and reacts correctly. Examples of this odd situation are,
– Load a game without a MEMORY card, or pulling out the MEMORY card during game loading.
– Running a game for more than 48 hours in order to test for memory allocation and management. When the game component does not handle the use of memory properly, some of the memory may become unusable (the memory does not get recycled). When it happens, a game will become non-playable (e.g. freeze) after along-playing time.
– For the playability testing, say, a scenario is developed for the “Head-to-Head” mode in Snowboarding, we have Player-1 boarding very fast, and finishing much earlier than Player-2, which is boarding extra slow. We expect the game should be able handle this extreme situation (e.g. no freeze, the scoring continues to work the way it should be) till Player-2 is finished. Define Smoke Testing to test a new CD burn. The name of Smoke Testing comes from the engineering lab in testing new car engines. Before a new engine is put out for a road test, the engineers would simply start the engine and see if it works. If smoke comes out from the engine, they would know immediately that some parts do not work properly. Just like creating a new build, we simply run the game, if the game keeps on crashing, we would know some parts are not right. The Smoke Test for a new CD burn needs to be defined. The test plan is a simplelist of what main game features and options need to be tested so you are able to confirm a new build is successful. Just like testing a new car engine in the lab, you would start it and let it runfor a few cycles.
Thank you for reading my blog and, as I said, if you know the author or this text, let me know so I can give the RIGHT CREDITS to him/her/it.
Advertisements

Test Plan and Testing Documentation

Hello,

How are you guys today? Hope everything is good.

I found this text in a part of a huge document here. If you know who is the RIGHT AUTHOR, please, let me know so I can give the right credits to him/her/them.

TEST PLAN AND TESTING DOCUMENTATION

Effective testing relies on a complete Test Plan package. The document should be written in away that promotes reusability (reuse a large portion of the previous test plan documents for anew game) and consistency (to minimize learning curve; the testing documentation produced for the game projects all have a similar format).The hierarchical listing provided below is to outline a structure and the testing documentframework that the testing will be handled for each game project.
* Testing Requirements Document (as explained earlier)
* Software Testing Test Plan
* Game Testing Test Plan (positive testing – what is expected?)
* Game Testing Scenario (negative testing – how robust is the game?)
* Manufacture Platform Technical Requirements (e.g. SONY TechnicalRequirement Checklist for the PSX/PS2 game titles, and Microsoft TechnicalRequirement for the X-Box game titles)
The Game Testing Test Plan is a “positive testing” document that explains WHAT and HOW to test for the playability of the game (e.g. options and features).
QA has developed a guideline to explain the SONY requirements, section by section. We alsoreviewed the SONY Bug Report from our previous game submissions and develop a “our lessonlearned” checklist so “ we won’t make the same mistake twice”. As far as timing goes, thereview for the manufacture standards and technical requirements is often conducted after Alpha.
In the Test Design document, you will learn the details (e.g. definition, guideline, templates andideas) of preparing the test plan documents.
Once more, thanks for reading my blog and IF YOU KNOW THE RIGHT AUTHOR OF THIS TEXT, please, let me know so I can give him/her/them the right credits.

How to Conduct Effective Testing

Hi guys!
Conducting a test can be something really hard to do. You have to focus on everything that is going on and keep yourself on the line all the time. FOCUS is the word here and sometimes that can be really easy to loose.
However, the text that I found here may help you with that. Hope you can use that on you eveyday work.
Enjoy your reading!
IMPORTANT: If you know the Author of this text, please let me know. As far as I know, the source you can find here and I don’t know who is the owner of the text. Let me know so I can give the RIGHT CREDITS to him/her/them.
HOW TO CONDUCT EFFECTIVE TESTING

These are some of the “good ideas” from the experienced QA testers:
* Make sure you are testing on the most up-to-dated version of the game. With at least oneversion possibly created on the same day, it is easy to find yourself testing the wrongversion.
* Make sure that every time you get a new game build, you are getting every part of a newgame (a new game has more than one file!) and it may not work against a previous savedgame from a Game Save/Load device like MEMORY card.
* Keep track of your hours so you have time to look for new bugs and regression test the“fixed” bugs that you need to verify for closure.
* A good understanding of the Tracker database is a MUST. You must understand theTracker template, the definition and attributes of each field. An optional field does notmean the field is not
required; the field is not required only if it is not applicable to the bug.
* We must regression test the previous bugs. Just because something got fixed in the previous versions is not a guarantee that it still works in the current version. This is our mandate tore-test the previously closed bugs with the severity of A-Crash, B-Non functioning Feature, and C- Major Functional every week and for each major milestone.
* Regression testing can also be frustrating, make sure you stay focused and don’t think of itas a trivial task.
* It is NOT only tester who can enter bugs to the defect database. However, you should review the bug report if that is entered by a non-tester person (why? It is because you are responsible for regression testing, and the testers are responsible for all the bugs in the defect database).
* Keep in mind that you will find bugs, however your productivity is not measured by the number of bugs that you find. There is no fixed rule for a game tester as the number of  bugs that he or she must report in a day.
Once again, thank you for reading my blog and, if you now the AUTHOR of this text, let me know so I can give him/her/them the RIGHT CREDITS of it!

Have a good night and drive safe! 🙂

External Testing

Good morning!

First of all, I would like to ask that, if anyone knows the AUTHOR of this text, let me know so I can give the RIGHT CREDITS to him/her/them. I only found it here and there is nothing else about it.

However, the text is pretty good and I think it deserve to be shared.

Enjoy the reading!

EXTERNAL TESTING AND TESTING BY THE PUBLISHER

When a game is getting ready, we send the game to the publisher so they can take a closer look of the game and provide us with their feedback. Roughly 2 to 4 weeks before the Beta milestone date, we start sending the Publisher with the game build twice a week (e.g. every Tuesday and Friday). When the game is Beta, we will send the Publisher with our daily build of the game. Most of the publishers have their own FTP (File Transfer Protocol) site where we can upload the game file over the Internet. You should find out the FTP instructions from the Publisher.
The Publisher sends us their bug reports weekly to almost daily as the game is getting closed to Final. You must be aware that the publisher tests the game remotely, and uses a build that is afew days older than our latest version. We often receive “crack” bugs (i.e. misinterpretation of the feature) or bugs that have already been identified. This is the responsibility of the Test Lead to rectify these defect records before they are reviewed by the game designer or Producer. To facilitate the bug submission by the publisher, the Test Lead will provide with a file exchange format and the PVCS Tracker Defect Reporting template to the publisher so their bug data can be imported to our defect database.
When the game is (or closed to) Final, the game is sent to the game platform manufacturer for testing. The primary focus of testing is on the peripherals and devices (e.g. controller and game pad), the naming convention, standards and the legal text appeared in the NIS or menus. The tester must be familiar with the Technical Requirements for each of the game platform.
Usually there are two kinds of submission to the manufacturer:
– Pre-submission:
The game is sent to the manufacturer (e.g. SONY, Microsoft) so they will briefly look at it and provide us with some feedback. Pre-submission gets us some idea about the readiness of the game. This is sort of an informal review, if we don’t receive any problem, it doesn’t necessarily mean the game is bug free. We often have a couple of pre-submissions for a brand new game (e.g. our first action game of Jackie Chan Stuntmaster),and we may skip the pre-submission for the second generation of a previous game title.
– Submission:
The game is formally reviewed by the manufacturer, usually a team of people is assigned to test the game. The result in the form of a manufacturer bug report is sent tous through the publisher. The submission process takes about a week. A game must meet certain standards and requirements (e.g. no crash bugs, no more than certain number of non-critical bugs identified by the manufacturer) before it is considered “commercial ready” by the manufacturer, and no game can be published without manufacturer’s approval.
There are certain rules that must be followed for the submission procedure, e.g. the use of special CD media, the number of copies, and the game packaging, etc. Please refer to the manufacturer specification for details. The Test Lead must work closely with the Producer, the Publisher and the testing group of the manufacturer for the submission process.
Well, I think this is the end of our text – SOURCE here. Hope everything posted here could help you improve your work over the Game Testing because, after reading all of it, I guess I am a little bit more close to get my dream come true. 🙂
See you guys around and thanks for reading my blog!

Game Testing Methodology -Part II

Welcome to the Second Part of GAME TESTING METHODOLOGY.
Once more, if you find or know who is the owner of this text, please, let me know right away so I can give the right credits to him/her/it.
Part one can be read here!
SOFTWARE TESTING AND TESTING TECHNIQUES
The Game Testing Primer document outlines the role of software testing. The processes that are commonly used for Software Testing are:
A) Code Inspection:
review the source code and focus on logic structure, conformity with standards, the use of comments, and the completeness of program documentation and/or definition.
B) Incremental Focus Testing:
test chunk or pieces of code by feeding values/input toan isolated module, and analyze the intermediate result from the module. Usually a run-time facility, a debugging tool, a “dummy” front-end, which allows invocation of objects by their underlying method(s).
C) Data Analysis:
examine the data in the database; or analyze which part of the system uses or modifies any item of data. By tracing data items through the game play, the tester can validate if data usage, interpretation and manipulation by the modules or objects are appropriate.
D) Path and Flow testing:
verify if the correct sequence of objects or components is executed for an end-to-end path.
E) Algorithm-Specific testing:
test a specific game scenario or feature by setting (or tweaking) data variables, data values to the code, and execute it in the run-time environment. Different from the Incremental Focus Testing, the testing is to verify a suite of codes that make up a specific game feature.
F) Artificial Intelligence Analysis:
generate the run statistic of the programmable moves and plays of the AI components. The result is to validate if all the pre-programmed moves (e.g. side grip on the snowboard) and plays (e.g. Jackie Chan’scombination punch/kick in a multi-directional action) are used. Verify if the AI logic is appropriate and reasonable (e.g. in a basketball game, it would not be reasonable to havethe scoring 90% by dunk; and 10% by jump shot).
The following provides a brief overview of specific tasks and techniques that a Software Tester should learn or know while working on a game project.
1) File structures
A new software tester should get familiar with Radical’s file structure. There are several good documents pertaining to this from the global game and foundation technologies technical documents. The learning objective is basically to learn the overall game component architecture, and understand how it all flows together and to familiar yourself with the file dependencies of the game. When checking a game module, it is important to understand what parts its inner workings are dependent on. This will help you and speed up your buginvestigation and problem analysis process.
2) Make Files
A software tester will be responsible for the nightly builds. Every night the build is built using the latest modules so that in the morning everyone can be working on the same up-to-date code. Occasionally some thing will go wrong with the nightly build. A file should generate when the build is broken and it should say what part it failed on. You should use this file to in your  problem investigation. Usually some of the code may not work with another new part or maybe the pipelines are messed up. It is the software tester’s responsibility to go through the broken module and to find out why it’s broken and fix it if possible. If its too much for you to fix, you should then inform the person responsible for the code or the pipeline that something is broken and get them to fix it. Once the problem is fixed the build should be rebuilt to verify. The build number should be automatically amended with a post fix to represent the build number of theday. These build numbers will help you track down bugs or identify bugs within a specific build. For example someone might come to you with a bug on an earlier version. If you can identify what build they are working on and what build is completed, it could just be a simple fix of them getting the new build for the bug fix. It would be a good idea to send an email that tells people a new build is completed.
Another note on build is to make sure to mark the build release or debug because there could besome problems with one but not the other.
3) Memory management
It is a good idea to check the current status of the memory after a nightly build. If there are significant changes between the new day and day before, make a note of it. Something could be wrong. If the exe size grows too large then the game will take longer to load.
4) Debugger and Code inspection
Allot of your time is going to be spent on debugging or at least helping someone debug the game. One of the main skills a software tester should have is a STRONG understanding on how a debugger (Visual C++, Linux Debugger) works so that they can accomplish this task. They should be able to step through or into parts of code and be able to understand what is happening and how to set and change variables as well as run a function at any time during the program. At this time you will be doing some code inspection. The debugger can give you a lot of information on variables. Use this information to verify that the code is using all of its generated nodes and arrays and that any discarded arrays or nodes are properly cleaned up so no null pointer or such are created. Check the code for correct documentation for ease of use (e.g.no funky variables like temp;) ensure that the programmers are documenting correctly so their code can be picked up easily in the off case someone will have to continue their work. You may have to also do some sort of data inspection depending on the game. If a game uses some sort of statistics program and randomization, make sure the data is being stored correctly and is only available to the parts of the program that use it. Ensure that the data is not getting corrupted as it is coveted and used in the pipelines.
5) Test Programs
A very important part of the job will be for a software tester to write test programs. Some examples would be programs to enable cheat codes, play sound files, enable/disable the AI, recreate test scenarios, and whatever other debug information may be required. The Game Tester also uses these test programs so they can test for specific gameplay scenarioand/or situations.
6) Sound Testing
Sound testing is simply listening to the sound and making sure the startup point and the length are appropriate. The software Tester often requires to write a “sound test” program to play all the individual sound files from inside the game (just a quick tip, you should check with the sound artist and see if they have one already).
Sound testing includes testing if there will be any error in loading the files, listening to the sound files for errors or distortion. Also, test to make sure all the sound files are being loaded correctly. If the game has Colour Commentary (or CC), use the cc profiler to analyze. The analysis helps you to determine how many times a CC was called, and if it gets played or overwritten.
The analyzer relates the scene or situations to the cc and checks to see if the information gets played, if you notice any problem in cc, you should check if the information is being passed and played correctly. Of course one way to analyze the cc is actually listening to it and note any irregularities and problems.
7) P3D and pipelines
It is important to generally understand how the pipelines work. How the sound is being processed and converted, how the Maya art is being converted and used, generally how all the object data is exported from a raw data to a useable product. It would be a good idea to understand how P3D files are created and used. Look into the Anim classes, P3D chunks andcomplex messes.
8) Database and Game Statistics
The main purpose of the database for a game is:
1) Player or team statistics. It could be used solely for display, or it could b used by the AI engine to simulate the actual game play or for randomization.
2) Run-time statistics. It is used by the game engine to remember certain state, attributes or scoring of the player as you move from level to level (or to a different game state).
The easiest way to verify the database is to go into the game itself using a debugger, and analyze the game is using the data correctly. Obtain a printout of the database and look at the screens to make sure the data is being loaded, in the right place, showing the right information. On ahigher note, one could check the database export to make sure no error found in the export. Another good technique is to do random checking, say, on a player statistic, to make sure thedata is correct.
9) Overlays
Make sure the overlays are loaded correctly. A tester could write a program to test that theoverlays are being called correctly, and identify if any one does not get used at all.
10) Front End
Most of the Front End screens could relate to statistics (you know, in the database). The main thing to look into is the “Text Bible”, i.e. the appropriateness of the text, the naming convention,checking against any manufacture standards or requirements, and even spelling.
11) Bug Tracking Software
The software tester should know how to use PVCS Tracker to track and report bugs, and know the use of the standard template to create/customize a new defect database for a new project. It would be preferred if you could have administrator rights to tracker to set up others with accounts and manage the database (new client sign-on ID, fields and information).
12) Game Tester and love of the game
A software tester must also have “to have a love of games” in mind. A significant amount of your time will be playing games to find the bugs. You have to have the skills to find and report game bugs as well as help repair them. It is important to test for the fun factor of a game aswell, if you stop having fun while playing the game then make sure to discuss with your peerslike the Game Tester. If the changes happen too late they may not get implemented. Still,remember the debugger is your best friend.
13) Burning CDs
Simple, burn CD copies of the game for people who need them (game testers, management). A bigger part of this would be to track down problems of bugs being discovered on a different medium.
Well, once more I kindly ask you that if you know the Author of this text, please let me know so I can give the RIGHT CREDITS to him/her/them.
Thanks for reading my blog!

Game Testing and Testing Techniques

Hi!
Due to the importance that I find on this information, I though it would be better to make it’s own topic.
This is a part of the document that I found here.
So, if you know the author of this text, please, let me know so I can give the right credits!



GAME TESTING AND TESTING TECHNIQUES
The Game Testing Primer document describes a few aspects and focuses of game testing. This section outlines a systematic testing approach, and explains how you break down a game feature into its smallest testable components:
Systematic game testing means examining every part of a game. These parts include:
* the menu and the menu functions,
*art (character model, texture, terrain or world, crowd, objects, etc.),
* animation (the like and feel of the movement, realism, frame rate),
* sound and the sound effect (in conjunction with the facial animation, e.g. lip sync, and theanimation sequence),
*music,
* camera (cinematic view, zoom in and out, replay),
* game flow and logic,
* world/level/scene,
* the player attributes,
*the action attributes,
*the condition to advance to the next level (what are the rules?)
* the use of environmental objects,
* the event/object triggers, and
* the scoring
* progressive levels of difficulty,
* the AI logic (for both defensive play and offensive play; player movement and positioning),
* statistics (pre-game and in-game like player statistics and high score),
* title screens,
* NIS (Non-Interactive Sequence),
* SFX (Special effect)
* any movie clip,
* the game pad,
* the use of multi-button actions (also called button mashing),
* the ease of use of the button functions,
* the shock/vibration effect of the game pad,
* the use of digital and analog mode
* legal text, and
* the game options (game start/menu selection, hints, game pause, pause menu options , and scrolling, i.e. cycling through the available options on the screen, etc.)
(Author’s Note: Still, you don’t have to right it down everything in here. The game designers will provide you with all the information that you need and everything that should be checked in the game. It also may be much more than you can find here….)

Game Testing Methodology

Hi, everyone!
Here I come with one more post, but this time, talking about Game Testing Methodology as the title said!
Unfortunately, I couldn’t find the source of this text and all I know that this was available here. And also, I couldn’t find the author of it. So, if you want to have the full version of it, just check the website above.
Further more, if any of you know who is the author of this text, please, let me know so I can give the right credits.
Enjoy!
Effective testing comes from a well-structured approach and a well-defined testing methodology so the game product is highly satisfying to our Publisher and the game player. In contrast, poor testing results in a buggy game or software that gives rise to a long stream of repeated testingand project delays.
[…]
There are few, if any “fixed rules” for this testing methodology; however, there are manysuggestions, ideas, and guidelines for improving the quality and effectiveness of testing for our game projects. Hopefully, testers will be able to learn, understand, plan and carry out effectiveand efficient testing in a structured manner. If you have any questions about the information, please contact your Software Quality Director.
SCOPE AND DEFINITION
In a simplistic view, testing is to identify bugs found in the software, so the problem can be removed. There are different forms of tests and testing that can be categorized as “Black-Box” testing and “Clear-Box” testing (“Clear Box” testing is also known as “White-Box” testing inthe software industry). Their testing objective and overall processes are indifference (e.g., test planning, test design, testing execution, regression testing and bug reporting), but their focus of attention puts emphasis on different aspects of the game:
* “Black Box” focuses on the functional or the playability aspects of the game. For examples, testing the user interface (e.g., the selection menus and the use of buttons), the “look and feel” (e.g., the graphics and animation), and the actual gameplay.
* “Clear Box” is focus on the architecture and integration aspects of the game software. For examples, the use of a database, pipelines, the interaction/integration of game components like the rendering engine, the AI engine, sound, and so on.
For Black Box testing, the tester must know how to play the game (e.g., use of the game pad, know the rules and the game flow). For Clear Box testing, the tester must understand what coding is. The Software Tester uses a run-time debugging environment, feeds the code or chunks of code with input (i.e., data, setting variables, etc.) and analyzes the test result.
TESTING PHILOSOPHY
Testing is NOT a single person’s job, nor solely the responsibility of the Game Tester and the Software Tester for a game project. Every team member working in a game project must have“quality” in mind, and each person is responsible for the accuracy and completeness of the work that he/she produces.
This testing methodology is NOT the only process and it should NOT be used in isolation. Thereader must be aware that this testing methodology is considered as an integral part to the GamePre-production and Production processes.
In reality, no one can test a program COMPLETELY , i.e., testing every single part of the gameusing different and all available characters, so triggering different path of the logic and all the possible variations of input, interfaces, AI, and then output. Our testing strategy is to develop excellent, full-coverage, and effective testing (i.e., 80/20 rule).
TESTING PLANNING AND TESTING REQUIREMENTS
A requirement is an objective that must be met. The tester must understand most of the game requirements and translate them into functional terms (e.g., what needs to be tested, what things are testable and what not, what are the targets and measures, etc.), leaving the design and implementation details to the developers. As part of the testing requirement development process, the testers read the system documentation, gather all the visual and written information, analyze what and how the game components can be tested. It is the responsibility of testers to read all the relevant documentation so they can share (to understand and appreciate) the mission of the project […]. You are required to develop a Testing Requirements document for each game by outlining what and how the game and game components will be tested. The document includes:
* a list of features,
* the specifics of the internal design and external designs of the game. This mayrequire a description of the possible implementations if it makes the testing requirements easier to understand (e.g., certain theme of the game, the characters, the animation, the AI, cinematic or camera view, and so on). For example, to test the multi-directional fight action for the Chan PS2 game, you must make reference to the use of “Ring of Engagement”, describe how the opponents engage into the fight scene, and what you expect the single/combination fighting actions.
* a testing structure detailing if and how Game Testing and/or Software Testing is applied (i.e., in a spreadsheet format for the items identified above),
* the testing criteria (e.g., ideas for testing), and
* the completion criteria (e.g., what are the expected results, what does “something is done” or “something is working” mean to you in game testing?) After the testing requirements are identified, the Technical Director for the game and the Software Quality Director will review the document to confirm scope and priority. Following the test requirements, the testers work on their own test design and develop a Test Plan and Test Cases. Any testing dependency requirements must be identified and communicated to the game team so the game code is “test-enabled”, i.e., what kind of cheat codes, or “test automation-enabled” code are required?
The testing documentation is expected to be developed in the early stage of the project, i.e., draft testing documentation is produced when we have the first playable build. It is important to note that the testing requirements will not cover every single detail of the game, but it must cover testing all the contractually required elements (e.g. specific features and the major game functionality). You can obtain this information from the game Technical Director.
DEFINE THE TIMELINE REQUIREMENT
When a game project is pressed for time, we must recognize the existence of a threshold point where sufficient timeline must be provided for the testers to perform:
* a number of iterations for testing each new or updated game features,
* a complete cycle of regression testing for each build,
*sufficient regression testing of the previously Critical, Closed bugs, and
* a full regression testing expecting to test every event/world/environment object and triggers in the game for Alpha, Beta and Final. This “threshold” point varies from game to game, the tester is expected to communicate the“bare bone” testing requirement with the Producer and/or Project Manager.
Well, as you can see, Testing Games is not that simple. There is a lot, and when I say a LOT, I mean a LOT of things that you have to read and study about. However, just reading and studying is not enought to be a good Game Tester. You have to deal with the real thing and get into the market – as hard as that can be.
Wait for the second part of this text and hope someone out there can understand and use this at work!