Category: Bugs

How To Write A Game Bug Report

On By Ricardo CristofoliniIn BugsLeave a comment

Hello!

Once more, Tom Sloper gives us an opportunity to enjoy one more of his texts and experience as below.

“How To Write a Game Bug Report” is one of the many information that you can find at www.sloperama.com

Enjoy the reading!

How to Write a Bug Report

[…] In this article, my goal is to teach you how to write a good bug report. I teach video game quality assurance at the University of Southern California, one of the top game schools in the world. I’ve read a lot of bug reports in my time, and written a lot of them too. If you ever want to get a QA job at a game company, you may need to write a bug report (also called a “bug”) as a test of your communication skills.

The Bug Database

QA departments use an online database for bug reporting. There are several different tools used for this: DevTrack, Mantis, JIRA, FogBugz, to name just a few. When a tester encounters a problem in the game he (or she) is testing, s/he then writes a report of the problem in the database. More about that in a bit (that being the main focus of this article).

 

The Bug Process

Once some new bugs have been written, the lead tester (the senior tester who leads the test effort on the game) filters the bugs. That means that the lead tester reads the bugs to make sure they’re “good” bugs. A “good” bug is not a duplicate of a bug already written by someone else, it describes a real problem worth looking into, and it contains enough information so that the developer (more on what this means below) can see the problem and fix it.

The game’s producer or project manager also filters the bugs. It wouldn’t do for bad bugs to be reported to the development team. Bad bugs waste time.

Once the bugs have been filtered, they are released to the development team. Development leads (the lead artist, the lead programmer, the lead designer) read the bugs appropriate to their own areas of responsibility and assign them to individual developers to fix. So if it’s an art bug, the lead artist knows which artist is responsible for that art asset, so the lead artist assigns the bug to that artist. If it’s a programming bug, the lead programmer can probably tell which programmer wrote the code that needs to be fixed.

The developer (the person to whom the bug was assigned, be s/he an artist, designer, programmer) is notified that a bug has been assigned. Then the developer opens the database and reads the bug. Assuming the bug is well-written enough that the developer understands and accepts the validity of the problem, the developer then fixes the bug, and clicks the “Fixed” selection in the database.

If the developer reads the bug and has a question that needs to be answered in order to fully understand the problem, the developer can post the question directly in the bug and send it back to the tester. Once the tester answers the question, the bug bounces back into the developer’s court. This little game of badminton continues until the developer has enough information and can fix the bug. Hopefully, the tester provides enough information in the initial report so that no back-and-forth is even necessary.

Once a bug has been fixed, the bug goes back to the tester who wrote it for retesting. If the tester determines that the bug is indeed fixed, he or she clicks the “Verified Fixed” selection, and the bug is on its way to being “Closed.” If the tester determines that there is still a problem, he or she “Reopens” the bug, and it goes back to the developer to try again to fix.

There’s more to it than that, but hopefully you get the idea. The database is a communication tool that the testers and developers use to track and resolve issues with the game.

How To Write A Good Bug

I can hear voices in the distance, crying “Finally!!” Well, I figured it was helpful to set the stage first. Good to have the context, right?

A bug contains the following five main parts:

  1. Summary line (short, sweet, summation of the problem)
  2. The Actual Result (a description of what happened; the problem)
  3. The Expected Result (what you thought should happen instead of the problem)
  4. Steps to Replicate (how someone else can make the problem happen too)
  5. Severity (how bad is the problem)

Let’s examine those in detail.

2. The Actual Result

Writing in first person point of view, describe the problem. Provide enough information pertinent to the problem so that anyone reading the bug report will understand what happened. Write in complete sentences. Use proper punctuation, grammar, syntax, and spelling.

Example:

I was playing the Dodgers versus the Zombies, at Pebbled on Downs. I was at bat, and I hit a high fly ball. The ball flew high and long, but it never came down. Instead, the ball got stuck in my onscreen score. The ball flew right into the O and stayed there.

3. The Expected Result

Now that you’ve described the problem, explain why it’s a problem. What did you expect to happen, instead of the thing that actually happened? And what is the basis of that expectation? It may seem to you that it’s a waste of time to have to explain why it’s a problem — maybe you think it’s self-evident that it’s a problem (that any idiot can plainly see why it’s a problem, and what any idiot would expect to happen instead). Well, it’s your job as a QA tester to communicate. So communicate! What seems self-evident to you may not be so self-evident to your reader. And that even extends to the basis for the expectation.

Example:

The onscreen score is an artificial construct; it’s not really a part of the physical world being represented in the game. The score remains stationary on the screen while the camera pans around the sports arena. The score doesn’t smack any sports fans in the head (the sports fans are not affected by the relative motion of the onscreen score), and the score doesn’t collide with structures in the sports arena. So it’s unexpected for the ball to interact with the onscreen score in any way.

Now it’s dead easy for anyone reading your bug to understand what you expected to happen (and why you expected that to happen).

4. Steps To Replicate

Great, so you found a bug and you’re proud of yourself for spotting it. But if the developer can’t make the problem happen, how can he or she fix it? You have to tell the developer how to recreate (replicate) the problem. Writing in second person point of view, give the developer complete instructions about how to make the problem happen, so he can see it for himself. And to really communicate thoroughly, conclude with an observation step. Tell the developer what to look for, and what he will see, when he has performed your steps.

Example:

The problem has only been observed when playing in Pebbledon Downs, with the Dodgers and the Zombies. Your onscreen score has to be zero, and you have to aim for the zero. It can be tricky to make this problem happen, because it requires a good strong hit, a fly ball, and a trajectory that intersects perfectly with the middle of the O in the score. But I was able to make it happen a second time by simply being persistent. I recommend using the practice mode (that way the test is not prolonged by inning changes, vampire attacks, and timeouts). Using the left stick, position the batter’s feet so that a line drawn across both shoetips points directly towards the O. Press the right trigger just as the ball crosses the 5-point line, and hit the ball, then watch as it flies. When the ball flies into the O, observe that it gets stuck in the O.

See how that’s written in second person (I told the developer what he has to do). And notice that in the observation step I told him what to look for.

5. Severity

Now that you’ve described the problem, why it’s a problem, and how to recreate the problem, you need to say how severe the problem is, and why you say it’s that severe. You can’t just say “it’s a bad bug and needs to be fixed.” You have to communicate the reason why it’s bad. And you should use the A-B-C-D scale […]. (Remember: A is worst, B is bad, C is normal, D is minor.)

Example:

When the problem happens, it’s quite severe, because the ball is stuck and the game can not continue. The only way out of it (other than rebooting) is to hit Start, then Quit the game. But the problem happens only on those extremely rare occasions when the ball perfectly enters the O. So I’m assigning this a severity of C.

1. Summary

I’ll bet you thought I made a mistake when I didn’t list this above! The reason I put this last is that the best time to write the summary is after you’ve written the rest of the report. A summary needs to be short, and it needs to summarize the problem you’re reporting.

Example:

Ball gets stuck in the O

A bug report summary is a lot like the subject line of an email. It has to be short, and it has to give an idea of what the email is about (in this case, what the bug is). You’ll find that it’s easier to write the most concise summation of the problem after you’ve already written the full bug description.

Some Common Mistakes

* Non-summing summary line. “Found a bug.” No kidding! Like all these other bugs in the database aren’t bugs that somebody found? That’s like sending an email with the subject line, “Hey” – or “I wrote an email” – or “From me.” Sum up the problem! Say what the problem is. Just like when you write an email, you give the recipient an idea what the email is about before he or she opens it.

* Too-long summary line. “The slithy toves gyred and gimbled in the wabe when the borogoves were all mimsy and the mome raths outgrabe.” Dude, just say “The slithy toves gyred and gimbled.” That condenses the essence of the problem. You can give us the details about all the excessively mimsy borogoves in the body of the report.

* Tester as designer. “The slithy toves need to have a 5-second cooldown after gyring so they don’t gimble too soon.” No, don’t tell the developer how to fix it. Just say what the problem is. It’s the designer’s job to figure out what’s the best way to balance the slithy toves.

* Not giving step by step instructions. Tell the developer what to do, so he can see the problem himself. Don’t just say “I did this, then I did that.” Tell him, “do this, then do that.” Give step-by-step instructions, in detail, in the second person.

* Unclear basis for an expectation. “Usually, pressing X causes the door to open.” What do you mean, “usually”? Do you mean that’s how one opens doors in other games? Do you mean that’s how one opens other doors in this game? Do you mean doors in your home open when you press an X button? What does “usually” mean?? Be specific, dude!

* Confusing “player” with “player character.” The player is the human who’s holding the game controller. That digital being on the screen is a character. Don’t use the terms interchangeably.

* Wishy-washy observation step. “Then watch to see if the problem happens or not.” Wrong. Tell the developer he will see the problem. Tell him to observe that it does happen.

* Inappropriate use of the word “should,” as in: “After you follow these steps, you should see the bug happen.” Um, what? The bug should not happen — that’s why it’s a bug! If it was supposed to happen, then it wouldn’t be a bug. So you shouldn’t say “should” in this way. Just say “after following these steps, observe that the bug happens.”

So that’s how to write a bug report. It’s all about communication. A little extra attention to detail in writing your initial communication can save a lot of time later on. And this principle applies to a lot of written communications, in business and in life.

And that is Tom Sloper once more helping us understanding a little bit over the game biz QA.

Thank you Tom!

Advertisements

How To Write A Game Bug Report

On By Ricardo CristofoliniIn BugsLeave a comment

Hello!

Once more, Tom Sloper gives us an opportunity to enjoy one more of his texts and experience as below.

“How To Write a Game Bug Report” is one of the many information that you can find at www.sloperama.com

Enjoy the reading!
How to Write a Bug Report

[…] In this article, my goal is to teach you how to write a good bug report. I teach video game quality assurance at the University of Southern California, one of the top game schools in the world. I’ve read a lot of bug reports in my time, and written a lot of them too. If you ever want to get a QA job at a game company, you may need to write a bug report (also called a “bug”) as a test of your communication skills.
The Bug Database

QA departments use an online database for bug reporting. There are several different tools used for this: DevTrack, Mantis, JIRA, FogBugz, to name just a few. When a tester encounters a problem in the game he (or she) is testing, s/he then writes a report of the problem in the database. More about that in a bit (that being the main focus of this article).

 
 

The Bug Process

 

Once some new bugs have been written, the lead tester (the senior tester who leads the test effort on the game) filters the bugs. That means that the lead tester reads the bugs to make sure they’re “good” bugs. A “good” bug is not a duplicate of a bug already written by someone else, it describes a real problem worth looking into, and it contains enough information so that the developer (more on what this means below) can see the problem and fix it. 

The game’s producer or project manager also filters the bugs. It wouldn’t do for bad bugs to be reported to the development team. Bad bugs waste time. 

Once the bugs have been filtered, they are released to the development team. Development leads (the lead artist, the lead programmer, the lead designer) read the bugs appropriate to their own areas of responsibility and assign them to individual developers to fix. So if it’s an art bug, the lead artist knows which artist is responsible for that art asset, so the lead artist assigns the bug to that artist. If it’s a programming bug, the lead programmer can probably tell which programmer wrote the code that needs to be fixed. 

The developer (the person to whom the bug was assigned, be s/he an artist, designer, programmer) is notified that a bug has been assigned. Then the developer opens the database and reads the bug. Assuming the bug is well-written enough that the developer understands and accepts the validity of the problem, the developer then fixes the bug, and clicks the “Fixed” selection in the database. 

If the developer reads the bug and has a question that needs to be answered in order to fully understand the problem, the developer can post the question directly in the bug and send it back to the tester. Once the tester answers the question, the bug bounces back into the developer’s court. This little game of badminton continues until the developer has enough information and can fix the bug. Hopefully, the tester provides enough information in the initial report so that no back-and-forth is even necessary. 

Once a bug has been fixed, the bug goes back to the tester who wrote it for retesting. If the tester determines that the bug is indeed fixed, he or she clicks the “Verified Fixed” selection, and the bug is on its way to being “Closed.” If the tester determines that there is still a problem, he or she “Reopens” the bug, and it goes back to the developer to try again to fix. 

There’s more to it than that, but hopefully you get the idea. The database is a communication tool that the testers and developers use to track and resolve issues with the game. 

How To Write A Good Bug

 

I can hear voices in the distance, crying “Finally!!” Well, I figured it was helpful to set the stage first. Good to have the context, right? 

A bug contains the following five main parts:

  1. Summary line (short, sweet, summation of the problem)
  2. The Actual Result (a description of what happened; the problem)
  3. The Expected Result (what you thought should happen instead of the problem)
  4. Steps to Replicate (how someone else can make the problem happen too)
  5. Severity (how bad is the problem)

Let’s examine those in detail. 

2. The Actual Result

 

Writing in first person point of view, describe the problem. Provide enough information pertinent to the problem so that anyone reading the bug report will understand what happened. Write in complete sentences. Use proper punctuation, grammar, syntax, and spelling. 

Example: 

I was playing the Dodgers versus the Zombies, at Pebbled on Downs. I was at bat, and I hit a high fly ball. The ball flew high and long, but it never came down. Instead, the ball got stuck in my onscreen score. The ball flew right into the O and stayed there.

 

3. The Expected Result

 

Now that you’ve described the problem, explain why it’s a problem. What did you expect to happen, instead of the thing that actually happened? And what is the basis of that expectation? It may seem to you that it’s a waste of time to have to explain why it’s a problem — maybe you think it’s self-evident that it’s a problem (that any idiot can plainly see why it’s a problem, and what any idiot would expect to happen instead). Well, it’s your job as a QA tester to communicate. So communicate! What seems self-evident to you may not be so self-evident to your reader. And that even extends to the basis for the expectation. 

Example: 

The onscreen score is an artificial construct; it’s not really a part of the physical world being represented in the game. The score remains stationary on the screen while the camera pans around the sports arena. The score doesn’t smack any sports fans in the head (the sports fans are not affected by the relative motion of the onscreen score), and the score doesn’t collide with structures in the sports arena. So it’s unexpected for the ball to interact with the onscreen score in any way. 

Now it’s dead easy for anyone reading your bug to understand what you expected to happen (and why you expected that to happen). 

4. Steps To Replicate

 

Great, so you found a bug and you’re proud of yourself for spotting it. But if the developer can’t make the problem happen, how can he or she fix it? You have to tell the developer how to recreate (replicate) the problem. Writing in second person point of view, give the developer complete instructions about how to make the problem happen, so he can see it for himself. And to really communicate thoroughly, conclude with an observation step. Tell the developer what to look for, and what he will see, when he has performed your steps. 

Example: 

The problem has only been observed when playing in Pebbledon Downs, with the Dodgers and the Zombies. Your onscreen score has to be zero, and you have to aim for the zero. It can be tricky to make this problem happen, because it requires a good strong hit, a fly ball, and a trajectory that intersects perfectly with the middle of the O in the score. But I was able to make it happen a second time by simply being persistent. I recommend using the practice mode (that way the test is not prolonged by inning changes, vampire attacks, and timeouts). Using the left stick, position the batter’s feet so that a line drawn across both shoetips points directly towards the O. Press the right trigger just as the ball crosses the 5-point line, and hit the ball, then watch as it flies. When the ball flies into the O, observe that it gets stuck in the O. 

See how that’s written in second person (I told the developer what he has to do). And notice that in the observation step I told him what to look for. 

5. Severity

 

Now that you’ve described the problem, why it’s a problem, and how to recreate the problem, you need to say how severe the problem is, and why you say it’s that severe. You can’t just say “it’s a bad bug and needs to be fixed.” You have to communicate the reason why it’s bad. And you should use the A-B-C-D scale […]. (Remember: A is worst, B is bad, C is normal, D is minor.) 

Example: 

When the problem happens, it’s quite severe, because the ball is stuck and the game can not continue. The only way out of it (other than rebooting) is to hit Start, then Quit the game. But the problem happens only on those extremely rare occasions when the ball perfectly enters the O. So I’m assigning this a severity of C. 

1. Summary

 

I’ll bet you thought I made a mistake when I didn’t list this above! The reason I put this last is that the best time to write the summary is after you’ve written the rest of the report. A summary needs to be short, and it needs to summarize the problem you’re reporting. 

Example: 

Ball gets stuck in the O 

A bug report summary is a lot like the subject line of an email. It has to be short, and it has to give an idea of what the email is about (in this case, what the bug is). You’ll find that it’s easier to write the most concise summation of the problem after you’ve already written the full bug description. 

Some Common Mistakes

 

* Non-summing summary line. “Found a bug.” No kidding! Like all these other bugs in the database aren’t bugs that somebody found? That’s like sending an email with the subject line, “Hey” – or “I wrote an email” – or “From me.” Sum up the problem! Say what the problem is. Just like when you write an email, you give the recipient an idea what the email is about before he or she opens it. 

* Too-long summary line. “The slithy toves gyred and gimbled in the wabe when the borogoves were all mimsy and the mome raths outgrabe.” Dude, just say “The slithy toves gyred and gimbled.” That condenses the essence of the problem. You can give us the details about all the excessively mimsy borogoves in the body of the report. 

* Tester as designer. “The slithy toves need to have a 5-second cooldown after gyring so they don’t gimble too soon.” No, don’t tell the developer how to fix it. Just say what the problem is. It’s the designer’s job to figure out what’s the best way to balance the slithy toves. 

* Not giving step by step instructions. Tell the developer what to do, so he can see the problem himself. Don’t just say “I did this, then I did that.” Tell him, “do this, then do that.” Give step-by-step instructions, in detail, in the second person. 

* Unclear basis for an expectation. “Usually, pressing X causes the door to open.” What do you mean, “usually”? Do you mean that’s how one opens doors in other games? Do you mean that’s how one opens other doors in this game? Do you mean doors in your home open when you press an X button? What does “usually” mean?? Be specific, dude! 

* Confusing “player” with “player character.” The player is the human who’s holding the game controller. That digital being on the screen is a character. Don’t use the terms interchangeably. 

* Wishy-washy observation step. “Then watch to see if the problem happens or not.” Wrong. Tell the developer he willsee the problem. Tell him to observe that it does happen. 

* Inappropriate use of the word “should,” as in: “After you follow these steps, you should see the bug happen.” Um, what? The bug should not happen — that’s why it’s a bug! If it was supposed to happen, then it wouldn’t be a bug. So you shouldn’t say “should” in this way. Just say “after following these steps, observe that the bug happens.” 

So that’s how to write a bug report. It’s all about communication. A little extra attention to detail in writing your initial communication can save a lot of time later on. And this principle applies to a lot of written communications, in business and in life.
And that is Tom Sloper once more helping us understanding a little bit over the game biz QA.
Thank you Tom!