Best Practices for testing Bodhi Packages in Fedora QA

When one attempts to do update-testing in Bodhi , there are few stuffs you check for.

**This one is mostly important for the people who are new.

I will take the example of two packages for F24 and explain the things one should look for.

Firstly, whenever you open up a Bodhi update please try looking for a test cases . The test case will generally give the tester a better idea of what the developer has built and wants to gets tested as an enhancement to the existing package or a totally new package.

Here is very good example(s)

                           

Update Testing for Live-usb Creator

Update Testing for Firefox 47

Now, if you are lucky enough to get the test cases , you can get inside the test cases , have a look at the "Setup" - which will tell you how to set up the system for testing , "How to test" - will help the tester with the steps to test it. "Expected Results" - will help testers with what are the things they are supposed to look for in order to give the package +1/-1 karma while they are executing the "How to test" instructions.
 

The test case page for Firefox Browse

Steps if you don't get the test case

Let's consider you are attempting to test packages when you don't see any Test case(s) . This becomes tough for you , I will be taking one example of how you can attempt to test of the package.

If the package is familiar to you and you know how and what the package does , you simply go and install the package and try performing activities which the package should perform . Example : if its "Firefox" you can end up browsing , adding addons and such.

But , In case you are pretty much unfamiliar with it , here is the next step you can use .

Check if the package is a "New" or its an "Enhancement"

Now , you can check out this snapshot as an example of a new package.
                         

Since , this is a new package , a good way will be check out the bug which will tell you in a bit details about what the package is supposed to do.If its not there , you can either do some research on the internet or contact the developer requesting the developer to explain in details about the package or write a "test case" of how to test the specified package.

SOP for writing the testcase goes like : https://fedoraproject.org/wiki/QA:SOP_test_case_creation and https://fedoraproject.org/wiki/QA:SOP_package_test_plan_creation
The stuff in https://fedoraproject.org/wiki/QA:SOP_package_test_plan_creation is important - the naming scheme described there is how Bodhi detects the package test cases to show up.
                                

The bug explaining the functionality of the package

Only , once that you have enough information about how the package is supposed to be tested and what the developer needs it to get tested , You should do ahead and test it .

Let's talk about Enhancement and bugfix packages

You can look for the "Notes for the Update" which will give you more idea about what were fixed or what are the new stuffs which were added in the packages before . Then do some research / or look for the test case (if attached) . A screenshot of how it should look like , will help !

Notes about the update and the Test cases for checking Bugfix or Enhancements

Checking for bugsWhen checking for bugs, you generally just have to open the bug and look out for something like "Steps to Reproduce". I will take an example of Live-usb creator for Fedora 24 which was a "enhancement" with a tons of bugfixes.

                                

The bug will explain the issue that the user got when s/he executed the package.

Run the steps in "Steps to reproduce" and match them with "Expected Results"

In the above mentioned screenshots , you can always check if the bug was fixed by executing the Steps to Reproduce which then becomes the "test case" and the "Expected Results" become the desired outcome which when satisfied , the tester can give a +1 karma.

Once that you have installed , You need to use it  and test out the package for a considerable amount of time before giving out karmas . Once, you are done , make sure you execute dnf remove <package_name> to ensure the completion of testing it . Here's the SOP to give out positive karma [https://fedoraproject.org/wiki/QA:Update_feedback_guidelines]

Thanks for the read!