TestLink UserManual


Table of Contents

1. General
1. Overall structure
1.1. Products and Test Plans
1.2. Test Case Categorization
1.3. Users
2. Test Specification
1. Creating Test Cases
2. Deleting Test Cases
3. Keywords
1. Creating new keywords
2. Keyword Creation
3. Assigning Keywords
4. Importing, executing, and metrics
4. Products
1. Overview
2. Creating new products
5. Test Plans
1. Creating a new Test Plan
2. Builds
3. Deleting TestPlans
6. Test Case Suite
1. Adding new Test Cases
2. Removing Test Cases from Test Case Suite
3. Priority
4. Ownership
7. Testcases
1. Creating new testcases
2. Execution
3. Metrics
8. User Administration
1. User Info
2. Permissions
2.1. How Rights are Implemented
2.2. Rights Definitions
2.2.1. Test Specification
2.2.2. Test Plan
2.3. Default Roles

Chapter 1. General

1. Overall structure

There are three cornerstones: Product, TestPlan and User. All other data are relations or attributes for this base. First, definition of a couple of terms that are used throughout the documentation.

1.1. Products and Test Plans

Product: A product is something that will exist forever in TestLink. Products will undergo many different versions throughout their life times. Product includes Test Specification with Test Cases and should be sorted via Keywords.

TestPlan: Test Plans are created when you'd like to execute test cases. Test plans can be made up of the test cases of one or many products. Test Plan includes Builds, Test Case Suite and Test Results.

1.2. Test Case Categorization

TestLink breaks down the test case structure into three levels components, categories, and test cases. These levels are persisted throughout the application.

Component: Components are the parents of categories. Each component can have many categories.

Category: Categories are the parents of test cases. Each category can have many test cases.

Test Case: Test cases are the fundamental piece of TestLink.

Test Specification: All components, categories and test cases within Product.

Test Case Suite: All components, categories and test cases within Test Plan.

1.3. Users

An User has a Role, that defines available TestLink features.

Chapter 2. Test Specification

1. Creating Test Cases

Tester must follow this structure: component, category and test case. At first you create component(s) for your product. You can fill description which can be printed then. Component includes categories. Category has the similar meaning but is second level of Test Specification and includes just Test Cases.

User can also copy or move Test Cases.

Test Cases has next parts:

  • Title: could include either short description or abbreviation (e.g. TL-USER-LOGIN)

  • Summary: should be really short; just for overview.

  • Steps: describe test scenario (input actions); can also include precondition and cleanup information here.

  • Expected results: describe checkpoints and expected behaviour a tested product or system.

2. Deleting Test Cases

Test cases, categories, and components may be deleted from a test plan by users with lead permissions from the "delete test cases" screen. Deleting data may be useful when first creating a test plan since there are no results. However, Deleting test cases will cause the loss of all results associated with them. Therefore, extreme caution is recommended when using this functionality.

Chapter 3. Keywords

1. Creating new keywords

Keywords were created to gives users another level of depth when categorizing test cases.

2. Keyword Creation

At this time keywords can only be created by users with the mgt_modify_key rights. These rights are currently held only by leads. Once a keyword or grouping of keywords have been created users may assign them to test cases

3. Assigning Keywords

Keywords may be assigned to test cases either from the assign keyword screen (in batch) or via the test case management (individually)

4. Importing, executing, and metrics

Users have the ability to import and execute groups of test cases in a test plan by keywords. Once these test cases are executed their status can be tracked in the metrics section

Chapter 4. Products

1. Overview

Products are the cornerstone of TestLink. Products are releases of your company that may change their features and functionality over time but for the most part remain the same.

2. Creating new products

Create a new product is admin right. You cannot create products with the same name. You can assign a color of backgroung to product for a better lucidity.

Things to note when creating a new product:

  • Products themselves is not recommended to delete from the system, because the deletion of products would either orphan lots of test plan cases or lead to their deletion.

  • Test Plans represent the testing of a product at a certain point of time. What this means is that All Test Plans are created from Product test cases.

  • TestLink has the ability to import your data into a product. Data is read in CSVs form and is explained further in the import section.

Chapter 5. Test Plans

1. Creating a new Test Plan

Test plans are the basis for test case execution. Test plans are made up of test cases imported from products at a specific point of time. Test plans can only be created by leads Test plans may be created from other test plans. This allows users to create test plans from test cases that at a desired point in time. This may be necessary when creating a test plan for a patch. In order for a user to see a test plan they must have the propper rights. Rights may be assigned (by leads) in the define User/Project Rights section. This is an important thing to remember when users tell you they can't see the project they are working on

2. Builds

Builds are a specific release of software. Each project in a company is most likely made up of many different builds. In TestLink execution is made up of both builds and test cases. If there are no builds created for a project the execution screen will not allow you to execute. The metrics screen will also be completely blank. Builds currently cannot be edited or deleted.

3. Deleting TestPlans

Test plans may be deleted from the main page by users with lead permissions. Deleting test plans permanently erases the test plan and all of its corresponding data (test cases, results, etc). The deleting of data is a scary prospect and should be reserved to extreme cases. TestLink also allows users to deactivate test plans so that they no longer appear as a menu option.

Chapter 6. Test Case Suite

1. Adding new Test Cases

Test Case Suite is set of test cases which are defined to be run within Test Plan. Test case Suite is created via Add Test Cases from Test Specification to Test Plan. Test cases are added including Steps and Expected result. So, you must use 'Update modified Test Cases' page to update test scenario (version of test case).

Data from multiple products can be added into one test plan. Data can be filtered by keywords.

Once data has been imported into a test plan it will be marked with checkmark. If a test case has already been imported it will be ignored if it is imported again.

2. Removing Test Cases from Test Case Suite

Test cases, categories, and components may be deleted from a test plan by users with Leader permissions from the "Remove test cases" page. Deleting data may be useful when first creating a test plan since there are no results. However, Deleting test cases will cause the loss of all results associated with them. Therefore, extreme caution is recommended when using this functionality.

3. Priority

TestLink gives users with Leader rights the ability to assign ownership and priority to test cases. General Risk/Ownership is done at the category level. TestLink currently does not allow users to assign risk or ownership at the individual test case level.

Risk levels are low, medium, high and Importance levels are 3, 2, 1. Users can rank the combinations of risk and importance (L1, L2, L3, M1, H2, M3, H1, H2, H3) as priority A,B,C.

Assigning risk, importance, ownership, and priority are all optional and will default to priority B in the metrics screen

4. Ownership

Ownership affects both the execution and metrics screens. In the execution screen users have the ability to sort the executable test cases by the ones they have ownership of. In the main metrics screen there is a table that shows the remaining test cases by ownership. If there are no test case owners it defaults to none.

Chapter 7. Testcases

1. Creating new testcases

2. Execution

Execution is the process of assigning a result (pass, fail, blocked) to a test case for a specific build.

Filtering Test Cases

This table allows the user to filter test cases before they are executed.

Ownership: Users can filter test cases by their owner. Ownership is determined at the category level, is determined by leads, and can be changed at the Assign Risk and Ownership page under metrics.

Keyword: Users can filter test cases by keyword. Keywords are set either using the Create/Edit/Delete Test Cases or by the Assign Keywords To Multiple Cases. Keywords can only be created, edited, or deleted by leads but may be assigned to test cases by testers.

Build: Users can filter test cases by builds. Builds are the basic component for how test cases are tracked. Each test case may be run once and only once per build. Builds can be created by leads using the Create New Build page.

Result: Users can filter test cases by results. Results are what happened to that test case during a particular build. Test cases can pass, fail, be blocked, or not be run.

Most Current Result: By default or if the "most current" checkbox is unchecked, the tree will be sorted by the build that is chosen from the dropdown box. In this state the tree will display the test cases status. Ex: User selects build 2 from the dropdown box and doesn't check the "most current" checkbox. All test cases will be shown with their status from build 2. So, if test case 1 passed in build 2 it will be colored green.

If the user decideds to check the "most current" checkbox the tree will be colored by the test cases most recent result.

Ex: User selects build 2 from the dropdown box and this time checks the "most current" checkbox. All test cases will be shown with most current status. So, if test case 1 passed in build 3, even though the user has also selected build 2, it will be colored green.

Updated Test Case: Users will see the American flag if the original version of the test case (on the management side) has been updated. If users have the proper rights they can go to the update/delete test case page either through clicking on the test case number next to the flag or through the link on main page. Note:It is not necessary for users to update test cases if there has been a change. They simply have the option of doing so if they wish.

Deleted Test Case: Users will see the "x" symbol if the original version of the test case (on the management side) has been deleted. If users have the propper rights they can go to the update/delete test case page either through clicking on the test case number next to the "x" or through the link on main page.

3. Metrics

The metrics pages sum up the results of execution into reports. Metrics are broken down by both individual builds and across all builds.

View Project Status Across All Builds This page shows you only the most current status of a test plan. For instance, you have test case 1 which was executed in builds 1,2, and 3. Build 1 2 3 Status Pass Fail Blocked Since the most recent result of the test case is blocked the result on the "Across All Builds" page would be blocked. If a user would go and change the status of build 3 to something else or not run the current result would be fail.

View Status by an Individual Build This report shows the detailed results for a particular build.

View The Overall Build Status This report show a high level view of each build's result

View Status By Individual Test Cases This report shows each test case's result for every build

Blocked/Failed Test Cases These reports show all of the currently blocked or failing test cases

Total Bugs For Each Test Case This report shows each test case with all of the bugs filed against it for the entire project

Email Test Plan Info This page allows users to email the results of the entire project or for a particular build. It also allows users to email the status of a component

Chapter 8. User Administration

1. User Info

TestLink allows users with administrator rights to create, edit, and delete users within the system. However, TestLink does not allow administrators to view or edit user's passwords. If users forget their passwords there is link on the login screen, that will mail the user their password based upon their user name and the email address they entered.

Every user on the system will also be able to edit their own information via the "My User Info" page.

2. Permissions

TestLink is built with 5 different permission levels built in. These permission levels are as follows:

  • Guest: A guest only has permission to view test cases and project metrics.

  • Tester: A tester outside of the company that only has permissions to run tests allotted to them. (initially otester)

  • Senior Tester: A tester can view,create, edit, and delete test cases as well as execute them. Testers lack the permissions to manage test plans, manage products, create milestones, or assign rights. (initially tester)

  • Leader: A lead has all of the same permissions as a Tester but also gains the ability to manage test plans, assign rights, create milestones, and manage keywords

  • Admin: An admin has all of the same permissions as a lead but gains the ability to manage products

Note

All users in the system will by default not have permissions to view newly created test plans (except for the test plan creator who can give themselves permissions at creation). Zero test plan permissions means that users will not see any Test Plans in the Test Plan dropdown box.

In order to gain test plan permissions a user with lead or admin status must give them rights through the “Define user/project rights” link under “Test Plan Management”.

2.1. How Rights are Implemented

In TestLink there are 2 tables that deal with rights. The first table is the rights table which defines the levels mentioned above

If you view the table you will see rows for each of the permissions levels (guest ,tester, senior tester, leader, admin). The column next to the row holds all of the different rights levels which will be defined below. These levels have been determined as standard for the use but they are free to be edited. The user table contains a foreign key that points to the appropriate permission level in the rights table. Changing of these rights is handled by the user administration link which is accessible by admins.

The second table deals with Test Plan rights (i.e. which users can see which projects). This table is made up of a combined user id and project id. The main page (mainPage.php) contains code which checks to see if the logged in user has the appropriate permissions (set from the define project rights page) and then shows the allowed projects. It is not recommended that this be hacked with.

2.2. Rights Definitions

Thera are listed keywords used for definition of role abilities:

2.2.1. Test Specification

mgt_view_tc: Viewing component, category, and test case info

mgt_modify_tc: create,edit,delete,order, move, and copy components, categories, and test cases

mgt_view_key: Viewing keywords mgt_modify_key: Modifying keywords

mgt_modify_product: create,edit,import,and delete products

Note

mgt_view_product was removed.

2.2.2. Test Plan

tp_execute: Ability to execute test cases

tp_create_build: Ability to create builds

tp_metrics: Viewing metrics

tp_planning: risk/ownership, milestones, smartlink, update, create, edit, delete test plans

tp_assign_rights: Assigning the rights to view projects

2.3. Default Roles

Tester: tp_execute,tp_metrics

Guest: mgt_view_tc, mgt_view_key, tp_metrics

Senior Tester: mgt_view_tc, mgt_modify_tc (create,edit,delete,order), mgt_view_key, mgt_view_product, tp_execute tp_metrics

Leader: mgt_view_tc, mgt_modify_tc (create,edit,delete,order), mgt_view_key, mgt_modify_key, tp_execute, tp_create_build, tp_metrics, tp_planning, tp_assign_rights

Admin: mgt_view_tc, mgt_modify_tc, mgt_view_key, mgt_modify_key, mgt_view_product, mgt_modify_product, tp_execute, tp_create_build ,tp_metrics, tp_planning, tp_assign_rights