First, I'll define a couple of terms that I'll be using throughout the documentation
Products and Test Plans
Product: A product is something that will exist forever in TestLink. Products will undergo many different versions throughout their life times.
Test Plan: Test Plans are created when you'd like to execute test cases. The word Test Plan might be used synonomously with the word project throughout the documentation. Test plans can be made up of the test cases of one or many products
SmartLink: "SmartLink"ing is the process of importing (or moving) test cases from a product into a test plan. The test cases from many products can be imported into one test case.
Test Case Categorization
Database StructureComponent: Components are the parents of categories. Each component can have many categories
Category: Categories are the parents of test cases. Each category can have many test cases.
Test Case: Test cases are the fundamental piece of TestLink.
Directory | Sub-Directory | Purpose |
admin | This directory used to hold only information that was related to administrative functionality. Since the lines have started to blur between administrative and lead rights it now holds a mixture of the two | |
build | Build creation files | |
category | This is actually the category ownership and risk/priority pages | |
product | Product creation and importing | |
project |
Project creation, editing, and deletion | |
TC | This directory holds the files that handle updating files in a test plan | |
user | User administration files | |
color | Color picker javascript files from softcomplex | |
execution | All files related to test case execution | |
functions | Generic functions that are used across all pages | |
help | Holds all popup help files | |
htmlarea | Turns <textarea> fields into wysiwyg editors. Interactive Tools | |
icons | Directory where all icons are held | |
jtree | Javascript tree files from softcomplex | |
manage | The manage directory holds files that are all related to manipulating products. You can find the component/category/testcase editing pages here as well as the product-testplan import pages | |
keyword | Keyword viewing and editing pages | |
search | Search page files | |
This directory has been merged with the other print directory and is no longer used | ||
metrics | All files related to metrics | |
All files related to printing test cases |
Guest: A guest only has permission to view test cases and project metrics
Otester: A tester outside of the company that only has permissions to run tests allotted to them
Tester: A tester can view,create, edit, and delete test cases as well as execute them. Testers lack the permissions to manage test plans, manage products, create milestones, or assign rights
Lead: A lead has all of the same permissions as a Tester but also gains the ability to manage test plans, assign rights, create milestones, and manage keywords
Admin: An admin has all of the same permissions as a lead but gains the ability to manage products
Notes
-All users in the system will by default not have permissions to view newly created test plans (except for the test plan creator who can give themselves permissions at creation).
-Zero test plan permissions means that users will not see any Test Plans in the Test Plan dropdown box. In order to gain test plan permissions a user with lead or admin status must give them rights through the “Define user/project rights” link under “Test Plan Management”
If you view the table you will see rows for each of the permissions levels
(guest ,Otester, tester, lead, admin). The column next to the row holds all
of the different rights levels which will be defined below. These levels have
been determined for the use at my company but they are free to be edited. The
user table contains a foreign key that points to the appropriate permission
level in the rights table. Changing of these rights is handled by the user administration
link which is accessible by admins.
The second table deals with project rights (i.e. which users can see which projects).
This table is made up of a combined user id and project id. The main page (mainPage.php)
contains code which checks to see if the logged in user has the appropriate
permissions (set from the define project rights page) and then shows the allowed
projects. It is not recommended that this be hacked with.
Management
mgt_view_tc: Viewing component, category, and test case info
mgt_modify_tc: create,edit,delete,order, move, and copy components, categories,
and test cases
mgt_view_key: Viewing keywords
mgt_modify_key: Modifying keywords
mgt_view_product: The ability to view products
mgt_modify_product: create,edit,import,and delete products
Test Plan
tp_execute: Ability to execute test cases
tp_create_build: Ability to create builds
tp_metrics: Viewing metrics
tp_planning: risk/ownership, milestones, smartlink, update, create, edit,
delete test plans
tp_assign_rights: Assigning the rights to view projects
Management Rights
None
Test Plans Rights
tp_execute
tp_metrics
Management Rights
mgt_view_product
mgt_view_tc
mgt_view_key
Test Plan Rights
tp_metrics
Management Rights
mgt_view_tc
mgt_modify_tc (create,edit,delete,order)
mgt_view_key
mgt_view_product
Test Plan Rights
tp_execute
tp_metrics
Management Rights
mgt_view_tc
mgt_modify_tc (create,edit,delete,order)
mgt_view_key
mgt_modify_key
mgt_view_product
Test Plan Rights
tp_execute
tp_create_build
tp_metrics
tp_planning (risk/ownership, milestones, smartlink, update, create, edit, delete)
tp_assign_rights
Management Rights
mgt_view_tc
mgt_modify_tc (create,edit,delete,order)
mgt_view_key
mgt_modify_key
mgt_view_product
mgt_modify_product (new,edit,import,delete)
Test Plan Rights
tp_execute
tp_create_build
tp_metrics
tp_planning (risk/ownership, milestones, smartlink, update, create, edit, delete)
tp_assign_rights
TestLink allows users with administrator rights to create, edit, and delete users within the system. However, TestLink does not allow administrators to view or edit user's passwords. If users forget their passwords there is link on the login screen (actually it isnt there yet.. There's a bug on this) that will mail the user their password based upon their user name and the email address they entered.
Every user on the system will also be able to edit their own information via the "My User Info"
TestLink and Bug Tracking SystemsCurrently, TestLink only supports one bug tracking system, Bugzilla. However, I eventually plan on supporting other tracking systems like Mantis. If there are other systems that you'd like to see supported please file an enhancement bug in the SourceForge site.
Setting up bug system support
Bug system support can be setup in the functions/header.php file.
Simply, change the line that says to true
$bugzillaOn = false;
You'll also need to set these lines correctly
$bzHost= "pesky.good.com"; //bugzilla host
$bzUser= "dvanhorn"; //bugzilla user
$bzPasswd= "dvanhorn"; //bugzilla password
$bzName = "bugs"; //bugzilla default db
$bzUrl = "http://box.good.com/bugzilla/show_bug.cgi?id="; //this line creates the link to bugzilla
Execution and Metrics
If a bug system has been enabled users will be able to link executed test cases with bugs. These bugs will then be tracked in the metrics section. In these pages bugs will be crossed out if their status is resolved or verified.
Failed and blocked test cases:
This page shows all of the currently failing and blocked test cases. If a bug tracking system has been activated you will see all of the bugs filed against that test case.
Bugs per test case:
This page actually will sum up all of the bugs for test case in the test plan.
Viewing Test Cases from your bug tracking system
Backing Up DataThere is a file in the main TestLink directory named viewTC.php. When this file is loaded in the browser with a test case id passed in the url string the user will be shown a view of the test case and its results against all test plans it's part of.
ex: viewTC.php?tc=1456
At our company we have edited bugzilla's comment section so that anytime a user puts in the words "test case xxx" or "testcase xxx" it will link directly to TestLink's viewTC.php page.
Here are the changes we made in our Bugzilla installation (version 2.16.4)
For the changes, you need to update the file "<bugzilla_installation_directory>/globals.pl".
$things[$count++] = $item;
}
# Support finding "test case XXX" for TestLink
#
while ($text =~ s/\btest[\s]*case(\s|%\#)*([\d\.]+)/"##$count##"/ei) {
my $item = $&;
my $num = $2;
$item = "<a href=\"http://www.qagood.com/testlink/viewtc.php?tc=$2\">$item</a>";
$things[$count++] = $item;
}#
# Either a comment string or no comma and a compulsory #.
while ($text =~ s/\bbug(\s|%\#)*(\d+),?\s*comment\s*(\s|%\#)(\d+)/"##$count#
#"/ei) { .........
Creating Windows Backup
If your mySQL instance is running in a Windows environment, it is simply a matter of creating a batch file to call the mysqldump command. Once the batch file exists you can include it as a scheduled task on the mySQL server.
Example of a batch file for TestLink:
FOR /F "tokens=2" %%i in ('date /t') do set thedate=%%i
set mm=%thedate:~0,2%
set dd=%thedate:~3,2%
set yyyy=%thedate:~6,4%
if exist \\path\backup.sql move \\path\backup.sql \\path\backup.sql%mm%%dd%%yyyy%.sql
C:\mysql\bin\mysqldump -uroot -proot testlink > \\path\\backup.sql
Unix
If your mySQL instance is running in a Unix environment, a shell script can be created that invokes the mysqldump command. Once the shell script exists, you can establish a cron job to run the script periodically on the mySQL server.
InstallationTestLink relies upon MYSQL, PHP, and a webserver (apache, IIS) to run. If you have questions about how to set up these tools either visit their official sites or check out the setup document I created.
Note: My setup doc probably will become out of date quickly. The official apache,mysql, and php sites are still the best way to learn how to set up their services
Dowloading and Installing TestLink
Download either the Tar or Zip version of TestLink and uninstall it to your webserver's directory
Setup the header.php
The header.php file in the functions directory is the TestLink configuration file. This file sets up the database link, the base href, login page, logout path, and whether or not you have bugzilla
Setup the database schema
The first step is to open the schema file. The schema file is named TestLink.sql
Select all of the text in the schema file and copy it
To add the schema to your database you’ll need to open a cmd prompt, run mysql, and create a new database.
“start > run >cmd”
“cd c:\mysql\bin”
“mysqladmin create <your database name>”
“mysql”
“use <your database name>”
You then need to paste the sql your copied earlier into the command prompt. This will create all of the tables and add all of the permissions
Note: If the schema creation encounters any errors make sure that you copy the table and add it again. TestLink will not function properly without all of the tables.
Now that the database and the configuration file have been added you can log into TestLink.
Open up a browser window
Goto http://localhost/<your TestLink Directory>/login.php
Username: admin
Password: admin
Create yourself a user
DependenciesWhen you have entered TestLink click the “my user info” link in the upper right hand corner. Create yourself a user and off you go
Note: This document explains in general how to setup these tools. It is probable that they are outdate or that my instructions are wrong. All of the tools listed have setup documentation on their site.
OR use one these packaged deals (installation has not been tested with them i.e. use at your own risk)
Apache
One thing to note here is that you are going to be installing an older version of apache. I know that PHP works with the 2.0 version of apache I just have had problems installing it on 2.0 so I use 1.3.38
Apache is a very straight forward setup. Just run the installer. You can verify that apache has been setup correctly by opening internet explorer and typing “localhost”. This should bring up an Apache web page.
MYSQL
In general MYSQL is also a very simple installation. After you download the zip and run the installer you’ll need to start the server. The server can be started by going to the MYSQL directory (c:\mysql\bin) and running the winmysqladmin file. After the server starts you will need to enter a username and password
You can verify that MYSQL is working correctly by opening a command window, going to the MYSQL bin directory (c:\mysql\bin), and typing mysql. This should open the MYSQL command prompt.
PHP
Setup for LinuxPHP is a little trickier to install than Apache and MYSQL. The first thing you’ll need to do is unzip the PHP zip file into your c: directory. Don’t worry about specifying a folder because PHP will create its own directory.
After the files have unzipped, you will notice that there is another folder named PHP-4.3.2-Win32. Rename this directory simply to PHP. I find this a lot easier to manage.
Here’s comes the tricky part of installing PHP. You’ll find all of these steps in the PHP install instructions (c:\php\install.txt) but I think they are a little confusing and hard to find.
Copy the php.ini-dist file to your windows directory. Typically for Windows NT/2000/XP this directory is c:\windows
Rename the php.ini-dist file to php.ini
Edit the php.ini file. Scroll to the line that says extension_dir = ".\"
Change the line above to extension_dir = "c:\php\extensions"
Now edit the apache config file located at c:\program files\apache group\apache\config\htdoc.conf
At the very bottom of this file add the following lines
LoadModule php4_module c:/php/sapi/php4apache.dll
AddModule mod_php4.c
AddType application/x-httpd-php .phpNext you need to move the file c:\php\php4ts.dll to c:\program files\apache group\apache\
Now, stop and start your apache server
To test that PHP has been installed correctly create a file in c:\program files\apache group\apache\htdocs\ named phpinfo.php
In the file put the following line
<? Phpinfo(); ?>
Finally, open a browser and type http://localhost/phpinfo.php
If you have installed PHP correctly you will see your php installation config info
Session Setup
You may get some weird session errors if you don't have the session directory created. In the php.ini file there is a line that allows you to specify where the session data is saved. By default the directory is /tmp. You have the choice of either making this directory on your system or specifying and creating a different one.
Undefined Indexes
I've received a whole bunch of email from people about seeing weird errors when using TestLink. These errors look like this
Notice: Undefined index: submit in c:\inetpub\wwwroot\testlink\login.php at line 21
I'm not exactly sure why these errors are happening but I have found a work around that makes them go away.
It can be turned off in php.ini by changing the following line.
error_reporting = E_ALL & ~E_NOTICE
to
error_reporting = E_ERROR
Here are some instructions that a user sent me to that describes the setup process on Linux.
Linux version: RH9 x86
MySQL version: 3.23.58
Apache version: 2.0.49
PHP Version: 4.3.4
I used the MySQL that’s already installed on the system.
MySQL Setup:
1-Install/create the default DB
/usr/bin/mysql_install_db
2-Start mysql
/usr/bin/safe_mysqld –user=mysql &
I ran into problems here because I did the above as user “root”
And sql startup was failing. I had to go to /var/lib/mysql where all
The mysql DB were located and changed the file ownership
To mysql and group mysql
- chown mysql <file>
- chgrp mysql <file>
After I did that, mysql started up.
Apache 2.0 Setup.
1- Downloaded the apache tar.gz from the Apache website (www.apache.com) and save in /tmp
2- Unzip and Untar via “gunzip tar_file | tar xvf –“
3- This creates a directory called “/tmp/httpd-2.0.49”
4- Cd to this directory
5- I followed the instructions that is documented on the INSTALL document
6- I installed apache in /usr/local/apache2
7- Cd to “/usr/local/apache2/conf”
8- Edit the httpd.conf and set -> ServerRoot "/usr/local/apache2"
9- Restart the httpd daemon via “/usr/sbin/apachectl restart”
PHP Setup:
1 – Download the tar.gz file from the php website (www.php.net) and save in /tmp
2 – Unzip and Untar via “gunzip tar_file | tar vxf –“
3 – This creates a directory called “/tmp/php-4.3.4”
4 – Run “./configure –with-mysql –with-apxs=/usr/local/apache2/bin/apxs”
5 – Cd to /user/local/apache2/conf”
6 – Add the following to httpd.conf
# Example:
# LoadModule foo_module modules/mod_foo.so
LoadModule php4_module modules/libphp4.so
#
# AddType for php
AddType application/x-httpd-php .php .phtml
AddType application/x-httpd-php-source .phps
7 – cp /tmp/php-4.3.4/php.ini-dist to /usr/local/lib/php.ini
8 – Edit /usr/local/lib/php.ini and set
extension_dir = "/usr/local/lib/php/extensions"
9 – Restart the apache server
10- Create a file called phpinfo.php in /usr/local/apache2/htdocs.
11-Add the line below to the phpinfo.info file.
<? Phpinfo(); ?>
12-Open a browser and type URL of “http:/<system_name>/phpinfo.php
13-If you installed PHP correctly; you should see your php installation configuration information
Testlink Setup:
1 – Download the testlink.tar file in /usr/local/apache2
2 – Untar via “tar vxf testlink_tar”
3 – This will create a testlink directory
4 – cd to the testlink directory
5 – Edit the functions/header.php file and put the following information. I used the naming conventions below for my install.
I did not setup bugzilla yet.
$dbhost = "localhost"; //the host name for the server. Use either localhost,server name, or IP
$dbuser = "testlink"; //the mysql user
$dbpasswd = ""; //the mysql password
$dbname = "testlink_db"; //the name of the testlink database
$basehref = "http://<system_FQDN>/testlink/"; //Sets the basehref variable which is your testlink directory location
seen through your websever. So, if your testlink is installed in htdocs/testlink on your local host you would use the variable that
is there by default
//Important to note that a forward slash "/" is needed in the end of the basehref
$loginurl = "http://<system_FQDN>/testlink/login.php"; // where you go back to login
6 – Now setup the Testlink DB schema. Use DB name of testlink_db
- Run “mysqladmin create testlink_db”
- Run “mysql”. This brings you into mysql
mysql> use testlink_db;
From here, cut and paste SQL statements in /usr/local/apache2/testlink/testlink.sql to create the
Testlink tables and values.
7 – Cd to /usr/local/apache2/htdocs
8 – Create a link to the testlink directory via “ln –s /usr/local/apache2/testlink testlink”
This is the part that I did not do for the DB rights that caused all that php errors.
>mysql -u root
use mysql;
GRANT ALL PRIVILEGES ON testlink_db.* to testlink@localhost with GRANT
OPTION;
Now, launch a browser and point to http://<system_name>:80/tesltink/login.php
Login as admin/admin.
Login should be successful!
Creating New ProductsProducts are the cornerstone of TestLink. Products are releases of your company that may change their features and functionality over time but for the most part remain the same.
Things to note when creating a new product:
1. Products themselves can never be deleted from the system (unless of course we decide to change this). This is necessary because the deletion of products would either orphan lots of test plan cases or lead to their deletion
2. Test Plans represent the testing of a product at a certain point of time. What this means is that All Test Plans are created from Product test cases
3. There is no way to delete products but you can rename them
4. You cannot create products with the same name
5. TestLink has the ability to import your data into a product. Data is read in CSVs form and is explained further in the import section.
Importing New ProductsTestLink admins have the ability to import data, in the form of a CSV, into existing products. This means that users with already existing test cases don't have to input all of the data by hand.
Note: TestLink has no way to determine that the data being imported already exists. Therefore, if the data that is being imported already exists you will have two copies.
There is an example of a CSV for users to examine on the TestLink Project Page
Managing Components/Categories/Test CasesTestLink breaks down the test case structure into three levels components, categories, and test cases. These levels are persisted throughout the application.
Note:
Depending on the user's rights, TestLink gives the ability to do the following..
Create | |||
Edit | |||
Delete | |||
Copy | |||
Move | |||
Re-order | |||
Multiple Creation |
Rights and actions
Your view of the management screen depends on your TestLink rights. At this time any action that manipulates data (ex: creation, editing, moving, etc. ) all require the mgt_modify_tc right. See the rights documentation for more info.
KeywordsKeywords were created to gives users another level of depth when categorizing test cases.
Keyword Creation
At this time keywords can only be created by users with the mgt_modify_key rights. These rights are currently held only by leads. Once a keyword or grouping of keywords have been created users may assign them to test cases.
Assigning Keywords
Keywords may be assigned to test cases either from the assign keyword screen (in batch) or via the test case management (individually)
Importing, executing, and metrics
Users have the ability to import and execute groups of test cases in a test plan by keywords. Once these test cases are executed their status can be tracked in the metrics section
Creating New Test PlansTest plans are the basis for test case execution. Test plans are made up of test cases imported from products at a specific point of time.
Test plans can only be created by leads
Test plans may be created from other test plans. This allows users to create test plans from test cases that at a desired point in time. This may be necessary when creating a test plan for a patch.
In order for a user to see a test plan they must have the propper rights. Rights may be assigned (by leads) in the define User/Project Rights section. This is an important thing to remember when users tell you they can't see the project they are working on
Importing into Test Plans (SmartLink)Importing (SmartLinking) data is the process of adding data (test cases) from a product into a test plan.
-Data from multiple products can be imported into one test plan
-Data can be filtered by keywords
-Once data has been imported into a test plan it will be marked with checkmark
-If a test case has already been imported it will be ignored if it is imported again
BuildsBuilds are a specific release of software. Each project in a company is most likely made up of many different builds. In TestLink execution is made up of both builds and test cases.
If there are no builds created for a project the execution screen will not allow you to execute.
The metrics screen will also be completely blank.
Builds currently cannot be edited or deleted
MetricsThe metrics pages sum up the results of execution into reports. Metrics are broken down by both individual builds and across all builds.
View Project Status Across All Builds
This page shows you only the most current status of a test plan. For instance, you have test case 1 which was executed in builds 1,2, and 3.
Build | 1 | 2 | 3 |
Status | Pass | Fail | Blocked |
Since the most recent result of the test case is blocked the result on the "Across All Builds" page would be blocked. If a user would go and change the status of build 3 to something else or not run the current result would be fail.
View Status by an Individual Build
This report shows the detailed results for a particular build.
View The Overall Build Status
This report show a high level view of each build's result
View Status By Individual Test Cases
This report shows each test case's result for every build
Blocked/Failed Test Cases
These reports show all of the currently blocked or failing test cases
Total Bugs For Each Test Case
This report shows each test case with all of the bugs filed against it for the entire project
Email Test Plan Info
This page allows users to email the results of the entire project or for a particular build. It also allows users to email the status of a component
Assigning Risk/OwnershipTestLink gives users with lead rights the ability to assign ownership and priority to test cases.
General
-Risk/Ownership is done at the category level. TestLink currently does not allow users to assign risk or ownership at the individual test case level.
Risk, importance, and priority
-Risk levels are low, medium, high
-Importance levels are 3, 2, 1
-Users can rank the combinations of risk and importance (L1, L2, L3, M1, H2, M3, H1, H2, H3) as priority A,B,C
-Assigning risk, importance, ownership, and priority are all optional and will default to priority B in the metrics screen
Understanding Ownership
Ownership affects both the execution and metrics screens.
-In the execution screen users have the ability to sort the executible test cases by the ones they have ownership of
-In the main metrics screen there is a table that shows the remaining test cases by ownership. If there are no test case owners it defaults to none
Updating Test CasesIn TestLink products are meant to be constantly evolving while test plans are meant to be a snap shot in time. Occasionally, difference between the a product test case and a test plan test case will need to be merged.
In the "view modified test case" section, users with lead permissions have the ability to view all test cases that have been changed since initial import. Users will then have the ability to see the differences between the two test cases and choose to update them.
Note: when a test case is updated the only thing affected are the fields. All results that were previously entered will still remain the same.
Executing Test CasesExecution is the process of assigning a result (pass, fail, blocked) to a test case for a specific build.
Filtering Test Cases
This table allows the user to filter test cases before they are executed.
-Ownership:
Users can filter test cases by their owner. Ownership is determined at the category
level, is determined by leads, and can be changed at the Assign Risk and Ownership
page under metrics.
-Keyword:
Users can filter test cases by keyword. Keywords are set either using the Create/Edit/Delete
Test Cases or by the Assign Keywords To Multiple Cases. Keywords can only be
created, edited, or deleted by leads but may be assigned to test cases by testers.
-Build:
Users can filter test cases by builds. Builds are the basic component for how
test cases are tracked. Each test case may be run once and only once per build.
Builds can be created by leads using the Create New Build page.
-Result:
Users can filter test cases by results. Results are what happened to that test
case during a particular build. Test cases can pass, fail, be blocked, or not
be run.
-Most Current Result:
By default or if the "most current" checkbox is unchecked, the tree
will be sorted by the build that is chosen from the dropdown box. In this state
the tree will display the test cases status.
Ex: User selects build 2 from the dropdown box and doesn't check the "most current" checkbox. All test cases will be shown with their status from build 2. So, if test case 1 passed in build 2 it will be colored green.
If the user decideds to check the "most current" checkbox the tree will be colored by the test cases most recent result.
Ex: User selects build 2 from the dropdown box and this time checks the "most current" checkbox. All test cases will be shown with most current status. So, if test case 1 passed in build 3, even though the user has also selected build 2, it will be colored green.
Updated Test Case:
Users will see the American flag if the original version of the test case (on
the management side) has been updated. If users have the propper rights they
can go to the update/delete test case page either through clicking on the test
case number next to the flag or through the link on main page.
Note:It is not necessary for users to update test cases if there has been a change. They simply have the option of doing so if they wish.
Deleted Test Case:
Users will see the "x" symbol if the original version of the test
case (on the management side) has been deleted. If users have the propper rights
they can go to the update/delete test case page either through clicking on the
test case number next to the "x" or through the link on main page.
Test cases, categories, and components may be deleted from a test plan by users with lead permissions from the "delete test cases" screen. Deleting data may be useful when first creating a test plan since there are no results. However, Deleting test cases will cause the loss of all results associated with them. Therefore, extreme caution is recommended when using this functionality.
Archiving Test PlansUsers with lead permissions have the ability to "archive" test plans from the edit/delete test plan page. Technically, when a test plan is archived it is merely removed from the list of selectable test plans. Archived test plans also will disappear from the user/project rights page.
-Archiving of test plans is recommended when the UI begins to get crowded.
-Test plans can always be un-archived from the edit/delete test plan page
Deleting Test PlansTest plans may be deleted from the main page by users with lead permissions.
Deleting test plans permenently erases the test plan and all of its corresponding data (test cases, metrics, etc).
The deleting of data is a scary prospect and should be reserved to extreme cases. TestLink also allows users to archive test plans so that they no longer appear as a menu option.