140 lines
5.5 KiB
Markdown
140 lines
5.5 KiB
Markdown
Piwik comes with unit tests, integration tests, Javascript tests and Webtests.
|
|
This document briefly describes how to use and modify Piwik tests.
|
|
|
|
## Continuous Integration
|
|
|
|
We use Travis CI for our continuous integration server. It automatically runs our battery of thousands of unit/integration/screenshot tests
|
|
after each commit to our GIT repo. More information at the links:
|
|
|
|
* Piwik on Travis CI: https://travis-ci.org/piwik/piwik
|
|
* QA in Piwik: http://piwik.org/qa/
|
|
|
|
Each core Piwik developer is responsible to keep the build green. If a developer breaks the build, he will receive an email from Travis CI.
|
|
|
|
The next section explains how you can run the test suite on your own dev machine.
|
|
|
|
## How To Run Piwik Tests
|
|
|
|
To run tests, you must use the Git master. Tests files are not in the Piwik zip archive.
|
|
|
|
You can get the latest Git revision at: http://github.com/piwik/piwik
|
|
|
|
```
|
|
$ git clone https://github.com/piwik/piwik.git
|
|
```
|
|
|
|
Next install Composer which will lets you download the libraries used in Piwik:
|
|
```
|
|
$ curl -sS https://getcomposer.org/installer | php
|
|
$ php composer.phar install
|
|
```
|
|
|
|
To execute the tests:
|
|
|
|
* In your php.ini make sure you have the setting to show all errors:
|
|
`error_reporting = E_ALL | E_STRICT`
|
|
|
|
* Go to tests/index.html to see the tests homepage
|
|
and run the Integration tests via a visual UI, or run JS Tests
|
|
|
|
* Next you will need to install PHPUnit
|
|
|
|
## PHPUnit Tests
|
|
|
|
1. Install PHPUnit on your system
|
|
|
|
$ cd your/php/directory
|
|
$ sudo pear upgrade PEAR
|
|
$ pear config-set auto_discover 1
|
|
$ sudo pear install --alldeps pear.phpunit.de/PHPUnit
|
|
|
|
Doc at: http://www.phpunit.de/manual/current/en/installation.html
|
|
|
|
2. Configure PHPUnit: Copy the file `piwik/tests/PHPUnit/phpunit.xml.dist` to `phpunit.xml`.
|
|
In this file, you will find the following lines.
|
|
Please edit HTTP_HOST and REQUEST_URI to match the hostname and path of the Piwik files.
|
|
For example if your Piwik is available at http://localhost/path/to/piwik/ you would write:
|
|
|
|
<server name="HTTP_HOST" value="localhost"/>
|
|
<server name="REQUEST_URI" value="/path/to/piwik/"/>
|
|
|
|
3. Ensure the `[database_tests]` section in `piwik/config/config.php.ini` is set up correctly,
|
|
i.e. with the correct password to prevent the following error:
|
|
`SQLSTATE[28000] [1045] Access denied for user 'root'@'localhost' (using password: NO)`
|
|
|
|
|
|
4. Run the tests
|
|
|
|
$ cd /path/to/piwik/tests/PHPUnit
|
|
$ phpunit --group Core
|
|
$ phpunit --group Plugins
|
|
$ phpunit --group Integration
|
|
|
|
There are three main groups of tests: Core, Plugins and Integration
|
|
For example run `phpunit --group Core`
|
|
to run all Core Piwik tests.
|
|
|
|
5. Write more tests :)
|
|
See ["Writing Unit tests with PHPUnit"](http://www.phpunit.de/manual/current/en/writing-tests-for-phpunit.html)
|
|
|
|
|
|
|
|
## Integration Tests
|
|
|
|
Integration tests files are in `tests/PHPUnit/Integration/*Test.php`
|
|
|
|
Integration tests allow to test how major Piwik components interact together.
|
|
A test will typically generate hits to the Tracker (record visits and page views)
|
|
and then test all API responses and for each API output. It then checks that they match expected XML (or CSV, json, etc.).
|
|
If a test fails, you can compare the processed/ and expected/ directories in a graphical
|
|
text compare tool, such as WinMerge on Win, or MELD on Linux, to easily view changes between files.
|
|
|
|
For example using Meld, click on "Start new comparison", "Directory comparison",
|
|
in "Original" select "path/to/piwik/tests/PHPUnit/Integration/expected"
|
|
in "Mine" select "path/to/piwik/tests/PHPUnit/Integration/processed"
|
|
|
|
If changes are expected due to the code changes you make, simply copy the file from processed/ to
|
|
expected/, and test will then pass. Copying files is done easily using Meld (ALT+LEFT).
|
|
Otherwise, if you didn't expect to modify the API outputs, it might be that your changes are breaking some features unexpectedly.
|
|
|
|
### Scheduled Reports Tests
|
|
|
|
As part of our integration tests we generate the scheduled reports (in HTML, PDF & SMS).
|
|
Some of these scheduled reports contain PNG graphs. Depending on the system under test, generated images can differ.
|
|
Therefore, PNG graphs are only tested and compared against "expected" graphs, if the system under test has the same characteristics as the integration server.
|
|
The characteristics of the integration server are described in `IntegrationTestCase::canImagesBeIncludedInScheduledReports()`
|
|
|
|
## JavaScript Tests
|
|
|
|
piwik.js is unit tested and you can run the Javascript tests at: /piwik/tests/javascript/
|
|
|
|
## Testing Data
|
|
|
|
See [tests/README.testing-data.md](https://github.com/piwik/piwik/blob/master/tests/README.testing-data.md) to import testing data in Piwik.
|
|
|
|
## UI Screenshots Tests
|
|
|
|
See [tests/README.screenshots.md](https://github.com/piwik/piwik/blob/master/tests/README.screenshots.md)
|
|
|
|
## Profiling
|
|
|
|
See [tests/README.xhprof.md](https://github.com/piwik/piwik/blob/master/tests/README.xhprof.md) for help on how to profile Piwik with XHProf.
|
|
|
|
## Benchmarks
|
|
|
|
See [tests/PHPUnit/Benchmarks/README.md](https://github.com/piwik/piwik/blob/master/tests/PHPUnit/Benchmarks/README.md) to learn about running Benchmark tests.
|
|
|
|
## Build artifacts
|
|
|
|
You can retrieve the files generated during the build (the build artifacts) at [builds-artifacts.piwik.org](http://builds-artifacts.piwik.org/)
|
|
|
|
## Troubleshooting
|
|
|
|
See [tests/README.troubleshooting.md](https://github.com/piwik/piwik/blob/master/tests/README.troubleshooting.md) for troubleshooting the tests.
|
|
|
|
## Participate
|
|
|
|
You can help by improving existing tests, or identify some missing tests and implement them.
|
|
See http://piwik.org/participate/development-process
|
|
Please contact us at hello@piwik.org
|
|
|