coderkun/questlab
1
0
Fork 0
Code Issues 15 Pull requests Projects Releases Wiki Activity

merge

Browse source
This commit is contained in:
coderkun 2015-04-27 16:42:05 +02:00
commit 046a724272
4209 changed files with 1186656 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

265
www/analytics/LEGALNOTICE Normal file
View file

@ -0,0 +1,265 @@
COPYRIGHT
Piwik - Open Source Web Analytics
The software package is:
Copyright (C) 2013 Matthieu Aubry
Individual contributions, components, and libraries are copyright
of their respective authors.
SOFTWARE LICENSE
The free software license of Piwik is GNU General Public License v3
or later. A copy of GNU GPL v3 should have been included in this
software package in misc/gpl-3.0.txt.
TRADEMARK
Piwik (TM) is an internationally registered trademark.
The software license does not grant any rights under trademark
law for use of the trademark. Refer to http://piwik.org/trademark/
for up-to-date trademark licensing information.
*
* The software license applies to both the aggregate software and
* application-specific portions of the software.
*
* You may not remove this legal notice or modify the software
* in such a way that misrepresents the origin of the software.
*
CREDITS
The software consists of contributions made by many individuals.
Major contributors are listed in http://piwik.org/the-piwik-team/.
For detailed contribution history, refer to the source, tickets,
patches, and Git revision history, available at
http://dev.piwik.org/trac/
https://github.com/piwik/piwik
SEPARATELY LICENSED COMPONENTS AND LIBRARIES
The following components/libraries are distributed in this package,
and subject to their respective licenses.
Name: javascriptCode.tpl - tracking tag to embed in your web pages
Link: https://github.com/piwik/piwik/blob/master/core/Tracker/javascriptTag.tpl
License: Public Domain
Name: jquery.truncate
Link: https://github.com/piwik/piwik/blob/master/libs/jquery/truncate/
License: New BSD
Name: piwik.js - JavaScript tracker
Link: https://github.com/piwik/piwik/blob/master/js/piwik.js
License: New BSD
Name: PiwikTracker - server-side tracker (PHP)
Link: https://github.com/piwik/piwik/blob/master/libs/PiwikTracker/
License: New BSD
Name: UserAgentParser
Link: https://github.com/piwik/piwik/blob/master/libs/UserAgentParser/
License: New BSD
THIRD-PARTY COMPONENTS AND LIBRARIES
The following components/libraries are redistributed in this package,
and subject to their respective licenses.
Name: jqPlot
Link: http://www.jqplot.com/
License: Dual-licensed: MIT or GPL v2
Name: jQuery
Link: http://jquery.com/
License: Dual-licensed: MIT or GPL
Notes:
- GPL version not explicitly stated in source but GPL v2 is in git
- includes Sizzle.js - multi-licensed: MIT, New BSD, or GPL [v2]
Name: jQuery UI
Link: http://jqueryui.com/
License: Dual-licensed: MIT or GPL
Notes:
- GPL version not explicitly stated in source but GPL v2 is in git
Name: jquery.history
Link: http://tkyk.github.com/jquery-history-plugin/
License: MIT
Name: jquery.scrollTo
Link: http://plugins.jquery.com/project/ScrollTo
License: Dual licensed: MIT or GPL
Name: jquery Tooltip
Link: http://bassistance.de/jquery-plugins/jquery-plugin-tooltip/
License: Dual licensed: MIT or GPL
Name: jquery placeholder
Link: http://mths.be/placeholder
License: Dual licensed: MIT or GPL
Name: jquery smartbanner
Link: https://github.com/jasny/jquery.smartbanner
License: Dual licensed: MIT
Name: json2.js
Link: http://json.org/
License: Public domain
Notes:
- reference implementation
Name: jshrink
Link: https://github.com/tedivm/jshrink
License: BSD-3-Clause
Name: sparkline
Link: https//sourceforge.net/projects/sparkline/
License: Dual-licensed: New BSD or GPL v2
Name: sprintf
Link: http://www.diveintojavascript.com/projects/javascript-sprintf
License: New BSD
Name: upgrade.php
Link: http://upgradephp.berlios.de/
License: Public domain
Name: Archive Tar
Link: http://pear.php.net/package/Archive_Tar
License: New BSD
Name: Event Dispatcher (and Notification)
Link: http://pear.php.net/package/Event_Dispatcher/
License: New BSD
Name: HTML Common2
Link: http://pear.php.net/package/HTML_Common2/
License: New BSD
Name: HTML QuickForm2
Link: http://pear.php.net/package/HTML_QuickForm2/
License: New BSD
Name: HTML QuickForm2_Renderer_Smarty
Link: http://www.phcomp.co.uk/tmp/Smarty.phps
License: New BSD
Name: MaxMindGeoIP
Link: http://dev.maxmind.com/geoip/downloadable#PHP-7
License: LGPL
Name: PclZip
Link: http://www.phpconcept.net/pclzip/
License: LGPL
Notes:
- GPL version not explicitly stated but tarball contains LGPL v2.1
Name: PEAR (base system)
Link: http://pear.php.net/package/PEAR
License: New BSD
Name: PhpSecInfo
Link: http://phpsec.org/projects/phpsecinfo/
License: New BSD
Name: RankChecker
Link: http://www.getrank.org/free-pagerank-script
License: GPL
Name: Twig
Link: http://twig.sensiolabs.org/
License: BSD
Name: TCPDF
Link: http://sourceforge.net/projects/tcpdf
License: LGPL v3 or later
Name: Zend Framework
Link: http://www.zendframework.com/
License: New BSD
Name: pChart 2.1.3
Link: http://www.pchart.net
License: GPL v3
Name: Chroma.js
Link: https://github.com/gka/chroma.js
License: GPL v3
Name: qTip2 - Pretty powerful tooltips
Link: http://craigsworks.com/projects/qtip2/
License: GPL
Name: Kartograph.js
Link: http://kartograph.org/
License: LGPL v3 or later
Name: Raphaël - JavaScript Vector Library
Link: http://raphaeljs.com/
License: MIT
Name: lessphp
Link: http://leafo.net/lessphp
License: GPL3/MIT
Name: Symfony Console Component
Link: https://github.com/symfony/Console
License: MIT
THIRD-PARTY CONTENT
Name: FamFamFam icons - Mark James
Link: http://www.famfamfam.com/lab/icons/
License: CC BY 3.0
Name: Solar System icons - Dan Wiersema
Link: http://www.iconspedia.com/icon/neptune-4672.html
License: Free for non-commercial use
Notes:
- used in Piwik's ExampleUI plugin
Name: Wine project - tahoma.ttf font
Link: http://source.winehq.org/git/wine.git/blob_plain/HEAD:/fonts/tahoma.ttf
License: LGPL v2.1
Notes:
- used in ImageGraph plugin
Name: plugins/CorePluginsAdmin/images/themes.png
Link: https://www.iconfinder.com/icons/17022/colors_draw_paint_icon
License: Free for commercial use
Name: plugins/Feedback/angularjs/ratefeature/thumbs-down.png
Link: https://www.iconfinder.com/icons/216428/down_thumbs_icon
License: Creative Commons (Attribution-Share Alike 3.0 Unported)
Name: plugins/Feedback/angularjs/ratefeature/thumbs-up.png
Link: https://www.iconfinder.com/icons/216429/thumbs_up_icon
License: Creative Commons (Attribution-Share Alike 3.0 Unported)
Name: plugins/CorePluginsAdmin/images/plugins.png
Link: http://findicons.com/icon/94051/tools_wizard?id=396912
License: GNU/GPL
Name: plugins/Insights/images/idea.png
Link: https://www.iconfinder.com/icons/6074/brainstorm_bulb_idea_jabber_light_icon
License: GPL
By: Alessandro Rei - http://www.kde-look.org/usermanager/search.php?username=mentalrey
Notes:
- the "New BSD" license refers to either the "Modified BSD" and "Simplified BSD"
licenses (2- or 3-clause), which are GPL compatible.
- the "MIT" license is also referred to as the "X11" license
- icons for browsers, operating systems, browser plugins, search engines, and
and flags of countries are nominative use of third-party trademarks when
referring to the corresponding product or entity

92
www/analytics/README.md Normal file
View file

@ -0,0 +1,92 @@
# Piwik - piwik.org
## Description
Piwik is the leading Free/Libre open source Web Analytics platform.
Piwik is a full featured PHP MySQL software program that you download and install on your own webserver.
At the end of the five minute installation process you will be given a JavaScript code.
Simply copy and paste this tag on websites you wish to track and access your analytics reports in real time.
Piwik aims to be a Free software alternative to Google Analytics, and is already used on more than 1,000,000 websites.
## Mission Statement
> « To create, as a community, the leading international Free/Libre web analytics platform, providing access to all functionality through open components and open APIs. »
Or in short:
> « Liberate Web Analytics »
## License
Piwik is released under the GPL v3 (or later) license, see [misc/gpl-3.0.txt](misc/gpl-3.0.txt)
## Requirements
* PHP 5.3.2 or greater
* MySQL 4.1 or greater, and either MySQLi or PDO library must be enabled
* Piwik is OS / server independent
See http://piwik.org/docs/requirements/
## Install
* Upload piwik to your webserver
* Point your browser to the directory
* Follow the steps
* Add the given javascript code to your pages
* (You may also generate fake data to experiment, by enabling the plugin VisitorGenerator)
See http://piwik.org/docs/installation/
If you do not have a server, consider our Piwik Hosting partner: http://piwik.org/hosting/
## Changelog
For the list of all tickets closed in the current and past releases, see http://piwik.org/changelog/
## Participate!
We believe in liberating Web Analytics, providing a free platform for simple and advanced analytics. Piwik was built by dozens of people like you,
and we need your help to make Piwik better… Why not participate in a useful project today?
You will find pointers on how you can participate in Piwik at http://piwik.org/get-involved/
## Contact
http://piwik.org
hello@piwik.org
About us: http://piwik.org/the-piwik-team/
## More information
What makes Piwik unique from the competition:
* Real time web analytics reports: in Piwik, reports are by default generated in real time.
For high traffic websites, you can choose the frequency for reports to be processed.
* You own your web analytics data: since Piwik is installed on your server, the data is stored in your own database and you can get all the statistics
using the powerful Piwik Analytics API.
* Piwik is a Free Software which can easily be configured to respect your visitors privacy.
* Modern, easy to use User Interface: you can fully customize your dashboard, drag and drop widgets and more.
* Piwik features are built inside plugins: you can add new features and remove the ones you don’t need.
You can build your own web analytics plugins or hire a consultant to have your custom feature built in Piwik
* Vibrant international Open community of more than 200,000 active users (tracking even more websites!)
* Advanced Web Analytics capabilities such as Ecommerce Tracking, Goal tracking, Campaign tracking,
Custom Variables, Email Reports, Custom Segment Editor, Geo Location, Real time maps, and more!
Documentation and more info on http://piwik.org
## Code Status
The Piwik project uses an ever-expanding comprehensive set of thousands of unit tests and dozens of integration [tests](https://github.com/piwik/piwik/tree/master/tests),
running on the hosted distributed continuous integration platform Travis-CI.
Build status (master branch) [![Build Status](https://travis-ci.org/piwik/piwik.png?branch=master)](https://travis-ci.org/piwik/piwik) - Screenshot tests Build [![Build Status](https://travis-ci.org/piwik/piwik-ui-tests.png?branch=master)](https://travis-ci.org/piwik/piwik-ui-tests)

30
www/analytics/composer.json Normal file
View file

@ -0,0 +1,30 @@
{
"name": "piwik/piwik",
"type": "application",
"description": "Open Source Real Time Web Analytics Platform",
"keywords": ["piwik","web","analytics"],
"homepage": "http://piwik.org",
"license": "GPL-3.0+",
"authors": [
{
"name": "The Piwik Team",
"email": "hello@piwik.org",
"homepage": "http://piwik.org/the-piwik-team/"
}
],
"support": {
"forum": "http://forum.piwik.org/",
"issues": "http://dev.piwik.org/trac/roadmap",
"wiki": "http://dev.piwik.org/",
"source": "https://github.com/piwik/piwik"
},
"require": {
"php": ">=5.3.2",
"twig/twig": "1.*",
"leafo/lessphp": "~0.3",
"symfony/console": ">=v2.3.5",
"tedivm/jshrink": "v0.5.1",
"mustangostang/spyc": "0.5.*",
"piwik/device-detector": "*"
}
}

310
www/analytics/composer.lock generated Normal file
View file

@ -0,0 +1,310 @@
{
"_readme": [
"This file locks the dependencies of your project to a known state",
"Read more about it at http://getcomposer.org/doc/01-basic-usage.md#composer-lock-the-lock-file"
],
"hash": "69246584e6b57bfbc8d39799cd3b9213",
"packages": [
{
"name": "leafo/lessphp",
"version": "v0.4.0",
"source": {
"type": "git",
"url": "https://github.com/leafo/lessphp.git",
"reference": "51f3f06f0fe78a722dabfd14578444bdd078d9de"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/leafo/lessphp/zipball/51f3f06f0fe78a722dabfd14578444bdd078d9de",
"reference": "51f3f06f0fe78a722dabfd14578444bdd078d9de",
"shasum": ""
},
"type": "library",
"extra": {
"branch-alias": {
"dev-master": "0.3-dev"
}
},
"autoload": {
"classmap": [
"lessc.inc.php"
]
},
"notification-url": "https://packagist.org/downloads/",
"license": [
"MIT",
"GPL-3.0"
],
"authors": [
{
"name": "Leaf Corcoran",
"email": "leafot@gmail.com",
"homepage": "http://leafo.net"
}
],
"description": "lessphp is a compiler for LESS written in PHP.",
"homepage": "http://leafo.net/lessphp/",
"time": "2013-08-09 17:09:19"
},
{
"name": "mustangostang/spyc",
"version": "0.5.1",
"source": {
"type": "git",
"url": "https://github.com/mustangostang/spyc.git",
"reference": "dc4785b4d7227fd9905e086d499fb8abfadf9977"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/mustangostang/spyc/zipball/dc4785b4d7227fd9905e086d499fb8abfadf9977",
"reference": "dc4785b4d7227fd9905e086d499fb8abfadf9977",
"shasum": ""
},
"require": {
"php": ">=5.3.1"
},
"type": "library",
"extra": {
"branch-alias": {
"dev-master": "0.5.x-dev"
}
},
"autoload": {
"files": [
"Spyc.php"
]
},
"notification-url": "https://packagist.org/downloads/",
"license": [
"MIT License"
],
"authors": [
{
"name": "mustangostang",
"email": "vlad.andersen@gmail.com"
}
],
"description": "A simple YAML loader/dumper class for PHP",
"homepage": "https://github.com/mustangostang/spyc/",
"keywords": [
"spyc",
"yaml",
"yml"
],
"time": "2013-02-21 10:52:01"
},
{
"name": "piwik/device-detector",
"version": "1.0",
"source": {
"type": "git",
"url": "https://github.com/piwik/device-detector.git",
"reference": "ea7c5d8b76def0d8345a4eba59c5f98ec0109de6"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/piwik/device-detector/zipball/ea7c5d8b76def0d8345a4eba59c5f98ec0109de6",
"reference": "ea7c5d8b76def0d8345a4eba59c5f98ec0109de6",
"shasum": ""
},
"require": {
"mustangostang/spyc": "*",
"php": ">=5.3.1"
},
"type": "library",
"autoload": {
"files": [
"DeviceDetector.php"
]
},
"notification-url": "https://packagist.org/downloads/",
"license": [
"GPL-3.0+"
],
"authors": [
{
"name": "The Piwik Team",
"email": "hello@piwik.org",
"homepage": "http://piwik.org/the-piwik-team/"
}
],
"description": "The Universal Device Detection library, that parses User Agents and detects devices (desktop, tablet, mobile, tv, cars, console, etc.), and detects browsers, operating systems, devices, brands and models.",
"homepage": "http://piwik.org",
"keywords": [
"devicedetection",
"parser",
"useragent"
],
"time": "2014-04-03 08:59:48"
},
{
"name": "symfony/console",
"version": "v2.4.3",
"target-dir": "Symfony/Component/Console",
"source": {
"type": "git",
"url": "https://github.com/symfony/Console.git",
"reference": "ef20f1f58d7f693ee888353962bd2db336e3bbcb"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/symfony/Console/zipball/ef20f1f58d7f693ee888353962bd2db336e3bbcb",
"reference": "ef20f1f58d7f693ee888353962bd2db336e3bbcb",
"shasum": ""
},
"require": {
"php": ">=5.3.3"
},
"require-dev": {
"symfony/event-dispatcher": "~2.1"
},
"suggest": {
"symfony/event-dispatcher": ""
},
"type": "library",
"extra": {
"branch-alias": {
"dev-master": "2.4-dev"
}
},
"autoload": {
"psr-0": {
"Symfony\\Component\\Console\\": ""
}
},
"notification-url": "https://packagist.org/downloads/",
"license": [
"MIT"
],
"authors": [
{
"name": "Fabien Potencier",
"email": "fabien@symfony.com",
"homepage": "http://fabien.potencier.org",
"role": "Lead Developer"
},
{
"name": "Symfony Community",
"homepage": "http://symfony.com/contributors"
}
],
"description": "Symfony Console Component",
"homepage": "http://symfony.com",
"time": "2014-03-01 17:35:04"
},
{
"name": "tedivm/jshrink",
"version": "v0.5.1",
"source": {
"type": "git",
"url": "https://github.com/tedivm/JShrink.git",
"reference": "2d3f1a7d336ad54bdf2180732b806c768a791cbf"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/tedivm/JShrink/zipball/2d3f1a7d336ad54bdf2180732b806c768a791cbf",
"reference": "2d3f1a7d336ad54bdf2180732b806c768a791cbf",
"shasum": ""
},
"require": {
"php": ">=5.3.0"
},
"type": "library",
"autoload": {
"psr-0": {
"JShrink": "src/"
}
},
"notification-url": "https://packagist.org/downloads/",
"license": [
"BSD-3-Clause"
],
"authors": [
{
"name": "Robert Hafner",
"email": "tedivm@tedivm.com"
}
],
"description": "Javascript Minifier built in PHP",
"homepage": "http://github.com/tedivm/JShrink",
"keywords": [
"javascript",
"minifier"
],
"time": "2012-11-26 04:48:55"
},
{
"name": "twig/twig",
"version": "v1.15.1",
"source": {
"type": "git",
"url": "https://github.com/fabpot/Twig.git",
"reference": "1fb5784662f438d7d96a541e305e28b812e2eeed"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/fabpot/Twig/zipball/1fb5784662f438d7d96a541e305e28b812e2eeed",
"reference": "1fb5784662f438d7d96a541e305e28b812e2eeed",
"shasum": ""
},
"require": {
"php": ">=5.2.4"
},
"type": "library",
"extra": {
"branch-alias": {
"dev-master": "1.15-dev"
}
},
"autoload": {
"psr-0": {
"Twig_": "lib/"
}
},
"notification-url": "https://packagist.org/downloads/",
"license": [
"BSD-3-Clause"
],
"authors": [
{
"name": "Fabien Potencier",
"email": "fabien@symfony.com",
"homepage": "http://fabien.potencier.org",
"role": "Lead Developer"
},
{
"name": "Armin Ronacher",
"email": "armin.ronacher@active-4.com",
"role": "Project Founder"
},
{
"name": "Twig Team",
"homepage": "https://github.com/fabpot/Twig/graphs/contributors",
"role": "Contributors"
}
],
"description": "Twig, the flexible, fast, and secure template language for PHP",
"homepage": "http://twig.sensiolabs.org",
"keywords": [
"templating"
],
"time": "2014-02-13 10:19:29"
}
],
"packages-dev": [
],
"aliases": [
],
"minimum-stability": "stable",
"stability-flags": [
],
"platform": {
"php": ">=5.3.2"
},
"platform-dev": [
]
}

13
www/analytics/config/.htaccess Normal file
View file

@ -0,0 +1,13 @@
<Files "*">
<IfModule mod_access.c>
Deny from all
</IfModule>
<IfModule !mod_access_compat>
<IfModule mod_authz_host.c>
Deny from all
</IfModule>
</IfModule>
<IfModule mod_access_compat>
Deny from all
</IfModule>
</Files>

627
www/analytics/config/global.ini.php Normal file
View file

@ -0,0 +1,627 @@
; <?php exit; ?> DO NOT REMOVE THIS LINE
; If you want to change some of these default values, the best practise is to override
; them in your configuration file in config/config.ini.php. If you directly edit this file,
; you will lose your changes when you upgrade Piwik.
; For example if you want to override action_title_category_delimiter,
; edit config/config.ini.php and add the following:
; [General]
; action_title_category_delimiter = "-"
;--------
; WARNING - YOU SHOULD NOT EDIT THIS FILE DIRECTLY - Edit config.ini.php instead.
;--------
[database]
host =
username =
password =
dbname =
tables_prefix =
port = 3306
adapter = PDO_MYSQL
type = InnoDB
schema = Mysql
; if charset is set to utf8, Piwik will ensure that it is storing its data using UTF8 charset.
; it will add a sql query SET at each page view.
; Piwik should work correctly without this setting.
;charset = utf8
[database_tests]
host = localhost
username = root
password =
dbname = piwik_tests
tables_prefix = piwiktests_
port = 3306
adapter = PDO_MYSQL
type = InnoDB
schema = Mysql
[log]
; possible values for log: screen, database, file
log_writers[] = screen
; log level, everything logged w/ this level or one of greater severity
; will be logged. everything else will be ignored. possible values are:
; NONE, ERROR, WARN, INFO, DEBUG, VERBOSE
log_level = WARN
; if set to 1, only requests done in CLI mode (eg. the archive.php cron run) will be logged
; NOTE: log_only_when_debug_parameter will also be checked for
log_only_when_cli = 0
; if set to 1, only requests with "&debug" parameter will be logged
; NOTE: log_only_when_cli will also be checked for
log_only_when_debug_parameter = 0
; if configured to log in a file, log entries will be made to this file
logger_file_path = tmp/logs/piwik.log
[Debug]
; if set to 1, the archiving process will always be triggered, even if the archive has already been computed
; this is useful when making changes to the archiving code so we can force the archiving process
always_archive_data_period = 0;
always_archive_data_day = 0;
; Force archiving Custom date range (without re-archiving sub-periods used to process this date range)
always_archive_data_range = 0;
; if set to 1, all the SQL queries will be recorded by the profiler
; and a profiling summary will be printed at the end of the request
; NOTE: you must also set [log] log_writers[] = "screen" to enable the profiler to print on screen
enable_sql_profiler = 0
; if set to 1, a Piwik tracking code will be included in the Piwik UI footer and will track visits, pages, etc. to idsite = 1
; this is useful for Piwik developers as an easy way to create data in their local Piwik
track_visits_inside_piwik_ui = 0
; if set to 1, javascript files will be included individually and neither merged nor minified.
; this option must be set to 1 when adding, removing or modifying javascript files
disable_merged_assets = 0
; If set to 1, all requests to piwik.php will be forced to be 'new visitors'
tracker_always_new_visitor = 0
; Allow automatic upgrades to Beta or RC releases
allow_upgrades_to_beta = 0
[DebugTests]
; Set to 1 by default. If you set to 0, the standalone plugins (with their own git repositories)
; will not be loaded when executing tests.
enable_load_standalone_plugins_during_tests = 1
[General]
; the following settings control whether Unique Visitors will be processed for different period types.
; year and range periods are disabled by default, to ensure optimal performance for high traffic Piwik instances
; if you set it to 1 and want the Unique Visitors to be re-processed for reports in the past, drop all piwik_archive_* tables
; it is recommended to always enable Unique Visitors processing for 'day' periods
enable_processing_unique_visitors_day = 1
enable_processing_unique_visitors_week = 1
enable_processing_unique_visitors_month = 1
enable_processing_unique_visitors_year = 0
enable_processing_unique_visitors_range = 0
; when set to 1, all requests to Piwik will return a maintenance message without connecting to the DB
; this is useful when upgrading using the shell command, to prevent other users from accessing the UI while Upgrade is in progress
maintenance_mode = 0
; character used to automatically create categories in the Actions > Pages, Outlinks and Downloads reports
; for example a URL like "example.com/blog/development/first-post" will create
; the page first-post in the subcategory development which belongs to the blog category
action_url_category_delimiter = /
; similar to above, but this delimiter is only used for page titles in the Actions > Page titles report
action_title_category_delimiter = /
; the maximum url category depth to track. if this is set to 2, then a url such as
; "example.com/blog/development/first-post" would be treated as "example.com/blog/development".
; this setting is used mainly to limit the amount of data that is stored by Piwik.
action_category_level_limit = 10
; minimum number of websites to run autocompleter
autocomplete_min_sites = 5
; maximum number of websites showed in search results in autocompleter
site_selector_max_sites = 15
; if set to 1, shows sparklines (evolution graph) in 'All Websites' report (MultiSites plugin)
show_multisites_sparklines = 1
; number of websites to display per page in the All Websites dashboard
all_websites_website_per_page = 50
; if set to 0, the anonymous user will not be able to use the 'segments' parameter in the API request
; this is useful to prevent full DB access to the anonymous user, or to limit performance usage
anonymous_user_enable_use_segments_API = 1
; if browser trigger archiving is disabled, API requests with a &segment= parameter will still trigger archiving.
; You can force the browser archiving to be disabled in most cases by setting this setting to 1
; The only time that the browser will still trigger archiving is when requesting a custom date range that is not pre-processed yet
browser_archiving_disabled_enforce = 0
; By default, users can create Segments which are to be processed in Real-time.
; Setting this to 0 will force all newly created Custom Segments to be "Pre-processed (faster, requires archive.php cron)"
; This can be useful if you want to prevent users from adding much load on the server.
; Note: any existing Segment set to "processed in Real time", will still be set to Real-time.
; this will only affect custom segments added or modified after this setting is changed.
enable_create_realtime_segments = 1
; this action name is used when the URL ends with a slash /
; it is useful to have an actual string to write in the UI
action_default_name = index
; if you want all your users to use Piwik in only one language, disable the LanguagesManager
; plugin, and set this default_language (users won't see the language drop down)
default_language = en
; default number of elements in the datatable
datatable_default_limit = 10
; default number of rows returned in API responses
; this value is overwritten by the '# Rows to display' selector.
; if set to -1, a click on 'Export as' will export all rows independently of the current '# Rows to display'.
API_datatable_default_limit = 100
; When period=range, below the datatables, when user clicks on "export", the data will be aggregate of the range.
; Here you can specify the comma separated list of formats for which the data will be exported aggregated by day
; (ie. there will be a new "date" column). For example set to: "rss,tsv,csv"
datatable_export_range_as_day = "rss"
; This setting is overriden in the UI, under "User Settings".
; The date and period loaded by Piwik uses the defaults below. Possible values: yesterday, today.
default_day = yesterday
; Possible values: day, week, month, year.
default_period = day
; Time in seconds after which an archive will be computed again. This setting is used only for today's statistics.
; Defaults to 10 seconds so that by default, Piwik provides real time reporting.
; This setting is overriden in the UI, under "General Settings".
; This setting is only used if it hasn't been overriden via the UI yet, or if enable_general_settings_admin=0
time_before_today_archive_considered_outdated = 10
; This setting is overriden in the UI, under "General Settings".
; The default value is to allow browsers to trigger the Piwik archiving process.
; This setting is only used if it hasn't been overriden via the UI yet, or if enable_general_settings_admin=0
enable_browser_archiving_triggering = 1
; By default Piwik runs OPTIMIZE TABLE SQL queries to free spaces after deleting some data.
; If your Piwik tracks millions of pages, the OPTIMIZE TABLE queries might run for hours (seen in "SHOW FULL PROCESSLIST \g")
; so you can disable these special queries here:
enable_sql_optimize_queries = 1
; MySQL minimum required version
; note: timezone support added in 4.1.3
minimum_mysql_version = 4.1
; PostgreSQL minimum required version
minimum_pgsql_version = 8.3
; Minimum adviced memory limit in php.ini file (see memory_limit value)
minimum_memory_limit = 128
; Minimum memory limit enforced when archived via misc/cron/archive.php
minimum_memory_limit_when_archiving = 768
; Piwik will check that usernames and password have a minimum length, and will check that characters are "allowed"
; This can be disabled, if for example you wish to import an existing User database in Piwik and your rules are less restrictive
disable_checks_usernames_attributes = 0
; Piwik will use the configured hash algorithm where possible.
; For legacy data, fallback or non-security scenarios, we use md5.
hash_algorithm = whirlpool
; by default, Piwik uses PHP's built-in file-based session save handler with lock files.
; For clusters, use dbtable.
session_save_handler = files
; If set to 1, Piwik will automatically redirect all http:// requests to https://
; If SSL / https is not correctly configured on the server, this will break Piwik
; If you set this to 1, and your SSL configuration breaks later on, you can always edit this back to 0
; it is recommended for security reasons to always use Piwik over https
force_ssl = 0
; login cookie name
login_cookie_name = piwik_auth
; login cookie expiration (14 days)
login_cookie_expire = 1209600
; The path on the server in which the cookie will be available on.
; Defaults to empty. See spec in http://curl.haxx.se/rfc/cookie_spec.html
login_cookie_path =
; email address that appears as a Sender in the password recovery email
; if specified, {DOMAIN} will be replaced by the current Piwik domain
login_password_recovery_email_address = "password-recovery@{DOMAIN}"
; name that appears as a Sender in the password recovery email
login_password_recovery_email_name = Piwik
; By default when user logs out he is redirected to Piwik "homepage" usually the Login form.
; Uncomment the next line to set a URL to redirect the user to after he logs out of Piwik.
; login_logout_url = http://...
; Set to 1 to disable the framebuster on standard Non-widgets pages (a click-jacking countermeasure).
; Default is 0 (i.e., bust frames on all non Widget pages such as Login, API, Widgets, Email reports, etc.).
enable_framed_pages = 0
; Set to 1 to disable the framebuster on Admin pages (a click-jacking countermeasure).
; Default is 0 (i.e., bust frames on the Settings forms).
enable_framed_settings = 0
; language cookie name for session
language_cookie_name = piwik_lang
; standard email address displayed when sending emails
noreply_email_address = "noreply@{DOMAIN}"
; feedback email address;
; when testing, use your own email address or "nobody"
feedback_email_address = "feedback@piwik.org"
; during archiving, Piwik will limit the number of results recorded, for performance reasons
; maximum number of rows for any of the Referrers tables (keywords, search engines, campaigns, etc.)
datatable_archiving_maximum_rows_referrers = 1000
; maximum number of rows for any of the Referrers subtable (search engines by keyword, keyword by campaign, etc.)
datatable_archiving_maximum_rows_subtable_referrers = 50
; maximum number of rows for the Custom Variables names report
; Note: if the website is Ecommerce enabled, the two values below will be automatically set to 50000
datatable_archiving_maximum_rows_custom_variables = 1000
; maximum number of rows for the Custom Variables values reports
datatable_archiving_maximum_rows_subtable_custom_variables = 1000
; maximum number of rows for any of the Actions tables (pages, downloads, outlinks)
datatable_archiving_maximum_rows_actions = 500
; maximum number of rows for pages in categories (sub pages, when clicking on the + for a page category)
; note: should not exceed the display limit in Piwik\Actions\Controller::ACTIONS_REPORT_ROWS_DISPLAY
; because each subdirectory doesn't have paging at the bottom, so all data should be displayed if possible.
datatable_archiving_maximum_rows_subtable_actions = 100
; maximum number of rows for any of the Events tables (Categories, Actions, Names)
datatable_archiving_maximum_rows_events = 500
; maximum number of rows for sub-tables of the Events tables (eg. for the subtables Categories>Actions or Categories>Names).
datatable_archiving_maximum_rows_subtable_events = 100
; maximum number of rows for other tables (Providers, User settings configurations)
datatable_archiving_maximum_rows_standard = 500
; maximum number of rows to fetch from the database when archiving. if set to 0, no limit is used.
; this can be used to speed up the archiving process, but is only useful if you're site has a large
; amount of actions, referrers or custom variable name/value pairs.
archiving_ranking_query_row_limit = 50000
; maximum number of actions that is shown in the visitor log for each visitor
visitor_log_maximum_actions_per_visit = 500
; by default, the real time Live! widget will update every 5 seconds and refresh with new visits/actions/etc.
; you can change the timeout so the widget refreshes more often, or not as frequently
live_widget_refresh_after_seconds = 5
; by default, the Live! real time visitor count widget will check to see how many visitors your
; website received in the last 3 minutes. changing this value will change the number of minutes
; the widget looks in.
live_widget_visitor_count_last_minutes = 3
; In "All Websites" dashboard, when looking at today's reports (or a date range including today),
; the page will automatically refresh every 5 minutes. Set to 0 to disable automatic refresh
multisites_refresh_after_seconds = 300
; Set to 1 if you're using https on your Piwik server and Piwik can't detect it,
; e.g., a reverse proxy using https-to-http, or a web server that doesn't
; set the HTTPS environment variable.
assume_secure_protocol = 0
; List of proxy headers for client IP addresses
;
; CloudFlare (CF-Connecting-IP)
;proxy_client_headers[] = HTTP_CF_CONNECTING_IP
;
; ISP proxy (Client-IP)
;proxy_client_headers[] = HTTP_CLIENT_IP
;
; de facto standard (X-Forwarded-For)
;proxy_client_headers[] = HTTP_X_FORWARDED_FOR
; List of proxy headers for host IP addresses
;
; de facto standard (X-Forwarded-Host)
;proxy_host_headers[] = HTTP_X_FORWARDED_HOST
; List of proxy IP addresses (or IP address ranges) to skip (if present in the above headers).
; Generally, only required if there's more than one proxy between the visitor and the backend web server.
;
; Examples:
;proxy_ips[] = 204.93.240.*
;proxy_ips[] = 204.93.177.0/24
;proxy_ips[] = 199.27.128.0/21
;proxy_ips[] = 173.245.48.0/20
; Whether to enable trusted host checking. This can be disabled if you're running Piwik
; on several URLs and do not wish to constantly edit the trusted host list.
enable_trusted_host_check = 1
; List of trusted hosts (eg domain or subdomain names) when generating absolute URLs.
;
; Examples:
;trusted_hosts[] = example.com
;trusted_hosts[] = stats.example.com
; The release server is an essential part of the Piwik infrastructure/ecosystem
; to provide the latest software version.
latest_version_url = http://builds.piwik.org/latest.zip
; The API server is an essential part of the Piwik infrastructure/ecosystem to
; provide services to Piwik installations, e.g., getLatestVersion and
; subscribeNewsletter.
api_service_url = http://api.piwik.org
; When the ImageGraph plugin is activated, report metadata have an additional entry : 'imageGraphUrl'.
; This entry can be used to request a static graph for the requested report.
; When requesting report metadata with $period=range, Piwik needs to translate it to multiple periods for evolution graphs.
; eg. $period=range&date=previous10 becomes $period=day&date=previous10. Use this setting to override the $period value.
graphs_default_period_to_plot_when_period_range = day
; The Overlay plugin shows the Top X following pages, Top X downloads and Top X outlinks which followed
; a view of the current page. The value X can be set here.
overlay_following_pages_limit = 300
; With this option, you can disable the framed mode of the Overlay plugin. Use it if your website contains a framebuster.
overlay_disable_framed_mode = 0
; By default we check whether the Custom logo is writable or not, before we display the Custom logo file uploader
enable_custom_logo_check = 1
; If php is running in a chroot environment, when trying to import CSV files with createTableFromCSVFile(),
; Mysql will try to load the chrooted path (which is imcomplete). To prevent an error, here you can specify the
; absolute path to the chroot environment. eg. '/path/to/piwik/chrooted/'
absolute_chroot_path =
; In some rare cases it may be useful to explicitely tell Piwik not to use LOAD DATA INFILE
; This may for example be useful when doing Mysql AWS replication
enable_load_data_infile = 1
; By setting this option to 0, you can disable the Piwik marketplace. This is useful to prevent giving the Super user
; the access to disk and install custom PHP code (Piwik plugins).
enable_marketplace = 1
; By setting this option to 0:
; - links to Enable/Disable/Uninstall plugins will be hidden and disabled
; - links to Uninstall themes will be disabled (but user can still enable/disable themes)
; - as well as disabling plugins admin actions (such as "Upload new plugin"), setting this to 1 will have same effect as setting enable_marketplace=1
enable_plugins_admin = 1
; By setting this option to 0, you can prevent Super User from editing the Geolocation settings.
enable_geolocation_admin = 1
; By setting this option to 0, the old log data and old report data features will be hidden from the UI
; Note: log purging and old data purging still occurs, just the Super User cannot change the settings.
enable_delete_old_data_settings_admin = 1
; By setting this option to 0, the following settings will be hidden and disabled from being set in the UI:
; - "Archiving Settings"
; - "Update settings"
; - "Email server settings"
enable_general_settings_admin = 1
; By setting this option to 0, it will disable the "Auto update" feature
enable_auto_update = 1
; By setting this option to 0, no emails will be sent in case of an available core.
; If set to 0 it also disables the "sent plugin update emails" feature in general and the related setting in the UI.
enable_update_communication = 1
[Tracker]
; Piwik uses first party cookies by default. If set to 1,
; the visit ID cookie will be set on the Piwik server domain as well
; this is useful when you want to do cross websites analysis
use_third_party_id_cookie = 0
; If tracking does not work for you or you are stuck finding an issue, you might want to enable the tracker debug mode.
; Once enabled (set to 1) messages will be logged to all loggers defined in "[log] log_writers" config.
debug = 0
; There is a feature in the Tracking API that lets you create new visit at any given time, for example if you know that a different user/customer is using
; the app then you would want to tell Piwik to create a new visit (even though both users are using the same browser/computer).
; To prevent abuse and easy creation of fake visits, this feature requires admin token_auth by default
; If you wish to use this feature using the Javascript tracker, you can set the setting new_visit_api_requires_admin=0, and in Javascript write:
; _paq.push(['appendToTrackingUrl', 'new_visit=1']);
new_visit_api_requires_admin = 1
; This setting should only be set to 1 in an intranet setting, where most users have the same configuration (browsers, OS)
; and the same IP. If left to 0 in this setting, all visitors will be counted as one single visitor.
trust_visitors_cookies = 0
; name of the cookie used to store the visitor information
; This is used only if use_third_party_id_cookie = 1
cookie_name = _pk_uid
; by default, the Piwik tracking cookie expires in 2 years
; This is used only if use_third_party_id_cookie = 1
cookie_expire = 63072000
; The path on the server in which the cookie will be available on.
; Defaults to empty. See spec in http://curl.haxx.se/rfc/cookie_spec.html
; This is used for the Ignore cookie, and the third party cookie if use_third_party_id_cookie = 1
cookie_path =
; set to 0 if you want to stop tracking the visitors. Useful if you need to stop all the connections on the DB.
record_statistics = 1
; length of a visit in seconds. If a visitor comes back on the website visit_standard_length seconds
; after his last page view, it will be recorded as a new visit
visit_standard_length = 1800
; The window to look back for a previous visit by this current visitor. Defaults to visit_standard_length.
; If you are looking for higher accuracy of "returning visitors" metrics, you may set this value to 86400 or more.
; This is especially useful when you use the Tracking API where tracking Returning Visitors often depends on this setting.
; The value window_look_back_for_visitor is used only if it is set to greater than visit_standard_length
window_look_back_for_visitor = 0
; visitors that stay on the website and view only one page will be considered as time on site of 0 second
default_time_one_page_visit = 0
; if set to 1, Piwik attempts a "best guess" at the visitor's country of
; origin when the preferred language tag omits region information.
; The mapping is defined in core/DataFiles/LanguageToCountry.php,
enable_language_to_country_guess = 1
; When the misc/cron/archive.php cron hasn't been setup, we still need to regularly run some maintenance tasks.
; Visits to the Tracker will try to trigger Scheduled Tasks (eg. scheduled PDF/HTML reports by email).
; Scheduled tasks will only run if 'Enable Piwik Archiving from Browser' is enabled in the General Settings.
; Tasks run once every hour maximum, they might not run every hour if traffic is low.
; Set to 0 to disable Scheduled tasks completely.
scheduled_tasks_min_interval = 3600
; name of the cookie to ignore visits
ignore_visits_cookie_name = piwik_ignore
; Comma separated list of variable names that will be read to define a Campaign name, for example CPC campaign
; Example: If a visitor first visits 'index.php?piwik_campaign=Adwords-CPC' then it will be counted as a campaign referrer named 'Adwords-CPC'
; Includes by default the GA style campaign parameters
campaign_var_name = "pk_campaign,piwik_campaign,utm_campaign,utm_source,utm_medium"
; Comma separated list of variable names that will be read to track a Campaign Keyword
; Example: If a visitor first visits 'index.php?piwik_campaign=Adwords-CPC&piwik_kwd=My killer keyword' ;
; then it will be counted as a campaign referrer named 'Adwords-CPC' with the keyword 'My killer keyword'
; Includes by default the GA style campaign keyword parameter utm_term
campaign_keyword_var_name = "pk_kwd,piwik_kwd,pk_keyword,utm_term"
; maximum length of a Page Title or a Page URL recorded in the log_action.name table
page_maximum_length = 1024;
; Tracker cache files are the simple caching layer for Tracking.
; TTL: Time to live for cache files, in seconds. Default to 5 minutes.
tracker_cache_file_ttl = 300
; DO NOT USE THIS SETTING ON PUBLICLY AVAILABLE PIWIK SERVER
; !!! Security risk: if set to 0, it would allow anyone to push data to Piwik with custom dates in the past/future and even with fake IPs!
; When using the Tracking API, to override either the datetime and/or the visitor IP,
; token_auth with an "admin" access is required. If you set this setting to 0, the token_auth will not be required anymore.
; DO NOT USE THIS SETTING ON PUBLIC PIWIK SERVERS
tracking_requests_require_authentication = 1
[Segments]
; Reports with segmentation in API requests are processed in real time.
; On high traffic websites it is recommended to pre-process the data
; so that the analytics reports are always fast to load.
; You can define below the list of Segments strings
; for which all reports should be Archived during the cron execution
; All segment values MUST be URL encoded.
;Segments[]="visitorType==new"
;Segments[]="visitorType==returning"
; If you define Custom Variables for your visitor, for example set the visit type
;Segments[]="customVariableName1==VisitType;customVariableValue1==Customer"
[Deletelogs]
; delete_logs_enable - enable (1) or disable (0) delete log feature. Make sure that all archives for the given period have been processed (setup a cronjob!),
; otherwise you may lose tracking data.
; delete_logs_schedule_lowest_interval - lowest possible interval between two table deletes (in days, 1|7|30). Default: 7.
; delete_logs_older_than - delete data older than XX (days). Default: 180
delete_logs_enable = 0
delete_logs_schedule_lowest_interval = 7
delete_logs_older_than = 180
delete_logs_max_rows_per_query = 100000
enable_auto_database_size_estimate = 1
[Deletereports]
delete_reports_enable = 0
delete_reports_older_than = 12
delete_reports_keep_basic_metrics = 1
delete_reports_keep_day_reports = 0
delete_reports_keep_week_reports = 0
delete_reports_keep_month_reports = 1
delete_reports_keep_year_reports = 1
delete_reports_keep_range_reports = 0
delete_reports_keep_segment_reports = 0
[mail]
defaultHostnameIfEmpty = defaultHostnameIfEmpty.example.org ; default Email @hostname, if current host can't be read from system variables
transport = ; smtp (using the configuration below) or empty (using built-in mail() function)
port = ; optional; defaults to 25 when security is none or tls; 465 for ssl
host = ; SMTP server address
type = ; SMTP Auth type. By default: NONE. For example: LOGIN
username = ; SMTP username
password = ; SMTP password
encryption = ; SMTP transport-layer encryption, either 'ssl', 'tls', or empty (i.e., none).
[proxy]
type = BASIC ; proxy type for outbound/outgoing connections; currently, only BASIC is supported
host = ; Proxy host: the host name of your proxy server (mandatory)
port = ; Proxy port: the port that the proxy server listens to. There is no standard default, but 80, 1080, 3128, and 8080 are popular
username = ; Proxy username: optional; if specified, password is mandatory
password = ; Proxy password: optional; if specified, username is mandatory
[Plugins]
Plugins[] = CorePluginsAdmin
Plugins[] = CoreAdminHome
Plugins[] = CoreHome
Plugins[] = CoreVisualizations
Plugins[] = Proxy
Plugins[] = API
Plugins[] = ExamplePlugin
Plugins[] = Widgetize
Plugins[] = Transitions
Plugins[] = LanguagesManager
Plugins[] = Actions
Plugins[] = Dashboard
Plugins[] = MultiSites
Plugins[] = Referrers
Plugins[] = UserSettings
Plugins[] = Goals
Plugins[] = SEO
Plugins[] = Events
Plugins[] = UserCountry
Plugins[] = VisitsSummary
Plugins[] = VisitFrequency
Plugins[] = VisitTime
Plugins[] = VisitorInterest
Plugins[] = ExampleAPI
Plugins[] = ExampleRssWidget
Plugins[] = Provider
Plugins[] = Feedback
Plugins[] = Login
Plugins[] = UsersManager
Plugins[] = SitesManager
Plugins[] = Installation
Plugins[] = CoreUpdater
Plugins[] = CoreConsole
Plugins[] = ScheduledReports
Plugins[] = UserCountryMap
Plugins[] = Live
Plugins[] = CustomVariables
Plugins[] = PrivacyManager
Plugins[] = ImageGraph
Plugins[] = Annotations
Plugins[] = MobileMessaging
Plugins[] = Overlay
Plugins[] = SegmentEditor
Plugins[] = Insights
Plugins[] = Morpheus
[PluginsInstalled]
PluginsInstalled[] = Login
PluginsInstalled[] = CoreAdminHome
PluginsInstalled[] = UsersManager
PluginsInstalled[] = SitesManager
PluginsInstalled[] = Installation
[Plugins_Tracker]
Plugins_Tracker[] = Provider
Plugins_Tracker[] = Goals
Plugins_Tracker[] = PrivacyManager
Plugins_Tracker[] = UserCountry
Plugins_Tracker[] = Login
[APISettings]
; Any key/value pair can be added in this section, they will be available via the REST call
; index.php?module=API&method=API.getSettings
; This can be used to expose values from Piwik, to control for example a Mobile app tracking
SDK_batch_size = 10
SDK_interval_value = 30
; NOTE: do not directly edit this file! See notice at the top

3200
www/analytics/config/manifest.inc.php Normal file
View file

File diff suppressed because it is too large Load diff

28
www/analytics/console Executable file
View file

@ -0,0 +1,28 @@
#!/usr/bin/env php
<?php
if (!defined('PIWIK_DOCUMENT_ROOT')) {
define('PIWIK_DOCUMENT_ROOT', dirname(__FILE__) == '/' ? '' : dirname(__FILE__));
}
if (!defined('PIWIK_INCLUDE_PATH')) {
define('PIWIK_INCLUDE_PATH', PIWIK_DOCUMENT_ROOT);
}
if (!defined('PIWIK_USER_PATH')) {
define('PIWIK_USER_PATH', PIWIK_DOCUMENT_ROOT);
}
require_once PIWIK_INCLUDE_PATH . '/core/testMinimumPhpVersion.php';
require_once file_exists(PIWIK_INCLUDE_PATH . '/vendor/autoload.php')
? PIWIK_INCLUDE_PATH . '/vendor/autoload.php' // Piwik is the main project
: PIWIK_INCLUDE_PATH . '/../../autoload.php'; // Piwik is installed as a dependency
require_once PIWIK_INCLUDE_PATH . '/core/Loader.php';
require_once PIWIK_INCLUDE_PATH . '/libs/upgradephp/upgrade.php';
Piwik\Translate::loadEnglishTranslation();
if (!Piwik\Common::isPhpCliMode()) {
exit;
}
$console = new Piwik\Console();
$console->init();
$console->run();

13
www/analytics/core/.htaccess Normal file
View file

@ -0,0 +1,13 @@
<Files "*">
<IfModule mod_access.c>
Deny from all
</IfModule>
<IfModule !mod_access_compat>
<IfModule mod_authz_host.c>
Deny from all
</IfModule>
</IfModule>
<IfModule mod_access_compat>
Deny from all
</IfModule>
</Files>

149
www/analytics/core/API/DataTableGenericFilter.php Normal file
View file

@ -0,0 +1,149 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\API;
use Exception;
use Piwik\Common;
use Piwik\DataTable;
use Piwik\DataTable\Filter\AddColumnsProcessedMetricsGoal;
class DataTableGenericFilter
{
private static $genericFiltersInfo = null;
/**
* Constructor
*
* @param $request
*/
function __construct($request)
{
$this->request = $request;
}
/**
* Filters the given data table
*
* @param DataTable $table
*/
public function filter($table)
{
$this->applyGenericFilters($table);
}
/**
* Returns an array containing the information of the generic Filter
* to be applied automatically to the data resulting from the API calls.
*
* Order to apply the filters:
* 1 - Filter that remove filtered rows
* 2 - Filter that sort the remaining rows
* 3 - Filter that keep only a subset of the results
* 4 - Presentation filters
*
* @return array See the code for spec
*/
public static function getGenericFiltersInformation()
{
if (is_null(self::$genericFiltersInfo)) {
self::$genericFiltersInfo = array(
'Pattern' => array(
'filter_column' => array('string', 'label'),
'filter_pattern' => array('string'),
),
'PatternRecursive' => array(
'filter_column_recursive' => array('string', 'label'),
'filter_pattern_recursive' => array('string'),
),
'ExcludeLowPopulation' => array(
'filter_excludelowpop' => array('string'),
'filter_excludelowpop_value' => array('float', '0'),
),
'AddColumnsProcessedMetrics' => array(
'filter_add_columns_when_show_all_columns' => array('integer')
),
'AddColumnsProcessedMetricsGoal' => array(
'filter_update_columns_when_show_all_goals' => array('integer'),
'idGoal' => array('string', AddColumnsProcessedMetricsGoal::GOALS_OVERVIEW),
),
'Sort' => array(
'filter_sort_column' => array('string'),
'filter_sort_order' => array('string', 'desc'),
),
'Truncate' => array(
'filter_truncate' => array('integer'),
),
'Limit' => array(
'filter_offset' => array('integer', '0'),
'filter_limit' => array('integer'),
'keep_summary_row' => array('integer', '0'),
),
);
}
return self::$genericFiltersInfo;
}
/**
* Apply generic filters to the DataTable object resulting from the API Call.
* Disable this feature by setting the parameter disable_generic_filters to 1 in the API call request.
*
* @param DataTable $datatable
* @return bool
*/
protected function applyGenericFilters($datatable)
{
if ($datatable instanceof DataTable\Map) {
$tables = $datatable->getDataTables();
foreach ($tables as $table) {
$this->applyGenericFilters($table);
}
return;
}
$genericFilters = self::getGenericFiltersInformation();
$filterApplied = false;
foreach ($genericFilters as $filterName => $parameters) {
$filterParameters = array();
$exceptionRaised = false;
foreach ($parameters as $name => $info) {
// parameter type to cast to
$type = $info[0];
// default value if specified, when the parameter doesn't have a value
$defaultValue = null;
if (isset($info[1])) {
$defaultValue = $info[1];
}
// third element in the array, if it exists, overrides the name of the request variable
$varName = $name;
if (isset($info[2])) {
$varName = $info[2];
}
try {
$value = Common::getRequestVar($name, $defaultValue, $type, $this->request);
settype($value, $type);
$filterParameters[] = $value;
} catch (Exception $e) {
$exceptionRaised = true;
break;
}
}
if (!$exceptionRaised) {
$datatable->filter($filterName, $filterParameters);
$filterApplied = true;
}
}
return $filterApplied;
}
}

192
www/analytics/core/API/DataTableManipulator.php Normal file
View file

@ -0,0 +1,192 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\API;
use Exception;
use Piwik\Archive\DataTableFactory;
use Piwik\Common;
use Piwik\DataTable\Row;
use Piwik\DataTable;
use Piwik\Period\Range;
use Piwik\Plugins\API\API;
/**
* Base class for manipulating data tables.
* It provides generic mechanisms like iteration and loading subtables.
*
* The manipulators are used in ResponseBuilder and are triggered by
* API parameters. They are not filters because they don't work on the pre-
* fetched nested data tables. Instead, they load subtables using this base
* class. This way, they can only load the tables they really need instead
* of using expanded=1. Another difference between manipulators and filters
* is that filters keep the overall structure of the table intact while
* manipulators can change the entire thing.
*/
abstract class DataTableManipulator
{
protected $apiModule;
protected $apiMethod;
protected $request;
private $apiMethodForSubtable;
/**
* Constructor
*
* @param bool $apiModule
* @param bool $apiMethod
* @param array $request
*/
public function __construct($apiModule = false, $apiMethod = false, $request = array())
{
$this->apiModule = $apiModule;
$this->apiMethod = $apiMethod;
$this->request = $request;
}
/**
* This method can be used by subclasses to iterate over data tables that might be
* data table maps. It calls back the template method self::doManipulate for each table.
* This way, data table arrays can be handled in a transparent fashion.
*
* @param DataTable\Map|DataTable $dataTable
* @throws Exception
* @return DataTable\Map|DataTable
*/
protected function manipulate($dataTable)
{
if ($dataTable instanceof DataTable\Map) {
return $this->manipulateDataTableMap($dataTable);
} else if ($dataTable instanceof DataTable) {
return $this->manipulateDataTable($dataTable);
} else {
return $dataTable;
}
}
/**
* Manipulates child DataTables of a DataTable\Map. See @manipulate for more info.
*
* @param DataTable\Map $dataTable
* @return DataTable\Map
*/
protected function manipulateDataTableMap($dataTable)
{
$result = $dataTable->getEmptyClone();
foreach ($dataTable->getDataTables() as $tableLabel => $childTable) {
$newTable = $this->manipulate($childTable);
$result->addTable($newTable, $tableLabel);
}
return $result;
}
/**
* Manipulates a single DataTable instance. Derived classes must define
* this function.
*/
protected abstract function manipulateDataTable($dataTable);
/**
* Load the subtable for a row.
* Returns null if none is found.
*
* @param DataTable $dataTable
* @param Row $row
*
* @return DataTable
*/
protected function loadSubtable($dataTable, $row)
{
if (!($this->apiModule && $this->apiMethod && count($this->request))) {
return null;
}
$request = $this->request;
$idSubTable = $row->getIdSubDataTable();
if ($idSubTable === null) {
return null;
}
$request['idSubtable'] = $idSubTable;
if ($dataTable) {
$period = $dataTable->getMetadata(DataTableFactory::TABLE_METADATA_PERIOD_INDEX);
if ($period instanceof Range) {
$request['date'] = $period->getDateStart() . ',' . $period->getDateEnd();
} else {
$request['date'] = $period->getDateStart()->toString();
}
}
$method = $this->getApiMethodForSubtable();
return $this->callApiAndReturnDataTable($this->apiModule, $method, $request);
}
/**
* In this method, subclasses can clean up the request array for loading subtables
* in order to make ResponseBuilder behave correctly (e.g. not trigger the
* manipulator again).
*
* @param $request
* @return
*/
protected abstract function manipulateSubtableRequest($request);
/**
* Extract the API method for loading subtables from the meta data
*
* @return string
*/
private function getApiMethodForSubtable()
{
if (!$this->apiMethodForSubtable) {
$meta = API::getInstance()->getMetadata('all', $this->apiModule, $this->apiMethod);
if(empty($meta)) {
throw new Exception(sprintf(
"The DataTable cannot be manipulated: Metadata for report %s.%s could not be found. You can define the metadata in a hook, see example at: http://developer.piwik.org/api-reference/events#apigetreportmetadata",
$this->apiModule, $this->apiMethod
));
}
if (isset($meta[0]['actionToLoadSubTables'])) {
$this->apiMethodForSubtable = $meta[0]['actionToLoadSubTables'];
} else {
$this->apiMethodForSubtable = $this->apiMethod;
}
}
return $this->apiMethodForSubtable;
}
protected function callApiAndReturnDataTable($apiModule, $method, $request)
{
$class = Request::getClassNameAPI($apiModule);
$request = $this->manipulateSubtableRequest($request);
$request['serialize'] = 0;
$request['expanded'] = 0;
// don't want to run recursive filters on the subtables as they are loaded,
// otherwise the result will be empty in places (or everywhere). instead we
// run it on the flattened table.
unset($request['filter_pattern_recursive']);
$dataTable = Proxy::getInstance()->call($class, $method, $request);
$response = new ResponseBuilder($format = 'original', $request);
$dataTable = $response->getResponse($dataTable);
if (Common::getRequestVar('disable_queued_filters', 0, 'int', $request) == 0) {
if (method_exists($dataTable, 'applyQueuedFilters')) {
$dataTable->applyQueuedFilters();
}
}
return $dataTable;
}
}

137
www/analytics/core/API/DataTableManipulator/Flattener.php Normal file
View file

@ -0,0 +1,137 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\API\DataTableManipulator;
use Piwik\API\DataTableManipulator;
use Piwik\Common;
use Piwik\DataTable;
use Piwik\DataTable\Row;
/**
* This class is responsible for flattening data tables.
*
* It loads subtables and combines them into a single table by concatenating the labels.
* This manipulator is triggered by using flat=1 in the API request.
*/
class Flattener extends DataTableManipulator
{
private $includeAggregateRows = false;
/**
* If the flattener is used after calling this method, aggregate rows will
* be included in the result. This can be useful when they contain data that
* the leafs don't have (e.g. conversion stats in some cases).
*/
public function includeAggregateRows()
{
$this->includeAggregateRows = true;
}
/**
* Separator for building recursive labels (or paths)
* @var string
*/
public $recursiveLabelSeparator = ' - ';
/**
* @param DataTable $dataTable
* @return DataTable|DataTable\Map
*/
public function flatten($dataTable)
{
if ($this->apiModule == 'Actions' || $this->apiMethod == 'getWebsites') {
$this->recursiveLabelSeparator = '/';
}
return $this->manipulate($dataTable);
}
/**
* Template method called from self::manipulate.
* Flatten each data table.
*
* @param DataTable $dataTable
* @return DataTable
*/
protected function manipulateDataTable($dataTable)
{
// apply filters now since subtables have their filters applied before generic filters. if we don't do this
// now, we'll try to apply filters to rows that have already been manipulated. this results in errors like
// 'column ... already exists'.
$keepFilters = true;
if (Common::getRequestVar('disable_queued_filters', 0, 'int', $this->request) == 0) {
$dataTable->applyQueuedFilters();
$keepFilters = false;
}
$newDataTable = $dataTable->getEmptyClone($keepFilters);
foreach ($dataTable->getRows() as $row) {
$this->flattenRow($row, $newDataTable);
}
return $newDataTable;
}
/**
* @param Row $row
* @param DataTable $dataTable
* @param string $labelPrefix
* @param bool $parentLogo
*/
private function flattenRow(Row $row, DataTable $dataTable,
$labelPrefix = '', $parentLogo = false)
{
$label = $row->getColumn('label');
if ($label !== false) {
$label = trim($label);
if (substr($label, 0, 1) == '/' && $this->recursiveLabelSeparator == '/') {
$label = substr($label, 1);
}
$label = $labelPrefix . $label;
$row->setColumn('label', $label);
}
$logo = $row->getMetadata('logo');
if ($logo === false && $parentLogo !== false) {
$logo = $parentLogo;
$row->setMetadata('logo', $logo);
}
$subTable = $this->loadSubtable($dataTable, $row);
$row->removeSubtable();
if ($subTable === null) {
if ($this->includeAggregateRows) {
$row->setMetadata('is_aggregate', 0);
}
$dataTable->addRow($row);
} else {
if ($this->includeAggregateRows) {
$row->setMetadata('is_aggregate', 1);
$dataTable->addRow($row);
}
$prefix = $label . $this->recursiveLabelSeparator;
foreach ($subTable->getRows() as $row) {
$this->flattenRow($row, $dataTable, $prefix, $logo);
}
}
}
/**
* Remove the flat parameter from the subtable request
*
* @param array $request
*/
protected function manipulateSubtableRequest($request)
{
unset($request['flat']);
return $request;
}
}

167
www/analytics/core/API/DataTableManipulator/LabelFilter.php Normal file
View file

@ -0,0 +1,167 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\API\DataTableManipulator;
use Piwik\API\DataTableManipulator;
use Piwik\Common;
use Piwik\DataTable;
use Piwik\DataTable\Row;
/**
* This class is responsible for handling the label parameter that can be
* added to every API call. If the parameter is set, only the row with the matching
* label is returned.
*
* The labels passed to this class should be urlencoded.
* Some reports use recursive labels (e.g. action reports). Use > to join them.
*/
class LabelFilter extends DataTableManipulator
{
const SEPARATOR_RECURSIVE_LABEL = '>';
private $labels;
private $addLabelIndex;
const FLAG_IS_ROW_EVOLUTION = 'label_index';
/**
* Filter a data table by label.
* The filtered table is returned, which might be a new instance.
*
* $apiModule, $apiMethod and $request are needed load sub-datatables
* for the recursive search. If the label is not recursive, these parameters
* are not needed.
*
* @param string $labels the labels to search for
* @param DataTable $dataTable the data table to be filtered
* @param bool $addLabelIndex Whether to add label_index metadata describing which
* label a row corresponds to.
* @return DataTable
*/
public function filter($labels, $dataTable, $addLabelIndex = false)
{
if (!is_array($labels)) {
$labels = array($labels);
}
$this->labels = $labels;
$this->addLabelIndex = (bool)$addLabelIndex;
return $this->manipulate($dataTable);
}
/**
* Method for the recursive descend
*
* @param array $labelParts
* @param DataTable $dataTable
* @return Row|bool
*/
private function doFilterRecursiveDescend($labelParts, $dataTable)
{
// search for the first part of the tree search
$labelPart = array_shift($labelParts);
$row = false;
foreach ($this->getLabelVariations($labelPart) as $labelPart) {
$row = $dataTable->getRowFromLabel($labelPart);
if ($row !== false) {
break;
}
}
if ($row === false) {
// not found
return false;
}
// end of tree search reached
if (count($labelParts) == 0) {
return $row;
}
$subTable = $this->loadSubtable($dataTable, $row);
if ($subTable === null) {
// no more subtables but label parts left => no match found
return false;
}
return $this->doFilterRecursiveDescend($labelParts, $subTable);
}
/**
* Clean up request for ResponseBuilder to behave correctly
*
* @param $request
*/
protected function manipulateSubtableRequest($request)
{
unset($request['label']);
return $request;
}
/**
* Use variations of the label to make it easier to specify the desired label
*
* Note: The HTML Encoded version must be tried first, since in ResponseBuilder the $label is unsanitized
* via Common::unsanitizeLabelParameter.
*
* @param string $label
* @return array
*/
private function getLabelVariations($label)
{
static $pageTitleReports = array('getPageTitles', 'getEntryPageTitles', 'getExitPageTitles');
$variations = array();
$label = urldecode($label);
$label = trim($label);
$sanitizedLabel = Common::sanitizeInputValue($label);
$variations[] = $sanitizedLabel;
if ($this->apiModule == 'Actions'
&& in_array($this->apiMethod, $pageTitleReports)
) {
// special case: the Actions.getPageTitles report prefixes some labels with a blank.
// the blank might be passed by the user but is removed in Request::getRequestArrayFromString.
$variations[] = ' ' . $sanitizedLabel;
$variations[] = ' ' . $label;
}
$variations[] = $label;
return $variations;
}
/**
* Filter a DataTable instance. See @filter for more info.
*
* @param DataTable\Simple|DataTable\Map $dataTable
* @return mixed
*/
protected function manipulateDataTable($dataTable)
{
$result = $dataTable->getEmptyClone();
foreach ($this->labels as $labelIndex => $label) {
$row = null;
foreach ($this->getLabelVariations($label) as $labelVariation) {
$labelVariation = explode(self::SEPARATOR_RECURSIVE_LABEL, $labelVariation);
$row = $this->doFilterRecursiveDescend($labelVariation, $dataTable);
if ($row) {
if ($this->addLabelIndex) {
$row->setMetadata(self::FLAG_IS_ROW_EVOLUTION, $labelIndex);
}
$result->addRow($row);
break;
}
}
}
return $result;
}
}

250
www/analytics/core/API/DataTableManipulator/ReportTotalsCalculator.php Normal file
View file

@ -0,0 +1,250 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\API\DataTableManipulator;
use Piwik\API\DataTableManipulator;
use Piwik\DataTable;
use Piwik\DataTable\Row;
use Piwik\DataTable\BaseFilter;
use Piwik\Period\Range;
use Piwik\Period;
use Piwik\Piwik;
use Piwik\Metrics;
use Piwik\Plugins\API\API;
/**
* This class is responsible for setting the metadata property 'totals' on each dataTable if the report
* has a dimension. 'Totals' means it tries to calculate the total report value for each metric. For each
* the total number of visits, actions, ... for a given report / dataTable.
*/
class ReportTotalsCalculator extends DataTableManipulator
{
/**
* Cached report metadata array.
* @var array
*/
private static $reportMetadata = array();
/**
* @param DataTable $table
* @return \Piwik\DataTable|\Piwik\DataTable\Map
*/
public function calculate($table)
{
// apiModule and/or apiMethod is empty for instance in case when flat=1 is called. Basically whenever a
// datamanipulator calls the API and wants the dataTable in return, see callApiAndReturnDataTable().
// it is also not set for some settings API request etc.
if (empty($this->apiModule) || empty($this->apiMethod)) {
return $table;
}
try {
return $this->manipulate($table);
} catch(\Exception $e) {
// eg. requests with idSubtable may trigger this exception
// (where idSubtable was removed in
// ?module=API&method=Events.getNameFromCategoryId&idSubtable=1&secondaryDimension=eventName&format=XML&idSite=1&period=day&date=yesterday&flat=0
return $table;
}
}
/**
* Adds ratio metrics if possible.
*
* @param DataTable $dataTable
* @return DataTable
*/
protected function manipulateDataTable($dataTable)
{
$report = $this->findCurrentReport();
if (!empty($report) && empty($report['dimension'])) {
// we currently do not calculate the total value for reports having no dimension
return $dataTable;
}
// Array [readableMetric] => [summed value]
$totalValues = array();
$firstLevelTable = $this->makeSureToWorkOnFirstLevelDataTable($dataTable);
$metricsToCalculate = Metrics::getMetricIdsToProcessReportTotal();
foreach ($metricsToCalculate as $metricId) {
if (!$this->hasDataTableMetric($firstLevelTable, $metricId)) {
continue;
}
foreach ($firstLevelTable->getRows() as $row) {
$totalValues = $this->sumColumnValueToTotal($row, $metricId, $totalValues);
}
}
$dataTable->setMetadata('totals', $totalValues);
return $dataTable;
}
private function hasDataTableMetric(DataTable $dataTable, $metricId)
{
$firstRow = $dataTable->getFirstRow();
if (empty($firstRow)) {
return false;
}
if (false === $this->getColumn($firstRow, $metricId)) {
return false;
}
return true;
}
/**
* Returns column from a given row.
* Will work with 2 types of datatable
* - raw datatables coming from the archive DB, which columns are int indexed
* - datatables processed resulting of API calls, which columns have human readable english names
*
* @param Row|array $row
* @param int $columnIdRaw see consts in Metrics::
* @return mixed Value of column, false if not found
*/
private function getColumn($row, $columnIdRaw)
{
$columnIdReadable = Metrics::getReadableColumnName($columnIdRaw);
if ($row instanceof Row) {
$raw = $row->getColumn($columnIdRaw);
if ($raw !== false) {
return $raw;
}
return $row->getColumn($columnIdReadable);
}
return false;
}
private function makeSureToWorkOnFirstLevelDataTable($table)
{
if (!array_key_exists('idSubtable', $this->request)) {
return $table;
}
$firstLevelReport = $this->findFirstLevelReport();
if (empty($firstLevelReport)) {
// it is not a subtable report
$module = $this->apiModule;
$action = $this->apiMethod;
} else {
$module = $firstLevelReport['module'];
$action = $firstLevelReport['action'];
}
$request = $this->request;
/** @var \Piwik\Period $period */
$period = $table->getMetadata('period');
if (!empty($period)) {
// we want a dataTable, not a dataTable\map
if (Period::isMultiplePeriod($request['date'], $request['period']) || 'range' == $period->getLabel()) {
$request['date'] = $period->getRangeString();
$request['period'] = 'range';
} else {
$request['date'] = $period->getDateStart()->toString();
$request['period'] = $period->getLabel();
}
}
return $this->callApiAndReturnDataTable($module, $action, $request);
}
private function sumColumnValueToTotal(Row $row, $metricId, $totalValues)
{
$value = $this->getColumn($row, $metricId);
if (false === $value) {
return $totalValues;
}
$metricName = Metrics::getReadableColumnName($metricId);
if (array_key_exists($metricName, $totalValues)) {
$totalValues[$metricName] += $value;
} else {
$totalValues[$metricName] = $value;
}
return $totalValues;
}
/**
* Make sure to get all rows of the first level table.
*
* @param array $request
*/
protected function manipulateSubtableRequest($request)
{
$request['totals'] = 0;
$request['expanded'] = 0;
$request['filter_limit'] = -1;
$request['filter_offset'] = 0;
$parametersToRemove = array('flat');
if (!array_key_exists('idSubtable', $this->request)) {
$parametersToRemove[] = 'idSubtable';
}
foreach ($parametersToRemove as $param) {
if (array_key_exists($param, $request)) {
unset($request[$param]);
}
}
return $request;
}
private function getReportMetadata()
{
if (!empty(static::$reportMetadata)) {
return static::$reportMetadata;
}
static::$reportMetadata = API::getInstance()->getReportMetadata();
return static::$reportMetadata;
}
private function findCurrentReport()
{
foreach ($this->getReportMetadata() as $report) {
if ($this->apiMethod == $report['action']
&& $this->apiModule == $report['module']) {
return $report;
}
}
}
private function findFirstLevelReport()
{
foreach ($this->getReportMetadata() as $report) {
if (!empty($report['actionToLoadSubTables'])
&& $this->apiMethod == $report['actionToLoadSubTables']
&& $this->apiModule == $report['module']
) {
return $report;
}
}
}
}

238
www/analytics/core/API/DocumentationGenerator.php Normal file
View file

@ -0,0 +1,238 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\API;
use Exception;
use Piwik\Common;
use Piwik\Piwik;
use Piwik\Url;
class DocumentationGenerator
{
protected $modulesToHide = array('CoreAdminHome', 'DBStats');
protected $countPluginsLoaded = 0;
/**
* trigger loading all plugins with an API.php file in the Proxy
*/
public function __construct()
{
$plugins = \Piwik\Plugin\Manager::getInstance()->getLoadedPluginsName();
foreach ($plugins as $plugin) {
try {
$className = Request::getClassNameAPI($plugin);
Proxy::getInstance()->registerClass($className);
} catch (Exception $e) {
}
}
}
/**
* Returns a HTML page containing help for all the successfully loaded APIs.
* For each module it will return a mini help with the method names, parameters to give,
* links to get the result in Xml/Csv/etc
*
* @param bool $outputExampleUrls
* @param string $prefixUrls
* @return string
*/
public function getAllInterfaceString($outputExampleUrls = true, $prefixUrls = '')
{
if (!empty($prefixUrls)) {
$prefixUrls = 'http://demo.piwik.org/';
}
$str = $toc = '';
$token_auth = "&token_auth=" . Piwik::getCurrentUserTokenAuth();
$parametersToSet = array(
'idSite' => Common::getRequestVar('idSite', 1, 'int'),
'period' => Common::getRequestVar('period', 'day', 'string'),
'date' => Common::getRequestVar('date', 'today', 'string')
);
foreach (Proxy::getInstance()->getMetadata() as $class => $info) {
$moduleName = Proxy::getInstance()->getModuleNameFromClassName($class);
if (in_array($moduleName, $this->modulesToHide)) {
continue;
}
$toc .= "<a href='#$moduleName'>$moduleName</a><br/>";
$str .= "\n<a name='$moduleName' id='$moduleName'></a><h2>Module " . $moduleName . "</h2>";
$str .= "<div class='apiDescription'> " . $info['__documentation'] . " </div>";
foreach ($info as $methodName => $infoMethod) {
if ($methodName == '__documentation') {
continue;
}
$params = $this->getParametersString($class, $methodName);
$str .= "\n <div class='apiMethod'>- <b>$moduleName.$methodName </b>" . $params . "";
$str .= '<small>';
if ($outputExampleUrls) {
// we prefix all URLs with $prefixUrls
// used when we include this output in the Piwik official documentation for example
$str .= "<span class=\"example\">";
$exampleUrl = $this->getExampleUrl($class, $methodName, $parametersToSet);
if ($exampleUrl !== false) {
$lastNUrls = '';
if (preg_match('/(&period)|(&date)/', $exampleUrl)) {
$exampleUrlRss1 = $prefixUrls . $this->getExampleUrl($class, $methodName, array('date' => 'last10', 'period' => 'day') + $parametersToSet);
$exampleUrlRss2 = $prefixUrls . $this->getExampleUrl($class, $methodName, array('date' => 'last5', 'period' => 'week',) + $parametersToSet);
$lastNUrls = ", RSS of the last <a target=_blank href='$exampleUrlRss1&format=rss$token_auth&translateColumnNames=1'>10 days</a>";
}
$exampleUrl = $prefixUrls . $exampleUrl;
$str .= " [ Example in
<a target=_blank href='$exampleUrl&format=xml$token_auth'>XML</a>,
<a target=_blank href='$exampleUrl&format=JSON$token_auth'>Json</a>,
<a target=_blank href='$exampleUrl&format=Tsv$token_auth&translateColumnNames=1'>Tsv (Excel)</a>
$lastNUrls
]";
} else {
$str .= " [ No example available ]";
}
$str .= "</span>";
}
$str .= '</small>';
$str .= "</div>\n";
}
$str .= '<div style="margin:15px;"><a href="#topApiRef">↑ Back to top</a></div>';
}
$str = "<h2 id='topApiRef' name='topApiRef'>Quick access to APIs</h2>
$toc
$str";
return $str;
}
/**
* Returns a string containing links to examples on how to call a given method on a given API
* It will export links to XML, CSV, HTML, JSON, PHP, etc.
* It will not export links for methods such as deleteSite or deleteUser
*
* @param string $class the class
* @param string $methodName the method
* @param array $parametersToSet parameters to set
* @return string|bool when not possible
*/
public function getExampleUrl($class, $methodName, $parametersToSet = array())
{
$knowExampleDefaultParametersValues = array(
'access' => 'view',
'userLogin' => 'test',
'passwordMd5ied' => 'passwordExample',
'email' => 'test@example.org',
'languageCode' => 'fr',
'url' => 'http://forum.piwik.org/',
'pageUrl' => 'http://forum.piwik.org/',
'apiModule' => 'UserCountry',
'apiAction' => 'getCountry',
'lastMinutes' => '30',
'abandonedCarts' => '0',
'segmentName' => 'pageTitle',
'ip' => '194.57.91.215',
'idSites' => '1,2',
'idAlert' => '1',
// 'segmentName' => 'browserCode',
);
foreach ($parametersToSet as $name => $value) {
$knowExampleDefaultParametersValues[$name] = $value;
}
// no links for these method names
$doNotPrintExampleForTheseMethods = array(
//Sites
'deleteSite',
'addSite',
'updateSite',
'addSiteAliasUrls',
//Users
'deleteUser',
'addUser',
'updateUser',
'setUserAccess',
//Goals
'addGoal',
'updateGoal',
'deleteGoal',
);
if (in_array($methodName, $doNotPrintExampleForTheseMethods)) {
return false;
}
// we try to give an URL example to call the API
$aParameters = Proxy::getInstance()->getParametersList($class, $methodName);
// Kindly force some known generic parameters to appear in the final list
// the parameter 'format' can be set to all API methods (used in tests)
// the parameter 'hideIdSubDatable' is used for integration tests only
// the parameter 'serialize' sets php outputs human readable, used in integration tests and debug
// the parameter 'language' sets the language for the response (eg. country names)
// the parameter 'flat' reduces a hierarchical table to a single level by concatenating labels
// the parameter 'include_aggregate_rows' can be set to include inner nodes in flat reports
// the parameter 'translateColumnNames' can be set to translate metric names in csv/tsv exports
$aParameters['format'] = false;
$aParameters['hideIdSubDatable'] = false;
$aParameters['serialize'] = false;
$aParameters['language'] = false;
$aParameters['translateColumnNames'] = false;
$aParameters['label'] = false;
$aParameters['flat'] = false;
$aParameters['include_aggregate_rows'] = false;
$aParameters['filter_limit'] = false; //@review without adding this, I can not set filter_limit in $otherRequestParameters integration tests
$aParameters['filter_sort_column'] = false; //@review without adding this, I can not set filter_sort_column in $otherRequestParameters integration tests
$aParameters['filter_truncate'] = false;
$aParameters['hideColumns'] = false;
$aParameters['showColumns'] = false;
$aParameters['filter_pattern_recursive'] = false;
$moduleName = Proxy::getInstance()->getModuleNameFromClassName($class);
$aParameters = array_merge(array('module' => 'API', 'method' => $moduleName . '.' . $methodName), $aParameters);
foreach ($aParameters as $nameVariable => &$defaultValue) {
if (isset($knowExampleDefaultParametersValues[$nameVariable])) {
$defaultValue = $knowExampleDefaultParametersValues[$nameVariable];
} // if there isn't a default value for a given parameter,
// we need a 'know default value' or we can't generate the link
elseif ($defaultValue instanceof NoDefaultValue) {
return false;
}
}
return '?' . Url::getQueryStringFromParameters($aParameters);
}
/**
* Returns the methods $class.$name parameters (and default value if provided) as a string.
*
* @param string $class The class name
* @param string $name The method name
* @return string For example "(idSite, period, date = 'today')"
*/
public function getParametersString($class, $name)
{
$aParameters = Proxy::getInstance()->getParametersList($class, $name);
$asParameters = array();
foreach ($aParameters as $nameVariable => $defaultValue) {
// Do not show API parameters starting with _
// They are supposed to be used only in internal API calls
if (strpos($nameVariable, '_') === 0) {
continue;
}
$str = $nameVariable;
if (!($defaultValue instanceof NoDefaultValue)) {
if (is_array($defaultValue)) {
$str .= " = 'Array'";
} else {
$str .= " = '$defaultValue'";
}
}
$asParameters[] = $str;
}
$sParameters = implode(", ", $asParameters);
return "($sParameters)";
}
}

514
www/analytics/core/API/Proxy.php Normal file
View file

@ -0,0 +1,514 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\API;
use Exception;
use Piwik\Common;
use Piwik\Piwik;
use Piwik\Singleton;
use ReflectionClass;
use ReflectionMethod;
/**
* Proxy is a singleton that has the knowledge of every method available, their parameters
* and default values.
* Proxy receives all the API calls requests via call() and forwards them to the right
* object, with the parameters in the right order.
*
* It will also log the performance of API calls (time spent, parameter values, etc.) if logger available
*
* @method static \Piwik\API\Proxy getInstance()
*/
class Proxy extends Singleton
{
// array of already registered plugins names
protected $alreadyRegistered = array();
private $metadataArray = array();
private $hideIgnoredFunctions = true;
// when a parameter doesn't have a default value we use this
private $noDefaultValue;
/**
* protected constructor
*/
protected function __construct()
{
$this->noDefaultValue = new NoDefaultValue();
}
/**
* Returns array containing reflection meta data for all the loaded classes
* eg. number of parameters, method names, etc.
*
* @return array
*/
public function getMetadata()
{
ksort($this->metadataArray);
return $this->metadataArray;
}
/**
* Registers the API information of a given module.
*
* The module to be registered must be
* - a singleton (providing a getInstance() method)
* - the API file must be located in plugins/ModuleName/API.php
* for example plugins/Referrers/API.php
*
* The method will introspect the methods, their parameters, etc.
*
* @param string $className ModuleName eg. "API"
*/
public function registerClass($className)
{
if (isset($this->alreadyRegistered[$className])) {
return;
}
$this->includeApiFile($className);
$this->checkClassIsSingleton($className);
$rClass = new ReflectionClass($className);
foreach ($rClass->getMethods() as $method) {
$this->loadMethodMetadata($className, $method);
}
$this->setDocumentation($rClass, $className);
$this->alreadyRegistered[$className] = true;
}
/**
* Will be displayed in the API page
*
* @param ReflectionClass $rClass Instance of ReflectionClass
* @param string $className Name of the class
*/
private function setDocumentation($rClass, $className)
{
// Doc comment
$doc = $rClass->getDocComment();
$doc = str_replace(" * " . PHP_EOL, "<br>", $doc);
// boldify the first line only if there is more than one line, otherwise too much bold
if (substr_count($doc, '<br>') > 1) {
$firstLineBreak = strpos($doc, "<br>");
$doc = "<div class='apiFirstLine'>" . substr($doc, 0, $firstLineBreak) . "</div>" . substr($doc, $firstLineBreak + strlen("<br>"));
}
$doc = preg_replace("/(@package)[a-z _A-Z]*/", "", $doc);
$doc = preg_replace("/(@method).*/", "", $doc);
$doc = str_replace(array("\t", "\n", "/**", "*/", " * ", " *", " ", "\t*", " * @package"), " ", $doc);
$this->metadataArray[$className]['__documentation'] = $doc;
}
/**
* Returns number of classes already loaded
* @return int
*/
public function getCountRegisteredClasses()
{
return count($this->alreadyRegistered);
}
/**
* Will execute $className->$methodName($parametersValues)
* If any error is detected (wrong number of parameters, method not found, class not found, etc.)
* it will throw an exception
*
* It also logs the API calls, with the parameters values, the returned value, the performance, etc.
* You can enable logging in config/global.ini.php (log_api_call)
*
* @param string $className The class name (eg. API)
* @param string $methodName The method name
* @param array $parametersRequest The parameters pairs (name=>value)
*
* @return mixed|null
* @throws Exception|\Piwik\NoAccessException
*/
public function call($className, $methodName, $parametersRequest)
{
$returnedValue = null;
// Temporarily sets the Request array to this API call context
$saveGET = $_GET;
$saveQUERY_STRING = @$_SERVER['QUERY_STRING'];
foreach ($parametersRequest as $param => $value) {
$_GET[$param] = $value;
}
try {
$this->registerClass($className);
// instanciate the object
$object = $className::getInstance();
// check method exists
$this->checkMethodExists($className, $methodName);
// get the list of parameters required by the method
$parameterNamesDefaultValues = $this->getParametersList($className, $methodName);
// load parameters in the right order, etc.
$finalParameters = $this->getRequestParametersArray($parameterNamesDefaultValues, $parametersRequest);
// allow plugins to manipulate the value
$pluginName = $this->getModuleNameFromClassName($className);
/**
* Triggered before an API request is dispatched.
*
* This event can be used to modify the arguments passed to one or more API methods.
*
* **Example**
*
* Piwik::addAction('API.Request.dispatch', function (&$parameters, $pluginName, $methodName) {
* if ($pluginName == 'Actions') {
* if ($methodName == 'getPageUrls') {
* // ... do something ...
* } else {
* // ... do something else ...
* }
* }
* });
*
* @param array &$finalParameters List of parameters that will be passed to the API method.
* @param string $pluginName The name of the plugin the API method belongs to.
* @param string $methodName The name of the API method that will be called.
*/
Piwik::postEvent('API.Request.dispatch', array(&$finalParameters, $pluginName, $methodName));
/**
* Triggered before an API request is dispatched.
*
* This event exists for convenience and is triggered directly after the {@hook API.Request.dispatch}
* event is triggered. It can be used to modify the arguments passed to a **single** API method.
*
* _Note: This is can be accomplished with the {@hook API.Request.dispatch} event as well, however
* event handlers for that event will have to do more work._
*
* **Example**
*
* Piwik::addAction('API.Actions.getPageUrls', function (&$parameters) {
* // force use of a single website. for some reason.
* $parameters['idSite'] = 1;
* });
*
* @param array &$finalParameters List of parameters that will be passed to the API method.
*/
Piwik::postEvent(sprintf('API.%s.%s', $pluginName, $methodName), array(&$finalParameters));
// call the method
$returnedValue = call_user_func_array(array($object, $methodName), $finalParameters);
$endHookParams = array(
&$returnedValue,
array('className' => $className,
'module' => $pluginName,
'action' => $methodName,
'parameters' => $finalParameters)
);
/**
* Triggered directly after an API request is dispatched.
*
* This event exists for convenience and is triggered immediately before the
* {@hook API.Request.dispatch.end} event. It can be used to modify the output of a **single**
* API method.
*
* _Note: This can be accomplished with the {@hook API.Request.dispatch.end} event as well,
* however event handlers for that event will have to do more work._
*
* **Example**
*
* // append (0 hits) to the end of row labels whose row has 0 hits
* Piwik::addAction('API.Actions.getPageUrls', function (&$returnValue, $info)) {
* $returnValue->filter('ColumnCallbackReplace', 'label', function ($label, $hits) {
* if ($hits === 0) {
* return $label . " (0 hits)";
* } else {
* return $label;
* }
* }, null, array('nb_hits'));
* }
*
* @param mixed &$returnedValue The API method's return value. Can be an object, such as a
* {@link Piwik\DataTable DataTable} instance.
* could be a {@link Piwik\DataTable DataTable}.
* @param array $extraInfo An array holding information regarding the API request. Will
* contain the following data:
*
* - **className**: The namespace-d class name of the API instance
* that's being called.
* - **module**: The name of the plugin the API request was
* dispatched to.
* - **action**: The name of the API method that was executed.
* - **parameters**: The array of parameters passed to the API
* method.
*/
Piwik::postEvent(sprintf('API.%s.%s.end', $pluginName, $methodName), $endHookParams);
/**
* Triggered directly after an API request is dispatched.
*
* This event can be used to modify the output of any API method.
*
* **Example**
*
* // append (0 hits) to the end of row labels whose row has 0 hits for any report that has the 'nb_hits' metric
* Piwik::addAction('API.Actions.getPageUrls', function (&$returnValue, $info)) {
* // don't process non-DataTable reports and reports that don't have the nb_hits column
* if (!($returnValue instanceof DataTableInterface)
* || in_array('nb_hits', $returnValue->getColumns())
* ) {
* return;
* }
*
* $returnValue->filter('ColumnCallbackReplace', 'label', function ($label, $hits) {
* if ($hits === 0) {
* return $label . " (0 hits)";
* } else {
* return $label;
* }
* }, null, array('nb_hits'));
* }
*
* @param mixed &$returnedValue The API method's return value. Can be an object, such as a
* {@link Piwik\DataTable DataTable} instance.
* @param array $extraInfo An array holding information regarding the API request. Will
* contain the following data:
*
* - **className**: The namespace-d class name of the API instance
* that's being called.
* - **module**: The name of the plugin the API request was
* dispatched to.
* - **action**: The name of the API method that was executed.
* - **parameters**: The array of parameters passed to the API
* method.
*/
Piwik::postEvent('API.Request.dispatch.end', $endHookParams);
// Restore the request
$_GET = $saveGET;
$_SERVER['QUERY_STRING'] = $saveQUERY_STRING;
} catch (Exception $e) {
$_GET = $saveGET;
throw $e;
}
return $returnedValue;
}
/**
* Returns the parameters names and default values for the method $name
* of the class $class
*
* @param string $class The class name
* @param string $name The method name
* @return array Format array(
* 'testParameter' => null, // no default value
* 'life' => 42, // default value = 42
* 'date' => 'yesterday',
* );
*/
public function getParametersList($class, $name)
{
return $this->metadataArray[$class][$name]['parameters'];
}
/**
* Returns the 'moduleName' part of '\\Piwik\\Plugins\\moduleName\\API'
*
* @param string $className "API"
* @return string "Referrers"
*/
public function getModuleNameFromClassName($className)
{
return str_replace(array('\\Piwik\\Plugins\\', '\\API'), '', $className);
}
public function isExistingApiAction($pluginName, $apiAction)
{
$namespacedApiClassName = "\\Piwik\\Plugins\\$pluginName\\API";
$api = $namespacedApiClassName::getInstance();
return method_exists($api, $apiAction);
}
public function buildApiActionName($pluginName, $apiAction)
{
return sprintf("%s.%s", $pluginName, $apiAction);
}
/**
* Sets whether to hide '@ignore'd functions from method metadata or not.
*
* @param bool $hideIgnoredFunctions
*/
public function setHideIgnoredFunctions($hideIgnoredFunctions)
{
$this->hideIgnoredFunctions = $hideIgnoredFunctions;
// make sure metadata gets reloaded
$this->alreadyRegistered = array();
$this->metadataArray = array();
}
/**
* Returns an array containing the values of the parameters to pass to the method to call
*
* @param array $requiredParameters array of (parameter name, default value)
* @param array $parametersRequest
* @throws Exception
* @return array values to pass to the function call
*/
private function getRequestParametersArray($requiredParameters, $parametersRequest)
{
$finalParameters = array();
foreach ($requiredParameters as $name => $defaultValue) {
try {
if ($defaultValue instanceof NoDefaultValue) {
$requestValue = Common::getRequestVar($name, null, null, $parametersRequest);
} else {
try {
if ($name == 'segment' && !empty($parametersRequest['segment'])) {
// segment parameter is an exception: we do not want to sanitize user input or it would break the segment encoding
$requestValue = ($parametersRequest['segment']);
} else {
$requestValue = Common::getRequestVar($name, $defaultValue, null, $parametersRequest);
}
} catch (Exception $e) {
// Special case: empty parameter in the URL, should return the empty string
if (isset($parametersRequest[$name])
&& $parametersRequest[$name] === ''
) {
$requestValue = '';
} else {
$requestValue = $defaultValue;
}
}
}
} catch (Exception $e) {
throw new Exception(Piwik::translate('General_PleaseSpecifyValue', array($name)));
}
$finalParameters[] = $requestValue;
}
return $finalParameters;
}
/**
* Includes the class API by looking up plugins/UserSettings/API.php
*
* @param string $fileName api class name eg. "API"
* @throws Exception
*/
private function includeApiFile($fileName)
{
$module = self::getModuleNameFromClassName($fileName);
$path = PIWIK_INCLUDE_PATH . '/plugins/' . $module . '/API.php';
if (is_readable($path)) {
require_once $path; // prefixed by PIWIK_INCLUDE_PATH
} else {
throw new Exception("API module $module not found.");
}
}
/**
* @param string $class name of a class
* @param ReflectionMethod $method instance of ReflectionMethod
*/
private function loadMethodMetadata($class, $method)
{
if ($method->isPublic()
&& !$method->isConstructor()
&& $method->getName() != 'getInstance'
&& false === strstr($method->getDocComment(), '@deprecated')
&& (!$this->hideIgnoredFunctions || false === strstr($method->getDocComment(), '@ignore'))
) {
$name = $method->getName();
$parameters = $method->getParameters();
$aParameters = array();
foreach ($parameters as $parameter) {
$nameVariable = $parameter->getName();
$defaultValue = $this->noDefaultValue;
if ($parameter->isDefaultValueAvailable()) {
$defaultValue = $parameter->getDefaultValue();
}
$aParameters[$nameVariable] = $defaultValue;
}
$this->metadataArray[$class][$name]['parameters'] = $aParameters;
$this->metadataArray[$class][$name]['numberOfRequiredParameters'] = $method->getNumberOfRequiredParameters();
}
}
/**
* Checks that the method exists in the class
*
* @param string $className The class name
* @param string $methodName The method name
* @throws Exception If the method is not found
*/
private function checkMethodExists($className, $methodName)
{
if (!$this->isMethodAvailable($className, $methodName)) {
throw new Exception(Piwik::translate('General_ExceptionMethodNotFound', array($methodName, $className)));
}
}
/**
* Returns the number of required parameters (parameters without default values).
*
* @param string $class The class name
* @param string $name The method name
* @return int The number of required parameters
*/
private function getNumberOfRequiredParameters($class, $name)
{
return $this->metadataArray[$class][$name]['numberOfRequiredParameters'];
}
/**
* Returns true if the method is found in the API of the given class name.
*
* @param string $className The class name
* @param string $methodName The method name
* @return bool
*/
private function isMethodAvailable($className, $methodName)
{
return isset($this->metadataArray[$className][$methodName]);
}
/**
* Checks that the class is a Singleton (presence of the getInstance() method)
*
* @param string $className The class name
* @throws Exception If the class is not a Singleton
*/
private function checkClassIsSingleton($className)
{
if (!method_exists($className, "getInstance")) {
throw new Exception("$className that provide an API must be Singleton and have a 'static public function getInstance()' method.");
}
}
}
/**
* To differentiate between "no value" and default value of null
*
*/
class NoDefaultValue
{
}

398
www/analytics/core/API/Request.php Normal file
View file

@ -0,0 +1,398 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\API;
use Exception;
use Piwik\Access;
use Piwik\Common;
use Piwik\DataTable;
use Piwik\Piwik;
use Piwik\PluginDeactivatedException;
use Piwik\SettingsServer;
use Piwik\Url;
use Piwik\UrlHelper;
/**
* Dispatches API requests to the appropriate API method.
*
* The Request class is used throughout Piwik to call API methods. The difference
* between using Request and calling API methods directly is that Request
* will do more after calling the API including: applying generic filters, applying queued filters,
* and handling the **flat** and **label** query parameters.
*
* Additionally, the Request class will **forward current query parameters** to the request
* which is more convenient than calling {@link Piwik\Common::getRequestVar()} many times over.
*
* In most cases, using a Request object to query the API is the correct approach.
*
* ### Post-processing
*
* The return value of API methods undergo some extra processing before being returned by Request.
* To learn more about what happens to API results, read [this](/guides/piwiks-web-api#extra-report-processing).
*
* ### Output Formats
*
* The value returned by Request will be serialized to a certain format before being returned.
* To see the list of supported output formats, read [this](/guides/piwiks-web-api#output-formats).
*
* ### Examples
*
* **Basic Usage**
*
* $request = new Request('method=UserSettings.getWideScreen&idSite=1&date=yesterday&period=week'
* . '&format=xml&filter_limit=5&filter_offset=0')
* $result = $request->process();
* echo $result;
*
* **Getting a unrendered DataTable**
*
* // use the convenience method 'processRequest'
* $dataTable = Request::processRequest('UserSettings.getWideScreen', array(
* 'idSite' => 1,
* 'date' => 'yesterday',
* 'period' => 'week',
* 'filter_limit' => 5,
* 'filter_offset' => 0
*
* 'format' => 'original', // this is the important bit
* ));
* echo "This DataTable has " . $dataTable->getRowsCount() . " rows.";
*
* @see http://piwik.org/docs/analytics-api
* @api
*/
class Request
{
protected $request = null;
/**
* Converts the supplied request string into an array of query paramater name/value
* mappings. The current query parameters (everything in `$_GET` and `$_POST`) are
* forwarded to request array before it is returned.
*
* @param string|array $request The base request string or array, eg,
* `'module=UserSettings&action=getWidescreen'`.
* @return array
*/
static public function getRequestArrayFromString($request)
{
$defaultRequest = $_GET + $_POST;
$requestRaw = self::getRequestParametersGET();
if (!empty($requestRaw['segment'])) {
$defaultRequest['segment'] = $requestRaw['segment'];
}
$requestArray = $defaultRequest;
if (!is_null($request)) {
if (is_array($request)) {
$url = array();
foreach ($request as $key => $value) {
$url[] = $key . "=" . $value;
}
$request = implode("&", $url);
}
$request = trim($request);
$request = str_replace(array("\n", "\t"), '', $request);
$requestParsed = UrlHelper::getArrayFromQueryString($request);
$requestArray = $requestParsed + $defaultRequest;
}
foreach ($requestArray as &$element) {
if (!is_array($element)) {
$element = trim($element);
}
}
return $requestArray;
}
/**
* Constructor.
*
* @param string|array $request Query string that defines the API call (must at least contain a **method** parameter),
* eg, `'method=UserSettings.getWideScreen&idSite=1&date=yesterday&period=week&format=xml'`
* If a request is not provided, then we use the values in the `$_GET` and `$_POST`
* superglobals.
*/
public function __construct($request = null)
{
$this->request = self::getRequestArrayFromString($request);
$this->sanitizeRequest();
}
/**
* For backward compatibility: Piwik API still works if module=Referers,
* we rewrite to correct renamed plugin: Referrers
*
* @param $module
* @return string
* @ignore
*/
public static function renameModule($module)
{
$moduleToRedirect = array(
'Referers' => 'Referrers',
'PDFReports' => 'ScheduledReports',
);
if (isset($moduleToRedirect[$module])) {
return $moduleToRedirect[$module];
}
return $module;
}
/**
* Make sure that the request contains no logical errors
*/
private function sanitizeRequest()
{
// The label filter does not work with expanded=1 because the data table IDs have a different meaning
// depending on whether the table has been loaded yet. expanded=1 causes all tables to be loaded, which
// is why the label filter can't descend when a recursive label has been requested.
// To fix this, we remove the expanded parameter if a label parameter is set.
if (isset($this->request['label']) && !empty($this->request['label'])
&& isset($this->request['expanded']) && $this->request['expanded']
) {
unset($this->request['expanded']);
}
}
/**
* Dispatches the API request to the appropriate API method and returns the result
* after post-processing.
*
* Post-processing includes:
*
* - flattening if **flat** is 0
* - running generic filters unless **disable_generic_filters** is set to 1
* - URL decoding label column values
* - running queued filters unless **disable_queued_filters** is set to 1
* - removing columns based on the values of the **hideColumns** and **showColumns** query parameters
* - filtering rows if the **label** query parameter is set
* - converting the result to the appropriate format (ie, XML, JSON, etc.)
*
* If `'original'` is supplied for the output format, the result is returned as a PHP
* object.
*
* @throws PluginDeactivatedException if the module plugin is not activated.
* @throws Exception if the requested API method cannot be called, if required parameters for the
* API method are missing or if the API method throws an exception and the **format**
* query parameter is **original**.
* @return DataTable|Map|string The data resulting from the API call.
*/
public function process()
{
// read the format requested for the output data
$outputFormat = strtolower(Common::getRequestVar('format', 'xml', 'string', $this->request));
// create the response
$response = new ResponseBuilder($outputFormat, $this->request);
try {
// read parameters
$moduleMethod = Common::getRequestVar('method', null, 'string', $this->request);
list($module, $method) = $this->extractModuleAndMethod($moduleMethod);
$module = $this->renameModule($module);
if (!\Piwik\Plugin\Manager::getInstance()->isPluginActivated($module)) {
throw new PluginDeactivatedException($module);
}
$apiClassName = $this->getClassNameAPI($module);
self::reloadAuthUsingTokenAuth($this->request);
// call the method
$returnedValue = Proxy::getInstance()->call($apiClassName, $method, $this->request);
$toReturn = $response->getResponse($returnedValue, $module, $method);
} catch (Exception $e) {
$toReturn = $response->getResponseException($e);
}
return $toReturn;
}
/**
* Returns the name of a plugin's API class by plugin name.
*
* @param string $plugin The plugin name, eg, `'Referrers'`.
* @return string The fully qualified API class name, eg, `'\Piwik\Plugins\Referrers\API'`.
*/
static public function getClassNameAPI($plugin)
{
return sprintf('\Piwik\Plugins\%s\API', $plugin);
}
/**
* If the token_auth is found in the $request parameter,
* the current session will be authenticated using this token_auth.
* It will overwrite the previous Auth object.
*
* @param array $request If null, uses the default request ($_GET)
* @return void
* @ignore
*/
static public function reloadAuthUsingTokenAuth($request = null)
{
// if a token_auth is specified in the API request, we load the right permissions
$token_auth = Common::getRequestVar('token_auth', '', 'string', $request);
if ($token_auth) {
/**
* Triggered when authenticating an API request, but only if the **token_auth**
* query parameter is found in the request.
*
* Plugins that provide authentication capabilities should subscribe to this event
* and make sure the global authentication object (the object returned by `Registry::get('auth')`)
* is setup to use `$token_auth` when its `authenticate()` method is executed.
*
* @param string $token_auth The value of the **token_auth** query parameter.
*/
Piwik::postEvent('API.Request.authenticate', array($token_auth));
Access::getInstance()->reloadAccess();
SettingsServer::raiseMemoryLimitIfNecessary();
}
}
/**
* Returns array($class, $method) from the given string $class.$method
*
* @param string $parameter
* @throws Exception
* @return array
*/
private function extractModuleAndMethod($parameter)
{
$a = explode('.', $parameter);
if (count($a) != 2) {
throw new Exception("The method name is invalid. Expected 'module.methodName'");
}
return $a;
}
/**
* Helper method that processes an API request in one line using the variables in `$_GET`
* and `$_POST`.
*
* @param string $method The API method to call, ie, `'Actions.getPageTitles'`.
* @param array $paramOverride The parameter name-value pairs to use instead of what's
* in `$_GET` & `$_POST`.
* @return mixed The result of the API request. See {@link process()}.
*/
public static function processRequest($method, $paramOverride = array())
{
$params = array();
$params['format'] = 'original';
$params['module'] = 'API';
$params['method'] = $method;
$params = $paramOverride + $params;
// process request
$request = new Request($params);
return $request->process();
}
/**
* Returns the original request parameters in the current query string as an array mapping
* query parameter names with values. The result of this function will not be affected
* by any modifications to `$_GET` and will not include parameters in `$_POST`.
*
* @return array
*/
public static function getRequestParametersGET()
{
if (empty($_SERVER['QUERY_STRING'])) {
return array();
}
$GET = UrlHelper::getArrayFromQueryString($_SERVER['QUERY_STRING']);
return $GET;
}
/**
* Returns the URL for the current requested report w/o any filter parameters.
*
* @param string $module The API module.
* @param string $action The API action.
* @param array $queryParams Query parameter overrides.
* @return string
*/
public static function getBaseReportUrl($module, $action, $queryParams = array())
{
$params = array_merge($queryParams, array('module' => $module, 'action' => $action));
return Request::getCurrentUrlWithoutGenericFilters($params);
}
/**
* Returns the current URL without generic filter query parameters.
*
* @param array $params Query parameter values to override in the new URL.
* @return string
*/
public static function getCurrentUrlWithoutGenericFilters($params)
{
// unset all filter query params so the related report will show up in its default state,
// unless the filter param was in $queryParams
$genericFiltersInfo = DataTableGenericFilter::getGenericFiltersInformation();
foreach ($genericFiltersInfo as $filter) {
foreach ($filter as $queryParamName => $queryParamInfo) {
if (!isset($params[$queryParamName])) {
$params[$queryParamName] = null;
}
}
}
return Url::getCurrentQueryStringWithParametersModified($params);
}
/**
* Returns whether the DataTable result will have to be expanded for the
* current request before rendering.
*
* @return bool
* @ignore
*/
public static function shouldLoadExpanded()
{
// if filter_column_recursive & filter_pattern_recursive are supplied, and flat isn't supplied
// we have to load all the child subtables.
return Common::getRequestVar('filter_column_recursive', false) !== false
&& Common::getRequestVar('filter_pattern_recursive', false) !== false
&& !self::shouldLoadFlatten();
}
/**
* @return bool
*/
public static function shouldLoadFlatten()
{
return Common::getRequestVar('flat', false) == 1;
}
/**
* Returns the segment query parameter from the original request, without modifications.
*
* @return array|bool
*/
static public function getRawSegmentFromRequest()
{
// we need the URL encoded segment parameter, we fetch it from _SERVER['QUERY_STRING'] instead of default URL decoded _GET
$segmentRaw = false;
$segment = Common::getRequestVar('segment', '', 'string');
if (!empty($segment)) {
$request = Request::getRequestParametersGET();
if (!empty($request['segment'])) {
$segmentRaw = $request['segment'];
}
}
return $segmentRaw;
}
}

478
www/analytics/core/API/ResponseBuilder.php Normal file
View file

@ -0,0 +1,478 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\API;
use Exception;
use Piwik\API\DataTableManipulator\Flattener;
use Piwik\API\DataTableManipulator\LabelFilter;
use Piwik\API\DataTableManipulator\ReportTotalsCalculator;
use Piwik\Common;
use Piwik\DataTable\Renderer\Json;
use Piwik\DataTable\Renderer;
use Piwik\DataTable\Simple;
use Piwik\DataTable;
/**
*/
class ResponseBuilder
{
private $request = null;
private $outputFormat = null;
private $apiModule = false;
private $apiMethod = false;
/**
* @param string $outputFormat
* @param array $request
*/
public function __construct($outputFormat, $request = array())
{
$this->request = $request;
$this->outputFormat = $outputFormat;
}
/**
* This method processes the data resulting from the API call.
*
* - If the data resulted from the API call is a DataTable then
* - we apply the standard filters if the parameters have been found
* in the URL. For example to offset,limit the Table you can add the following parameters to any API
* call that returns a DataTable: filter_limit=10&filter_offset=20
* - we apply the filters that have been previously queued on the DataTable
* @see DataTable::queueFilter()
* - we apply the renderer that generate the DataTable in a given format (XML, PHP, HTML, JSON, etc.)
* the format can be changed using the 'format' parameter in the request.
* Example: format=xml
*
* - If there is nothing returned (void) we display a standard success message
*
* - If there is a PHP array returned, we try to convert it to a dataTable
* It is then possible to convert this datatable to any requested format (xml/etc)
*
* - If a bool is returned we convert to a string (true is displayed as 'true' false as 'false')
*
* - If an integer / float is returned, we simply return it
*
* @param mixed $value The initial returned value, before post process. If set to null, success response is returned.
* @param bool|string $apiModule The API module that was called
* @param bool|string $apiMethod The API method that was called
* @return mixed Usually a string, but can still be a PHP data structure if the format requested is 'original'
*/
public function getResponse($value = null, $apiModule = false, $apiMethod = false)
{
$this->apiModule = $apiModule;
$this->apiMethod = $apiMethod;
if($this->outputFormat == 'original') {
@header('Content-Type: text/plain; charset=utf-8');
}
return $this->renderValue($value);
}
/**
* Returns an error $message in the requested $format
*
* @param Exception $e
* @throws Exception
* @return string
*/
public function getResponseException(Exception $e)
{
$format = strtolower($this->outputFormat);
if ($format == 'original') {
throw $e;
}
try {
$renderer = Renderer::factory($format);
} catch (Exception $exceptionRenderer) {
return "Error: " . $e->getMessage() . " and: " . $exceptionRenderer->getMessage();
}
$e = $this->decorateExceptionWithDebugTrace($e);
$renderer->setException($e);
if ($format == 'php') {
$renderer->setSerialize($this->caseRendererPHPSerialize());
}
return $renderer->renderException();
}
/**
* @param $value
* @return string
*/
protected function renderValue($value)
{
// when null or void is returned from the api call, we handle it as a successful operation
if (!isset($value)) {
return $this->handleSuccess();
}
// If the returned value is an object DataTable we
// apply the set of generic filters if asked in the URL
// and we render the DataTable according to the format specified in the URL
if ($value instanceof DataTable
|| $value instanceof DataTable\Map
) {
return $this->handleDataTable($value);
}
// Case an array is returned from the API call, we convert it to the requested format
// - if calling from inside the application (format = original)
// => the data stays unchanged (ie. a standard php array or whatever data structure)
// - if any other format is requested, we have to convert this data structure (which we assume
// to be an array) to a DataTable in order to apply the requested DataTable_Renderer (for example XML)
if (is_array($value)) {
return $this->handleArray($value);
}
// original data structure requested, we return without process
if ($this->outputFormat == 'original') {
return $value;
}
if (is_object($value)
|| is_resource($value)
) {
return $this->getResponseException(new Exception('The API cannot handle this data structure.'));
}
// bool // integer // float // serialized object
return $this->handleScalar($value);
}
/**
* @param Exception $e
* @return Exception
*/
protected function decorateExceptionWithDebugTrace(Exception $e)
{
// If we are in tests, show full backtrace
if (defined('PIWIK_PATH_TEST_TO_ROOT')) {
if (\Piwik_ShouldPrintBackTraceWithMessage()) {
$message = $e->getMessage() . " in \n " . $e->getFile() . ":" . $e->getLine() . " \n " . $e->getTraceAsString();
} else {
$message = $e->getMessage() . "\n \n --> To temporarily debug this error further, set const PIWIK_PRINT_ERROR_BACKTRACE=true; in index.php";
}
return new Exception($message);
}
return $e;
}
/**
* Returns true if the user requested to serialize the output data (&serialize=1 in the request)
*
* @param mixed $defaultSerializeValue Default value in case the user hasn't specified a value
* @return bool
*/
protected function caseRendererPHPSerialize($defaultSerializeValue = 1)
{
$serialize = Common::getRequestVar('serialize', $defaultSerializeValue, 'int', $this->request);
if ($serialize) {
return true;
}
return false;
}
/**
* Apply the specified renderer to the DataTable
*
* @param DataTable|array $dataTable
* @return string
*/
protected function getRenderedDataTable($dataTable)
{
$format = strtolower($this->outputFormat);
// if asked for original dataStructure
if ($format == 'original') {
// by default "original" data is not serialized
if ($this->caseRendererPHPSerialize($defaultSerialize = 0)) {
$dataTable = serialize($dataTable);
}
return $dataTable;
}
$method = Common::getRequestVar('method', '', 'string', $this->request);
$renderer = Renderer::factory($format);
$renderer->setTable($dataTable);
$renderer->setRenderSubTables(Common::getRequestVar('expanded', false, 'int', $this->request));
$renderer->setHideIdSubDatableFromResponse(Common::getRequestVar('hideIdSubDatable', false, 'int', $this->request));
if ($format == 'php') {
$renderer->setSerialize($this->caseRendererPHPSerialize());
$renderer->setPrettyDisplay(Common::getRequestVar('prettyDisplay', false, 'int', $this->request));
} else if ($format == 'html') {
$renderer->setTableId($this->request['method']);
} else if ($format == 'csv' || $format == 'tsv') {
$renderer->setConvertToUnicode(Common::getRequestVar('convertToUnicode', true, 'int', $this->request));
}
// prepare translation of column names
if ($format == 'html' || $format == 'csv' || $format == 'tsv' || $format = 'rss') {
$renderer->setApiMethod($method);
$renderer->setIdSite(Common::getRequestVar('idSite', false, 'int', $this->request));
$renderer->setTranslateColumnNames(Common::getRequestVar('translateColumnNames', false, 'int', $this->request));
}
return $renderer->render();
}
/**
* Returns a success $message in the requested $format
*
* @param string $message
* @return string
*/
protected function handleSuccess($message = 'ok')
{
// return a success message only if no content has already been buffered, useful when APIs return raw text or html content to the browser
if (!ob_get_contents()) {
switch ($this->outputFormat) {
case 'xml':
@header("Content-Type: text/xml;charset=utf-8");
$return =
"<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n" .
"<result>\n" .
"\t<success message=\"" . $message . "\" />\n" .
"</result>";
break;
case 'json':
@header("Content-Type: application/json");
$return = '{"result":"success", "message":"' . $message . '"}';
break;
case 'php':
$return = array('result' => 'success', 'message' => $message);
if ($this->caseRendererPHPSerialize()) {
$return = serialize($return);
}
break;
case 'csv':
@header("Content-Type: application/vnd.ms-excel");
@header("Content-Disposition: attachment; filename=piwik-report-export.csv");
$return = "message\n" . $message;
break;
default:
$return = 'Success:' . $message;
break;
}
return $return;
}
}
/**
* Converts the given scalar to an data table
*
* @param mixed $scalar
* @return string
*/
protected function handleScalar($scalar)
{
$dataTable = new Simple();
$dataTable->addRowsFromArray(array($scalar));
return $this->getRenderedDataTable($dataTable);
}
/**
* Handles the given data table
*
* @param DataTable $datatable
* @return string
*/
protected function handleDataTable($datatable)
{
// if requested, flatten nested tables
if (Common::getRequestVar('flat', '0', 'string', $this->request) == '1') {
$flattener = new Flattener($this->apiModule, $this->apiMethod, $this->request);
if (Common::getRequestVar('include_aggregate_rows', '0', 'string', $this->request) == '1') {
$flattener->includeAggregateRows();
}
$datatable = $flattener->flatten($datatable);
}
if (1 == Common::getRequestVar('totals', '1', 'integer', $this->request)) {
$genericFilter = new ReportTotalsCalculator($this->apiModule, $this->apiMethod, $this->request);
$datatable = $genericFilter->calculate($datatable);
}
// if the flag disable_generic_filters is defined we skip the generic filters
if (0 == Common::getRequestVar('disable_generic_filters', '0', 'string', $this->request)) {
$genericFilter = new DataTableGenericFilter($this->request);
$genericFilter->filter($datatable);
}
// we automatically safe decode all datatable labels (against xss)
$datatable->queueFilter('SafeDecodeLabel');
// if the flag disable_queued_filters is defined we skip the filters that were queued
if (Common::getRequestVar('disable_queued_filters', 0, 'int', $this->request) == 0) {
$datatable->applyQueuedFilters();
}
// use the ColumnDelete filter if hideColumns/showColumns is provided (must be done
// after queued filters are run so processed metrics can be removed, too)
$hideColumns = Common::getRequestVar('hideColumns', '', 'string', $this->request);
$showColumns = Common::getRequestVar('showColumns', '', 'string', $this->request);
if ($hideColumns !== '' || $showColumns !== '') {
$datatable->filter('ColumnDelete', array($hideColumns, $showColumns));
}
// apply label filter: only return rows matching the label parameter (more than one if more than one label)
$label = $this->getLabelFromRequest($this->request);
if (!empty($label)) {
$addLabelIndex = Common::getRequestVar('labelFilterAddLabelIndex', 0, 'int', $this->request) == 1;
$filter = new LabelFilter($this->apiModule, $this->apiMethod, $this->request);
$datatable = $filter->filter($label, $datatable, $addLabelIndex);
}
return $this->getRenderedDataTable($datatable);
}
/**
* Converts the given simple array to a data table
*
* @param array $array
* @return string
*/
protected function handleArray($array)
{
if ($this->outputFormat == 'original') {
// we handle the serialization. Because some php array have a very special structure that
// couldn't be converted with the automatic DataTable->addRowsFromSimpleArray
// the user may want to request the original PHP data structure serialized by the API
// in case he has to setup serialize=1 in the URL
if ($this->caseRendererPHPSerialize($defaultSerialize = 0)) {
return serialize($array);
}
return $array;
}
$multiDimensional = $this->handleMultiDimensionalArray($array);
if ($multiDimensional !== false) {
return $multiDimensional;
}
return $this->getRenderedDataTable($array);
}
/**
* Is this a multi dimensional array?
* Multi dim arrays are not supported by the Datatable renderer.
* We manually render these.
*
* array(
* array(
* 1,
* 2 => array( 1,
* 2
* )
* ),
* array( 2,
* 3
* )
* );
*
* @param array $array
* @return string|bool false if it isn't a multidim array
*/
protected function handleMultiDimensionalArray($array)
{
$first = reset($array);
foreach ($array as $first) {
if (is_array($first)) {
foreach ($first as $key => $value) {
// Yes, this is a multi dim array
if (is_array($value)) {
switch ($this->outputFormat) {
case 'json':
@header("Content-Type: application/json");
return self::convertMultiDimensionalArrayToJson($array);
break;
case 'php':
if ($this->caseRendererPHPSerialize($defaultSerialize = 0)) {
return serialize($array);
}
return $array;
case 'xml':
@header("Content-Type: text/xml;charset=utf-8");
return $this->getRenderedDataTable($array);
default:
break;
}
}
}
}
}
return false;
}
/**
* Render a multidimensional array to Json
* Handle DataTable|Set elements in the first dimension only, following case does not work:
* array(
* array(
* DataTable,
* 2 => array(
* 1,
* 2
* ),
* ),
* );
*
* @param array $array can contain scalar, arrays, DataTable and Set
* @return string
*/
public static function convertMultiDimensionalArrayToJson($array)
{
$jsonRenderer = new Json();
$jsonRenderer->setTable($array);
$renderedReport = $jsonRenderer->render();
return $renderedReport;
}
/**
* Returns the value for the label query parameter which can be either a string
* (ie, label=...) or array (ie, label[]=...).
*
* @param array $request
* @return array
*/
static public function getLabelFromRequest($request)
{
$label = Common::getRequestVar('label', array(), 'array', $request);
if (empty($label)) {
$label = Common::getRequestVar('label', '', 'string', $request);
if (!empty($label)) {
$label = array($label);
}
}
$label = self::unsanitizeLabelParameter($label);
return $label;
}
static public function unsanitizeLabelParameter($label)
{
// this is needed because Proxy uses Common::getRequestVar which in turn
// uses Common::sanitizeInputValue. This causes the > that separates recursive labels
// to become &gt; and we need to undo that here.
$label = Common::unsanitizeInputValues($label);
return $label;
}
}

418
www/analytics/core/Access.php Normal file
View file

@ -0,0 +1,418 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik;
use Piwik\Db;
use Piwik\Plugins\UsersManager\API as APIUsersManager;
/**
* Singleton that manages user access to Piwik resources.
*
* To check whether a user has access to a resource, use one of the {@link Piwik Piwik::checkUser...}
* methods.
*
* In Piwik there are four different access levels:
*
* - **no access**: Users with this access level cannot view the resource.
* - **view access**: Users with this access level can view the resource, but cannot modify it.
* - **admin access**: Users with this access level can view and modify the resource.
* - **Super User access**: Only the Super User has this access level. It means the user can do
* whatever he/she wants.
*
* Super user access is required to set some configuration options.
* All other options are specific to the user or to a website.
*
* Access is granted per website. Uses with access for a website can view all
* data associated with that website.
*
*/
class Access
{
private static $instance = null;
/**
* Gets the singleton instance. Creates it if necessary.
*/
public static function getInstance()
{
if (self::$instance == null) {
self::$instance = new self;
Piwik::postEvent('Access.createAccessSingleton', array(&self::$instance));
}
return self::$instance;
}
/**
* Sets the singleton instance. For testing purposes.
*/
public static function setSingletonInstance($instance)
{
self::$instance = $instance;
}
/**
* Array of idsites available to the current user, indexed by permission level
* @see getSitesIdWith*()
*
* @var array
*/
protected $idsitesByAccess = null;
/**
* Login of the current user
*
* @var string
*/
protected $login = null;
/**
* token_auth of the current user
*
* @var string
*/
protected $token_auth = null;
/**
* Defines if the current user is the Super User
* @see hasSuperUserAccess()
*
* @var bool
*/
protected $hasSuperUserAccess = false;
/**
* List of available permissions in Piwik
*
* @var array
*/
private static $availableAccess = array('noaccess', 'view', 'admin', 'superuser');
/**
* Authentification object (see Auth)
*
* @var Auth
*/
private $auth = null;
/**
* Returns the list of the existing Access level.
* Useful when a given API method requests a given acccess Level.
* We first check that the required access level exists.
*
* @return array
*/
public static function getListAccess()
{
return self::$availableAccess;
}
/**
* Constructor
*/
public function __construct()
{
$this->idsitesByAccess = array(
'view' => array(),
'admin' => array(),
'superuser' => array()
);
}
/**
* Loads the access levels for the current user.
*
* Calls the authentication method to try to log the user in the system.
* If the user credentials are not correct we don't load anything.
* If the login/password is correct the user is either the SuperUser or a normal user.
* We load the access levels for this user for all the websites.
*
* @param null|Auth $auth Auth adapter
* @return bool true on success, false if reloading access failed (when auth object wasn't specified and user is not enforced to be Super User)
*/
public function reloadAccess(Auth $auth = null)
{
if (!is_null($auth)) {
$this->auth = $auth;
}
// if the Auth wasn't set, we may be in the special case of setSuperUser(), otherwise we fail
if (is_null($this->auth)) {
if ($this->hasSuperUserAccess()) {
return $this->reloadAccessSuperUser();
}
return false;
}
// access = array ( idsite => accessIdSite, idsite2 => accessIdSite2)
$result = $this->auth->authenticate();
if (!$result->wasAuthenticationSuccessful()) {
return false;
}
$this->login = $result->getIdentity();
$this->token_auth = $result->getTokenAuth();
// case the superUser is logged in
if ($result->hasSuperUserAccess()) {
return $this->reloadAccessSuperUser();
}
// in case multiple calls to API using different tokens, we ensure we reset it as not SU
$this->setSuperUserAccess(false);
// we join with site in case there are rows in access for an idsite that doesn't exist anymore
// (backward compatibility ; before we deleted the site without deleting rows in _access table)
$accessRaw = $this->getRawSitesWithSomeViewAccess($this->login);
foreach ($accessRaw as $access) {
$this->idsitesByAccess[$access['access']][] = $access['idsite'];
}
return true;
}
public function getRawSitesWithSomeViewAccess($login)
{
return Db::fetchAll(self::getSqlAccessSite("access, t2.idsite"), $login);
}
/**
* Returns the SQL query joining sites and access table for a given login
*
* @param string $select Columns or expression to SELECT FROM table, eg. "MIN(ts_created)"
* @return string SQL query
*/
public static function getSqlAccessSite($select)
{
return "SELECT " . $select . "
FROM " . Common::prefixTable('access') . " as t1
JOIN " . Common::prefixTable('site') . " as t2 USING (idsite) " .
" WHERE login = ?";
}
/**
* Reload Super User access
*
* @return bool
*/
protected function reloadAccessSuperUser()
{
$this->hasSuperUserAccess = true;
try {
$allSitesId = Plugins\SitesManager\API::getInstance()->getAllSitesId();
} catch (\Exception $e) {
$allSitesId = array();
}
$this->idsitesByAccess['superuser'] = $allSitesId;
return true;
}
/**
* We bypass the normal auth method and give the current user Super User rights.
* This should be very carefully used.
*
* @param bool $bool
*/
public function setSuperUserAccess($bool = true)
{
if ($bool) {
$this->reloadAccessSuperUser();
} else {
$this->hasSuperUserAccess = false;
$this->idsitesByAccess['superuser'] = array();
}
}
/**
* Returns true if the current user is logged in as the Super User
*
* @return bool
*/
public function hasSuperUserAccess()
{
return $this->hasSuperUserAccess;
}
/**
* Returns the current user login
*
* @return string|null
*/
public function getLogin()
{
return $this->login;
}
/**
* Returns the token_auth used to authenticate this user in the API
*
* @return string|null
*/
public function getTokenAuth()
{
return $this->token_auth;
}
/**
* Returns an array of ID sites for which the user has at least a VIEW access.
* Which means VIEW or ADMIN or SUPERUSER.
*
* @return array Example if the user is ADMIN for 4
* and has VIEW access for 1 and 7, it returns array(1, 4, 7);
*/
public function getSitesIdWithAtLeastViewAccess()
{
return array_unique(array_merge(
$this->idsitesByAccess['view'],
$this->idsitesByAccess['admin'],
$this->idsitesByAccess['superuser'])
);
}
/**
* Returns an array of ID sites for which the user has an ADMIN access.
*
* @return array Example if the user is ADMIN for 4 and 8
* and has VIEW access for 1 and 7, it returns array(4, 8);
*/
public function getSitesIdWithAdminAccess()
{
return array_unique(array_merge(
$this->idsitesByAccess['admin'],
$this->idsitesByAccess['superuser'])
);
}
/**
* Returns an array of ID sites for which the user has a VIEW access only.
*
* @return array Example if the user is ADMIN for 4
* and has VIEW access for 1 and 7, it returns array(1, 7);
* @see getSitesIdWithAtLeastViewAccess()
*/
public function getSitesIdWithViewAccess()
{
return $this->idsitesByAccess['view'];
}
/**
* Throws an exception if the user is not the SuperUser
*
* @throws \Piwik\NoAccessException
*/
public function checkUserHasSuperUserAccess()
{
if (!$this->hasSuperUserAccess()) {
throw new NoAccessException(Piwik::translate('General_ExceptionPrivilege', array("'superuser'")));
}
}
/**
* If the user doesn't have an ADMIN access for at least one website, throws an exception
*
* @throws \Piwik\NoAccessException
*/
public function checkUserHasSomeAdminAccess()
{
if ($this->hasSuperUserAccess()) {
return;
}
$idSitesAccessible = $this->getSitesIdWithAdminAccess();
if (count($idSitesAccessible) == 0) {
throw new NoAccessException(Piwik::translate('General_ExceptionPrivilegeAtLeastOneWebsite', array('admin')));
}
}
/**
* If the user doesn't have any view permission, throw exception
*
* @throws \Piwik\NoAccessException
*/
public function checkUserHasSomeViewAccess()
{
if ($this->hasSuperUserAccess()) {
return;
}
$idSitesAccessible = $this->getSitesIdWithAtLeastViewAccess();
if (count($idSitesAccessible) == 0) {
throw new NoAccessException(Piwik::translate('General_ExceptionPrivilegeAtLeastOneWebsite', array('view')));
}
}
/**
* This method checks that the user has ADMIN access for the given list of websites.
* If the user doesn't have ADMIN access for at least one website of the list, we throw an exception.
*
* @param int|array $idSites List of ID sites to check
* @throws \Piwik\NoAccessException If for any of the websites the user doesn't have an ADMIN access
*/
public function checkUserHasAdminAccess($idSites)
{
if ($this->hasSuperUserAccess()) {
return;
}
$idSites = $this->getIdSites($idSites);
$idSitesAccessible = $this->getSitesIdWithAdminAccess();
foreach ($idSites as $idsite) {
if (!in_array($idsite, $idSitesAccessible)) {
throw new NoAccessException(Piwik::translate('General_ExceptionPrivilegeAccessWebsite', array("'admin'", $idsite)));
}
}
}
/**
* This method checks that the user has VIEW or ADMIN access for the given list of websites.
* If the user doesn't have VIEW or ADMIN access for at least one website of the list, we throw an exception.
*
* @param int|array|string $idSites List of ID sites to check (integer, array of integers, string comma separated list of integers)
* @throws \Piwik\NoAccessException If for any of the websites the user doesn't have an VIEW or ADMIN access
*/
public function checkUserHasViewAccess($idSites)
{
if ($this->hasSuperUserAccess()) {
return;
}
$idSites = $this->getIdSites($idSites);
$idSitesAccessible = $this->getSitesIdWithAtLeastViewAccess();
foreach ($idSites as $idsite) {
if (!in_array($idsite, $idSitesAccessible)) {
throw new NoAccessException(Piwik::translate('General_ExceptionPrivilegeAccessWebsite', array("'view'", $idsite)));
}
}
}
/**
* @param int|array|string $idSites
* @return array
* @throws \Piwik\NoAccessException
*/
protected function getIdSites($idSites)
{
if ($idSites === 'all') {
$idSites = $this->getSitesIdWithAtLeastViewAccess();
}
$idSites = Site::getIdSitesFromIdSitesString($idSites);
if (empty($idSites)) {
throw new NoAccessException("The parameter 'idSite=' is missing from the request.");
}
return $idSites;
}
}
/**
* Exception thrown when a user doesn't have sufficient access to a resource.
*
* @api
*/
class NoAccessException extends \Exception
{
}

809
www/analytics/core/Archive.php Normal file
View file

@ -0,0 +1,809 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik;
use Piwik\Archive\Parameters;
use Piwik\ArchiveProcessor\Rules;
use Piwik\DataAccess\ArchiveSelector;
use Piwik\Period\Range;
/**
* The **Archive** class is used to query cached analytics statistics
* (termed "archive data").
*
* You can use **Archive** instances to get data that was archived for one or more sites,
* for one or more periods and one optional segment.
*
* If archive data is not found, this class will initiate the archiving process. [1](#footnote-1)
*
* **Archive** instances must be created using the {@link build()} factory method;
* they cannot be constructed.
*
* You can search for metrics (such as `nb_visits`) using the {@link getNumeric()} and
* {@link getDataTableFromNumeric()} methods. You can search for
* reports using the {@link getBlob()}, {@link getDataTable()} and {@link getDataTableExpanded()} methods.
*
* If you're creating an API that returns report data, you may want to use the
* {@link getDataTableFromArchive()} helper function.
*
* ### Learn more
*
* Learn more about _archiving_ [here](/guides/all-about-analytics-data).
*
* ### Limitations
*
* - You cannot get data for multiple range periods in a single query.
* - You cannot get data for periods of different types in a single query.
*
* ### Examples
*
* **_Querying metrics for an API method_**
*
* // one site and one period
* $archive = Archive::build($idSite = 1, $period = 'week', $date = '2013-03-08');
* return $archive->getDataTableFromNumeric(array('nb_visits', 'nb_actions'));
*
* // all sites and multiple dates
* $archive = Archive::build($idSite = 'all', $period = 'month', $date = '2013-01-02,2013-03-08');
* return $archive->getDataTableFromNumeric(array('nb_visits', 'nb_actions'));
*
* **_Querying and using metrics immediately_**
*
* // one site and one period
* $archive = Archive::build($idSite = 1, $period = 'week', $date = '2013-03-08');
* $data = $archive->getNumeric(array('nb_visits', 'nb_actions'));
*
* $visits = $data['nb_visits'];
* $actions = $data['nb_actions'];
*
* // ... do something w/ metric data ...
*
* // multiple sites and multiple dates
* $archive = Archive::build($idSite = '1,2,3', $period = 'month', $date = '2013-01-02,2013-03-08');
* $data = $archive->getNumeric('nb_visits');
*
* $janSite1Visits = $data['1']['2013-01-01,2013-01-31']['nb_visits'];
* $febSite1Visits = $data['1']['2013-02-01,2013-02-28']['nb_visits'];
* // ... etc.
*
* **_Querying for reports_**
*
* $archive = Archive::build($idSite = 1, $period = 'week', $date = '2013-03-08');
* $dataTable = $archive->getDataTable('MyPlugin_MyReport');
* // ... manipulate $dataTable ...
* return $dataTable;
*
* **_Querying a report for an API method_**
*
* public function getMyReport($idSite, $period, $date, $segment = false, $expanded = false)
* {
* $dataTable = Archive::getDataTableFromArchive('MyPlugin_MyReport', $idSite, $period, $date, $segment, $expanded);
* $dataTable->queueFilter('ReplaceColumnNames');
* return $dataTable;
* }
*
* **_Querying data for multiple range periods_**
*
* // get data for first range
* $archive = Archive::build($idSite = 1, $period = 'range', $date = '2013-03-08,2013-03-12');
* $dataTable = $archive->getDataTableFromNumeric(array('nb_visits', 'nb_actions'));
*
* // get data for second range
* $archive = Archive::build($idSite = 1, $period = 'range', $date = '2013-03-15,2013-03-20');
* $dataTable = $archive->getDataTableFromNumeric(array('nb_visits', 'nb_actions'));
*
* <a name="footnote-1"></a>
* [1]: The archiving process will not be launched if browser archiving is disabled
* and the current request came from a browser (and not the **archive.php** cron
* script).
*
*
* @api
*/
class Archive
{
const REQUEST_ALL_WEBSITES_FLAG = 'all';
const ARCHIVE_ALL_PLUGINS_FLAG = 'all';
const ID_SUBTABLE_LOAD_ALL_SUBTABLES = 'all';
/**
* List of archive IDs for the site, periods and segment we are querying with.
* Archive IDs are indexed by done flag and period, ie:
*
* array(
* 'done.Referrers' => array(
* '2010-01-01' => 1,
* '2010-01-02' => 2,
* ),
* 'done.VisitsSummary' => array(
* '2010-01-01' => 3,
* '2010-01-02' => 4,
* ),
* )
*
* or,
*
* array(
* 'done.all' => array(
* '2010-01-01' => 1,
* '2010-01-02' => 2
* )
* )
*
* @var array
*/
private $idarchives = array();
/**
* If set to true, the result of all get functions (ie, getNumeric, getBlob, etc.)
* will be indexed by the site ID, even if we're only querying data for one site.
*
* @var bool
*/
private $forceIndexedBySite;
/**
* If set to true, the result of all get functions (ie, getNumeric, getBlob, etc.)
* will be indexed by the period, even if we're only querying data for one period.
*
* @var bool
*/
private $forceIndexedByDate;
/**
* @var Parameters
*/
private $params;
/**
* @param Parameters $params
* @param bool $forceIndexedBySite Whether to force index the result of a query by site ID.
* @param bool $forceIndexedByDate Whether to force index the result of a query by period.
*/
protected function __construct(Parameters $params, $forceIndexedBySite = false,
$forceIndexedByDate = false)
{
$this->params = $params;
$this->forceIndexedBySite = $forceIndexedBySite;
$this->forceIndexedByDate = $forceIndexedByDate;
}
/**
* Returns a new Archive instance that will query archive data for the given set of
* sites and periods, using an optional Segment.
*
* This method uses data that is found in query parameters, so the parameters to this
* function can be string values.
*
* If you want to create an Archive instance with an array of Period instances, use
* {@link Archive::factory()}.
*
* @param string|int|array $idSites A single ID (eg, `'1'`), multiple IDs (eg, `'1,2,3'` or `array(1, 2, 3)`),
* or `'all'`.
* @param string $period 'day', `'week'`, `'month'`, `'year'` or `'range'`
* @param Date|string $strDate 'YYYY-MM-DD', magic keywords (ie, 'today'; {@link Date::factory()}
* or date range (ie, 'YYYY-MM-DD,YYYY-MM-DD').
* @param bool|false|string $segment Segment definition or false if no segment should be used. {@link Piwik\Segment}
* @param bool|false|string $_restrictSitesToLogin Used only when running as a scheduled task.
* @param bool $skipAggregationOfSubTables Whether the archive, when it is processed, should also aggregate all sub-tables
* @return Archive
*/
public static function build($idSites, $period, $strDate, $segment = false, $_restrictSitesToLogin = false, $skipAggregationOfSubTables = false)
{
$websiteIds = Site::getIdSitesFromIdSitesString($idSites, $_restrictSitesToLogin);
if (Period::isMultiplePeriod($strDate, $period)) {
$oPeriod = new Range($period, $strDate);
$allPeriods = $oPeriod->getSubperiods();
} else {
$timezone = count($websiteIds) == 1 ? Site::getTimezoneFor($websiteIds[0]) : false;
$oPeriod = Period::makePeriodFromQueryParams($timezone, $period, $strDate);
$allPeriods = array($oPeriod);
}
$segment = new Segment($segment, $websiteIds);
$idSiteIsAll = $idSites == self::REQUEST_ALL_WEBSITES_FLAG;
$isMultipleDate = Period::isMultiplePeriod($strDate, $period);
return Archive::factory($segment, $allPeriods, $websiteIds, $idSiteIsAll, $isMultipleDate, $skipAggregationOfSubTables);
}
/**
* Returns a new Archive instance that will query archive data for the given set of
* sites and periods, using an optional segment.
*
* This method uses an array of Period instances and a Segment instance, instead of strings
* like {@link build()}.
*
* If you want to create an Archive instance using data found in query parameters,
* use {@link build()}.
*
* @param Segment $segment The segment to use. For no segment, use `new Segment('', $idSites)`.
* @param array $periods An array of Period instances.
* @param array $idSites An array of site IDs (eg, `array(1, 2, 3)`).
* @param bool $idSiteIsAll Whether `'all'` sites are being queried or not. If true, then
* the result of querying functions will be indexed by site, regardless
* of whether `count($idSites) == 1`.
* @param bool $isMultipleDate Whether multiple dates are being queried or not. If true, then
* the result of querying functions will be indexed by period,
* regardless of whether `count($periods) == 1`.
* @param bool $skipAggregationOfSubTables Whether the archive should skip aggregation of all sub-tables
*
* @return Archive
*/
public static function factory(Segment $segment, array $periods, array $idSites, $idSiteIsAll = false, $isMultipleDate = false, $skipAggregationOfSubTables = false)
{
$forceIndexedBySite = false;
$forceIndexedByDate = false;
if ($idSiteIsAll || count($idSites) > 1) {
$forceIndexedBySite = true;
}
if (count($periods) > 1 || $isMultipleDate) {
$forceIndexedByDate = true;
}
$params = new Parameters($idSites, $periods, $segment, $skipAggregationOfSubTables);
return new Archive($params, $forceIndexedBySite, $forceIndexedByDate);
}
/**
* Queries and returns metric data in an array.
*
* If multiple sites were requested in {@link build()} or {@link factory()} the result will
* be indexed by site ID.
*
* If multiple periods were requested in {@link build()} or {@link factory()} the result will
* be indexed by period.
*
* The site ID index is always first, so if multiple sites & periods were requested, the result
* will be indexed by site ID first, then period.
*
* @param string|array $names One or more archive names, eg, `'nb_visits'`, `'Referrers_distinctKeywords'`,
* etc.
* @return false|numeric|array `false` if there is no data to return, a single numeric value if we're not querying
* for multiple sites/periods, or an array if multiple sites, periods or names are
* queried for.
*/
public function getNumeric($names)
{
$data = $this->get($names, 'numeric');
$resultIndices = $this->getResultIndices();
$result = $data->getIndexedArray($resultIndices);
// if only one metric is returned, just return it as a numeric value
if (empty($resultIndices)
&& count($result) <= 1
&& (!is_array($names) || count($names) == 1)
) {
$result = (float)reset($result); // convert to float in case $result is empty
}
return $result;
}
/**
* Queries and returns blob data in an array.
*
* Reports are stored in blobs as serialized arrays of {@link DataTable\Row} instances, but this
* data can technically be anything. In other words, you can store whatever you want
* as archive data blobs.
*
* If multiple sites were requested in {@link build()} or {@link factory()} the result will
* be indexed by site ID.
*
* If multiple periods were requested in {@link build()} or {@link factory()} the result will
* be indexed by period.
*
* The site ID index is always first, so if multiple sites & periods were requested, the result
* will be indexed by site ID first, then period.
*
* @param string|array $names One or more archive names, eg, `'Referrers_keywordBySearchEngine'`.
* @param null|string $idSubtable If we're returning serialized DataTable data, then this refers
* to the subtable ID to return. If set to 'all', all subtables
* of each requested report are returned.
* @return array An array of appropriately indexed blob data.
*/
public function getBlob($names, $idSubtable = null)
{
$data = $this->get($names, 'blob', $idSubtable);
return $data->getIndexedArray($this->getResultIndices());
}
/**
* Queries and returns metric data in a DataTable instance.
*
* If multiple sites were requested in {@link build()} or {@link factory()} the result will
* be a DataTable\Map that is indexed by site ID.
*
* If multiple periods were requested in {@link build()} or {@link factory()} the result will
* be a {@link DataTable\Map} that is indexed by period.
*
* The site ID index is always first, so if multiple sites & periods were requested, the result
* will be a {@link DataTable\Map} indexed by site ID which contains {@link DataTable\Map} instances that are
* indexed by period.
*
* _Note: Every DataTable instance returned will have at most one row in it. The contents of each
* row will be the requested metrics for the appropriate site and period._
*
* @param string|array $names One or more archive names, eg, 'nb_visits', 'Referrers_distinctKeywords',
* etc.
* @return DataTable|DataTable\Map A DataTable if multiple sites and periods were not requested.
* An appropriately indexed DataTable\Map if otherwise.
*/
public function getDataTableFromNumeric($names)
{
$data = $this->get($names, 'numeric');
return $data->getDataTable($this->getResultIndices());
}
/**
* Queries and returns one or more reports as DataTable instances.
*
* This method will query blob data that is a serialized array of of {@link DataTable\Row}'s and
* unserialize it.
*
* If multiple sites were requested in {@link build()} or {@link factory()} the result will
* be a {@link DataTable\Map} that is indexed by site ID.
*
* If multiple periods were requested in {@link build()} or {@link factory()} the result will
* be a DataTable\Map that is indexed by period.
*
* The site ID index is always first, so if multiple sites & periods were requested, the result
* will be a {@link DataTable\Map} indexed by site ID which contains {@link DataTable\Map} instances that are
* indexed by period.
*
* @param string $name The name of the record to get. This method can only query one record at a time.
* @param int|string|null $idSubtable The ID of the subtable to get (if any).
* @return DataTable|DataTable\Map A DataTable if multiple sites and periods were not requested.
* An appropriately indexed {@link DataTable\Map} if otherwise.
*/
public function getDataTable($name, $idSubtable = null)
{
$data = $this->get($name, 'blob', $idSubtable);
return $data->getDataTable($this->getResultIndices());
}
/**
* Queries and returns one report with all of its subtables loaded.
*
* If multiple sites were requested in {@link build()} or {@link factory()} the result will
* be a DataTable\Map that is indexed by site ID.
*
* If multiple periods were requested in {@link build()} or {@link factory()} the result will
* be a DataTable\Map that is indexed by period.
*
* The site ID index is always first, so if multiple sites & periods were requested, the result
* will be a {@link DataTable\Map indexed} by site ID which contains {@link DataTable\Map} instances that are
* indexed by period.
*
* @param string $name The name of the record to get.
* @param int|string|null $idSubtable The ID of the subtable to get (if any). The subtable will be expanded.
* @param int|null $depth The maximum number of subtable levels to load. If null, all levels are loaded.
* For example, if `1` is supplied, then the DataTable returned will have its subtables
* loaded. Those subtables, however, will NOT have their subtables loaded.
* @param bool $addMetadataSubtableId Whether to add the database subtable ID as metadata to each datatable,
* or not.
* @return DataTable|DataTable\Map
*/
public function getDataTableExpanded($name, $idSubtable = null, $depth = null, $addMetadataSubtableId = true)
{
$data = $this->get($name, 'blob', self::ID_SUBTABLE_LOAD_ALL_SUBTABLES);
return $data->getExpandedDataTable($this->getResultIndices(), $idSubtable, $depth, $addMetadataSubtableId);
}
/**
* Returns the list of plugins that archive the given reports.
*
* @param array $archiveNames
* @return array
*/
private function getRequestedPlugins($archiveNames)
{
$result = array();
foreach ($archiveNames as $name) {
$result[] = self::getPluginForReport($name);
}
return array_unique($result);
}
/**
* Returns an object describing the set of sites, the set of periods and the segment
* this Archive will query data for.
*
* @return Parameters
*/
public function getParams()
{
return $this->params;
}
/**
* Helper function that creates an Archive instance and queries for report data using
* query parameter data. API methods can use this method to reduce code redundancy.
*
* @param string $name The name of the report to return.
* @param int|string|array $idSite @see {@link build()}
* @param string $period @see {@link build()}
* @param string $date @see {@link build()}
* @param string $segment @see {@link build()}
* @param bool $expanded If true, loads all subtables. See {@link getDataTableExpanded()}
* @param int|null $idSubtable See {@link getDataTableExpanded()}
* @param bool $skipAggregationOfSubTables Whether or not we should skip the aggregation of all sub-tables and only aggregate parent DataTable.
* @param int|null $depth See {@link getDataTableExpanded()}
* @return DataTable|DataTable\Map See {@link getDataTable()} and
* {@link getDataTableExpanded()} for more
* information
*/
public static function getDataTableFromArchive($name, $idSite, $period, $date, $segment, $expanded,
$idSubtable = null, $skipAggregationOfSubTables = false, $depth = null)
{
Piwik::checkUserHasViewAccess($idSite);
if($skipAggregationOfSubTables && ($expanded || $idSubtable)) {
throw new \Exception("Not expected to skipAggregationOfSubTables when expanded=1 or idSubtable is set.");
}
$archive = Archive::build($idSite, $period, $date, $segment, $_restrictSitesToLogin = false, $skipAggregationOfSubTables);
if ($idSubtable === false) {
$idSubtable = null;
}
if ($expanded) {
$dataTable = $archive->getDataTableExpanded($name, $idSubtable, $depth);
} else {
$dataTable = $archive->getDataTable($name, $idSubtable);
}
$dataTable->queueFilter('ReplaceSummaryRowLabel');
return $dataTable;
}
private function appendIdSubtable($recordName, $id)
{
return $recordName . "_" . $id;
}
/**
* Queries archive tables for data and returns the result.
* @param array|string $archiveNames
* @param $archiveDataType
* @param null|int $idSubtable
* @return Archive\DataCollection
*/
private function get($archiveNames, $archiveDataType, $idSubtable = null)
{
if (!is_array($archiveNames)) {
$archiveNames = array($archiveNames);
}
// apply idSubtable
if ($idSubtable !== null
&& $idSubtable != self::ID_SUBTABLE_LOAD_ALL_SUBTABLES
) {
foreach ($archiveNames as &$name) {
$name = $this->appendIdsubtable($name, $idSubtable);
}
}
$result = new Archive\DataCollection(
$archiveNames, $archiveDataType, $this->params->getIdSites(), $this->params->getPeriods(), $defaultRow = null);
$archiveIds = $this->getArchiveIds($archiveNames);
if (empty($archiveIds)) {
return $result;
}
$loadAllSubtables = $idSubtable == self::ID_SUBTABLE_LOAD_ALL_SUBTABLES;
$archiveData = ArchiveSelector::getArchiveData($archiveIds, $archiveNames, $archiveDataType, $loadAllSubtables);
foreach ($archiveData as $row) {
// values are grouped by idsite (site ID), date1-date2 (date range), then name (field name)
$idSite = $row['idsite'];
$periodStr = $row['date1'] . "," . $row['date2'];
if ($archiveDataType == 'numeric') {
$value = $this->formatNumericValue($row['value']);
} else {
$value = $this->uncompress($row['value']);
$result->addMetadata($idSite, $periodStr, 'ts_archived', $row['ts_archived']);
}
$resultRow = & $result->get($idSite, $periodStr);
$resultRow[$row['name']] = $value;
}
return $result;
}
/**
* Returns archive IDs for the sites, periods and archive names that are being
* queried. This function will use the idarchive cache if it has the right data,
* query archive tables for IDs w/o launching archiving, or launch archiving and
* get the idarchive from ArchiveProcessor instances.
*/
private function getArchiveIds($archiveNames)
{
$plugins = $this->getRequestedPlugins($archiveNames);
// figure out which archives haven't been processed (if an archive has been processed,
// then we have the archive IDs in $this->idarchives)
$doneFlags = array();
$archiveGroups = array();
foreach ($plugins as $plugin) {
$doneFlag = $this->getDoneStringForPlugin($plugin);
$doneFlags[$doneFlag] = true;
if (!isset($this->idarchives[$doneFlag])) {
$archiveGroup = $this->getArchiveGroupOfPlugin($plugin);
if($archiveGroup == self::ARCHIVE_ALL_PLUGINS_FLAG) {
$archiveGroup = reset($plugins);
}
$archiveGroups[] = $archiveGroup;
}
}
$archiveGroups = array_unique($archiveGroups);
// cache id archives for plugins we haven't processed yet
if (!empty($archiveGroups)) {
if (!Rules::isArchivingDisabledFor($this->params->getIdSites(), $this->params->getSegment(), $this->getPeriodLabel())) {
$this->cacheArchiveIdsAfterLaunching($archiveGroups, $plugins);
} else {
$this->cacheArchiveIdsWithoutLaunching($plugins);
}
}
// order idarchives by the table month they belong to
$idArchivesByMonth = array();
foreach (array_keys($doneFlags) as $doneFlag) {
if (empty($this->idarchives[$doneFlag])) {
continue;
}
foreach ($this->idarchives[$doneFlag] as $dateRange => $idarchives) {
foreach ($idarchives as $id) {
$idArchivesByMonth[$dateRange][] = $id;
}
}
}
return $idArchivesByMonth;
}
/**
* Gets the IDs of the archives we're querying for and stores them in $this->archives.
* This function will launch the archiving process for each period/site/plugin if
* metrics/reports have not been calculated/archived already.
*
* @param array $archiveGroups @see getArchiveGroupOfReport
* @param array $plugins List of plugin names to archive.
*/
private function cacheArchiveIdsAfterLaunching($archiveGroups, $plugins)
{
$today = Date::today();
foreach ($this->params->getPeriods() as $period) {
$twoDaysBeforePeriod = $period->getDateStart()->subDay(2);
$twoDaysAfterPeriod = $period->getDateEnd()->addDay(2);
foreach ($this->params->getIdSites() as $idSite) {
$site = new Site($idSite);
// if the END of the period is BEFORE the website creation date
// we already know there are no stats for this period
// we add one day to make sure we don't miss the day of the website creation
if ($twoDaysAfterPeriod->isEarlier($site->getCreationDate())) {
Log::verbose("Archive site %s, %s (%s) skipped, archive is before the website was created.",
$idSite, $period->getLabel(), $period->getPrettyString());
continue;
}
// if the starting date is in the future we know there is no visiidsite = ?t
if ($twoDaysBeforePeriod->isLater($today)) {
Log::verbose("Archive site %s, %s (%s) skipped, archive is after today.",
$idSite, $period->getLabel(), $period->getPrettyString());
continue;
}
$this->prepareArchive($archiveGroups, $site, $period);
}
}
}
/**
* Gets the IDs of the archives we're querying for and stores them in $this->archives.
* This function will not launch the archiving process (and is thus much, much faster
* than cacheArchiveIdsAfterLaunching).
*
* @param array $plugins List of plugin names from which data is being requested.
*/
private function cacheArchiveIdsWithoutLaunching($plugins)
{
$idarchivesByReport = ArchiveSelector::getArchiveIds(
$this->params->getIdSites(), $this->params->getPeriods(), $this->params->getSegment(), $plugins, $this->params->isSkipAggregationOfSubTables());
// initialize archive ID cache for each report
foreach ($plugins as $plugin) {
$doneFlag = $this->getDoneStringForPlugin($plugin);
$this->initializeArchiveIdCache($doneFlag);
}
foreach ($idarchivesByReport as $doneFlag => $idarchivesByDate) {
foreach ($idarchivesByDate as $dateRange => $idarchives) {
foreach ($idarchives as $idarchive) {
$this->idarchives[$doneFlag][$dateRange][] = $idarchive;
}
}
}
}
/**
* Returns the done string flag for a plugin using this instance's segment & periods.
* @param string $plugin
* @return string
*/
private function getDoneStringForPlugin($plugin)
{
return Rules::getDoneStringFlagFor(
$this->params->getIdSites(),
$this->params->getSegment(),
$this->getPeriodLabel(),
$plugin,
$this->params->isSkipAggregationOfSubTables()
);
}
private function getPeriodLabel()
{
$periods = $this->params->getPeriods();
return reset($periods)->getLabel();
}
/**
* Returns an array describing what metadata to use when indexing a query result.
* For use with DataCollection.
*
* @return array
*/
private function getResultIndices()
{
$indices = array();
if (count($this->params->getIdSites()) > 1
|| $this->forceIndexedBySite
) {
$indices['site'] = 'idSite';
}
if (count($this->params->getPeriods()) > 1
|| $this->forceIndexedByDate
) {
$indices['period'] = 'date';
}
return $indices;
}
private function formatNumericValue($value)
{
// If there is no dot, we return as is
// Note: this could be an integer bigger than 32 bits
if (strpos($value, '.') === false) {
if ($value === false) {
return 0;
}
return (float)$value;
}
// Round up the value with 2 decimals
// we cast the result as float because returns false when no visitors
return round((float)$value, 2);
}
private function uncompress($data)
{
return @gzuncompress($data);
}
/**
* Initializes the archive ID cache ($this->idarchives) for a particular 'done' flag.
*
* It is necessary that each archive ID caching function call this method for each
* unique 'done' flag it encounters, since the getArchiveIds function determines
* whether archiving should be launched based on whether $this->idarchives has a
* an entry for a specific 'done' flag.
*
* If this function is not called, then periods with no visits will not add
* entries to the cache. If the archive is used again, SQL will be executed to
* try and find the archive IDs even though we know there are none.
*/
private function initializeArchiveIdCache($doneFlag)
{
if (!isset($this->idarchives[$doneFlag])) {
$this->idarchives[$doneFlag] = array();
}
}
/**
* Returns the archiving group identifier given a plugin.
*
* More than one plugin can be called at once when archiving. In such a case
* we don't want to launch archiving three times for three plugins if doing
* it once is enough, so getArchiveIds makes sure to get the archive group of
* all reports.
*
* If the period isn't a range, then all plugins' archiving code is executed.
* If the period is a range, then archiving code is executed individually for
* each plugin.
*/
private function getArchiveGroupOfPlugin($plugin)
{
if ($this->getPeriodLabel() != 'range') {
return self::ARCHIVE_ALL_PLUGINS_FLAG;
}
return $plugin;
}
/**
* Returns the name of the plugin that archives a given report.
*
* @param string $report Archive data name, eg, `'nb_visits'`, `'UserSettings_...'`, etc.
* @return string Plugin name.
* @throws \Exception If a plugin cannot be found or if the plugin for the report isn't
* activated.
*/
private static function getPluginForReport($report)
{
// Core metrics are always processed in Core, for the requested date/period/segment
if (in_array($report, Metrics::getVisitsMetricNames())) {
$report = 'VisitsSummary_CoreMetrics';
} // Goal_* metrics are processed by the Goals plugin (HACK)
else if (strpos($report, 'Goal_') === 0) {
$report = 'Goals_Metrics';
} else if (strrpos($report, '_returning') === strlen($report) - strlen('_returning')) { // HACK
$report = 'VisitFrequency_Metrics';
}
$plugin = substr($report, 0, strpos($report, '_'));
if (empty($plugin)
|| !\Piwik\Plugin\Manager::getInstance()->isPluginActivated($plugin)
) {
throw new \Exception("Error: The report '$report' was requested but it is not available at this stage."
. " (Plugin '$plugin' is not activated.)");
}
return $plugin;
}
/**
* @param $archiveGroups
* @param $site
* @param $period
*/
private function prepareArchive(array $archiveGroups, Site $site, Period $period)
{
$parameters = new ArchiveProcessor\Parameters($site, $period, $this->params->getSegment(), $this->params->isSkipAggregationOfSubTables());
$archiveLoader = new ArchiveProcessor\Loader($parameters);
$periodString = $period->getRangeString();
// process for each plugin as well
foreach ($archiveGroups as $plugin) {
$doneFlag = $this->getDoneStringForPlugin($plugin);
$this->initializeArchiveIdCache($doneFlag);
$idArchive = $archiveLoader->prepareArchive($plugin);
if($idArchive) {
$this->idarchives[$doneFlag][$periodString][] = $idArchive;
}
}
}
}

336
www/analytics/core/Archive/DataCollection.php Normal file
View file

@ -0,0 +1,336 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\Archive;
use Exception;
use Piwik\DataTable;
/**
* This class is used to hold and transform archive data for the Archive class.
*
* Archive data is loaded into an instance of this type, can be indexed by archive
* metadata (such as the site ID, period string, etc.), and can be transformed into
* DataTable and Map instances.
*/
class DataCollection
{
const METADATA_CONTAINER_ROW_KEY = '_metadata';
/**
* The archive data, indexed first by site ID and then by period date range. Eg,
*
* array(
* '0' => array(
* array(
* '2012-01-01,2012-01-01' => array(...),
* '2012-01-02,2012-01-02' => array(...),
* )
* ),
* '1' => array(
* array(
* '2012-01-01,2012-01-01' => array(...),
* )
* )
* )
*
* Archive data can be either a numeric value or a serialized string blob. Every
* piece of archive data is associated by it's archive name. For example,
* the array(...) above could look like:
*
* array(
* 'nb_visits' => 1,
* 'nb_actions' => 2
* )
*
* There is a special element '_metadata' in data rows that holds values treated
* as DataTable metadata.
*/
private $data = array();
/**
* The whole list of metric/record names that were used in the archive query.
*
* @var array
*/
private $dataNames;
/**
* The type of data that was queried for (ie, "blob" or "numeric").
*
* @var string
*/
private $dataType;
/**
* The default values to use for each metric/record name that's being queried
* for.
*
* @var array
*/
private $defaultRow;
/**
* The list of all site IDs that were queried for.
*
* @var array
*/
private $sitesId;
/**
* The list of all periods that were queried for. Each period is associated with
* the period's range string. Eg,
*
* array(
* '2012-01-01,2012-01-31' => new Period(...),
* '2012-02-01,2012-02-28' => new Period(...),
* )
*
* @var \Piwik\Period[]
*/
private $periods;
/**
* Constructor.
*
* @param array $dataNames @see $this->dataNames
* @param string $dataType @see $this->dataType
* @param array $sitesId @see $this->sitesId
* @param \Piwik\Period[] $periods @see $this->periods
* @param array $defaultRow @see $this->defaultRow
*/
public function __construct($dataNames, $dataType, $sitesId, $periods, $defaultRow = null)
{
$this->dataNames = $dataNames;
$this->dataType = $dataType;
if ($defaultRow === null) {
$defaultRow = array_fill_keys($dataNames, 0);
}
$this->sitesId = $sitesId;
foreach ($periods as $period) {
$this->periods[$period->getRangeString()] = $period;
}
$this->defaultRow = $defaultRow;
}
/**
* Returns a reference to the data for a specific site & period. If there is
* no data for the given site ID & period, it is set to the default row.
*
* @param int $idSite
* @param string $period eg, '2012-01-01,2012-01-31'
*/
public function &get($idSite, $period)
{
if (!isset($this->data[$idSite][$period])) {
$this->data[$idSite][$period] = $this->defaultRow;
}
return $this->data[$idSite][$period];
}
/**
* Adds a new metadata to the data for specific site & period. If there is no
* data for the given site ID & period, it is set to the default row.
*
* Note: Site ID and period range string are two special types of metadata. Since
* the data stored in this class is indexed by site & period, this metadata is not
* stored in individual data rows.
*
* @param int $idSite
* @param string $period eg, '2012-01-01,2012-01-31'
* @param string $name The metadata name.
* @param mixed $value The metadata name.
*/
public function addMetadata($idSite, $period, $name, $value)
{
$row = & $this->get($idSite, $period);
$row[self::METADATA_CONTAINER_ROW_KEY][$name] = $value;
}
/**
* Returns archive data as an array indexed by metadata.
*
* @param array $resultIndices An array mapping metadata names to pretty labels
* for them. Each archive data row will be indexed
* by the metadata specified here.
*
* Eg, array('site' => 'idSite', 'period' => 'Date')
* @return array
*/
public function getIndexedArray($resultIndices)
{
$indexKeys = array_keys($resultIndices);
$result = $this->createOrderedIndex($indexKeys);
foreach ($this->data as $idSite => $rowsByPeriod) {
foreach ($rowsByPeriod as $period => $row) {
// FIXME: This hack works around a strange bug that occurs when getting
// archive IDs through ArchiveProcessing instances. When a table
// does not already exist, for some reason the archive ID for
// today (or from two days ago) will be added to the Archive
// instances list. The Archive instance will then select data
// for periods outside of the requested set.
// working around the bug here, but ideally, we need to figure
// out why incorrect idarchives are being selected.
if (empty($this->periods[$period])) {
continue;
}
$this->putRowInIndex($result, $indexKeys, $row, $idSite, $period);
}
}
return $result;
}
/**
* Returns archive data as a DataTable indexed by metadata. Indexed data will
* be represented by Map instances.
*
* @param array $resultIndices An array mapping metadata names to pretty labels
* for them. Each archive data row will be indexed
* by the metadata specified here.
*
* Eg, array('site' => 'idSite', 'period' => 'Date')
* @return DataTable|DataTable\Map
*/
public function getDataTable($resultIndices)
{
$dataTableFactory = new DataTableFactory(
$this->dataNames, $this->dataType, $this->sitesId, $this->periods, $this->defaultRow);
$index = $this->getIndexedArray($resultIndices);
return $dataTableFactory->make($index, $resultIndices);
}
/**
* Returns archive data as a DataTable indexed by metadata. Indexed data will
* be represented by Map instances. Each DataTable will have
* its subtable IDs set.
*
* This function will only work if blob data was loaded and only one record
* was loaded (not including subtables of the record).
*
* @param array $resultIndices An array mapping metadata names to pretty labels
* for them. Each archive data row will be indexed
* by the metadata specified here.
*
* Eg, array('site' => 'idSite', 'period' => 'Date')
* @param int|null $idSubTable The subtable to return.
* @param int|null $depth max depth for subtables.
* @param bool $addMetadataSubTableId Whether to add the DB subtable ID as metadata
* to each datatable, or not.
* @throws Exception
* @return DataTable|DataTable\Map
*/
public function getExpandedDataTable($resultIndices, $idSubTable = null, $depth = null, $addMetadataSubTableId = false)
{
if ($this->dataType != 'blob') {
throw new Exception("DataCollection: cannot call getExpandedDataTable with "
. "{$this->dataType} data types. Only works with blob data.");
}
if (count($this->dataNames) !== 1) {
throw new Exception("DataCollection: cannot call getExpandedDataTable with "
. "more than one record.");
}
$dataTableFactory = new DataTableFactory(
$this->dataNames, 'blob', $this->sitesId, $this->periods, $this->defaultRow);
$dataTableFactory->expandDataTable($depth, $addMetadataSubTableId);
$dataTableFactory->useSubtable($idSubTable);
$index = $this->getIndexedArray($resultIndices);
return $dataTableFactory->make($index, $resultIndices);
}
/**
* Returns metadata for a data row.
*
* @param array $data The data row.
* @return array
*/
public static function getDataRowMetadata($data)
{
if (isset($data[self::METADATA_CONTAINER_ROW_KEY])) {
return $data[self::METADATA_CONTAINER_ROW_KEY];
} else {
return array();
}
}
/**
* Removes all table metadata from a data row.
*
* @param array $data The data row.
*/
public static function removeMetadataFromDataRow(&$data)
{
unset($data[self::METADATA_CONTAINER_ROW_KEY]);
}
/**
* Creates an empty index using a list of metadata names. If the 'site' and/or
* 'period' metadata names are supplied, empty rows are added for every site/period
* that was queried for.
*
* Using this function ensures consistent ordering in the indexed result.
*
* @param array $metadataNamesToIndexBy List of metadata names to index archive data by.
* @return array
*/
private function createOrderedIndex($metadataNamesToIndexBy)
{
$result = array();
if (!empty($metadataNamesToIndexBy)) {
$metadataName = array_shift($metadataNamesToIndexBy);
if ($metadataName == DataTableFactory::TABLE_METADATA_SITE_INDEX) {
$indexKeyValues = array_values($this->sitesId);
} else if ($metadataName == DataTableFactory::TABLE_METADATA_PERIOD_INDEX) {
$indexKeyValues = array_keys($this->periods);
}
foreach ($indexKeyValues as $key) {
$result[$key] = $this->createOrderedIndex($metadataNamesToIndexBy);
}
}
return $result;
}
/**
* Puts an archive data row in an index.
*/
private function putRowInIndex(&$index, $metadataNamesToIndexBy, $row, $idSite, $period)
{
$currentLevel = & $index;
foreach ($metadataNamesToIndexBy as $metadataName) {
if ($metadataName == DataTableFactory::TABLE_METADATA_SITE_INDEX) {
$key = $idSite;
} else if ($metadataName == DataTableFactory::TABLE_METADATA_PERIOD_INDEX) {
$key = $period;
} else {
$key = $row[self::METADATA_CONTAINER_ROW_KEY][$metadataName];
}
if (!isset($currentLevel[$key])) {
$currentLevel[$key] = array();
}
$currentLevel = & $currentLevel[$key];
}
$currentLevel = $row;
}
}

426
www/analytics/core/Archive/DataTableFactory.php Normal file
View file

@ -0,0 +1,426 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\Archive;
use Piwik\DataTable;
use Piwik\DataTable\Row;
use Piwik\Site;
/**
* Creates a DataTable or Set instance based on an array
* index created by DataCollection.
*
* This class is only used by DataCollection.
*/
class DataTableFactory
{
/**
* @see DataCollection::$dataNames.
*/
private $dataNames;
/**
* @see DataCollection::$dataType.
*/
private $dataType;
/**
* Whether to expand the DataTables that're created or not. Expanding a DataTable
* means creating DataTables using subtable blobs and correctly setting the subtable
* IDs of all DataTables.
*
* @var bool
*/
private $expandDataTable = false;
/**
* Whether to add the subtable ID used in the database to the in-memory DataTables
* as metadata or not.
*
* @var bool
*/
private $addMetadataSubtableId = false;
/**
* The maximum number of subtable levels to create when creating an expanded
* DataTable.
*
* @var int
*/
private $maxSubtableDepth = null;
/**
* @see DataCollection::$sitesId.
*/
private $sitesId;
/**
* @see DataCollection::$periods.
*/
private $periods;
/**
* The ID of the subtable to create a DataTable for. Only relevant for blob data.
*
* @var int|null
*/
private $idSubtable = null;
/**
* @see DataCollection::$defaultRow.
*/
private $defaultRow;
const TABLE_METADATA_SITE_INDEX = 'site';
const TABLE_METADATA_PERIOD_INDEX = 'period';
/**
* Constructor.
*/
public function __construct($dataNames, $dataType, $sitesId, $periods, $defaultRow)
{
$this->dataNames = $dataNames;
$this->dataType = $dataType;
$this->sitesId = $sitesId;
//here index period by string only
$this->periods = $periods;
$this->defaultRow = $defaultRow;
}
/**
* Tells the factory instance to expand the DataTables that are created by
* creating subtables and setting the subtable IDs of rows w/ subtables correctly.
*
* @param null|int $maxSubtableDepth max depth for subtables.
* @param bool $addMetadataSubtableId Whether to add the subtable ID used in the
* database to the in-memory DataTables as
* metadata or not.
*/
public function expandDataTable($maxSubtableDepth = null, $addMetadataSubtableId = false)
{
$this->expandDataTable = true;
$this->maxSubtableDepth = $maxSubtableDepth;
$this->addMetadataSubtableId = $addMetadataSubtableId;
}
/**
* Tells the factory instance to create a DataTable using a blob with the
* supplied subtable ID.
*
* @param int $idSubtable An in-database subtable ID.
* @throws \Exception
*/
public function useSubtable($idSubtable)
{
if (count($this->dataNames) !== 1) {
throw new \Exception("DataTableFactory: Getting subtables for multiple records in one"
. " archive query is not currently supported.");
}
$this->idSubtable = $idSubtable;
}
/**
* Creates a DataTable|Set instance using an index of
* archive data.
*
* @param array $index @see DataCollection
* @param array $resultIndices an array mapping metadata names with pretty metadata
* labels.
* @return DataTable|DataTable\Map
*/
public function make($index, $resultIndices)
{
if (empty($resultIndices)) {
// for numeric data, if there's no index (and thus only 1 site & period in the query),
// we want to display every queried metric name
if (empty($index)
&& $this->dataType == 'numeric'
) {
$index = $this->defaultRow;
}
$dataTable = $this->createDataTable($index, $keyMetadata = array());
} else {
$dataTable = $this->createDataTableMapFromIndex($index, $resultIndices, $keyMetadata = array());
}
$this->transformMetadata($dataTable);
return $dataTable;
}
/**
* Creates a DataTable|Set instance using an array
* of blobs.
*
* If only one record is being queried, a single DataTable will
* be returned. Otherwise, a DataTable\Map is returned that indexes
* DataTables by record name.
*
* If expandDataTable was called, and only one record is being queried,
* the created DataTable's subtables will be expanded.
*
* @param array $blobRow
* @return DataTable|DataTable\Map
*/
private function makeFromBlobRow($blobRow)
{
if ($blobRow === false) {
return new DataTable();
}
if (count($this->dataNames) === 1) {
return $this->makeDataTableFromSingleBlob($blobRow);
} else {
return $this->makeIndexedByRecordNameDataTable($blobRow);
}
}
/**
* Creates a DataTable for one record from an archive data row.
*
* @see makeFromBlobRow
*
* @param array $blobRow
* @return DataTable
*/
private function makeDataTableFromSingleBlob($blobRow)
{
$recordName = reset($this->dataNames);
if ($this->idSubtable !== null) {
$recordName .= '_' . $this->idSubtable;
}
if (!empty($blobRow[$recordName])) {
$table = DataTable::fromSerializedArray($blobRow[$recordName]);
} else {
$table = new DataTable();
}
// set table metadata
$table->setMetadataValues(DataCollection::getDataRowMetadata($blobRow));
if ($this->expandDataTable) {
$table->enableRecursiveFilters();
$this->setSubtables($table, $blobRow);
}
return $table;
}
/**
* Creates a DataTable for every record in an archive data row and puts them
* in a DataTable\Map instance.
*
* @param array $blobRow
* @return DataTable\Map
*/
private function makeIndexedByRecordNameDataTable($blobRow)
{
$table = new DataTable\Map();
$table->setKeyName('recordName');
$tableMetadata = DataCollection::getDataRowMetadata($blobRow);
foreach ($blobRow as $name => $blob) {
$newTable = DataTable::fromSerializedArray($blob);
$newTable->setAllTableMetadata($tableMetadata);
$table->addTable($newTable, $name);
}
return $table;
}
/**
* Creates a Set from an array index.
*
* @param array $index @see DataCollection
* @param array $resultIndices @see make
* @param array $keyMetadata The metadata to add to the table when it's created.
* @return DataTable\Map
*/
private function createDataTableMapFromIndex($index, $resultIndices, $keyMetadata = array())
{
$resultIndexLabel = reset($resultIndices);
$resultIndex = key($resultIndices);
array_shift($resultIndices);
$result = new DataTable\Map();
$result->setKeyName($resultIndexLabel);
foreach ($index as $label => $value) {
$keyMetadata[$resultIndex] = $label;
if (empty($resultIndices)) {
$newTable = $this->createDataTable($value, $keyMetadata);
} else {
$newTable = $this->createDataTableMapFromIndex($value, $resultIndices, $keyMetadata);
}
$result->addTable($newTable, $this->prettifyIndexLabel($resultIndex, $label));
}
return $result;
}
/**
* Creates a DataTable instance from an index row.
*
* @param array $data An archive data row.
* @param array $keyMetadata The metadata to add to the table(s) when created.
* @return DataTable|DataTable\Map
*/
private function createDataTable($data, $keyMetadata)
{
if ($this->dataType == 'blob') {
$result = $this->makeFromBlobRow($data);
} else {
$result = $this->makeFromMetricsArray($data);
}
$this->setTableMetadata($keyMetadata, $result);
return $result;
}
/**
* Creates DataTables from $dataTable's subtable blobs (stored in $blobRow) and sets
* the subtable IDs of each DataTable row.
*
* @param DataTable $dataTable
* @param array $blobRow An array associating record names (w/ subtable if applicable)
* with blob values. This should hold every subtable blob for
* the loaded DataTable.
* @param int $treeLevel
*/
private function setSubtables($dataTable, $blobRow, $treeLevel = 0)
{
if ($this->maxSubtableDepth
&& $treeLevel >= $this->maxSubtableDepth
) {
// unset the subtables so DataTableManager doesn't throw
foreach ($dataTable->getRows() as $row) {
$row->removeSubtable();
}
return;
}
$dataName = reset($this->dataNames);
foreach ($dataTable->getRows() as $row) {
$sid = $row->getIdSubDataTable();
if ($sid === null) {
continue;
}
$blobName = $dataName . "_" . $sid;
if (isset($blobRow[$blobName])) {
$subtable = DataTable::fromSerializedArray($blobRow[$blobName]);
$this->setSubtables($subtable, $blobRow, $treeLevel + 1);
// we edit the subtable ID so that it matches the newly table created in memory
// NB: we dont overwrite the datatableid in the case we are displaying the table expanded.
if ($this->addMetadataSubtableId) {
// this will be written back to the column 'idsubdatatable' just before rendering,
// see Renderer/Php.php
$row->addMetadata('idsubdatatable_in_db', $row->getIdSubDataTable());
}
$row->setSubtable($subtable);
}
}
}
/**
* Converts site IDs and period string ranges into Site instances and
* Period instances in DataTable metadata.
*/
private function transformMetadata($table)
{
$periods = $this->periods;
$table->filter(function ($table) use ($periods) {
$table->setMetadata(DataTableFactory::TABLE_METADATA_SITE_INDEX, new Site($table->getMetadata(DataTableFactory::TABLE_METADATA_SITE_INDEX)));
$table->setMetadata(DataTableFactory::TABLE_METADATA_PERIOD_INDEX, $periods[$table->getMetadata(DataTableFactory::TABLE_METADATA_PERIOD_INDEX)]);
});
}
/**
* Returns the pretty version of an index label.
*
* @param string $labelType eg, 'site', 'period', etc.
* @param string $label eg, '0', '1', '2012-01-01,2012-01-31', etc.
* @return string
*/
private function prettifyIndexLabel($labelType, $label)
{
if ($labelType == self::TABLE_METADATA_PERIOD_INDEX) { // prettify period labels
return $this->periods[$label]->getPrettyString();
}
return $label;
}
/**
* @param $keyMetadata
* @param $result
*/
private function setTableMetadata($keyMetadata, $result)
{
if (!isset($keyMetadata[DataTableFactory::TABLE_METADATA_SITE_INDEX])) {
$keyMetadata[DataTableFactory::TABLE_METADATA_SITE_INDEX] = reset($this->sitesId);
}
if (!isset($keyMetadata[DataTableFactory::TABLE_METADATA_PERIOD_INDEX])) {
reset($this->periods);
$keyMetadata[DataTableFactory::TABLE_METADATA_PERIOD_INDEX] = key($this->periods);
}
// Note: $result can be a DataTable\Map
$result->filter(function ($table) use ($keyMetadata) {
foreach ($keyMetadata as $name => $value) {
$table->setMetadata($name, $value);
}
});
}
/**
* @param $data
* @return DataTable\Simple
*/
private function makeFromMetricsArray($data)
{
$table = new DataTable\Simple();
if (!empty($data)) {
$table->setAllTableMetadata(DataCollection::getDataRowMetadata($data));
DataCollection::removeMetadataFromDataRow($data);
$table->addRow(new Row(array(Row::COLUMNS => $data)));
} else {
// if we're querying numeric data, we couldn't find any, and we're only
// looking for one metric, add a row w/ one column w/ value 0. this is to
// ensure that the PHP renderer outputs 0 when only one column is queried.
// w/o this code, an empty array would be created, and other parts of Piwik
// would break.
if (count($this->dataNames) == 1
&& $this->dataType == 'numeric'
) {
$name = reset($this->dataNames);
$table->addRow(new Row(array(Row::COLUMNS => array($name => 0))));
}
}
$result = $table;
return $result;
}
}

73
www/analytics/core/Archive/Parameters.php Normal file
View file

@ -0,0 +1,73 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\Archive;
use Exception;
use Piwik\Period;
use Piwik\Segment;
class Parameters
{
/**
* The list of site IDs to query archive data for.
*
* @var array
*/
private $idSites = array();
/**
* The list of Period's to query archive data for.
*
* @var Period[]
*/
private $periods = array();
/**
* Segment applied to the visits set.
*
* @var Segment
*/
private $segment;
/**
* @var bool
*/
private $skipAggregationOfSubTables;
public function getSegment()
{
return $this->segment;
}
public function __construct($idSites, $periods, Segment $segment, $skipAggregationOfSubTables)
{
$this->idSites = $idSites;
$this->periods = $periods;
$this->segment = $segment;
$this->skipAggregationOfSubTables = $skipAggregationOfSubTables;
}
public function getPeriods()
{
return $this->periods;
}
public function getIdSites()
{
return $this->idSites;
}
public function isSkipAggregationOfSubTables()
{
return $this->skipAggregationOfSubTables;
}
}

489
www/analytics/core/ArchiveProcessor.php Normal file
View file

@ -0,0 +1,489 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik;
use Exception;
use Piwik\ArchiveProcessor\Parameters;
use Piwik\DataAccess\ArchiveWriter;
use Piwik\DataAccess\LogAggregator;
use Piwik\DataTable\Manager;
use Piwik\DataTable\Map;
use Piwik\DataTable\Row;
use Piwik\Db;
use Piwik\Period;
/**
* Used by {@link Piwik\Plugin\Archiver} instances to insert and aggregate archive data.
*
* ### See also
*
* - **{@link Piwik\Plugin\Archiver}** - to learn how plugins should implement their own analytics
* aggregation logic.
* - **{@link Piwik\DataAccess\LogAggregator}** - to learn how plugins can perform data aggregation
* across Piwik's log tables.
*
* ### Examples
*
* **Inserting numeric data**
*
* // function in an Archiver descendant
* public function aggregateDayReport()
* {
* $archiveProcessor = $this->getProcessor();
*
* $myFancyMetric = // ... calculate the metric value ...
* $archiveProcessor->insertNumericRecord('MyPlugin_myFancyMetric', $myFancyMetric);
* }
*
* **Inserting serialized DataTables**
*
* // function in an Archiver descendant
* public function aggregateDayReport()
* {
* $archiveProcessor = $this->getProcessor();
*
* $maxRowsInTable = Config::getInstance()->General['datatable_archiving_maximum_rows_standard'];j
*
* $dataTable = // ... build by aggregating visits ...
* $serializedData = $dataTable->getSerialized($maxRowsInTable, $maxRowsInSubtable = $maxRowsInTable,
* $columnToSortBy = Metrics::INDEX_NB_VISITS);
*
* $archiveProcessor->insertBlobRecords('MyPlugin_myFancyReport', $serializedData);
* }
*
* **Aggregating archive data**
*
* // function in Archiver descendant
* public function aggregateMultipleReports()
* {
* $archiveProcessor = $this->getProcessor();
*
* // aggregate a metric
* $archiveProcessor->aggregateNumericMetrics('MyPlugin_myFancyMetric');
* $archiveProcessor->aggregateNumericMetrics('MyPlugin_mySuperFancyMetric', 'max');
*
* // aggregate a report
* $archiveProcessor->aggregateDataTableRecords('MyPlugin_myFancyReport');
* }
*
*/
class ArchiveProcessor
{
/**
* @var \Piwik\DataAccess\ArchiveWriter
*/
protected $archiveWriter;
/**
* @var \Piwik\DataAccess\LogAggregator
*/
protected $logAggregator;
/**
* @var Archive
*/
public $archive = null;
/**
* @var Parameters
*/
protected $params;
/**
* @var int
*/
protected $numberOfVisits = false;
protected $numberOfVisitsConverted = false;
public function __construct(Parameters $params, ArchiveWriter $archiveWriter)
{
$this->params = $params;
$this->logAggregator = new LogAggregator($params);
$this->archiveWriter = $archiveWriter;
}
protected function getArchive()
{
if(empty($this->archive)) {
$subPeriods = $this->params->getSubPeriods();
$idSites = $this->params->getIdSites();
$this->archive = Archive::factory($this->params->getSegment(), $subPeriods, $idSites);
}
return $this->archive;
}
public function setNumberOfVisits($visits, $visitsConverted)
{
$this->numberOfVisits = $visits;
$this->numberOfVisitsConverted = $visitsConverted;
}
/**
* Returns the {@link Parameters} object containing the site, period and segment we're archiving
* data for.
*
* @return Parameters
* @api
*/
public function getParams()
{
return $this->params;
}
/**
* Returns a `{@link Piwik\DataAccess\LogAggregator}` instance for the site, period and segment this
* ArchiveProcessor will insert archive data for.
*
* @return LogAggregator
* @api
*/
public function getLogAggregator()
{
return $this->logAggregator;
}
/**
* Array of (column name before => column name renamed) of the columns for which sum operation is invalid.
* These columns will be renamed as per this mapping.
* @var array
*/
protected static $columnsToRenameAfterAggregation = array(
Metrics::INDEX_NB_UNIQ_VISITORS => Metrics::INDEX_SUM_DAILY_NB_UNIQ_VISITORS
);
/**
* Sums records for every subperiod of the current period and inserts the result as the record
* for this period.
*
* DataTables are summed recursively so subtables will be summed as well.
*
* @param string|array $recordNames Name(s) of the report we are aggregating, eg, `'Referrers_type'`.
* @param int $maximumRowsInDataTableLevelZero Maximum number of rows allowed in the top level DataTable.
* @param int $maximumRowsInSubDataTable Maximum number of rows allowed in each subtable.
* @param string $columnToSortByBeforeTruncation The name of the column to sort by before truncating a DataTable.
* @param array $columnsAggregationOperation Operations for aggregating columns, see {@link Row::sumRow()}.
* @param array $columnsToRenameAfterAggregation Columns mapped to new names for columns that must change names
* when summed because they cannot be summed, eg,
* `array('nb_uniq_visitors' => 'sum_daily_nb_uniq_visitors')`.
* @return array Returns the row counts of each aggregated report before truncation, eg,
*
* array(
* 'report1' => array('level0' => $report1->getRowsCount,
* 'recursive' => $report1->getRowsCountRecursive()),
* 'report2' => array('level0' => $report2->getRowsCount,
* 'recursive' => $report2->getRowsCountRecursive()),
* ...
* )
* @api
*/
public function aggregateDataTableRecords($recordNames,
$maximumRowsInDataTableLevelZero = null,
$maximumRowsInSubDataTable = null,
$columnToSortByBeforeTruncation = null,
&$columnsAggregationOperation = null,
$columnsToRenameAfterAggregation = null)
{
if (!is_array($recordNames)) {
$recordNames = array($recordNames);
}
$nameToCount = array();
foreach ($recordNames as $recordName) {
$latestUsedTableId = Manager::getInstance()->getMostRecentTableId();
$table = $this->aggregateDataTableRecord($recordName, $columnsAggregationOperation, $columnsToRenameAfterAggregation);
$rowsCount = $table->getRowsCount();
$nameToCount[$recordName]['level0'] = $rowsCount;
$rowsCountRecursive = $rowsCount;
if($this->isAggregateSubTables()) {
$rowsCountRecursive = $table->getRowsCountRecursive();
}
$nameToCount[$recordName]['recursive'] = $rowsCountRecursive;
$blob = $table->getSerialized($maximumRowsInDataTableLevelZero, $maximumRowsInSubDataTable, $columnToSortByBeforeTruncation);
Common::destroy($table);
$this->insertBlobRecord($recordName, $blob);
unset($blob);
DataTable\Manager::getInstance()->deleteAll($latestUsedTableId);
}
return $nameToCount;
}
/**
* Aggregates one or more metrics for every subperiod of the current period and inserts the results
* as metrics for the current period.
*
* @param array|string $columns Array of metric names to aggregate.
* @param bool|string $operationToApply The operation to apply to the metric. Either `'sum'`, `'max'` or `'min'`.
* @return array|int Returns the array of aggregate values. If only one metric was aggregated,
* the aggregate value will be returned as is, not in an array.
* For example, if `array('nb_visits', 'nb_hits')` is supplied for `$columns`,
*
* array(
* 'nb_visits' => 3040,
* 'nb_hits' => 405
* )
*
* could be returned. If `array('nb_visits')` or `'nb_visits'` is used for `$columns`,
* then `3040` would be returned.
* @api
*/
public function aggregateNumericMetrics($columns, $operationToApply = false)
{
$metrics = $this->getAggregatedNumericMetrics($columns, $operationToApply);
foreach($metrics as $column => $value) {
$this->archiveWriter->insertRecord($column, $value);
}
// if asked for only one field to sum
if (count($metrics) == 1) {
return reset($metrics);
}
// returns the array of records once summed
return $metrics;
}
public function getNumberOfVisits()
{
if($this->numberOfVisits === false) {
throw new Exception("visits should have been set here");
}
return $this->numberOfVisits;
}
public function getNumberOfVisitsConverted()
{
return $this->numberOfVisitsConverted;
}
/**
* Caches multiple numeric records in the archive for this processor's site, period
* and segment.
*
* @param array $numericRecords A name-value mapping of numeric values that should be
* archived, eg,
*
* array('Referrers_distinctKeywords' => 23, 'Referrers_distinctCampaigns' => 234)
* @api
*/
public function insertNumericRecords($numericRecords)
{
foreach ($numericRecords as $name => $value) {
$this->insertNumericRecord($name, $value);
}
}
/**
* Caches a single numeric record in the archive for this processor's site, period and
* segment.
*
* Numeric values are not inserted if they equal `0`.
*
* @param string $name The name of the numeric value, eg, `'Referrers_distinctKeywords'`.
* @param float $value The numeric value.
* @api
*/
public function insertNumericRecord($name, $value)
{
$value = round($value, 2);
$this->archiveWriter->insertRecord($name, $value);
}
/**
* Caches one or more blob records in the archive for this processor's site, period
* and segment.
*
* @param string $name The name of the record, eg, 'Referrers_type'.
* @param string|array $values A blob string or an array of blob strings. If an array
* is used, the first element in the array will be inserted
* with the `$name` name. The others will be inserted with
* `$name . '_' . $index` as the record name (where $index is
* the index of the blob record in `$values`).
* @api
*/
public function insertBlobRecord($name, $values)
{
$this->archiveWriter->insertBlobRecord($name, $values);
}
/**
* This method selects all DataTables that have the name $name over the period.
* All these DataTables are then added together, and the resulting DataTable is returned.
*
* @param string $name
* @param array $columnsAggregationOperation Operations for aggregating columns, @see Row::sumRow()
* @param array $columnsToRenameAfterAggregation columns in the array (old name, new name) to be renamed as the sum operation is not valid on them (eg. nb_uniq_visitors->sum_daily_nb_uniq_visitors)
* @return DataTable
*/
protected function aggregateDataTableRecord($name, $columnsAggregationOperation = null, $columnsToRenameAfterAggregation = null)
{
if($this->isAggregateSubTables()) {
// By default we shall aggregate all sub-tables.
$dataTable = $this->getArchive()->getDataTableExpanded($name, $idSubTable = null, $depth = null, $addMetadataSubtableId = false);
} else {
// In some cases (eg. Actions plugin when period=range),
// for better performance we will only aggregate the parent table
$dataTable = $this->getArchive()->getDataTable($name, $idSubTable = null);
}
$dataTable = $this->getAggregatedDataTableMap($dataTable, $columnsAggregationOperation);
$this->renameColumnsAfterAggregation($dataTable, $columnsToRenameAfterAggregation);
return $dataTable;
}
protected function getOperationForColumns($columns, $defaultOperation)
{
$operationForColumn = array();
foreach ($columns as $name) {
$operation = $defaultOperation;
if (empty($operation)) {
$operation = $this->guessOperationForColumn($name);
}
$operationForColumn[$name] = $operation;
}
return $operationForColumn;
}
protected function enrichWithUniqueVisitorsMetric(Row $row)
{
if(!$this->getParams()->isSingleSite() ) {
// we only compute unique visitors for a single site
return;
}
if ( $row->getColumn('nb_uniq_visitors') !== false) {
if (SettingsPiwik::isUniqueVisitorsEnabled($this->getParams()->getPeriod()->getLabel())) {
$uniqueVisitors = (float)$this->computeNbUniqVisitors();
$row->setColumn('nb_uniq_visitors', $uniqueVisitors);
} else {
$row->deleteColumn('nb_uniq_visitors');
}
}
}
protected function guessOperationForColumn($column)
{
if (strpos($column, 'max_') === 0) {
return 'max';
}
if (strpos($column, 'min_') === 0) {
return 'min';
}
return 'sum';
}
/**
* Processes number of unique visitors for the given period
*
* This is the only Period metric (ie. week/month/year/range) that we process from the logs directly,
* since unique visitors cannot be summed like other metrics.
*
* @return int
*/
protected function computeNbUniqVisitors()
{
$logAggregator = $this->getLogAggregator();
$query = $logAggregator->queryVisitsByDimension(array(), false, array(), array(Metrics::INDEX_NB_UNIQ_VISITORS));
$data = $query->fetch();
return $data[Metrics::INDEX_NB_UNIQ_VISITORS];
}
/**
* If the DataTable is a Map, sums all DataTable in the map and return the DataTable.
*
*
* @param $data DataTable|DataTable\Map
* @param $columnsToRenameAfterAggregation array
* @return DataTable
*/
protected function getAggregatedDataTableMap($data, $columnsAggregationOperation)
{
$table = new DataTable();
if (!empty($columnsAggregationOperation)) {
$table->setMetadata(DataTable::COLUMN_AGGREGATION_OPS_METADATA_NAME, $columnsAggregationOperation);
}
if ($data instanceof DataTable\Map) {
// as $date => $tableToSum
$this->aggregatedDataTableMapsAsOne($data, $table);
} else {
$table->addDataTable($data, $this->isAggregateSubTables());
}
return $table;
}
/**
* Aggregates the DataTable\Map into the destination $aggregated
* @param $map
* @param $aggregated
*/
protected function aggregatedDataTableMapsAsOne(Map $map, DataTable $aggregated)
{
foreach ($map->getDataTables() as $tableToAggregate) {
if($tableToAggregate instanceof Map) {
$this->aggregatedDataTableMapsAsOne($tableToAggregate, $aggregated);
} else {
$aggregated->addDataTable($tableToAggregate, $this->isAggregateSubTables());
}
}
}
protected function renameColumnsAfterAggregation(DataTable $table, $columnsToRenameAfterAggregation = null)
{
// Rename columns after aggregation
if (is_null($columnsToRenameAfterAggregation)) {
$columnsToRenameAfterAggregation = self::$columnsToRenameAfterAggregation;
}
foreach ($columnsToRenameAfterAggregation as $oldName => $newName) {
$table->renameColumn($oldName, $newName, $this->isAggregateSubTables());
}
}
protected function getAggregatedNumericMetrics($columns, $operationToApply)
{
if (!is_array($columns)) {
$columns = array($columns);
}
$operationForColumn = $this->getOperationForColumns($columns, $operationToApply);
$dataTable = $this->getArchive()->getDataTableFromNumeric($columns);
$results = $this->getAggregatedDataTableMap($dataTable, $operationForColumn);
if ($results->getRowsCount() > 1) {
throw new Exception("A DataTable is an unexpected state:" . var_export($results, true));
}
$rowMetrics = $results->getFirstRow();
if($rowMetrics === false) {
$rowMetrics = new Row;
}
$this->enrichWithUniqueVisitorsMetric($rowMetrics);
$this->renameColumnsAfterAggregation($results);
$metrics = $rowMetrics->getColumns();
foreach ($columns as $name) {
if (!isset($metrics[$name])) {
$metrics[$name] = 0;
}
}
return $metrics;
}
/**
* @return bool
*/
protected function isAggregateSubTables()
{
return !$this->getParams()->isSkipAggregationOfSubTables();
}
}

214
www/analytics/core/ArchiveProcessor/Loader.php Normal file
View file

@ -0,0 +1,214 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\ArchiveProcessor;
use Piwik\Archive;
use Piwik\ArchiveProcessor;
use Piwik\Config;
use Piwik\DataAccess\ArchiveSelector;
use Piwik\Date;
use Piwik\Period;
/**
* This class uses PluginsArchiver class to trigger data aggregation and create archives.
*/
class Loader
{
/**
* Is the current archive temporary. ie.
* - today
* - current week / month / year
*/
protected $temporaryArchive;
/**
* Idarchive in the DB for the requested archive
*
* @var int
*/
protected $idArchive;
/**
* @var Parameters
*/
protected $params;
public function __construct(Parameters $params)
{
$this->params = $params;
}
/**
* @return bool
*/
protected function isThereSomeVisits($visits)
{
return $visits > 0;
}
/**
* @return bool
*/
protected function mustProcessVisitCount($visits)
{
return $visits === false;
}
public function prepareArchive($pluginName)
{
$this->params->setRequestedPlugin($pluginName);
list($idArchive, $visits, $visitsConverted) = $this->loadExistingArchiveIdFromDb();
if (!empty($idArchive)) {
return $idArchive;
}
list($visits, $visitsConverted) = $this->prepareCoreMetricsArchive($visits, $visitsConverted);
list($idArchive, $visits) = $this->prepareAllPluginsArchive($visits, $visitsConverted);
if ($this->isThereSomeVisits($visits)) {
return $idArchive;
}
return false;
}
/**
* Prepares the core metrics if needed.
*
* @param $visits
*/
protected function prepareCoreMetricsArchive($visits, $visitsConverted)
{
$createSeparateArchiveForCoreMetrics = $this->mustProcessVisitCount($visits)
&& !$this->doesRequestedPluginIncludeVisitsSummary();
if ($createSeparateArchiveForCoreMetrics) {
$requestedPlugin = $this->params->getRequestedPlugin();
$this->params->setRequestedPlugin('VisitsSummary');
$pluginsArchiver = new PluginsArchiver($this->params, $this->isArchiveTemporary());
$metrics = $pluginsArchiver->callAggregateCoreMetrics();
$pluginsArchiver->finalizeArchive();
$this->params->setRequestedPlugin($requestedPlugin);
$visits = $metrics['nb_visits'];
$visitsConverted = $metrics['nb_visits_converted'];
}
return array($visits, $visitsConverted);
}
protected function prepareAllPluginsArchive($visits, $visitsConverted)
{
$pluginsArchiver = new PluginsArchiver($this->params, $this->isArchiveTemporary());
if ($this->mustProcessVisitCount($visits)
|| $this->doesRequestedPluginIncludeVisitsSummary()
) {
$metrics = $pluginsArchiver->callAggregateCoreMetrics();
$visits = $metrics['nb_visits'];
$visitsConverted = $metrics['nb_visits_converted'];
}
if ($this->isThereSomeVisits($visits)) {
$pluginsArchiver->callAggregateAllPlugins($visits, $visitsConverted);
}
$idArchive = $pluginsArchiver->finalizeArchive();
if (!$this->params->isSingleSiteDayArchive() && $visits) {
ArchiveSelector::purgeOutdatedArchives($this->params->getPeriod()->getDateStart());
}
return array($idArchive, $visits);
}
protected function doesRequestedPluginIncludeVisitsSummary()
{
$processAllReportsIncludingVisitsSummary =
Rules::shouldProcessReportsAllPlugins($this->params->getIdSites(), $this->params->getSegment(), $this->params->getPeriod()->getLabel());
$doesRequestedPluginIncludeVisitsSummary = $processAllReportsIncludingVisitsSummary
|| $this->params->getRequestedPlugin() == 'VisitsSummary';
return $doesRequestedPluginIncludeVisitsSummary;
}
protected function isArchivingForcedToTrigger()
{
$period = $this->params->getPeriod()->getLabel();
$debugSetting = 'always_archive_data_period'; // default
if ($period == 'day') {
$debugSetting = 'always_archive_data_day';
} elseif ($period == 'range') {
$debugSetting = 'always_archive_data_range';
}
return (bool) Config::getInstance()->Debug[$debugSetting];
}
/**
* Returns the idArchive if the archive is available in the database for the requested plugin.
* Returns false if the archive needs to be processed.
*
* @return array
*/
protected function loadExistingArchiveIdFromDb()
{
$noArchiveFound = array(false, false, false);
// see isArchiveTemporary()
$minDatetimeArchiveProcessedUTC = $this->getMinTimeArchiveProcessed();
if ($this->isArchivingForcedToTrigger()) {
return $noArchiveFound;
}
$idAndVisits = ArchiveSelector::getArchiveIdAndVisits($this->params, $minDatetimeArchiveProcessedUTC);
if (!$idAndVisits) {
return $noArchiveFound;
}
return $idAndVisits;
}
/**
* Returns the minimum archive processed datetime to look at. Only public for tests.
*
* @return int|bool Datetime timestamp, or false if must look at any archive available
*/
protected function getMinTimeArchiveProcessed()
{
$endDateTimestamp = self::determineIfArchivePermanent($this->params->getDateEnd());
$isArchiveTemporary = ($endDateTimestamp === false);
$this->temporaryArchive = $isArchiveTemporary;
if ($endDateTimestamp) {
// Permanent archive
return $endDateTimestamp;
}
// Temporary archive
return Rules::getMinTimeProcessedForTemporaryArchive($this->params->getDateStart(), $this->params->getPeriod(), $this->params->getSegment(), $this->params->getSite());
}
protected static function determineIfArchivePermanent(Date $dateEnd)
{
$now = time();
$endTimestampUTC = strtotime($dateEnd->getDateEndUTC());
if ($endTimestampUTC <= $now) {
// - if the period we are looking for is finished, we look for a ts_archived that
// is greater than the last day of the archive
return $endTimestampUTC;
}
return false;
}
protected function isArchiveTemporary()
{
if (is_null($this->temporaryArchive)) {
throw new \Exception("getMinTimeArchiveProcessed() should be called prior to isArchiveTemporary()");
}
return $this->temporaryArchive;
}
}

194
www/analytics/core/ArchiveProcessor/Parameters.php Normal file
View file

@ -0,0 +1,194 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\ArchiveProcessor;
use Piwik\Date;
use Piwik\Log;
use Piwik\Period;
use Piwik\Piwik;
use Piwik\Segment;
use Piwik\Site;
/**
* Contains the analytics parameters for the reports that are currently being archived. The analytics
* parameters include the **website** the reports describe, the **period** of time the reports describe
* and the **segment** used to limit the visit set.
*/
class Parameters
{
/**
* @var Site
*/
private $site = null;
/**
* @var Period
*/
private $period = null;
/**
* @var Segment
*/
private $segment = null;
/**
* @var string Plugin name which triggered this archive processor
*/
private $requestedPlugin = false;
/**
* Constructor.
*
* @ignore
*/
public function __construct(Site $site, Period $period, Segment $segment, $skipAggregationOfSubTables = false)
{
$this->site = $site;
$this->period = $period;
$this->segment = $segment;
$this->skipAggregationOfSubTables = $skipAggregationOfSubTables;
}
/**
* @ignore
*/
public function setRequestedPlugin($plugin)
{
$this->requestedPlugin = $plugin;
}
/**
* @ignore
*/
public function getRequestedPlugin()
{
return $this->requestedPlugin;
}
/**
* Returns the period we are computing statistics for.
*
* @return Period
* @api
*/
public function getPeriod()
{
return $this->period;
}
/**
* Returns the array of Period which make up this archive.
*
* @return \Piwik\Period[]
* @ignore
*/
public function getSubPeriods()
{
if($this->getPeriod()->getLabel() == 'day') {
return array( $this->getPeriod() );
}
return $this->getPeriod()->getSubperiods();
}
/**
* @return array
* @ignore
*/
public function getIdSites()
{
$idSite = $this->getSite()->getId();
$idSites = array($idSite);
Piwik::postEvent('ArchiveProcessor.Parameters.getIdSites', array(&$idSites, $this->getPeriod()));
return $idSites;
}
/**
* Returns the site we are computing statistics for.
*
* @return Site
* @api
*/
public function getSite()
{
return $this->site;
}
/**
* The Segment used to limit the set of visits that are being aggregated.
*
* @return Segment
* @api
*/
public function getSegment()
{
return $this->segment;
}
/**
* Returns the end day of the period in the site's timezone.
*
* @return Date
*/
public function getDateEnd()
{
return $this->getPeriod()->getDateEnd()->setTimezone($this->getSite()->getTimezone());
}
/**
* Returns the start day of the period in the site's timezone.
*
* @return Date
*/
public function getDateStart()
{
return $this->getPeriod()->getDateStart()->setTimezone($this->getSite()->getTimezone());
}
/**
* @return bool
*/
public function isSingleSiteDayArchive()
{
$oneSite = $this->isSingleSite();
$oneDay = $this->getPeriod()->getLabel() == 'day';
return $oneDay && $oneSite;
}
public function isSingleSite()
{
return count($this->getIdSites()) == 1;
}
public function isSkipAggregationOfSubTables()
{
return $this->skipAggregationOfSubTables;
}
public function logStatusDebug($isTemporary)
{
$temporary = 'definitive archive';
if ($isTemporary) {
$temporary = 'temporary archive';
}
Log::verbose(
"%s archive, idSite = %d (%s), segment '%s', report = '%s', UTC datetime [%s -> %s]",
$this->getPeriod()->getLabel(),
$this->getSite()->getId(),
$temporary,
$this->getSegment()->getString(),
$this->getRequestedPlugin(),
$this->getDateStart()->getDateStartUTC(),
$this->getDateEnd()->getDateEndUTC()
);
}
}

197
www/analytics/core/ArchiveProcessor/PluginsArchiver.php Normal file
View file

@ -0,0 +1,197 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\ArchiveProcessor;
use Piwik\Archive;
use Piwik\ArchiveProcessor;
use Piwik\DataAccess\ArchiveSelector;
use Piwik\DataAccess\ArchiveWriter;
use Piwik\DataTable\Manager;
use Piwik\Metrics;
use Piwik\Plugin\Archiver;
/**
* This class creates the Archiver objects found in plugins and will trigger aggregation,
* so each plugin can process their reports.
*/
class PluginsArchiver
{
/**
* @param ArchiveProcessor $archiveProcessor
*/
public $archiveProcessor;
/**
* @var Parameters
*/
protected $params;
/**
* @var Archiver[] $archivers
*/
private static $archivers = array();
public function __construct(Parameters $params, $isTemporaryArchive)
{
$this->params = $params;
$this->archiveWriter = new ArchiveWriter($this->params, $isTemporaryArchive);
$this->archiveWriter->initNewArchive();
$this->archiveProcessor = new ArchiveProcessor($this->params, $this->archiveWriter);
$this->isSingleSiteDayArchive = $this->params->isSingleSiteDayArchive();
}
/**
* If period is day, will get the core metrics (including visits) from the logs.
* If period is != day, will sum the core metrics from the existing archives.
* @return array Core metrics
*/
public function callAggregateCoreMetrics()
{
if($this->isSingleSiteDayArchive) {
$metrics = $this->aggregateDayVisitsMetrics();
} else {
$metrics = $this->aggregateMultipleVisitsMetrics();
}
if (empty($metrics)) {
return array(
'nb_visits' => false,
'nb_visits_converted' => false
);
}
return array(
'nb_visits' => $metrics['nb_visits'],
'nb_visits_converted' => $metrics['nb_visits_converted']
);
}
/**
* Instantiates the Archiver class in each plugin that defines it,
* and triggers Aggregation processing on these plugins.
*/
public function callAggregateAllPlugins($visits, $visitsConverted)
{
$this->archiveProcessor->setNumberOfVisits($visits, $visitsConverted);
$archivers = $this->getPluginArchivers();
foreach($archivers as $pluginName => $archiverClass) {
// We clean up below all tables created during this function call (and recursive calls)
$latestUsedTableId = Manager::getInstance()->getMostRecentTableId();
/** @var Archiver $archiver */
$archiver = new $archiverClass($this->archiveProcessor);
if(!$archiver->isEnabled()) {
continue;
}
if($this->shouldProcessReportsForPlugin($pluginName)) {
if($this->isSingleSiteDayArchive) {
$archiver->aggregateDayReport();
} else {
$archiver->aggregateMultipleReports();
}
}
Manager::getInstance()->deleteAll($latestUsedTableId);
unset($archiver);
}
}
public function finalizeArchive()
{
$this->params->logStatusDebug( $this->archiveWriter->isArchiveTemporary );
$this->archiveWriter->finalizeArchive();
return $this->archiveWriter->getIdArchive();
}
/**
* Loads Archiver class from any plugin that defines one.
*
* @return \Piwik\Plugin\Archiver[]
*/
protected function getPluginArchivers()
{
if (empty(static::$archivers)) {
$pluginNames = \Piwik\Plugin\Manager::getInstance()->getActivatedPlugins();
$archivers = array();
foreach ($pluginNames as $pluginName) {
$archivers[$pluginName] = self::getPluginArchiverClass($pluginName);
}
static::$archivers = array_filter($archivers);
}
return static::$archivers;
}
private static function getPluginArchiverClass($pluginName)
{
$klassName = 'Piwik\\Plugins\\' . $pluginName . '\\Archiver';
if (class_exists($klassName)
&& is_subclass_of($klassName, 'Piwik\\Plugin\\Archiver')) {
return $klassName;
}
return false;
}
/**
* Whether the specified plugin's reports should be archived
* @param string $pluginName
* @return bool
*/
protected function shouldProcessReportsForPlugin($pluginName)
{
if ($this->params->getRequestedPlugin() == $pluginName) {
return true;
}
if (Rules::shouldProcessReportsAllPlugins(
$this->params->getIdSites(),
$this->params->getSegment(),
$this->params->getPeriod()->getLabel())) {
return true;
}
if (!\Piwik\Plugin\Manager::getInstance()->isPluginLoaded($this->params->getRequestedPlugin())) {
return true;
}
return false;
}
protected function aggregateDayVisitsMetrics()
{
$query = $this->archiveProcessor->getLogAggregator()->queryVisitsByDimension();
$data = $query->fetch();
$metrics = $this->convertMetricsIdToName($data);
$this->archiveProcessor->insertNumericRecords($metrics);
return $metrics;
}
protected function convertMetricsIdToName($data)
{
$metrics = array();
foreach ($data as $metricId => $value) {
$readableMetric = Metrics::$mappingFromIdToName[$metricId];
$metrics[$readableMetric] = $value;
}
return $metrics;
}
protected function aggregateMultipleVisitsMetrics()
{
$toSum = Metrics::getVisitsMetricNames();
$metrics = $this->archiveProcessor->aggregateNumericMetrics($toSum);
return $metrics;
}
}

304
www/analytics/core/ArchiveProcessor/Rules.php Normal file
View file

@ -0,0 +1,304 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\ArchiveProcessor;
use Exception;
use Piwik\Common;
use Piwik\Config;
use Piwik\Date;
use Piwik\Log;
use Piwik\Option;
use Piwik\Piwik;
use Piwik\Plugins\CoreAdminHome\Controller;
use Piwik\Plugins\CoreAdminHome\CoreAdminHome;
use Piwik\Segment;
use Piwik\SettingsPiwik;
use Piwik\SettingsServer;
use Piwik\Site;
use Piwik\Tracker\Cache;
/**
* This class contains Archiving rules/logic which are used when creating and processing Archives.
*
*/
class Rules
{
const OPTION_TODAY_ARCHIVE_TTL = 'todayArchiveTimeToLive';
const OPTION_BROWSER_TRIGGER_ARCHIVING = 'enableBrowserTriggerArchiving';
const FLAG_TABLE_PURGED = 'lastPurge_';
/** Old Archives purge can be disabled (used in tests only) */
static public $purgeDisabledByTests = false;
/** Flag that will forcefully disable the archiving process (used in tests only) */
public static $archivingDisabledByTests = false;
/**
* Returns the name of the archive field used to tell the status of an archive, (ie,
* whether the archive was created successfully or not).
*
* @param Segment $segment
* @param string $periodLabel
* @param string $plugin
* @return string
*/
public static function getDoneStringFlagFor(array $idSites, $segment, $periodLabel, $plugin, $isSkipAggregationOfSubTables)
{
if (!self::shouldProcessReportsAllPlugins($idSites, $segment, $periodLabel)) {
return self::getDoneFlagArchiveContainsOnePlugin($segment, $plugin, $isSkipAggregationOfSubTables);
}
return self::getDoneFlagArchiveContainsAllPlugins($segment);
}
public static function shouldProcessReportsAllPlugins(array $idSites, Segment $segment, $periodLabel)
{
if ($segment->isEmpty() && $periodLabel != 'range') {
return true;
}
return self::isSegmentPreProcessed($idSites, $segment);
}
/**
* @param $idSites
* @return array
*/
private static function getSegmentsToProcess($idSites)
{
$knownSegmentsToArchiveAllSites = SettingsPiwik::getKnownSegmentsToArchive();
$segmentsToProcess = $knownSegmentsToArchiveAllSites;
foreach ($idSites as $idSite) {
$segmentForThisWebsite = SettingsPiwik::getKnownSegmentsToArchiveForSite($idSite);
$segmentsToProcess = array_merge($segmentsToProcess, $segmentForThisWebsite);
}
$segmentsToProcess = array_unique($segmentsToProcess);
return $segmentsToProcess;
}
public static function getDoneFlagArchiveContainsOnePlugin(Segment $segment, $plugin, $isSkipAggregationOfSubTables = false)
{
$partial = self::isFlagArchivePartial($plugin, $isSkipAggregationOfSubTables);
return 'done' . $segment->getHash() . '.' . $plugin . $partial ;
}
private static function getDoneFlagArchiveContainsAllPlugins(Segment $segment)
{
return 'done' . $segment->getHash();
}
/**
* @param $plugin
* @param $isSkipAggregationOfSubTables
* @return string
*/
private static function isFlagArchivePartial($plugin, $isSkipAggregationOfSubTables)
{
$partialArchive = '';
if ($plugin != "VisitsSummary" // VisitsSummary is always called when segmenting and should not have its own .partial archive
&& $isSkipAggregationOfSubTables
) {
$partialArchive = '.partial';
}
return $partialArchive;
}
/**
* @param array $plugins
* @param $segment
* @return array
*/
public static function getDoneFlags(array $plugins, Segment $segment, $isSkipAggregationOfSubTables)
{
$doneFlags = array();
$doneAllPlugins = self::getDoneFlagArchiveContainsAllPlugins($segment);
$doneFlags[$doneAllPlugins] = $doneAllPlugins;
$plugins = array_unique($plugins);
foreach ($plugins as $plugin) {
$doneOnePlugin = self::getDoneFlagArchiveContainsOnePlugin($segment, $plugin, $isSkipAggregationOfSubTables);
$doneFlags[$plugin] = $doneOnePlugin;
}
return $doneFlags;
}
/**
* Given a monthly archive table, will delete all reports that are now outdated,
* or reports that ended with an error
*
* @param \Piwik\Date $date
* @return int|bool False, or timestamp indicating which archives to delete
*/
public static function shouldPurgeOutdatedArchives(Date $date)
{
if (self::$purgeDisabledByTests) {
return false;
}
$key = self::FLAG_TABLE_PURGED . "blob_" . $date->toString('Y_m');
$timestamp = Option::get($key);
// we shall purge temporary archives after their timeout is finished, plus an extra 6 hours
// in case archiving is disabled or run once a day, we give it this extra time to run
// and re-process more recent records...
$temporaryArchivingTimeout = self::getTodayArchiveTimeToLive();
$hoursBetweenPurge = 6;
$purgeEveryNSeconds = max($temporaryArchivingTimeout, $hoursBetweenPurge * 3600);
// we only delete archives if we are able to process them, otherwise, the browser might process reports
// when &segment= is specified (or custom date range) and would below, delete temporary archives that the
// browser is not able to process until next cron run (which could be more than 1 hour away)
if (self::isRequestAuthorizedToArchive()
&& (!$timestamp
|| $timestamp < time() - $purgeEveryNSeconds)
) {
Option::set($key, time());
if (self::isBrowserTriggerEnabled()) {
// If Browser Archiving is enabled, it is likely there are many more temporary archives
// We delete more often which is safe, since reports are re-processed on demand
$purgeArchivesOlderThan = Date::factory(time() - 2 * $temporaryArchivingTimeout)->getDateTime();
} else {
// If archive.php via Cron is building the reports, we should keep all temporary reports from today
$purgeArchivesOlderThan = Date::factory('today')->getDateTime();
}
return $purgeArchivesOlderThan;
}
Log::info("Purging temporary archives: skipped.");
return false;
}
public static function getMinTimeProcessedForTemporaryArchive(
Date $dateStart, \Piwik\Period $period, Segment $segment, Site $site)
{
$now = time();
$minimumArchiveTime = $now - Rules::getTodayArchiveTimeToLive();
$idSites = array($site->getId());
$isArchivingDisabled = Rules::isArchivingDisabledFor($idSites, $segment, $period->getLabel());
if ($isArchivingDisabled) {
if ($period->getNumberOfSubperiods() == 0
&& $dateStart->getTimestamp() <= $now
) {
// Today: accept any recent enough archive
$minimumArchiveTime = false;
} else {
// This week, this month, this year:
// accept any archive that was processed today after 00:00:01 this morning
$timezone = $site->getTimezone();
$minimumArchiveTime = Date::factory(Date::factory('now', $timezone)->getDateStartUTC())->setTimezone($timezone)->getTimestamp();
}
}
return $minimumArchiveTime;
}
public static function setTodayArchiveTimeToLive($timeToLiveSeconds)
{
$timeToLiveSeconds = (int)$timeToLiveSeconds;
if ($timeToLiveSeconds <= 0) {
throw new Exception(Piwik::translate('General_ExceptionInvalidArchiveTimeToLive'));
}
Option::set(self::OPTION_TODAY_ARCHIVE_TTL, $timeToLiveSeconds, $autoLoad = true);
}
public static function getTodayArchiveTimeToLive()
{
$uiSettingIsEnabled = Controller::isGeneralSettingsAdminEnabled();
if($uiSettingIsEnabled) {
$timeToLive = Option::get(self::OPTION_TODAY_ARCHIVE_TTL);
if ($timeToLive !== false) {
return $timeToLive;
}
}
return Config::getInstance()->General['time_before_today_archive_considered_outdated'];
}
public static function isArchivingDisabledFor(array $idSites, Segment $segment, $periodLabel)
{
if ($periodLabel == 'range') {
return false;
}
$processOneReportOnly = !self::shouldProcessReportsAllPlugins($idSites, $segment, $periodLabel);
$isArchivingDisabled = !self::isRequestAuthorizedToArchive() || self::$archivingDisabledByTests;
if ($processOneReportOnly) {
// When there is a segment, we disable archiving when browser_archiving_disabled_enforce applies
if (!$segment->isEmpty()
&& $isArchivingDisabled
&& Config::getInstance()->General['browser_archiving_disabled_enforce']
&& !SettingsServer::isArchivePhpTriggered() // Only applies when we are not running archive.php
) {
Log::debug("Archiving is disabled because of config setting browser_archiving_disabled_enforce=1");
return true;
}
// Always allow processing one report
return false;
}
return $isArchivingDisabled;
}
protected static function isRequestAuthorizedToArchive()
{
return Rules::isBrowserTriggerEnabled() || SettingsServer::isArchivePhpTriggered();
}
public static function isBrowserTriggerEnabled()
{
$uiSettingIsEnabled = Controller::isGeneralSettingsAdminEnabled();
if($uiSettingIsEnabled) {
$browserArchivingEnabled = Option::get(self::OPTION_BROWSER_TRIGGER_ARCHIVING);
if ($browserArchivingEnabled !== false) {
return (bool)$browserArchivingEnabled;
}
}
return (bool)Config::getInstance()->General['enable_browser_archiving_triggering'];
}
public static function setBrowserTriggerArchiving($enabled)
{
if (!is_bool($enabled)) {
throw new Exception('Browser trigger archiving must be set to true or false.');
}
Option::set(self::OPTION_BROWSER_TRIGGER_ARCHIVING, (int)$enabled, $autoLoad = true);
Cache::clearCacheGeneral();
}
/**
* @param array $idSites
* @param Segment $segment
* @return bool
*/
protected static function isSegmentPreProcessed(array $idSites, Segment $segment)
{
$segmentsToProcess = self::getSegmentsToProcess($idSites);
if (empty($segmentsToProcess)) {
return false;
}
// If the requested segment is one of the segments to pre-process
// we ensure that any call to the API will trigger archiving of all reports for this segment
$segment = $segment->getString();
// Turns out the getString() above returns the URL decoded segment string
$segmentsToProcessUrlDecoded = array_map('urldecode', $segmentsToProcess);
if (in_array($segment, $segmentsToProcess)
|| in_array($segment, $segmentsToProcessUrlDecoded)
) {
return true;
}
return false;
}
}

405
www/analytics/core/AssetManager.php Normal file
View file

@ -0,0 +1,405 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik;
use Exception;
use Piwik\AssetManager\UIAsset;
use Piwik\AssetManager\UIAsset\InMemoryUIAsset;
use Piwik\AssetManager\UIAsset\OnDiskUIAsset;
use Piwik\AssetManager\UIAssetCacheBuster;
use Piwik\AssetManager\UIAssetFetcher\JScriptUIAssetFetcher;
use Piwik\AssetManager\UIAssetFetcher\StaticUIAssetFetcher;
use Piwik\AssetManager\UIAssetFetcher\StylesheetUIAssetFetcher;
use Piwik\AssetManager\UIAssetFetcher;
use Piwik\AssetManager\UIAssetMerger\JScriptUIAssetMerger;
use Piwik\AssetManager\UIAssetMerger\StylesheetUIAssetMerger;
use Piwik\Plugin\Manager;
use Piwik\Translate;
use Piwik\Config as PiwikConfig;
/**
* AssetManager is the class used to manage the inclusion of UI assets:
* JavaScript and CSS files.
*
* It performs the following actions:
* - Identifies required assets
* - Includes assets in the rendered HTML page
* - Manages asset merging and minifying
* - Manages server-side cache
*
* Whether assets are included individually or as merged files is defined by
* the global option 'disable_merged_assets'. See the documentation in the global
* config for more information.
*
* @method static \Piwik\AssetManager getInstance()
*/
class AssetManager extends Singleton
{
const MERGED_CSS_FILE = "asset_manager_global_css.css";
const MERGED_CORE_JS_FILE = "asset_manager_core_js.js";
const MERGED_NON_CORE_JS_FILE = "asset_manager_non_core_js.js";
const CSS_IMPORT_DIRECTIVE = "<link rel=\"stylesheet\" type=\"text/css\" href=\"%s\" />\n";
const JS_IMPORT_DIRECTIVE = "<script type=\"text/javascript\" src=\"%s\"></script>\n";
const GET_CSS_MODULE_ACTION = "index.php?module=Proxy&action=getCss";
const GET_CORE_JS_MODULE_ACTION = "index.php?module=Proxy&action=getCoreJs";
const GET_NON_CORE_JS_MODULE_ACTION = "index.php?module=Proxy&action=getNonCoreJs";
/**
* @var UIAssetCacheBuster
*/
private $cacheBuster;
/**
* @var UIAssetFetcher
*/
private $minimalStylesheetFetcher;
/**
* @var Theme
*/
private $theme;
function __construct()
{
$this->cacheBuster = UIAssetCacheBuster::getInstance();
$this->minimalStylesheetFetcher = new StaticUIAssetFetcher(array('plugins/Zeitgeist/stylesheets/base.less'), array(), $this->theme);
$theme = Manager::getInstance()->getThemeEnabled();
if(!empty($theme)) {
$this->theme = new Theme();
}
}
/**
* @param UIAssetCacheBuster $cacheBuster
*/
public function setCacheBuster($cacheBuster)
{
$this->cacheBuster = $cacheBuster;
}
/**
* @param UIAssetFetcher $minimalStylesheetFetcher
*/
public function setMinimalStylesheetFetcher($minimalStylesheetFetcher)
{
$this->minimalStylesheetFetcher = $minimalStylesheetFetcher;
}
/**
* @param Theme $theme
*/
public function setTheme($theme)
{
$this->theme = $theme;
}
/**
* Return CSS file inclusion directive(s) using the markup <link>
*
* @return string
*/
public function getCssInclusionDirective()
{
return sprintf(self::CSS_IMPORT_DIRECTIVE, self::GET_CSS_MODULE_ACTION);
}
/**
* Return JS file inclusion directive(s) using the markup <script>
*
* @return string
*/
public function getJsInclusionDirective()
{
$result = "<script type=\"text/javascript\">\n" . Translate::getJavascriptTranslations() . "\n</script>";
if ($this->isMergedAssetsDisabled()) {
$this->getMergedCoreJSAsset()->delete();
$this->getMergedNonCoreJSAsset()->delete();
$result .= $this->getIndividualJsIncludes();
} else {
$result .= sprintf(self::JS_IMPORT_DIRECTIVE, self::GET_CORE_JS_MODULE_ACTION);
$result .= sprintf(self::JS_IMPORT_DIRECTIVE, self::GET_NON_CORE_JS_MODULE_ACTION);
}
return $result;
}
/**
* Return the base.less compiled to css
*
* @return UIAsset
*/
public function getCompiledBaseCss()
{
$mergedAsset = new InMemoryUIAsset();
$assetMerger = new StylesheetUIAssetMerger($mergedAsset, $this->minimalStylesheetFetcher, $this->cacheBuster);
$assetMerger->generateFile();
return $mergedAsset;
}
/**
* Return the css merged file absolute location.
* If there is none, the generation process will be triggered.
*
* @return UIAsset
*/
public function getMergedStylesheet()
{
$mergedAsset = $this->getMergedStylesheetAsset();
$assetFetcher = new StylesheetUIAssetFetcher(Manager::getInstance()->getLoadedPluginsName(), $this->theme);
$assetMerger = new StylesheetUIAssetMerger($mergedAsset, $assetFetcher, $this->cacheBuster);
$assetMerger->generateFile();
return $mergedAsset;
}
/**
* Return the core js merged file absolute location.
* If there is none, the generation process will be triggered.
*
* @return UIAsset
*/
public function getMergedCoreJavaScript()
{
return $this->getMergedJavascript($this->getCoreJScriptFetcher(), $this->getMergedCoreJSAsset());
}
/**
* Return the non core js merged file absolute location.
* If there is none, the generation process will be triggered.
*
* @return UIAsset
*/
public function getMergedNonCoreJavaScript()
{
return $this->getMergedJavascript($this->getNonCoreJScriptFetcher(), $this->getMergedNonCoreJSAsset());
}
/**
* @param boolean $core
* @return string[]
*/
public function getLoadedPlugins($core)
{
$loadedPlugins = array();
foreach(Manager::getInstance()->getPluginsLoadedAndActivated() as $plugin) {
$pluginName = $plugin->getPluginName();
$pluginIsCore = Manager::getInstance()->isPluginBundledWithCore($pluginName);
if(($pluginIsCore && $core) || (!$pluginIsCore && !$core))
$loadedPlugins[] = $pluginName;
}
return $loadedPlugins;
}
/**
* Remove previous merged assets
*/
public function removeMergedAssets($pluginName = false)
{
$assetsToRemove = array($this->getMergedStylesheetAsset());
if($pluginName) {
if($this->pluginContainsJScriptAssets($pluginName)) {
PiwikConfig::getInstance()->init();
if(Manager::getInstance()->isPluginBundledWithCore($pluginName)) {
$assetsToRemove[] = $this->getMergedCoreJSAsset();
} else {
$assetsToRemove[] = $this->getMergedNonCoreJSAsset();
}
}
} else {
$assetsToRemove[] = $this->getMergedCoreJSAsset();
$assetsToRemove[] = $this->getMergedNonCoreJSAsset();
}
$this->removeAssets($assetsToRemove);
}
/**
* Check if the merged file directory exists and is writable.
*
* @return string The directory location
* @throws Exception if directory is not writable.
*/
public function getAssetDirectory()
{
$mergedFileDirectory = PIWIK_USER_PATH . "/tmp/assets";
$mergedFileDirectory = SettingsPiwik::rewriteTmpPathWithHostname($mergedFileDirectory);
if (!is_dir($mergedFileDirectory)) {
Filesystem::mkdir($mergedFileDirectory);
}
if (!is_writable($mergedFileDirectory)) {
throw new Exception("Directory " . $mergedFileDirectory . " has to be writable.");
}
return $mergedFileDirectory;
}
/**
* Return the global option disable_merged_assets
*
* @return boolean
*/
public function isMergedAssetsDisabled()
{
return Config::getInstance()->Debug['disable_merged_assets'];
}
/**
* @param UIAssetFetcher $assetFetcher
* @param UIAsset $mergedAsset
* @return UIAsset
*/
private function getMergedJavascript($assetFetcher, $mergedAsset)
{
$assetMerger = new JScriptUIAssetMerger($mergedAsset, $assetFetcher, $this->cacheBuster);
$assetMerger->generateFile();
return $mergedAsset;
}
/**
* Return individual JS file inclusion directive(s) using the markup <script>
*
* @return string
*/
private function getIndividualJsIncludes()
{
return
$this->getIndividualJsIncludesFromAssetFetcher($this->getCoreJScriptFetcher()) .
$this->getIndividualJsIncludesFromAssetFetcher($this->getNonCoreJScriptFetcher());
}
/**
* @param UIAssetFetcher $assetFetcher
* @return string
*/
private function getIndividualJsIncludesFromAssetFetcher($assetFetcher)
{
$jsIncludeString = '';
foreach ($assetFetcher->getCatalog()->getAssets() as $jsFile) {
$jsFile->validateFile();
$jsIncludeString = $jsIncludeString . sprintf(self::JS_IMPORT_DIRECTIVE, $jsFile->getRelativeLocation());
}
return $jsIncludeString;
}
private function getCoreJScriptFetcher()
{
return new JScriptUIAssetFetcher($this->getLoadedPlugins(true), $this->theme);
}
private function getNonCoreJScriptFetcher()
{
return new JScriptUIAssetFetcher($this->getLoadedPlugins(false), $this->theme);
}
/**
* @param string $pluginName
* @return boolean
*/
private function pluginContainsJScriptAssets($pluginName)
{
$fetcher = new JScriptUIAssetFetcher(array($pluginName), $this->theme);
try {
$assets = $fetcher->getCatalog()->getAssets();
} catch(\Exception $e) {
// This can happen when a plugin is not valid (eg. Piwik 1.x format)
// When posting the event to the plugin, it returns an exception "Plugin has not been loaded"
return false;
}
$plugin = Manager::getInstance()->getLoadedPlugin($pluginName);
if($plugin->isTheme()) {
$theme = Manager::getInstance()->getTheme($pluginName);
$javaScriptFiles = $theme->getJavaScriptFiles();
if(!empty($javaScriptFiles))
$assets = array_merge($assets, $javaScriptFiles);
}
return !empty($assets);
}
/**
* @param UIAsset[] $uiAssets
*/
private function removeAssets($uiAssets)
{
foreach($uiAssets as $uiAsset) {
$uiAsset->delete();
}
}
/**
* @return UIAsset
*/
private function getMergedStylesheetAsset()
{
return $this->getMergedUIAsset(self::MERGED_CSS_FILE);
}
/**
* @return UIAsset
*/
private function getMergedCoreJSAsset()
{
return $this->getMergedUIAsset(self::MERGED_CORE_JS_FILE);
}
/**
* @return UIAsset
*/
private function getMergedNonCoreJSAsset()
{
return $this->getMergedUIAsset(self::MERGED_NON_CORE_JS_FILE);
}
/**
* @param string $fileName
* @return UIAsset
*/
private function getMergedUIAsset($fileName)
{
return new OnDiskUIAsset($this->getAssetDirectory(), $fileName);
}
}

61
www/analytics/core/AssetManager/UIAsset.php Normal file
View file

@ -0,0 +1,61 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\AssetManager;
use Exception;
abstract class UIAsset
{
abstract public function validateFile();
/**
* @return string
*/
abstract public function getAbsoluteLocation();
/**
* @return string
*/
abstract public function getRelativeLocation();
/**
* @return string
*/
abstract public function getBaseDirectory();
/**
* Removes the previous file if it exists.
* Also tries to remove compressed version of the file.
*
* @see ProxyStaticFile::serveStaticFile(serveFile
* @throws Exception if the file couldn't be deleted
*/
abstract public function delete();
/**
* @param string $content
* @throws \Exception
*/
abstract public function writeContent($content);
/**
* @return string
*/
abstract public function getContent();
/**
* @return boolean
*/
abstract public function exists();
/**
* @return int
*/
abstract public function getModificationDate();
}

63
www/analytics/core/AssetManager/UIAsset/InMemoryUIAsset.php Normal file
View file

@ -0,0 +1,63 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\AssetManager\UIAsset;
use Exception;
use Piwik\AssetManager\UIAsset;
class InMemoryUIAsset extends UIAsset
{
private $content;
public function validateFile()
{
return;
}
public function getAbsoluteLocation()
{
throw new Exception('invalid operation');
}
public function getRelativeLocation()
{
throw new Exception('invalid operation');
}
public function getBaseDirectory()
{
throw new Exception('invalid operation');
}
public function delete()
{
$this->content = null;
}
public function exists()
{
return false;
}
public function writeContent($content)
{
$this->content = $content;
}
public function getContent()
{
return $this->content;
}
public function getModificationDate()
{
throw new Exception('invalid operation');
}
}

113
www/analytics/core/AssetManager/UIAsset/OnDiskUIAsset.php Normal file
View file

@ -0,0 +1,113 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\AssetManager\UIAsset;
use Exception;
use Piwik\AssetManager\UIAsset;
class OnDiskUIAsset extends UIAsset
{
/**
* @var string
*/
private $baseDirectory;
/**
* @var string
*/
private $relativeLocation;
/**
* @param string $baseDirectory
* @param string $fileLocation
*/
function __construct($baseDirectory, $fileLocation)
{
$this->baseDirectory = $baseDirectory;
$this->relativeLocation = $fileLocation;
}
public function getAbsoluteLocation()
{
return $this->baseDirectory . '/' . $this->relativeLocation;
}
public function getRelativeLocation()
{
return $this->relativeLocation;
}
public function getBaseDirectory()
{
return $this->baseDirectory;
}
public function validateFile()
{
if (!$this->assetIsReadable())
throw new Exception("The ui asset with 'href' = " . $this->getAbsoluteLocation() . " is not readable");
}
public function delete()
{
if ($this->exists()) {
if (!unlink($this->getAbsoluteLocation()))
throw new Exception("Unable to delete merged file : " . $this->getAbsoluteLocation() . ". Please delete the file and refresh");
// try to remove compressed version of the merged file.
@unlink($this->getAbsoluteLocation() . ".deflate");
@unlink($this->getAbsoluteLocation() . ".gz");
}
}
/**
* @param string $content
* @throws \Exception
*/
public function writeContent($content)
{
$this->delete();
$newFile = @fopen($this->getAbsoluteLocation(), "w");
if (!$newFile)
throw new Exception ("The file : " . $newFile . " can not be opened in write mode.");
fwrite($newFile, $content);
fclose($newFile);
}
/**
* @return string
*/
public function getContent()
{
return file_get_contents($this->getAbsoluteLocation());
}
public function exists()
{
return $this->assetIsReadable();
}
/**
* @return boolean
*/
private function assetIsReadable()
{
return is_readable($this->getAbsoluteLocation());
}
public function getModificationDate()
{
return filemtime($this->getAbsoluteLocation());
}
}

54
www/analytics/core/AssetManager/UIAssetCacheBuster.php Normal file
View file

@ -0,0 +1,54 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
* @method static \Piwik\AssetManager\UIAssetCacheBuster getInstance()
*/
namespace Piwik\AssetManager;
use Piwik\Plugin\Manager;
use Piwik\Singleton;
use Piwik\Version;
class UIAssetCacheBuster extends Singleton
{
/**
* Cache buster based on
* - Piwik version
* - Loaded plugins (name and version)
* - Super user salt
* - Latest
*
* @param string[] $pluginNames
* @return string
*/
public function piwikVersionBasedCacheBuster($pluginNames = false)
{
$masterFile = PIWIK_INCLUDE_PATH . '/.git/refs/heads/master';
$currentGitHash = file_exists($masterFile) ? @file_get_contents($masterFile) : null;
$pluginNames = !$pluginNames ? Manager::getInstance()->getLoadedPluginsName() : $pluginNames;
sort($pluginNames);
$pluginsInfo = '';
foreach ($pluginNames as $pluginName) {
$plugin = Manager::getInstance()->getLoadedPlugin($pluginName);
$pluginsInfo .= $plugin->getPluginName() . $plugin->getVersion() . ',';
}
$cacheBuster = md5($pluginsInfo . PHP_VERSION . Version::VERSION . trim($currentGitHash));
return $cacheBuster;
}
/**
* @param string $content
* @return string
*/
public function md5BasedCacheBuster($content)
{
return md5($content);
}
}

70
www/analytics/core/AssetManager/UIAssetCatalog.php Normal file
View file

@ -0,0 +1,70 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\AssetManager;
class UIAssetCatalog
{
/**
* @var UIAsset[]
*/
private $uiAssets = array();
/**
* @var UIAssetCatalogSorter
*/
private $catalogSorter;
/**
* @param UIAssetCatalogSorter $catalogSorter
*/
function __construct($catalogSorter)
{
$this->catalogSorter = $catalogSorter;
}
/**
* @param UIAsset $uiAsset
*/
public function addUIAsset($uiAsset)
{
if(!$this->assetAlreadyInCatalog($uiAsset)) {
$this->uiAssets[] = $uiAsset;
}
}
/**
* @return UIAsset[]
*/
public function getAssets()
{
return $this->uiAssets;
}
/**
* @return UIAssetCatalog
*/
public function getSortedCatalog()
{
return $this->catalogSorter->sortUIAssetCatalog($this);
}
/**
* @param UIAsset $uiAsset
* @return boolean
*/
private function assetAlreadyInCatalog($uiAsset)
{
foreach($this->uiAssets as $existingAsset)
if($uiAsset->getAbsoluteLocation() == $existingAsset->getAbsoluteLocation())
return true;
return false;
}
}

59
www/analytics/core/AssetManager/UIAssetCatalogSorter.php Normal file
View file

@ -0,0 +1,59 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\AssetManager;
class UIAssetCatalogSorter
{
/**
* @var string[]
*/
private $priorityOrder;
/**
* @param string[] $priorityOrder
*/
function __construct($priorityOrder)
{
$this->priorityOrder = $priorityOrder;
}
/**
* @param UIAssetCatalog $uiAssetCatalog
* @return UIAssetCatalog
*/
public function sortUIAssetCatalog($uiAssetCatalog)
{
$sortedCatalog = new UIAssetCatalog($this);
foreach ($this->priorityOrder as $filePattern) {
$assetsMatchingPattern = array_filter($uiAssetCatalog->getAssets(), function($uiAsset) use ($filePattern) {
return preg_match('~^' . $filePattern . '~', $uiAsset->getRelativeLocation());
});
foreach($assetsMatchingPattern as $assetMatchingPattern) {
$sortedCatalog->addUIAsset($assetMatchingPattern);
}
}
$this->addUnmatchedAssets($uiAssetCatalog, $sortedCatalog);
return $sortedCatalog;
}
/**
* @param UIAssetCatalog $uiAssetCatalog
* @param UIAssetCatalog $sortedCatalog
*/
private function addUnmatchedAssets($uiAssetCatalog, $sortedCatalog)
{
foreach ($uiAssetCatalog->getAssets() as $uiAsset) {
$sortedCatalog->addUIAsset($uiAsset);
}
}
}

119
www/analytics/core/AssetManager/UIAssetFetcher.php Normal file
View file

@ -0,0 +1,119 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\AssetManager;
use Piwik\AssetManager\UIAsset\OnDiskUIAsset;
use Piwik\Theme;
abstract class UIAssetFetcher
{
/**
* @var UIAssetCatalog
*/
protected $catalog;
/**
* @var string[]
*/
protected $fileLocations = array();
/**
* @var string[]
*/
protected $plugins;
/**
* @var Theme
*/
private $theme;
/**
* @param string[] $plugins
* @param Theme $theme
*/
function __construct($plugins, $theme)
{
$this->plugins = $plugins;
$this->theme = $theme;
}
/**
* @return string[]
*/
public function getPlugins()
{
return $this->plugins;
}
/**
* $return UIAssetCatalog
*/
public function getCatalog()
{
if($this->catalog == null)
$this->createCatalog();
return $this->catalog;
}
abstract protected function retrieveFileLocations();
/**
* @return string[]
*/
abstract protected function getPriorityOrder();
private function createCatalog()
{
$this->retrieveFileLocations();
$this->initCatalog();
$this->populateCatalog();
$this->sortCatalog();
}
private function initCatalog()
{
$catalogSorter = new UIAssetCatalogSorter($this->getPriorityOrder());
$this->catalog = new UIAssetCatalog($catalogSorter);
}
private function populateCatalog()
{
foreach ($this->fileLocations as $fileLocation) {
$newUIAsset = new OnDiskUIAsset($this->getBaseDirectory(), $fileLocation);
$this->catalog->addUIAsset($newUIAsset);
}
}
private function sortCatalog()
{
$this->catalog = $this->catalog->getSortedCatalog();
}
/**
* @return string
*/
private function getBaseDirectory()
{
// served by web server directly, so must be a public path
return PIWIK_USER_PATH;
}
/**
* @return Theme
*/
public function getTheme()
{
return $this->theme;
}
}

90
www/analytics/core/AssetManager/UIAssetFetcher/JScriptUIAssetFetcher.php Normal file
View file

@ -0,0 +1,90 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\AssetManager\UIAssetFetcher;
use Piwik\AssetManager\UIAssetFetcher;
use Piwik\Piwik;
use string;
class JScriptUIAssetFetcher extends UIAssetFetcher
{
protected function retrieveFileLocations()
{
if(!empty($this->plugins)) {
/**
* Triggered when gathering the list of all JavaScript files needed by Piwik
* and its plugins.
*
* Plugins that have their own JavaScript should use this event to make those
* files load in the browser.
*
* JavaScript files should be placed within a **javascripts** subdirectory in your
* plugin's root directory.
*
* _Note: While you are developing your plugin you should enable the config setting
* `[Debug] disable_merged_assets` so JavaScript files will be reloaded immediately
* after every change._
*
* **Example**
*
* public function getJsFiles(&$jsFiles)
* {
* $jsFiles[] = "plugins/MyPlugin/javascripts/myfile.js";
* $jsFiles[] = "plugins/MyPlugin/javascripts/anotherone.js";
* }
*
* @param string[] $jsFiles The JavaScript files to load.
*/
Piwik::postEvent('AssetManager.getJavaScriptFiles', array(&$this->fileLocations), null, $this->plugins);
}
$this->addThemeFiles();
}
protected function addThemeFiles()
{
$theme = $this->getTheme();
if(!$theme) {
return;
}
if(in_array($theme->getThemeName(), $this->plugins)) {
$jsInThemes = $this->getTheme()->getJavaScriptFiles();
if(!empty($jsInThemes)) {
foreach($jsInThemes as $jsFile) {
$this->fileLocations[] = $jsFile;
}
}
}
}
protected function getPriorityOrder()
{
return array(
'libs/jquery/jquery.js',
'libs/jquery/jquery-ui.js',
'libs/jquery/jquery.browser.js',
'libs/',
'plugins/CoreHome/javascripts/require.js',
'plugins/Zeitgeist/javascripts/piwikHelper.js',
'plugins/Zeitgeist/javascripts/',
'plugins/CoreHome/javascripts/uiControl.js',
'plugins/CoreHome/javascripts/broadcast.js',
'plugins/CoreHome/javascripts/', // load CoreHome JS before other plugins
'plugins/',
'tests/',
);
}
}

37
www/analytics/core/AssetManager/UIAssetFetcher/StaticUIAssetFetcher.php Normal file
View file

@ -0,0 +1,37 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\AssetManager\UIAssetFetcher;
use Piwik\AssetManager\UIAssetFetcher;
class StaticUIAssetFetcher extends UIAssetFetcher
{
/**
* @var string[]
*/
private $priorityOrder;
function __construct($fileLocations, $priorityOrder, $theme)
{
parent::__construct(array(), $theme);
$this->fileLocations = $fileLocations;
$this->priorityOrder = $priorityOrder;
}
protected function retrieveFileLocations()
{
}
protected function getPriorityOrder()
{
return $this->priorityOrder;
}
}

64
www/analytics/core/AssetManager/UIAssetFetcher/StylesheetUIAssetFetcher.php Normal file
View file

@ -0,0 +1,64 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\AssetManager\UIAssetFetcher;
use Piwik\AssetManager\UIAssetFetcher;
use Piwik\Piwik;
class StylesheetUIAssetFetcher extends UIAssetFetcher
{
protected function getPriorityOrder()
{
return array(
'libs/',
'plugins/CoreHome/stylesheets/color_manager.css', // must be before other Piwik stylesheets
'plugins/Zeitgeist/stylesheets/base.less',
'plugins/Zeitgeist/stylesheets/',
'plugins/',
'plugins/Dashboard/stylesheets/dashboard.less',
'tests/',
);
}
protected function retrieveFileLocations()
{
/**
* Triggered when gathering the list of all stylesheets (CSS and LESS) needed by
* Piwik and its plugins.
*
* Plugins that have stylesheets should use this event to make those stylesheets
* load.
*
* Stylesheets should be placed within a **stylesheets** subdirectory in your plugin's
* root directory.
*
* **Example**
*
* public function getStylesheetFiles(&$stylesheets)
* {
* $stylesheets[] = "plugins/MyPlugin/stylesheets/myfile.less";
* $stylesheets[] = "plugins/MyPlugin/stylesheets/myotherfile.css";
* }
*
* @param string[] &$stylesheets The list of stylesheet paths.
*/
Piwik::postEvent('AssetManager.getStylesheetFiles', array(&$this->fileLocations));
$this->addThemeFiles();
}
protected function addThemeFiles()
{
$themeStylesheet = $this->getTheme()->getStylesheet();
if($themeStylesheet) {
$this->fileLocations[] = $themeStylesheet;
}
}
}

209
www/analytics/core/AssetManager/UIAssetMerger.php Normal file
View file

@ -0,0 +1,209 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\AssetManager;
use Piwik\AssetManager\PiwikLessCompiler;
use Piwik\AssetManager\UIAsset\StylesheetUIAsset;
use Piwik\AssetManager;
abstract class UIAssetMerger
{
/**
* @var UIAssetFetcher
*/
private $assetFetcher;
/**
* @var UIAsset
*/
private $mergedAsset;
/**
* @var string
*/
private $mergedContent;
/**
* @var UIAssetCacheBuster
*/
protected $cacheBuster;
/**
* @param UIAsset $mergedAsset
* @param UIAssetFetcher $assetFetcher
* @param UIAssetCacheBuster $cacheBuster
*/
function __construct($mergedAsset, $assetFetcher, $cacheBuster)
{
$this->mergedAsset = $mergedAsset;
$this->assetFetcher = $assetFetcher;
$this->cacheBuster = $cacheBuster;
}
public function generateFile()
{
if(!$this->shouldGenerate())
return;
$this->mergedContent = $this->getMergedAssets();
$this->postEvent($this->mergedContent);
$this->adjustPaths();
$this->addPreamble();
$this->writeContentToFile();
}
/**
* @return string
*/
abstract protected function getMergedAssets();
/**
* @return string
*/
abstract protected function generateCacheBuster();
/**
* @return string
*/
abstract protected function getPreamble();
/**
* @return string
*/
abstract protected function getFileSeparator();
/**
* @param UIAsset $uiAsset
* @return string
*/
abstract protected function processFileContent($uiAsset);
/**
* @param string $mergedContent
*/
abstract protected function postEvent(&$mergedContent);
protected function getConcatenatedAssets()
{
if(empty($this->mergedContent))
$this->concatenateAssets();
return $this->mergedContent;
}
private function concatenateAssets()
{
$mergedContent = '';
foreach ($this->getAssetCatalog()->getAssets() as $uiAsset) {
$uiAsset->validateFile();
$content = $this->processFileContent($uiAsset);
$mergedContent .= $this->getFileSeparator() . $content;
}
$this->mergedContent = $mergedContent;
}
/**
* @return string[]
*/
protected function getPlugins()
{
return $this->assetFetcher->getPlugins();
}
/**
* @return UIAssetCatalog
*/
protected function getAssetCatalog()
{
return $this->assetFetcher->getCatalog();
}
/**
* @return boolean
*/
private function shouldGenerate()
{
if(!$this->mergedAsset->exists())
return true;
return !$this->isFileUpToDate();
}
/**
* @return boolean
*/
private function isFileUpToDate()
{
$f = fopen($this->mergedAsset->getAbsoluteLocation(), 'r');
$firstLine = fgets($f);
fclose($f);
if (!empty($firstLine) && trim($firstLine) == trim($this->getCacheBusterValue())) {
return true;
}
// Some CSS file in the merge, has changed since last merged asset was generated
// Note: we do not detect changes in @import'ed LESS files
return false;
}
/**
* @return boolean
*/
private function isMergedAssetsDisabled()
{
return AssetManager::getInstance()->isMergedAssetsDisabled();
}
private function adjustPaths()
{
$theme = $this->assetFetcher->getTheme();
// During installation theme is not yet ready
if($theme) {
$this->mergedContent = $this->assetFetcher->getTheme()->rewriteAssetsPathToTheme($this->mergedContent);
}
}
private function writeContentToFile()
{
$this->mergedAsset->writeContent($this->mergedContent);
}
/**
* @return string
*/
protected function getCacheBusterValue()
{
if(empty($this->cacheBusterValue))
$this->cacheBusterValue = $this->generateCacheBuster();
return $this->cacheBusterValue;
}
private function addPreamble()
{
$this->mergedContent = $this->getPreamble() . $this->mergedContent;
}
/**
* @return boolean
*/
private function shouldCompareExistingVersion()
{
return $this->isMergedAssetsDisabled();
}
}

89
www/analytics/core/AssetManager/UIAssetMerger/JScriptUIAssetMerger.php Normal file
View file

@ -0,0 +1,89 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\AssetManager\UIAssetMerger;
use Piwik\AssetManager\UIAsset;
use Piwik\AssetManager\UIAssetCacheBuster;
use Piwik\AssetManager\UIAssetFetcher\JScriptUIAssetFetcher;
use Piwik\AssetManager\UIAssetMerger;
use Piwik\AssetManager;
use Piwik\AssetManager\UIAssetMinifier;
use Piwik\Piwik;
class JScriptUIAssetMerger extends UIAssetMerger
{
/**
* @var UIAssetMinifier
*/
private $assetMinifier;
/**
* @param UIAsset $mergedAsset
* @param JScriptUIAssetFetcher $assetFetcher
* @param UIAssetCacheBuster $cacheBuster
*/
function __construct($mergedAsset, $assetFetcher, $cacheBuster)
{
parent::__construct($mergedAsset, $assetFetcher, $cacheBuster);
$this->assetMinifier = UIAssetMinifier::getInstance();
}
protected function getMergedAssets()
{
$concatenatedAssets = $this->getConcatenatedAssets();
return str_replace("\n", "\r\n", $concatenatedAssets);
}
protected function generateCacheBuster()
{
$cacheBuster = $this->cacheBuster->piwikVersionBasedCacheBuster($this->getPlugins());
return "/* Piwik Javascript - cb=" . $cacheBuster . "*/\r\n";
}
protected function getPreamble()
{
return $this->getCacheBusterValue();
}
protected function postEvent(&$mergedContent)
{
$plugins = $this->getPlugins();
if(!empty($plugins)) {
/**
* Triggered after all the JavaScript files Piwik uses are minified and merged into a
* single file, but before the merged JavaScript is written to disk.
*
* Plugins can use this event to modify merged JavaScript or do something else
* with it.
*
* @param string $mergedContent The minified and merged JavaScript.
*/
Piwik::postEvent('AssetManager.filterMergedJavaScripts', array(&$mergedContent), null, $plugins);
}
}
public function getFileSeparator()
{
return PHP_EOL;
}
protected function processFileContent($uiAsset)
{
$content = $uiAsset->getContent();
if (!$this->assetMinifier->isMinifiedJs($content))
$content = $this->assetMinifier->minifyJs($content);
return $content;
}
}

147
www/analytics/core/AssetManager/UIAssetMerger/StylesheetUIAssetMerger.php Normal file
View file

@ -0,0 +1,147 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\AssetManager\UIAssetMerger;
use Exception;
use Piwik\AssetManager\UIAsset;
use Piwik\AssetManager\UIAssetMerger;
use Piwik\Piwik;
use lessc;
class StylesheetUIAssetMerger extends UIAssetMerger
{
/**
* @var lessc
*/
private $lessCompiler;
function __construct($mergedAsset, $assetFetcher, $cacheBuster)
{
parent::__construct($mergedAsset, $assetFetcher, $cacheBuster);
$this->lessCompiler = self::getLessCompiler();
}
protected function getMergedAssets()
{
foreach($this->getAssetCatalog()->getAssets() as $uiAsset) {
$content = $uiAsset->getContent();
if (false !== strpos($content, '@import')) {
$this->lessCompiler->addImportDir(dirname($uiAsset->getAbsoluteLocation()));
}
}
return $this->lessCompiler->compile($this->getConcatenatedAssets());
}
/**
* @return lessc
* @throws Exception
*/
private static function getLessCompiler()
{
if (!class_exists("lessc")) {
throw new Exception("Less was added to composer during 2.0. ==> Execute this command to update composer packages: \$ php composer.phar install");
}
$less = new lessc();
return $less;
}
protected function generateCacheBuster()
{
$fileHash = $this->cacheBuster->md5BasedCacheBuster($this->getConcatenatedAssets());
return "/* compile_me_once=$fileHash */";
}
protected function getPreamble()
{
return $this->getCacheBusterValue() . "\n"
. "/* Piwik CSS file is compiled with Less. You may be interested in writing a custom Theme for Piwik! */\n";
}
protected function postEvent(&$mergedContent)
{
/**
* Triggered after all less stylesheets are compiled to CSS, minified and merged into
* one file, but before the generated CSS is written to disk.
*
* This event can be used to modify merged CSS.
*
* @param string $mergedContent The merged and minified CSS.
*/
Piwik::postEvent('AssetManager.filterMergedStylesheets', array(&$mergedContent));
}
public function getFileSeparator()
{
return '';
}
protected function processFileContent($uiAsset)
{
return $this->rewriteCssPathsDirectives($uiAsset);
}
/**
* Rewrite css url directives
* - rewrites paths defined relatively to their css/less definition file
* - rewrite windows directory separator \\ to /
*
* @param UIAsset $uiAsset
* @return string
*/
private function rewriteCssPathsDirectives($uiAsset)
{
static $rootDirectoryLength = null;
if (is_null($rootDirectoryLength)) {
$rootDirectoryLength = self::countDirectoriesInPathToRoot($uiAsset);
}
$baseDirectory = dirname($uiAsset->getRelativeLocation());
$content = preg_replace_callback(
"/(url\(['\"]?)([^'\")]*)/",
function ($matches) use ($rootDirectoryLength, $baseDirectory) {
$absolutePath = realpath(PIWIK_USER_PATH . "/$baseDirectory/" . $matches[2]);
if($absolutePath) {
$relativePath = substr($absolutePath, $rootDirectoryLength);
$relativePath = str_replace('\\', '/', $relativePath);
return $matches[1] . $relativePath;
} else {
return $matches[1] . $matches[2];
}
},
$uiAsset->getContent()
);
return $content;
}
/**
* @param UIAsset $uiAsset
* @return int
*/
protected function countDirectoriesInPathToRoot($uiAsset)
{
$rootDirectory = realpath($uiAsset->getBaseDirectory());
if ($rootDirectory != PATH_SEPARATOR
&& substr_compare($rootDirectory, PATH_SEPARATOR, -1)) {
$rootDirectory .= PATH_SEPARATOR;
}
$rootDirectoryLen = strlen($rootDirectory);
return $rootDirectoryLen;
}
}

66
www/analytics/core/AssetManager/UIAssetMinifier.php Normal file
View file

@ -0,0 +1,66 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
* @method static \Piwik\AssetManager\UIAssetMinifier getInstance()
*/
namespace Piwik\AssetManager;
use Exception;
use Piwik\Singleton;
use JShrink\Minifier;
class UIAssetMinifier extends Singleton
{
const MINIFIED_JS_RATIO = 100;
protected function __construct()
{
self::validateDependency();
parent::__construct();
}
/**
* Indicates if the provided JavaScript content has already been minified or not.
* The heuristic is based on a custom ratio : (size of file) / (number of lines).
* The threshold (100) has been found empirically on existing files :
* - the ratio never exceeds 50 for non-minified content and
* - it never goes under 150 for minified content.
*
* @param string $content Contents of the JavaScript file
* @return boolean
*/
public function isMinifiedJs($content)
{
$lineCount = substr_count($content, "\n");
if ($lineCount == 0) {
return true;
}
$contentSize = strlen($content);
$ratio = $contentSize / $lineCount;
return $ratio > self::MINIFIED_JS_RATIO;
}
/**
* @param string $content
* @return string
*/
public function minifyJs($content)
{
return Minifier::minify($content);
}
private static function validateDependency()
{
if (!class_exists("JShrink\\Minifier"))
throw new Exception("JShrink could not be found, maybe you are using Piwik from git and need to have update Composer. <br>php composer.phar update");
}
}

147
www/analytics/core/Auth.php Normal file
View file

@ -0,0 +1,147 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik;
/**
* Base for authentication modules
*/
interface Auth
{
/**
* Authentication module's name, e.g., "Login"
*
* @return string
*/
public function getName();
/**
* Authenticates user
*
* @return AuthResult
*/
public function authenticate();
/**
* Authenticates the user and initializes the session.
*/
public function initSession($login, $md5Password, $rememberMe);
/**
* Accessor to set authentication token. If set, you can authenticate the tokenAuth by calling the authenticate()
* method afterwards.
*
* @param string $token_auth authentication token
*/
public function setTokenAuth($token_auth);
/**
* Accessor to set login name
*
* @param string $login user login
*/
public function setLogin($login);
}
/**
* Authentication result
*
*/
class AuthResult
{
const FAILURE = 0;
const SUCCESS = 1;
const SUCCESS_SUPERUSER_AUTH_CODE = 42;
/**
* token_auth parameter used to authenticate in the API
*
* @var string
*/
protected $tokenAuth = null;
/**
* The login used to authenticate.
*
* @var string
*/
protected $login = null;
/**
* The authentication result code. Can be self::FAILURE, self::SUCCESS, or
* self::SUCCESS_SUPERUSER_AUTH_CODE.
*
* @var int
*/
protected $code = null;
/**
* Constructor for AuthResult
*
* @param int $code
* @param string $login identity
* @param string $tokenAuth
*/
public function __construct($code, $login, $tokenAuth)
{
$this->code = (int)$code;
$this->login = $login;
$this->tokenAuth = $tokenAuth;
}
/**
* Returns the login used to authenticate.
*
* @return string
*/
public function getIdentity()
{
return $this->login;
}
/**
* Returns the token_auth to authenticate the current user in the API
*
* @return string
*/
public function getTokenAuth()
{
return $this->tokenAuth;
}
/**
* Returns the authentication result code.
*
* @return int
*/
public function getCode()
{
return $this->code;
}
/**
* Returns true if the user has Super User access, false otherwise.
*
* @return bool
*/
public function hasSuperUserAccess()
{
return $this->getCode() == self::SUCCESS_SUPERUSER_AUTH_CODE;
}
/**
* Returns true if this result was successfully authentication.
*
* @return bool
*/
public function wasAuthenticationSuccessful()
{
return $this->code > self::FAILURE;
}
}

206
www/analytics/core/CacheFile.php Normal file
View file

@ -0,0 +1,206 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik;
use Exception;
/**
* This class is used to cache data on the filesystem.
*
* It is for example used by the Tracker process to cache various settings and websites attributes in tmp/cache/tracker/*
*
*/
class CacheFile
{
// for testing purposes since tests run on both CLI/FPM (changes in CLI can't invalidate
// opcache in FPM, so we have to invalidate before reading)
public static $invalidateOpCacheBeforeRead = false;
/**
* @var string
*/
protected $cachePath;
/**
* @var
*/
protected $cachePrefix;
/**
* Minimum enforced TTL in seconds
*/
const MINIMUM_TTL = 60;
/**
* @param string $directory directory to use
* @param int $timeToLiveInSeconds TTL
*/
public function __construct($directory, $timeToLiveInSeconds = 300)
{
$cachePath = PIWIK_USER_PATH . '/tmp/cache/' . $directory . '/';
$this->cachePath = SettingsPiwik::rewriteTmpPathWithHostname($cachePath);
if ($timeToLiveInSeconds < self::MINIMUM_TTL) {
$timeToLiveInSeconds = self::MINIMUM_TTL;
}
$this->ttl = $timeToLiveInSeconds;
}
/**
* Function to fetch a cache entry
*
* @param string $id The cache entry ID
* @return array|bool False on error, or array the cache content
*/
public function get($id)
{
if (empty($id)) {
return false;
}
$id = $this->cleanupId($id);
$cache_complete = false;
$content = '';
$expires_on = false;
// We are assuming that most of the time cache will exists
$cacheFilePath = $this->cachePath . $id . '.php';
if (self::$invalidateOpCacheBeforeRead) {
$this->opCacheInvalidate($cacheFilePath);
}
$ok = @include($cacheFilePath);
if ($ok && $cache_complete == true) {
if (empty($expires_on)
|| $expires_on < time()
) {
return false;
}
return $content;
}
return false;
}
private function getExpiresTime()
{
return time() + $this->ttl;
}
protected function cleanupId($id)
{
if (!Filesystem::isValidFilename($id)) {
throw new Exception("Invalid cache ID request $id");
}
return $id;
}
/**
* A function to store content a cache entry.
*
* @param string $id The cache entry ID
* @param array $content The cache content
* @throws \Exception
* @return bool True if the entry was succesfully stored
*/
public function set($id, $content)
{
if (empty($id)) {
return false;
}
if (!is_dir($this->cachePath)) {
Filesystem::mkdir($this->cachePath);
}
if (!is_writable($this->cachePath)) {
return false;
}
$id = $this->cleanupId($id);
$id = $this->cachePath . $id . '.php';
if (is_object($content)) {
throw new \Exception('You cannot use the CacheFile to cache an object, only arrays, strings and numbers.');
}
$cache_literal = "<" . "?php\n";
$cache_literal .= "$" . "content = " . var_export($content, true) . ";\n";
$cache_literal .= "$" . "expires_on = " . $this->getExpiresTime() . ";\n";
$cache_literal .= "$" . "cache_complete = true;\n";
$cache_literal .= "?" . ">";
// Write cache to a temp file, then rename it, overwriting the old cache
// On *nix systems this should guarantee atomicity
$tmp_filename = tempnam($this->cachePath, 'tmp_');
@chmod($tmp_filename, 0640);
if ($fp = @fopen($tmp_filename, 'wb')) {
@fwrite($fp, $cache_literal, strlen($cache_literal));
@fclose($fp);
if (!@rename($tmp_filename, $id)) {
// On some systems rename() doesn't overwrite destination
@unlink($id);
if (!@rename($tmp_filename, $id)) {
// Make sure that no temporary file is left over
// if the destination is not writable
@unlink($tmp_filename);
}
}
$this->opCacheInvalidate($id);
return true;
}
return false;
}
/**
* A function to delete a single cache entry
*
* @param string $id The cache entry ID
* @return bool True if the entry was succesfully deleted
*/
public function delete($id)
{
if (empty($id)) {
return false;
}
$id = $this->cleanupId($id);
$filename = $this->cachePath . $id . '.php';
if (file_exists($filename)) {
$this->opCacheInvalidate($filename);
@unlink($filename);
return true;
}
return false;
}
/**
* A function to delete all cache entries in the directory
*/
public function deleteAll()
{
$self = $this;
$beforeUnlink = function ($path) use ($self) {
$self->opCacheInvalidate($path);
};
Filesystem::unlinkRecursive($this->cachePath, $deleteRootToo = false, $beforeUnlink);
}
public function opCacheInvalidate($filepath)
{
if (function_exists('opcache_invalidate')
&& is_file($filepath)
) {
opcache_invalidate($filepath, $force = true);
}
}
}

280
www/analytics/core/CliMulti.php Normal file
View file

@ -0,0 +1,280 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*/
namespace Piwik;
use Piwik\CliMulti\Output;
use Piwik\CliMulti\Process;
/**
* Class CliMulti.
*/
class CliMulti {
/**
* If set to true or false it will overwrite whether async is supported or not.
*
* @var null|bool
*/
public $supportsAsync = null;
/**
* @var \Piwik\CliMulti\Process[]
*/
private $processes = array();
/**
* @var \Piwik\CliMulti\Output[]
*/
private $outputs = array();
private $acceptInvalidSSLCertificate = false;
public function __construct()
{
$this->supportsAsync = $this->supportsAsync();
}
/**
* It will request all given URLs in parallel (async) using the CLI and wait until all requests are finished.
* If multi cli is not supported (eg windows) it will initiate an HTTP request instead (not async).
*
* @param string[] $piwikUrls An array of urls, for instance:
* array('http://www.example.com/piwik?module=API...')
* @return array The response of each URL in the same order as the URLs. The array can contain null values in case
* there was a problem with a request, for instance if the process died unexpected.
*/
public function request(array $piwikUrls)
{
$this->start($piwikUrls);
do {
usleep(100000); // 100 * 1000 = 100ms
} while (!$this->hasFinished());
$results = $this->getResponse($piwikUrls);
$this->cleanup();
self::cleanupNotRemovedFiles();
return $results;
}
/**
* Ok, this sounds weird. Why should we care about ssl certificates when we are in CLI mode? It is needed for
* our simple fallback mode for Windows where we initiate HTTP requests instead of CLI.
* @param $acceptInvalidSSLCertificate
*/
public function setAcceptInvalidSSLCertificate($acceptInvalidSSLCertificate)
{
$this->acceptInvalidSSLCertificate = $acceptInvalidSSLCertificate;
}
private function start($piwikUrls)
{
foreach ($piwikUrls as $index => $url) {
$cmdId = $this->generateCommandId($url) . $index;
$output = new Output($cmdId);
if ($this->supportsAsync) {
$this->executeAsyncCli($url, $output, $cmdId);
} else {
$this->executeNotAsyncHttp($url, $output);
}
$this->outputs[] = $output;
}
}
private function buildCommand($hostname, $query, $outputFile)
{
$bin = $this->findPhpBinary();
return sprintf('%s -q %s/console climulti:request --piwik-domain=%s %s > %s 2>&1 &',
$bin, PIWIK_INCLUDE_PATH, escapeshellarg($hostname), escapeshellarg($query), $outputFile);
}
private function getResponse()
{
$response = array();
foreach ($this->outputs as $output) {
$response[] = $output->get();
}
return $response;
}
private function hasFinished()
{
foreach ($this->processes as $index => $process) {
$hasStarted = $process->hasStarted();
if (!$hasStarted && 8 <= $process->getSecondsSinceCreation()) {
// if process was created more than 8 seconds ago but still not started there must be something wrong.
// ==> declare the process as finished
$process->finishProcess();
continue;
} elseif (!$hasStarted) {
return false;
}
if ($process->isRunning()) {
return false;
}
if ($process->hasFinished()) {
// prevent from checking this process over and over again
unset($this->processes[$index]);
}
}
return true;
}
private function generateCommandId($command)
{
return substr(Common::hash($command . microtime(true) . rand(0, 99999)), 0, 100);
}
/**
* What is missing under windows? Detection whether a process is still running in Process::isProcessStillRunning
* and how to send a process into background in start()
*/
private function supportsAsync()
{
return !SettingsServer::isWindows()
&& Process::isSupported()
&& $this->findPhpBinary();
}
private function cleanup()
{
foreach ($this->processes as $pid) {
$pid->finishProcess();
}
foreach ($this->outputs as $output) {
$output->destroy();
}
$this->processes = array();
$this->outputs = array();
}
/**
* Remove files older than one week. They should be cleaned up automatically after each request but for whatever
* reason there can be always some files left.
*/
public static function cleanupNotRemovedFiles()
{
$timeOneWeekAgo = strtotime('-1 week');
$files = _glob(self::getTmpPath() . '/*');
if(empty($files)) {
return;
}
foreach ($files as $file) {
$timeLastModified = filemtime($file);
if ($timeOneWeekAgo > $timeLastModified) {
unlink($file);
}
}
}
public static function getTmpPath()
{
$dir = PIWIK_INCLUDE_PATH . '/tmp/climulti';
return SettingsPiwik::rewriteTmpPathWithHostname($dir);
}
private function findPhpBinary()
{
if (defined('PHP_BINARY') && false === strpos(PHP_BINARY, 'fpm')) {
return PHP_BINARY;
}
$bin = '';
if (!empty($_SERVER['_']) && Common::isPhpCliMode()) {
$bin = $this->getPhpCommandIfValid($_SERVER['_']);
}
if (empty($bin) && !empty($_SERVER['argv'][0]) && Common::isPhpCliMode()) {
$bin = $this->getPhpCommandIfValid($_SERVER['argv'][0]);
}
if (empty($bin)) {
$bin = shell_exec('which php');
}
if (empty($bin)) {
$bin = shell_exec('which php5');
}
if (!empty($bin)) {
return trim($bin);
}
}
private function executeAsyncCli($url, Output $output, $cmdId)
{
$this->processes[] = new Process($cmdId);
$url = $this->appendTestmodeParamToUrlIfNeeded($url);
$query = UrlHelper::getQueryFromUrl($url, array('pid' => $cmdId));
$hostname = UrlHelper::getHostFromUrl($url);
$command = $this->buildCommand($hostname, $query, $output->getPathToFile());
Log::debug($command);
shell_exec($command);
}
private function executeNotAsyncHttp($url, Output $output)
{
try {
Log::debug("Execute HTTP API request: " . $url);
$response = Http::sendHttpRequestBy('curl', $url, $timeout = 0, $userAgent = null, $destinationPath = null, $file = null, $followDepth = 0, $acceptLanguage = false, $this->acceptInvalidSSLCertificate);
$output->write($response);
} catch (\Exception $e) {
$message = "Got invalid response from API request: $url. ";
if (empty($response)) {
$message .= "The response was empty. This usually means a server error. This solution to this error is generally to increase the value of 'memory_limit' in your php.ini file. Please check your Web server Error Log file for more details.";
} else {
$message .= "Response was '" . $e->getMessage() . "'";
}
$output->write($message);
}
}
private function appendTestmodeParamToUrlIfNeeded($url)
{
$isTestMode = $url && false !== strpos($url, 'tests/PHPUnit/proxy');
if ($isTestMode && false === strpos($url, '?')) {
$url .= "?testmode=1";
} elseif ($isTestMode) {
$url .= "&testmode=1";
}
return $url;
}
private function getPhpCommandIfValid($path)
{
if (!empty($path) && is_executable($path)) {
if (0 === strpos($path, PHP_BINDIR) && false === strpos($path, 'phpunit')) {
return $path;
}
}
}
}

53
www/analytics/core/CliMulti/Output.php Normal file
View file

@ -0,0 +1,53 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*/
namespace Piwik\CliMulti;
use Piwik\CliMulti;
use Piwik\Filesystem;
class Output {
private $tmpFile = '';
public function __construct($outputId)
{
if (!Filesystem::isValidFilename($outputId)) {
throw new \Exception('The given output id has an invalid format');
}
$dir = CliMulti::getTmpPath();
Filesystem::mkdir($dir, true);
$this->tmpFile = $dir . '/' . $outputId . '.output';
}
public function write($content)
{
file_put_contents($this->tmpFile, $content);
}
public function getPathToFile()
{
return $this->tmpFile;
}
public function exists()
{
return file_exists($this->tmpFile);
}
public function get()
{
return @file_get_contents($this->tmpFile);
}
public function destroy()
{
Filesystem::deleteFileIfExists($this->tmpFile);
}
}

195
www/analytics/core/CliMulti/Process.php Normal file
View file

@ -0,0 +1,195 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*/
namespace Piwik\CliMulti;
use Piwik\CliMulti;
use Piwik\Filesystem;
use Piwik\SettingsServer;
/**
* There are three different states
* - PID file exists with empty content: Process is created but not started
* - PID file exists with the actual process PID as content: Process is runnning
* - PID file does not exist: Process is marked as finished
*
* Class Process
*/
class Process
{
private $pidFile = '';
private $timeCreation = null;
private $isSupported = null;
public function __construct($pid)
{
if (!Filesystem::isValidFilename($pid)) {
throw new \Exception('The given pid has an invalid format');
}
$pidDir = CliMulti::getTmpPath();
Filesystem::mkdir($pidDir, true);
$this->isSupported = self::isSupported();
$this->pidFile = $pidDir . '/' . $pid . '.pid';
$this->timeCreation = time();
$this->markAsNotStarted();
}
private function markAsNotStarted()
{
$content = $this->getPidFileContent();
if ($this->doesPidFileExist($content)) {
return;
}
$this->writePidFileContent('');
}
public function hasStarted($content = null)
{
if (is_null($content)) {
$content = $this->getPidFileContent();
}
if (!$this->doesPidFileExist($content)) {
// process is finished, this means there was a start before
return true;
}
if ('' === trim($content)) {
// pid file is overwritten by startProcess()
return false;
}
// process is probably running or pid file was not removed
return true;
}
public function hasFinished()
{
$content = $this->getPidFileContent();
return !$this->doesPidFileExist($content);
}
public function getSecondsSinceCreation()
{
return time() - $this->timeCreation;
}
public function startProcess()
{
$this->writePidFileContent(getmypid());
}
public function isRunning()
{
$content = $this->getPidFileContent();
if (!$this->doesPidFileExist($content)) {
return false;
}
if ($this->isProcessStillRunning($content)) {
return true;
}
if ($this->hasStarted($content)) {
$this->finishProcess();
}
return false;
}
public function finishProcess()
{
Filesystem::deleteFileIfExists($this->pidFile);
}
private function doesPidFileExist($content)
{
return false !== $content;
}
private function isProcessStillRunning($content)
{
if (!$this->isSupported) {
return true;
}
$lockedPID = trim($content);
$runningPIDs = explode("\n", trim( `ps -e | awk '{print $1}'` ));
return !empty($lockedPID) && in_array($lockedPID, $runningPIDs);
}
private function getPidFileContent()
{
return @file_get_contents($this->pidFile);
}
private function writePidFileContent($content)
{
file_put_contents($this->pidFile, $content);
}
public static function isSupported()
{
if (SettingsServer::isWindows()) {
return false;
}
if (self::shellExecFunctionIsDisabled()) {
return false;
}
if (self::isSystemNotSupported()) {
return false;
}
if (static::commandExists('ps') && self::returnsSuccessCode('ps') && self::commandExists('awk')) {
return true;
}
return false;
}
private static function isSystemNotSupported()
{
$uname = shell_exec('uname -a');
if(strpos($uname, 'synology') !== false) {
return true;
}
return false;
}
private static function shellExecFunctionIsDisabled()
{
$command = 'shell_exec';
$disabled = explode(',', ini_get('disable_functions'));
$disabled = array_map('trim', $disabled);
return in_array($command, $disabled);
}
private static function returnsSuccessCode($command)
{
$exec = $command . ' > /dev/null 2>&1 & echo $?';
$returnCode = shell_exec($exec);
$returnCode = trim($returnCode);
return 0 == (int) $returnCode;
}
private static function commandExists($command)
{
$result = shell_exec('which ' . escapeshellarg($command));
return !empty($result);
}
}

84
www/analytics/core/CliMulti/RequestCommand.php Normal file
View file

@ -0,0 +1,84 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*/
namespace Piwik\CliMulti;
use Piwik\Plugin\ConsoleCommand;
use Piwik\Url;
use Piwik\UrlHelper;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Output\OutputInterface;
use Piwik\Config;
use Piwik\Common;
use Piwik\FrontController;
/**
* RequestCommand
*/
class RequestCommand extends ConsoleCommand
{
protected function configure()
{
$this->setName('climulti:request');
$this->setDescription('Parses and executes the given query. See Piwik\CliMulti. Intended only for system usage.');
$this->addArgument('url-query', null, InputOption::VALUE_REQUIRED, 'Piwik URL query string, for instance: "module=API&method=API.getPiwikVersion&token_auth=123456789"');
}
protected function execute(InputInterface $input, OutputInterface $output)
{
$this->initHostAndQueryString($input);
if ($this->isTestModeEnabled()) {
Config::getInstance()->setTestEnvironment();
$indexFile = '/tests/PHPUnit/proxy/index.php';
} else {
$indexFile = '/index.php';
}
if (!empty($_GET['pid'])) {
$process = new Process($_GET['pid']);
if ($process->hasFinished()) {
return;
}
$process->startProcess();
}
require_once PIWIK_INCLUDE_PATH . $indexFile;
if (!empty($process)) {
$process->finishProcess();
}
}
private function isTestModeEnabled()
{
return !empty($_GET['testmode']);
}
/**
* @param InputInterface $input
*/
protected function initHostAndQueryString(InputInterface $input)
{
$_GET = array();
$hostname = $input->getOption('piwik-domain');
Url::setHost($hostname);
$query = $input->getArgument('url-query');
$query = UrlHelper::getArrayFromQueryString($query);
foreach ($query as $name => $value) {
$_GET[$name] = $value;
}
}
}

1088
www/analytics/core/Common.php Normal file
View file

File diff suppressed because it is too large Load diff

714
www/analytics/core/Config.php Normal file
View file

@ -0,0 +1,714 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik;
use Exception;
/**
* Singleton that provides read & write access to Piwik's INI configuration.
*
* This class reads and writes to the `config/config.ini.php` file. If config
* options are missing from that file, this class will look for their default
* values in `config/global.ini.php`.
*
* ### Examples
*
* **Getting a value:**
*
* // read the minimum_memory_limit option under the [General] section
* $minValue = Config::getInstance()->General['minimum_memory_limit'];
*
* **Setting a value:**
*
* // set the minimum_memory_limit option
* Config::getInstance()->General['minimum_memory_limit'] = 256;
* Config::getInstance()->forceSave();
*
* **Setting an entire section:**
*
* Config::getInstance()->MySection = array('myoption' => 1);
* Config::getInstance()->forceSave();
*
* @method static \Piwik\Config getInstance()
*/
class Config extends Singleton
{
const DEFAULT_LOCAL_CONFIG_PATH = '/config/config.ini.php';
const DEFAULT_COMMON_CONFIG_PATH = '/config/common.config.ini.php';
const DEFAULT_GLOBAL_CONFIG_PATH = '/config/global.ini.php';
/**
* Contains configuration files values
*
* @var array
*/
protected $initialized = false;
protected $configGlobal = array();
protected $configLocal = array();
protected $configCommon = array();
protected $configCache = array();
protected $pathGlobal = null;
protected $pathCommon = null;
protected $pathLocal = null;
/**
* @var boolean
*/
protected $isTest = false;
/**
* Constructor
*/
public function __construct($pathGlobal = null, $pathLocal = null, $pathCommon = null)
{
$this->pathGlobal = $pathGlobal ?: self::getGlobalConfigPath();
$this->pathCommon = $pathCommon ?: self::getCommonConfigPath();
$this->pathLocal = $pathLocal ?: self::getLocalConfigPath();
}
/**
* Returns the path to the local config file used by this instance.
*
* @return string
*/
public function getLocalPath()
{
return $this->pathLocal;
}
/**
* Returns the path to the global config file used by this instance.
*
* @return string
*/
public function getGlobalPath()
{
return $this->pathGlobal;
}
/**
* Returns the path to the common config file used by this instance.
*
* @return string
*/
public function getCommonPath()
{
return $this->pathCommon;
}
/**
* Enable test environment
*
* @param string $pathLocal
* @param string $pathGlobal
* @param string $pathCommon
*/
public function setTestEnvironment($pathLocal = null, $pathGlobal = null, $pathCommon = null, $allowSaving = false)
{
if (!$allowSaving) {
$this->isTest = true;
}
$this->clear();
$this->pathLocal = $pathLocal ?: Config::getLocalConfigPath();
$this->pathGlobal = $pathGlobal ?: Config::getGlobalConfigPath();
$this->pathCommon = $pathCommon ?: Config::getCommonConfigPath();
$this->init();
// this proxy will not record any data in the production database.
// this provides security for Piwik installs and tests were setup.
if (isset($this->configGlobal['database_tests'])
|| isset($this->configLocal['database_tests'])
) {
$this->__get('database_tests');
$this->configCache['database'] = $this->configCache['database_tests'];
}
// Ensure local mods do not affect tests
if (empty($pathGlobal)) {
$this->configCache['log'] = $this->configGlobal['log'];
$this->configCache['Debug'] = $this->configGlobal['Debug'];
$this->configCache['mail'] = $this->configGlobal['mail'];
$this->configCache['General'] = $this->configGlobal['General'];
$this->configCache['Segments'] = $this->configGlobal['Segments'];
$this->configCache['Tracker'] = $this->configGlobal['Tracker'];
$this->configCache['Deletelogs'] = $this->configGlobal['Deletelogs'];
$this->configCache['Deletereports'] = $this->configGlobal['Deletereports'];
}
// for unit tests, we set that no plugin is installed. This will force
// the test initialization to create the plugins tables, execute ALTER queries, etc.
$this->configCache['PluginsInstalled'] = array('PluginsInstalled' => array());
// DevicesDetection plugin is not yet enabled by default
if (isset($configGlobal['Plugins'])) {
$this->configCache['Plugins'] = $this->configGlobal['Plugins'];
$this->configCache['Plugins']['Plugins'][] = 'DevicesDetection';
}
if (isset($configGlobal['Plugins_Tracker'])) {
$this->configCache['Plugins_Tracker'] = $this->configGlobal['Plugins_Tracker'];
$this->configCache['Plugins_Tracker']['Plugins_Tracker'][] = 'DevicesDetection';
}
// to avoid weird session error in travis
if (empty($pathGlobal)) {
$configArray = &$this->configCache;
} else {
$configArray = &$this->configLocal;
}
$configArray['General']['session_save_handler'] = 'dbtables';
}
/**
* Returns absolute path to the global configuration file
*
* @return string
*/
protected static function getGlobalConfigPath()
{
return PIWIK_USER_PATH . self::DEFAULT_GLOBAL_CONFIG_PATH;
}
/**
* Returns absolute path to the common configuration file.
*
* @return string
*/
public static function getCommonConfigPath()
{
return PIWIK_USER_PATH . self::DEFAULT_COMMON_CONFIG_PATH;
}
/**
* Returns absolute path to the local configuration file
*
* @return string
*/
public static function getLocalConfigPath()
{
$path = self::getByDomainConfigPath();
if ($path) {
return $path;
}
return PIWIK_USER_PATH . self::DEFAULT_LOCAL_CONFIG_PATH;
}
private static function getLocalConfigInfoForHostname($hostname)
{
$perHostFilename = $hostname . '.config.ini.php';
$pathDomainConfig = PIWIK_USER_PATH . '/config/' . $perHostFilename;
return array('file' => $perHostFilename, 'path' => $pathDomainConfig);
}
public function getConfigHostnameIfSet()
{
if ($this->getByDomainConfigPath() === false) {
return false;
}
return $this->getHostname();
}
public function getClientSideOptions()
{
$general = $this->General;
return array(
'action_url_category_delimiter' => $general['action_url_category_delimiter'],
'autocomplete_min_sites' => $general['autocomplete_min_sites'],
'datatable_export_range_as_day' => $general['datatable_export_range_as_day']
);
}
protected static function getByDomainConfigPath()
{
$host = self::getHostname();
$hostConfig = self::getLocalConfigInfoForHostname($host);
if (Filesystem::isValidFilename($hostConfig['file'])
&& file_exists($hostConfig['path'])
) {
return $hostConfig['path'];
}
return false;
}
protected static function getHostname()
{
$host = Url::getHost($checkIfTrusted = false); // Check trusted requires config file which is not ready yet
return $host;
}
/**
* If set, Piwik will use the hostname config no matter if it exists or not. Useful for instance if you want to
* create a new hostname config:
*
* $config = Config::getInstance();
* $config->forceUsageOfHostnameConfig('piwik.example.com');
* $config->save();
*
* @param string $hostname eg piwik.example.com
* @return string
* @throws \Exception In case the domain contains not allowed characters
*/
public function forceUsageOfLocalHostnameConfig($hostname)
{
$hostConfig = static::getLocalConfigInfoForHostname($hostname);
if (!Filesystem::isValidFilename($hostConfig['file'])) {
throw new Exception('Hostname is not valid');
}
$this->pathLocal = $hostConfig['path'];
$this->configLocal = array();
$this->initialized = false;
return $this->pathLocal;
}
/**
* Returns `true` if the local configuration file is writable.
*
* @return bool
*/
public function isFileWritable()
{
return is_writable($this->pathLocal);
}
/**
* Clear in-memory configuration so it can be reloaded
*/
public function clear()
{
$this->configGlobal = array();
$this->configLocal = array();
$this->configCache = array();
$this->initialized = false;
}
/**
* Read configuration from files into memory
*
* @throws Exception if local config file is not readable; exits for other errors
*/
public function init()
{
$this->initialized = true;
$reportError = SettingsServer::isTrackerApiRequest();
// read defaults from global.ini.php
if (!is_readable($this->pathGlobal) && $reportError) {
Piwik_ExitWithMessage(Piwik::translate('General_ExceptionConfigurationFileNotFound', array($this->pathGlobal)));
}
$this->configGlobal = _parse_ini_file($this->pathGlobal, true);
if (empty($this->configGlobal) && $reportError) {
Piwik_ExitWithMessage(Piwik::translate('General_ExceptionUnreadableFileDisabledMethod', array($this->pathGlobal, "parse_ini_file()")));
}
$this->configCommon = _parse_ini_file($this->pathCommon, true);
// Check config.ini.php last
$this->checkLocalConfigFound();
$this->configLocal = _parse_ini_file($this->pathLocal, true);
if (empty($this->configLocal) && $reportError) {
Piwik_ExitWithMessage(Piwik::translate('General_ExceptionUnreadableFileDisabledMethod', array($this->pathLocal, "parse_ini_file()")));
}
}
public function existsLocalConfig()
{
return is_readable($this->pathLocal);
}
public function checkLocalConfigFound()
{
if (!$this->existsLocalConfig()) {
throw new Exception(Piwik::translate('General_ExceptionConfigurationFileNotFound', array($this->pathLocal)));
}
}
/**
* Decode HTML entities
*
* @param mixed $values
* @return mixed
*/
protected function decodeValues($values)
{
if (is_array($values)) {
foreach ($values as &$value) {
$value = $this->decodeValues($value);
}
return $values;
}
return html_entity_decode($values, ENT_COMPAT, 'UTF-8');
}
/**
* Encode HTML entities
*
* @param mixed $values
* @return mixed
*/
protected function encodeValues($values)
{
if (is_array($values)) {
foreach ($values as &$value) {
$value = $this->encodeValues($value);
}
} else {
$values = htmlentities($values, ENT_COMPAT, 'UTF-8');
}
return $values;
}
/**
* Returns a configuration value or section by name.
*
* @param string $name The value or section name.
* @return string|array The requested value requested. Returned by reference.
* @throws Exception If the value requested not found in either `config.ini.php` or
* `global.ini.php`.
* @api
*/
public function &__get($name)
{
if (!$this->initialized) {
$this->init();
// must be called here, not in init(), since setTestEnvironment() calls init(). (this avoids
// infinite recursion)
Piwik::postTestEvent('Config.createConfigSingleton',
array($this, &$this->configCache, &$this->configLocal));
}
// check cache for merged section
if (isset($this->configCache[$name])) {
$tmp =& $this->configCache[$name];
return $tmp;
}
$section = $this->getFromGlobalConfig($name);
$sectionCommon = $this->getFromCommonConfig($name);
if(empty($section) && !empty($sectionCommon)) {
$section = $sectionCommon;
} elseif(!empty($section) && !empty($sectionCommon)) {
$section = $this->array_merge_recursive_distinct($section, $sectionCommon);
}
if (isset($this->configLocal[$name])) {
// local settings override the global defaults
$section = $section
? array_merge($section, $this->configLocal[$name])
: $this->configLocal[$name];
}
if ($section === null && $name = 'superuser') {
$user = $this->getConfigSuperUserForBackwardCompatibility();
return $user;
} else if ($section === null) {
throw new Exception("Error while trying to read a specific config file entry <strong>'$name'</strong> from your configuration files.</b>If you just completed a Piwik upgrade, please check that the file config/global.ini.php was overwritten by the latest Piwik version.");
}
// cache merged section for later
$this->configCache[$name] = $this->decodeValues($section);
$tmp =& $this->configCache[$name];
return $tmp;
}
/**
* @deprecated since version 2.0.4
*/
public function getConfigSuperUserForBackwardCompatibility()
{
try {
$db = Db::get();
$user = $db->fetchRow("SELECT login, email, password
FROM " . Common::prefixTable("user") . "
WHERE superuser_access = 1
ORDER BY date_registered ASC LIMIT 1");
if (!empty($user)) {
$user['bridge'] = 1;
return $user;
}
} catch (Exception $e) {}
return array();
}
public function getFromGlobalConfig($name)
{
if (isset($this->configGlobal[$name])) {
return $this->configGlobal[$name];
}
return null;
}
public function getFromCommonConfig($name)
{
if (isset($this->configCommon[$name])) {
return $this->configCommon[$name];
}
return null;
}
/**
* Sets a configuration value or section.
*
* @param string $name This section name or value name to set.
* @param mixed $value
* @api
*/
public function __set($name, $value)
{
$this->configCache[$name] = $value;
}
/**
* Comparison function
*
* @param mixed $elem1
* @param mixed $elem2
* @return int;
*/
public static function compareElements($elem1, $elem2)
{
if (is_array($elem1)) {
if (is_array($elem2)) {
return strcmp(serialize($elem1), serialize($elem2));
}
return 1;
}
if (is_array($elem2)) {
return -1;
}
if ((string)$elem1 === (string)$elem2) {
return 0;
}
return ((string)$elem1 > (string)$elem2) ? 1 : -1;
}
/**
* Compare arrays and return difference, such that:
*
* $modified = array_merge($original, $difference);
*
* @param array $original original array
* @param array $modified modified array
* @return array differences between original and modified
*/
public function array_unmerge($original, $modified)
{
// return key/value pairs for keys in $modified but not in $original
// return key/value pairs for keys in both $modified and $original, but values differ
// ignore keys that are in $original but not in $modified
return array_udiff_assoc($modified, $original, array(__CLASS__, 'compareElements'));
}
/**
* Dump config
*
* @param array $configLocal
* @param array $configGlobal
* @param array $configCommon
* @param array $configCache
* @return string
*/
public function dumpConfig($configLocal, $configGlobal, $configCommon, $configCache)
{
$dirty = false;
$output = "; <?php exit; ?> DO NOT REMOVE THIS LINE\n";
$output .= "; file automatically generated or modified by Piwik; you can manually override the default values in global.ini.php by redefining them in this file.\n";
if (!$configCache) {
return false;
}
// If there is a common.config.ini.php, this will ensure config.ini.php does not duplicate its values
if(!empty($configCommon)) {
$configGlobal = $this->array_merge_recursive_distinct($configGlobal, $configCommon);
}
if ($configLocal) {
foreach ($configLocal as $name => $section) {
if (!isset($configCache[$name])) {
$configCache[$name] = $this->decodeValues($section);
}
}
}
$sectionNames = array_unique(array_merge(array_keys($configGlobal), array_keys($configCache)));
foreach ($sectionNames as $section) {
if (!isset($configCache[$section])) {
continue;
}
// Only merge if the section exists in global.ini.php (in case a section only lives in config.ini.php)
// get local and cached config
$local = isset($configLocal[$section]) ? $configLocal[$section] : array();
$config = $configCache[$section];
// remove default values from both (they should not get written to local)
if (isset($configGlobal[$section])) {
$config = $this->array_unmerge($configGlobal[$section], $configCache[$section]);
$local = $this->array_unmerge($configGlobal[$section], $local);
}
// if either local/config have non-default values and the other doesn't,
// OR both have values, but different values, we must write to config.ini.php
if (empty($local) xor empty($config)
|| (!empty($local)
&& !empty($config)
&& self::compareElements($config, $configLocal[$section]))
) {
$dirty = true;
}
// no point in writing empty sections, so skip if the cached section is empty
if (empty($config)) {
continue;
}
$output .= "[$section]\n";
foreach ($config as $name => $value) {
$value = $this->encodeValues($value);
if (is_numeric($name)) {
$name = $section;
$value = array($value);
}
if (is_array($value)) {
foreach ($value as $currentValue) {
$output .= $name . "[] = \"$currentValue\"\n";
}
} else {
if (!is_numeric($value)) {
$value = "\"$value\"";
}
$output .= $name . ' = ' . $value . "\n";
}
}
$output .= "\n";
}
if ($dirty) {
return $output;
}
return false;
}
/**
* Write user configuration file
*
* @param array $configLocal
* @param array $configGlobal
* @param array $configCommon
* @param array $configCache
* @param string $pathLocal
* @param bool $clear
*
* @throws \Exception if config file not writable
*/
protected function writeConfig($configLocal, $configGlobal, $configCommon, $configCache, $pathLocal, $clear = true)
{
if ($this->isTest) {
return;
}
$output = $this->dumpConfig($configLocal, $configGlobal, $configCommon, $configCache);
if ($output !== false) {
$success = @file_put_contents($pathLocal, $output);
if (!$success) {
throw $this->getConfigNotWritableException();
}
}
if ($clear) {
$this->clear();
}
}
/**
* Writes the current configuration to the **config.ini.php** file. Only writes options whose
* values are different from the default.
*
* @api
*/
public function forceSave()
{
$this->writeConfig($this->configLocal, $this->configGlobal, $this->configCommon, $this->configCache, $this->pathLocal);
}
/**
* @throws \Exception
*/
public function getConfigNotWritableException()
{
$path = "config/" . basename($this->pathLocal);
return new Exception(Piwik::translate('General_ConfigFileIsNotWritable', array("(" . $path . ")", "")));
}
/**
* array_merge_recursive does indeed merge arrays, but it converts values with duplicate
* keys to arrays rather than overwriting the value in the first array with the duplicate
* value in the second array, as array_merge does. I.e., with array_merge_recursive,
* this happens (documented behavior):
*
* array_merge_recursive(array('key' => 'org value'), array('key' => 'new value'));
* => array('key' => array('org value', 'new value'));
*
* array_merge_recursive_distinct does not change the datatypes of the values in the arrays.
* Matching keys' values in the second array overwrite those in the first array, as is the
* case with array_merge, i.e.:
*
* array_merge_recursive_distinct(array('key' => 'org value'), array('key' => 'new value'));
* => array('key' => array('new value'));
*
* Parameters are passed by reference, though only for performance reasons. They're not
* altered by this function.
*
* @param array $array1
* @param array $array2
* @return array
* @author Daniel <daniel (at) danielsmedegaardbuus (dot) dk>
* @author Gabriel Sobrinho <gabriel (dot) sobrinho (at) gmail (dot) com>
*/
function array_merge_recursive_distinct ( array &$array1, array &$array2 )
{
$merged = $array1;
foreach ( $array2 as $key => &$value ) {
if ( is_array ( $value ) && isset ( $merged [$key] ) && is_array ( $merged [$key] ) ) {
$merged [$key] = $this->array_merge_recursive_distinct ( $merged [$key], $value );
} else {
$merged [$key] = $value;
}
}
return $merged;
}
}

166
www/analytics/core/Console.php Normal file
View file

@ -0,0 +1,166 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik;
use Symfony\Component\Console\Application;
use Symfony\Component\Console\Input\ArgvInput;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
use Piwik\Plugin\Manager as PluginManager;
class Console extends Application
{
public function __construct()
{
parent::__construct();
$option = new InputOption('piwik-domain',
null,
InputOption::VALUE_OPTIONAL,
'Piwik URL (protocol and domain) eg. "http://piwik.example.org"'
);
$this->getDefinition()->addOption($option);
}
/**
* @deprecated
*/
public function init()
{
// TODO: remove
}
public function doRun(InputInterface $input, OutputInterface $output)
{
$this->initPiwikHost($input);
$this->initConfig($output);
try {
self::initPlugins();
} catch(\Exception $e) {
// Piwik not installed yet, no config file?
}
Translate::reloadLanguage('en');
$commands = $this->getAvailableCommands();
foreach ($commands as $command) {
if (!class_exists($command)) {
Log::warning(sprintf('Cannot add command %s, class does not exist', $command));
} elseif (!is_subclass_of($command, 'Piwik\Plugin\ConsoleCommand')) {
Log::warning(sprintf('Cannot add command %s, class does not extend Piwik\Plugin\ConsoleCommand', $command));
} else {
$this->add(new $command);
}
}
return parent::doRun($input, $output);
}
/**
* Returns a list of available command classnames.
*
* @return string[]
*/
private function getAvailableCommands()
{
$commands = $this->getDefaultPiwikCommands();
$pluginNames = PluginManager::getInstance()->getLoadedPluginsName();
foreach ($pluginNames as $pluginName) {
$commands = array_merge($commands, $this->findCommandsInPlugin($pluginName));
}
/**
* Triggered to filter / restrict console commands. Plugins that want to restrict commands
* should subscribe to this event and remove commands from the existing list.
*
* **Example**
*
* public function filterConsoleCommands(&$commands)
* {
* $key = array_search('Piwik\Plugins\MyPlugin\Commands\MyCommand', $commands);
* if (false !== $key) {
* unset($commands[$key]);
* }
* }
*
* @param array &$commands An array containing a list of command class names.
*/
Piwik::postEvent('Console.filterCommands', array(&$commands));
$commands = array_values(array_unique($commands));
return $commands;
}
private function findCommandsInPlugin($pluginName)
{
$commands = array();
$files = Filesystem::globr(PIWIK_INCLUDE_PATH . '/plugins/' . $pluginName .'/Commands', '*.php');
foreach ($files as $file) {
$klassName = sprintf('Piwik\\Plugins\\%s\\Commands\\%s', $pluginName, basename($file, '.php'));
if (!class_exists($klassName) || !is_subclass_of($klassName, 'Piwik\\Plugin\\ConsoleCommand')) {
continue;
}
$klass = new \ReflectionClass($klassName);
if ($klass->isAbstract()) {
continue;
}
$commands[] = $klassName;
}
return $commands;
}
protected function initPiwikHost(InputInterface $input)
{
$piwikHostname = $input->getParameterOption('--piwik-domain');
$piwikHostname = UrlHelper::getHostFromUrl($piwikHostname);
Url::setHost($piwikHostname);
}
protected function initConfig(OutputInterface $output)
{
$config = Config::getInstance();
try {
$config->checkLocalConfigFound();
return $config;
} catch (\Exception $e) {
$output->writeln($e->getMessage() . "\n");
}
}
public static function initPlugins()
{
Plugin\Manager::getInstance()->loadActivatedPlugins();
}
private function getDefaultPiwikCommands()
{
$commands = array(
'Piwik\CliMulti\RequestCommand'
);
if (class_exists('Piwik\Plugins\EnterpriseAdmin\EnterpriseAdmin')) {
$extra = new \Piwik\Plugins\EnterpriseAdmin\EnterpriseAdmin();
$extra->addConsoleCommands($commands);
}
return $commands;
}
}

383
www/analytics/core/Cookie.php Normal file
View file

@ -0,0 +1,383 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik;
/**
* Simple class to handle the cookies:
* - read a cookie values
* - edit an existing cookie and save it
* - create a new cookie, set values, expiration date, etc. and save it
*
*/
class Cookie
{
/**
* Don't create a cookie bigger than 1k
*/
const MAX_COOKIE_SIZE = 1024;
/**
* The name of the cookie
* @var string
*/
protected $name = null;
/**
* The expire time for the cookie (expressed in UNIX Timestamp)
* @var int
*/
protected $expire = null;
/**
* Restrict cookie path
* @var string
*/
protected $path = '';
/**
* Restrict cookie to a domain (or subdomains)
* @var string
*/
protected $domain = '';
/**
* If true, cookie should only be transmitted over secure HTTPS
* @var bool
*/
protected $secure = false;
/**
* If true, cookie will only be made available via the HTTP protocol.
* Note: not well supported by browsers.
* @var bool
*/
protected $httponly = false;
/**
* The content of the cookie
* @var array
*/
protected $value = array();
/**
* The character used to separate the tuple name=value in the cookie
*/
const VALUE_SEPARATOR = ':';
/**
* Instantiate a new Cookie object and tries to load the cookie content if the cookie
* exists already.
*
* @param string $cookieName cookie Name
* @param int $expire The timestamp after which the cookie will expire, eg time() + 86400;
* use 0 (int zero) to expire cookie at end of browser session
* @param string $path The path on the server in which the cookie will be available on.
* @param bool|string $keyStore Will be used to store several bits of data (eg. one array per website)
*/
public function __construct($cookieName, $expire = null, $path = null, $keyStore = false)
{
$this->name = $cookieName;
$this->path = $path;
$this->expire = $expire;
if (is_null($expire)
|| !is_numeric($expire)
|| $expire < 0
) {
$this->expire = $this->getDefaultExpire();
}
$this->keyStore = $keyStore;
if ($this->isCookieFound()) {
$this->loadContentFromCookie();
}
}
/**
* Returns true if the visitor already has the cookie.
*
* @return bool
*/
public function isCookieFound()
{
return isset($_COOKIE[$this->name]);
}
/**
* Returns the default expiry time, 2 years
*
* @return int Timestamp in 2 years
*/
protected function getDefaultExpire()
{
return time() + 86400 * 365 * 2;
}
/**
* setcookie() replacement -- we don't use the built-in function because
* it is buggy for some PHP versions.
*
* @link http://php.net/setcookie
*
* @param string $Name Name of cookie
* @param string $Value Value of cookie
* @param int $Expires Time the cookie expires
* @param string $Path
* @param string $Domain
* @param bool $Secure
* @param bool $HTTPOnly
*/
protected function setCookie($Name, $Value, $Expires, $Path = '', $Domain = '', $Secure = false, $HTTPOnly = false)
{
if (!empty($Domain)) {
// Fix the domain to accept domains with and without 'www.'.
if (!strncasecmp($Domain, 'www.', 4)) {
$Domain = substr($Domain, 4);
}
$Domain = '.' . $Domain;
// Remove port information.
$Port = strpos($Domain, ':');
if ($Port !== false) $Domain = substr($Domain, 0, $Port);
}
$header = 'Set-Cookie: ' . rawurlencode($Name) . '=' . rawurlencode($Value)
. (empty($Expires) ? '' : '; expires=' . gmdate('D, d-M-Y H:i:s', $Expires) . ' GMT')
. (empty($Path) ? '' : '; path=' . $Path)
. (empty($Domain) ? '' : '; domain=' . $Domain)
. (!$Secure ? '' : '; secure')
. (!$HTTPOnly ? '' : '; HttpOnly');
Common::sendHeader($header, false);
}
/**
* We set the privacy policy header
*/
protected function setP3PHeader()
{
Common::sendHeader("P3P: CP='OTI DSP COR NID STP UNI OTPa OUR'");
}
/**
* Delete the cookie
*/
public function delete()
{
$this->setP3PHeader();
$this->setCookie($this->name, 'deleted', time() - 31536001, $this->path, $this->domain);
}
/**
* Saves the cookie (set the Cookie header).
* You have to call this method before sending any text to the browser or you would get the
* "Header already sent" error.
*/
public function save()
{
$cookieString = $this->generateContentString();
if (strlen($cookieString) > self::MAX_COOKIE_SIZE) {
// If the cookie was going to be too large, instead, delete existing cookie and start afresh
$this->delete();
return;
}
$this->setP3PHeader();
$this->setCookie($this->name, $cookieString, $this->expire, $this->path, $this->domain, $this->secure, $this->httponly);
}
/**
* Extract signed content from string: content VALUE_SEPARATOR '_=' signature
*
* @param string $content
* @return string|bool Content or false if unsigned
*/
private function extractSignedContent($content)
{
$signature = substr($content, -40);
if (substr($content, -43, 3) == self::VALUE_SEPARATOR . '_=' &&
$signature == sha1(substr($content, 0, -40) . SettingsPiwik::getSalt())
) {
// strip trailing: VALUE_SEPARATOR '_=' signature"
return substr($content, 0, -43);
}
return false;
}
/**
* Load the cookie content into a php array.
* Parses the cookie string to extract the different variables.
* Unserialize the array when necessary.
* Decode the non numeric values that were base64 encoded.
*/
protected function loadContentFromCookie()
{
$cookieStr = $this->extractSignedContent($_COOKIE[$this->name]);
if ($cookieStr === false) {
return;
}
$values = explode(self::VALUE_SEPARATOR, $cookieStr);
foreach ($values as $nameValue) {
$equalPos = strpos($nameValue, '=');
$varName = substr($nameValue, 0, $equalPos);
$varValue = substr($nameValue, $equalPos + 1);
// no numeric value are base64 encoded so we need to decode them
if (!is_numeric($varValue)) {
$tmpValue = base64_decode($varValue);
$varValue = safe_unserialize($tmpValue);
// discard entire cookie
// note: this assumes we never serialize a boolean
if ($varValue === false && $tmpValue !== 'b:0;') {
$this->value = array();
unset($_COOKIE[$this->name]);
break;
}
}
$this->value[$varName] = $varValue;
}
}
/**
* Returns the string to save in the cookie from the $this->value array of values.
* It goes through the array and generates the cookie content string.
*
* @return string Cookie content
*/
protected function generateContentString()
{
$cookieStr = '';
foreach ($this->value as $name => $value) {
if (!is_numeric($value)) {
$value = base64_encode(safe_serialize($value));
}
$cookieStr .= "$name=$value" . self::VALUE_SEPARATOR;
}
if (!empty($cookieStr)) {
$cookieStr .= '_=';
// sign cookie
$signature = sha1($cookieStr . SettingsPiwik::getSalt());
return $cookieStr . $signature;
}
return '';
}
/**
* Set cookie domain
*
* @param string $domain
*/
public function setDomain($domain)
{
$this->domain = $domain;
}
/**
* Set secure flag
*
* @param bool $secure
*/
public function setSecure($secure)
{
$this->secure = $secure;
}
/**
* Set HTTP only
*
* @param bool $httponly
*/
public function setHttpOnly($httponly)
{
$this->httponly = $httponly;
}
/**
* Registers a new name => value association in the cookie.
*
* Registering new values is optimal if the value is a numeric value.
* If the value is a string, it will be saved as a base64 encoded string.
* If the value is an array, it will be saved as a serialized and base64 encoded
* string which is not very good in terms of bytes usage.
* You should save arrays only when you are sure about their maximum data size.
* A cookie has to stay small and its size shouldn't increase over time!
*
* @param string $name Name of the value to save; the name will be used to retrieve this value
* @param string|array|number $value Value to save. If null, entry will be deleted from cookie.
*/
public function set($name, $value)
{
$name = self::escapeValue($name);
// Delete value if $value === null
if (is_null($value)) {
if ($this->keyStore === false) {
unset($this->value[$name]);
return;
}
unset($this->value[$this->keyStore][$name]);
return;
}
if ($this->keyStore === false) {
$this->value[$name] = $value;
return;
}
$this->value[$this->keyStore][$name] = $value;
}
/**
* Returns the value defined by $name from the cookie.
*
* @param string|integer Index name of the value to return
* @return mixed The value if found, false if the value is not found
*/
public function get($name)
{
$name = self::escapeValue($name);
if ($this->keyStore === false) {
return isset($this->value[$name])
? self::escapeValue($this->value[$name])
: false;
}
return isset($this->value[$this->keyStore][$name])
? self::escapeValue($this->value[$this->keyStore][$name])
: false;
}
/**
* Returns an easy to read cookie dump
*
* @return string The cookie dump
*/
public function __toString()
{
$str = 'COOKIE ' . $this->name . ', rows count: ' . count($this->value) . ', cookie size = ' . strlen($this->generateContentString()) . " bytes\n";
$str .= var_export($this->value, $return = true);
return $str;
}
/**
* Escape values from the cookie before sending them back to the client
* (when using the get() method).
*
* @param string $value Value to be escaped
* @return mixed The value once cleaned.
*/
protected static function escapeValue($value)
{
return Common::sanitizeInputValues($value);
}
}

1247
www/analytics/core/CronArchive.php Normal file
View file

File diff suppressed because it is too large Load diff

68
www/analytics/core/CronArchive/FixedSiteIds.php Normal file
View file

@ -0,0 +1,68 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\CronArchive;
use Piwik\CronArchive;
class FixedSiteIds
{
private $siteIds = array();
private $index = -1;
public function __construct($websiteIds)
{
if (!empty($websiteIds)) {
$this->siteIds = $websiteIds;
}
}
public function getInitialSiteIds()
{
return $this->siteIds;
}
/**
* Get the number of total websites that needs to be processed.
*
* @return int
*/
public function getNumSites()
{
return count($this->siteIds);
}
/**
* Get the number of already processed websites. All websites were processed by the current archiver.
*
* @return int
*/
public function getNumProcessedWebsites()
{
$numProcessed = $this->index + 1;
if ($numProcessed > $this->getNumSites()) {
return $this->getNumSites();
}
return $numProcessed;
}
public function getNextSiteId()
{
$this->index++;
if (!empty($this->siteIds[$this->index])) {
return $this->siteIds[$this->index];
}
return null;
}
}

176
www/analytics/core/CronArchive/SharedSiteIds.php Normal file
View file

@ -0,0 +1,176 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\CronArchive;
use Exception;
use Piwik\CliMulti\Process;
use Piwik\Option;
/**
* This class saves all to be processed siteIds in an Option named 'SharedSiteIdsToArchive' and processes all sites
* within that list. If a user starts multiple archiver those archiver will help to finish processing that list.
*/
class SharedSiteIds
{
private $siteIds = array();
private $currentSiteId;
private $done = false;
public function __construct($websiteIds)
{
if (empty($websiteIds)) {
$websiteIds = array();
}
$self = $this;
$this->siteIds = $this->runExclusive(function () use ($self, $websiteIds) {
// if there are already sites to be archived registered, prefer the list of existing archive, meaning help
// to finish this queue of sites instead of starting a new queue
$existingWebsiteIds = $self->getAllSiteIdsToArchive();
if (!empty($existingWebsiteIds)) {
return $existingWebsiteIds;
}
$self->setSiteIdsToArchive($websiteIds);
return $websiteIds;
});
}
public function getInitialSiteIds()
{
return $this->siteIds;
}
/**
* Get the number of total websites that needs to be processed.
*
* @return int
*/
public function getNumSites()
{
return count($this->siteIds);
}
/**
* Get the number of already processed websites (not necessarily all of those where processed by this archiver).
*
* @return int
*/
public function getNumProcessedWebsites()
{
if ($this->done) {
return $this->getNumSites();
}
if (empty($this->currentSiteId)) {
return 0;
}
$index = array_search($this->currentSiteId, $this->siteIds);
if (false === $index) {
return 0;
}
return $index + 1;
}
public function setSiteIdsToArchive($siteIds)
{
if (!empty($siteIds)) {
Option::set('SharedSiteIdsToArchive', implode(',', $siteIds));
} else {
Option::delete('SharedSiteIdsToArchive');
}
}
public function getAllSiteIdsToArchive()
{
Option::clearCachedOption('SharedSiteIdsToArchive');
$siteIdsToArchive = Option::get('SharedSiteIdsToArchive');
if (empty($siteIdsToArchive)) {
return array();
}
return explode(',', trim($siteIdsToArchive));
}
/**
* If there are multiple archiver running on the same node it makes sure only one of them performs an action and it
* will wait until another one has finished. Any closure you pass here should be very fast as other processes wait
* for this closure to finish otherwise. Currently only used for making multiple archivers at the same time work.
* If a closure takes more than 5 seconds we assume it is dead and simply continue.
*
* @param \Closure $closure
* @return mixed
* @throws \Exception
*/
private function runExclusive($closure)
{
$process = new Process('archive.sharedsiteids');
while ($process->isRunning() && $process->getSecondsSinceCreation() < 5) {
// wait max 5 seconds, such an operation should not take longer
usleep(25 * 1000);
}
$process->startProcess();
try {
$result = $closure();
} catch (Exception $e) {
$process->finishProcess();
throw $e;
}
$process->finishProcess();
return $result;
}
/**
* Get the next site id that needs to be processed or null if all site ids where processed.
*
* @return int|null
*/
public function getNextSiteId()
{
$self = $this;
$this->currentSiteId = $this->runExclusive(function () use ($self) {
$siteIds = $self->getAllSiteIdsToArchive();
if (empty($siteIds)) {
return null;
}
$nextSiteId = array_shift($siteIds);
$self->setSiteIdsToArchive($siteIds);
return $nextSiteId;
});
if (is_null($this->currentSiteId)) {
$this->done = true;
}
return $this->currentSiteId;
}
public static function isSupported()
{
return Process::isSupported();
}
}

360
www/analytics/core/DataAccess/ArchiveSelector.php Normal file
View file

@ -0,0 +1,360 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataAccess;
use Exception;
use Piwik\ArchiveProcessor\Rules;
use Piwik\ArchiveProcessor;
use Piwik\Common;
use Piwik\Date;
use Piwik\Db;
use Piwik\Log;
use Piwik\Period;
use Piwik\Period\Range;
use Piwik\Piwik;
use Piwik\Segment;
use Piwik\Site;
/**
* Data Access object used to query archives
*
* A record in the Database for a given report is defined by
* - idarchive = unique ID that is associated to all the data of this archive (idsite+period+date)
* - idsite = the ID of the website
* - date1 = starting day of the period
* - date2 = ending day of the period
* - period = integer that defines the period (day/week/etc.). @see period::getId()
* - ts_archived = timestamp when the archive was processed (UTC)
* - name = the name of the report (ex: uniq_visitors or search_keywords_by_search_engines)
* - value = the actual data (a numeric value, or a blob of compressed serialized data)
*
*/
class ArchiveSelector
{
const NB_VISITS_RECORD_LOOKED_UP = "nb_visits";
const NB_VISITS_CONVERTED_RECORD_LOOKED_UP = "nb_visits_converted";
static public function getArchiveIdAndVisits(ArchiveProcessor\Parameters $params, $minDatetimeArchiveProcessedUTC)
{
$dateStart = $params->getPeriod()->getDateStart();
$bindSQL = array($params->getSite()->getId(),
$dateStart->toString('Y-m-d'),
$params->getPeriod()->getDateEnd()->toString('Y-m-d'),
$params->getPeriod()->getId(),
);
$timeStampWhere = '';
if ($minDatetimeArchiveProcessedUTC) {
$timeStampWhere = " AND ts_archived >= ? ";
$bindSQL[] = Date::factory($minDatetimeArchiveProcessedUTC)->getDatetime();
}
$requestedPlugin = $params->getRequestedPlugin();
$segment = $params->getSegment();
$isSkipAggregationOfSubTables = $params->isSkipAggregationOfSubTables();
$plugins = array("VisitsSummary", $requestedPlugin);
$sqlWhereArchiveName = self::getNameCondition($plugins, $segment, $isSkipAggregationOfSubTables);
$sqlQuery = " SELECT idarchive, value, name, date1 as startDate
FROM " . ArchiveTableCreator::getNumericTable($dateStart) . "``
WHERE idsite = ?
AND date1 = ?
AND date2 = ?
AND period = ?
AND ( ($sqlWhereArchiveName)
OR name = '" . self::NB_VISITS_RECORD_LOOKED_UP . "'
OR name = '" . self::NB_VISITS_CONVERTED_RECORD_LOOKED_UP . "')
$timeStampWhere
ORDER BY idarchive DESC";
$results = Db::fetchAll($sqlQuery, $bindSQL);
if (empty($results)) {
return false;
}
$idArchive = self::getMostRecentIdArchiveFromResults($segment, $requestedPlugin, $isSkipAggregationOfSubTables, $results);
$idArchiveVisitsSummary = self::getMostRecentIdArchiveFromResults($segment, "VisitsSummary", $isSkipAggregationOfSubTables, $results);
list($visits, $visitsConverted) = self::getVisitsMetricsFromResults($idArchive, $idArchiveVisitsSummary, $results);
if ($visits === false
&& $idArchive === false
) {
return false;
}
return array($idArchive, $visits, $visitsConverted);
}
protected static function getVisitsMetricsFromResults($idArchive, $idArchiveVisitsSummary, $results)
{
$visits = $visitsConverted = false;
$archiveWithVisitsMetricsWasFound = ($idArchiveVisitsSummary !== false);
if ($archiveWithVisitsMetricsWasFound) {
$visits = $visitsConverted = 0;
}
foreach ($results as $result) {
if (in_array($result['idarchive'], array($idArchive, $idArchiveVisitsSummary))) {
$value = (int)$result['value'];
if (empty($visits)
&& $result['name'] == self::NB_VISITS_RECORD_LOOKED_UP
) {
$visits = $value;
}
if (empty($visitsConverted)
&& $result['name'] == self::NB_VISITS_CONVERTED_RECORD_LOOKED_UP
) {
$visitsConverted = $value;
}
}
}
return array($visits, $visitsConverted);
}
protected static function getMostRecentIdArchiveFromResults(Segment $segment, $requestedPlugin, $isSkipAggregationOfSubTables, $results)
{
$idArchive = false;
$namesRequestedPlugin = Rules::getDoneFlags(array($requestedPlugin), $segment, $isSkipAggregationOfSubTables);
foreach ($results as $result) {
if ($idArchive === false
&& in_array($result['name'], $namesRequestedPlugin)
) {
$idArchive = $result['idarchive'];
break;
}
}
return $idArchive;
}
/**
* Queries and returns archive IDs for a set of sites, periods, and a segment.
*
* @param array $siteIds
* @param array $periods
* @param Segment $segment
* @param array $plugins List of plugin names for which data is being requested.
* @param bool $isSkipAggregationOfSubTables Whether we are selecting an archive that may be partial (no sub-tables)
* @return array Archive IDs are grouped by archive name and period range, ie,
* array(
* 'VisitsSummary.done' => array(
* '2010-01-01' => array(1,2,3)
* )
* )
*/
static public function getArchiveIds($siteIds, $periods, $segment, $plugins, $isSkipAggregationOfSubTables = false)
{
$getArchiveIdsSql = "SELECT idsite, name, date1, date2, MAX(idarchive) as idarchive
FROM %s
WHERE %s
AND " . self::getNameCondition($plugins, $segment, $isSkipAggregationOfSubTables) . "
AND idsite IN (" . implode(',', $siteIds) . ")
GROUP BY idsite, date1, date2";
$monthToPeriods = array();
foreach ($periods as $period) {
/** @var Period $period */
$table = ArchiveTableCreator::getNumericTable($period->getDateStart());
$monthToPeriods[$table][] = $period;
}
// for every month within the archive query, select from numeric table
$result = array();
foreach ($monthToPeriods as $table => $periods) {
$firstPeriod = reset($periods);
$bind = array();
if ($firstPeriod instanceof Range) {
$dateCondition = "period = ? AND date1 = ? AND date2 = ?";
$bind[] = $firstPeriod->getId();
$bind[] = $firstPeriod->getDateStart()->toString('Y-m-d');
$bind[] = $firstPeriod->getDateEnd()->toString('Y-m-d');
} else {
// we assume there is no range date in $periods
$dateCondition = '(';
foreach ($periods as $period) {
if (strlen($dateCondition) > 1) {
$dateCondition .= ' OR ';
}
$dateCondition .= "(period = ? AND date1 = ? AND date2 = ?)";
$bind[] = $period->getId();
$bind[] = $period->getDateStart()->toString('Y-m-d');
$bind[] = $period->getDateEnd()->toString('Y-m-d');
}
$dateCondition .= ')';
}
$sql = sprintf($getArchiveIdsSql, $table, $dateCondition);
// get the archive IDs
foreach (Db::fetchAll($sql, $bind) as $row) {
$archiveName = $row['name'];
//FIXMEA duplicate with Archive.php
$dateStr = $row['date1'] . "," . $row['date2'];
$result[$archiveName][$dateStr][] = $row['idarchive'];
}
}
return $result;
}
/**
* Queries and returns archive data using a set of archive IDs.
*
* @param array $archiveIds The IDs of the archives to get data from.
* @param array $recordNames The names of the data to retrieve (ie, nb_visits, nb_actions, etc.)
* @param string $archiveDataType The archive data type (either, 'blob' or 'numeric').
* @param bool $loadAllSubtables Whether to pre-load all subtables
* @throws Exception
* @return array
*/
static public function getArchiveData($archiveIds, $recordNames, $archiveDataType, $loadAllSubtables)
{
// create the SQL to select archive data
$inNames = Common::getSqlStringFieldsArray($recordNames);
if ($loadAllSubtables) {
$name = reset($recordNames);
// select blobs w/ name like "$name_[0-9]+" w/o using RLIKE
$nameEnd = strlen($name) + 2;
$whereNameIs = "(name = ?
OR (name LIKE ?
AND SUBSTRING(name, $nameEnd, 1) >= '0'
AND SUBSTRING(name, $nameEnd, 1) <= '9') )";
$bind = array($name, $name . '%');
} else {
$whereNameIs = "name IN ($inNames)";
$bind = array_values($recordNames);
}
$getValuesSql = "SELECT value, name, idsite, date1, date2, ts_archived
FROM %s
WHERE idarchive IN (%s)
AND " . $whereNameIs;
// get data from every table we're querying
$rows = array();
foreach ($archiveIds as $period => $ids) {
if (empty($ids)) {
throw new Exception("Unexpected: id archive not found for period '$period' '");
}
// $period = "2009-01-04,2009-01-04",
$date = Date::factory(substr($period, 0, 10));
if ($archiveDataType == 'numeric') {
$table = ArchiveTableCreator::getNumericTable($date);
} else {
$table = ArchiveTableCreator::getBlobTable($date);
}
$sql = sprintf($getValuesSql, $table, implode(',', $ids));
$dataRows = Db::fetchAll($sql, $bind);
foreach ($dataRows as $row) {
$rows[] = $row;
}
}
return $rows;
}
/**
* Returns the SQL condition used to find successfully completed archives that
* this instance is querying for.
*
* @param array $plugins
* @param Segment $segment
* @param bool $isSkipAggregationOfSubTables
* @return string
*/
static private function getNameCondition(array $plugins, Segment $segment, $isSkipAggregationOfSubTables)
{
// the flags used to tell how the archiving process for a specific archive was completed,
// if it was completed
$doneFlags = Rules::getDoneFlags($plugins, $segment, $isSkipAggregationOfSubTables);
$allDoneFlags = "'" . implode("','", $doneFlags) . "'";
// create the SQL to find archives that are DONE
return "((name IN ($allDoneFlags)) AND " .
" (value = '" . ArchiveWriter::DONE_OK . "' OR " .
" value = '" . ArchiveWriter::DONE_OK_TEMPORARY . "'))";
}
static public function purgeOutdatedArchives(Date $dateStart)
{
$purgeArchivesOlderThan = Rules::shouldPurgeOutdatedArchives($dateStart);
if (!$purgeArchivesOlderThan) {
return;
}
$idArchivesToDelete = self::getTemporaryArchiveIdsOlderThan($dateStart, $purgeArchivesOlderThan);
if (!empty($idArchivesToDelete)) {
self::deleteArchiveIds($dateStart, $idArchivesToDelete);
}
self::deleteArchivesWithPeriodRange($dateStart);
Log::debug("Purging temporary archives: done [ purged archives older than %s in %s ] [Deleted IDs: %s]",
$purgeArchivesOlderThan, $dateStart->toString("Y-m"), implode(',', $idArchivesToDelete));
}
/*
* Deleting "Custom Date Range" reports after 1 day, since they can be re-processed and would take up un-necessary space
*/
protected static function deleteArchivesWithPeriodRange(Date $date)
{
$query = "DELETE FROM %s WHERE period = ? AND ts_archived < ?";
$yesterday = Date::factory('yesterday')->getDateTime();
$bind = array(Piwik::$idPeriods['range'], $yesterday);
$numericTable = ArchiveTableCreator::getNumericTable($date);
Db::query(sprintf($query, $numericTable), $bind);
Log::debug("Purging Custom Range archives: done [ purged archives older than %s from %s / blob ]", $yesterday, $numericTable);
try {
Db::query(sprintf($query, ArchiveTableCreator::getBlobTable($date)), $bind);
} catch (Exception $e) {
// Individual blob tables could be missing
}
}
protected static function deleteArchiveIds(Date $date, $idArchivesToDelete)
{
$query = "DELETE FROM %s WHERE idarchive IN (" . implode(',', $idArchivesToDelete) . ")";
Db::query(sprintf($query, ArchiveTableCreator::getNumericTable($date)));
try {
Db::query(sprintf($query, ArchiveTableCreator::getBlobTable($date)));
} catch (Exception $e) {
// Individual blob tables could be missing
}
}
protected static function getTemporaryArchiveIdsOlderThan(Date $date, $purgeArchivesOlderThan)
{
$query = "SELECT idarchive
FROM " . ArchiveTableCreator::getNumericTable($date) . "
WHERE name LIKE 'done%'
AND (( value = " . ArchiveWriter::DONE_OK_TEMPORARY . "
AND ts_archived < ?)
OR value = " . ArchiveWriter::DONE_ERROR . ")";
$result = Db::fetchAll($query, array($purgeArchivesOlderThan));
$idArchivesToDelete = array();
if (!empty($result)) {
foreach ($result as $row) {
$idArchivesToDelete[] = $row['idarchive'];
}
}
return $idArchivesToDelete;
}
}

119
www/analytics/core/DataAccess/ArchiveTableCreator.php Normal file
View file

@ -0,0 +1,119 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataAccess;
use Exception;
use Piwik\Common;
use Piwik\Date;
use Piwik\Db;
use Piwik\DbHelper;
class ArchiveTableCreator
{
const NUMERIC_TABLE = "numeric";
const BLOB_TABLE = "blob";
static public $tablesAlreadyInstalled = null;
static public function getNumericTable(Date $date)
{
return self::getTable($date, self::NUMERIC_TABLE);
}
static public function getBlobTable(Date $date)
{
return self::getTable($date, self::BLOB_TABLE);
}
static protected function getTable(Date $date, $type)
{
$tableNamePrefix = "archive_" . $type;
$tableName = $tableNamePrefix . "_" . $date->toString('Y_m');
$tableName = Common::prefixTable($tableName);
self::createArchiveTablesIfAbsent($tableName, $tableNamePrefix);
return $tableName;
}
static protected function createArchiveTablesIfAbsent($tableName, $tableNamePrefix)
{
if (is_null(self::$tablesAlreadyInstalled)) {
self::refreshTableList();
}
if (!in_array($tableName, self::$tablesAlreadyInstalled)) {
$db = Db::get();
$sql = DbHelper::getTableCreateSql($tableNamePrefix);
// replace table name template by real name
$tableNamePrefix = Common::prefixTable($tableNamePrefix);
$sql = str_replace($tableNamePrefix, $tableName, $sql);
try {
$db->query($sql);
} catch (Exception $e) {
// accept mysql error 1050: table already exists, throw otherwise
if (!$db->isErrNo($e, '1050')) {
throw $e;
}
}
self::$tablesAlreadyInstalled[] = $tableName;
}
}
static public function clear()
{
self::$tablesAlreadyInstalled = null;
}
static public function refreshTableList($forceReload = false)
{
self::$tablesAlreadyInstalled = DbHelper::getTablesInstalled($forceReload);
}
/**
* Returns all table names archive_*
*
* @return array
*/
static public function getTablesArchivesInstalled()
{
if (is_null(self::$tablesAlreadyInstalled)) {
self::refreshTableList();
}
$archiveTables = array();
foreach (self::$tablesAlreadyInstalled as $table) {
if (strpos($table, 'archive_numeric_') !== false
|| strpos($table, 'archive_blob_') !== false
) {
$archiveTables[] = $table;
}
}
return $archiveTables;
}
static public function getDateFromTableName($tableName)
{
$tableName = Common::unprefixTable($tableName);
$date = str_replace(array('archive_numeric_', 'archive_blob_'), '', $tableName);
return $date;
}
static public function getTypeFromTableName($tableName)
{
if (strpos($tableName, 'archive_numeric_') !== false) {
return self::NUMERIC_TABLE;
}
if (strpos($tableName, 'archive_blob_') !== false) {
return self::BLOB_TABLE;
}
return false;
}
}

317
www/analytics/core/DataAccess/ArchiveWriter.php Normal file
View file

@ -0,0 +1,317 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataAccess;
use Exception;
use Piwik\ArchiveProcessor\Rules;
use Piwik\ArchiveProcessor;
use Piwik\Common;
use Piwik\Config;
use Piwik\Db;
use Piwik\Db\BatchInsert;
use Piwik\Log;
use Piwik\Period;
use Piwik\Segment;
use Piwik\SettingsPiwik;
/**
* This class is used to create a new Archive.
* An Archive is a set of reports (numeric and data tables).
* New data can be inserted in the archive with insertRecord/insertBulkRecords
*/
class ArchiveWriter
{
const PREFIX_SQL_LOCK = "locked_";
/**
* Flag stored at the end of the archiving
*
* @var int
*/
const DONE_OK = 1;
/**
* Flag stored at the start of the archiving
* When requesting an Archive, we make sure that non-finished archive are not considered valid
*
* @var int
*/
const DONE_ERROR = 2;
/**
* Flag indicates the archive is over a period that is not finished, eg. the current day, current week, etc.
* Archives flagged will be regularly purged from the DB.
*
* @var int
*/
const DONE_OK_TEMPORARY = 3;
protected $fields = array('idarchive',
'idsite',
'date1',
'date2',
'period',
'ts_archived',
'name',
'value');
public function __construct(ArchiveProcessor\Parameters $params, $isArchiveTemporary)
{
$this->idArchive = false;
$this->idSite = $params->getSite()->getId();
$this->segment = $params->getSegment();
$this->period = $params->getPeriod();
$idSites = array($this->idSite);
$this->doneFlag = Rules::getDoneStringFlagFor($idSites, $this->segment, $this->period->getLabel(), $params->getRequestedPlugin(), $params->isSkipAggregationOfSubTables());
$this->isArchiveTemporary = $isArchiveTemporary;
$this->dateStart = $this->period->getDateStart();
}
/**
* @param string $name
* @param string[] $values
*/
public function insertBlobRecord($name, $values)
{
if (is_array($values)) {
$clean = array();
foreach ($values as $id => $value) {
// for the parent Table we keep the name
// for example for the Table of searchEngines we keep the name 'referrer_search_engine'
// but for the child table of 'Google' which has the ID = 9 the name would be 'referrer_search_engine_9'
$newName = $name;
if ($id != 0) {
//FIXMEA: refactor
$newName = $name . '_' . $id;
}
$value = $this->compress($value);
$clean[] = array($newName, $value);
}
$this->insertBulkRecords($clean);
return;
}
$values = $this->compress($values);
$this->insertRecord($name, $values);
}
public function getIdArchive()
{
if ($this->idArchive === false) {
throw new Exception("Must call allocateNewArchiveId() first");
}
return $this->idArchive;
}
public function initNewArchive()
{
$this->allocateNewArchiveId();
$this->logArchiveStatusAsIncomplete();
}
public function finalizeArchive()
{
$this->deletePreviousArchiveStatus();
$this->logArchiveStatusAsFinal();
}
static protected function compress($data)
{
if (Db::get()->hasBlobDataType()) {
return gzcompress($data);
}
return $data;
}
protected function getArchiveLockName()
{
$numericTable = $this->getTableNumeric();
$dbLockName = "allocateNewArchiveId.$numericTable";
return $dbLockName;
}
protected function acquireArchiveTableLock()
{
$dbLockName = $this->getArchiveLockName();
if (Db::getDbLock($dbLockName, $maxRetries = 30) === false) {
throw new Exception("allocateNewArchiveId: Cannot get named lock $dbLockName.");
}
}
protected function releaseArchiveTableLock()
{
$dbLockName = $this->getArchiveLockName();
Db::releaseDbLock($dbLockName);
}
protected function allocateNewArchiveId()
{
$this->idArchive = $this->insertNewArchiveId();
return $this->idArchive;
}
/**
* Locks the archive table to generate a new archive ID.
*
* We lock to make sure that
* if several archiving processes are running at the same time (for different websites and/or periods)
* then they will each use a unique archive ID.
*
* @return int
*/
protected function insertNewArchiveId()
{
$numericTable = $this->getTableNumeric();
$idSite = $this->idSite;
$this->acquireArchiveTableLock();
$locked = self::PREFIX_SQL_LOCK . Common::generateUniqId();
$date = date("Y-m-d H:i:s");
$insertSql = "INSERT INTO $numericTable "
. " SELECT IFNULL( MAX(idarchive), 0 ) + 1,
'" . $locked . "',
" . (int)$idSite . ",
'" . $date . "',
'" . $date . "',
0,
'" . $date . "',
0 "
. " FROM $numericTable as tb1";
Db::get()->exec($insertSql);
$this->releaseArchiveTableLock();
$selectIdSql = "SELECT idarchive FROM $numericTable WHERE name = ? LIMIT 1";
$id = Db::get()->fetchOne($selectIdSql, $locked);
return $id;
}
protected function logArchiveStatusAsIncomplete()
{
$statusWhileProcessing = self::DONE_ERROR;
$this->insertRecord($this->doneFlag, $statusWhileProcessing);
}
protected function deletePreviousArchiveStatus()
{
// without advisory lock here, the DELETE would acquire Exclusive Lock
$this->acquireArchiveTableLock();
Db::query("DELETE FROM " . $this->getTableNumeric() . "
WHERE idarchive = ? AND (name = '" . $this->doneFlag
. "' OR name LIKE '" . self::PREFIX_SQL_LOCK . "%')",
array($this->getIdArchive())
);
$this->releaseArchiveTableLock();
}
protected function logArchiveStatusAsFinal()
{
$status = self::DONE_OK;
if ($this->isArchiveTemporary) {
$status = self::DONE_OK_TEMPORARY;
}
$this->insertRecord($this->doneFlag, $status);
}
protected function insertBulkRecords($records)
{
// Using standard plain INSERT if there is only one record to insert
if ($DEBUG_DO_NOT_USE_BULK_INSERT = false
|| count($records) == 1
) {
foreach ($records as $record) {
$this->insertRecord($record[0], $record[1]);
}
return true;
}
$bindSql = $this->getInsertRecordBind();
$values = array();
$valueSeen = false;
foreach ($records as $record) {
// don't record zero
if (empty($record[1])) continue;
$bind = $bindSql;
$bind[] = $record[0]; // name
$bind[] = $record[1]; // value
$values[] = $bind;
$valueSeen = $record[1];
}
if (empty($values)) return true;
$tableName = $this->getTableNameToInsert($valueSeen);
BatchInsert::tableInsertBatch($tableName, $this->getInsertFields(), $values);
return true;
}
/**
* Inserts a record in the right table (either NUMERIC or BLOB)
*
* @param string $name
* @param mixed $value
*
* @return bool
*/
public function insertRecord($name, $value)
{
if ($this->isRecordZero($value)) {
return false;
}
$tableName = $this->getTableNameToInsert($value);
// duplicate idarchives are Ignored, see http://dev.piwik.org/trac/ticket/987
$query = "INSERT IGNORE INTO " . $tableName . "
(" . implode(", ", $this->getInsertFields()) . ")
VALUES (?,?,?,?,?,?,?,?)";
$bindSql = $this->getInsertRecordBind();
$bindSql[] = $name;
$bindSql[] = $value;
Db::query($query, $bindSql);
return true;
}
protected function getInsertRecordBind()
{
return array($this->getIdArchive(),
$this->idSite,
$this->dateStart->toString('Y-m-d'),
$this->period->getDateEnd()->toString('Y-m-d'),
$this->period->getId(),
date("Y-m-d H:i:s"));
}
protected function getTableNameToInsert($value)
{
if (is_numeric($value)) {
return $this->getTableNumeric();
}
return ArchiveTableCreator::getBlobTable($this->dateStart);
}
protected function getTableNumeric()
{
return ArchiveTableCreator::getNumericTable($this->dateStart);
}
protected function getInsertFields()
{
return $this->fields;
}
protected function isRecordZero($value)
{
return ($value === '0' || $value === false || $value === 0 || $value === 0.0);
}
}

880
www/analytics/core/DataAccess/LogAggregator.php Normal file
View file

@ -0,0 +1,880 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataAccess;
use PDOStatement;
use Piwik\ArchiveProcessor\Parameters;
use Piwik\Common;
use Piwik\DataArray;
use Piwik\Db;
use Piwik\Metrics;
use Piwik\Tracker\GoalManager;
/**
* Contains methods that calculate metrics by aggregating log data (visits, actions, conversions,
* ecommerce items).
*
* You can use the methods in this class within {@link Piwik\Plugin\Archiver Archiver} descendants
* to aggregate log data without having to write SQL queries.
*
* ### Aggregation Dimension
*
* All aggregation methods accept a **dimension** parameter. These parameters are important as
* they control how rows in a table are aggregated together.
*
* A **_dimension_** is just a table column. Rows that have the same values for these columns are
* aggregated together. The result of these aggregations is a set of metrics for every recorded value
* of a **dimension**.
*
* _Note: A dimension is essentially the same as a **GROUP BY** field._
*
* ### Examples
*
* **Aggregating visit data**
*
* $archiveProcessor = // ...
* $logAggregator = $archiveProcessor->getLogAggregator();
*
* // get metrics for every used browser language of all visits by returning visitors
* $query = $logAggregator->queryVisitsByDimension(
* $dimensions = array('log_visit.location_browser_lang'),
* $where = 'log_visit.visitor_returning = 1',
*
* // also count visits for each browser language that are not located in the US
* $additionalSelects = array('sum(case when log_visit.location_country <> 'us' then 1 else 0 end) as nonus'),
*
* // we're only interested in visits, unique visitors & actions, so don't waste time calculating anything else
* $metrics = array(Metrics::INDEX_NB_UNIQ_VISITORS, Metrics::INDEX_NB_VISITS, Metrics::INDEX_NB_ACTIONS),
* );
* if ($query === false) {
* return;
* }
*
* while ($row = $query->fetch()) {
* $uniqueVisitors = $row[Metrics::INDEX_NB_UNIQ_VISITORS];
* $visits = $row[Metrics::INDEX_NB_VISITS];
* $actions = $row[Metrics::INDEX_NB_ACTIONS];
*
* // ... do something w/ calculated metrics ...
* }
*
* **Aggregating conversion data**
*
* $archiveProcessor = // ...
* $logAggregator = $archiveProcessor->getLogAggregator();
*
* // get metrics for ecommerce conversions for each country
* $query = $logAggregator->queryConversionsByDimension(
* $dimensions = array('log_conversion.location_country'),
* $where = 'log_conversion.idgoal = 0', // 0 is the special ecommerceOrder idGoal value in the table
*
* // also calculate average tax and max shipping per country
* $additionalSelects = array(
* 'AVG(log_conversion.revenue_tax) as avg_tax',
* 'MAX(log_conversion.revenue_shipping) as max_shipping'
* )
* );
* if ($query === false) {
* return;
* }
*
* while ($row = $query->fetch()) {
* $country = $row['location_country'];
* $numEcommerceSales = $row[Metrics::INDEX_GOAL_NB_CONVERSIONS];
* $numVisitsWithEcommerceSales = $row[Metrics::INDEX_GOAL_NB_VISITS_CONVERTED];
* $avgTaxForCountry = $country['avg_tax'];
* $maxShippingForCountry = $country['max_shipping'];
*
* // ... do something with aggregated data ...
* }
*/
class LogAggregator
{
const LOG_VISIT_TABLE = 'log_visit';
const LOG_ACTIONS_TABLE = 'log_link_visit_action';
const LOG_CONVERSION_TABLE = "log_conversion";
const REVENUE_SUBTOTAL_FIELD = 'revenue_subtotal';
const REVENUE_TAX_FIELD = 'revenue_tax';
const REVENUE_SHIPPING_FIELD = 'revenue_shipping';
const REVENUE_DISCOUNT_FIELD = 'revenue_discount';
const TOTAL_REVENUE_FIELD = 'revenue';
const ITEMS_COUNT_FIELD = "items";
const CONVERSION_DATETIME_FIELD = "server_time";
const ACTION_DATETIME_FIELD = "server_time";
const VISIT_DATETIME_FIELD = 'visit_last_action_time';
const IDGOAL_FIELD = 'idgoal';
const FIELDS_SEPARATOR = ", \n\t\t\t";
/** @var \Piwik\Date */
protected $dateStart;
/** @var \Piwik\Date */
protected $dateEnd;
/** @var \Piwik\Site */
protected $site;
/** @var \Piwik\Segment */
protected $segment;
/**
* Constructor.
*
* @param \Piwik\ArchiveProcessor\Parameters $params
*/
public function __construct(Parameters $params)
{
$this->dateStart = $params->getDateStart();
$this->dateEnd = $params->getDateEnd();
$this->segment = $params->getSegment();
$this->site = $params->getSite();
}
public function generateQuery($select, $from, $where, $groupBy, $orderBy)
{
$bind = $this->getBindDatetimeSite();
$query = $this->segment->getSelectQuery($select, $from, $where, $bind, $orderBy, $groupBy);
return $query;
}
protected function getVisitsMetricFields()
{
return array(
Metrics::INDEX_NB_UNIQ_VISITORS => "count(distinct " . self::LOG_VISIT_TABLE . ".idvisitor)",
Metrics::INDEX_NB_VISITS => "count(*)",
Metrics::INDEX_NB_ACTIONS => "sum(" . self::LOG_VISIT_TABLE . ".visit_total_actions)",
Metrics::INDEX_MAX_ACTIONS => "max(" . self::LOG_VISIT_TABLE . ".visit_total_actions)",
Metrics::INDEX_SUM_VISIT_LENGTH => "sum(" . self::LOG_VISIT_TABLE . ".visit_total_time)",
Metrics::INDEX_BOUNCE_COUNT => "sum(case " . self::LOG_VISIT_TABLE . ".visit_total_actions when 1 then 1 when 0 then 1 else 0 end)",
Metrics::INDEX_NB_VISITS_CONVERTED => "sum(case " . self::LOG_VISIT_TABLE . ".visit_goal_converted when 1 then 1 else 0 end)",
);
}
static public function getConversionsMetricFields()
{
return array(
Metrics::INDEX_GOAL_NB_CONVERSIONS => "count(*)",
Metrics::INDEX_GOAL_NB_VISITS_CONVERTED => "count(distinct " . self::LOG_CONVERSION_TABLE . ".idvisit)",
Metrics::INDEX_GOAL_REVENUE => self::getSqlConversionRevenueSum(self::TOTAL_REVENUE_FIELD),
Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_SUBTOTAL => self::getSqlConversionRevenueSum(self::REVENUE_SUBTOTAL_FIELD),
Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_TAX => self::getSqlConversionRevenueSum(self::REVENUE_TAX_FIELD),
Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_SHIPPING => self::getSqlConversionRevenueSum(self::REVENUE_SHIPPING_FIELD),
Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_DISCOUNT => self::getSqlConversionRevenueSum(self::REVENUE_DISCOUNT_FIELD),
Metrics::INDEX_GOAL_ECOMMERCE_ITEMS => "SUM(" . self::LOG_CONVERSION_TABLE . "." . self::ITEMS_COUNT_FIELD . ")",
);
}
static private function getSqlConversionRevenueSum($field)
{
return self::getSqlRevenue('SUM(' . self::LOG_CONVERSION_TABLE . '.' . $field . ')');
}
static public function getSqlRevenue($field)
{
return "ROUND(" . $field . "," . GoalManager::REVENUE_PRECISION . ")";
}
/**
* Helper function that returns an array with common metrics for a given log_visit field distinct values.
*
* The statistics returned are:
* - number of unique visitors
* - number of visits
* - number of actions
* - maximum number of action for a visit
* - sum of the visits' length in sec
* - count of bouncing visits (visits with one page view)
*
* For example if $dimension = 'config_os' it will return the statistics for every distinct Operating systems
* The returned array will have a row per distinct operating systems,
* and a column per stat (nb of visits, max actions, etc)
*
* 'label' Metrics::INDEX_NB_UNIQ_VISITORS Metrics::INDEX_NB_VISITS etc.
* Linux 27 66 ...
* Windows XP 12 ...
* Mac OS 15 36 ...
*
* @param string $dimension Table log_visit field name to be use to compute common stats
* @return DataArray
*/
public function getMetricsFromVisitByDimension($dimension)
{
if (!is_array($dimension)) {
$dimension = array($dimension);
}
if (count($dimension) == 1) {
$dimension = array("label" => reset($dimension));
}
$query = $this->queryVisitsByDimension($dimension);
$metrics = new DataArray();
while ($row = $query->fetch()) {
$metrics->sumMetricsVisits($row["label"], $row);
}
return $metrics;
}
/**
* Executes and returns a query aggregating visit logs, optionally grouping by some dimension. Returns
* a DB statement that can be used to iterate over the result
*
* **Result Set**
*
* The following columns are in each row of the result set:
*
* - **{@link Piwik\Metrics::INDEX_NB_UNIQ_VISITORS}**: The total number of unique visitors in this group
* of aggregated visits.
* - **{@link Piwik\Metrics::INDEX_NB_VISITS}**: The total number of visits aggregated.
* - **{@link Piwik\Metrics::INDEX_NB_ACTIONS}**: The total number of actions performed in this group of
* aggregated visits.
* - **{@link Piwik\Metrics::INDEX_MAX_ACTIONS}**: The maximum actions perfomred in one visit for this group of
* visits.
* - **{@link Piwik\Metrics::INDEX_SUM_VISIT_LENGTH}**: The total amount of time spent on the site for this
* group of visits.
* - **{@link Piwik\Metrics::INDEX_BOUNCE_COUNT}**: The total number of bounced visits in this group of
* visits.
* - **{@link Piwik\Metrics::INDEX_NB_VISITS_CONVERTED}**: The total number of visits for which at least one
* conversion occurred, for this group of visits.
*
* Additional data can be selected by setting the `$additionalSelects` parameter.
*
* _Note: The metrics returned by this query can be customized by the `$metrics` parameter._
*
* @param array|string $dimensions `SELECT` fields (or just one field) that will be grouped by,
* eg, `'referrer_name'` or `array('referrer_name', 'referrer_keyword')`.
* The metrics retrieved from the query will be specific to combinations
* of these fields. So if `array('referrer_name', 'referrer_keyword')`
* is supplied, the query will aggregate visits for each referrer/keyword
* combination.
* @param bool|string $where Additional condition for the `WHERE` clause. Can be used to filter
* the set of visits that are considered for aggregation.
* @param array $additionalSelects Additional `SELECT` fields that are not included in the group by
* clause. These can be aggregate expressions, eg, `SUM(somecol)`.
* @param bool|array $metrics The set of metrics to calculate and return. If false, the query will select
* all of them. The following values can be used:
*
* - {@link Piwik\Metrics::INDEX_NB_UNIQ_VISITORS}
* - {@link Piwik\Metrics::INDEX_NB_VISITS}
* - {@link Piwik\Metrics::INDEX_NB_ACTIONS}
* - {@link Piwik\Metrics::INDEX_MAX_ACTIONS}
* - {@link Piwik\Metrics::INDEX_SUM_VISIT_LENGTH}
* - {@link Piwik\Metrics::INDEX_BOUNCE_COUNT}
* - {@link Piwik\Metrics::INDEX_NB_VISITS_CONVERTED}
* @param bool|\Piwik\RankingQuery $rankingQuery
* A pre-configured ranking query instance that will be used to limit the result.
* If set, the return value is the array returned by {@link Piwik\RankingQuery::execute()}.
* @return mixed A Zend_Db_Statement if `$rankingQuery` isn't supplied, otherwise the result of
* {@link Piwik\RankingQuery::execute()}. Read {@link queryVisitsByDimension() this}
* to see what aggregate data is calculated by the query.
* @api
*/
public function queryVisitsByDimension(array $dimensions = array(), $where = false, array $additionalSelects = array(),
$metrics = false, $rankingQuery = false)
{
$tableName = self::LOG_VISIT_TABLE;
$availableMetrics = $this->getVisitsMetricFields();
$select = $this->getSelectStatement($dimensions, $tableName, $additionalSelects, $availableMetrics, $metrics);
$from = array($tableName);
$where = $this->getWhereStatement($tableName, self::VISIT_DATETIME_FIELD, $where);
$groupBy = $this->getGroupByStatement($dimensions, $tableName);
$orderBy = false;
if ($rankingQuery) {
$orderBy = '`' . Metrics::INDEX_NB_VISITS . '` DESC';
}
$query = $this->generateQuery($select, $from, $where, $groupBy, $orderBy);
if ($rankingQuery) {
unset($availableMetrics[Metrics::INDEX_MAX_ACTIONS]);
$sumColumns = array_keys($availableMetrics);
if ($metrics) {
$sumColumns = array_intersect($sumColumns, $metrics);
}
$rankingQuery->addColumn($sumColumns, 'sum');
if ($this->isMetricRequested(Metrics::INDEX_MAX_ACTIONS, $metrics)) {
$rankingQuery->addColumn(Metrics::INDEX_MAX_ACTIONS, 'max');
}
return $rankingQuery->execute($query['sql'], $query['bind']);
}
return $this->getDb()->query($query['sql'], $query['bind']);
}
protected function getSelectsMetrics($metricsAvailable, $metricsRequested = false)
{
$selects = array();
foreach ($metricsAvailable as $metricId => $statement) {
if ($this->isMetricRequested($metricId, $metricsRequested)) {
$aliasAs = $this->getSelectAliasAs($metricId);
$selects[] = $statement . $aliasAs;
}
}
return $selects;
}
protected function getSelectStatement($dimensions, $tableName, $additionalSelects, array $availableMetrics, $requestedMetrics = false)
{
$dimensionsToSelect = $this->getDimensionsToSelect($dimensions, $additionalSelects);
$selects = array_merge(
$this->getSelectDimensions($dimensionsToSelect, $tableName),
$this->getSelectsMetrics($availableMetrics, $requestedMetrics),
!empty($additionalSelects) ? $additionalSelects : array()
);
$select = implode(self::FIELDS_SEPARATOR, $selects);
return $select;
}
/**
* Will return the subset of $dimensions that are not found in $additionalSelects
*
* @param $dimensions
* @param array $additionalSelects
* @return array
*/
protected function getDimensionsToSelect($dimensions, $additionalSelects)
{
if (empty($additionalSelects)) {
return $dimensions;
}
$dimensionsToSelect = array();
foreach ($dimensions as $selectAs => $dimension) {
$asAlias = $this->getSelectAliasAs($dimension);
foreach ($additionalSelects as $additionalSelect) {
if (strpos($additionalSelect, $asAlias) === false) {
$dimensionsToSelect[$selectAs] = $dimension;
}
}
}
$dimensionsToSelect = array_unique($dimensionsToSelect);
return $dimensionsToSelect;
}
/**
* Returns the dimensions array, where
* (1) the table name is prepended to the field
* (2) the "AS `label` " is appended to the field
*
* @param $dimensions
* @param $tableName
* @param bool $appendSelectAs
* @return mixed
*/
protected function getSelectDimensions($dimensions, $tableName, $appendSelectAs = true)
{
foreach ($dimensions as $selectAs => &$field) {
$selectAsString = $field;
if (!is_numeric($selectAs)) {
$selectAsString = $selectAs;
} else {
// if function, do not alias or prefix
if ($this->isFieldFunctionOrComplexExpression($field)) {
$selectAsString = $appendSelectAs = false;
}
}
$isKnownField = !in_array($field, array('referrer_data'));
if ($selectAsString == $field
&& $isKnownField
) {
$field = $this->prefixColumn($field, $tableName);
}
if ($appendSelectAs && $selectAsString) {
$field = $this->prefixColumn($field, $tableName) . $this->getSelectAliasAs($selectAsString);
}
}
return $dimensions;
}
/**
* Prefixes a column name with a table name if not already done.
*
* @param string $column eg, 'location_provider'
* @param string $tableName eg, 'log_visit'
* @return string eg, 'log_visit.location_provider'
*/
private function prefixColumn($column, $tableName)
{
if (strpos($column, '.') === false) {
return $tableName . '.' . $column;
} else {
return $column;
}
}
protected function isFieldFunctionOrComplexExpression($field)
{
return strpos($field, "(") !== false
|| strpos($field, "CASE") !== false;
}
protected function getSelectAliasAs($metricId)
{
return " AS `" . $metricId . "`";
}
protected function isMetricRequested($metricId, $metricsRequested)
{
return $metricsRequested === false
|| in_array($metricId, $metricsRequested);
}
protected function getWhereStatement($tableName, $datetimeField, $extraWhere = false)
{
$where = "$tableName.$datetimeField >= ?
AND $tableName.$datetimeField <= ?
AND $tableName.idsite = ?";
if (!empty($extraWhere)) {
$extraWhere = sprintf($extraWhere, $tableName, $tableName);
$where .= ' AND ' . $extraWhere;
}
return $where;
}
protected function getGroupByStatement($dimensions, $tableName)
{
$dimensions = $this->getSelectDimensions($dimensions, $tableName, $appendSelectAs = false);
$groupBy = implode(", ", $dimensions);
return $groupBy;
}
protected function getBindDatetimeSite()
{
return array($this->dateStart->getDateStartUTC(), $this->dateEnd->getDateEndUTC(), $this->site->getId());
}
/**
* Executes and returns a query aggregating ecommerce item data (everything stored in the
* **log\_conversion\_item** table) and returns a DB statement that can be used to iterate over the result
*
* <a name="queryEcommerceItems-result-set"></a>
* **Result Set**
*
* Each row of the result set represents an aggregated group of ecommerce items. The following
* columns are in each row of the result set:
*
* - **{@link Piwik\Metrics::INDEX_ECOMMERCE_ITEM_REVENUE}**: The total revenue for the group of items.
* - **{@link Piwik\Metrics::INDEX_ECOMMERCE_ITEM_QUANTITY}**: The total number of items in this group.
* - **{@link Piwik\Metrics::INDEX_ECOMMERCE_ITEM_PRICE}**: The total price for the group of items.
* - **{@link Piwik\Metrics::INDEX_ECOMMERCE_ORDERS}**: The total number of orders this group of items
* belongs to. This will be <= to the total number
* of items in this group.
* - **{@link Piwik\Metrics::INDEX_NB_VISITS}**: The total number of visits that caused these items to be logged.
* - **ecommerceType**: Either {@link Piwik\Tracker\GoalManager::IDGOAL_CART} if the items in this group were
* abandoned by a visitor, or {@link Piwik\Tracker\GoalManager::IDGOAL_ORDER} if they
* were ordered by a visitor.
*
* **Limitations**
*
* Segmentation is not yet supported for this aggregation method.
*
* @param string $dimension One or more **log\_conversion\_item** columns to group aggregated data by.
* Eg, `'idaction_sku'` or `'idaction_sku, idaction_category'`.
* @return Zend_Db_Statement A statement object that can be used to iterate through the query's
* result set. See [above](#queryEcommerceItems-result-set) to learn more
* about what this query selects.
* @api
*/
public function queryEcommerceItems($dimension)
{
$query = $this->generateQuery(
// SELECT ...
implode(
', ',
array(
"log_action.name AS label",
sprintf("log_conversion_item.%s AS labelIdAction", $dimension),
sprintf(
'%s AS `%d`',
self::getSqlRevenue('SUM(log_conversion_item.quantity * log_conversion_item.price)'),
Metrics::INDEX_ECOMMERCE_ITEM_REVENUE
),
sprintf(
'%s AS `%d`',
self::getSqlRevenue('SUM(log_conversion_item.quantity)'),
Metrics::INDEX_ECOMMERCE_ITEM_QUANTITY
),
sprintf(
'%s AS `%d`',
self::getSqlRevenue('SUM(log_conversion_item.price)'),
Metrics::INDEX_ECOMMERCE_ITEM_PRICE
),
sprintf(
'COUNT(distinct log_conversion_item.idorder) AS `%d`',
Metrics::INDEX_ECOMMERCE_ORDERS
),
sprintf(
'COUNT(distinct log_conversion_item.idvisit) AS `%d`',
Metrics::INDEX_NB_VISITS
),
sprintf(
'CASE log_conversion_item.idorder WHEN \'0\' THEN %d ELSE %d END AS ecommerceType',
GoalManager::IDGOAL_CART,
GoalManager::IDGOAL_ORDER
)
)
),
// FROM ...
array(
"log_conversion_item",
array(
"table" => "log_action",
"joinOn" => sprintf("log_conversion_item.%s = log_action.idaction", $dimension)
)
),
// WHERE ... AND ...
implode(
' AND ',
array(
'log_conversion_item.server_time >= ?',
'log_conversion_item.server_time <= ?',
'log_conversion_item.idsite = ?',
'log_conversion_item.deleted = 0'
)
),
// GROUP BY ...
sprintf(
"ecommerceType, log_conversion_item.%s",
$dimension
),
// ORDER ...
false
);
return $this->getDb()->query($query['sql'], $query['bind']);
}
/**
* Executes and returns a query aggregating action data (everything in the log_action table) and returns
* a DB statement that can be used to iterate over the result
*
* <a name="queryActionsByDimension-result-set"></a>
* **Result Set**
*
* Each row of the result set represents an aggregated group of actions. The following columns
* are in each aggregate row:
*
* - **{@link Piwik\Metrics::INDEX_NB_UNIQ_VISITORS}**: The total number of unique visitors that performed
* the actions in this group.
* - **{@link Piwik\Metrics::INDEX_NB_VISITS}**: The total number of visits these actions belong to.
* - **{@link Piwik\Metrics::INDEX_NB_ACTIONS}**: The total number of actions in this aggregate group.
*
* Additional data can be selected through the `$additionalSelects` parameter.
*
* _Note: The metrics calculated by this query can be customized by the `$metrics` parameter._
*
* @param array|string $dimensions One or more SELECT fields that will be used to group the log_action
* rows by. This parameter determines which log_action rows will be
* aggregated together.
* @param bool|string $where Additional condition for the WHERE clause. Can be used to filter
* the set of visits that are considered for aggregation.
* @param array $additionalSelects Additional SELECT fields that are not included in the group by
* clause. These can be aggregate expressions, eg, `SUM(somecol)`.
* @param bool|array $metrics The set of metrics to calculate and return. If `false`, the query will select
* all of them. The following values can be used:
*
* - {@link Piwik\Metrics::INDEX_NB_UNIQ_VISITORS}
* - {@link Piwik\Metrics::INDEX_NB_VISITS}
* - {@link Piwik\Metrics::INDEX_NB_ACTIONS}
* @param bool|\Piwik\RankingQuery $rankingQuery
* A pre-configured ranking query instance that will be used to limit the result.
* If set, the return value is the array returned by {@link Piwik\RankingQuery::execute()}.
* @param bool|string $joinLogActionOnColumn One or more columns from the **log_link_visit_action** table that
* log_action should be joined on. The table alias used for each join
* is `"log_action$i"` where `$i` is the index of the column in this
* array.
*
* If a string is used for this parameter, the table alias is not
* suffixed (since there is only one column).
* @return mixed A Zend_Db_Statement if `$rankingQuery` isn't supplied, otherwise the result of
* {@link Piwik\RankingQuery::execute()}. Read [this](#queryEcommerceItems-result-set)
* to see what aggregate data is calculated by the query.
* @api
*/
public function queryActionsByDimension($dimensions, $where = '', $additionalSelects = array(), $metrics = false, $rankingQuery = null, $joinLogActionOnColumn = false)
{
$tableName = self::LOG_ACTIONS_TABLE;
$availableMetrics = $this->getActionsMetricFields();
$select = $this->getSelectStatement($dimensions, $tableName, $additionalSelects, $availableMetrics, $metrics);
$from = array($tableName);
$where = $this->getWhereStatement($tableName, self::ACTION_DATETIME_FIELD, $where);
$groupBy = $this->getGroupByStatement($dimensions, $tableName);
$orderBy = false;
if ($joinLogActionOnColumn !== false) {
$multiJoin = is_array($joinLogActionOnColumn);
if (!$multiJoin) {
$joinLogActionOnColumn = array($joinLogActionOnColumn);
}
foreach ($joinLogActionOnColumn as $i => $joinColumn) {
$tableAlias = 'log_action' . ($multiJoin ? $i + 1 : '');
if (strpos($joinColumn, ' ') === false) {
$joinOn = $tableAlias . '.idaction = ' . $tableName . '.' . $joinColumn;
} else {
// more complex join column like IF(...)
$joinOn = $tableAlias . '.idaction = ' . $joinColumn;
}
$from[] = array(
'table' => 'log_action',
'tableAlias' => $tableAlias,
'joinOn' => $joinOn
);
}
}
if ($rankingQuery) {
$orderBy = '`' . Metrics::INDEX_NB_ACTIONS . '` DESC';
}
$query = $this->generateQuery($select, $from, $where, $groupBy, $orderBy);
if ($rankingQuery !== null) {
$sumColumns = array_keys($availableMetrics);
if ($metrics) {
$sumColumns = array_intersect($sumColumns, $metrics);
}
$rankingQuery->addColumn($sumColumns, 'sum');
return $rankingQuery->execute($query['sql'], $query['bind']);
}
return $this->getDb()->query($query['sql'], $query['bind']);
}
protected function getActionsMetricFields()
{
return $availableMetrics = array(
Metrics::INDEX_NB_VISITS => "count(distinct " . self::LOG_ACTIONS_TABLE . ".idvisit)",
Metrics::INDEX_NB_UNIQ_VISITORS => "count(distinct " . self::LOG_ACTIONS_TABLE . ".idvisitor)",
Metrics::INDEX_NB_ACTIONS => "count(*)",
);
}
/**
* Executes a query aggregating conversion data (everything in the **log_conversion** table) and returns
* a DB statement that can be used to iterate over the result.
*
* <a name="queryConversionsByDimension-result-set"></a>
* **Result Set**
*
* Each row of the result set represents an aggregated group of conversions. The
* following columns are in each aggregate row:
*
* - **{@link Piwik\Metrics::INDEX_GOAL_NB_CONVERSIONS}**: The total number of conversions in this aggregate
* group.
* - **{@link Piwik\Metrics::INDEX_GOAL_NB_VISITS_CONVERTED}**: The total number of visits during which these
* conversions were converted.
* - **{@link Piwik\Metrics::INDEX_GOAL_REVENUE}**: The total revenue generated by these conversions. This value
* includes the revenue from individual ecommerce items.
* - **{@link Piwik\Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_SUBTOTAL}**: The total cost of all ecommerce items sold
* within these conversions. This value does not
* include tax, shipping or any applied discount.
*
* _This metric is only applicable to the special
* **ecommerce** goal (where `idGoal == 'ecommerceOrder'`)._
* - **{@link Piwik\Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_TAX}**: The total tax applied to every transaction in these
* conversions.
*
* _This metric is only applicable to the special
* **ecommerce** goal (where `idGoal == 'ecommerceOrder'`)._
* - **{@link Piwik\Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_SHIPPING}**: The total shipping cost for every transaction
* in these conversions.
*
* _This metric is only applicable to the special
* **ecommerce** goal (where `idGoal == 'ecommerceOrder'`)._
* - **{@link Piwik\Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_DISCOUNT}**: The total discount applied to every transaction
* in these conversions.
*
* _This metric is only applicable to the special
* **ecommerce** goal (where `idGoal == 'ecommerceOrder'`)._
* - **{@link Piwik\Metrics::INDEX_GOAL_ECOMMERCE_ITEMS}**: The total number of ecommerce items sold in each transaction
* in these conversions.
*
* _This metric is only applicable to the special
* **ecommerce** goal (where `idGoal == 'ecommerceOrder'`)._
*
* Additional data can be selected through the `$additionalSelects` parameter.
*
* _Note: This method will only query the **log_conversion** table. Other tables cannot be joined
* using this method._
*
* @param array|string $dimensions One or more **SELECT** fields that will be used to group the log_conversion
* rows by. This parameter determines which **log_conversion** rows will be
* aggregated together.
* @param bool|string $where An optional SQL expression used in the SQL's **WHERE** clause.
* @param array $additionalSelects Additional SELECT fields that are not included in the group by
* clause. These can be aggregate expressions, eg, `SUM(somecol)`.
* @return Zend_Db_Statement
*/
public function queryConversionsByDimension($dimensions = array(), $where = false, $additionalSelects = array())
{
$dimensions = array_merge(array(self::IDGOAL_FIELD), $dimensions);
$availableMetrics = $this->getConversionsMetricFields();
$tableName = self::LOG_CONVERSION_TABLE;
$select = $this->getSelectStatement($dimensions, $tableName, $additionalSelects, $availableMetrics);
$from = array($tableName);
$where = $this->getWhereStatement($tableName, self::CONVERSION_DATETIME_FIELD, $where);
$groupBy = $this->getGroupByStatement($dimensions, $tableName);
$orderBy = false;
$query = $this->generateQuery($select, $from, $where, $groupBy, $orderBy);
return $this->getDb()->query($query['sql'], $query['bind']);
}
/**
* Creates and returns an array of SQL `SELECT` expressions that will each count how
* many rows have a column whose value is within a certain range.
*
* **Note:** The result of this function is meant for use in the `$additionalSelects` parameter
* in one of the query... methods (for example {@link queryVisitsByDimension()}).
*
* **Example**
*
* // summarize one column
* $visitTotalActionsRanges = array(
* array(1, 1),
* array(2, 10),
* array(10)
* );
* $selects = LogAggregator::getSelectsFromRangedColumn('visit_total_actions', $visitTotalActionsRanges, 'log_visit', 'vta');
*
* // summarize another column in the same request
* $visitCountVisitsRanges = array(
* array(1, 1),
* array(2, 20),
* array(20)
* );
* $selects = array_merge(
* $selects,
* LogAggregator::getSelectsFromRangedColumn('visitor_count_visits', $visitCountVisitsRanges, 'log_visit', 'vcv')
* );
*
* // perform the query
* $logAggregator = // get the LogAggregator somehow
* $query = $logAggregator->queryVisitsByDimension($dimensions = array(), $where = false, $selects);
* $tableSummary = $query->fetch();
*
* $numberOfVisitsWithOneAction = $tableSummary['vta0'];
* $numberOfVisitsBetweenTwoAnd10 = $tableSummary['vta1'];
*
* $numberOfVisitsWithVisitCountOfOne = $tableSummary['vcv0'];
*
* @param string $column The name of a column in `$table` that will be summarized.
* @param array $ranges The array of ranges over which the data in the table
* will be summarized. For example,
* ```
* array(
* array(1, 1),
* array(2, 2),
* array(3, 8),
* array(8) // everything over 8
* )
* ```
* @param string $table The unprefixed name of the table whose rows will be summarized.
* @param string $selectColumnPrefix The prefix to prepend to each SELECT expression. This
* prefix is used to differentiate different sets of
* range summarization SELECTs. You can supply different
* values to this argument to summarize several columns
* in one query (see above for an example).
* @param bool $restrictToReturningVisitors Whether to only summarize rows that belong to
* visits of returning visitors or not. If this
* argument is true, then the SELECT expressions
* returned can only be used with the
* {@link queryVisitsByDimension()} method.
* @return array An array of SQL SELECT expressions, for example,
* ```
* array(
* 'sum(case when log_visit.visit_total_actions between 0 and 2 then 1 else 0 end) as vta0',
* 'sum(case when log_visit.visit_total_actions > 2 then 1 else 0 end) as vta1'
* )
* ```
* @api
*/
public static function getSelectsFromRangedColumn($column, $ranges, $table, $selectColumnPrefix, $restrictToReturningVisitors = false)
{
$selects = array();
$extraCondition = '';
if ($restrictToReturningVisitors) {
// extra condition for the SQL SELECT that makes sure only returning visits are counted
// when creating the 'days since last visit' report
$extraCondition = 'and log_visit.visitor_returning = 1';
$extraSelect = "sum(case when log_visit.visitor_returning = 0 then 1 else 0 end) "
. " as `" . $selectColumnPrefix . 'General_NewVisits' . "`";
$selects[] = $extraSelect;
}
foreach ($ranges as $gap) {
if (count($gap) == 2) {
$lowerBound = $gap[0];
$upperBound = $gap[1];
$selectAs = "$selectColumnPrefix$lowerBound-$upperBound";
$selects[] = "sum(case when $table.$column between $lowerBound and $upperBound $extraCondition" .
" then 1 else 0 end) as `$selectAs`";
} else {
$lowerBound = $gap[0];
$selectAs = $selectColumnPrefix . ($lowerBound + 1) . urlencode('+');
$selects[] = "sum(case when $table.$column > $lowerBound $extraCondition then 1 else 0 end) as `$selectAs`";
}
}
return $selects;
}
/**
* Clean up the row data and return values.
* $lookForThisPrefix can be used to make sure only SOME of the data in $row is used.
*
* The array will have one column $columnName
*
* @param $row
* @param $columnName
* @param bool $lookForThisPrefix A string that identifies which elements of $row to use
* in the result. Every key of $row that starts with this
* value is used.
* @return array
*/
static public function makeArrayOneColumn($row, $columnName, $lookForThisPrefix = false)
{
$cleanRow = array();
foreach ($row as $label => $count) {
if (empty($lookForThisPrefix)
|| strpos($label, $lookForThisPrefix) === 0
) {
$cleanLabel = substr($label, strlen($lookForThisPrefix));
$cleanRow[$cleanLabel] = array($columnName => $count);
}
}
return $cleanRow;
}
public function getDb()
{
return Db::get();
}
}

396
www/analytics/core/DataArray.php Normal file
View file

@ -0,0 +1,396 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik;
use Exception;
use Piwik\Tracker\GoalManager;
/**
* The DataArray is a data structure used to aggregate datasets,
* ie. sum arrays made of rows made of columns,
* data from the logs is stored in a DataArray before being converted in a DataTable
*
*/
class DataArray
{
protected $data = array();
protected $dataTwoLevels = array();
public function __construct($data = array(), $dataArrayByLabel = array())
{
$this->data = $data;
$this->dataTwoLevels = $dataArrayByLabel;
}
/**
* This returns the actual raw data array
*
* @return array
*/
public function &getDataArray()
{
return $this->data;
}
public function getDataArrayWithTwoLevels()
{
return $this->dataTwoLevels;
}
public function sumMetricsVisits($label, $row)
{
if (!isset($this->data[$label])) {
$this->data[$label] = self::makeEmptyRow();
}
$this->doSumVisitsMetrics($row, $this->data[$label]);
}
/**
* Returns an empty row containing default metrics
*
* @return array
*/
static public function makeEmptyRow()
{
return array(Metrics::INDEX_NB_UNIQ_VISITORS => 0,
Metrics::INDEX_NB_VISITS => 0,
Metrics::INDEX_NB_ACTIONS => 0,
Metrics::INDEX_MAX_ACTIONS => 0,
Metrics::INDEX_SUM_VISIT_LENGTH => 0,
Metrics::INDEX_BOUNCE_COUNT => 0,
Metrics::INDEX_NB_VISITS_CONVERTED => 0,
);
}
/**
* Adds the given row $newRowToAdd to the existing $oldRowToUpdate passed by reference
* The rows are php arrays Name => value
*
* @param array $newRowToAdd
* @param array $oldRowToUpdate
* @param bool $onlyMetricsAvailableInActionsTable
*
* @return void
*/
protected function doSumVisitsMetrics($newRowToAdd, &$oldRowToUpdate, $onlyMetricsAvailableInActionsTable = false)
{
// Pre 1.2 format: string indexed rows are returned from the DB
// Left here for Backward compatibility with plugins doing custom SQL queries using these metrics as string
if (!isset($newRowToAdd[Metrics::INDEX_NB_VISITS])) {
$oldRowToUpdate[Metrics::INDEX_NB_VISITS] += $newRowToAdd['nb_visits'];
$oldRowToUpdate[Metrics::INDEX_NB_ACTIONS] += $newRowToAdd['nb_actions'];
$oldRowToUpdate[Metrics::INDEX_NB_UNIQ_VISITORS] += $newRowToAdd['nb_uniq_visitors'];
if ($onlyMetricsAvailableInActionsTable) {
return;
}
$oldRowToUpdate[Metrics::INDEX_MAX_ACTIONS] = (float)max($newRowToAdd['max_actions'], $oldRowToUpdate[Metrics::INDEX_MAX_ACTIONS]);
$oldRowToUpdate[Metrics::INDEX_SUM_VISIT_LENGTH] += $newRowToAdd['sum_visit_length'];
$oldRowToUpdate[Metrics::INDEX_BOUNCE_COUNT] += $newRowToAdd['bounce_count'];
$oldRowToUpdate[Metrics::INDEX_NB_VISITS_CONVERTED] += $newRowToAdd['nb_visits_converted'];
return;
}
$oldRowToUpdate[Metrics::INDEX_NB_VISITS] += $newRowToAdd[Metrics::INDEX_NB_VISITS];
$oldRowToUpdate[Metrics::INDEX_NB_ACTIONS] += $newRowToAdd[Metrics::INDEX_NB_ACTIONS];
$oldRowToUpdate[Metrics::INDEX_NB_UNIQ_VISITORS] += $newRowToAdd[Metrics::INDEX_NB_UNIQ_VISITORS];
if ($onlyMetricsAvailableInActionsTable) {
return;
}
// In case the existing Row had no action metrics (eg. Custom Variable XYZ with "visit" scope)
// but the new Row has action metrics (eg. same Custom Variable XYZ this time with a "page" scope)
if(!isset($oldRowToUpdate[Metrics::INDEX_MAX_ACTIONS])) {
$toZero = array(Metrics::INDEX_MAX_ACTIONS,
Metrics::INDEX_SUM_VISIT_LENGTH,
Metrics::INDEX_BOUNCE_COUNT,
Metrics::INDEX_NB_VISITS_CONVERTED);
foreach($toZero as $metric) {
$oldRowToUpdate[$metric] = 0;
}
}
$oldRowToUpdate[Metrics::INDEX_MAX_ACTIONS] = (float)max($newRowToAdd[Metrics::INDEX_MAX_ACTIONS], $oldRowToUpdate[Metrics::INDEX_MAX_ACTIONS]);
$oldRowToUpdate[Metrics::INDEX_SUM_VISIT_LENGTH] += $newRowToAdd[Metrics::INDEX_SUM_VISIT_LENGTH];
$oldRowToUpdate[Metrics::INDEX_BOUNCE_COUNT] += $newRowToAdd[Metrics::INDEX_BOUNCE_COUNT];
$oldRowToUpdate[Metrics::INDEX_NB_VISITS_CONVERTED] += $newRowToAdd[Metrics::INDEX_NB_VISITS_CONVERTED];
}
public function sumMetricsGoals($label, $row)
{
$idGoal = $row['idgoal'];
if (!isset($this->data[$label][Metrics::INDEX_GOALS][$idGoal])) {
$this->data[$label][Metrics::INDEX_GOALS][$idGoal] = self::makeEmptyGoalRow($idGoal);
}
$this->doSumGoalsMetrics($row, $this->data[$label][Metrics::INDEX_GOALS][$idGoal]);
}
/**
* @param $idGoal
* @return array
*/
protected static function makeEmptyGoalRow($idGoal)
{
if ($idGoal > GoalManager::IDGOAL_ORDER) {
return array(Metrics::INDEX_GOAL_NB_CONVERSIONS => 0,
Metrics::INDEX_GOAL_NB_VISITS_CONVERTED => 0,
Metrics::INDEX_GOAL_REVENUE => 0,
);
}
if ($idGoal == GoalManager::IDGOAL_ORDER) {
return array(Metrics::INDEX_GOAL_NB_CONVERSIONS => 0,
Metrics::INDEX_GOAL_NB_VISITS_CONVERTED => 0,
Metrics::INDEX_GOAL_REVENUE => 0,
Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_SUBTOTAL => 0,
Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_TAX => 0,
Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_SHIPPING => 0,
Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_DISCOUNT => 0,
Metrics::INDEX_GOAL_ECOMMERCE_ITEMS => 0,
);
}
// idGoal == GoalManager::IDGOAL_CART
return array(Metrics::INDEX_GOAL_NB_CONVERSIONS => 0,
Metrics::INDEX_GOAL_NB_VISITS_CONVERTED => 0,
Metrics::INDEX_GOAL_REVENUE => 0,
Metrics::INDEX_GOAL_ECOMMERCE_ITEMS => 0,
);
}
/**
*
* @param $newRowToAdd
* @param $oldRowToUpdate
*/
protected function doSumGoalsMetrics($newRowToAdd, &$oldRowToUpdate)
{
$oldRowToUpdate[Metrics::INDEX_GOAL_NB_CONVERSIONS] += $newRowToAdd[Metrics::INDEX_GOAL_NB_CONVERSIONS];
$oldRowToUpdate[Metrics::INDEX_GOAL_NB_VISITS_CONVERTED] += $newRowToAdd[Metrics::INDEX_GOAL_NB_VISITS_CONVERTED];
$oldRowToUpdate[Metrics::INDEX_GOAL_REVENUE] += $newRowToAdd[Metrics::INDEX_GOAL_REVENUE];
// Cart & Order
if (isset($oldRowToUpdate[Metrics::INDEX_GOAL_ECOMMERCE_ITEMS])) {
$oldRowToUpdate[Metrics::INDEX_GOAL_ECOMMERCE_ITEMS] += $newRowToAdd[Metrics::INDEX_GOAL_ECOMMERCE_ITEMS];
// Order only
if (isset($oldRowToUpdate[Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_SUBTOTAL])) {
$oldRowToUpdate[Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_SUBTOTAL] += $newRowToAdd[Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_SUBTOTAL];
$oldRowToUpdate[Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_TAX] += $newRowToAdd[Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_TAX];
$oldRowToUpdate[Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_SHIPPING] += $newRowToAdd[Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_SHIPPING];
$oldRowToUpdate[Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_DISCOUNT] += $newRowToAdd[Metrics::INDEX_GOAL_ECOMMERCE_REVENUE_DISCOUNT];
}
}
}
public function sumMetricsActions($label, $row)
{
if (!isset($this->data[$label])) {
$this->data[$label] = self::makeEmptyActionRow();
}
$this->doSumVisitsMetrics($row, $this->data[$label], $onlyMetricsAvailableInActionsTable = true);
}
static protected function makeEmptyActionRow()
{
return array(
Metrics::INDEX_NB_UNIQ_VISITORS => 0,
Metrics::INDEX_NB_VISITS => 0,
Metrics::INDEX_NB_ACTIONS => 0,
);
}
public function sumMetricsEvents($label, $row)
{
if (!isset($this->data[$label])) {
$this->data[$label] = self::makeEmptyEventRow();
}
$this->doSumEventsMetrics($row, $this->data[$label], $onlyMetricsAvailableInActionsTable = true);
}
static protected function makeEmptyEventRow()
{
return array(
Metrics::INDEX_NB_UNIQ_VISITORS => 0,
Metrics::INDEX_NB_VISITS => 0,
Metrics::INDEX_EVENT_NB_HITS => 0,
Metrics::INDEX_EVENT_NB_HITS_WITH_VALUE => 0,
Metrics::INDEX_EVENT_SUM_EVENT_VALUE => 0,
Metrics::INDEX_EVENT_MIN_EVENT_VALUE => 0,
Metrics::INDEX_EVENT_MAX_EVENT_VALUE => 0,
);
}
const EVENT_VALUE_PRECISION = 2;
/**
* @param array $newRowToAdd
* @param array $oldRowToUpdate
* @return void
*/
protected function doSumEventsMetrics($newRowToAdd, &$oldRowToUpdate)
{
$oldRowToUpdate[Metrics::INDEX_NB_VISITS] += $newRowToAdd[Metrics::INDEX_NB_VISITS];
$oldRowToUpdate[Metrics::INDEX_NB_UNIQ_VISITORS] += $newRowToAdd[Metrics::INDEX_NB_UNIQ_VISITORS];
$oldRowToUpdate[Metrics::INDEX_EVENT_NB_HITS] += $newRowToAdd[Metrics::INDEX_EVENT_NB_HITS];
$oldRowToUpdate[Metrics::INDEX_EVENT_NB_HITS_WITH_VALUE] += $newRowToAdd[Metrics::INDEX_EVENT_NB_HITS_WITH_VALUE];
$newRowToAdd[Metrics::INDEX_EVENT_SUM_EVENT_VALUE] = round($newRowToAdd[Metrics::INDEX_EVENT_SUM_EVENT_VALUE], self::EVENT_VALUE_PRECISION);
$oldRowToUpdate[Metrics::INDEX_EVENT_SUM_EVENT_VALUE] += $newRowToAdd[Metrics::INDEX_EVENT_SUM_EVENT_VALUE];
$oldRowToUpdate[Metrics::INDEX_EVENT_MAX_EVENT_VALUE] = round(max($newRowToAdd[Metrics::INDEX_EVENT_MAX_EVENT_VALUE], $oldRowToUpdate[Metrics::INDEX_EVENT_MAX_EVENT_VALUE]), self::EVENT_VALUE_PRECISION);
// Update minimum only if it is set
if($newRowToAdd[Metrics::INDEX_EVENT_MIN_EVENT_VALUE] !== false) {
if($oldRowToUpdate[Metrics::INDEX_EVENT_MIN_EVENT_VALUE] === false) {
$oldRowToUpdate[Metrics::INDEX_EVENT_MIN_EVENT_VALUE] = round($newRowToAdd[Metrics::INDEX_EVENT_MIN_EVENT_VALUE], self::EVENT_VALUE_PRECISION);
} else {
$oldRowToUpdate[Metrics::INDEX_EVENT_MIN_EVENT_VALUE] = round(min($newRowToAdd[Metrics::INDEX_EVENT_MIN_EVENT_VALUE], $oldRowToUpdate[Metrics::INDEX_EVENT_MIN_EVENT_VALUE]), self::EVENT_VALUE_PRECISION);
}
}
}
/**
* Generic function that will sum all columns of the given row, at the specified label's row.
*
* @param $label
* @param $row
* @throws Exception if the the data row contains non numeric values
*/
public function sumMetrics($label, $row)
{
foreach ($row as $columnName => $columnValue) {
if (empty($columnValue)) {
continue;
}
if (empty($this->data[$label][$columnName])) {
$this->data[$label][$columnName] = 0;
}
if (!is_numeric($columnValue)) {
throw new Exception("DataArray->sumMetricsPivot expects rows of numeric values, non numeric found: " . var_export($columnValue, true) . " for column $columnName");
}
$this->data[$label][$columnName] += $columnValue;
}
}
public function sumMetricsVisitsPivot($parentLabel, $label, $row)
{
if (!isset($this->dataTwoLevels[$parentLabel][$label])) {
$this->dataTwoLevels[$parentLabel][$label] = self::makeEmptyRow();
}
$this->doSumVisitsMetrics($row, $this->dataTwoLevels[$parentLabel][$label]);
}
public function sumMetricsGoalsPivot($parentLabel, $label, $row)
{
$idGoal = $row['idgoal'];
if (!isset($this->dataTwoLevels[$parentLabel][$label][Metrics::INDEX_GOALS][$idGoal])) {
$this->dataTwoLevels[$parentLabel][$label][Metrics::INDEX_GOALS][$idGoal] = self::makeEmptyGoalRow($idGoal);
}
$this->doSumGoalsMetrics($row, $this->dataTwoLevels[$parentLabel][$label][Metrics::INDEX_GOALS][$idGoal]);
}
public function sumMetricsActionsPivot($parentLabel, $label, $row)
{
if (!isset($this->dataTwoLevels[$parentLabel][$label])) {
$this->dataTwoLevels[$parentLabel][$label] = $this->makeEmptyActionRow();
}
$this->doSumVisitsMetrics($row, $this->dataTwoLevels[$parentLabel][$label], $onlyMetricsAvailableInActionsTable = true);
}
public function sumMetricsEventsPivot($parentLabel, $label, $row)
{
if (!isset($this->dataTwoLevels[$parentLabel][$label])) {
$this->dataTwoLevels[$parentLabel][$label] = $this->makeEmptyEventRow();
}
$this->doSumEventsMetrics($row, $this->dataTwoLevels[$parentLabel][$label]);
}
public function setRowColumnPivot($parentLabel, $label, $column, $value)
{
$this->dataTwoLevels[$parentLabel][$label][$column] = $value;
}
public function enrichMetricsWithConversions()
{
$this->enrichWithConversions($this->data);
foreach ($this->dataTwoLevels as &$metricsBySubLabel) {
$this->enrichWithConversions($metricsBySubLabel);
}
}
/**
* Given an array of stats, it will process the sum of goal conversions
* and sum of revenue and add it in the stats array in two new fields.
*
* @param array $data Passed by reference, two new columns
* will be added: total conversions, and total revenue, for all goals for this label/row
*/
protected function enrichWithConversions(&$data)
{
foreach ($data as $label => &$values) {
if (!isset($values[Metrics::INDEX_GOALS])) {
continue;
}
// When per goal metrics are processed, general 'visits converted' is not meaningful because
// it could differ from the sum of each goal conversions
unset($values[Metrics::INDEX_NB_VISITS_CONVERTED]);
$revenue = $conversions = 0;
foreach ($values[Metrics::INDEX_GOALS] as $idgoal => $goalValues) {
// Do not sum Cart revenue since it is a lost revenue
if ($idgoal >= GoalManager::IDGOAL_ORDER) {
$revenue += $goalValues[Metrics::INDEX_GOAL_REVENUE];
$conversions += $goalValues[Metrics::INDEX_GOAL_NB_CONVERSIONS];
}
}
$values[Metrics::INDEX_NB_CONVERSIONS] = $conversions;
// 25.00 recorded as 25
if (round($revenue) == $revenue) {
$revenue = round($revenue);
}
$values[Metrics::INDEX_REVENUE] = $revenue;
// if there are no "visit" column, we force one to prevent future complications
// eg. This helps the setDefaultColumnsToDisplay() call
if(!isset($values[Metrics::INDEX_NB_VISITS])) {
$values[Metrics::INDEX_NB_VISITS] = 0;
}
}
}
/**
* Returns true if the row looks like an Action metrics row
*
* @param $row
* @return bool
*/
static public function isRowActions($row)
{
return (count($row) == count(self::makeEmptyActionRow())) && isset($row[Metrics::INDEX_NB_ACTIONS]);
}
/**
* Converts array to a datatable
*
* @return \Piwik\DataTable
*/
public function asDataTable()
{
$dataArray = $this->getDataArray();
$dataArrayTwoLevels = $this->getDataArrayWithTwoLevels();
$subtableByLabel = null;
if (!empty($dataArrayTwoLevels)) {
$subtableByLabel = array();
foreach ($dataArrayTwoLevels as $label => $subTable) {
$subtableByLabel[$label] = DataTable::makeFromIndexedArray($subTable);
}
}
return DataTable::makeFromIndexedArray($dataArray, $subtableByLabel);
}
}

326
www/analytics/core/DataFiles/Countries.php Normal file
View file

@ -0,0 +1,326 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
/**
* Country code and continent database.
*
* The mapping of countries to continents is from MaxMind with the exception
* of Central America. MaxMind groups Central American countries with
* North America. Piwik previously grouped Central American countries with
* South America. Given this conflict and the fact that most of Central
* America lies on its own continental plate (i.e., the Caribbean Plate), we
* currently use a separate continent code (amc).
*/
if (!isset($GLOBALS['Piwik_CountryList'])) {
// Primary reference: ISO 3166-1 alpha-2
$GLOBALS['Piwik_CountryList'] = array(
'ad' => 'eur',
'ae' => 'asi',
'af' => 'asi',
'ag' => 'amc',
'ai' => 'amc',
'al' => 'eur',
'am' => 'asi',
'ao' => 'afr',
'aq' => 'ant',
'ar' => 'ams',
'as' => 'oce',
'at' => 'eur',
'au' => 'oce',
'aw' => 'amc',
'ax' => 'eur',
'az' => 'asi',
'ba' => 'eur',
'bb' => 'amc',
'bd' => 'asi',
'be' => 'eur',
'bf' => 'afr',
'bg' => 'eur',
'bh' => 'asi',
'bi' => 'afr',
'bj' => 'afr',
'bl' => 'amc',
'bm' => 'amc',
'bn' => 'asi',
'bo' => 'ams',
'bq' => 'amc',
'br' => 'ams',
'bs' => 'amc',
'bt' => 'asi',
'bv' => 'ant',
'bw' => 'afr',
'by' => 'eur',
'bz' => 'amc',
'ca' => 'amn',
'cc' => 'asi',
'cd' => 'afr',
'cf' => 'afr',
'cg' => 'afr',
'ch' => 'eur',
'ci' => 'afr',
'ck' => 'oce',
'cl' => 'ams',
'cm' => 'afr',
'cn' => 'asi',
'co' => 'ams',
'cr' => 'amc',
'cu' => 'amc',
'cv' => 'afr',
'cw' => 'amc',
'cx' => 'asi',
'cy' => 'eur',
'cz' => 'eur',
'de' => 'eur',
'dj' => 'afr',
'dk' => 'eur',
'dm' => 'amc',
'do' => 'amc',
'dz' => 'afr',
'ec' => 'ams',
'ee' => 'eur',
'eg' => 'afr',
'eh' => 'afr',
'er' => 'afr',
'es' => 'eur',
'et' => 'afr',
'fi' => 'eur',
'fj' => 'oce',
'fk' => 'ams',
'fm' => 'oce',
'fo' => 'eur',
'fr' => 'eur',
'ga' => 'afr',
'gb' => 'eur',
'gd' => 'amc',
'ge' => 'asi',
'gf' => 'ams',
'gg' => 'eur',
'gh' => 'afr',
'gi' => 'eur',
'gl' => 'amn',
'gm' => 'afr',
'gn' => 'afr',
'gp' => 'amc',
'gq' => 'afr',
'gr' => 'eur',
'gs' => 'ant',
'gt' => 'amc',
'gu' => 'oce',
'gw' => 'afr',
'gy' => 'ams',
'hk' => 'asi',
'hm' => 'ant',
'hn' => 'amc',
'hr' => 'eur',
'ht' => 'amc',
'hu' => 'eur',
'id' => 'asi',
'ie' => 'eur',
'il' => 'asi',
'im' => 'eur',
'in' => 'asi',
'io' => 'asi',
'iq' => 'asi',
'ir' => 'asi',
'is' => 'eur',
'it' => 'eur',
'je' => 'eur',
'jm' => 'amc',
'jo' => 'asi',
'jp' => 'asi',
'ke' => 'afr',
'kg' => 'asi',
'kh' => 'asi',
'ki' => 'oce',
'km' => 'afr',
'kn' => 'amc',
'kp' => 'asi',
'kr' => 'asi',
'kw' => 'asi',
'ky' => 'amc',
'kz' => 'asi',
'la' => 'asi',
'lb' => 'asi',
'lc' => 'amc',
'li' => 'eur',
'lk' => 'asi',
'lr' => 'afr',
'ls' => 'afr',
'lt' => 'eur',
'lu' => 'eur',
'lv' => 'eur',
'ly' => 'afr',
'ma' => 'afr',
'mc' => 'eur',
'md' => 'eur',
'me' => 'eur',
'mf' => 'amc',
'mg' => 'afr',
'mh' => 'oce',
'mk' => 'eur',
'ml' => 'afr',
'mm' => 'asi',
'mn' => 'asi',
'mo' => 'asi',
'mp' => 'oce',
'mq' => 'amc',
'mr' => 'afr',
'ms' => 'amc',
'mt' => 'eur',
'mu' => 'afr',
'mv' => 'asi',
'mw' => 'afr',
'mx' => 'amn',
'my' => 'asi',
'mz' => 'afr',
'na' => 'afr',
'nc' => 'oce',
'ne' => 'afr',
'nf' => 'oce',
'ng' => 'afr',
'ni' => 'amc',
'nl' => 'eur',
'no' => 'eur',
'np' => 'asi',
'nr' => 'oce',
'nu' => 'oce',
'nz' => 'oce',
'om' => 'asi',
'pa' => 'amc',
'pe' => 'ams',
'pf' => 'oce',
'pg' => 'oce',
'ph' => 'asi',
'pk' => 'asi',
'pl' => 'eur',
'pm' => 'amn',
'pn' => 'oce',
'pr' => 'amc',
'ps' => 'asi',
'pt' => 'eur',
'pw' => 'oce',
'py' => 'ams',
'qa' => 'asi',
're' => 'afr',
'ro' => 'eur',
'rs' => 'eur',
'ru' => 'eur',
'rw' => 'afr',
'sa' => 'asi',
'sb' => 'oce',
'sc' => 'afr',
'sd' => 'afr',
'se' => 'eur',
'sg' => 'asi',
'sh' => 'afr',
'si' => 'eur',
'sj' => 'eur',
'sk' => 'eur',
'sl' => 'afr',
'sm' => 'eur',
'sn' => 'afr',
'so' => 'afr',
'sr' => 'ams',
'ss' => 'afr',
'st' => 'afr',
'sv' => 'amc',
'sx' => 'amc',
'sy' => 'asi',
'sz' => 'afr',
'tc' => 'amc',
'td' => 'afr',
'tf' => 'ant',
'tg' => 'afr',
'th' => 'asi',
'ti' => 'asi',
'tj' => 'asi',
'tk' => 'oce',
'tl' => 'asi',
'tm' => 'asi',
'tn' => 'afr',
'to' => 'oce',
'tr' => 'eur',
'tt' => 'amc',
'tv' => 'oce',
'tw' => 'asi',
'tz' => 'afr',
'ua' => 'eur',
'ug' => 'afr',
'um' => 'oce',
'us' => 'amn',
'uy' => 'ams',
'uz' => 'asi',
'va' => 'eur',
'vc' => 'amc',
've' => 'ams',
'vg' => 'amc',
'vi' => 'amc',
'vn' => 'asi',
'vu' => 'oce',
'wf' => 'oce',
'ws' => 'oce',
'ye' => 'asi',
'yt' => 'afr',
'za' => 'afr',
'zm' => 'afr',
'zw' => 'afr',
);
// codes for internal use
$GLOBALS['Piwik_CountryList_Extras'] = array(
// unknown
'xx' => 'unk',
// exceptionally reserved
'ac' => 'afr', // .ac TLD
'cp' => 'amc',
'dg' => 'asi',
'ea' => 'afr',
'eu' => 'eur', // .eu TLD
'fx' => 'eur',
'ic' => 'afr',
'su' => 'eur', // .su TLD
'ta' => 'afr',
'uk' => 'eur', // .uk TLD
// transitionally reserved
'an' => 'amc', // former Netherlands Antilles
'bu' => 'asi',
'cs' => 'eur', // former Serbia and Montenegro
'nt' => 'asi',
'sf' => 'eur',
'tp' => 'oce', // .tp TLD
'yu' => 'eur', // .yu TLD
'zr' => 'afr',
// MaxMind GeoIP specific
'a1' => 'unk',
'a2' => 'unk',
'ap' => 'asi',
'o1' => 'unk',
// Catalonia (Spain)
'cat' => 'eur',
);
}
if (!isset($GLOBALS['Piwik_ContinentList'])) {
// Primary reference: ISO 3166-1 alpha-2
$GLOBALS['Piwik_ContinentList'] = array(
'unk', // unknown
'amn', // North America
'amc', // Central America
'ams', // South America
'eur', // Europe
'afr', // Africa
'asi', // Asia
'oce', // Oceania
'ant', // Antarctica
);
}

186
www/analytics/core/DataFiles/Currencies.php Normal file
View file

@ -0,0 +1,186 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
/**
* International currencies in circulation.
*
* @see http://en.wikipedia.org/wiki/List_of_circulating_currencies
*/
if (!isset($GLOBALS['Piwik_CurrencyList'])) {
$GLOBALS['Piwik_CurrencyList'] = array(
// 'ISO-4217 CODE' => array('currency symbol', 'description'),
// Top 5 by global trading volume
'USD' => array('$', 'US dollar'),
'EUR' => array('€', 'Euro'),
'JPY' => array('¥', 'Japanese yen'),
'GBP' => array('£', 'British pound'),
'CHF' => array('Fr', 'Swiss franc'),
'AFN' => array('؋', 'Afghan afghani'),
'ALL' => array('L', 'Albanian lek'),
'DZD' => array('د.ج', 'Algerian dinar'),
'AOA' => array('Kz', 'Angolan kwanza'),
'ARS' => array('$', 'Argentine peso'),
'AMD' => array('դր.', 'Armenian dram'),
'AWG' => array('ƒ', 'Aruban florin'),
'AUD' => array('$', 'Australian dollar'),
'AZN' => array('m', 'Azerbaijani manat'),
'BSD' => array('$', 'Bahamian dollar'),
'BHD' => array('.د.ب', 'Bahraini dinar'),
'BDT' => array('৳', 'Bangladeshi taka'),
'BBD' => array('$', 'Barbadian dollar'),
'BYR' => array('Br', 'Belarusian ruble'),
'BZD' => array('$', 'Belize dollar'),
'BMD' => array('$', 'Bermudian dollar'),
'BTC' => array('BTC', 'Bitcoin'),
'BTN' => array('Nu.', 'Bhutanese ngultrum'),
'BOB' => array('Bs.', 'Bolivian boliviano'),
'BAM' => array('KM', 'Bosnia Herzegovina mark'),
'BWP' => array('P', 'Botswana pula'),
'BRL' => array('R$', 'Brazilian real'),
// 'GBP' => array('£', 'British pound'),
'BND' => array('$', 'Brunei dollar'),
'BGN' => array('лв', 'Bulgarian lev'),
'BIF' => array('Fr', 'Burundian franc'),
'KHR' => array('៛', 'Cambodian riel'),
'CAD' => array('$', 'Canadian dollar'),
'CVE' => array('$', 'Cape Verdean escudo'),
'KYD' => array('$', 'Cayman Islands dollar'),
'XAF' => array('Fr', 'Central African CFA franc'),
'CLP' => array('$', 'Chilean peso'),
'CNY' => array('元', 'Chinese yuan'),
'COP' => array('$', 'Colombian peso'),
'KMF' => array('Fr', 'Comorian franc'),
'CDF' => array('Fr', 'Congolese franc'),
'CRC' => array('₡', 'Costa Rican colón'),
'HRK' => array('kn', 'Croatian kuna'),
'XPF' => array('F', 'CFP franc'),
'CUC' => array('$', 'Cuban convertible peso'),
'CUP' => array('$', 'Cuban peso'),
'CMG' => array('ƒ', 'Curaçao and Sint Maarten guilder'),
'CZK' => array('Kč', 'Czech koruna'),
'DKK' => array('kr', 'Danish krone'),
'DJF' => array('Fr', 'Djiboutian franc'),
'DOP' => array('$', 'Dominican peso'),
'XCD' => array('$', 'East Caribbean dollar'),
'EGP' => array('ج.م', 'Egyptian pound'),
'ERN' => array('Nfk', 'Eritrean nakfa'),
'ETB' => array('Br', 'Ethiopian birr'),
// 'EUR' => array('€', 'Euro'),
'FKP' => array('£', 'Falkland Islands pound'),
'FJD' => array('$', 'Fijian dollar'),
'GMD' => array('D', 'Gambian dalasi'),
'GEL' => array('ლ', 'Georgian lari'),
'GHS' => array('₵', 'Ghanaian cedi'),
'GIP' => array('£', 'Gibraltar pound'),
'GTQ' => array('Q', 'Guatemalan quetzal'),
'GNF' => array('Fr', 'Guinean franc'),
'GYD' => array('$', 'Guyanese dollar'),
'HTG' => array('G', 'Haitian gourde'),
'HNL' => array('L', 'Honduran lempira'),
'HKD' => array('$', 'Hong Kong dollar'),
'HUF' => array('Ft', 'Hungarian forint'),
'ISK' => array('kr', 'Icelandic króna'),
'INR' => array('‎₹', 'Indian rupee'),
'IDR' => array('Rp', 'Indonesian rupiah'),
'IRR' => array('﷼', 'Iranian rial'),
'IQD' => array('ع.د', 'Iraqi dinar'),
'ILS' => array('₪', 'Israeli new shekel'),
'JMD' => array('$', 'Jamaican dollar'),
// 'JPY' => array('¥', 'Japanese yen'),
'JOD' => array('د.ا', 'Jordanian dinar'),
'KZT' => array('₸', 'Kazakhstani tenge'),
'KES' => array('Sh', 'Kenyan shilling'),
'KWD' => array('د.ك', 'Kuwaiti dinar'),
'KGS' => array('лв', 'Kyrgyzstani som'),
'LAK' => array('₭', 'Lao kip'),
'LBP' => array('ل.ل', 'Lebanese pound'),
'LSL' => array('L', 'Lesotho loti'),
'LRD' => array('$', 'Liberian dollar'),
'LYD' => array('ل.د', 'Libyan dinar'),
'LTL' => array('Lt', 'Lithuanian litas'),
'MOP' => array('P', 'Macanese pataca'),
'MKD' => array('ден', 'Macedonian denar'),
'MGA' => array('Ar', 'Malagasy ariary'),
'MWK' => array('MK', 'Malawian kwacha'),
'MYR' => array('RM', 'Malaysian ringgit'),
'MVR' => array('ރ.', 'Maldivian rufiyaa'),
'MRO' => array('UM', 'Mauritanian ouguiya'),
'MUR' => array('₨', 'Mauritian rupee'),
'MXN' => array('$', 'Mexican peso'),
'MDL' => array('L', 'Moldovan leu'),
'MNT' => array('₮', 'Mongolian tögrög'),
'MAD' => array('د.م.', 'Moroccan dirham'),
'MZN' => array('MTn', 'Mozambican metical'),
'MMK' => array('K', 'Myanma kyat'),
'NAD' => array('$', 'Namibian dollar'),
'NPR' => array('₨', 'Nepalese rupee'),
'ANG' => array('ƒ', 'Netherlands Antillean guilder'),
'TWD' => array('$', 'New Taiwan dollar'),
'NZD' => array('$', 'New Zealand dollar'),
'NIO' => array('C$', 'Nicaraguan córdoba'),
'NGN' => array('₦', 'Nigerian naira'),
'KPW' => array('₩', 'North Korean won'),
'NOK' => array('kr', 'Norwegian krone'),
'OMR' => array('ر.ع.', 'Omani rial'),
'PKR' => array('₨', 'Pakistani rupee'),
'PAB' => array('B/.', 'Panamanian balboa'),
'PGK' => array('K', 'Papua New Guinean kina'),
'PYG' => array('₲', 'Paraguayan guaraní'),
'PEN' => array('S/.', 'Peruvian nuevo sol'),
'PHP' => array('₱', 'Philippine peso'),
'PLN' => array('zł', 'Polish złoty'),
'QAR' => array('ر.ق', 'Qatari riyal'),
'RON' => array('L', 'Romanian leu'),
'RUB' => array('руб.', 'Russian ruble'),
'RWF' => array('Fr', 'Rwandan franc'),
'SHP' => array('£', 'Saint Helena pound'),
'SVC' => array('₡', 'Salvadoran colón'),
'WST' => array('T', 'Samoan tala'),
'STD' => array('Db', 'São Tomé and Príncipe dobra'),
'SAR' => array('ر.س', 'Saudi riyal'),
'RSD' => array('дин. or din.', 'Serbian dinar'),
'SCR' => array('₨', 'Seychellois rupee'),
'SLL' => array('Le', 'Sierra Leonean leone'),
'SGD' => array('$', 'Singapore dollar'),
'SBD' => array('$', 'Solomon Islands dollar'),
'SOS' => array('Sh', 'Somali shilling'),
'ZAR' => array('R', 'South African rand'),
'KRW' => array('₩', 'South Korean won'),
'LKR' => array('Rs', 'Sri Lankan rupee'),
'SDG' => array('جنيه سوداني', 'Sudanese pound'),
'SRD' => array('$', 'Surinamese dollar'),
'SZL' => array('L', 'Swazi lilangeni'),
'SEK' => array('kr', 'Swedish krona'),
// 'CHF' => array('Fr', 'Swiss franc'),
'SYP' => array('ل.س', 'Syrian pound'),
'TJS' => array('ЅМ', 'Tajikistani somoni'),
'TZS' => array('Sh', 'Tanzanian shilling'),
'THB' => array('฿', 'Thai baht'),
'TOP' => array('T$', 'Tongan paʻanga'),
'TTD' => array('$', 'Trinidad and Tobago dollar'),
'TND' => array('د.ت', 'Tunisian dinar'),
'TRY' => array('TL', 'Turkish lira'),
'TMM' => array('m', 'Turkmenistani manat'),
'UGX' => array('Sh', 'Ugandan shilling'),
'UAH' => array('₴', 'Ukrainian hryvnia'),
'AED' => array('د.إ', 'United Arab Emirates dirham'),
// 'USD' => array('$', 'United States dollar'),
'UYU' => array('$', 'Uruguayan peso'),
'UZS' => array('лв', 'Uzbekistani som'),
'VUV' => array('Vt', 'Vanuatu vatu'),
'VEF' => array('Bs F', 'Venezuelan bolívar'),
'VND' => array('₫', 'Vietnamese đồng'),
'XOF' => array('Fr', 'West African CFA franc'),
'YER' => array('﷼', 'Yemeni rial'),
'ZMW' => array('ZK', 'Zambian kwacha'),
'ZWL' => array('$', 'Zimbabwean dollar'),
);
}

63
www/analytics/core/DataFiles/LanguageToCountry.php Normal file
View file

@ -0,0 +1,63 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
/**
* Language to Country mapping
*
* This list is used to guess the visitor's country when the region is
* not specified in the first language tag. Inclusion/exclusion is
* based on Piwik.org visitor statistics and probability of disambiguation.
* (Notably, "en" and "zh" are excluded.)
*
* If you want to add a new entry, please email us at hello at piwik.org
*/
if (!isset($GLOBALS['Piwik_LanguageToCountry'])) {
$GLOBALS['Piwik_LanguageToCountry'] = array(
'bg' => 'bg', // Bulgarian => Bulgaria
'ca' => 'es', // Catalan => Spain
'cs' => 'cz', // Czech => Czech Republic
'da' => 'dk', // Danish => Denmark
'de' => 'de', // German => Germany
'el' => 'gr', // Greek => Greece
'es' => 'es', // Spanish => Spain
'et' => 'ee', // Estonian => Estonia
'fa' => 'ir', // Farsi => Iran
'fi' => 'fi', // Finnish => Finland
'fr' => 'fr', // French => France
'he' => 'il', // Hebrew => Israel
'hr' => 'hr', // Croatian => Croatia
'hu' => 'hu', // Hungarian => Hungary
'id' => 'id', // Indonesian => Indonesia
'is' => 'is', // Icelandic => Iceland
'it' => 'it', // Italian => Italy
'ja' => 'jp', // Japanese => Japan
'ko' => 'kr', // Korean => South Korea
'lt' => 'lt', // Lithuanian => Lithuania
'lv' => 'lv', // Latvian => Latvia
'mk' => 'mk', // Macedonian => Macedonia
'ms' => 'my', // Malay => Malaysia
'nb' => 'no', // Bokmål => Norway
'nl' => 'nl', // Dutch => Netherlands
'nn' => 'no', // Nynorsk => Norway
'no' => 'no', // Norwegian => Norway
'pl' => 'pl', // Polish => Poland
'pt' => 'pt', // Portugese => Portugal
'ro' => 'ro', // Romanian => Romania
'ru' => 'ru', // Russian => Russia
'sk' => 'sk', // Slovak => Slovakia
'sl' => 'si', // Slovene => Slovenia
'sq' => 'al', // Albanian => Albania
'sr' => 'rs', // Serbian => Serbia
'sv' => 'se', // Swedish => Sweden
'th' => 'th', // Thai => Thailand
'bo' => 'ti', // Tibetan => Tibet
'tr' => 'tr', // Turkish => Turkey
'uk' => 'ua', // Ukrainian => Ukraine
);
}

203
www/analytics/core/DataFiles/Languages.php Normal file
View file

@ -0,0 +1,203 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
/*
* Language database
*/
if (!isset($GLOBALS['Piwik_LanguageList'])) {
// Reference: ISO 639-1 alpha-2
$GLOBALS['Piwik_LanguageList'] = array(
'aa' => array('Afar'),
'ab' => array('Abkhazian'),
'ae' => array('Avestan'),
'af' => array('Afrikaans'),
'ak' => array('Akan'),
'am' => array('Amharic'),
'an' => array('Aragonese'),
'ar' => array('Arabic'),
'as' => array('Assamese'),
'av' => array('Avaric'),
'ay' => array('Aymara'),
'az' => array('Azerbaijani'),
'ba' => array('Bashkir'),
'be' => array('Belarusian'),
'bg' => array('Bulgarian'),
'bh' => array('Bihari'), // 'Bihari languages'
'bi' => array('Bislama'),
'bm' => array('Bambara'),
'bn' => array('Bengali'),
'bo' => array('Tibetan'),
'br' => array('Breton'),
'bs' => array('Bosnian'),
'ca' => array('Catalan', 'Valencian'),
'ce' => array('Chechen'),
'ch' => array('Chamorro'),
'co' => array('Corsican'),
'cr' => array('Cree'),
'cs' => array('Czech'),
'cu' => array('Church Slavic', 'Old Slavonic', 'Church Slavonic', 'Old Bulgarian', 'Old Church Slavonic'),
'cv' => array('Chuvash'),
'cy' => array('Welsh'),
'da' => array('Danish'),
'de' => array('German'),
'dv' => array('Divehi', 'Dhivehi', 'Maldivian'),
'dz' => array('Dzongkha'),
'ee' => array('Ewe'),
'el' => array('Greek', 'Modern Greek', 'Hellenic'), // Greek, Modern (1453-)
'en' => array('English'),
'eo' => array('Esperanto'),
'es' => array('Spanish', 'Castilian'),
'et' => array('Estonian'),
'eu' => array('Basque'),
'fa' => array('Persian'),
'ff' => array('Fulah'),
'fi' => array('Finnish'),
'fj' => array('Fijian'),
'fo' => array('Faroese'),
'fr' => array('French'),
'fy' => array('Western Frisian'),
'ga' => array('Irish'),
'gd' => array('Gaelic', 'Scottish Gaelic'),
'gl' => array('Galician'),
'gn' => array('Guarani'),
'gu' => array('Gujarati'),
'gv' => array('Manx'),
'ha' => array('Hausa'),
'he' => array('Hebrew'),
'hi' => array('Hindi'),
'ho' => array('Hiri Motu'),
'hr' => array('Croatian'),
'ht' => array('Haitian', 'Haitian Creole'),
'hu' => array('Hungarian'),
'hy' => array('Armenian'),
'hz' => array('Herero'),
'ia' => array('Interlingua'), // 'Interlingua (International Auxiliary Language Association)'
'id' => array('Indonesian'),
'ie' => array('Interlingue', 'Occidental'),
'ig' => array('Igbo'),
'ii' => array('Sichuan Yi', 'Nuosu'),
'ik' => array('Inupiaq'),
'io' => array('Ido'),
'is' => array('Icelandic'),
'it' => array('Italian'),
'iu' => array('Inuktitut'),
'ja' => array('Japanese'),
'jv' => array('Javanese'),
'ka' => array('Georgian'),
'kg' => array('Kongo'),
'ki' => array('Kikuyu', 'Gikuyu'),
'kj' => array('Kuanyama', 'Kwanyama'),
'kk' => array('Kazakh'),
'kl' => array('Kalaallisut', 'Greenlandic'),
'km' => array('Central Khmer'),
'kn' => array('Kannada'),
'ko' => array('Korean'),
'kr' => array('Kanuri'),
'ks' => array('Kashmiri'),
'ku' => array('Kurdish'),
'kv' => array('Komi'),
'kw' => array('Cornish'),
'ky' => array('Kirghiz', 'Kyrgyz'),
'la' => array('Latin'),
'lb' => array('Luxembourgish', 'Letzeburgesch'),
'lg' => array('Ganda'),
'li' => array('Limburgan', 'Limburger', 'Limburgish'),
'ln' => array('Lingala'),
'lo' => array('Lao'),
'lt' => array('Lithuanian'),
'lu' => array('Luba-Katanga'),
'lv' => array('Latvian'),
'mg' => array('Malagasy'),
'mh' => array('Marshallese'),
'mi' => array('Maori'),
'mk' => array('Macedonian'),
'ml' => array('Malayalam'),
'mn' => array('Mongolian'),
// 'mo' => array('Moldavian'), // deprecated
'mr' => array('Marathi'),
'ms' => array('Malay'),
'mt' => array('Maltese'),
'my' => array('Burmese'),
'na' => array('Nauru'),
'nb' => array('Norwegian Bokmål'),
'nd' => array('North Ndebele'),
'ne' => array('Nepali'),
'ng' => array('Ndonga'),
'nl' => array('Dutch', 'Flemish'),
'nn' => array('Norwegian Nynorsk'),
'no' => array('Norwegian'),
'nr' => array('South Ndebele'),
'nv' => array('Navajo', 'Navaho'),
'ny' => array('Chichewa', 'Chewa', 'Nyanja'),
'oc' => array('Occitan', 'Provençal'), // Occitan (post 1500)
'oj' => array('Ojibwa'),
'om' => array('Oromo'),
'or' => array('Oriya'),
'os' => array('Ossetian', 'Ossetic'),
'pa' => array('Panjabi', 'Punjabi'),
'pi' => array('Pali'),
'pl' => array('Polish'),
'ps' => array('Pushto', 'Pashto'),
'pt' => array('Portuguese'),
'qu' => array('Quechua'),
'rm' => array('Romansh'),
'rn' => array('Rundi'),
'ro' => array('Romanian', 'Moldavian', 'Moldovan'),
'ru' => array('Russian'),
'rw' => array('Kinyarwanda'),
'sa' => array('Sanskrit'),
'sc' => array('Sardinian'),
'sd' => array('Sindhi'),
'se' => array('Northern Sami'),
'sg' => array('Sango'),
// 'sh' => array('Serbo-Croatian'), // deprecated
'si' => array('Sinhala', 'Sinhalese'),
'sk' => array('Slovak'),
'sl' => array('Slovenian'),
'sm' => array('Samoan'),
'sn' => array('Shona'),
'so' => array('Somali'),
'sq' => array('Albanian'),
'sr' => array('Serbian'),
'ss' => array('Swati'),
'st' => array('Southern Soth'),
'su' => array('Sundanese'),
'sv' => array('Swedish'),
'sw' => array('Swahili'),
'ta' => array('Tamil'),
'te' => array('Telugu'),
'tg' => array('Tajik'),
'th' => array('Thai'),
'ti' => array('Tigrinya'),
'tk' => array('Turkmen'),
'tl' => array('Tagalog'),
'tn' => array('Tswana'),
'to' => array('Tonga'), // Tonga (Tonga Islands)
'tr' => array('Turkish'),
'ts' => array('Tsonga'),
'tt' => array('Tatar'),
'tw' => array('Twi'),
'ty' => array('Tahitian'),
'ug' => array('Uighur', 'Uyghur'),
'uk' => array('Ukrainian'),
'ur' => array('Urdu'),
'uz' => array('Uzbek'),
've' => array('Venda'),
'vi' => array('Vietnamese'),
'vo' => array('Volapük'),
'wa' => array('Walloon'),
'wo' => array('Wolof'),
'xh' => array('Xhosa'),
'yi' => array('Yiddish'),
'yo' => array('Yoruba'),
'za' => array('Zhuang', 'Chuang'),
'zh' => array('Chinese'),
'zu' => array('Zulu'),
);
}

48
www/analytics/core/DataFiles/Providers.php Normal file
View file

@ -0,0 +1,48 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
/**
* Providers names
*/
if (!isset($GLOBALS['Piwik_ProviderNames'])) {
$GLOBALS['Piwik_ProviderNames'] = array(
// France
"wanadoo" => "Orange",
"proxad" => "Free",
"bbox" => "Bouygues Telecom",
"bouyguestelecom" => "Bouygues Telecom",
"coucou-networks" => "Free Mobile",
"sfr" => "SFR", //Acronym, keep in uppercase
"univ-metz" => "Université de Lorraine",
"unilim" => "Université de Limoges",
"univ-paris5" => "Université Paris Descartes",
// US
"rr" => "Time Warner Cable Internet", // Not sure
"uu" => "Verizon",
// UK
'zen.net' => 'Zen Internet',
// DE
't-ipconnect' => 'Deutsche Telekom',
't-dialin' => 'Deutsche Telekom',
'dtag' => 'Deutsche Telekom',
't-ipnet' => 'Deutsche Telekom',
'd1-online' => 'Deutsche Telekom (Mobile)',
'superkabel' => 'Kabel Deutschland',
'unitymediagroup' => 'Unitymedia',
'arcor-ip' => 'Vodafone',
'kabel-badenwuerttemberg' => 'Kabel BW',
'alicedsl' => 'O2',
'komdsl' => 'komDSL - Thüga MeteringService',
'mediaways' => 'mediaWays - Telefonica',
'citeq' => 'Citeq - Stadt Münster',
);
}

1033
www/analytics/core/DataFiles/SearchEngines.php Normal file
View file

File diff suppressed because it is too large Load diff

226
www/analytics/core/DataFiles/Socials.php Executable file
View file

@ -0,0 +1,226 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
if (!isset($GLOBALS['Piwik_socialUrl'])) {
// Note: the key of the array should have max 3 elements eg. sub.domain.ext
$GLOBALS['Piwik_socialUrl'] = array(
// Facebook
'facebook.com' => 'Facebook',
'fb.me' => 'Facebook',
// Ozone
'qzone.qq.com' => 'Qzone',
// Haboo
'habbo.com' => 'Haboo',
// Twitter
'twitter.com' => 'Twitter',
't.co' => 'Twitter',
// Renren
'renren.com' => 'Renren',
// Windows Live Spaces
'login.live.com' => 'Windows Live Spaces',
// LinkedIn
'linkedin.com' => 'LinkedIn',
// Bebo
'bebo.com' => 'Bebo',
// Vkontakte
'vk.com' => 'Vkontakte',
'vkontakte.ru' => 'Vkontakte',
// Tagged
'login.tagged.com' => 'Tagged',
// Orkut
'orkut.com' => 'Orkut',
// Myspace
'myspace.com' => 'Myspace',
// Frinedster
'friendster.com' => 'Friendster',
// Badoo
'badoo.com' => 'Badoo',
// hi5
'hi5.com' => 'hi5',
// Netlog
'netlog.com' => 'Netlog',
// Flixster
'flixster.com' => 'Flixster',
// MyLife
'mylife.ru' => 'MyLife',
// Classmates.com
'classmates.com' => 'Classmates.com',
// Github
'github.com' => 'Github',
// Google+
'url.google.com' => 'Google%2B',
// douban
'douban.com' => 'douban',
// Odnoklassniki
'odnoklassniki.ru' => 'Odnoklassniki',
// Viadeo
'viadeo.com' => 'Viadeo',
// Flickr
'flickr.com' => 'Flickr',
// WeeWorld
'weeworld.com' => 'WeeWorld',
// Last.fm
'last.fm' => 'Last.fm',
'lastfm.ru' => 'Last.fm',
'lastfm.de' => 'Last.fm',
'lastfm.es' => 'Last.fm',
'lastfm.fr' => 'Last.fm',
'lastfm.it' => 'Last.fm',
'lastfm.jp' => 'Last.fm',
'lastfm.pl' => 'Last.fm',
'lastfm.com.br' => 'Last.fm',
'lastfm.se' => 'Last.fm',
'lastfm.com.tr' => 'Last.fm',
// MyHeritage
'myheritage.com' => 'MyHeritage',
// Xanga
'xanga.com' => 'Xanga',
// Mixi
'mixi.jp' => 'Mixi',
// Cyworld
'global.cyworld.com' => 'Cyworld',
// Gaia Online
'gaiaonline.com' => 'Gaia Online',
// Skyrock
'skyrock.com' => 'Skyrock',
// BlackPlanet
'blackplanet.com' => 'BlackPlanet',
// myYearbook
'myyearbook.com' => 'myYearbook',
// Fotolog
'fotolog.com' => 'Fotolog',
// Friends Reunited
'friendsreunited.com' => 'Friends Reunited',
// LiveJournal
'livejournal.ru' => 'LiveJournal',
'livejournal.com' => 'LiveJournal',
// StudiVZ/MeinVZ
'studivz.net' => 'StudiVZ',
'meinvz.net' => 'MeinVZ',
// StackOverflow
'stackoverflow.com' => 'StackOverflow',
// Sonico.com
'sonico.com' => 'Sonico.com',
// Pinterest
'pinterest.com' => 'Pinterest',
// Plaxo
'plaxo.com' => 'Plaxo',
// Geni.com
'geni.com' => 'Geni.com',
// Tuenti
'tuenti.com' => 'Tuenti',
// XING
'xing.com' => 'XING',
// Taringa!
'taringa.net' => 'Taringa!',
// Nasza-klasa.pl
'nk.pl' => 'Nasza-klasa.pl',
// StumbleUpon
'stumbleupon.com' => 'StumbleUpon',
// Sourceforge
'sourceforge.net' => 'SourceForge',
// Hyves
'hyves.nl' => 'Hyves',
// WAYN
'wayn.com' => 'WAYN',
// Buzznet
'buzznet.com' => 'Buzznet',
// Multiply
'multiply.com' => 'Multiply',
// Foursquare
'foursquare.com' => 'Foursquare',
// vkrugudruzei.ru
'vkrugudruzei.ru' => 'vkrugudruzei.ru',
// my.mail.ru
'my.mail.ru' => 'my.mail.ru',
//MoiKrug.ru
'moikrug.ru' => 'moikrug.ru',
// Reddit
'reddit.com' => 'reddit',
// HackerNews
'news.ycombinator.com' => 'Hacker News',
// Identi.ca
'identi.ca' => 'identi.ca',
// Weibo
'weibo.com' => 'Weibo',
't.cn' => 'Weibo',
// YouTube
'youtube.com' => 'YouTube',
'youtu.be' => 'YouTube',
// Vimeo
'vimeo.com' => 'Vimeo',
//tumblr
'tumblr.com' => 'tumblr',
);
}

1638
www/analytics/core/DataTable.php Normal file
View file

File diff suppressed because it is too large Load diff

81
www/analytics/core/DataTable/BaseFilter.php Normal file
View file

@ -0,0 +1,81 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable;
use Piwik\DataTable;
use Piwik\DataTable\Row;
/**
* A filter is set of logic that manipulates a DataTable. Existing filters do things like,
*
* - remove rows
* - change column values (change string to lowercase, truncate, etc.)
* - add/remove columns or metadata (compute percentage values, add an 'icon' metadata based on the label, etc.)
* - add/remove/edit subtable associated with rows
* - etc.
*
* Filters are called with a DataTable instance and extra parameters that are specified
* in {@link Piwik\DataTable::filter()} and {@link Piwik\DataTable::queueFilter()}.
*
* To see examples of Filters look at the existing ones in the Piwik\DataTable\BaseFilter
* namespace.
*
* @api
*/
abstract class BaseFilter
{
/**
* @var bool
*/
protected $enableRecursive = false;
/**
* Constructor.
*
* @param DataTable $table
*/
public function __construct(DataTable $table)
{
// empty
}
/**
* Manipulates a {@link DataTable} in some way.
*
* @param DataTable $table
*/
abstract public function filter($table);
/**
* Enables/Disables recursive filtering. Whether this property is actually used
* is up to the derived BaseFilter class.
*
* @param bool $enable
*/
public function enableRecursive($enable)
{
$this->enableRecursive = (bool)$enable;
}
/**
* Filters a row's subtable, if one exists and is loaded in memory.
*
* @param Row $row The row whose subtable should be filter.
*/
public function filterSubTable(Row $row)
{
if (!$this->enableRecursive) {
return;
}
if ($row->isSubtableLoaded()) {
$subTable = Manager::getInstance()->getTable($row->getIdSubDataTable());
$this->filter($subTable);
}
}
}

28
www/analytics/core/DataTable/Bridges.php Normal file
View file

@ -0,0 +1,28 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
/**
* This contains the bridge classes which were used prior to Piwik 2.0
* The serialized reports contains these classes below, which were not using namespaces yet
*/
namespace {
use Piwik\DataTable\Row\DataTableSummaryRow;
use Piwik\DataTable\Row;
class Piwik_DataTable_Row_DataTableSummary extends DataTableSummaryRow
{
}
class Piwik_DataTable_Row extends Row
{
}
}

30
www/analytics/core/DataTable/DataTableInterface.php Normal file
View file

@ -0,0 +1,30 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable;
/**
* The DataTable Interface
*
*/
interface DataTableInterface
{
public function getRowsCount();
public function queueFilter($className, $parameters = array());
public function applyQueuedFilters();
public function filter($className, $parameters = array());
public function getFirstRow();
public function __toString();
public function enableRecursiveSort();
public function renameColumn($oldName, $newName);
public function deleteColumns($columns, $deleteRecursiveInSubtables = false);
public function deleteRow($id);
public function deleteColumn($name);
public function getColumn($name);
public function getColumns();
}

149
www/analytics/core/DataTable/Filter/AddColumnsProcessedMetrics.php Normal file
View file

@ -0,0 +1,149 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable\BaseFilter;
use Piwik\DataTable\Row;
use Piwik\DataTable;
use Piwik\Metrics;
/**
* Adds processed metrics columns to a {@link DataTable} using metrics that already exist.
*
* Columns added are:
*
* - **conversion_rate**: percent value of `nb_visits_converted / nb_visits
* - **nb_actions_per_visit**: `nb_actions / nb_visits`
* - **avg_time_on_site**: in number of seconds, `round(visit_length / nb_visits)`. Not
* pretty formatted.
* - **bounce_rate**: percent value of `bounce_count / nb_visits`
*
* Adding the **filter_add_columns_when_show_all_columns** query parameter to
* an API request will trigger the execution of this Filter.
*
* _Note: This filter must be called before {@link ReplaceColumnNames} is called._
*
* **Basic usage example**
*
* $dataTable->filter('AddColumnsProcessedMetrics');
*
* @api
*/
class AddColumnsProcessedMetrics extends BaseFilter
{
protected $invalidDivision = 0;
protected $roundPrecision = 2;
protected $deleteRowsWithNoVisit = true;
/**
* Constructor.
*
* @param DataTable $table The table to eventually filter.
* @param bool $deleteRowsWithNoVisit Whether to delete rows with no visits or not.
*/
public function __construct($table, $deleteRowsWithNoVisit = true)
{
$this->deleteRowsWithNoVisit = $deleteRowsWithNoVisit;
parent::__construct($table);
}
/**
* Adds the processed metrics. See {@link AddColumnsProcessedMetrics} for
* more information.
*
* @param DataTable $table
*/
public function filter($table)
{
$rowsIdToDelete = array();
foreach ($table->getRows() as $key => $row) {
$nbVisits = $this->getColumn($row, Metrics::INDEX_NB_VISITS);
$nbActions = $this->getColumn($row, Metrics::INDEX_NB_ACTIONS);
if ($nbVisits == 0
&& $nbActions == 0
&& $this->deleteRowsWithNoVisit
) {
// case of keyword/website/campaign with a conversion for this day,
// but no visit, we don't show it
$rowsIdToDelete[] = $key;
continue;
}
$nbVisitsConverted = (int)$this->getColumn($row, Metrics::INDEX_NB_VISITS_CONVERTED);
if ($nbVisitsConverted > 0) {
$conversionRate = round(100 * $nbVisitsConverted / $nbVisits, $this->roundPrecision);
try {
$row->addColumn('conversion_rate', $conversionRate . "%");
} catch (\Exception $e) {
// conversion_rate can be defined upstream apparently? FIXME
}
}
if ($nbVisits == 0) {
$actionsPerVisit = $averageTimeOnSite = $bounceRate = $this->invalidDivision;
} else {
// nb_actions / nb_visits => Actions/visit
// sum_visit_length / nb_visits => Avg. Time on Site
// bounce_count / nb_visits => Bounce Rate
$actionsPerVisit = round($nbActions / $nbVisits, $this->roundPrecision);
$visitLength = $this->getColumn($row, Metrics::INDEX_SUM_VISIT_LENGTH);
$averageTimeOnSite = round($visitLength / $nbVisits, $rounding = 0);
$bounceRate = round(100 * $this->getColumn($row, Metrics::INDEX_BOUNCE_COUNT) / $nbVisits, $this->roundPrecision);
}
try {
$row->addColumn('nb_actions_per_visit', $actionsPerVisit);
$row->addColumn('avg_time_on_site', $averageTimeOnSite);
// It could be useful for API users to have raw sum length value.
//$row->addMetadata('sum_visit_length', $visitLength);
} catch (\Exception $e) {
}
try {
$row->addColumn('bounce_rate', $bounceRate . "%");
} catch (\Exception $e) {
}
$this->filterSubTable($row);
}
$table->deleteRows($rowsIdToDelete);
}
/**
* Returns column from a given row.
* Will work with 2 types of datatable
* - raw datatables coming from the archive DB, which columns are int indexed
* - datatables processed resulting of API calls, which columns have human readable english names
*
* @param Row|array $row
* @param int $columnIdRaw see consts in Archive::
* @param bool|array $mappingIdToName
* @return mixed Value of column, false if not found
*/
protected function getColumn($row, $columnIdRaw, $mappingIdToName = false)
{
if (empty($mappingIdToName)) {
$mappingIdToName = Metrics::$mappingFromIdToName;
}
$columnIdReadable = $mappingIdToName[$columnIdRaw];
if ($row instanceof Row) {
$raw = $row->getColumn($columnIdRaw);
if ($raw !== false) {
return $raw;
}
return $row->getColumn($columnIdReadable);
}
if (isset($row[$columnIdRaw])) {
return $row[$columnIdRaw];
}
if (isset($row[$columnIdReadable])) {
return $row[$columnIdReadable];
}
return false;
}
}

226
www/analytics/core/DataTable/Filter/AddColumnsProcessedMetricsGoal.php Normal file
View file

@ -0,0 +1,226 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Exception;
use Piwik\DataTable;
use Piwik\Metrics;
use Piwik\Piwik;
use Piwik\Tracker\GoalManager;
/**
* Adds goal related metrics to a {@link DataTable} using metrics that already exist.
*
* Metrics added are:
* - **revenue_per_visit**: total goal and ecommerce revenue / nb_visits
* - **goal_%idGoal%_conversion_rate**: the conversion rate. There will be one of
* these columns for each goal that exists
* for the site.
* - **goal_%idGoal%_nb_conversions**: the number of conversions. There will be one of
* these columns for each goal that exists
* for the site.
* - **goal_%idGoal%_revenue_per_visit**: goal revenue / nb_visits. There will be one of
* these columns for each goal that exists
* for the site.
* - **goal_%idGoal%_revenue**: goal revenue. There will be one of
* these columns for each goal that exists
* for the site.
* - **goal_%idGoal%_avg_order_revenue**: goal revenue / number of orders or abandoned
* carts. Only for ecommerce order and abandoned cart
* reports.
* - **goal_%idGoal%_items**: number of items. Only for ecommerce order and abandoned cart
* reports.
*
* Adding the **filter_update_columns_when_show_all_goals** query parameter to
* an API request will trigger the execution of this Filter.
*
* _Note: This filter must be called before {@link ReplaceColumnNames} is called._
*
* **Basic usage example**
*
* $dataTable->filter('AddColumnsProcessedMetricsGoal',
* array($enable = true, $idGoal = Piwik::LABEL_ID_GOAL_IS_ECOMMERCE_ORDER));
*
* @api
*/
class AddColumnsProcessedMetricsGoal extends AddColumnsProcessedMetrics
{
/**
* Process main goal metrics: conversion rate, revenue per visit
*/
const GOALS_MINIMAL_REPORT = -2;
/**
* Process main goal metrics, and conversion rate per goal
*/
const GOALS_OVERVIEW = -1;
/**
* Process all goal and per-goal metrics
*/
const GOALS_FULL_TABLE = 0;
/**
* Constructor.
*
* @param DataTable $table The table that will eventually filtered.
* @param bool $enable Always set to true.
* @param string $processOnlyIdGoal Defines what metrics to add (don't process metrics when you don't display them).
* If self::GOALS_FULL_TABLE, all Goal metrics (and per goal metrics) will be processed.
* If self::GOALS_OVERVIEW, only the main goal metrics will be added.
* If an int > 0, then will process only metrics for this specific Goal.
*/
public function __construct($table, $enable = true, $processOnlyIdGoal)
{
$this->processOnlyIdGoal = $processOnlyIdGoal;
$this->isEcommerce = $this->processOnlyIdGoal == Piwik::LABEL_ID_GOAL_IS_ECOMMERCE_ORDER || $this->processOnlyIdGoal == Piwik::LABEL_ID_GOAL_IS_ECOMMERCE_CART;
parent::__construct($table);
// Ensure that all rows with no visit but conversions will be displayed
$this->deleteRowsWithNoVisit = false;
}
/**
* Adds the processed metrics. See {@link AddColumnsProcessedMetrics} for
* more information.
*
* @param DataTable $table
*/
public function filter($table)
{
// Add standard processed metrics
parent::filter($table);
$roundingPrecision = GoalManager::REVENUE_PRECISION;
$expectedColumns = array();
foreach ($table->getRows() as $key => $row) {
$currentColumns = $row->getColumns();
$newColumns = array();
// visits could be undefined when there is a conversion but no visit
$nbVisits = (int)$this->getColumn($row, Metrics::INDEX_NB_VISITS);
$conversions = (int)$this->getColumn($row, Metrics::INDEX_NB_CONVERSIONS);
$goals = $this->getColumn($currentColumns, Metrics::INDEX_GOALS);
if ($goals) {
$revenue = 0;
foreach ($goals as $goalId => $goalMetrics) {
if ($goalId == Piwik::LABEL_ID_GOAL_IS_ECOMMERCE_CART) {
continue;
}
if ($goalId >= GoalManager::IDGOAL_ORDER
|| $goalId == Piwik::LABEL_ID_GOAL_IS_ECOMMERCE_ORDER
) {
$revenue += (int)$this->getColumn($goalMetrics, Metrics::INDEX_GOAL_REVENUE, Metrics::$mappingFromIdToNameGoal);
}
}
if ($revenue == 0) {
$revenue = (int)$this->getColumn($currentColumns, Metrics::INDEX_REVENUE);
}
if (!isset($currentColumns['revenue_per_visit'])) {
// If no visit for this metric, but some conversions, we still want to display some kind of "revenue per visit"
// even though it will actually be in this edge case "Revenue per conversion"
$revenuePerVisit = $this->invalidDivision;
if ($nbVisits > 0
|| $conversions > 0
) {
$revenuePerVisit = round($revenue / ($nbVisits == 0 ? $conversions : $nbVisits), $roundingPrecision);
}
$newColumns['revenue_per_visit'] = $revenuePerVisit;
}
if ($this->processOnlyIdGoal == self::GOALS_MINIMAL_REPORT) {
$row->addColumns($newColumns);
continue;
}
// Display per goal metrics
// - conversion rate
// - conversions
// - revenue per visit
foreach ($goals as $goalId => $goalMetrics) {
$goalId = str_replace("idgoal=", "", $goalId);
if (($this->processOnlyIdGoal > self::GOALS_FULL_TABLE
|| $this->isEcommerce)
&& $this->processOnlyIdGoal != $goalId
) {
continue;
}
$conversions = (int)$this->getColumn($goalMetrics, Metrics::INDEX_GOAL_NB_CONVERSIONS, Metrics::$mappingFromIdToNameGoal);
// Goal Conversion rate
$name = 'goal_' . $goalId . '_conversion_rate';
if ($nbVisits == 0) {
$value = $this->invalidDivision;
} else {
$value = round(100 * $conversions / $nbVisits, $roundingPrecision);
}
$newColumns[$name] = $value . "%";
$expectedColumns[$name] = true;
// When the table is displayed by clicking on the flag icon, we only display the columns
// Visits, Conversions, Per goal conversion rate, Revenue
if ($this->processOnlyIdGoal == self::GOALS_OVERVIEW) {
continue;
}
// Goal Conversions
$name = 'goal_' . $goalId . '_nb_conversions';
$newColumns[$name] = $conversions;
$expectedColumns[$name] = true;
// Goal Revenue per visit
$name = 'goal_' . $goalId . '_revenue_per_visit';
// See comment above for $revenuePerVisit
$goalRevenue = (float)$this->getColumn($goalMetrics, Metrics::INDEX_GOAL_REVENUE, Metrics::$mappingFromIdToNameGoal);
$revenuePerVisit = round($goalRevenue / ($nbVisits == 0 ? $conversions : $nbVisits), $roundingPrecision);
$newColumns[$name] = $revenuePerVisit;
$expectedColumns[$name] = true;
// Total revenue
$name = 'goal_' . $goalId . '_revenue';
$newColumns[$name] = $goalRevenue;
$expectedColumns[$name] = true;
if ($this->isEcommerce) {
// AOV Average Order Value
$name = 'goal_' . $goalId . '_avg_order_revenue';
$newColumns[$name] = $goalRevenue / $conversions;
$expectedColumns[$name] = true;
// Items qty
$name = 'goal_' . $goalId . '_items';
$newColumns[$name] = $this->getColumn($goalMetrics, Metrics::INDEX_GOAL_ECOMMERCE_ITEMS, Metrics::$mappingFromIdToNameGoal);
$expectedColumns[$name] = true;
}
}
}
// conversion_rate can be defined upstream apparently? FIXME
try {
$row->addColumns($newColumns);
} catch (Exception $e) {
}
}
$expectedColumns['revenue_per_visit'] = true;
// make sure all goals values are set, 0 by default
// if no value then sorting would put at the end
$expectedColumns = array_keys($expectedColumns);
$rows = $table->getRows();
foreach ($rows as &$row) {
foreach ($expectedColumns as $name) {
if (false === $row->getColumn($name)) {
$value = 0;
if (strpos($name, 'conversion_rate') !== false) {
$value = '0%';
}
$row->addColumn($name, $value);
}
}
}
}
}

52
www/analytics/core/DataTable/Filter/AddSummaryRow.php Normal file
View file

@ -0,0 +1,52 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable\BaseFilter;
use Piwik\DataTable;
use Piwik\DataTable\Row\DataTableSummaryRow;
/**
* Adds a summary row to {@link DataTable}s that contains the sum of all other table rows.
*
* **Basic usage example**
*
* $dataTable->filter('AddSummaryRow');
*
* // use a human readable label for the summary row (instead of '-1')
* $dataTable->filter('AddSummaryRow', array($labelSummaryRow = Piwik::translate('General_Total')));
*
* @api
*/
class AddSummaryRow extends BaseFilter
{
/**
* Constructor.
*
* @param DataTable $table The table that will be filtered.
* @param int $labelSummaryRow The value of the label column for the new row.
*/
public function __construct($table, $labelSummaryRow = DataTable::LABEL_SUMMARY_ROW)
{
parent::__construct($table);
$this->labelSummaryRow = $labelSummaryRow;
}
/**
* Executes the filter. See {@link AddSummaryRow}.
*
* @param DataTable $table
*/
public function filter($table)
{
$row = new DataTableSummaryRow($table);
$row->setColumn('label', $this->labelSummaryRow);
$table->addSummaryRow($row);
}
}

168
www/analytics/core/DataTable/Filter/BeautifyRangeLabels.php Normal file
View file

@ -0,0 +1,168 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable;
use Piwik\Piwik;
/**
* A {@link DataTable} filter that replaces range label columns with prettier,
* human-friendlier versions.
*
* When reports that summarize data over a set of ranges (such as the
* reports in the **VisitorInterest** plugin) are archived, they are
* archived with labels that read as: '$min-$max' or '$min+'. These labels
* have no units and can look like '1-1'.
*
* This filter can be used to clean up and add units to those range labels. To
* do this, you supply a string to use when the range specifies only
* one unit (ie '1-1') and another format string when the range specifies
* more than one unit (ie '2-2', '3-5' or '6+').
*
* This filter can be extended to vary exactly how ranges are prettified based
* on the range values found in the DataTable. To see an example of this,
* take a look at the {@link BeautifyTimeRangeLabels} filter.
*
* **Basic usage example**
*
* $dataTable->queueFilter('BeautifyRangeLabels', array("1 visit", "%s visits"));
*
* @api
*/
class BeautifyRangeLabels extends ColumnCallbackReplace
{
/**
* The string to use when the range being beautified is between 1-1 units.
* @var string
*/
protected $labelSingular;
/**
* The format string to use when the range being beautified references more than
* one unit.
* @var string
*/
protected $labelPlural;
/**
* Constructor.
*
* @param DataTable $table The DataTable that will be filtered.
* @param string $labelSingular The string to use when the range being beautified
* is equal to '1-1 units', eg `"1 visit"`.
* @param string $labelPlural The string to use when the range being beautified
* references more than one unit. This must be a format
* string that takes one string parameter, eg, `"%s visits"`.
*/
public function __construct($table, $labelSingular, $labelPlural)
{
parent::__construct($table, 'label', array($this, 'beautify'), array());
$this->labelSingular = $labelSingular;
$this->labelPlural = $labelPlural;
}
/**
* Beautifies a range label and returns the pretty result. See {@link BeautifyRangeLabels}.
*
* @param string $value The range string. This must be in either a '$min-$max' format
* a '$min+' format.
* @return string The pretty range label.
*/
public function beautify($value)
{
// if there's more than one element, handle as a range w/ an upper bound
if (strpos($value, "-") !== false) {
// get the range
sscanf($value, "%d - %d", $lowerBound, $upperBound);
// if the lower bound is the same as the upper bound make sure the singular label
// is used
if ($lowerBound == $upperBound) {
return $this->getSingleUnitLabel($value, $lowerBound);
} else {
return $this->getRangeLabel($value, $lowerBound, $upperBound);
}
} // if there's one element, handle as a range w/ no upper bound
else {
// get the lower bound
sscanf($value, "%d", $lowerBound);
if ($lowerBound !== null) {
$plusEncoded = urlencode('+');
$plusLen = strlen($plusEncoded);
$len = strlen($value);
// if the label doesn't end with a '+', append it
if ($len < $plusLen || substr($value, $len - $plusLen) != $plusEncoded) {
$value .= $plusEncoded;
}
return $this->getUnboundedLabel($value, $lowerBound);
} else {
// if no lower bound can be found, this isn't a valid range. in this case
// we assume its a translation key and try to translate it.
return Piwik::translate(trim($value));
}
}
}
/**
* Beautifies and returns a range label whose range spans over one unit, ie
* 1-1, 2-2 or 3-3.
*
* This function can be overridden in derived types to customize beautifcation
* behavior based on the range values.
*
* @param string $oldLabel The original label value.
* @param int $lowerBound The lower bound of the range.
* @return string The pretty range label.
*/
public function getSingleUnitLabel($oldLabel, $lowerBound)
{
if ($lowerBound == 1) {
return $this->labelSingular;
} else {
return sprintf($this->labelPlural, $lowerBound);
}
}
/**
* Beautifies and returns a range label whose range is bounded and spans over
* more than one unit, ie 1-5, 5-10 but NOT 11+.
*
* This function can be overridden in derived types to customize beautifcation
* behavior based on the range values.
*
* @param string $oldLabel The original label value.
* @param int $lowerBound The lower bound of the range.
* @param int $upperBound The upper bound of the range.
* @return string The pretty range label.
*/
public function getRangeLabel($oldLabel, $lowerBound, $upperBound)
{
return sprintf($this->labelPlural, $oldLabel);
}
/**
* Beautifies and returns a range label whose range is unbounded, ie
* 5+, 10+, etc.
*
* This function can be overridden in derived types to customize beautifcation
* behavior based on the range values.
*
* @param string $oldLabel The original label value.
* @param int $lowerBound The lower bound of the range.
* @return string The pretty range label.
*/
public function getUnboundedLabel($oldLabel, $lowerBound)
{
return sprintf($this->labelPlural, $oldLabel);
}
}

121
www/analytics/core/DataTable/Filter/BeautifyTimeRangeLabels.php Normal file
View file

@ -0,0 +1,121 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable;
/**
* A {@link DataTable} filter that replaces range labels whose values are in seconds with
* prettier, human-friendlier versions.
*
* This filter customizes the behavior of the {@link BeautifyRangeLabels} filter
* so range values that are less than one minute are displayed in seconds but
* other ranges are displayed in minutes.
*
* **Basic usage**
*
* $dataTable->filter('BeautifyTimeRangeLabels', array("%1$s-%2$s min", "1 min", "%s min"));
*
* @api
*/
class BeautifyTimeRangeLabels extends BeautifyRangeLabels
{
/**
* A format string used to create pretty range labels when the range's
* lower bound is between 0 and 60.
*
* This format string must take two numeric parameters, one for each
* range bound.
*/
protected $labelSecondsPlural;
/**
* Constructor.
*
* @param DataTable $table The DataTable this filter will run over.
* @param string $labelSecondsPlural A string to use when beautifying range labels
* whose lower bound is between 0 and 60. Must be
* a format string that takes two numeric params.
* @param string $labelMinutesSingular A string to use when replacing a range that
* equals 60-60 (or 1 minute - 1 minute).
* @param string $labelMinutesPlural A string to use when replacing a range that
* spans multiple minutes. This must be a
* format string that takes one string parameter.
*/
public function __construct($table, $labelSecondsPlural, $labelMinutesSingular, $labelMinutesPlural)
{
parent::__construct($table, $labelMinutesSingular, $labelMinutesPlural);
$this->labelSecondsPlural = $labelSecondsPlural;
}
/**
* Beautifies and returns a range label whose range spans over one unit, ie
* 1-1, 2-2 or 3-3.
*
* If the lower bound of the range is less than 60 the pretty range label
* will be in seconds. Otherwise, it will be in minutes.
*
* @param string $oldLabel The original label value.
* @param int $lowerBound The lower bound of the range.
* @return string The pretty range label.
*/
public function getSingleUnitLabel($oldLabel, $lowerBound)
{
if ($lowerBound < 60) {
return sprintf($this->labelSecondsPlural, $lowerBound, $lowerBound);
} else if ($lowerBound == 60) {
return $this->labelSingular;
} else {
return sprintf($this->labelPlural, ceil($lowerBound / 60));
}
}
/**
* Beautifies and returns a range label whose range is bounded and spans over
* more than one unit, ie 1-5, 5-10 but NOT 11+.
*
* If the lower bound of the range is less than 60 the pretty range label
* will be in seconds. Otherwise, it will be in minutes.
*
* @param string $oldLabel The original label value.
* @param int $lowerBound The lower bound of the range.
* @param int $upperBound The upper bound of the range.
* @return string The pretty range label.
*/
public function getRangeLabel($oldLabel, $lowerBound, $upperBound)
{
if ($lowerBound < 60) {
return sprintf($this->labelSecondsPlural, $lowerBound, $upperBound);
} else {
return sprintf($this->labelPlural, ceil($lowerBound / 60) . "-" . ceil($upperBound / 60));
}
}
/**
* Beautifies and returns a range label whose range is unbounded, ie
* 5+, 10+, etc.
*
* If the lower bound of the range is less than 60 the pretty range label
* will be in seconds. Otherwise, it will be in minutes.
*
* @param string $oldLabel The original label value.
* @param int $lowerBound The lower bound of the range.
* @return string The pretty range label.
*/
public function getUnboundedLabel($oldLabel, $lowerBound)
{
if ($lowerBound < 60) {
return sprintf($this->labelSecondsPlural, $lowerBound);
} else {
// since we're using minutes, we use floor so 1801s+ will be 30m+ and not 31m+
return sprintf($this->labelPlural, "" . floor($lowerBound / 60) . urlencode('+'));
}
}
}

187
www/analytics/core/DataTable/Filter/CalculateEvolutionFilter.php Executable file
View file

@ -0,0 +1,187 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable;
use Piwik\DataTable\Row;
use Piwik\Site;
/**
* A {@link DataTable} filter that calculates the evolution of a metric and adds
* it to each row as a percentage.
*
* **This filter cannot be used as an argument to {@link Piwik\DataTable::filter()}** since
* it requires corresponding data from another DataTable. Instead,
* you must manually perform a binary filter (see the **MultiSites** API for an
* example).
*
* The evolution metric is calculated as:
*
* ((currentValue - pastValue) / pastValue) * 100
*
* @api
*/
class CalculateEvolutionFilter extends ColumnCallbackAddColumnPercentage
{
/**
* The the DataTable that contains past data.
*
* @var DataTable
*/
protected $pastDataTable;
/**
* Tells if column being added is the revenue evolution column.
*/
protected $isRevenueEvolution = null;
/**
* Constructor.
*
* @param DataTable $table The DataTable being filtered.
* @param DataTable $pastDataTable The DataTable containing data for the period in the past.
* @param string $columnToAdd The column to add evolution data to, eg, `'visits_evolution'`.
* @param string $columnToRead The column to use to calculate evolution data, eg, `'nb_visits'`.
* @param int $quotientPrecision The precision to use when rounding the evolution value.
*/
public function __construct($table, $pastDataTable, $columnToAdd, $columnToRead, $quotientPrecision = 0)
{
parent::__construct(
$table, $columnToAdd, $columnToRead, $columnToRead, $quotientPrecision, $shouldSkipRows = true);
$this->pastDataTable = $pastDataTable;
$this->isRevenueEvolution = $columnToAdd == 'revenue_evolution';
}
/**
* Returns the difference between the column in the specific row and its
* sister column in the past DataTable.
*
* @param Row $row
* @return int|float
*/
protected function getDividend($row)
{
$currentValue = $row->getColumn($this->columnValueToRead);
// if the site this is for doesn't support ecommerce & this is for the revenue_evolution column,
// we don't add the new column
if ($currentValue === false
&& $this->isRevenueEvolution
&& !Site::isEcommerceEnabledFor($row->getColumn('label'))
) {
return false;
}
$pastRow = $this->getPastRowFromCurrent($row);
if ($pastRow) {
$pastValue = $pastRow->getColumn($this->columnValueToRead);
} else {
$pastValue = 0;
}
return $currentValue - $pastValue;
}
/**
* Returns the value of the column in $row's sister row in the past
* DataTable.
*
* @param Row $row
* @return int|float
*/
protected function getDivisor($row)
{
$pastRow = $this->getPastRowFromCurrent($row);
if (!$pastRow) return 0;
return $pastRow->getColumn($this->columnNameUsedAsDivisor);
}
/**
* Calculates and formats a quotient based on a divisor and dividend.
*
* Unlike ColumnCallbackAddColumnPercentage's,
* version of this method, this method will return 100% if the past
* value of a metric is 0, and the current value is not 0. For a
* value representative of an evolution, this makes sense.
*
* @param int|float $value The dividend.
* @param int|float $divisor
* @return string
*/
protected function formatValue($value, $divisor)
{
$value = self::getPercentageValue($value, $divisor, $this->quotientPrecision);
$value = self::appendPercentSign($value);
return $value;
}
/**
* Utility function. Returns the current row in the past DataTable.
*
* @param Row $row The row in the 'current' DataTable.
* @return bool|Row
*/
protected function getPastRowFromCurrent($row)
{
return $this->pastDataTable->getRowFromLabel($row->getColumn('label'));
}
/**
* Calculates the evolution percentage for two arbitrary values.
*
* @param float|int $currentValue The current metric value.
* @param float|int $pastValue The value of the metric in the past. We measure the % change
* from this value to $currentValue.
* @param float|int $quotientPrecision The quotient precision to round to.
* @param bool $appendPercentSign Whether to append a '%' sign to the end of the number or not.
*
* @return string The evolution percent, eg `'15%'`.
*/
public static function calculate($currentValue, $pastValue, $quotientPrecision = 0, $appendPercentSign = true)
{
$number = self::getPercentageValue($currentValue - $pastValue, $pastValue, $quotientPrecision);
if ($appendPercentSign) {
$number = self::appendPercentSign($number);
}
return $number;
}
public static function appendPercentSign($number)
{
return $number . '%';
}
public static function prependPlusSignToNumber($number)
{
if ($number > 0) {
$number = '+' . $number;
}
return $number;
}
/**
* Returns an evolution percent based on a value & divisor.
*/
private static function getPercentageValue($value, $divisor, $quotientPrecision)
{
if ($value == 0) {
$evolution = 0;
} elseif ($divisor == 0) {
$evolution = 100;
} else {
$evolution = ($value / $divisor) * 100;
}
$evolution = round($evolution, $quotientPrecision);
return $evolution;
}
}

96
www/analytics/core/DataTable/Filter/ColumnCallbackAddColumn.php Executable file
View file

@ -0,0 +1,96 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable;
use Piwik\DataTable\BaseFilter;
/**
* Adds a new column to every row of a {@link DataTable} based on the result of callback.
*
* **Basic usage example**
*
* $callback = function ($visits, $timeSpent) {
* return round($timeSpent / $visits, 2);
* };
*
* $dataTable->filter('ColumnCallbackAddColumn', array(array('nb_visits', 'sum_time_spent'), 'avg_time_on_site', $callback));
*
* @api
*/
class ColumnCallbackAddColumn extends BaseFilter
{
/**
* The names of the columns to pass to the callback.
*/
private $columns;
/**
* The name of the column to add.
*/
private $columnToAdd;
/**
* The callback to apply to each row of the DataTable. The result is added as
* the value of a new column.
*/
private $functionToApply;
/**
* Extra parameters to pass to the callback.
*/
private $functionParameters;
/**
* Constructor.
*
* @param DataTable $table The DataTable that will be filtered.
* @param array|string $columns The names of the columns to pass to the callback.
* @param string $columnToAdd The name of the column to add.
* @param callable $functionToApply The callback to apply to each row of a DataTable. The columns
* specified in `$columns` are passed to this callback.
* @param array $functionParameters deprecated - use an [anonymous function](http://php.net/manual/en/functions.anonymous.php)
* instead.
*/
public function __construct($table, $columns, $columnToAdd, $functionToApply, $functionParameters = array())
{
parent::__construct($table);
if (!is_array($columns)) {
$columns = array($columns);
}
$this->columns = $columns;
$this->columnToAdd = $columnToAdd;
$this->functionToApply = $functionToApply;
$this->functionParameters = $functionParameters;
}
/**
* See {@link ColumnCallbackAddColumn}.
*
* @param DataTable $table The table to filter.
*/
public function filter($table)
{
foreach ($table->getRows() as $row) {
$columnValues = array();
foreach ($this->columns as $column) {
$columnValues[] = $row->getColumn($column);
}
$parameters = array_merge($columnValues, $this->functionParameters);
$value = call_user_func_array($this->functionToApply, $parameters);
$row->setColumn($this->columnToAdd, $value);
$this->filterSubTable($row);
}
}
}

39
www/analytics/core/DataTable/Filter/ColumnCallbackAddColumnPercentage.php Normal file
View file

@ -0,0 +1,39 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\Piwik;
/**
* Calculates a percentage value for each row of a {@link DataTable} and adds the result
* to each row.
*
* See {@link ColumnCallbackAddColumnQuotient} for more information.
*
* **Basic usage example**
*
* $nbVisits = // ... get the visits for a period ...
* $dataTable->queueFilter('ColumnCallbackAddColumnPercentage', array('nb_visits', 'nb_visits_percentage', $nbVisits, 1));
*
* @api
*/
class ColumnCallbackAddColumnPercentage extends ColumnCallbackAddColumnQuotient
{
/**
* Formats the given value as a percentage.
*
* @param number $value
* @param number $divisor
* @return string
*/
protected function formatValue($value, $divisor)
{
return Piwik::getPercentageSafe($value, $divisor, $this->quotientPrecision) . '%';
}
}

145
www/analytics/core/DataTable/Filter/ColumnCallbackAddColumnQuotient.php Normal file
View file

@ -0,0 +1,145 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable\BaseFilter;
use Piwik\DataTable;
use Piwik\DataTable\Row;
/**
* Calculates the quotient of two columns and adds the result as a new column
* for each row of a DataTable.
*
* This filter is used to calculate rate values (eg, `'bounce_rate'`), averages
* (eg, `'avg_time_on_page'`) and other types of values.
*
* **Basic usage example**
*
* $dataTable->queueFilter('ColumnCallbackAddColumnQuotient', array('bounce_rate', 'bounce_count', 'nb_visits', $precision = 2));
*
* @api
*/
class ColumnCallbackAddColumnQuotient extends BaseFilter
{
protected $table;
protected $columnValueToRead;
protected $columnNameToAdd;
protected $columnNameUsedAsDivisor;
protected $totalValueUsedAsDivisor;
protected $quotientPrecision;
protected $shouldSkipRows;
protected $getDivisorFromSummaryRow;
/**
* Constructor.
*
* @param DataTable $table The DataTable that will eventually be filtered.
* @param string $columnNameToAdd The name of the column to add the quotient value to.
* @param string $columnValueToRead The name of the column that holds the dividend.
* @param number|string $divisorValueOrDivisorColumnName
* Either numeric value to use as the divisor for every row,
* or the name of the column whose value should be used as the
* divisor.
* @param int $quotientPrecision The precision to use when rounding the quotient.
* @param bool|number $shouldSkipRows Whether rows w/o the column to read should be skipped or not.
* @param bool $getDivisorFromSummaryRow Whether to get the divisor from the summary row or the current
* row iteration.
*/
public function __construct($table, $columnNameToAdd, $columnValueToRead, $divisorValueOrDivisorColumnName,
$quotientPrecision = 0, $shouldSkipRows = false, $getDivisorFromSummaryRow = false)
{
parent::__construct($table);
$this->table = $table;
$this->columnValueToRead = $columnValueToRead;
$this->columnNameToAdd = $columnNameToAdd;
if (is_numeric($divisorValueOrDivisorColumnName)) {
$this->totalValueUsedAsDivisor = $divisorValueOrDivisorColumnName;
} else {
$this->columnNameUsedAsDivisor = $divisorValueOrDivisorColumnName;
}
$this->quotientPrecision = $quotientPrecision;
$this->shouldSkipRows = $shouldSkipRows;
$this->getDivisorFromSummaryRow = $getDivisorFromSummaryRow;
}
/**
* See {@link ColumnCallbackAddColumnQuotient}.
*
* @param DataTable $table
*/
public function filter($table)
{
foreach ($table->getRows() as $key => $row) {
$value = $this->getDividend($row);
if ($value === false && $this->shouldSkipRows) {
continue;
}
// Delete existing column if it exists
$existingValue = $row->getColumn($this->columnNameToAdd);
if ($existingValue !== false) {
continue;
}
$divisor = $this->getDivisor($row);
$formattedValue = $this->formatValue($value, $divisor);
$row->addColumn($this->columnNameToAdd, $formattedValue);
$this->filterSubTable($row);
}
}
/**
* Formats the given value
*
* @param number $value
* @param number $divisor
* @return float|int
*/
protected function formatValue($value, $divisor)
{
$quotient = 0;
if ($divisor > 0 && $value > 0) {
$quotient = round($value / $divisor, $this->quotientPrecision);
}
return $quotient;
}
/**
* Returns the dividend to use when calculating the new column value. Can
* be overridden by descendent classes to customize behavior.
*
* @param Row $row The row being modified.
* @return int|float
*/
protected function getDividend($row)
{
return $row->getColumn($this->columnValueToRead);
}
/**
* Returns the divisor to use when calculating the new column value. Can
* be overridden by descendent classes to customize behavior.
*
* @param Row $row The row being modified.
* @return int|float
*/
protected function getDivisor($row)
{
if (!is_null($this->totalValueUsedAsDivisor)) {
return $this->totalValueUsedAsDivisor;
} else if ($this->getDivisorFromSummaryRow) {
$summaryRow = $this->table->getRowFromId(DataTable::ID_SUMMARY_ROW);
return $summaryRow->getColumn($this->columnNameUsedAsDivisor);
} else {
return $row->getColumn($this->columnNameUsedAsDivisor);
}
}
}

89
www/analytics/core/DataTable/Filter/ColumnCallbackAddMetadata.php Normal file
View file

@ -0,0 +1,89 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable;
use Piwik\DataTable\BaseFilter;
/**
* Executes a callback for each row of a {@link DataTable} and adds the result as a new
* row metadata value.
*
* **Basic usage example**
*
* $dataTable->filter('ColumnCallbackAddMetadata', array('label', 'logo', 'Piwik\Plugins\MyPlugin\getLogoFromLabel'));
*
* @api
*/
class ColumnCallbackAddMetadata extends BaseFilter
{
private $columnsToRead;
private $functionToApply;
private $functionParameters;
private $metadataToAdd;
private $applyToSummaryRow;
/**
* Constructor.
*
* @param DataTable $table The DataTable instance that will be filtered.
* @param string|array $columnsToRead The columns to read from each row and pass on to the callback.
* @param string $metadataToAdd The name of the metadata field that will be added to each row.
* @param callable $functionToApply The callback to apply for each row.
* @param array $functionParameters deprecated - use an [anonymous function](http://php.net/manual/en/functions.anonymous.php)
* instead.
* @param bool $applyToSummaryRow Whether the callback should be applied to the summary row or not.
*/
public function __construct($table, $columnsToRead, $metadataToAdd, $functionToApply = null,
$functionParameters = null, $applyToSummaryRow = true)
{
parent::__construct($table);
if (!is_array($columnsToRead)) {
$columnsToRead = array($columnsToRead);
}
$this->columnsToRead = $columnsToRead;
$this->functionToApply = $functionToApply;
$this->functionParameters = $functionParameters;
$this->metadataToAdd = $metadataToAdd;
$this->applyToSummaryRow = $applyToSummaryRow;
}
/**
* See {@link ColumnCallbackAddMetadata}.
*
* @param DataTable $table
*/
public function filter($table)
{
foreach ($table->getRows() as $key => $row) {
if (!$this->applyToSummaryRow && $key == DataTable::ID_SUMMARY_ROW) {
continue;
}
$parameters = array();
foreach ($this->columnsToRead as $columnsToRead) {
$parameters[] = $row->getColumn($columnsToRead);
}
if (!is_null($this->functionParameters)) {
$parameters = array_merge($parameters, $this->functionParameters);
}
if (!is_null($this->functionToApply)) {
$newValue = call_user_func_array($this->functionToApply, $parameters);
} else {
$newValue = $parameters[0];
}
if ($newValue !== false) {
$row->addMetadata($this->metadataToAdd, $newValue);
}
}
}
}

81
www/analytics/core/DataTable/Filter/ColumnCallbackDeleteRow.php Normal file
View file

@ -0,0 +1,81 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable;
use Piwik\DataTable\BaseFilter;
/**
* Deletes all rows for which a callback returns true.
*
* **Basic usage example**
*
* $labelsToRemove = array('label1', 'label2', 'label2');
* $dataTable->filter('ColumnCallbackDeleteRow', array('label', function ($label) use ($labelsToRemove) {
* return in_array($label, $labelsToRemove);
* }));
*
* @api
*/
class ColumnCallbackDeleteRow extends BaseFilter
{
private $columnToFilter;
private $function;
private $functionParams;
/**
* Constructor.
*
* @param DataTable $table The DataTable that will be filtered eventually.
* @param array|string $columnsToFilter The column or array of columns that should be
* passed to the callback.
* @param callback $function The callback that determines whether a row should be deleted
* or not. Should return `true` if the row should be deleted.
* @param array $functionParams deprecated - use an [anonymous function](http://php.net/manual/en/functions.anonymous.php)
* instead.
*/
public function __construct($table, $columnsToFilter, $function, $functionParams = array())
{
parent::__construct($table);
if (!is_array($functionParams)) {
$functionParams = array($functionParams);
}
if (!is_array($columnsToFilter)) {
$columnsToFilter = array($columnsToFilter);
}
$this->function = $function;
$this->columnsToFilter = $columnsToFilter;
$this->functionParams = $functionParams;
}
/**
* Filters the given data table
*
* @param DataTable $table
*/
public function filter($table)
{
foreach ($table->getRows() as $key => $row) {
$params = array();
foreach ($this->columnsToFilter as $column) {
$params[] = $row->getColumn($column);
}
$params = array_merge($params, $this->functionParams);
if (call_user_func_array($this->function, $params) === true) {
$table->deleteRow($key);
}
$this->filterSubTable($row);
}
}
}

122
www/analytics/core/DataTable/Filter/ColumnCallbackReplace.php Normal file
View file

@ -0,0 +1,122 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable\BaseFilter;
use Piwik\DataTable;
use Piwik\DataTable\Row;
/**
* Replaces one or more column values in each row of a DataTable with the results
* of a callback.
*
* **Basic usage example**
*
* $truncateString = function ($value, $truncateLength) {
* if (strlen($value) > $truncateLength) {
* return substr(0, $truncateLength);
* } else {
* return $value;
* }
* };
*
* // label, url and truncate_length are columns in $dataTable
* $dataTable->filter('ColumnCallbackReplace', array('label', 'url'), $truncateString, null, array('truncate_length'));
*
*/
class ColumnCallbackReplace extends BaseFilter
{
private $columnsToFilter;
private $functionToApply;
private $functionParameters;
private $extraColumnParameters;
/**
* Constructor.
*
* @param DataTable $table The DataTable to filter.
* @param array|string $columnsToFilter The columns whose values should be passed to the callback
* and then replaced with the callback's result.
* @param callable $functionToApply The function to execute. Must take the column value as a parameter
* and return a value that will be used to replace the original.
* @param array|null $functionParameters deprecated - use an [anonymous function](http://php.net/manual/en/functions.anonymous.php)
* instead.
* @param array $extraColumnParameters Extra column values that should be passed to the callback, but
* shouldn't be replaced.
*/
public function __construct($table, $columnsToFilter, $functionToApply, $functionParameters = null,
$extraColumnParameters = array())
{
parent::__construct($table);
$this->functionToApply = $functionToApply;
$this->functionParameters = $functionParameters;
if (!is_array($columnsToFilter)) {
$columnsToFilter = array($columnsToFilter);
}
$this->columnsToFilter = $columnsToFilter;
$this->extraColumnParameters = $extraColumnParameters;
}
/**
* See {@link ColumnCallbackReplace}.
*
* @param DataTable $table
*/
public function filter($table)
{
foreach ($table->getRows() as $key => $row) {
$extraColumnParameters = array();
foreach ($this->extraColumnParameters as $columnName) {
$extraColumnParameters[] = $row->getColumn($columnName);
}
foreach ($this->columnsToFilter as $column) {
// when a value is not defined, we set it to zero by default (rather than displaying '-')
$value = $this->getElementToReplace($row, $column);
if ($value === false) {
$value = 0;
}
$parameters = array_merge(array($value), $extraColumnParameters);
if (!is_null($this->functionParameters)) {
$parameters = array_merge($parameters, $this->functionParameters);
}
$newValue = call_user_func_array($this->functionToApply, $parameters);
$this->setElementToReplace($row, $column, $newValue);
$this->filterSubTable($row);
}
}
}
/**
* Replaces the given column within given row with the given value
*
* @param Row $row
* @param string $columnToFilter
* @param mixed $newValue
*/
protected function setElementToReplace($row, $columnToFilter, $newValue)
{
$row->setColumn($columnToFilter, $newValue);
}
/**
* Returns the element that should be replaced
*
* @param Row $row
* @param string $columnToFilter
* @return mixed
*/
protected function getElementToReplace($row, $columnToFilter)
{
return $row->getColumn($columnToFilter);
}
}

150
www/analytics/core/DataTable/Filter/ColumnDelete.php Normal file
View file

@ -0,0 +1,150 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable;
use Piwik\DataTable\BaseFilter;
/**
* Filter that will remove columns from a {@link DataTable} using either a blacklist,
* whitelist or both.
*
* This filter is used to handle the **hideColumn** and **showColumn** query parameters.
*
* **Basic usage example**
*
* $columnsToRemove = array('nb_hits', 'nb_pageviews');
* $dataTable->filter('ColumnDelete', array($columnsToRemove));
*
* $columnsToKeep = array('nb_visits');
* $dataTable->filter('ColumnDelete', array(array(), $columnsToKeep));
*
* @api
*/
class ColumnDelete extends BaseFilter
{
/**
* The columns that should be removed from DataTable rows.
*
* @var array
*/
private $columnsToRemove;
/**
* The columns that should be kept in DataTable rows. All other columns will be
* removed. If a column is in $columnsToRemove and this variable, it will NOT be kept.
*
* @var array
*/
private $columnsToKeep;
/**
* Hack: when specifying "showColumns", sometimes we'd like to also keep columns that "look" like a given column,
* without manually specifying all these columns (which may not be possible if column names are generated dynamically)
*
* Column will be kept, if they match any name in the $columnsToKeep, or if they look like anyColumnToKeep__anythingHere
*/
const APPEND_TO_COLUMN_NAME_TO_KEEP = '__';
/**
* Delete the column, only if the value was zero
*
* @var bool
*/
private $deleteIfZeroOnly;
/**
* Constructor.
*
* @param DataTable $table The DataTable instance that will eventually be filtered.
* @param array|string $columnsToRemove An array of column names or a comma-separated list of
* column names. These columns will be removed.
* @param array|string $columnsToKeep An array of column names that should be kept or a
* comma-separated list of column names. Columns not in
* this list will be removed.
* @param bool $deleteIfZeroOnly If true, columns will be removed only if their value is 0.
*/
public function __construct($table, $columnsToRemove, $columnsToKeep = array(), $deleteIfZeroOnly = false)
{
parent::__construct($table);
if (is_string($columnsToRemove)) {
$columnsToRemove = $columnsToRemove == '' ? array() : explode(',', $columnsToRemove);
}
if (is_string($columnsToKeep)) {
$columnsToKeep = $columnsToKeep == '' ? array() : explode(',', $columnsToKeep);
}
$this->columnsToRemove = $columnsToRemove;
$this->columnsToKeep = array_flip($columnsToKeep); // flip so we can use isset instead of in_array
$this->deleteIfZeroOnly = $deleteIfZeroOnly;
}
/**
* See {@link ColumnDelete}.
*
* @param DataTable $table
*/
public function filter($table)
{
// always do recursive filter
$this->enableRecursive(true);
$recurse = false; // only recurse if there are columns to remove/keep
// remove columns specified in $this->columnsToRemove
if (!empty($this->columnsToRemove)) {
foreach ($table->getRows() as $row) {
foreach ($this->columnsToRemove as $column) {
if ($this->deleteIfZeroOnly) {
$value = $row->getColumn($column);
if ($value === false || !empty($value)) {
continue;
}
}
$row->deleteColumn($column);
}
}
$recurse = true;
}
// remove columns not specified in $columnsToKeep
if (!empty($this->columnsToKeep)) {
foreach ($table->getRows() as $row) {
foreach ($row->getColumns() as $name => $value) {
$keep = false;
// @see self::APPEND_TO_COLUMN_NAME_TO_KEEP
foreach ($this->columnsToKeep as $nameKeep => $true) {
if (strpos($name, $nameKeep . self::APPEND_TO_COLUMN_NAME_TO_KEEP) === 0) {
$keep = true;
}
}
if (!$keep
&& $name != 'label' // label cannot be removed via whitelisting
&& !isset($this->columnsToKeep[$name])
) {
$row->deleteColumn($name);
}
}
}
$recurse = true;
}
// recurse
if ($recurse) {
foreach ($table->getRows() as $row) {
$this->filterSubTable($row);
}
}
}
}

90
www/analytics/core/DataTable/Filter/ExcludeLowPopulation.php Normal file
View file

@ -0,0 +1,90 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable;
use Piwik\DataTable\BaseFilter;
/**
* Deletes all rows for which a specific column has a value that is lower than
* specified minimum threshold value.
*
* **Basic usage examples**
*
* // remove all countries from UserCountry.getCountry that have less than 3 visits
* $dataTable = // ... get a DataTable whose queued filters have been run ...
* $dataTable->filter('ExcludeLowPopulation', array('nb_visits', 3));
*
* // remove all countries from UserCountry.getCountry whose percent of total visits is less than 5%
* $dataTable = // ... get a DataTable whose queued filters have been run ...
* $dataTable->filter('ExcludeLowPopulation', array('nb_visits', false, 0.05));
*
* // remove all countries from UserCountry.getCountry whose bounce rate is less than 10%
* $dataTable = // ... get a DataTable that has a numerical bounce_rate column ...
* $dataTable->filter('ExcludeLowPopulation', array('bounce_rate', 0.10));
*
* @api
*/
class ExcludeLowPopulation extends BaseFilter
{
const MINIMUM_SIGNIFICANT_PERCENTAGE_THRESHOLD = 0.02;
/**
* The minimum value to enforce in a datatable for a specified column. Rows found with
* a value less than this are removed.
*
* @var number
*/
private $minimumValue;
/**
* Constructor.
*
* @param DataTable $table The DataTable that will be filtered eventually.
* @param string $columnToFilter The name of the column whose value will determine whether
* a row is deleted or not.
* @param number|false $minimumValue The minimum column value. Rows with column values <
* this number will be deleted. If false,
* `$minimumPercentageThreshold` is used.
* @param bool|float $minimumPercentageThreshold If supplied, column values must be a greater
* percentage of the sum of all column values than
* this precentage.
*/
public function __construct($table, $columnToFilter, $minimumValue, $minimumPercentageThreshold = false)
{
parent::__construct($table);
$this->columnToFilter = $columnToFilter;
if ($minimumValue == 0) {
if ($minimumPercentageThreshold === false) {
$minimumPercentageThreshold = self::MINIMUM_SIGNIFICANT_PERCENTAGE_THRESHOLD;
}
$allValues = $table->getColumn($this->columnToFilter);
$sumValues = array_sum($allValues);
$minimumValue = $sumValues * $minimumPercentageThreshold;
}
$this->minimumValue = $minimumValue;
}
/**
* See {@link ExcludeLowPopulation}.
*
* @param DataTable $table
*/
public function filter($table)
{
$minimumValue = $this->minimumValue;
$isValueLowPopulation = function ($value) use ($minimumValue) {
return $value < $minimumValue;
};
$table->filter('ColumnCallbackDeleteRow', array($this->columnToFilter, $isValueLowPopulation));
}
}

104
www/analytics/core/DataTable/Filter/GroupBy.php Executable file
View file

@ -0,0 +1,104 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable;
use Piwik\DataTable\BaseFilter;
/**
* DataTable filter that will group {@link DataTable} rows together based on the results
* of a reduce function. Rows with the same reduce result will be summed and merged.
*
* _NOTE: This filter should never be queued, it must be applied directly on a {@link DataTable}._
*
* **Basic usage example**
*
* // group URLs by host
* $dataTable->filter('GroupBy', array('label', function ($labelUrl) {
* return parse_url($labelUrl, PHP_URL_HOST);
* }));
*
* @api
*/
class GroupBy extends BaseFilter
{
/**
* The name of the columns to reduce.
* @var string
*/
private $groupByColumn;
/**
* A callback that modifies the $groupByColumn of each row in some way. Rows with
* the same reduction result will be added together.
*/
private $reduceFunction;
/**
* Extra parameters to pass to the reduce function.
*/
private $parameters;
/**
* Constructor.
*
* @param DataTable $table The DataTable to filter.
* @param string $groupByColumn The column name to reduce.
* @param callable $reduceFunction The reduce function. This must alter the `$groupByColumn`
* columng in some way.
* @param array $parameters deprecated - use an [anonymous function](http://php.net/manual/en/functions.anonymous.php)
* instead.
*/
public function __construct($table, $groupByColumn, $reduceFunction, $parameters = array())
{
parent::__construct($table);
$this->groupByColumn = $groupByColumn;
$this->reduceFunction = $reduceFunction;
$this->parameters = $parameters;
}
/**
* See {@link GroupBy}.
*
* @param DataTable $table
*/
public function filter($table)
{
$groupByRows = array();
$nonGroupByRowIds = array();
foreach ($table->getRows() as $rowId => $row) {
// skip the summary row
if ($rowId == DataTable::ID_SUMMARY_ROW) {
continue;
}
// reduce the group by column of this row
$groupByColumnValue = $row->getColumn($this->groupByColumn);
$parameters = array_merge(array($groupByColumnValue), $this->parameters);
$groupByValue = call_user_func_array($this->reduceFunction, $parameters);
if (!isset($groupByRows[$groupByValue])) {
// if we haven't encountered this group by value before, we mark this row as a
// row to keep, and change the group by column to the reduced value.
$groupByRows[$groupByValue] = $row;
$row->setColumn($this->groupByColumn, $groupByValue);
} else {
// if we have already encountered this group by value, we add this row to the
// row that will be kept, and mark this one for deletion
$groupByRows[$groupByValue]->sumRow($row, $copyMeta = true, $table->getMetadata(DataTable::COLUMN_AGGREGATION_OPS_METADATA_NAME));
$nonGroupByRowIds[] = $rowId;
}
}
// delete the unneeded rows.
$table->deleteRows($nonGroupByRowIds);
}
}

69
www/analytics/core/DataTable/Filter/Limit.php Normal file
View file

@ -0,0 +1,69 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable;
use Piwik\DataTable\BaseFilter;
/**
* Delete all rows from the table that are not in the given [offset, offset+limit) range.
*
* **Basic example usage**
*
* // delete all rows from 5 -> 15
* $dataTable->filter('Limit', array(5, 10));
*
* @api
*/
class Limit extends BaseFilter
{
/**
* Constructor.
*
* @param DataTable $table The DataTable that will be filtered eventually.
* @param int $offset The starting row index to keep.
* @param int $limit Number of rows to keep (specify -1 to keep all rows).
* @param bool $keepSummaryRow Whether to keep the summary row or not.
*/
public function __construct($table, $offset, $limit = -1, $keepSummaryRow = false)
{
parent::__construct($table);
$this->offset = $offset;
$this->limit = $limit;
$this->keepSummaryRow = $keepSummaryRow;
}
/**
* See {@link Limit}.
*
* @param DataTable $table
*/
public function filter($table)
{
$table->setMetadata(DataTable::TOTAL_ROWS_BEFORE_LIMIT_METADATA_NAME, $table->getRowsCount());
if ($this->keepSummaryRow) {
$summaryRow = $table->getRowFromId(DataTable::ID_SUMMARY_ROW);
}
// we delete from 0 to offset
if ($this->offset > 0) {
$table->deleteRowsOffset(0, $this->offset);
}
// at this point the array has offset less elements. We delete from limit to the end
if ($this->limit >= 0) {
$table->deleteRowsOffset($this->limit);
}
if ($this->keepSummaryRow && !empty($summaryRow)) {
$table->addSummaryRow($summaryRow);
}
}
}

81
www/analytics/core/DataTable/Filter/MetadataCallbackAddMetadata.php Normal file
View file

@ -0,0 +1,81 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable;
use Piwik\DataTable\BaseFilter;
/**
* Executes a callback for each row of a {@link DataTable} and adds the result to the
* row as a metadata value. Only metadata values are passed to the callback.
*
* **Basic usage example**
*
* // add a logo metadata based on the url metadata
* $dataTable->filter('MetadataCallbackAddMetadata', array('url', 'logo', 'Piwik\Plugins\MyPlugin\getLogoFromUrl'));
*
* @api
*/
class MetadataCallbackAddMetadata extends BaseFilter
{
private $metadataToRead;
private $functionToApply;
private $metadataToAdd;
private $applyToSummaryRow;
/**
* Constructor.
*
* @param DataTable $table The DataTable that will eventually be filtered.
* @param string|array $metadataToRead The metadata to read from each row and pass to the callback.
* @param string $metadataToAdd The name of the metadata to add.
* @param callable $functionToApply The callback to execute for each row. The result will be
* added as metadata with the name `$metadataToAdd`.
* @param bool $applyToSummaryRow True if the callback should be applied to the summary row, false
* if otherwise.
*/
public function __construct($table, $metadataToRead, $metadataToAdd, $functionToApply,
$applyToSummaryRow = true)
{
parent::__construct($table);
$this->functionToApply = $functionToApply;
if (!is_array($metadataToRead)) {
$metadataToRead = array($metadataToRead);
}
$this->metadataToRead = $metadataToRead;
$this->metadataToAdd = $metadataToAdd;
$this->applyToSummaryRow = $applyToSummaryRow;
}
/**
* See {@link MetadataCallbackAddMetadata}.
*
* @param DataTable $table
*/
public function filter($table)
{
foreach ($table->getRows() as $key => $row) {
if (!$this->applyToSummaryRow && $key == DataTable::ID_SUMMARY_ROW) {
continue;
}
$params = array();
foreach ($this->metadataToRead as $name) {
$params[] = $row->getMetadata($name);
}
$newValue = call_user_func_array($this->functionToApply, $params);
if ($newValue !== false) {
$row->addMetadata($this->metadataToAdd, $newValue);
}
}
}
}

66
www/analytics/core/DataTable/Filter/MetadataCallbackReplace.php Normal file
View file

@ -0,0 +1,66 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable;
use Piwik\DataTable\Row;
/**
* Execute a callback for each row of a {@link DataTable} passing certain column values and metadata
* as metadata, and replaces row metadata with the callback result.
*
* **Basic usage example**
*
* $dataTable->filter('MetadataCallbackReplace', array('url', function ($url) {
* return $url . '#index';
* }));
*
* @api
*/
class MetadataCallbackReplace extends ColumnCallbackReplace
{
/**
* Constructor.
*
* @param DataTable $table The DataTable that will eventually be filtered.
* @param array|string $metadataToFilter The metadata whose values should be passed to the callback
* and then replaced with the callback's result.
* @param callable $functionToApply The function to execute. Must take the metadata value as a parameter
* and return a value that will be used to replace the original.
* @param array|null $functionParameters deprecated - use an [anonymous function](http://php.net/manual/en/functions.anonymous.php)
* instead.
* @param array $extraColumnParameters Extra column values that should be passed to the callback, but
* shouldn't be replaced.
*/
public function __construct($table, $metadataToFilter, $functionToApply, $functionParameters = null,
$extraColumnParameters = array())
{
parent::__construct($table, $metadataToFilter, $functionToApply, $functionParameters, $extraColumnParameters);
}
/**
* @param Row $row
* @param string $metadataToFilter
* @param mixed $newValue
*/
protected function setElementToReplace($row, $metadataToFilter, $newValue)
{
$row->setMetadata($metadataToFilter, $newValue);
}
/**
* @param Row $row
* @param string $metadataToFilter
* @return array|bool|mixed
*/
protected function getElementToReplace($row, $metadataToFilter)
{
return $row->getMetadata($metadataToFilter);
}
}

96
www/analytics/core/DataTable/Filter/Pattern.php Normal file
View file

@ -0,0 +1,96 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable;
use Piwik\DataTable\BaseFilter;
/**
* Deletes every row for which a specific column does not match a supplied regex pattern.
*
* **Example**
*
* // filter out all rows whose labels doesn't start with piwik
* $dataTable->filter('Pattern', array('label', '^piwik'));
*
* @api
*/
class Pattern extends BaseFilter
{
private $columnToFilter;
private $patternToSearch;
private $patternToSearchQuoted;
private $invertedMatch;
/**
* Constructor.
*
* @param DataTable $table The table to eventually filter.
* @param string $columnToFilter The column to match with the `$patternToSearch` pattern.
* @param string $patternToSearch The regex pattern to use.
* @param bool $invertedMatch Whether to invert the pattern or not. If true, will remove
* rows if they match the pattern.
*/
public function __construct($table, $columnToFilter, $patternToSearch, $invertedMatch = false)
{
parent::__construct($table);
$this->patternToSearch = $patternToSearch;
$this->patternToSearchQuoted = self::getPatternQuoted($patternToSearch);
$this->columnToFilter = $columnToFilter;
$this->invertedMatch = $invertedMatch;
}
/**
* Helper method to return the given pattern quoted
*
* @param string $pattern
* @return string
* @ignore
*/
static public function getPatternQuoted($pattern)
{
return '/' . str_replace('/', '\/', $pattern) . '/';
}
/**
* Performs case insensitive match
*
* @param string $patternQuoted
* @param string $string
* @param bool $invertedMatch
* @return int
* @ignore
*/
static public function match($patternQuoted, $string, $invertedMatch = false)
{
return preg_match($patternQuoted . "i", $string) == 1 ^ $invertedMatch;
}
/**
* See {@link Pattern}.
*
* @param DataTable $table
*/
public function filter($table)
{
foreach ($table->getRows() as $key => $row) {
//instead search must handle
// - negative search with -piwik
// - exact match with ""
// see (?!pattern) A subexpression that performs a negative lookahead search, which matches the search string at any point where a string not matching pattern begins.
$value = $row->getColumn($this->columnToFilter);
if ($value === false) {
$value = $row->getMetadata($this->columnToFilter);
}
if (!self::match($this->patternToSearchQuoted, $value, $this->invertedMatch)) {
$table->deleteRow($key);
}
}
}
}

88
www/analytics/core/DataTable/Filter/PatternRecursive.php Normal file
View file

@ -0,0 +1,88 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Exception;
use Piwik\DataTable\BaseFilter;
use Piwik\DataTable;
use Piwik\DataTable\Manager;
/**
* Deletes rows that do not contain a column that matches a regex pattern and do not contain a
* subtable that contains a column that matches a regex pattern.
*
* **Example**
*
* // only display index pageviews in Actions.getPageUrls
* $dataTable->filter('PatternRecursive', array('label', 'index'));
*
* @api
*/
class PatternRecursive extends BaseFilter
{
private $columnToFilter;
private $patternToSearch;
private $patternToSearchQuoted;
/**
* Constructor.
*
* @param DataTable $table The table to eventually filter.
* @param string $columnToFilter The column to match with the `$patternToSearch` pattern.
* @param string $patternToSearch The regex pattern to use.
*/
public function __construct($table, $columnToFilter, $patternToSearch)
{
parent::__construct($table);
$this->patternToSearch = $patternToSearch;
$this->patternToSearchQuoted = Pattern::getPatternQuoted($patternToSearch);
$this->patternToSearch = $patternToSearch; //preg_quote($patternToSearch);
$this->columnToFilter = $columnToFilter;
}
/**
* See {@link PatternRecursive}.
*
* @param DataTable $table
* @return int The number of deleted rows.
*/
public function filter($table)
{
$rows = $table->getRows();
foreach ($rows as $key => $row) {
// A row is deleted if
// 1 - its label doesnt contain the pattern
// AND 2 - the label is not found in the children
$patternNotFoundInChildren = false;
try {
$idSubTable = $row->getIdSubDataTable();
$subTable = Manager::getInstance()->getTable($idSubTable);
// we delete the row if we couldn't find the pattern in any row in the
// children hierarchy
if ($this->filter($subTable) == 0) {
$patternNotFoundInChildren = true;
}
} catch (Exception $e) {
// there is no subtable loaded for example
$patternNotFoundInChildren = true;
}
if ($patternNotFoundInChildren
&& !Pattern::match($this->patternToSearchQuoted, $row->getColumn($this->columnToFilter), $invertedMatch = false)
) {
$table->deleteRow($key);
}
}
return $table->getRowsCount();
}
}

59
www/analytics/core/DataTable/Filter/RangeCheck.php Normal file
View file

@ -0,0 +1,59 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable;
use Piwik\DataTable\BaseFilter;
/**
* Check range
*
*/
class RangeCheck extends BaseFilter
{
static public $minimumValue = 0.00;
static public $maximumValue = 100.0;
/**
* @param DataTable $table
* @param string $columnToFilter name of the column to filter
* @param float $minimumValue minimum value for range
* @param float $maximumValue maximum value for range
*/
public function __construct($table, $columnToFilter, $minimumValue = 0.00, $maximumValue = 100.0)
{
parent::__construct($table);
$this->columnToFilter = $columnToFilter;
if ($minimumValue < $maximumValue) {
self::$minimumValue = $minimumValue;
self::$maximumValue = $maximumValue;
}
}
/**
* Executes the filter an adjusts all columns to fit the defined range
*
* @param DataTable $table
*/
public function filter($table)
{
foreach ($table->getRows() as $row) {
$value = $row->getColumn($this->columnToFilter);
if ($value !== false) {
if ($value < self::$minimumValue) {
$row->setColumn($this->columnToFilter, self::$minimumValue);
} elseif ($value > self::$maximumValue) {
$row->setColumn($this->columnToFilter, self::$maximumValue);
}
}
}
}
}

171
www/analytics/core/DataTable/Filter/ReplaceColumnNames.php Normal file
View file

@ -0,0 +1,171 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable\BaseFilter;
use Piwik\DataTable\Simple;
use Piwik\DataTable;
use Piwik\Metrics;
use Piwik\Piwik;
use Piwik\Tracker\GoalManager;
/**
* Replaces column names in each row of a table using an array that maps old column
* names new ones.
*
* If no mapping is provided, this column will use one that maps index metric names
* (which are integers) with their string column names. In the database, reports are
* stored with integer metric names because it results in blobs that take up less space.
* When loading the reports, the column names must be replaced, which is handled by this
* class. (See {@link Piwik\Metrics} for more information about integer metric names.)
*
* **Basic example**
*
* // filter use in a plugin's API method
* public function getMyReport($idSite, $period, $date, $segment = false, $expanded = false)
* {
* $dataTable = Archive::getDataTableFromArchive('MyPlugin_MyReport', $idSite, $period, $date, $segment, $expanded);
* $dataTable->queueFilter('ReplaceColumnNames');
* return $dataTable;
* }
*
* @api
*/
class ReplaceColumnNames extends BaseFilter
{
protected $mappingToApply;
/**
* Constructor.
*
* @param DataTable $table The table that will be eventually filtered.
* @param array|null $mappingToApply The name mapping to apply. Must map old column names
* with new ones, eg,
*
* array('OLD_COLUMN_NAME' => 'NEW_COLUMN NAME',
* 'OLD_COLUMN_NAME2' => 'NEW_COLUMN NAME2')
*
* If null, {@link Piwik\Metrics::$mappingFromIdToName} is used.
*/
public function __construct($table, $mappingToApply = null)
{
parent::__construct($table);
$this->mappingToApply = Metrics::$mappingFromIdToName;
if (!is_null($mappingToApply)) {
$this->mappingToApply = $mappingToApply;
}
}
/**
* See {@link ReplaceColumnNames}.
*
* @param DataTable $table
*/
public function filter($table)
{
if ($table instanceof Simple) {
$this->filterSimple($table);
} else {
$this->filterTable($table);
}
}
/**
* @param DataTable $table
*/
protected function filterTable($table)
{
foreach ($table->getRows() as $key => $row) {
$oldColumns = $row->getColumns();
$newColumns = $this->getRenamedColumns($oldColumns);
$row->setColumns($newColumns);
$this->filterSubTable($row);
}
}
/**
* @param Simple $table
*/
protected function filterSimple(Simple $table)
{
foreach ($table->getRows() as $row) {
$columns = array_keys($row->getColumns());
foreach ($columns as $column) {
$newName = $this->getRenamedColumn($column);
if ($newName) {
$row->renameColumn($column, $newName);
}
}
}
}
protected function getRenamedColumn($column)
{
$newName = false;
if (isset($this->mappingToApply[$column])
&& $this->mappingToApply[$column] != $column
) {
$newName = $this->mappingToApply[$column];
}
return $newName;
}
/**
* Checks the given columns and renames them if required
*
* @param array $columns
* @return array
*/
protected function getRenamedColumns($columns)
{
$newColumns = array();
foreach ($columns as $columnName => $columnValue) {
$renamedColumn = $this->getRenamedColumn($columnName);
if ($renamedColumn) {
if ($renamedColumn == 'goals') {
$columnValue = $this->flattenGoalColumns($columnValue);
}
// If we happen to rename a column to a name that already exists,
// sum both values in the column. This should really not happen, but
// we introduced in 1.1 a new dataTable indexing scheme for Actions table, and
// could end up with both strings and their int indexes counterpart in a monthly/yearly dataTable
// built from DataTable with both formats
if (isset($newColumns[$renamedColumn])) {
$columnValue += $newColumns[$renamedColumn];
}
$columnName = $renamedColumn;
}
$newColumns[$columnName] = $columnValue;
}
return $newColumns;
}
/**
* @param $columnValue
* @return array
*/
protected function flattenGoalColumns($columnValue)
{
$newSubColumns = array();
foreach ($columnValue as $idGoal => $goalValues) {
$mapping = Metrics::$mappingFromIdToNameGoal;
if ($idGoal == GoalManager::IDGOAL_CART) {
$idGoal = Piwik::LABEL_ID_GOAL_IS_ECOMMERCE_CART;
} elseif ($idGoal == GoalManager::IDGOAL_ORDER) {
$idGoal = Piwik::LABEL_ID_GOAL_IS_ECOMMERCE_ORDER;
}
foreach ($goalValues as $id => $goalValue) {
$subColumnName = $mapping[$id];
$newSubColumns['idgoal=' . $idGoal][$subColumnName] = $goalValue;
}
}
return $newSubColumns;
}
}

74
www/analytics/core/DataTable/Filter/ReplaceSummaryRowLabel.php Normal file
View file

@ -0,0 +1,74 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable\BaseFilter;
use Piwik\DataTable;
use Piwik\DataTable\Manager;
use Piwik\Piwik;
/**
* Replaces the label of the summary row with a supplied label.
*
* This filter is only used to prettify the summary row label and so it should
* always be queued on a {@link DataTable}.
*
* This filter always recurses. In other words, this filter will always apply itself to
* all subtables in the given {@link DataTable}'s table hierarchy.
*
* **Basic example**
*
* $dataTable->queueFilter('ReplaceSummaryRowLabel', array(Piwik::translate('General_Others')));
*
* @api
*/
class ReplaceSummaryRowLabel extends BaseFilter
{
/**
* Constructor.
*
* @param DataTable $table The table that will eventually be filtered.
* @param string|null $newLabel The new label for summary row. If null, defaults to
* `Piwik::translate('General_Others')`.
*/
public function __construct($table, $newLabel = null)
{
parent::__construct($table);
if (is_null($newLabel)) {
$newLabel = Piwik::translate('General_Others');
}
$this->newLabel = $newLabel;
}
/**
* See {@link ReplaceSummaryRowLabel}.
*
* @param DataTable $table
*/
public function filter($table)
{
$rows = $table->getRows();
foreach ($rows as $id => $row) {
if ($row->getColumn('label') == DataTable::LABEL_SUMMARY_ROW
|| $id == DataTable::ID_SUMMARY_ROW
) {
$row->setColumn('label', $this->newLabel);
break;
}
}
// recurse
foreach ($rows as $row) {
if ($row->isSubtableLoaded()) {
$subTable = Manager::getInstance()->getTable($row->getIdSubDataTable());
$this->filter($subTable);
}
}
}
}

72
www/analytics/core/DataTable/Filter/SafeDecodeLabel.php Normal file
View file

@ -0,0 +1,72 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable;
use Piwik\DataTable\BaseFilter;
/**
* Sanitizes DataTable labels as an extra precaution. Called internally by Piwik.
*
*/
class SafeDecodeLabel extends BaseFilter
{
private $columnToDecode;
/**
* @param DataTable $table
*/
public function __construct($table)
{
parent::__construct($table);
$this->columnToDecode = 'label';
}
/**
* Decodes the given value
*
* @param string $value
* @return mixed|string
*/
public static function decodeLabelSafe($value)
{
if (empty($value)) {
return $value;
}
$raw = urldecode($value);
$value = htmlspecialchars_decode($raw, ENT_QUOTES);
// ENT_IGNORE so that if utf8 string has some errors, we simply discard invalid code unit sequences
$style = ENT_QUOTES | ENT_IGNORE;
// See changes in 5.4: http://nikic.github.com/2012/01/28/htmlspecialchars-improvements-in-PHP-5-4.html
// Note: at some point we should change ENT_IGNORE to ENT_SUBSTITUTE
$value = htmlspecialchars($value, $style, 'UTF-8');
return $value;
}
/**
* Decodes all columns of the given data table
*
* @param DataTable $table
*/
public function filter($table)
{
foreach ($table->getRows() as $row) {
$value = $row->getColumn($this->columnToDecode);
if ($value !== false) {
$value = self::decodeLabelSafe($value);
$row->setColumn($this->columnToDecode, $value);
$this->filterSubTable($row);
}
}
}
}

221
www/analytics/core/DataTable/Filter/Sort.php Normal file
View file

@ -0,0 +1,221 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable\BaseFilter;
use Piwik\DataTable\Row;
use Piwik\DataTable\Simple;
use Piwik\DataTable;
use Piwik\Metrics;
/**
* Sorts a {@link DataTable} based on the value of a specific column.
*
* It is possible to specify a natural sorting (see [php.net/natsort](http://php.net/natsort) for details).
*
* @api
*/
class Sort extends BaseFilter
{
protected $columnToSort;
protected $order;
/**
* Constructor.
*
* @param DataTable $table The table to eventually filter.
* @param string $columnToSort The name of the column to sort by.
* @param string $order order `'asc'` or `'desc'`.
* @param bool $naturalSort Whether to use a natural sort or not (see {@link http://php.net/natsort}).
* @param bool $recursiveSort Whether to sort all subtables or not.
*/
public function __construct($table, $columnToSort, $order = 'desc', $naturalSort = true, $recursiveSort = false)
{
parent::__construct($table);
if ($recursiveSort) {
$table->enableRecursiveSort();
}
$this->columnToSort = $columnToSort;
$this->naturalSort = $naturalSort;
$this->setOrder($order);
}
/**
* Updates the order
*
* @param string $order asc|desc
*/
public function setOrder($order)
{
if ($order == 'asc') {
$this->order = 'asc';
$this->sign = 1;
} else {
$this->order = 'desc';
$this->sign = -1;
}
}
/**
* Sorting method used for sorting numbers
*
* @param number $a
* @param number $b
* @return int
*/
public function numberSort($a, $b)
{
return !isset($a->c[Row::COLUMNS][$this->columnToSort])
&& !isset($b->c[Row::COLUMNS][$this->columnToSort])
? 0
: (
!isset($a->c[Row::COLUMNS][$this->columnToSort])
? 1
: (
!isset($b->c[Row::COLUMNS][$this->columnToSort])
? -1
: (($a->c[Row::COLUMNS][$this->columnToSort] != $b->c[Row::COLUMNS][$this->columnToSort]
|| !isset($a->c[Row::COLUMNS]['label']))
? ($this->sign * (
$a->c[Row::COLUMNS][$this->columnToSort]
< $b->c[Row::COLUMNS][$this->columnToSort]
? -1
: 1)
)
: -1 * $this->sign * strnatcasecmp(
$a->c[Row::COLUMNS]['label'],
$b->c[Row::COLUMNS]['label'])
)
)
);
}
/**
* Sorting method used for sorting values natural
*
* @param mixed $a
* @param mixed $b
* @return int
*/
function naturalSort($a, $b)
{
return !isset($a->c[Row::COLUMNS][$this->columnToSort])
&& !isset($b->c[Row::COLUMNS][$this->columnToSort])
? 0
: (!isset($a->c[Row::COLUMNS][$this->columnToSort])
? 1
: (!isset($b->c[Row::COLUMNS][$this->columnToSort])
? -1
: $this->sign * strnatcasecmp(
$a->c[Row::COLUMNS][$this->columnToSort],
$b->c[Row::COLUMNS][$this->columnToSort]
)
)
);
}
/**
* Sorting method used for sorting values
*
* @param mixed $a
* @param mixed $b
* @return int
*/
function sortString($a, $b)
{
return !isset($a->c[Row::COLUMNS][$this->columnToSort])
&& !isset($b->c[Row::COLUMNS][$this->columnToSort])
? 0
: (!isset($a->c[Row::COLUMNS][$this->columnToSort])
? 1
: (!isset($b->c[Row::COLUMNS][$this->columnToSort])
? -1
: $this->sign *
strcasecmp($a->c[Row::COLUMNS][$this->columnToSort],
$b->c[Row::COLUMNS][$this->columnToSort]
)
)
);
}
/**
* Sets the column to be used for sorting
*
* @param Row $row
* @return int
*/
protected function selectColumnToSort($row)
{
$value = $row->getColumn($this->columnToSort);
if ($value !== false) {
return $this->columnToSort;
}
$columnIdToName = Metrics::getMappingFromIdToName();
// sorting by "nb_visits" but the index is Metrics::INDEX_NB_VISITS in the table
if (isset($columnIdToName[$this->columnToSort])) {
$column = $columnIdToName[$this->columnToSort];
$value = $row->getColumn($column);
if ($value !== false) {
return $column;
}
}
// eg. was previously sorted by revenue_per_visit, but this table
// doesn't have this column; defaults with nb_visits
$column = Metrics::INDEX_NB_VISITS;
$value = $row->getColumn($column);
if ($value !== false) {
return $column;
}
// even though this column is not set properly in the table,
// we select it for the sort, so that the table's internal state is set properly
return $this->columnToSort;
}
/**
* See {@link Sort}.
*
* @param DataTable $table
* @return mixed
*/
public function filter($table)
{
if ($table instanceof Simple) {
return;
}
if (empty($this->columnToSort)) {
return;
}
$rows = $table->getRows();
if (count($rows) == 0) {
return;
}
$row = current($rows);
if ($row === false) {
return;
}
$this->columnToSort = $this->selectColumnToSort($row);
$value = $row->getColumn($this->columnToSort);
if (is_numeric($value)) {
$methodToUse = "numberSort";
} else {
if ($this->naturalSort) {
$methodToUse = "naturalSort";
} else {
$methodToUse = "sortString";
}
}
$table->sort(array($this, $methodToUse), $this->columnToSort);
}
}

113
www/analytics/core/DataTable/Filter/Truncate.php Normal file
View file

@ -0,0 +1,113 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable\Filter;
use Piwik\DataTable\BaseFilter;
use Piwik\DataTable;
use Piwik\DataTable\Row;
use Piwik\Piwik;
/**
* Truncates a {@link DataTable} by merging all rows after a certain index into a new summary
* row. If the count of rows is less than the index, nothing happens.
*
* The {@link ReplaceSummaryRowLabel} filter will be queued after the table is truncated.
*
* ### Examples
*
* **Basic usage**
*
* $dataTable->filter('Truncate', array($truncateAfter = 500));
*
* **Using a custom summary row label**
*
* $dataTable->filter('Truncate', array($truncateAfter = 500, $summaryRowLabel = Piwik::translate('General_Total')));
*
* @api
*/
class Truncate extends BaseFilter
{
/**
* Constructor.
*
* @param DataTable $table The table that will be filtered eventually.
* @param int $truncateAfter The row index to truncate at. All rows passed this index will
* be removed.
* @param string $labelSummaryRow The label to use for the summary row. Defaults to
* `Piwik::translate('General_Others')`.
* @param string $columnToSortByBeforeTruncating The column to sort by before truncation, eg,
* `'nb_visits'`.
* @param bool $filterRecursive If true executes this filter on all subtables descending from
* `$table`.
*/
public function __construct($table,
$truncateAfter,
$labelSummaryRow = null,
$columnToSortByBeforeTruncating = null,
$filterRecursive = true)
{
parent::__construct($table);
$this->truncateAfter = $truncateAfter;
if ($labelSummaryRow === null) {
$labelSummaryRow = Piwik::translate('General_Others');
}
$this->labelSummaryRow = $labelSummaryRow;
$this->columnToSortByBeforeTruncating = $columnToSortByBeforeTruncating;
$this->filterRecursive = $filterRecursive;
}
/**
* Executes the filter, see {@link Truncate}.
*
* @param DataTable $table
*/
public function filter($table)
{
$this->addSummaryRow($table);
$table->queueFilter('ReplaceSummaryRowLabel', array($this->labelSummaryRow));
if ($this->filterRecursive) {
foreach ($table->getRows() as $row) {
if ($row->isSubtableLoaded()) {
$this->filter($row->getSubtable());
}
}
}
}
private function addSummaryRow($table)
{
$table->filter('Sort', array($this->columnToSortByBeforeTruncating, 'desc'));
if ($table->getRowsCount() <= $this->truncateAfter + 1) {
return;
}
$rows = $table->getRows();
$count = $table->getRowsCount();
$newRow = new Row(array(Row::COLUMNS => array('label' => DataTable::LABEL_SUMMARY_ROW)));
for ($i = $this->truncateAfter; $i < $count; $i++) {
if (!isset($rows[$i])) {
// case when the last row is a summary row, it is not indexed by $cout but by DataTable::ID_SUMMARY_ROW
$summaryRow = $table->getRowFromId(DataTable::ID_SUMMARY_ROW);
//FIXME: I'm not sure why it could return false, but it was reported in: http://forum.piwik.org/read.php?2,89324,page=1#msg-89442
if ($summaryRow) {
$newRow->sumRow($summaryRow, $enableCopyMetadata = false, $table->getMetadata(DataTable::COLUMN_AGGREGATION_OPS_METADATA_NAME));
}
} else {
$newRow->sumRow($rows[$i], $enableCopyMetadata = false, $table->getMetadata(DataTable::COLUMN_AGGREGATION_OPS_METADATA_NAME));
}
}
$table->filter('Limit', array(0, $this->truncateAfter));
$table->addSummaryRow($newRow);
unset($rows);
}
}

156
www/analytics/core/DataTable/Manager.php Normal file
View file

@ -0,0 +1,156 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable;
use Exception;
use Piwik\Common;
use Piwik\DataTable;
use Piwik\Singleton;
/**
* The DataTable_Manager registers all the instanciated DataTable and provides an
* easy way to access them. This is used to store all the DataTable during the archiving process.
* At the end of archiving, the ArchiveProcessor will read the stored datatable and record them in the DB.
*
* @method static \Piwik\DataTable\Manager getInstance()
*/
class Manager extends Singleton
{
/**
* Array used to store the DataTable
*
* @var array
*/
private $tables = array();
/**
* Id of the next inserted table id in the Manager
* @var int
*/
protected $nextTableId = 1;
/**
* Add a DataTable to the registry
*
* @param DataTable $table
* @return int Index of the table in the manager array
*/
public function addTable($table)
{
$this->tables[$this->nextTableId] = $table;
$this->nextTableId++;
return $this->nextTableId - 1;
}
/**
* Returns the DataTable associated to the ID $idTable.
* NB: The datatable has to have been instanciated before!
* This method will not fetch the DataTable from the DB.
*
* @param int $idTable
* @throws Exception If the table can't be found
* @return DataTable The table
*/
public function getTable($idTable)
{
if (!isset($this->tables[$idTable])) {
throw new TableNotFoundException(sprintf("This report has been reprocessed since your last click. To see this error less often, please increase the timeout value in seconds in Settings > General Settings. (error: id %s not found).", $idTable));
}
return $this->tables[$idTable];
}
/**
* Returns the latest used table ID
*
* @return int
*/
public function getMostRecentTableId()
{
return $this->nextTableId - 1;
}
/**
* Delete all the registered DataTables from the manager
*/
public function deleteAll($deleteWhenIdTableGreaterThan = 0)
{
foreach ($this->tables as $id => $table) {
if ($id > $deleteWhenIdTableGreaterThan) {
$this->deleteTable($id);
}
}
if ($deleteWhenIdTableGreaterThan == 0) {
$this->tables = array();
$this->nextTableId = 1;
}
}
/**
* Deletes (unsets) the datatable given its id and removes it from the manager
* Subsequent get for this table will fail
*
* @param int $id
*/
public function deleteTable($id)
{
if (isset($this->tables[$id])) {
Common::destroy($this->tables[$id]);
$this->setTableDeleted($id);
}
}
/**
* Deletes all tables starting from the $firstTableId to the most recent table id except the ones that are
* supposed to be ignored.
*
* @param int[] $idsToBeIgnored
* @param int $firstTableId
*/
public function deleteTablesExceptIgnored($idsToBeIgnored, $firstTableId = 0)
{
$lastTableId = $this->getMostRecentTableId();
for ($index = $firstTableId; $index <= $lastTableId; $index++) {
if (!in_array($index, $idsToBeIgnored)) {
$this->deleteTable($index);
}
}
}
/**
* Remove the table from the manager (table has already been unset)
*
* @param int $id
*/
public function setTableDeleted($id)
{
$this->tables[$id] = null;
}
/**
* Debug only. Dumps all tables currently registered in the Manager
*/
public function dumpAllTables()
{
echo "<hr />Manager->dumpAllTables()<br />";
foreach ($this->tables as $id => $table) {
if (!($table instanceof DataTable)) {
echo "Error table $id is not instance of datatable<br />";
var_export($table);
} else {
echo "<hr />";
echo "Table (index=$id) TableId = " . $table->getId() . "<br />";
echo $table;
echo "<br />";
}
}
echo "<br />-- End Manager->dumpAllTables()<hr />";
}
}

445
www/analytics/core/DataTable/Map.php Normal file
View file

@ -0,0 +1,445 @@
<?php
/**
* Piwik - Open source web analytics
*
* @link http://piwik.org
* @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
*
*/
namespace Piwik\DataTable;
use Piwik\Common;
use Piwik\DataTable;
use Piwik\DataTable\Renderer\Console;
/**
* Stores an array of {@link DataTable}s indexed by one type of {@link DataTable} metadata (such as site ID
* or period).
*
* DataTable Maps are returned on all queries that involve multiple sites and/or multiple
* periods. The Maps will contain a {@link DataTable} for each site and period combination.
*
* The Map implements some {@link DataTable} such as {@link queueFilter()} and {@link getRowsCount}.
*
*
* @api
*/
class Map implements DataTableInterface
{
/**
* Array containing the DataTable withing this Set
*
* @var DataTable[]
*/
protected $array = array();
/**
* @see self::getKeyName()
* @var string
*/
protected $keyName = 'defaultKeyName';
/**
* Returns a string description of the data used to index the DataTables.
*
* This label is used by DataTable Renderers (it becomes a column name or the XML description tag).
*
* @return string eg, `'idSite'`, `'period'`
*/
public function getKeyName()
{
return $this->keyName;
}
/**
* Set the name of they metadata used to index {@link DataTable}s. See {@link getKeyName()}.
*
* @param string $name
*/
public function setKeyName($name)
{
$this->keyName = $name;
}
/**
* Returns the number of {@link DataTable}s in this DataTable\Map.
*
* @return int
*/
public function getRowsCount()
{
return count($this->getDataTables());
}
/**
* Queue a filter to {@link DataTable} child of contained by this instance.
*
* See {@link Piwik\DataTable::queueFilter()} for more information..
*
* @param string|Closure $className Filter name, eg. `'Limit'` or a Closure.
* @param array $parameters Filter parameters, eg. `array(50, 10)`.
*/
public function queueFilter($className, $parameters = array())
{
foreach ($this->getDataTables() as $table) {
$table->queueFilter($className, $parameters);
}
}
/**
* Apply the filters previously queued to each DataTable contained by this DataTable\Map.
*/
public function applyQueuedFilters()
{
foreach ($this->getDataTables() as $table) {
$table->applyQueuedFilters();
}
}
/**
* Apply a filter to all tables contained by this instance.
*
* @param string|Closure $className Name of filter class or a Closure.
* @param array $parameters Parameters to pass to the filter.
*/
public function filter($className, $parameters = array())
{
foreach ($this->getDataTables() as $id => $table) {
$table->filter($className, $parameters);
}
}
/**
* Returns the array of DataTables contained by this class.
*
* @return DataTable[]|Map[]
*/
public function getDataTables()
{
return $this->array;
}
/**
* Returns the table with the specific label.
*
* @param string $label
* @return DataTable|Map
*/
public function getTable($label)
{
return $this->array[$label];
}
/**
* Returns the first element in the Map's array.
*
* @return DataTable|Map|false
*/
public function getFirstRow()
{
return reset($this->array);
}
/**
* Returns the last element in the Map's array.
*
* @return DataTable|Map|false
*/
public function getLastRow()
{
return end($this->array);
}
/**
* Adds a new {@link DataTable} or Map instance to this DataTable\Map.
*
* @param DataTable|Map $table
* @param string $label Label used to index this table in the array.
*/
public function addTable($table, $label)
{
$this->array[$label] = $table;
}
/**
* Returns a string output of this DataTable\Map (applying the default renderer to every {@link DataTable}
* of this DataTable\Map).
*
* @return string
*/
public function __toString()
{
$renderer = new Console();
$renderer->setTable($this);
return (string)$renderer;
}
/**
* See {@link DataTable::enableRecursiveSort()}.
*/
public function enableRecursiveSort()
{
foreach ($this->getDataTables() as $table) {
$table->enableRecursiveSort();
}
}
/**
* Renames the given column in each contained {@link DataTable}.
*
* See {@link DataTable::renameColumn()}.
*
* @param string $oldName
* @param string $newName
*/
public function renameColumn($oldName, $newName)
{
foreach ($this->getDataTables() as $table) {
$table->renameColumn($oldName, $newName);
}
}
/**
* Deletes the specified columns in each contained {@link DataTable}.
*
* See {@link DataTable::deleteColumns()}.
*
* @param array $columns The columns to delete.
* @param bool $deleteRecursiveInSubtables This param is currently not used.
*/
public function deleteColumns($columns, $deleteRecursiveInSubtables = false)
{
foreach ($this->getDataTables() as $table) {
$table->deleteColumns($columns);
}
}
/**
* Deletes a table from the array of DataTables.
*
* @param string $id The label associated with {@link DataTable}.
*/
public function deleteRow($id)
{
unset($this->array[$id]);
}
/**
* Deletes the given column in every contained {@link DataTable}.
*
* @see DataTable::deleteColumn
* @param string $name
*/
public function deleteColumn($name)
{
foreach ($this->getDataTables() as $table) {
$table->deleteColumn($name);
}
}
/**
* Returns the array containing all column values in all contained {@link DataTable}s for the requested column.
*
* @param string $name The column name.
* @return array
*/
public function getColumn($name)
{
$values = array();
foreach ($this->getDataTables() as $table) {
$moreValues = $table->getColumn($name);
foreach ($moreValues as &$value) {
$values[] = $value;
}
}
return $values;
}
/**
* Merges the rows of every child {@link DataTable} into a new one and
* returns it. This function will also set the label of the merged rows
* to the label of the {@link DataTable} they were originally from.
*
* The result of this function is determined by the type of DataTable
* this instance holds. If this DataTable\Map instance holds an array
* of DataTables, this function will transform it from:
*
* Label 0:
* DataTable(row1)
* Label 1:
* DataTable(row2)
*
* to:
*
* DataTable(row1[label = 'Label 0'], row2[label = 'Label 1'])
*
* If this instance holds an array of DataTable\Maps, this function will
* transform it from:
*
* Outer Label 0: // the outer DataTable\Map
* Inner Label 0: // one of the inner DataTable\Maps
* DataTable(row1)
* Inner Label 1:
* DataTable(row2)
* Outer Label 1:
* Inner Label 0:
* DataTable(row3)
* Inner Label 1:
* DataTable(row4)
*
* to:
*
* Inner Label 0:
* DataTable(row1[label = 'Outer Label 0'], row3[label = 'Outer Label 1'])
* Inner Label 1:
* DataTable(row2[label = 'Outer Label 0'], row4[label = 'Outer Label 1'])
*
* If this instance holds an array of DataTable\Maps, the
* metadata of the first child is used as the metadata of the result.
*
* This function can be used, for example, to smoosh IndexedBySite archive
* query results into one DataTable w/ different rows differentiated by site ID.
*
* Note: This DataTable/Map will be destroyed and will be no longer usable after the tables have been merged into
* the new dataTable to reduce memory usage. Destroying all DataTables witihn the Map also seems to fix a
* Segmentation Fault that occurred in the AllWebsitesDashboard when having > 16k sites.
*
* @return DataTable|Map
*/
public function mergeChildren()
{
$firstChild = reset($this->array);
if ($firstChild instanceof Map) {
$result = $firstChild->getEmptyClone();
/** @var $subDataTableMap Map */
foreach ($this->getDataTables() as $label => $subDataTableMap) {
foreach ($subDataTableMap->getDataTables() as $innerLabel => $subTable) {
if (!isset($result->array[$innerLabel])) {
$dataTable = new DataTable();
$dataTable->setMetadataValues($subTable->getAllTableMetadata());
$result->addTable($dataTable, $innerLabel);
}
$this->copyRowsAndSetLabel($result->array[$innerLabel], $subTable, $label);
}
}
} else {
$result = new DataTable();
foreach ($this->getDataTables() as $label => $subTable) {
$this->copyRowsAndSetLabel($result, $subTable, $label);
Common::destroy($subTable);
}
$this->array = array();
}
return $result;
}
/**
* Utility function used by mergeChildren. Copies the rows from one table,
* sets their 'label' columns to a value and adds them to another table.
*
* @param DataTable $toTable The table to copy rows to.
* @param DataTable $fromTable The table to copy rows from.
* @param string $label The value to set the 'label' column of every copied row.
*/
private function copyRowsAndSetLabel($toTable, $fromTable, $label)
{
foreach ($fromTable->getRows() as $fromRow) {
$oldColumns = $fromRow->getColumns();
unset($oldColumns['label']);
$columns = array_merge(array('label' => $label), $oldColumns);
$row = new Row(array(
Row::COLUMNS => $columns,
Row::METADATA => $fromRow->getMetadata(),
Row::DATATABLE_ASSOCIATED => $fromRow->getIdSubDataTable()
));
$toTable->addRow($row);
}
}
/**
* Sums a DataTable to all the tables in this array.
*
* _Note: Will only add `$tableToSum` if the childTable has some rows._
*
* See {@link Piwik\DataTable::addDataTable()}.
*
* @param DataTable $tableToSum
*/
public function addDataTable(DataTable $tableToSum)
{
foreach ($this->getDataTables() as $childTable) {
$childTable->addDataTable($tableToSum);
}
}
/**
* Returns a new DataTable\Map w/ child tables that have had their
* subtables merged.
*
* See {@link DataTable::mergeSubtables()}.
*
* @return Map
*/
public function mergeSubtables()
{
$result = $this->getEmptyClone();
foreach ($this->getDataTables() as $label => $childTable) {
$result->addTable($childTable->mergeSubtables(), $label);
}
return $result;
}
/**
* Returns a new DataTable\Map w/o any child DataTables, but with
* the same key name as this instance.
*
* @return Map
*/
public function getEmptyClone()
{
$dataTableMap = new Map;
$dataTableMap->setKeyName($this->getKeyName());
return $dataTableMap;
}
/**
* Returns the intersection of children's metadata arrays (what they all have in common).
*
* @param string $name The metadata name.
* @return mixed
*/
public function getMetadataIntersectArray($name)
{
$data = array();
foreach ($this->getDataTables() as $childTable) {
$childData = $childTable->getMetadata($name);
if (is_array($childData)) {
$data = array_intersect($data, $childData);
}
}
return array_values($data);
}
/**
* See {@link DataTable::getColumns()}.
*
* @return array
*/
public function getColumns()
{
foreach ($this->getDataTables() as $childTable) {
if ($childTable->getRowsCount() > 0) {
return $childTable->getColumns();
}
}
return array();
}
}

Some files were not shown because too many files have changed in this diff Show more