matomo-logtail
Script to continuous import serverlogs to matomo with logtail
On GitHub Site Matomo Server Log Analytics is recommended to import
serverlogs automatically rotated every day.
With this script it is possible to continuous import serverlogs to matomo by cronjobs.
The script uses logtail2 to read logfiles and feed matomo with new lines.
From logtail2 manpage:
logtail2 reads a specified file (usually a log file) and writes to the standard output that part of it which has not been read by previous runs of logtail2. It prints the appro‐
priate number of bytes from the end of logfile, assuming that all changes that are made to it are to add new characters to it.
Howto:
Install logtail
Install script import_logs.py from https://github.com/matomo-org/matomo-log-analytics
Download matomo-logtail.sh and copy to /usr/local/bin
Create directory /etc/matomo-logtail
Create global configfile for matomo-logtail /etc/matomo-logtail/matomo-logtail.cfg:
MATOMO_DIR="/var/www/matomo" MATOMO_URL="http://mysite/matomo/" MATOMO_LOGTAIL_WORKDIR="/var/lib/matomo-logtail" LOGTAIL="/usr/sbin/logtail2" IMPORT_LOGS_BIN="/usr/local/bin/import_logs.py" IMPORT_LOGS_OPT="--config=config/config.ini.php --recorders=4 --enable-static"
Create a configfile for your every matomo website instance, for example /etc/matomo-logtail/mysite.sitecfg:
SITE_NAME="mysite" SITE_ID=1 SITE_LOG_FILE="/var/log/apache2/www.mysite.com-access.log" IMPORT_LOGS_SITE_OPT="--enable-reverse-dns --enable-bots"
run test matomo-logtail.sh from console:
root@server:~# matomo-logtail.sh processing idsite 1 mysite /var/log/apache2/www.mysite.com-access.log... 12 lines... done. processing idsite 5 myothersite /var/log/apache2/www.myothersite.com-ssl-access.log... 31 lines... done. processing idsite 12 mycloud /var/log/apache2/mycloud.example.com-ssl-access.log... 17 lines... done. root@server:~#
Create cronjob for import logs every 5 minutes /etc/cron.d/matomo-logtail:
MAILTO="" */5 * * * * root /usr/local/bin/matomo-logtail.sh
