Usage Statistics
General
When you bring multiple Shiny apps to your end users, it may be interesting to track usage of the different applications over time. It may a.o. help to understand your user base and to prioritize maintenance work for the different applications.
ShinyProxy has a general model on board that allows one to store Shiny app events and for different storage back-ends, different plug-ins are available. Currently, ShinyProxy ships with plug-ins that can store events in the following back-ends:
Alternative plug-ins can easily be developed and are on the roadmap.
Starting from ShinyProxy 2.5.0 an extra plug-in was added that supports for Micrometer.io. Micrometer itself is a modular library that allows to support many popular monitoring system. Our focus is to export Prometheus Metrics, however, any Micrometer supported monitoring system is supported.
InfluxDB
Installation
InfluxDB installation is very simple: on an Ubuntu 16.04 LTS system one just needs to download and install the package using something along
wget https://dl.influxdata.com/influxdb/releases/influxdb_1.0.2_amd64.deb'
sudo dpkg -i influxdb_1.0.2_amd64.deb
To start the database one can use
sudo service influxdb start
and querying its status can be done with
sudo service influxdb status
Database creation
In order to store the usage statistics, a specific database needs
to be created on the influxDB instance. In this example we will give it the name shinyproxy_usagestats
.
To do so one can use the CLI client:
influx
then, at the database prompt, enter
CREATE DATABASE shinyproxy_usagestats
quit
Configuration
Once the database is created, we can configure ShinyProxy to post all usage events to the shinyproxy_usagestats
database. It is sufficient to add a single property in the application.yml
file under
the shiny / proxy section:
proxy:
[...]
usage-stats-url: http://localhost:8086/write?db=shinyproxy_usagestats
[...]
As can be seen, influxDB has the advantage that the storage of usage events can be done on a separate server and should not necessarily happen on localhost
i.e. the machine on which the ShinyProxy server itself is running.
Note:
If authentication on the shinyproxy_usagestats
database is required, one can use the usage-stats-username
and usage-stats-password
properties.
More detailed information on configuring ShinyProxy can be found here
SQL Based Databases (JDBC)
Installation
Please refer to the official documentation of your database system for installation. For example, on an Ubuntu 16.04 system MonetDB can be installed from the official Debian package repository that is available here.
Database creation
When posting events to a database, it is assumed that an event
table exists that should be created upfront.
For MonetDB. MySQL, MariaDB or PostgreSQL use:
create table event(
event_time timestamp,
username varchar(128),
type varchar(128),
data text
);
For Microsoft SQL Server, use:
create table event(
event_time datetime,
username varchar(128),
type varchar(128),
data text
);
Configuration
Once the database is created, we can configure ShinyProxy to post all usage events to the database. It is sufficient to only add a JDBC url in the usage-stats-url
field in the shiny.proxy
section of the application.yml
file:
proxy:
[...]
# for MonetDB
usage-stats-url: jdbc:monetdb://localhost:50000/usage_stats
# for MySQL/MariaDB
usage-stats-url: jdbc:mysql://localhost/shinyproxy
# for PostgreSQL
usage-stats-url: jdbc:postgresql://localhost/postgres
# for MS SQL Server:
usage-stats-url: jdbc:sqlserver://localhost;databaseName=shinyproxy
[...]
Similarly to influxDB, MonetDB can be used to store events on a separate server and event logging should not necessarily happen on localhost
i.e. the machine on which the ShinyProxy server itself is running.
Note:
The default login and password that will be used to log into the database are monetdb
and monetdb
respectively. If other credentials have been set on the database, one can enter the usage-stats-username
and usage-stats-password
properties below the usage-stats-url
field.
More detailed information on configuring ShinyProxy can be found here.
Micrometer
This section describes on how to setup the Micrometer monitoring backend using
Prometheus. It assumes that ShinyProxy, Prometheus and Grafana (see later) are
all running on localhost
.
Configuration
Start by changing your ShinyProxy configuration so that it includes the following configuration:
proxy:
[...]
usage-stats-url: micrometer
management:
metrics:
export:
prometheus:
enabled: true
[...]
The Prometheus metrics are now available at the
localhost:9090/actuator/prometheus
endpoint. Note that you should not include
any context-path in this URL.
Setup Prometheus
The next step is to ensure that Prometheus scrapes your ShinyProxy server. For demonstrating purposes, it is enough to download it and extract it on your server. The next step is to adapt your configuration, so that Prometheus scrapes the ShinyProxy server:
scrape_configs:
- job_name: 'shinyproxy'
metrics_path: '/actuator/prometheus'
static_configs:
# note: this is the port of ShinyProxy Actuator services, not the port of Prometheus which is by default also 9090
- targets: ['localhost:9090']
Now you can start Prometheus. Make sure to run it on a different port than 9090, since that port is already being used by ShinyProxy:
./prometheus --web.listen-address="0.0.0.0:7070"
Now go to your browser and open
http://localhost:7070/targets, this page should
contain one target called shinyproxy
with the UP state.
Setup Grafana
Now that Prometheus scrapes the metrics, we need a way to visualize these metrics. An excellent tool for this is Grafana. Start by downloading Grafana, again you can use the standalone binaries and simple extract them. Then start Grafana using:
./bin/grafana-server
Next, access Grafana on http://localhost:3000 and login
using admin:admin
. Next add Prometheus as datasource, remember that Prometheus
is running at http://localhost:7070
.
The final step is to import the dashboard. Download the JSON file (right-click and choose save as )and import it into Grafana. If everything is correctly setup, you should see the graph changing when you login, start and app etc.
Exposed Metrics
ShinyProxy keeps track of the following actions:
- User Login
- User Logout (also by expiring session)
- Authentication failure
- Application start
- Application stop
- Duration of application startup
- Duration of application usage
- Application start failures