
= Database Reports
:description: Learn about database reports in {page-component-title}, including a list of templates, how to create a custom report, and info. on Jaspersoft Studio.

Reporting on data from your monitoring system is important for strategic and operational decisions.
Database reports provide access to the {page-component-title} JasperReports engine, and let you create and customize report templates.
You can run reports on demand or on a predefined schedule.

NOTE: Originally, database reports created reports based only on data stored in the {page-component-title} database.
This is no longer mandatory; you can also use performance data.
Theoretically, reports do not need to be {page-component-title}-related.

The {page-component-title} Report Engine lets you create reports and supports distributed report repositories.
This documentation does not cover those features; it describes only reports using the JasperReports engine and Grafana dashboards.

== Overview

The {page-component-title} Report Engine uses the JasperReport library to create reports in various formats.
Each report template must be saved as a `*.jrxml` file.

Every time a report runs, the Report Engine passes it a JDBC connection to the {page-component-title} database.

NOTE: Only a JDBC connection to the {page-component-title} database can be passed to the report.

[caption=]
.Report Engine features
[cols="1,1"]
|===
s| Supported Output Formats
| PDF, CSV

s| JasperReport Version
| {jasperreportsversion}
|===

For more details on how JasperReports works, see the http://community.jaspersoft.com/documentation/tibco-jaspersoft-studio-user-guide/v610/getting-started-jaspersoft-studio[official Jaspersoft Studio documentation].

[[db-report-sample-templates]]
== Report templates

{page-component-title} includes the following sample report templates:

[cols="2,1,3"]
|===
| Report Name | Category  | Description

| xref:deep-dive/database-reports/templates/avail-summary.adoc[Availability Summary]
| Status
| Summary of outages and availability for a specified node tag, spanning a specified number of days.

| xref:deep-dive/database-reports/templates/availability.adoc[Availability by Node]
| Status
| List of availability statistics for each node in a specified node tag, spanning a specified number of days.

| xref:deep-dive/database-reports/templates/calendar.adoc[Default Calendar]
| Status
| Network availability information across the previous month and the current month, to date, presented in a calendar format.
Summary of devices with the most problems, and monitored services.
This is not a JasperReport.

| xref:deep-dive/database-reports/templates/classic.adoc[Default Classic]
| Status
| Network availability information across the previous month and the current month, to date.
Summary of devices with the most problems, and monitored services.
This is not a JasperReport.

| xref:deep-dive/database-reports/templates/event.adoc[Event Analysis]
| Status
| Tables and charts showing different slices on the top 25 events over the past 30 days.

| xref:deep-dive/database-reports/templates/grafana.adoc[Grafana Dashboards]
| Grafana
| Three variations of reports created from a specified Grafana dashboard.

| xref:deep-dive/database-reports/templates/interface-avail.adoc[Interface Availability]
| Status
| Availability summary of interfaces which have experienced outages, spanning a specified time range.

| xref:deep-dive/database-reports/templates/maintenance-expired.adoc[Maintenance Contracts Expired]
| Asset Management
| Summary of assets whose maintenance contracts are set to expire within a specified time range.

| xref:deep-dive/database-reports/templates/maintenance-strategy.adoc[Maintenance Contracts Strategy]
| Asset Management
| Summary of assets, sorted by device age, and then by contract status.
It also shows how many contracts are set to expire within the next 12 months.

| xref:deep-dive/database-reports/templates/morning.adoc[]
| Status
| Summary of node status over the past 24 hours.

| xref:deep-dive/database-reports/templates/response-summary.adoc[Response Time Summary]
| Status
| Summary of response times for nodes with a specified tag.

| xref:deep-dive/database-reports/templates/response-time.adoc[Response Time by Node]
| Status
| Summary of response times, listed by node.

| xref:deep-dive/database-reports/templates/serial.adoc[Serial Interface Utilization Summary]
| Status
| Summary of serial interface utilization.

| xref:deep-dive/database-reports/templates/snmp.adoc[SNMP Interface Availability]
| Status
| Summary of SNMP interface status over a specified time range.

| xref:deep-dive/database-reports/templates/total-bytes.adoc[Total Bytes Transferred by Interface]
| Usage
| Summary of bytes transferred, listed by interface.

| xref:deep-dive/database-reports/templates/traffic-rates.adoc[Average and Peak Traffic Rates for Nodes by Interface]
| Usage
| Average and peak traffic rates for nodes, listed by interface.
|===

=== Modify report templates

All default report templates are stored in `$\{OPENNMS_HOME}/etc/report-templates`, and can be modified to suit your needs.
Any changes are applied the next time {page-component-title} creates a report.

When a subreport is modified, {page-component-title} detects the change based on the report's `lastModified` parameter and recompiles it.
A compiled version of the report is represented by a `.jasper` file with the same name as the `.jrxml` file.
Subreports are located in `$\{OPENNMS_HOME}/etc/report-templates/subreports`.

TIP: If you are not sure that your compiled subreports match the current templates, simply delete all `.jasper` files.
{page-component-title} automatically recompiles the subreports as needed.

== Add a custom report

To add a new report to your local {page-component-title} repository, follow these steps:

. Create a new entry in `$\{OPENNMS_HOME}/etc/database-reports.xml`:
+
[source, xml]
-----
<report
  id="MyReport" <1>
  display-name="My Report" <2>
  online="true" <3>
  report-service="jasperReportService" <4>
  description="This is an example description. It shows up in the web ui when creating an online report"  <5>
/>
-----
<1> A unique identifier.
<2> The name of the report.
This name is displayed in the web UI.
<3> Defines if this report can run on demand; otherwise, only scheduling is possible.
<4> The report service implementation to use.
In most cases, this is `jasperReportService`.
<5> A description of the report.
This description is displayed in the web UI.

. Create a new entry in `$\{OPENNMS_HOME}/etc/jasper-reports.xml`:
+
[source, xml]
-----
<report
  id="MyReport" <1>
  template="My-Report.jrxml" <2>
  engine="jdbc" <3>
/>
-----
<1> The identifier defined in the previous step.
It must exist in `$\{OPENNMS_HOME}/etc/database-reports.xml`.
<2> The name of the template.
The template must be located in `$\{OPENNMS_HOME}/etc/report-templates`.
<3> The engine to use (either `jdbc` or `null`).

== Jaspersoft Studio

We recommend that you use Jaspersoft Studio to develop new reports.
You can download it via the http://community.jaspersoft.com/project/jaspersoft-studio[official Jaspersoft website].

TIP: We recommend that you use the same version of Jaspersoft Studio that the {page-component-title} JasperReport library uses.
Currently, {page-component-title} uses version {jasperreportsversion}.

=== Connect to {page-component-title} database

You must create a database data adapter to run SQL queries against your {page-component-title} database (see the https://community.jaspersoft.com/documentation/tibco-jaspersoft-studio-user-guide/v610/data-sources[official Jaspersoft Studio documentation]).

=== Use Measurements Datasource and helpers

You must add the Measurements Datasource library to Jaspersoft Studio's build path to use the Measurements API:

. Open the Project Explorer.
. Right-click in the window and select *Configure Buildpath*.
. Navigate to the *Libraries* tab:
+
image::database-reports/1_configure-build-path-1.png["Project Explorer UI displaying the Properties dialog."]

. Click *Add External JARs*.
. Select `$\{OPENNMS_HOME}/bin/opennms-jasperstudio-extension-\{opennms-version}-jar-with-dependencies.jar`.
. Close the file selection dialog.
The Measurements Datasource and helpers should now be available:
+
image::database-reports/2_configure-build-path-2.png["Project Explorer UI displaying the Properties dialog with the external JAR configured."]

. Open Jaspersoft Studio and navigate to the *Dataset and Query* dialog.
. Select the *Measurement* language from the *Language* list:
+
image::database-reports/3_dataset-query-dialog.png["JasperSoft Studio UI displaying the Dataset and Query dialog."]

NOTE: If the Read Fields functionality is not available, use the Data Preview.
To access the Measurements API, use the `MEASUREMENT_URL`, `MEASUREMENT_USERNAME`, and `MEASUREMENT_PASSWORD` parameters (see <<ga-database-reports-fields, Datasource fields>>).

== Access performance data

The {page-component-title} Measurement Datasource lets you query the Measurements API and process the returned performance data in your reports.
See xref:development:rest/measurements.adoc[] for information on how to use the API.

NOTE: When you use the Measurements Datasource for a report, an HTTP connection to the Measurements API is established only if the report is not running within {page-component-title} (for example, when run in Jaspersoft Studio).

To receive data from the Measurements API, create a query:

.Sample `queryString` to receive data from Measurements API
[source, xml]
------
<query-request step="300000" start="$P{startDateTime}" end="$P{endDateTime}" maxrows="2000"> <1>
  <source aggregation="AVERAGE" label="IfInOctets" attribute="ifHCInOctets" transient="false" resourceId="node[$P\{nodeid}].interfaceSnmp[$P{interface}]"/>
  <source aggregation="AVERAGE" label="IfOutOctets" attribute="ifHCOutOctets" transient="false" resourceId="node[$P\{nodeid}].interfaceSnmp[$P{interface}]"/>
</query-request>
------
<1> The query language; in this case, `measurement`.
JasperReports supports many languages out of the box, including SQL, xpath, and so on.

[[ga-database-reports-fields]]
=== Datasource fields

Each datasource returns a number of fields which you can use in reports.
The Measurement Datasource supports the following fields:

[cols="1,3,1"]
|===
| Name  | Description | Type

| label
| Each source defined as `transient=false` can be used as a field.
The name of the field is the label (for example, `IfInOctets`).
| java.lang.Double

| timestamp
| Sample's timestamp
| java.util.Date

| step
| Response step size.
Returns the same value for all rows.
| java.lang.Long

| start
| Timestamp for the beginning of the response, in milliseconds.
Returns the same value for all rows.
| java.lang.Long

| end
| Timestamp for the end of the response, in milliseconds.
Returns the same value for all rows.
| java.lang.Long
|===

For more information, see xref:development:rest/measurements.adoc[].

=== Parameters

In addition to the `queryString`, the following JasperReports parameters are supported:

[options="autowidth"]
|===
| Parameter | Description

2+| *Required*

| MEASUREMENTURL
| The URL of the Measurements API (for example, `\http://localhost:8980/opennms/rest/measurements`).

2+| *Optional*

| MEASUREMENT_USERNAME
| Username to access the datasource, if required (for example, "admin").

| MEASUREMENT_PASSWORD
| Password to access the datasource, if required (for example, "admin").
|===

== Disable report scheduler

To disable the report scheduler, set `opennms.report.scheduler.enabled` to `false`.
You can set it in a `.properties` file in `$\{OPENNMS_HOME}/etc/opennms.properties.d/`.

== Helper methods

You can use helper methods to create reports in {page-component-title}.
These helpers are included with the Measurement Datasource:

[cols="1,1"]
|===
| Helper Method | Description

| getNodeOrNodeSourceDescriptor(nodeId, foreignSource, foreignId)
| Generates a node source descriptor according to the input parameters.
Either `node[nodeId]` or `nodeSource[foreignSource:foreignId]` is returned.
`nodeSource[foreignSource:foreignId]` is returned only if `foreignSource` and `foreignId` are not empty and not `null`.
Otherwise, `node[nodeId]` is returned. +
For more details, see <<ga-database-reports-node-source-descriptor, Node source descriptors>>.

| getInterfaceDescriptor(snmpifname, snmpifdescr, snmpphysaddr)
| Returns the interface descriptor of a given interface (for example, `en0-005e607e9e00`).
The input parameters are prioritized. +
If an `snmpifdescr` is specified, it is used instead of the `snmpifname`.
If an `snmpphysaddr` is defined, it will be appended to `snmpifname` or `snmpifdescr`. +
Because each input parameter may be `null`, at least one must be defined. +
For more details, see <<ga-database-reports-interface-descriptor, Interface descriptors>>.
|===

[[ga-database-reports-node-source-descriptor]]
=== Node source descriptors

A node is addressed by a node source descriptor.
The node source descriptor references a node using either its foreign source and foreign ID, or its node ID.
If `storeByForeignSource` is enabled, you can address the node using only its foreign source and foreign ID (see https://opennms.discourse.group/t/storing-data-with-foreign-sources/2057[Storing data with foreign sources] on Discourse).

To make report creation easier, the Measurement Datasource includes a helper method that creates the node source descriptor:

[source, xml]
------
<parameter name="nodeResourceDescriptor" class="java.lang.String" isForPrompting="false">
  <defaultValueExpression><![CDATA[org.opennms.netmgt.jasper.helper.MeasurementsHelper.getNodeOrNodeSourceDescriptor(String.valueOf($P{nodeid}), $Pforeignsource, $P{foreignid})]]></defaultValueExpression>
</parameter>
<queryString language="Measurement">
  <![CDATA[<query-request step="300000" start="$P{startDateTime}" end="$P{endDateTime}" maxrows="2000">
<source aggregation="AVERAGE" label="IfInOctets" attribute="ifHCInOctets" transient="false" resourceId="$P{nodeResourceDescriptor}.interfaceSnmp[en0-005e607e9e00]"/>
<source aggregation="AVERAGE" label="IfOutOctets" attribute="ifHCOutOctets" transient="false" resourceId="$P{nodeResourceDescriptor}.interfaceSnmp[en0-005e607e9e00]"/>
</query-request>]]>
------

Depending on the input parameters, the method returns either a node resource descriptor or a foreign source/foreign ID resource descriptor.

[[ga-database-reports-interface-descriptor]]
=== Interface descriptors

An interface is addressed using its exact interface descriptor.
The Measurement Datasource includes a helper method that allows easy access to the interface descriptor:

[source, xml]
------
<parameter name="interface" class="java.lang.String" isForPrompting="false">
  <parameterDescription><![CDATA[]]></parameterDescription>
  <defaultValueExpression><![CDATA[org.opennms.netmgt.jasper.helper.MeasurementsHelper.getInterfaceDescriptor($P{snmpifname}, $P{snmpifdescr}, $P{snmpphysaddr})]]></defaultValueExpression>
</parameter>
<queryString language="Measurement">
  <![CDATA[<query-request step="300000" start="$P{startDateTime}" end="$P{endDateTime}" maxrows="2000">
<source aggregation="AVERAGE" label="IfInOctets" attribute="ifHCInOctets" transient="false" resourceId="node[$P{nodeid}].interfaceSnmp[$P{interface}]"/>
<source aggregation="AVERAGE" label="IfOutOctets" attribute="ifHCOutOctets" transient="false" resourceId="node[$P{nodeid}].interfaceSnmp[$P{interface}]"/>
</query-request>]]>
------

=== Connect to Measurements API

To establish a secure connection to the Measurements API, you must import your {page-component-title} instance's public certificate to the Java truststore and configure {page-component-title} to use it (see <<deep-dive/admin/configuration/https/https-client.adoc#ga-operation-ssl-opennms-trust-store, Configure {page-component-title} to use the Java truststore>>).
You must also set `org.opennms.netmgt.jasper.measurement.ssl.enable` to `true` in `$\{OPENNMS_HOME}/etc/opennms.properties` to ensure that only secure connections are established.

WARNING: If `org.opennms.netmgt.jasper.measurement.ssl.enable` is set to `false`, an insecure connection to the Measurements API can be established.
