Metering

The act of measuring customer activity or consumption of resources to collect the metrics needed to generate an invoice.

Overview

📖

Definitions

Metrics represent all the data that you capture to analyze trends across your business and technology domains. Raw metrics are unfiltered data. Derived metrics, on the other hand, are generated when a certain combination of selection criteria (or filters) is applied on the raw metrics.

Metering is the act of measuring customer activity or consumption of resources to collect the metrics needed to generate an invoice.

Aggregates are derived metrics. Average, count, distinct, maximum, minimum, and sum are some selection criteria (or filters) applied on raw metrics to derive aggregates.

Your application can be instrumented with specific metering capabilities to extract the metrics needed to generate an invoice. These metrics could be number of messages sent or received, number of active users, or it could be some aggregate of resource consumption (CPU, memory, storage) that makes sense to your customers.

Zenskar can be configured to connect with your data sources to ingest raw metrics or aggregates. Zenskar can ingest data from various data sources seamlessly. Alternatively, you can send us your raw metrics using our application programming interfaces (APIs) to generate an invoice, as shown in Fig. 1.

Various data sources that Zenskar can connect to and various ways customers can send raw metrics to Zenskar

Fig. 1: Zenskar ingests data from various data sources and generates invoices

Data Ingestion

Zenskar ingests your metrics in the following two ways:

  1. Data Sources: We pull data from your data infrastructure, as shown in Fig. 2
  2. Raw Metrics: You push data to Zenskar, as shown in Fig. 4

Data Sources

Fig. 2: Raw data or aggregates ingested via configured data source

Fig. 2: Raw data or aggregates ingested via configured data source

To add and configure a Data Source, navigate to Metering > Data Sources in the side navigation menu:

Do the following to complete the process:

  1. Set Up Source: Provide a Source Name and select a Source Type from the drop-down
  2. Source Config: Provide configuration details and credentials to establish connection with the data source.

👷

Engineering effort required

The details needed for this step are source-type dependent. Therefore, some technical knowledge is expected to complete this step.

👍

Security

Zenskar takes the security of your data seriously. All connection details and credentials are stored in a secure encrypted vault.

Raw Metrics

Fig. 3: Raw metrics or aggregates ingested via Zenskar API

Fig. 4: Raw metrics or aggregates ingested via Zenskar API

Your can seamlessly integrate Zenskar in your application workflow by configuring a data-ingestion API. You can call this API like any other REST API and help Zenskar ingest your raw metrics or aggregates. Zenskar expects the input data in JavaScript Object Notation (JSON) format: a universal data-interchange format like XML. Some examples are shown below:

  1. If you are providing a compute resource (for example, CPU time) for rent, a JSON raw metric may look like the following:
{
    "data": {
        "cpu_usage_hours_per_day": "5"
    },
    "timestamp": "2023-05-21 09:30:38.731",
    "customer_id": "[email protected]"
}
  1. If you are providing a compute resource (for example, CPU time) for rent, a JSON aggregate data may look like the following:
{
    "data": {
        "cpu_usage_hours_per_month": "300"
    },
    "timestamp": "2023-05-21 09:30:38.731",
    "customer_id": "[email protected]"
}

The customer_id and timestamp are mandatory fields. A combination of these mandatory fields provides Zenskar the capability to uniquely identify a raw metric or an aggregate.

Sample API calls to send raw metrics or aggregate data to Zenskar would look like the following. The examples below use curl utility which can act as a REST client. However, the same can be done from any REST client:

curl --location --request POST 'https://api.zenskar.com/usage/test_1' \
-H 'organisation: 23507783-d5b4-4854-b012-2bf4ea922205' \
-H 'authorization: Bearer b3VyY2VfYWNjZXNzIjp7OiIwOTkxNTEyYy0wMzI3LTQzMzEtYjQ1NS1jNzgxZjg2YWRjNzMiLCJlbWFpbF92ZXJpZmllZCI6dHJ1ZSwibmFtZSI6Ik1vaGFuIFNyZWVrYW50IiwicHJlZmVycmVkX3VzZXJuYW1lIjoibW9oYW5AemVuc2thci5jb20iLCJnaXZlbl9uYW1lIjoiTW9oYW4iLCJmYW1pbHlfbmFtZSI6IlNyZWVrYW50IiwiZW1haWwiOiJtb2hhbkB6ZW5za2FyLmNvbSJ9.K5fuK2fBlzJjQQ06Etu_H3syFjT3Q7ZtZzQqFi51YDkdJCDc-_l9JQakXWRIyeXiAM4XmbnuOorUUItNYopwfD033kQL5aI6vPnDR5DoRRkelwTam7bn94Ce83IgXbzzi0rLBzhC47lMStj-CLnNY2qdEAFRyrL8eWDsjE_8IXUB-LYb1V9I3dMOpgme77zLKySo3dHYQkBC-rZc9sNTdnaLdZy7nLrJ2-uoDvPvtOq6JLeR5ozf3bn5XEBhaHRzY23VMAgOX64-Ow_CAeVng0xIEq5P2CgqFDuNFdT3gfB8-qjKlPHdC_ndAAGGcyxbcCwrhKMgwsP6q2ZvaFDwYA' \
-H 'Content-Type: application/json' \
--data-raw '{"data":{"cpu_usage_hours_per_day":5},"timestamp":"2023-05-21 09:30:38.731","customer_id":"[email protected]"}'

curl --location --request POST 'https://api.zenskar.com/usage/test_1' \
-H 'organisation: 23507783-d5b4-4854-b012-2bf4ea922205' \
-H 'authorization: Bearer b3VyY2VfYWNjZXNzIjp7OiIwOTkxNTEyYy0wMzI3LTQzMzEtYjQ1NS1jNzgxZjg2YWRjNzMiLCJlbWFpbF92ZXJpZmllZCI6dHJ1ZSwibmFtZSI6Ik1vaGFuIFNyZWVrYW50IiwicHJlZmVycmVkX3VzZXJuYW1lIjoibW9oYW5AemVuc2thci5jb20iLCJnaXZlbl9uYW1lIjoiTW9oYW4iLCJmYW1pbHlfbmFtZSI6IlNyZWVrYW50IiwiZW1haWwiOiJtb2hhbkB6ZW5za2FyLmNvbSJ9.K5fuK2fBlzJjQQ06Etu_H3syFjT3Q7ZtZzQqFi51YDkdJCDc-_l9JQakXWRIyeXiAM4XmbnuOorUUItNYopwfD033kQL5aI6vPnDR5DoRRkelwTam7bn94Ce83IgXbzzi0rLBzhC47lMStj-CLnNY2qdEAFRyrL8eWDsjE_8IXUB-LYb1V9I3dMOpgme77zLKySo3dHYQkBC-rZc9sNTdnaLdZy7nLrJ2-uoDvPvtOq6JLeR5ozf3bn5XEBhaHRzY23VMAgOX64-Ow_CAeVng0xIEq5P2CgqFDuNFdT3gfB8-qjKlPHdC_ndAAGGcyxbcCwrhKMgwsP6q2ZvaFDwYA' \
-H 'Content-Type: application/json' \
--data-raw '{"data":{"cpu_usage_hours_per_month":300},"timestamp":"2023-05-21 09:30:38.731","customer_id":"[email protected]"}'

The JSON data you send us is securely stored in our database as a table after the requisite transformations are applied:

📚

Database Table

Tables are database objects that contain all the data in a database. In tables, data is logically organized in a row-and-column format similar to a spreadsheet.

👷

Engineering effort required

Integrating Zenskar data-ingestion REST API in your application workflow will require engineering effort. However, to test all you need is a REST client. Prominent examples of REST clients are Postman (feature-rich) and Thunder Client (lightweight).

To add and configure a data-ingestion API, navigate to Metering > Raw Metrics page and click on the /ADD RAW METRIC button, as shown in .

Setting up a Zenskar data-ingestion API from ingesting raw metrics and aggregates.

Fig. 5: Set up data-ingestion API

The newly added data-ingestion API will be listed on the Raw Metrics page.

In addition to setting up data-ingestion APIs for automation, Zenskar lets you upload raw metrics in comma-separated values (CSV) files manually. Simply click on the UPLOAD CSV button and choose the desired CSV.

Data Processing

Raw Metrics versus Aggregates

You have seen how to configure data sources and data-ingestion APIs, also called Data Source and Raw Metrics, respectively, in Zenskar parlance. There are two types of data that these sources can send: raw metrics and aggregates. It is important to remember that aggregates are actionable data whereas raw metrics are not. If the data ingested by Zenskar is raw metrics, further filtering is necessary to derive aggregates, as shown in Fig. 8.

📖

The Structured Query Language

Zenskar stores all the data in databases. Therefore, the power of the Structured Query Language (SQL) must be employed to derive and\or retrieve aggregates from the databases. If the source data is raw metrics, SQL queries must be written to derive and retrieve specific aggregates for calculations. However, if the source data is aggregates, SQL queries must only retrieve specific aggregates for calculations.

Fig. 8: Deriving aggregates from raw metrics by applying necessary filters

Fig. 8: Deriving aggregates from raw metrics by applying necessary filters

Deriving Aggregates from Raw Metrics

Zenskar provides you two ways to create SQL queries to derive and\or retrieve aggregates. Once you pick your source, a list of relevant tables are displayed. You can define SQL queries on these tables to derive or retrieve the aggregates needed to generate an invoice. There are two ways to create SQL queries:

  1. Visual Query Builder
  2. SQL Aggregator

Visual Query Builder

Visual Query Builder is the no-code solution to aggregate customer's usage metrics. Following are the steps needed to build an aggregate query:

  1. Select the Table containing data you want to aggregate
  2. Select the Customer Column: this is the column that contains unique identifiers for customers
  3. Select the Zenskar Column which corresponds to the Customer Column
  4. Select the Usage Column which contains usage metrics
  5. Select the Aggregate Type: AVG (average), COUNT, MAX (maximum), MIN (minimum), and SUM are the filters available

SQL Aggregator

Zenskar allows you to take complete control over the SQL queries using the SQL Aggregator tool.

Raw events

After importing data via raw metrics or data sources, it is necessary to arrange the data in a specific format based on the billing method. To do this, you can write a SQL query to fetch the data in a way that meets your billing requirements.

For instance, in subscription-based pricing models, customers may be charged on a recurring monthly basis, but the usage data may be recorded daily. To ensure accurate billing information, the usage data must be aggregated into a monthly basis.

You can pick Usage aggregates, if your data if already in a desired format and skip to customer specific query.

Customer specific query

After aggregating the data into the required format, you can write a SQL query to extract the customer specific usage. This query should produce a single quantity that can be used to calculate the customer's bill.

An example of such a query can be seen in the image below, this query tells the system that whenever a customer name (customer.id) is provided it will extract its usage data (data.id) and give a singular key metric.