Query Builder

Introduction

Query Builder in SigNoz simplifies the process of filtering, aggregating, and visualizing data, making complex queries accessible to all users. This documentation will walk you through features of the Query Builder, showing you how to create detailed queries, visualize results, and understand your data better.

Logs and Traces Query Builder

Filtering

The Query Builder in SigNoz allows users to apply filtering based on various attributes of logs or traces.

Using the Filtering Feature

Access: Navigate to the Query Builder.
Open Filter Options: Click on the Search Filter field to open a dropdown list of available attributes.
Select Attributes: Choose the attribute you want to filter by (e.g., service.name, level, status). Start typing to filter the list.
Apply Operators: Choose the operator (e.g., =, !=, IN, NOT_IN) for the selected attribute.
Input Values: Input the value(s) you want to filter by.
Combine Filters: You can add multiple filters by repeating the above steps. Multiple filters are combined with AND.
View Results: Click the Stage & Run Query Button to display the logs or traces that match the applied filters.

Example

Suppose you want to filter logs from a specific service with a particular severity level. You can achieve this using the following filters:

selecting the service name. service.name = demo-app
selecting the severity text, severity_text = DEBUG

A gif explaining the Filtering feature in SigNoz Query Builder — *Example showing the filtering feature of SigNoz Query Builder*

Aggregation and Grouping

The Query Builder's Aggregation and Grouping feature enables you to combine data points and categorize them for complex insights.

Aggregation

Aggregation allows you to perform calculations on a set of values to return a single value. With the Query Builder, you can:

Count: Count the number of times a particular event occurs.
Count Distinct: Count unique occurrences of a specified attribute.
Sum: Calculate the total for a numerical attribute.
Avg: Find the average value of a numerical attribute.
Max/Min: Determine the maximum or minimum value of a numerical attribute in an aggregation interval.
Percentiles (P05, P10, P90, etc.): Understand the distribution of your data with percentile calculations.
Rate: Measure the frequency of occurrence within a given time frame. It is often expressed as a ratio, where the numerator represents the number of occurrences, and the denominator represents the length of the time period.
Rate Sum: The rate of change in the sum of the selected attribute over the specified rate period.
Rate Avg: The rate of change in the average of the selected attribute over the specified rate period.
Rate Min/Max: The rate of change in the minimum or maximum value of the selected attribute over the specified rate period.

Grouping

A gif explaining the Group by feature in SigNoz Query Builder — *Group by feature of SigNoz Query Builder*

Grouping allows you to segment your data based on chosen attributes, facilitating comparative analysis across different categories of the selected attributes:

You can group data by attributes like service.name or method to analyze patterns per service or HTTP method.
When combined with aggregation, grouping enables you to, for example, find the average response time per service or count errors per endpoint.

Using Aggregation and Grouping Together

Select an aggregation function from the dropdown, such as Avg.
Then, choose an attribute to apply it to, like response_time.
To group the data, you would then specify an attribute in the Group by field.
After running the query, you'll see the average response times for each group, allowing you to identify areas that may need optimization.

Example

To analyze the average response time of services within a system, we can do so in the following way:

In the aggregation dropdown, select Avg.
For the attribute to aggregate, choose durationNano.
To see the average duration for each of the service, enter serviceName in the Group by field.

A gif explaining the how to use Group by and Aggregation features together in SigNoz Query Builder — *Example combining Group By and Aggregation features*

Result Manipulation

Result Manipulation is a set of features in the Query Builder that enables you to refine your query. The features include:

Order By

Order your query results based on a specified attribute in either ascending or descending order. This can help in identifying the highest or lowest values in your data, such as the most frequently occurring errors.

Aggregate Every

Define the interval over which to aggregate data. For example, you can aggregate count data in 60-second intervals to get a per-minute count of events.

Limit

Set a limit on the number of results returned. This is useful when you only want to see the top N results, such as the top 10 most visited endpoints.

Having

Apply conditions to filter the results further based on aggregate value.

Legend Format

Customize the legend in your query's visual output to give more clarity, by formatting how grouped data will be labeled in your charts or graphs. We use the double curly braces - {{}} format to show the attribute.

For example, if you have grouped by serviceName attribute then you can write {{serviceName}} in your legend. You can also add text along with the attribute like {{serviceName}} - This is a service** will show be shown as sampleService - This is a service name.

Example

Suppose you want to find the top 2 endpoints with an average response time greater than 500 ms. The query is configured as follows:

Group by: serviceName
Aggregate: Avg on durationNano
HAVING: AVG(durationNano) >= 500000000 to filter for endpoints with average response times exceeding 500 ms
Order by: avg(durationNano) desc to list services with the highest response times first
Limit: 2 to focus on the top 2 services with the longest response times
Legend Format: {{serviceName}} to display the service name in the visualization legend

A gif explaining some more features in SigNoz Query Builder — *An example showcasing Order by, Limit, HAVING and Legend Format features of SigNoz Query Builder*

Multiple Queries and Functions

The SigNoz Query Builder allows you to run multiple queries simultaneously and perform functions on them. This feature facilitates analysis of complex data, such as comparing data or calculating ratios.

Multiple Queries

Create and run multiple independent queries within the same view. Each query can have its own set of filters, aggregations, and groupings. This is particularly useful for analyzing different dimensions of your data in parallel.

Functions on Queries

Apply mathematical functions to the results of your queries. This allows you to derive new insights by performing operations like addition, subtraction, multiplication, division, or more complex functions on your data.

List of supported functions

exp
log
ln
exp2
log2
exp10
log10
sqrt
cbrt
erf
erfc
lgamma
tgamma
sin
cos
tan
asin
acos
atan
degrees
radians
now

Example

Suppose you want to determiner the percentage of logs that contains errors relative to the total log count. You can achieve this using Multiple queries and a function as follows:

Query A: Filters logs to count the instances where the body contains the word error.
Query B: Represents a baseline count of all logs.
function F1 A**100/B

A gif explaining multiple queries and functions features in SigNoz Query Builder — *Multiple Queries and Functions features of SigNoz Query Builder*

Metrics Query Builder

SigNoz's Metrics Query Builder has all the features mentioned above like filtering, aggregation, Order by, Multiple Queries etc. but it also has some additional functionalities which can enable users to delve deeper into metrics analysis with ease.

Spatial and Temporal Aggregations

Efficiently aggregate metrics data across time and various dimensions to gain comprehensive insights into your system's performance.

Temporal Aggregation

Temporal aggregation simplifies metric analysis by consolidating data points over specific time periods. It's essential for managing data from applications that transmit metrics at regular intervals, helping to reduce the volume of data points into a more interpretable format. This aggregation is particularly crucial when dealing with long-term data visualization, where displaying each data point is neither practical nor informative due to limited screen space.

Spatial Aggregation

Modern applications will have many time series across dimensions such as container names, service names, etc.

While temporal aggregations are meant for reducing data points across a single time axis, spatial aggregation refers to the technique of merging metric data across various dimensions or labels, such as container names, service names, or geographical regions.

Example

If you select a counter metric called signoz_calls_total, you can apply temporal aggregations like Rate and Increase on it. Similarly, you can apply spatial aggregations across many dimensions, such as service_name, operation, deployment_environment, etc.

Spatial & temporal aggregations in the metrics query builder of SigNoz — *How to apply temporal and spatial aggregations in metrics query builder of SigNoz*

Functions for Extended Data Analysis

To extend the data analysis capabilities of the Metrics Query Builder, you can use four different categories of functions.

Function Types:

Exclusion Functions: These allow users to exclude data points based on certain conditions, such as values falling outside a specified range. This helps in focusing on the most relevant data. For example, Cut Off Min excludes data below a threshold, while Cut Off Max ignores values above a certain point.
Arithmetic Functions: Users can perform mathematical operations on data points, enabling the customization of metric calculations to suit specific analysis needs. For example, functions like Absolute can return the absolute value of data points, and logarithmic functions such as Log2 or Log10 can transform data to a log scale for better comparison of wide-ranging numbers.
Smoothing Functions: To deal with volatile data, smoothing functions can be applied, such as moving averages, which help in identifying underlying trends by reducing noise. For example, functions like EWMA 3/5/7 (Exponentially Weighted Moving Average) can smooth data fluctuations over 3, 5, or 7 periods.
Time Shift Functions: These functions enable comparisons of data across different time periods, useful for analyzing trends, changes, or anomalies over time. For example, using the Time Shift function for shifting a series by a fixed amount of seconds to compare current data with past data.

*Functions in metrics query builder will help extend data analysis capabilities on metrics*

Chain Functions:

Combine two or more functions, like applying a logarithmic scale after setting a minimum cutoff, to tailor your metrics.

Chained functions — *Chain two functions to get your required metrics*

Example

This example uses the Time Shift function to compare total calls for frontend service in an interval of 1hr.

You can plot charts for the same service an hour apart.

Compare performance of services — *Check the performance of frontend service an hour apart with the help of time shift function*

Users can use the time shift function for very practical use cases, like comparing performance metrics before and after a deployment.

To know more details about the Metrics Query Builder, you can checkout these resources from the first SigNoz Launch Week:

Was this page helpful?

Manage Variables

Metrics