Grouping Logs for a Unified View

correlated-logs

correlated-logs

If you are using AWS Lambda or Serverless and you want to correlate logs from your services, then this release is for you.

When using AWS Lambda, you typically have multiple Lambda functions or services that cooperate to respond to a single client request. This means that the log data for a single request is scattered over multiple AWS CloudWatch log groups. Diagnosing a single request by tracing the log events for that request in all the relevant log groups can be difficult and time consuming.

SenseDeep 2.3 addresses this problem by supporting meta logs to provide a unified, correlated view over multiple log groups.

SenseDeep combines multiple AWS CloudWatch log groups into unified, correlated log views. Perfect for Lambda to quickly pinpoint errors in serverless architectures.

We're happy to announce SenseDeep 2.3 which has some great new features including:

  • Meta logs that merge multiple log groups into a unified, correlated log view.
  • Log view activation control to temporarily deactivate views while keeping configuration.
  • Greatly optimized cloud and log synchronization. Faster, increased parallelism and more responsive.
  • Many minor UI improvements.
  • Improved subscription dialog with option to rerun the IAM Template.
  • Icon to copy log events to the clipboard.
  • Option to reload logs or clouds of logs.
  • Improved Viewer first time render of a log view.
  • Reduced default log max age to 7 days to focus on most recent data.

Meta Logs

A SenseLog meta log will combine multiple AWS CloudWatch log groups for different services into a unified and correlated log view. This single log can then be viewed and queried like any normal SenseDeep log view.

To create a meta log, click on the Logs icon in the left sidebar and then click "Add". Enter your log name, maximum amount of data to store and the maximum age of data.

correlated-logs

You can explicitly select the individual contributing logs by clicking the checkbox next to each dependent log. Alternatively you can enter a regular expression pattern that will be used to dynamically match contributing logs. Using a regular expression pattern is preferable if you have a changing set of log group names.

High Cardinality IDs

To get the most out of meta logs, you should define a unique request ID that is passed to all your Lambda functions and then emitted in all log events. This is sometimes called a High Cardinality ID. Using this ID, you can filter log events on this ID and display only those events. A single double click in the SenseDeep Viewer on the request ID will isolate and display the entire request over all participating services.

Match Logs

When creating a meta log you can explicitly select the individual contributing dependent logs. Alternatively you can select the logs using a dynamic regular expression pattern. Using a regular expression pattern is preferable if you have a changing set of log group names.

Fixes

This release also fixes:

  • Double click on Lambda report events to filter.
  • Log unwanted log pruning when changing organization.
  • Automatic upgrading to new versions.

Summary

SenseDeep is a fast and easy serverless troubleshooting service. SenseDeep displays Lambda results in real-time with full metrics, analysis and application log data in context. It offers alarms, alerts and notifications for 24x7 monitoring.

Getting Started

There is nothing to install. Just navigate your browser to: https://app.sensedeep.com/

To learn more about SenseDeep and how to use the app, please read the documentation at: https://www.sensedeep.com/doc/.

Please let us know what you think, we thrive on feedback. dev@sensedeep.com.

© SenseDeep® LLC. All rights reserved. Privacy Policy and Terms of Use.

Consent

This web site uses cookies to provide you with a better viewing experience. Without cookies, you will not be able to view videos, contact chat or use other site features. By continuing, you are giving your consent to cookies being used.

OK