SenseDeep DynamoDB Observability
I'm happy to announce exciting new DynamoDB capabilities in SenseDeep to give you deep insights and observability into your single-table DynamoDB designs.
This release breaks new ground with detailed metrics and visualizations for your DynamoDB implementations.
Best practices for DynamoDB have evolved to favor single-table patterns where one database table serves the entire application and holds multiple different application entities. But there is a dearth of tools to support single-table designs.
Previously, single-table designs with DynamoDB has been a black box and it has been difficult to peer inside and see how the components of your design are operating.
This release has new unique capabilities for single-table design patterns that unlock the hidden inner workings of your DynamoDB designs.
The new release of SenseDeep offers the following DynamoDB features:
- Discovery of your DynamoDB tables in connected clouds and regions.
- Per-table statistics and metrics tracking table size and number of items stored.
- Single-table item-level metrics for Tables, Invoking functions, Indexes, Entities and Operations.
- Per-query statistics and metrics. Profile individual queries in your serverless apps.
- Visualizations for account level DynamoDB utilization and limits.
- Operation metrics for DynamoDB API calls per table.
- Provisioning and capacity planning visualizations and metrics.
The scalable single-table metrics permit drill-down from the Table level down to the application entity with aggregated metrics for each level.
So what are the kinds of DynamoDB questions can SenseDeep answer?
- Which single-table entity/model is causing the most load and is consuming the most RCU or WCU?
- Which customer tenant is causing the most load and how much should they be billed?
- Which app or function is causing what percentage of load on DynamoDB and is consuming the most RCU or WCU?
- Which queries are the most inefficient (items vs scanned) and by which app or model?
- Which operations are being used the most?
- Which entity is using performing scans or other operations?
These questions and others are now easily answered.
In the side-menu, you'll see the new option for
Databases. This option displays the DynamoDB table list with account level and per-table metrics.
The page displays graphs and metrics for your DynamoDB account usage. The utilization graphs are for the account-level read and write provisioned utilization. The Max Account read / write show your account configurated limits.
Below the graphs is a list of all your DynamoDB tables that displays key information and metrics for each table over the selected period including the size of the table and number of items stored, as well as the billing mode.
To drill-down, click on a table to see per-table metrics.
For any table, SenseDeep can display standard AWS DynamoDB metrics to show read and write usage with the provisioned capacity overlaid in red.
The per-operation stats are displayed in a table at the bottom.
Now, things get event more interesting if you enable single-table metrics.
These libraries capture and emit detailed single-table metrics that track requests at function, index, application entity and operation levels.
See Understanding Your DynamoDB Single-Table Performance to learn more.
The SenseDeep single-table DynamoDB page displays metrics across 5 dimensions:
- Table — Per table metrics.
- Tenant — Per customer tenant metrics.
- Source — Per application, module or function identification.
- Index — Primary or global secondary index.
- Model — Application single-table entity / model name.
- Operation — DynamoDB low-level operation: GetItem, PutItem, etc.
You can drill-down through these dimensions to see metrics aggregated by table, tenant, source, index, model or operation. This enables you to pin-point exactly where performance issues may be lurking.
For each of these dimension combinations, SenseDeep displays the following metrics:
- read — Read capacity units consumed.
- write — Write capacity units consumed.
- latency — Aggregated request latency in milliseconds.
- count — Count of items returned.
- scanned — Number of items scanned.
- requests — Number of API requests issued.
With these metrics, you can see precisely who is consuming read and write capacity, which requests are running long, and which requests are inefficient and are scanning the table.
Enabling Single Table Metrics
Alternatively if you have an existing DynamoDB code base, you can use the DynamoDB Metrics library that captures and emits the same detailed DynamoDB metrics for existing code bases that use single-table designs.
On the Single-Table page, you can drill-down by clicking on any row in the table at the bottom of the page.
Initially, the table displays the Lambda functions or sources of the metrics. This enables you to see which application or function is loading your database table.
By clicking on a row, you can display metrics for the table primary and secondary indexes. Click again to display the application entities and so on.
As you drill-down, the graphs and metrics at the top of the page will display information for your selected choice.
Go back up the chain by clicking on the cookies to the right of the table filter box.
Selecting Time Periods
For any page, you can select the desired time range for your metrics. You can select last hour, day, week or month or any custom period.
Provisioning and capacity planning
SenseDeep also provides intuitive capacity planning and provisioning assistance.
The displayed graphs show DynamoDB table read and write capacity utilization with the provisioned capacity in red. You can get this view for the primary index or any secondary index.
If you are using on-demand provisioning, the provisioning is an estimate of what would be required if you switched to provisioned billing with a 70% utilization buffer.
Below the graphs is a section that compares two billing scenarios: current and estimated.
The current scenario displays the costs with your current billing plan (OnDemand vs Provisioned). The costs for your read and write capacity usage with storage costs are displayed.
The estimated scenario displays costs if you were to switch your billing plan from OnDemand to Provisioned or vice-versa. If you would gain by switching plans, a summary of benefits is displayed.
DynamoDB provisions indexes separately from the primary table. While your billing mode must be consistent across the primary and secondary indexes, you can vary the capacity and auto-scaling for each index.
You can analyze the provisioning and costs for secondary indexes by clicking on the index in the table at the bottom of the page.
Under the Hood
The OneTable and DynamoDB Metrics libraries emits metrics using the CloudWatch EMF log-based metrics format. This permits zero-latency creation of metrics without impacting the performance of your Lambdas. EMF allows metrics to be emitted without blocking as would be the case with a normal blocking API.
Gaining insight into single-table design patterns is the new frontier for DynamoDB and we've got more exciting new capabilities coming soon for DynamoDB. Tell us if there is anything you'd like
You may also like to read:
SenseDeep is an observability platform for AWS developers to accelerate the delivery and maintenance of serverless applications.
SenseDeep helps developers through the entire lifecycle to create observable, reliable and maintainable apps via an integrated serverless developer studio that includes deep insights into how your apps are performing.
To try SenseDeep, navigate your browser to: https://app.sensedeep.com.
To learn more about SenseDeep please see: https://www.sensedeep.com/product.
Please let us know what you think, we thrive on feedback. email@example.com.