Manage tables, tiers, and retention in Sentinel | Microsoft Learn
In brief
The article now covers managing tables, tiers, and retention, including supported table types, XDR and Basic Logs management limitations, retention tiers, regional availability, ISOC guidance, and updated permissions links.
What Defender admins need to know
Administrators can use the revised guidance to identify which tables they can manage in the Defender portal and where additional management is required.
Summaries are generated from the documentation change itself.
Documentation change
The comparison below shows only the changed extract. Use the full-page view for complete context.
Configure Table SettingsManage tables, tiers, and retention in Microsoft Sentinel | Microsoft Learn
The Microsoft Defender portal provides a centralized experience for configuring table-level data retention and tier settings across Microsoft Sentinel and Microsoft Defender XDR. You can view and manage retention settings, switch between Analytics and Data Lake tiers, and optimize storage based on operational and cost requirements.
ThisFor an overview of the Data & connectors experience, see Manage data in Microsoft Sentinel and Defender.
Note
If you use Integrated Security Operations Center (ISOC), see Manage tables, tiers, and retention in ISOC. Review Data ingestion and billing for ISOC in Microsoft Defender for eligibility and included retention before changing table settings. The general retention ranges in this article explains how to configuredon't establish which settings are supported for an individual ISOC XDR table. The XDR retention thresholds and tier settings forcost examples describe the existing Microsoft Sentinel and Defender XDR tables iningestion experience, not the Microsoft Defender portal. It also explains how to use Table insights, a built-in monitoring feature that surfaces ingestion health data for each table in your workspace.
For more information about how data tiers andISOC retention work, see Manage data tiers and retention in Microsoft Sentinel.benefit.
Prerequisites
- To use data lake retention, your workspace must be in a supported region.
- To manage Defender XDR hunting tables, you need to onboard to Microsoft Sentinel in the Defender portal. For more information, see Connect Microsoft Sentinel to the Microsoft Defender portal.
- To use Table insights, you need read access to the Tables page in your workspace. Table insights requires the same permissions as viewing table settings. For more information, see
Permissions required.Table insights permissions.
Permissions required
For more information about Microsoft Sentinel workspace permissions, see Roles and permissions in the Microsoft Sentinel platform.
Table insights
For permissions
Use the following permissions when you want to investigate ingestion healthtable health, see Table insights permissions.
Which tables can you manage in Table insights or take action from a flagged table.the Microsoft Defender portal?
This section describes the table types you can manage in the Microsoft Defender portal.
| Is in Microsoft Sentinel workspace? | |||
|---|---|---|---|
AzureDiagnostics, SigninLogs- Microsoft Sentinel tables: AWSCloudTrail, SecurityAlert- XDR tables: DeviceEvents,AlertInfo | Yes | ||
| Tables you create manually or |
_CL or _SRCH suffixes. |
||
| Tables in the XDR default tier, which have 30 days of analytics retention |
IdentityInfo |
||
Custom roles must include equivalent readNote
You can view Basic Logs tables in your Microsoft Sentinel workspace from the Microsoft Defender portal, but you can currently manage them only from your Log Analytics workspace. To manage these tables from the Microsoft Defender portal, change the table plan from Basic Logs to Analytics in your Microsoft Sentinel workspace.
How data tiers and write permissionsretention work
For regional support, see Data lake retention availability.
You can retain data in Microsoft Sentinel in one of two tiers:
Analytics tier: This tier makes data available for alerting, hunting, workbooks, and all Microsoft Sentinel features. It retains data in two states:
- Analytics retention: In this "hot" state, data is fully available for real-time analytics, including high-performance queries, analytics rules, and threat hunting. By default, Microsoft Sentinel and Microsoft Defender XDR retain data in this tier for 30 days. You can extend the retention period of all tables to up to two years at a prorated monthly long-term retention charge. You can extend the retention period of Microsoft Sentinel solution tables to 90 days for free.
- Total retention: By default, all data in the analytics tier is mirrored to the data lake for the same retention period. You can extend the retention of your data in the lake beyond the analytics retention, for up to 12 years of total retention at a low cost.
Data Lake tier: In this low-cost "cold" tier, Microsoft Sentinel retains your data in the data lake only. Data in the Data Lake tier isn't available for real-time analytics features and threat hunting. However, you can access data in the lake whenever you need it through KQL jobs, analyze trends over time by running scheduled KQL or Spark jobs, and aggregate insights from incoming data at a regular cadence by using summary rules.
XDR data: By default, Microsoft Defender XDR threat hunting data is always available in the Analytics tier for 30 days. You can extend retention of this data in the Analytics tier up to 90 days, which incurs ingestion costs, while storage remains free. Extending retention beyond 90 days in the Analytics tier also incurs storage costs. You can also ingest exclusively into the Data Lake tier, but the data remains available in the Analytics tier for 30 days. Ingesting XDR data directly into the Data Lake tier is more cost-effective, but it involves ingestion, storage, and processing costs. In this option, customers still get 30 days' worth of data in the Analytics tier at no additional cost.
For more information about the differences between the Analytics tier and the Data Lake tier, see Compare the analytics and data lake tiers.
The following diagram illustrates the retention components of the Analytics, Data Lake, and XDR default tiers, and the table types that apply to each tier:
For more information about the Microsoft Sentinel data lake, see What is Microsoft Sentinel data lake.
Compare the analytics and data lake tiers
This table compares the Analytics and Data Lake tiers and their key characteristics:
| Comparison | Analytics tier | Data Lake tier |
|---|---|---|
| Key characteristics | High-performance querying and indexing for logs (also known as hot, or interactive retention). | Cost-effective long-term retention of large data volumes (also known as cold storage). |
| Best for | Real-time analytics rules, alerting, hunting, workbooks, and all Microsoft Sentinel features. | - Compliance and regulatory logging.- Historical trend analysis and forensics.- Low-touch data that's not needed for real-time alerts. |
| Ingestion cost | Standard | Minimal |
| Query price included | ✅ | ❌ |
| Optimized query performance | ✅ | ❌ Slower queries.Good for auditing. Not optimized for real-time analysis. |
| Query capabilities | Full query capabilities in the Microsoft Defender portal, the Azure portal, and by using APIs. | - Full query capabilities, including unions and joins.- Run scheduled KQL or Spark jobs.- Use notebooks. |
| Full set of real-time analytics features | ✅ | ❌ Limitations on some features, including analytics rules, hunting queries, parsers, watchlists, workbooks, and playbooks. |
| Search jobs | ✅ | ✅ |
| Summary rules | ✅ | ✅ Full KQL on a single table, which you can extend with data from an analytics table using lookup |
| Restore | ✅ | ❌ KQL and Notebook jobs can promote data to the analytics tier. |
| Data export | ✅ | ❌ |
| Retention period | 90 days for Microsoft Sentinel, 30 days for Microsoft Defender XDR. Can be extended to up to two years at a prorated monthly long-term retention charge. | Same as analytics retention, by default. Can be extended to up to 12 years. |
What happens when you modify table settings
You can switch a table's tier and retention settings at any time.
When you change a table's tier from Analytics to Data Lake, all real-time analytics and hunting queries stop working.
When you shorten a table's total retention, Microsoft waits 30 days before removing the data, so you can revert the change and avoid data loss if you made a configuration error.
When you increase total retention, the new retention period applies to all data that was already ingested into the table and wasn't yet removed.
When you change the analytics retention settings of a table with existing data, the change takes effect immediately.
Example:
- You have a table in the Analytics tier with 180 days of analytics retention. By default, total retention is also set to 180 days.
- You change analytics retention to 90 days without changing total retention, which remains 180 days.
- Microsoft Sentinel automatically removes the last 90 days of data from analytics retention, but continues to store data that's 90-180 days old in the Data Lake tier.
Manage XDR data in Microsoft Sentinel
Before changing retention, review the table's current settings and health investigation, including the warnings in the Microsoft Defender portal. For ISOC-specific retention and Microsoft.OperationalInsights/workspaces/tables/readbilling, use the ISOC guidance rather than the retention thresholds and cost examples in this section.Microsoft.SecurityInsights/*/read
By default, Microsoft Defender XDR retains threat hunting data actions.in the XDR default tier for 30 days. This data isn't ingested into the Analytics or Data Lake tiers by default. If you extend the retention period of supported XDR tables beyond 30 days and up to 90 days, Sentinel ingestion costs apply, but there are no additional storage costs. The tables are created in your Microsoft Sentinel workspace in the Analytics tier and mirrored to the Data Lake tier.
If you enable the Microsoft Sentinel XDR connector in the Azure portal, the tables you select during setup are automatically ingested into the Analytics tier and mirrored to the Data Lake tier. The default retention is 30 days, and you can extend it up to 12 years. For a list of tables, see Microsoft Defender XDR integration with Microsoft Sentinel. You can ingest supported XDR tables that you didn't select during connector deployment into the Analytics tier and mirror them to the Data Lake tier by setting retention to more than 30 days.
If you don't enable the Microsoft Sentinel XDR connector, XDR tables aren't automatically ingested, but you can still ingest them by setting Analytics or Data Lake tier retention to more than 30 days in the Microsoft Defender portal.
You can choose to ingest supported XDR tables exclusively into the Data Lake tier by selecting the Data Lake tier option when configuring retention settings. For more information, see Configure data retention and tiering.
Warning
Resetting analytics retention and total retention to the default 30 days disables the connector in the Azure portal.
To stop ingesting data into the Analytics tier, reset analytics retention and total retention to the default 30 days.
For more information about managing your tables and data, see Manage table settings.
XDR data retention and costs
The following tables summarize the free retention periods and cost implications for the different tiers in Microsoft Sentinel:
| Tier | Retention | Notes |
|---|---|---|
| Advanced hunting (default) | 30 days | Default, included in the XDR license. |
| Analytics tier | 31-90 days | Free storage for Sentinel-enabled workspaces. Data is mirrored to the data lake. Sentinel ingestion charges apply. |
| Data Lake tier | Configurable. By default, the same as the analytics tier. | Free storage when total retention is the same as analytics tier retention.Retaining data in the Data Lake tier beyond analytics tier retention incurs Data Lake tier storage costs.Ingesting data directly into the Data Lake tier incurs ingestion, storage, and processing costs. |
For more information about billing and costs, see Understand the full billing model for Microsoft Sentinel.
In the following examples, XDR data is available through advanced hunting for at least 30 days, regardless of the retention settings in the analytics or data lake tiers.
| Analytics tier retention | Total retention | Analytics tier ingestion costs | Analytics tier storage costs | Data Lake tier costs |
|---|---|---|---|---|
| 30 days default | 30 days default | No additional costs | N/A | N/A |
| 90 days | 90 days | Costs apply for analytics tier ingestion. | No additional costs. 90 days included free. | No additional costs. Total retention matches analytics tier retention. |
| 90 days | 180 days | Costs apply for analytics tier ingestion. | No additional costs; 90 days included free. | Costs apply for 90 days of additional data lake retention (180 - 90 days). |
| 180 days | 1 year | Costs apply for analytics tier ingestion. | Costs apply for 90 days of additional analytics tier retention. | Costs apply for 185 days of additional data lake retention (365 - 180 days). |
| 0 days (data lake only) | 5 years | N/A | N/A | Costs apply for ingestion and for 5 years of data lake retention. |
Manage table settings
The **Manage table** screen lets you modify retention settings in the current tier and change the storage tier, if necessary.
Available settings depend on the table. Review the options and warnings shown for the selected table rather than assuming that every table supports the full ranges listed here.
[](media/manage-data-overview/manage-table-settings-microsoft-defender-portal.png#lightbox)
- **Analytics tier retention settings**:
Tier changes aren't available for all tables. For example, some XDR and Microsoft Sentinel solution tables must remain in the Analytics tier because Microsoft security services require the data in these tables for near-real-time analytics.
For more information about retention and tier settings, see [ManageHow data tiers and retention in Microsoft Sentinel](manage-data-overview).work.
Review warnings and messages. These messages help you understand important implications of changing table settings.
Examples include: - Advanced hunting - Analytics rules - Custom detection rules
\ No newline at end of file 6. Select Save to apply the new settings.
Use Table insights to monitor table health
Table insights is a built-in monitoring view on the Tables page in the Microsoft Defender portal. It gives security and platform engineers a single table-level view of ingestion health across a Microsoft Sentinel workspace, including tables populated by third-party data connectors. Use it to spot silent connector failures, investigate cost spikes, validate a new connector onboarding, and identify tables that might need a different tier or retention setting.
Review the Table insights section and table grid together to understand whether a table is healthy, expensive, or misconfigured.
To view all table insights and investigate table behavior:
Sign in to the Microsoft Defender portal.Go toMicrosoft Sentinel>Configuration>Tables.SelectView all.Review all insights.(Optional) Add additional filters such as:Minimum fluctuation percentage.Minimum fluctuation volume by kilobytes (KB), megabytes (MB), or gigabytes (GB).
(Optional) Sort the table by selecting one of the headers.
Table insights is useful in scenarios such as:
If a third-party connector silently stops sending data, sort byLast data receivedand review theTable ingestion fluctuationscard for tables that dropped to zero ingestion in the last 24 hours compared to the same period last week.If a noisy connector causes an unexpected cost spike, reviewTop 5 tables by daily ingestion volume, then inspectVolume anomalyandEst. daily ingestion costto find the table that needs transformation or tier review.If you just onboarded a new third-party data connector, confirm that its destination table appears in the grid, is receiving data, and is on the expected tier.If you need to right-size table settings, compareAvg. daily ingestion,Est. daily ingestion cost,Tier, andRetentionto identify candidates for the Data Lake tier or shorter retention.If detections suddenly drop, review the fluctuations card for tables that show a 100 percent decrease and correlate those tables with the detections that stopped firing.
Table insights limitations
Keep the following limits in mind when you interpret the Table insights data:
| |
| \ No newline at end of file | |
@@ -1,6 +1,6 @@ --- layout: Conceptual-title: Configure Table Settings in Microsoft Sentinel | Microsoft Learn+title: Manage tables, tiers, and retention in Sentinel | Microsoft Learn canonicalUrl: https://learn.microsoft.com/en-us/azure/sentinel/manage-table-tiers-retention breadcrumb_path: breadcrumb/toc.json feedback_help_link_url: https://learn.microsoft.com/answers/tags/423/microsoft-sentinel/@@ -21,7 +21,7 @@ manager: orspodek ms.service: microsoft-sentinel ms.subservice: sentinel-siem search.appverid: met150-description: Configure retention and data tier settings for Microsoft Sentinel and Defender XDR tables in the Microsoft Defender portal, and use Table insights to monitor ingestion health and optimize costs.+description: Configure retention and data tier settings for Microsoft Sentinel and Defender XDR tables in the Microsoft Defender portal. ms.author: guywild author: guywi-ms ms.reviewer: noak@@ -44,26 +44,30 @@ item_type: Content source_path: sentinel/manage-table-tiers-retention.md cmProducts: - https://authoring-docs-microsoft.poolparty.biz/devrel/8a94907f-2511-4271-b5ca-ec7f2e75067c-- https://microsoft-devrel.poolparty.biz/DevRelOfferingOntology/c671beaa-a830-4c9f-aceb-97379ee031ca+- https://microsoft-devrel.poolparty.biz/DevRelOfferingOntology/1577a46d-8446-40bd-bfae-0578362f4d94+- https://authoring-docs-microsoft.poolparty.biz/devrel/8f37329d-5c2f-4d50-b9b8-aa5cf54dbffe spProducts: - https://authoring-docs-microsoft.poolparty.biz/devrel/bffa8e88-f633-409d-a24d-083bdbc68872-- https://microsoft-devrel.poolparty.biz/DevRelOfferingOntology/8921374c-4dbe-4ed0-b011-a39e18bfbd98+- https://microsoft-devrel.poolparty.biz/DevRelOfferingOntology/cdf3f22d-5420-4d59-a2bf-66d6b3d9c828+- https://authoring-docs-microsoft.poolparty.biz/devrel/e13db295-3de6-46d5-bcdf-8785a76e3843 platformId: 4fa798b0-9a70-0a32-fd8e-1487bccf21f0 --- -# Configure Table Settings in Microsoft Sentinel | Microsoft Learn+# Manage tables, tiers, and retention in Sentinel | Microsoft Learn The Microsoft Defender portal provides a centralized experience for configuring table-level data retention and tier settings across Microsoft Sentinel and Microsoft Defender XDR. You can view and manage retention settings, switch between Analytics and Data Lake tiers, and optimize storage based on operational and cost requirements.
-This article explains how to configure retention and tier settings for Microsoft Sentinel and Defender XDR tables in the Microsoft Defender portal. It also explains how to use Table insights, a built-in monitoring feature that surfaces ingestion health data for each table in your workspace.
+For an overview of the **Data & connectors** experience, see [Manage data in Microsoft Sentinel and Defender](manage-data-overview).
-For more information about how data tiers and retention work, see [Manage data tiers and retention in Microsoft Sentinel](manage-data-overview).
+Note
+
+If you use Integrated Security Operations Center (ISOC), see [Manage tables, tiers, and retention in ISOC](manage-table-tiers-retention-isoc). Review [Data ingestion and billing for ISOC in Microsoft Defender](/en-us/defender-xdr/integrated-security-operations-data-billing-retention) for eligibility and included retention before changing table settings. The general retention ranges in this article don't establish which settings are supported for an individual ISOC XDR table. The XDR retention thresholds and cost examples describe the existing Microsoft Sentinel XDR ingestion experience, not the ISOC retention benefit.
## Prerequisites
- To use data lake retention, your workspace must be in a [supported region](geographical-availability-data-residency#supported-regions).
- To manage Defender XDR hunting tables, you need to onboard to Microsoft Sentinel in the Defender portal. For more information, see [Connect Microsoft Sentinel to the Microsoft Defender portal](/en-us/azure/sentinel/microsoft-sentinel-onboard).
-- To use Table insights, you need read access to the Tables page in your workspace. Table insights requires the same permissions as viewing table settings. For more information, see Permissions required.
+- To use Table insights, you need read access to the Tables page in your workspace. Table insights requires the same permissions as viewing table settings. For more information, see [Table insights permissions](manage-data-overview#table-insights-permissions).
## Permissions required
@@ -82,19 +86,123 @@ For more information about unified RBAC in the Defender portal, see [Microsoft D
For more information about Microsoft Sentinel workspace permissions, see [Roles and permissions in the Microsoft Sentinel platform](roles).
-### Table insights permissions
+For permissions to investigate table health, see [Table insights permissions](manage-data-overview#table-insights-permissions).
+
+## Which tables can you manage in the Microsoft Defender portal?
+
+This section describes the table types you can manage in the Microsoft Defender portal.
+
+[](media/manage-data-overview/view-table-properties-microsoft-defender-portal.png#lightbox)
+
+| Table type | Description | Examples | Is in Microsoft Sentinel workspace? |
+| --- | --- | --- | --- |
+| **Microsoft Sentinel** | Built-in tables, including:- Azure tables, such as AzureDiagnostics and SigninLogs.- Microsoft Sentinel tables.- [Microsoft Defender XDR integration with Microsoft Sentinel](connect-microsoft-365-defender?tabs=MDE#connect-events), which are created in your Microsoft Sentinel workspace when you increase the analytics retention period beyond 30 days. See the **XDR** table type for Defender XDR tables that are currently unsupported. | - Azure tables: `AzureDiagnostics`, `SigninLogs`- Microsoft Sentinel tables: `AWSCloudTrail`, `SecurityAlert`- XDR tables: `DeviceEvents`,`AlertInfo` | Yes |
+| **Custom** | Tables you create manually or through jobs in your Microsoft Sentinel workspace, including summary rule and search job results tables, and custom data source tables. | Tables with `_CL` or `_SRCH` suffixes. | Yes |
+| **XDR** | Tables in the XDR default tier, which have 30 days of analytics retention by default. You can view these tables, but you can't manage them from the Defender portal. | `IdentityInfo` | No |
+
+Note
+
+You can view Basic Logs tables in your Microsoft Sentinel workspace from the Microsoft Defender portal, but you can currently manage them only from your Log Analytics workspace. To manage these tables from the Microsoft Defender portal, change the table plan from Basic Logs to Analytics in your Microsoft Sentinel workspace.
+
+## How data tiers and retention work
+
+For regional support, see [Data lake retention availability](geographical-availability-data-residency#supported-regions).
+
+You can retain data in Microsoft Sentinel in one of two tiers:
+
+- **Analytics tier**: This tier makes data available for alerting, hunting, workbooks, and all Microsoft Sentinel features. It retains data in two states:
+
+ - **Analytics retention**: In this "hot" state, data is fully available for real-time analytics, including high-performance queries, analytics rules, and threat hunting. By default, Microsoft Sentinel and Microsoft Defender XDR retain data in this tier for 30 days. You can extend the retention period of all tables to up to two years at a prorated monthly long-term retention charge. You can extend the retention period of Microsoft Sentinel solution tables to 90 days for free.
+ - **Total retention**: By default, all data in the analytics tier is mirrored to the data lake for the same retention period. You can extend the retention of your data in the lake beyond the analytics retention, for up to 12 years of total retention at a low cost.
+- **Data Lake tier**: In this low-cost "cold" tier, Microsoft Sentinel retains your data in the data lake only. Data in the Data Lake tier isn't available for real-time analytics features and threat hunting. However, you can access data in the lake whenever you need it through [KQL jobs](datalake/kql-jobs), analyze trends over time by running scheduled KQL or Spark jobs, and aggregate insights from incoming data at a regular cadence by using summary rules.
+- **XDR data**: By default, Microsoft Defender XDR threat hunting data is always available in the Analytics tier for 30 days. You can extend retention of this data in the Analytics tier up to 90 days, which incurs ingestion costs, while storage remains free. Extending retention beyond 90 days in the Analytics tier also incurs storage costs. You can also ingest exclusively into the Data Lake tier, but the data remains available in the Analytics tier for 30 days. Ingesting XDR data directly into the Data Lake tier is more cost-effective, but it involves ingestion, storage, and processing costs. In this option, customers still get 30 days' worth of data in the Analytics tier at no additional cost.
-Use the following permissions when you want to investigate ingestion health in **Table insights** or take action from a flagged table.
+For more information about the differences between the Analytics tier and the Data Lake tier, see Compare the analytics and data lake tiers.
-| Task | Minimum role | Scope |
+The following diagram illustrates the retention components of the Analytics, Data Lake, and XDR default tiers, and the table types that apply to each tier:
+
+[](media/manage-data-overview/tiers-retention-defender-portal.png#lightbox)
+
+For more information about the Microsoft Sentinel data lake, see [What is Microsoft Sentinel data lake](datalake/sentinel-lake-overview).
+
+## Compare the analytics and data lake tiers
+
+This table compares the Analytics and Data Lake tiers and their key characteristics:
+
+| Comparison | Analytics tier | Data Lake tier |
| --- | --- | --- |
-| View **Table insights** cards and the table grid | [Microsoft Sentinel Reader built-in role](roles#built-in-azure-roles-for-microsoft-sentinel) | Resource group that contains the workspace |
-| Configure fluctuation thresholds or customize table columns | [Microsoft Sentinel Contributor built-in role](roles#built-in-azure-roles-for-microsoft-sentinel) | Workspace |
-| Change a table tier or retention setting | [Microsoft Sentinel Contributor built-in role](roles#built-in-azure-roles-for-microsoft-sentinel) and [Log Analytics Contributor built-in role](/en-us/azure/azure-monitor/logs/manage-access#log-analytics-contributor) | Workspace |
-| Create a data collection rule (DCR) or ingestion-time transformation from a flagged table | [Monitoring Contributor built-in role](/en-us/azure/role-based-access-control/built-in-roles/monitor#monitoring-contributor) and [Microsoft Sentinel Contributor built-in role](roles#built-in-azure-roles-for-microsoft-sentinel) | Subscription or resource group that contains the DCR |
-| Read the underlying health signals for custom alerting | [Log Analytics Reader built-in role](/en-us/azure/azure-monitor/logs/manage-access#log-analytics-reader) | Workspace |
+| Key characteristics | High-performance querying and indexing for logs (also known as hot, or interactive retention). | Cost-effective long-term retention of large data volumes (also known as cold storage). |
+| Best for | Real-time analytics rules, alerting, hunting, workbooks, and all Microsoft Sentinel features. | - Compliance and regulatory logging.- Historical trend analysis and forensics.- Low-touch data that's not needed for real-time alerts. |
+| Ingestion cost | Standard | Minimal |
+| Query price included | ✅ | ❌ |
+| Optimized query performance | ✅ | ❌ Slower queries.Good for auditing. Not optimized for real-time analysis. |
+| Query capabilities | [Full query capabilities](/en-us/azure/azure-monitor/logs/get-started-queries) in the Microsoft Defender portal, the Azure portal, and by using APIs. | - [Full query capabilities](/en-us/azure/azure-monitor/logs/get-started-queries), including unions and joins.- Run scheduled KQL or Spark jobs.- Use notebooks. |
+| Full set of real-time analytics features | ✅ | ❌ Limitations on some features, including analytics rules, hunting queries, parsers, watchlists, workbooks, and playbooks. |
+| [Search jobs](investigate-large-datasets) | ✅ | ✅ |
+| [Summary rules](summary-rules) | ✅ | ✅ [Full KQL on a single table](/en-us/azure/azure-monitor/logs/basic-logs-query), which you can extend with data from an analytics table using [lookup](/en-us/azure/data-explorer/kusto/query/lookup-operator) |
+| [Restore](restore) | ✅ | ❌ KQL and Notebook jobs can promote data to the analytics tier. |
+| [Data export](/en-us/azure/azure-monitor/logs/logs-data-export) | ✅ | ❌ |
+| Retention period | 90 days for Microsoft Sentinel, 30 days for Microsoft Defender XDR. Can be extended to up to two years at a prorated monthly long-term retention charge. | Same as analytics retention, by default. Can be extended to up to 12 years. |
+
+## What happens when you modify table settings
+
+You can switch a table's tier and retention settings at any time.
+
+When you change a table's tier from Analytics to Data Lake, all real-time analytics and hunting queries stop working.
+
+When you shorten a table's total retention, Microsoft waits 30 days before removing the data, so you can revert the change and avoid data loss if you made a configuration error.
+
+When you increase total retention, the new retention period applies to all data that was already ingested into the table and wasn't yet removed.
+
+When you change the analytics retention settings of a table with existing data, the change takes effect immediately.
+
+***Example***:
-Custom roles must include equivalent read and write permissions for table settings and health investigation, including `Microsoft.OperationalInsights/workspaces/tables/read` and `Microsoft.SecurityInsights/*/read` data actions.
+- You have a table in the Analytics tier with 180 days of analytics retention. By default, total retention is also set to 180 days.
+- You change analytics retention to 90 days without changing total retention, which remains 180 days.
+- Microsoft Sentinel automatically removes the last 90 days of data from analytics retention, but continues to store data that's 90-180 days old in the Data Lake tier.
+
+## Manage XDR data in Microsoft Sentinel
+
+Before changing retention, review the table's current settings and the warnings in the Microsoft Defender portal. For ISOC-specific retention and billing, use the [ISOC guidance](/en-us/defender-xdr/integrated-security-operations-data-billing-retention) rather than the retention thresholds and cost examples in this section.
+
+By default, Microsoft Defender XDR retains threat hunting data in the **XDR default tier** for 30 days. This data isn't ingested into the Analytics or Data Lake tiers by default. If you extend the retention period of supported XDR tables beyond 30 days and up to 90 days, Sentinel ingestion costs apply, but there are no additional storage costs. The tables are created in your Microsoft Sentinel workspace in the Analytics tier and mirrored to the Data Lake tier.
+
+If you enable the Microsoft Sentinel XDR connector in the Azure portal, the tables you select during setup are automatically ingested into the Analytics tier and mirrored to the Data Lake tier. The default retention is 30 days, and you can extend it up to 12 years. For a list of tables, see [Microsoft Defender XDR integration with Microsoft Sentinel](connect-microsoft-365-defender?tabs=MDE#connect-events). You can ingest supported XDR tables that you didn't select during connector deployment into the Analytics tier and mirror them to the Data Lake tier by setting retention to more than 30 days.
+
+If you don't enable the Microsoft Sentinel XDR connector, XDR tables aren't automatically ingested, but you can still ingest them by setting Analytics or Data Lake tier retention to more than 30 days in the Microsoft Defender portal.
+
+You can choose to ingest supported XDR tables exclusively into the Data Lake tier by selecting the **Data Lake tier** option when configuring retention settings. For more information, see [Configure data retention and tiering](configure-data-connector?tabs=defender-portal#configure-data-retention-and-tiering).
+
+Warning
+
+Resetting analytics retention and total retention to the default 30 days disables the connector in the Azure portal.
+
+To stop ingesting data into the Analytics tier, reset analytics retention and total retention to the default 30 days.
+
+For more information about managing your tables and data, see Manage table settings.
+
+### XDR data retention and costs
+
+The following tables summarize the free retention periods and cost implications for the different tiers in Microsoft Sentinel:
+
+| Tier | Retention | Notes |
+| --- | --- | --- |
+| Advanced hunting (default) | 30 days | Default, included in the XDR license. |
+| Analytics tier | 31-90 days | Free storage for Sentinel-enabled workspaces. Data is mirrored to the data lake. Sentinel ingestion charges apply. |
+| Data Lake tier | Configurable. By default, the same as the analytics tier. | Free storage when total retention is the same as analytics tier retention.Retaining data in the Data Lake tier beyond analytics tier retention incurs Data Lake tier storage costs.Ingesting data directly into the Data Lake tier incurs ingestion, storage, and processing costs. |
+
+For more information about billing and costs, see [Understand the full billing model for Microsoft Sentinel](billing#understand-the-full-billing-model-for-microsoft-sentinel).
+
+In the following examples, XDR data is available through advanced hunting for at least 30 days, regardless of the retention settings in the analytics or data lake tiers.
+
+| Analytics tier retention | Total retention | Analytics tier ingestion costs | Analytics tier storage costs | Data Lake tier costs |
+| --- | --- | --- | --- | --- |
+| 30 days default | 30 days default | No additional costs | N/A | N/A |
+| 90 days | 90 days | Costs apply for analytics tier ingestion. | No additional costs. 90 days included free. | No additional costs. Total retention matches analytics tier retention. |
+| 90 days | 180 days | Costs apply for analytics tier ingestion. | No additional costs; 90 days included free. | Costs apply for 90 days of additional data lake retention (180 - 90 days). |
+| 180 days | 1 year | Costs apply for analytics tier ingestion. | Costs apply for 90 days of additional analytics tier retention. | Costs apply for 185 days of additional data lake retention (365 - 180 days). |
+| 0 days (data lake only) | 5 years | N/A | N/A | Costs apply for ingestion and for 5 years of data lake retention. |
## Manage table settings
@@ -117,6 +225,8 @@ To configure table retention and tier settings in the Microsoft Defender portal,
The **Manage table** screen lets you modify retention settings in the current tier and change the storage tier, if necessary.
+ Available settings depend on the table. Review the options and warnings shown for the selected table rather than assuming that every table supports the full ranges listed here.
+
[](media/manage-data-overview/manage-table-settings-microsoft-defender-portal.png#lightbox)
- **Analytics tier retention settings**:
@@ -130,7 +240,7 @@ To configure table retention and tier settings in the Microsoft Defender portal,
Tier changes aren't available for all tables. For example, some XDR and Microsoft Sentinel solution tables must remain in the Analytics tier because Microsoft security services require the data in these tables for near-real-time analytics.
- For more information about retention and tier settings, see [Manage data tiers and retention in Microsoft Sentinel](manage-data-overview).
+ For more information about retention and tier settings, see How data tiers and retention work.
5. Review warnings and messages. These messages help you understand important implications of changing table settings.
Examples include:
@@ -141,55 +251,4 @@ To configure table retention and tier settings in the Microsoft Defender portal, - Advanced hunting
- Analytics rules
- Custom detection rules
-6. Select **Save** to apply the new settings.
-
-## Use Table insights to monitor table health
-
-Table insights is a built-in monitoring view on the Tables page in the Microsoft Defender portal. It gives security and platform engineers a single table-level view of ingestion health across a Microsoft Sentinel workspace, including tables populated by third-party data connectors. Use it to spot silent connector failures, investigate cost spikes, validate a new connector onboarding, and identify tables that might need a different tier or retention setting.
-
-Review the Table insights section and table grid together to understand whether a table is healthy, expensive, or misconfigured.
-
-| Signal | What it shows | How to use it |
-| --- | --- | --- |
-| **Ingestion volume per tier** | The rolling 30-day total of gigabytes ingested into the Analytics and Data Lake tiers for the current workspace | Compare how much data each tier is receiving before you move tables between tiers or shorten retention. |
-| **Table ingestion fluctuations** | Tables whose last 24-hour ingestion volume differs from the same day last week by at least the configured fluctuation threshold | Investigate large drops or spikes first. A negative value can indicate an upstream collection issue, while a positive value can signal unexpected cost growth. |
-| **Top 5 tables by daily ingestion volume** | The tables that drove the most daily ingestion during the last 30 days | Use this view to identify the main cost drivers in the workspace and decide whether a transformation or tier change might help. |
-| **Last data received** | How long it has been since the table last received a record | Use this value to detect a connector that has gone silent, especially after a deployment or credential change. |
-| **Avg. daily ingestion** | The average number of gigabytes per day over the lookback window | Compare this value with retention settings to find tables that are storing more data than you need for active investigations. |
-| **Est. daily ingestion cost** | The estimated daily ingestion cost for the table's current tier | Use this estimate to prioritize which tables to optimize first when costs increase. |
-| **Volume anomaly** | The signed percentage change from the baseline ingestion pattern | Focus on large positive values for unexpected cost growth and large negative values for possible data collection failures. |
-
-To view all table insights and investigate table behavior:
-
-1. Sign in to the Microsoft Defender portal.
-2. Go to **Microsoft Sentinel** > **Configuration** > **Tables**.
-3. Select **View all**.
-
- [](media/manage-table-tiers-retention/view-all-table-insights.png#lightbox)
-4. Review all insights.
-5. (Optional) Add additional filters such as:
-
- - Minimum fluctuation percentage.
- - Minimum fluctuation volume by kilobytes (KB), megabytes (MB), or gigabytes (GB).
-6. (Optional) Sort the table by selecting one of the headers.
-
-Table insights is useful in scenarios such as:
-
-- If a third-party connector silently stops sending data, sort by **Last data received** and review the **Table ingestion fluctuations** card for tables that dropped to zero ingestion in the last 24 hours compared to the same period last week.
-- If a noisy connector causes an unexpected cost spike, review **Top 5 tables by daily ingestion volume**, then inspect **Volume anomaly** and **Est. daily ingestion cost** to find the table that needs transformation or tier review.
-- If you just onboarded a new third-party data connector, confirm that its destination table appears in the grid, is receiving data, and is on the expected tier.
-- If you need to right-size table settings, compare **Avg. daily ingestion**, **Est. daily ingestion cost**, **Tier**, and **Retention** to identify candidates for the Data Lake tier or shorter retention.
-- If detections suddenly drop, review the fluctuations card for tables that show a 100 percent decrease and correlate those tables with the detections that stopped firing.
-
-### Table insights limitations
-
-Keep the following limits in mind when you interpret the **Table insights** data:
-
-| Area | Limitation |
-| --- | --- |
-| Lookback window | Ingestion volume cards show only the last 30 days. Fluctuations compare the last 24 hours with the same day of the previous week, so they don't account for longer seasonal patterns such as month-end batch jobs. |
-| Fluctuation thresholds | By default, fluctuations require at least a 10 percent change and 1 megabyte of absolute change. Tables below 1 MB per day aren't evaluated and won't appear on the card even if they fully stop sending data. |
-| Cost estimates | The **Est. daily ingestion cost** value is based on public list price for the table's current tier and region. It doesn't reflect commitment-tier discounts, reservations, or private pricing. |
-| Tier coverage | Auxiliary, Basic Logs, and Data Lake tier tables are listed, but the **Volume anomaly** signal is currently optimized for Analytics tier tables. Results for Data Lake tier tables can be less precise. |
-| Alerting | **Table insights** is a visualization surface only. To alert on table health, such as when a connector goes silent, create a scheduled analytics rule that uses the `SentinelHealth` table or a Kusto Query Language (KQL) query against `Usage` or a table-specific schema. |
-| Multi-workspace view | **Table insights** shows data for the currently selected workspace only. If you manage multiple workspaces, switch workspaces to review each one separately. |\ No newline at end of file+6. Select **Save** to apply the new settings.\ No newline at end of file 


