Microsoft Sentinel
Cloud and workloads

Manage tables, tiers, and retention in Sentinel | Microsoft Learn

In brief

The article now covers managing tables, tiers, and retention, including supported table types, XDR and Basic Logs management limitations, retention tiers, regional availability, ISOC guidance, and updated permissions links.

What Defender admins need to know

Administrators can use the revised guidance to identify which tables they can manage in the Defender portal and where additional management is required.

Summaries are generated from the documentation change itself.

Documentation change

The comparison below shows only the changed extract. Use the full-page view for complete context.

Configure Table SettingsManage tables, tiers, and retention in Microsoft Sentinel | Microsoft Learn

The Microsoft Defender portal provides a centralized experience for configuring table-level data retention and tier settings across Microsoft Sentinel and Microsoft Defender XDR. You can view and manage retention settings, switch between Analytics and Data Lake tiers, and optimize storage based on operational and cost requirements.

ThisFor an overview of the Data & connectors experience, see Manage data in Microsoft Sentinel and Defender.

Note

If you use Integrated Security Operations Center (ISOC), see Manage tables, tiers, and retention in ISOC. Review Data ingestion and billing for ISOC in Microsoft Defender for eligibility and included retention before changing table settings. The general retention ranges in this article explains how to configuredon't establish which settings are supported for an individual ISOC XDR table. The XDR retention thresholds and tier settings forcost examples describe the existing Microsoft Sentinel and Defender XDR tables iningestion experience, not the Microsoft Defender portal. It also explains how to use Table insights, a built-in monitoring feature that surfaces ingestion health data for each table in your workspace.

For more information about how data tiers andISOC retention work, see Manage data tiers and retention in Microsoft Sentinel.benefit.

Prerequisites

Permissions required

For more information about Microsoft Sentinel workspace permissions, see Roles and permissions in the Microsoft Sentinel platform.

Table insights

For permissions

Use the following permissions when you want to investigate ingestion healthtable health, see Table insights permissions.

Which tables can you manage in Table insights or take action from a flagged table.the Microsoft Defender portal?

This section describes the table types you can manage in the Microsoft Defender portal.

Screenshot that shows the Table Management screen in the Defender portal.

TaskTable type Minimum roleDescription ScopeExamplesIs in Microsoft Sentinel workspace?
View Table insightsMicrosoft Sentinel cards and the table grid Microsoft Sentinel Reader built-in roleBuilt-in tables, including:- Azure tables, such as AzureDiagnostics and SigninLogs.- Microsoft Sentinel tables.- Microsoft Defender XDR integration with Microsoft Sentinel, which are created in your Microsoft Sentinel workspace when you increase the analytics retention period beyond 30 days. See the XDR table type for Defender XDR tables that are currently unsupported. Resource group that contains the workspace- Azure tables: AzureDiagnostics, SigninLogs- Microsoft Sentinel tables: AWSCloudTrail, SecurityAlert- XDR tables: DeviceEvents,AlertInfoYes
Configure fluctuation thresholdsCustomTables you create manually or customize table columnsthrough jobs in your Microsoft Sentinel workspace, including summary rule and search job results tables, and custom data source tables. Microsoft Sentinel Contributor built-in roleTables with _CL or _SRCH suffixes. WorkspaceYes
Change a table tier orXDRTables in the XDR default tier, which have 30 days of analytics retention settingby default. You can view these tables, but you can't manage them from the Defender portal. Microsoft Sentinel Contributor built-in role and Log Analytics Contributor built-in roleIdentityInfo Workspace
Create a data collection rule (DCR) or ingestion-time transformation from a flagged tableMonitoring Contributor built-in role and Microsoft Sentinel Contributor built-in roleSubscription or resource group that contains the DCR
Read the underlying health signals for custom alertingLog Analytics Reader built-in roleWorkspaceNo

Custom roles must include equivalent readNote

You can view Basic Logs tables in your Microsoft Sentinel workspace from the Microsoft Defender portal, but you can currently manage them only from your Log Analytics workspace. To manage these tables from the Microsoft Defender portal, change the table plan from Basic Logs to Analytics in your Microsoft Sentinel workspace.

How data tiers and write permissionsretention work

For regional support, see Data lake retention availability.

You can retain data in Microsoft Sentinel in one of two tiers:

  • Analytics tier: This tier makes data available for alerting, hunting, workbooks, and all Microsoft Sentinel features. It retains data in two states:

    • Analytics retention: In this "hot" state, data is fully available for real-time analytics, including high-performance queries, analytics rules, and threat hunting. By default, Microsoft Sentinel and Microsoft Defender XDR retain data in this tier for 30 days. You can extend the retention period of all tables to up to two years at a prorated monthly long-term retention charge. You can extend the retention period of Microsoft Sentinel solution tables to 90 days for free.
    • Total retention: By default, all data in the analytics tier is mirrored to the data lake for the same retention period. You can extend the retention of your data in the lake beyond the analytics retention, for up to 12 years of total retention at a low cost.
  • Data Lake tier: In this low-cost "cold" tier, Microsoft Sentinel retains your data in the data lake only. Data in the Data Lake tier isn't available for real-time analytics features and threat hunting. However, you can access data in the lake whenever you need it through KQL jobs, analyze trends over time by running scheduled KQL or Spark jobs, and aggregate insights from incoming data at a regular cadence by using summary rules.

  • XDR data: By default, Microsoft Defender XDR threat hunting data is always available in the Analytics tier for 30 days. You can extend retention of this data in the Analytics tier up to 90 days, which incurs ingestion costs, while storage remains free. Extending retention beyond 90 days in the Analytics tier also incurs storage costs. You can also ingest exclusively into the Data Lake tier, but the data remains available in the Analytics tier for 30 days. Ingesting XDR data directly into the Data Lake tier is more cost-effective, but it involves ingestion, storage, and processing costs. In this option, customers still get 30 days' worth of data in the Analytics tier at no additional cost.

For more information about the differences between the Analytics tier and the Data Lake tier, see Compare the analytics and data lake tiers.

The following diagram illustrates the retention components of the Analytics, Data Lake, and XDR default tiers, and the table types that apply to each tier:

Diagram that depicts the analytics and data lake tiers in the Microsoft Defender portal.

For more information about the Microsoft Sentinel data lake, see What is Microsoft Sentinel data lake.

Compare the analytics and data lake tiers

This table compares the Analytics and Data Lake tiers and their key characteristics:

ComparisonAnalytics tierData Lake tier
Key characteristicsHigh-performance querying and indexing for logs (also known as hot, or interactive retention).Cost-effective long-term retention of large data volumes (also known as cold storage).
Best forReal-time analytics rules, alerting, hunting, workbooks, and all Microsoft Sentinel features.- Compliance and regulatory logging.- Historical trend analysis and forensics.- Low-touch data that's not needed for real-time alerts.
Ingestion costStandardMinimal
Query price included✅❌
Optimized query performance✅❌ Slower queries.Good for auditing. Not optimized for real-time analysis.
Query capabilitiesFull query capabilities in the Microsoft Defender portal, the Azure portal, and by using APIs.- Full query capabilities, including unions and joins.- Run scheduled KQL or Spark jobs.- Use notebooks.
Full set of real-time analytics features✅❌ Limitations on some features, including analytics rules, hunting queries, parsers, watchlists, workbooks, and playbooks.
Search jobs✅✅
Summary rules✅✅ Full KQL on a single table, which you can extend with data from an analytics table using lookup
Restore✅❌ KQL and Notebook jobs can promote data to the analytics tier.
Data export✅❌
Retention period90 days for Microsoft Sentinel, 30 days for Microsoft Defender XDR. Can be extended to up to two years at a prorated monthly long-term retention charge.Same as analytics retention, by default. Can be extended to up to 12 years.

What happens when you modify table settings

You can switch a table's tier and retention settings at any time.

When you change a table's tier from Analytics to Data Lake, all real-time analytics and hunting queries stop working.

When you shorten a table's total retention, Microsoft waits 30 days before removing the data, so you can revert the change and avoid data loss if you made a configuration error.

When you increase total retention, the new retention period applies to all data that was already ingested into the table and wasn't yet removed.

When you change the analytics retention settings of a table with existing data, the change takes effect immediately.

Example:

  • You have a table in the Analytics tier with 180 days of analytics retention. By default, total retention is also set to 180 days.
  • You change analytics retention to 90 days without changing total retention, which remains 180 days.
  • Microsoft Sentinel automatically removes the last 90 days of data from analytics retention, but continues to store data that's 90-180 days old in the Data Lake tier.

Manage XDR data in Microsoft Sentinel

Before changing retention, review the table's current settings and health investigation, including Microsoft.OperationalInsights/workspaces/tables/readthe warnings in the Microsoft Defender portal. For ISOC-specific retention and Microsoft.SecurityInsights/*/readbilling, use the ISOC guidance rather than the retention thresholds and cost examples in this section.

By default, Microsoft Defender XDR retains threat hunting data actions.in the XDR default tier for 30 days. This data isn't ingested into the Analytics or Data Lake tiers by default. If you extend the retention period of supported XDR tables beyond 30 days and up to 90 days, Sentinel ingestion costs apply, but there are no additional storage costs. The tables are created in your Microsoft Sentinel workspace in the Analytics tier and mirrored to the Data Lake tier.

If you enable the Microsoft Sentinel XDR connector in the Azure portal, the tables you select during setup are automatically ingested into the Analytics tier and mirrored to the Data Lake tier. The default retention is 30 days, and you can extend it up to 12 years. For a list of tables, see Microsoft Defender XDR integration with Microsoft Sentinel. You can ingest supported XDR tables that you didn't select during connector deployment into the Analytics tier and mirror them to the Data Lake tier by setting retention to more than 30 days.

If you don't enable the Microsoft Sentinel XDR connector, XDR tables aren't automatically ingested, but you can still ingest them by setting Analytics or Data Lake tier retention to more than 30 days in the Microsoft Defender portal.

You can choose to ingest supported XDR tables exclusively into the Data Lake tier by selecting the Data Lake tier option when configuring retention settings. For more information, see Configure data retention and tiering.

Warning

Resetting analytics retention and total retention to the default 30 days disables the connector in the Azure portal.

To stop ingesting data into the Analytics tier, reset analytics retention and total retention to the default 30 days.

For more information about managing your tables and data, see Manage table settings.

XDR data retention and costs

The following tables summarize the free retention periods and cost implications for the different tiers in Microsoft Sentinel:

TierRetentionNotes
Advanced hunting (default)30 daysDefault, included in the XDR license.
Analytics tier31-90 daysFree storage for Sentinel-enabled workspaces. Data is mirrored to the data lake. Sentinel ingestion charges apply.
Data Lake tierConfigurable. By default, the same as the analytics tier.Free storage when total retention is the same as analytics tier retention.Retaining data in the Data Lake tier beyond analytics tier retention incurs Data Lake tier storage costs.Ingesting data directly into the Data Lake tier incurs ingestion, storage, and processing costs.

For more information about billing and costs, see Understand the full billing model for Microsoft Sentinel.

In the following examples, XDR data is available through advanced hunting for at least 30 days, regardless of the retention settings in the analytics or data lake tiers.

Analytics tier retentionTotal retentionAnalytics tier ingestion costsAnalytics tier storage costsData Lake tier costs
30 days default30 days defaultNo additional costsN/AN/A
90 days90 daysCosts apply for analytics tier ingestion.No additional costs. 90 days included free.No additional costs. Total retention matches analytics tier retention.
90 days180 daysCosts apply for analytics tier ingestion.No additional costs; 90 days included free.Costs apply for 90 days of additional data lake retention (180 - 90 days).
180 days1 yearCosts apply for analytics tier ingestion.Costs apply for 90 days of additional analytics tier retention.Costs apply for 185 days of additional data lake retention (365 - 180 days).
0 days (data lake only)5 yearsN/AN/ACosts apply for ingestion and for 5 years of data lake retention.

Manage table settings

The **Manage table** screen lets you modify retention settings in the current tier and change the storage tier, if necessary.

Available settings depend on the table. Review the options and warnings shown for the selected table rather than assuming that every table supports the full ranges listed here.

[![Screenshot that shows the Manage table screen for the CommonSecurityLog table in the Defender portal.](media/manage-data-overview/manage-table-settings-microsoft-defender-portal.png)](media/manage-data-overview/manage-table-settings-microsoft-defender-portal.png#lightbox)

- **Analytics tier retention settings**:

    Tier changes aren't available for all tables. For example, some XDR and Microsoft Sentinel solution tables must remain in the Analytics tier because Microsoft security services require the data in these tables for near-real-time analytics.

For more information about retention and tier settings, see [ManageHow data tiers and retention in Microsoft Sentinel](manage-data-overview).work.
  1. Review warnings and messages. These messages help you understand important implications of changing table settings.

    Examples include: - Advanced hunting - Analytics rules - Custom detection rules

\ No newline at end of file 6. Select Save to apply the new settings.

Use Table insights to monitor table health

Table insights is a built-in monitoring view on the Tables page in the Microsoft Defender portal. It gives security and platform engineers a single table-level view of ingestion health across a Microsoft Sentinel workspace, including tables populated by third-party data connectors. Use it to spot silent connector failures, investigate cost spikes, validate a new connector onboarding, and identify tables that might need a different tier or retention setting.

Review the Table insights section and table grid together to understand whether a table is healthy, expensive, or misconfigured.

SignalWhat it showsHow to use it
Ingestion volume per tierThe rolling 30-day total of gigabytes ingested into the Analytics and Data Lake tiers for the current workspaceCompare how much data each tier is receiving before you move tables between tiers or shorten retention.
Table ingestion fluctuationsTables whose last 24-hour ingestion volume differs from the same day last week by at least the configured fluctuation thresholdInvestigate large drops or spikes first. A negative value can indicate an upstream collection issue, while a positive value can signal unexpected cost growth.
Top 5 tables by daily ingestion volumeThe tables that drove the most daily ingestion during the last 30 daysUse this view to identify the main cost drivers in the workspace and decide whether a transformation or tier change might help.
Last data receivedHow long it has been since the table last received a recordUse this value to detect a connector that has gone silent, especially after a deployment or credential change.
Avg. daily ingestionThe average number of gigabytes per day over the lookback windowCompare this value with retention settings to find tables that are storing more data than you need for active investigations.
Est. daily ingestion costThe estimated daily ingestion cost for the table's current tierUse this estimate to prioritize which tables to optimize first when costs increase.
Volume anomalyThe signed percentage change from the baseline ingestion patternFocus on large positive values for unexpected cost growth and large negative values for possible data collection failures.

To view all table insights and investigate table behavior:

  1. Sign in to the Microsoft Defender portal.

  2. Go to Microsoft Sentinel > Configuration > Tables.

  3. Select View all.

    Screenshot that shows where the View all button is located in the Table insights section of the page.

  4. Review all insights.

  5. (Optional) Add additional filters such as:

    • Minimum fluctuation percentage.
    • Minimum fluctuation volume by kilobytes (KB), megabytes (MB), or gigabytes (GB).
  6. (Optional) Sort the table by selecting one of the headers.

Table insights is useful in scenarios such as:

  • If a third-party connector silently stops sending data, sort by Last data received and review the Table ingestion fluctuations card for tables that dropped to zero ingestion in the last 24 hours compared to the same period last week.
  • If a noisy connector causes an unexpected cost spike, review Top 5 tables by daily ingestion volume, then inspect Volume anomaly and Est. daily ingestion cost to find the table that needs transformation or tier review.
  • If you just onboarded a new third-party data connector, confirm that its destination table appears in the grid, is receiving data, and is on the expected tier.
  • If you need to right-size table settings, compare Avg. daily ingestion, Est. daily ingestion cost, Tier, and Retention to identify candidates for the Data Lake tier or shorter retention.
  • If detections suddenly drop, review the fluctuations card for tables that show a 100 percent decrease and correlate those tables with the detections that stopped firing.

Table insights limitations

Keep the following limits in mind when you interpret the Table insights data:

AreaLimitation
Lookback windowIngestion volume cards show only the last 30 days. Fluctuations compare the last 24 hours with the same day of the previous week, so they don't account for longer seasonal patterns such as month-end batch jobs.
Fluctuation thresholdsBy default, fluctuations require at least a 10 percent change and 1 megabyte of absolute change. Tables below 1 MB per day aren't evaluated and won't appear on the card even if they fully stop sending data.
Cost estimatesThe Est. daily ingestion cost value is based on public list price for the table's current tier and region. It doesn't reflect commitment-tier discounts, reservations, or private pricing.
Tier coverageAuxiliary, Basic Logs, and Data Lake tier tables are listed, but the Volume anomaly signal is currently optimized for Analytics tier tables. Results for Data Lake tier tables can be less precise.
AlertingTable insights is a visualization surface only. To alert on table health, such as when a connector goes silent, create a scheduled analytics rule that uses the SentinelHealth table or a Kusto Query Language (KQL) query against Usage or a table-specific schema.
Multi-workspace viewTable insights shows data for the currently selected workspace only. If you manage multiple workspaces, switch workspaces to review each one separately.
\ No newline at end of file
\ No newline at end of file