Microsoft Sentinel
Hunting and detection

Run KQL queries on the Microsoft Sentinel data lake

In brief

The documentation now describes running long-running KQL queries asynchronously, including queries that may exceed the 8-minute synchronous timeout. It also clarifies external-table examples, time-range requirements for stored results, and adds `estimate_data_size()`.

What Defender admins need to know

Administrators can use async queries for broad or long-running investigations and follow the clarified query guidance. No action is required.

Summaries are generated from the documentation change itself.

Documentation change

The comparison below shows only the changed extract. Use the full-page view for complete context.

Run KQL queries on the Microsoft Sentinel data lake

Data lake exploration in the Microsoft Defender portal provides a unified interface to analyze your data lake. It lets you run KQL (Kusto Query Language) queries, create jobs, and manage them. Before you begin, make sure you meet the prerequisites for querying the data lake, including data lake onboarding and required permissions.

The KQL queries page under Data lake exploration lets you edit and run KQL queries on data lake resources and federated tables. Create jobs to promote data from the data lake to the analytics tier, or create aggregate tables in the data lake tier. Run jobs on demand or schedule them. The Jobs page lets you manage jobs; enable, disable, edit, or delete. For more information, see Create jobs in the Microsoft Sentinel data lake.

Async queries

Async queries let you run long-running KQL queries in the background so you can continue working in the portal while the query processes on the server. Use async queries when your query might exceed the 8-minute synchronous timeout or when working with broad time ranges.

Run async queries

You can run long-running queries asynchronously, so you can keep working while the query runs on the server. To run a query asynchronously:

You can run KQL queries against the Microsoft Sentinel data lake using Azure Data Explorer (ADX). ADX provides a powerful query engine and advanced analytics capabilities. To connect to the data lake using ADX, create a new connection using the following URI: https://api.securityplatform.microsoft.com/lake/kql

When querying tables in the data lake using ADX, you must use the external_table() function to access the data. For example:The following example query retrieves a sample of 100 records from the AADRiskyUsers external table:

external_table("AADRiskyUsers")
    + `.show databases entities`
    + `.show database`

+ When you use the `stored_query_results` command, provide the time range in the KQL query. The time selector above the query editor time range selector doesn't work with thisthe `stored_query_results` command.

+ Using out-of-the-box or custom functions isn't supported in KQL queries against the data lake.

    + `arg()`
    + `externaldata()`
    + `ingestion_time()`
    + `estimate_data_size()`

+ There is a 15-minute latency between when data is ingested into the data lake or federated tables, and when it becomes available for querying. This means that newly ingested data may not be immediately queryable.