Daily Defender News
Aboutmerill.net
Microsoft Sentinel
Cloud and workloads

Normalization Develop Parsers

8 September 2026Feature update+34 / −88 lines

In brief

The documentation now restricts parsers to the declared source table, one normalized output per source record, and scalar or local static mappings. Cross-table enrichment, watchlists, multi-value expansion, joins, aggregation, and deduplication are prohibited for parser operations.

What Defender admins need to know

Review custom parsers and connectors for these patterns. Source filtering must use fields in the current event; missing source information should be addressed in the connector or source-specific table.

Summaries are generated from the documentation change itself.

Documentation change

View full page diffView on Microsoft Learn

The comparison below shows only the changed extract. Use the full-page view for complete context.

#Customer intent: As a security analyst, I want to develop custom ASIM parsers so that I can normalize and analyze security event data from various sources in a consistent format.

Filter > Parse > Prepare fields

Keep parser operations record-local

Normalize each source record independently. A source record can produce zero records after filtering or one normalized record.

  • Read event records from only the declared source table. Don't perform same-table or cross-table event enrichment with a second table read, event-record join, workspace-table or watchlist reference, externaldata, or another external tabular source.
  • Preserve record cardinality. Don't turn one source record into multiple normalized records. If the source combines multiple logical events in one record, correct the connector or source event format.
  • Don't use any mv-* operator, including mv-expand and mv-apply.
  • Don't correlate, deduplicate, aggregate, or reaggregate event records with summarize, distinct, arg_min, arg_max, or an equivalent operation.

Query-local static mappings created with datatable and applied with lookup are allowed when each lookup key is unique. Use scalar expressions, direct access to dynamic values, and values available in the current row. If a field can't be mapped without a prohibited pattern, correct the connector or source event shape, or leave a nonmandatory field unmapped.

Filter relevant source records

Filter by source fields

Use physical fields in the current event to identify the source type. Don't query a watchlist or another table to identify relevant records.

If the event doesn't contain enough information to distinguish its source or event type, update the connector to include a source identifier or route the events to a source-specific table. Don't compensate for missing source information with table enrichment.

Filtering based on parser parameters

Derived fields and values

The value of the source field, once extracted, might need to be mapped to the set of values specified for the target schema field. Use scalar expressions such as iff and case, or a query-local static datatable with lookup, to map available data to target values.

For example, the Microsoft DNS parser derives a normalized success or failure outcome from source-specific event and response codes. The parser assigns the EventResult field based on the Event ID and Response Code using an iff statement, as follows:

extend EventResult = iff(EventId==257 and ResponseCode==0 ,'Success','Failure') 


Use `case` when a source value can map to several normalized values. For example:

```kusto
| extend NetworkProtocol = case(
    Proto == 6, "TCP",
    Proto == 17, "UDP",
    ""
)

For larger static mappings, define a query-local dimension table and apply it with lookup. The right side of the lookup must be a locally defined static datatable, not a workspace table, watchlist, or external data source. Define only one row for each lookup key so one source record can't produce multiple normalized records. For example: 

let NetworkProtocolLookup = datatable(Proto:real, NetworkProtocol:string)
[
    6, "TCP",
    17, "UDP"
];
...
| lookup NetworkProtocolLookup on Proto

Enrichment fields

In addition to the fields available from the source, a resulting ASIM event includes enrichment fields that the parser should generate. These schema fields use values from the current row or constants and don't require table enrichment. In many cases, the parsers can assign a constant value to these fields. Populate the standard enrichment fields so each parsed record includes consistent product, vendor, and schema metadata, for example: 

  | extend

### Handle parsing variants

If variants of the same event type require different parsing logic, use scalar conditional expressions such as iff and case while preserving one output record for each source record. Don't create tabular branches and recombine them with union, because branches can process or return the same source record more than once.

Deploy parsers

You can also combine multiple templates to a single deploy process using linked templates watchlist.

To use the ASimSourceType watchlist in your parsers, use the _ASIM_GetSourceBySourceType function in the parser filtering section. For example, the Infoblox DNS parser restricts records to only Infoblox NIOS sources by including the following filter, ensuring the parser processes only relevant Syslog records:

  | where Computer in (_ASIM_GetSourceBySourceType('InfobloxNIOS'))

To use this sample in your parser:

  • Replace Computer with the name of the field that includes the source information for your source. You can keep this as Computer for any parsers based on Syslog.

  • Replace the InfobloxNIOS token with a value of your choice for your parser. Inform parser users that they must update the ASimSourceType watchlist using your selected value, as well as the list of sources that send events of this type.

Filtering based on parser parameters

Derived fields and values

The value of the source field, once extracted, may need to be mapped to the set of values specified for the target schema field. The functions iff, case, and lookup can be helpful to map available data to target values.

For example, the Microsoft DNS parser derives a normalized success or failure outcome from source-specific event and response codes. The parser assigns the EventResult field based on the Event ID and Response Code using an iff statement, as follows:

extend EventResult = iff(EventId==257 and ResponseCode==0 ,'Success','Failure')


To map several values, define the mapping using the `datatable` operator and use `lookup` to perform the mapping. For example, some sources report numeric DNS response codes and the network protocol, while the schema mandates the more common text labels representation for both. The following example demonstrates how to create lookup tables that map numeric protocol identifiers and DNS response codes to their normalized text labels, and then apply those lookups to the parsed data using `datatable` and `lookup`:

```kusto
   let NetworkProtocolLookup = datatable(Proto:real, NetworkProtocol:string)[
        6, 'TCP',
        17, 'UDP'
   ];
    let DnsResponseCodeLookup=datatable(DnsResponseCode:int,DnsResponseCodeName:string)[
      0,'NOERROR',
      1,'FORMERR',
      2,'SERVFAIL',
      3,'NXDOMAIN',
      ...
   ];
   ...
   | lookup DnsResponseCodeLookup on DnsResponseCode
   | lookup NetworkProtocolLookup on Proto

Notice that lookup is useful and efficient also when the mapping has only two possible values.

When the mapping conditions are more complex combine iff, case, and lookup. The example below shows how to combine lookup and case. The lookup example above returns an empty value in the field DnsResponseCodeName if the lookup value is not found. The case example below augments it by using the result of the lookup operation if available, and specifying additional conditions otherwise. Use this approach to handle unmatched lookup values by falling back to additional conditions or a default label:

   | extend DnsResponseCodeName =
      case (
        DnsResponseCodeName != "", DnsResponseCodeName,
        DnsResponseCode between (3841 .. 4095), 'Reserved for Private Use',
        'Unassigned'
      )

Microsoft Sentinel provides built-in helper functions for common lookup values. Instead of manually building a datatable and lookup for well-known mappings, you can use these functions to populate the normalized field directly. For example, the DnsResponseCodeName lookup above can be implemented using one of the following functions:


| extend DnsResponseCodeName = _ASIM_LookupDnsResponseCode(DnsResponseCode)

| invoke _ASIM_ResolveDnsResponseCode('DnsResponseCode')

_ASIM_LookupDnsResponseCode accepts the value to look up as a parameter and lets you choose the output field, making it useful as a general lookup function. _ASIM_ResolveDnsResponseCode is more geared toward parsers: it takes the name of the source field as input and updates the needed ASIM field, in this case DnsResponseCodeName.

For a full list of ASIM help functions, refer to ASIM functions

Enrichment fields

In addition to the fields available from the source, a resulting ASIM event includes enrichment fields that the parser should generate. In many cases, the parsers can assign a constant value to these fields. Populate the standard enrichment fields so each parsed record includes consistent product, vendor, and schema metadata, for example:

  | extend

### Handle parsing variants

In many cases, events in an eventstream include variants that require different parsing logic. To parse different variants in a single parser either use conditional statements such as iff and case, or use a union structure.

To use union to handle multiple variants, create a separate function for each variant and use the union statement to combine the results: 

let AzureFirewallNetworkRuleLogs = AzureDiagnostics
    | where Category == "AzureFirewallNetworkRule"
    | where isnotempty(msg_s);
let parseLogs = AzureFirewallNetworkRuleLogs
    | where msg_s has_any("TCP", "UDP")
    | parse-where
        msg_s with           networkProtocol:string
        " request from "     srcIpAddr:string
        ":"                  srcPortNumber:int
    …
    | project-away msg_s;
let parseLogsWithUrls = AzureFirewallNetworkRuleLogs
    | where msg_s has_all ("Url:","ThreatIntel:")
    | parse-where
        msg_s with           networkProtocol:string
        " request from "     srcIpAddr:string
        " to "               dstIpAddr:string
    ...
union parseLogs,  parseLogsWithUrls…

To avoid duplicate events and excessive processing, make sure each function starts by filtering, using native fields, only the events that it is intended to parse. Also, if needed, use project-away at each branch, before the union.

Deploy parsers

You can also combine multiple templates to a single deploy process using linked templates

Test parsers