Microsoft Sentinel
Developer and API

Create a pull codeless connector for Microsoft Sentinel

In brief

The article distinguishes pull/polling connectors from push connectors and focuses its instructions on connectors that periodically fetch data through APIs. It also links to the Microsoft Sentinel Visual Studio Code extension.

What Defender admins need to know

Connector builders should use this guide for pull-based integrations and the linked extension when implementing or testing components.

Summaries are generated from the documentation change itself.

Documentation change

The comparison below shows only the changed extract. Use the full-page view for complete context.

Create a pull codeless connector for Microsoft Sentinel

The Codeless Connector Framework (CCF) provides partners, advanced users, and developers the ability to create custom connectors for ingesting data to Microsoft Sentinel.

Connectors can be pull/polling connectors or push connectors, depending on the data source and the preference of the builder. This article focuses on pull connectors, which are polling-based and periodically fetch data via APIs. For information on push connectors, see Create a push codeless connector for Microsoft Sentinel.

Connectors created using the CCF are fully SaaS, with no requirements for service installations. They also include health monitoring and full support from Microsoft Sentinel.

Use the following steps to create your CCF connector and connect your data source to Microsoft Sentinel

[!div class="checklist"]Sentinel:

  • Build the data connector
  • Create the ARM template
  • Deploy the connector
  • Connect Microsoft Sentinel to your data source and start ingesting data

This article will show you how to complete eachEach step and provideincludes an example codeless connector to build along the way.

If you're a software development partner and need support to build a CCF data connector, contact Microsoft Sentinel Partners at [email protected] for assistance.

How is this CCF different fromImprovements over the previous version?CCF version

The initial version of the CCF was announced on the Microsoft Tech Community blog in January of 2022. Since then, we've improved upon the platform has been improved and the legacy releaselegacy release is no longer recommended. This new version of the CCF has the following key improvements:

  1. Better support for various authentication and pagination types.

Testing APIs

We recommend testingTest your components with an API testing tool likesuch as one of the following:

Build the data connector

There are four components required to build the CCF data connector.

Output table definition

If your data source doesn't conform to the schema of a standard table, you have two options:

  • Create a custom table for all the data
  • Create a custom table for some data and split conforming data out to a standard table

Use the Log Analytics UI for a straight forwardstraightforward method to create a custom table together with a DCR. If you create the custom table using the Tables API or another programmatic method, add the _CL suffix manually to the table name. For more information, see Create a custom table.

For more information on splitting your data to more than one table, see the example data and the example custom table created for that data.

Build the data connector user interface with the Data Connector Definition API. Use the Data connector definitions reference as a supplement to explain the API elements in greater detail.

Notes: 1)

  1. The kind property for API polling connector should always be Customizable. 2)
  2. Since this is a type of API polling connector, set the connectivityCriteria type to hasDataConnectors 3) .
  3. The example instructionSteps include a button of type ConnectionToggleButton. This button helps trigger the deployment of data connector rules based on the connection parameters specified.

Use an API testing tool to call the data connector definitions API to create the data connector UI in order toand validate it in the data connectors gallery.

To learn from an example, see the Data connector definitions reference example section.

There are currently three kinds of data connection rules possible for defining your CCF data connector.

Use an API testing tool to call the data connector API to create the data connector which combines the connection rules and previous components. Verify the connector is now connected in the UI.

Whatever authentication is used by your CCF data connector, take these steps to ensure confidential information is kept secure. The goal is to pass along credentials from the ARM template to the CCF without leaving readable confidential objects in your deployments history.

Create the credential label in the connector UI

The data connector definition creates a UI element to prompt for security credentials. For example, if your data connector authenticates to a log source with OAuth, your data connector definition section includes the OAuthForm type in the instructions. This sets up the ARM template to prompt for the credentials.

            "description": "Enter the API key, client secret or password required to connect."
        }
    },
// more deployment template information
}

}


### Use the securestring objects

Finally, the CCF utilizesuses the credential objects in the data connector section.

```json
"auth": {
    "TokenEndpointQueryParameters": {
        "grant_type": "client_credentials"
    }
},}

Deploy your codeless connector as a custom template.

  1. Copy the contents of the ARM deployment template.
  2. Follow the Edit and deploy the template instructions from the article, Quickstart: Create and deploy ARM templates by using the Azure portal.

View your codeless connector in the data connector gallery. Open the data connector and complete any authentication parameters required to connect. Once successfully connected, the DCR and custom tables are created. View the DCR resource in your resource group and any custom tables from the logs analytics workspace.

Example

The following DCR defines a single stream Custom-ExampleConnectorInput using the example data source and transforms the output into two tables.

  1. The first dataflow directs eventType = Alert to the custom ExampleConnectorAlerts_CL table.
  2. The second dataflow directs eventType = File to the normalized standard table,ASimFileEventLogs.

For more information on the structure of this example, see Structure of a data collection rule.

}


Stitch the sections together with a JSON-aware editor like Visual Studio Code to minimize syntax errors like commas and closing brackets and parentheses.

To guide the template building process, comments appear in the **metadata** `description` or inline with `//` comment notation. For more information, see [ARM template best practices - comments](/azure/azure-resource-manager/templates/best-practices#comments).


For more information, see [Parameters in ARM templates](/azure/azure-resource-manager/templates/parameters).
},
// Next is the resources sections here
#### Example ARM template - resources

There are 5 ARM deployment resources in this template guide which house the 4 CCF data connector building components.
                "icon": "[variables('_packageIcon')]"
            }
        }

    // that's the end!
    ]
}

Related content

For more information, see