Domain 1 of 3 · Chapter 1 of 3

Implement and manage data classification

From a business requirement to a detection requirement

A privacy officer asks you to make sure nobody emails a completed patient intake form outside the company. Before you open any policy, split that sentence into two decisions: what Microsoft Purview must recognize (the form, the identifiers on it) and what must happen when it does (a label, a block, a retention action). This topic covers the first decision, the detection side of Microsoft Purview Information Protection, and the classification taxonomy it serves. For handling, the Microsoft SC-401 study guide skills measured[1] lists 'Define and create sensitivity labels for items and containers' under Implement and manage sensitivity labels in Microsoft Purview and 'Create and manage data loss prevention policies' under Create and configure data loss prevention policies.

Know your data, then protect it

Purview presents its information protection capabilities as tools to know your data, protect your data, and prevent data loss[2]; classification is the know-your-data step. Purview identifies items in three ways[3]: manually by users, by automated pattern recognition with sensitive information types (SITs), and by machine learning with trainable classifiers. Classifier is the umbrella term: the Classifiers page[4] of the Microsoft Purview portal (formerly the Microsoft Purview compliance portal) brings SITs, named entities, exact data match, trainable classifiers, credentials, and fingerprinting together, and you reach it under Information Protection, Data Loss Prevention, or Data Security Posture Management (DSPM).

Requirements become categories people understand

Classification has a second sense: your organization's classification taxonomy of sensitivity levels, which falls under this skill's first bullet[1], 'Identify sensitive information requirements for an organization's data'. Microsoft's guidance is to name sensitivity labels according to your organization's classification taxonomy[5] for different sensitivity levels, using terms that make sense to users (for example Personal, Public, General, Confidential, and Highly Confidential), to give each label a tooltip with specific examples, and to always test and tailor label names and tooltips with the people who need to apply them. The same guidance recommends a working virtual team that identifies the business and technical requirements. Because the taxonomy describes sensitivity levels, a scheme that only names departments (Finance, HR) leaves users without the handling decision a label is meant to carry.

Detection and handling are separate settings

A classifier supplies evidence about content; it never lists who may read it. SITs are consumed by data loss prevention (DLP) policies, sensitivity labels, retention labels, insider risk management, communication compliance, and auto-labeling policies[3], and each consumer decides the action. The label side can also stop at identification: you can apply a label without any protection settings[6], which gives users a visual mapping of data sensitivity and produces usage reports and activity data, and add protection settings later.

The figure below traces the classification loop: a business requirement selects a classifier, a label or policy acts on the classifier's matches, the explorers (Data explorer and Content explorer below) show what was classified, and Match and Not a Match feedback[7] is used to tune the classifier.

Write the detection requirement and the handling requirement as two separate lines: a classifier answers the first, a label or policy answers the second.

Business requirementClassifier detectsLabel or policy actsExplorers show resultsFeedback tunes classifier
The classification loop this topic follows: a business requirement, a classifier that detects, a label or policy that acts, explorers, and feedback.

Classifier types and what each compares content against

Choosing a classifier is easier once you see that each type compares content with something different. This section sorts the types on the Classifiers page by that comparison and covers the ones Microsoft supplies ready-made.

All types except trainable classifiers are SITs. SITs are pattern-based classifiers[3] that detect information such as social security, credit card, or bank account numbers, and Microsoft groups them as built-in, named entity, custom, and exact data match SITs; document fingerprinting[8] also creates a SIT, from a standard form. Microsoft's pattern-based there describes every SIT, EDM included. Within SITs, this guide's narrower pattern-based SIT (a grouping for this guide, not a Purview menu name) means built-in, named entity, credential, and custom SITs matching a regular expression, keyword list, or function, beside the two SIT types that compare content with something you upload, EDM SITs and fingerprint SITs created from a standard form:

Classifier type Compares content against Where this topic covers it
Pattern-based SIT A regular expression, keyword list, keyword dictionary, or function, plus supporting evidence This section and the four custom SIT sections that follow
Fingerprint SIT The word pattern of a blank form Document fingerprinting
EDM SIT Hashed values from your own source table The two exact data match sections
Trainable classifier What it learned from example items Trainable classifiers

Built-in SITs: use as-is or copy

Microsoft-created SITs show up in the portal by default; they can't be edited, but you can copy them as templates[3] for custom SITs. When a built-in pattern meets the requirement, use it and avoid maintaining a detector of your own; when it needs organization-specific keywords, proximity, or checks, copy it and edit the copy. A few built-in SITs can't be copied[9]: Canada driver's license number, EU driver's license number, EU national identification number, EU passport number, EU social security number or equivalent identification, EU tax identification number, International classification of diseases (ICD-10-CM), International classification of diseases (ICD-9-CM), and U.S. driver's license number. A SIT with 10 text processors or restricted text processors (such as keyword lists, regex, keyword dictionaries, or functions) can't be copied from the UI directly; copy it with PowerShell instead.

Named entity SITs: bundled or unbundled

Named entity SITs[10] are dictionary- and pattern-based classifiers that detect person names, physical addresses, and medical terms and conditions, and they can't be edited or copied[3]. They come in two forms:

  • A bundled named entity SIT, such as All full names or All physical addresses, detects all possible matches in its class; use it as broad criteria.
  • An unbundled named entity SIT, such as Australia Physical Addresses or Blood Test Terms, has a narrower focus, such as a single country or region or a single class of terms; use it when a policy needs a narrower detection scope.

They suit open-ended requirements such as recognizing names you have never seen; a keyword list of known customer names only matches the names on it. Two prerequisites apply: Entity Match and other advanced classifiers are offered only to users with E5 licenses[10], and using a bundled SIT in an Endpoint DLP policy requires the Endpoint DLP setting Advanced classification scanning and protection[3].

Credential SITs: the bundle or one member

The All credentials SIT contains all the credential scanning SITs available in the portal[3], and each member is a credential scanning SIT that can be used standalone. Choose All credentials when any credential counts; choose the single member when only one credential type is in scope, because the bundle also matches every other credential type. Credential scanning SITs require an E5 license.

The figure below shows the classifier types as nested groups: SITs (pattern-based, fingerprint, and EDM) beside trainable classifiers (pretrained and custom).

Start from what the requirement must compare content against, check whether a built-in, named entity, or credential SIT already does it, and build something custom only when none does.

ClassifiersSensitive information typesPattern-based SITBuilt-inNamed entityCredentialCustomFingerprint SITblank formEDM SITyour own tableTrainable classifiersPretrainedready to useCustomyour samples
Classifier types on the Purview Classifiers page: SITs (pattern-based, fingerprint, EDM) and trainable classifiers (pretrained, custom).

Anatomy of a custom SIT pattern

Every custom SIT, whether built from scratch or copied from a built-in one, is assembled from the same parts; this section defines them before the next sections refine them. A SIT entity has a name, a description, and a pattern[3], and each pattern consists of four components:

Pattern component What it is
Primary element What the SIT looks for: a regular expression with or without checksum validation, a keyword list, a keyword dictionary, or a function
Supporting element Corroborative evidence, such as the keyword Social Security Number near a nine-digit number, that raises confidence that a match is real
Confidence level High, medium, or low, reflecting how much supporting evidence was found with the primary element
Proximity The number of characters allowed between a supporting element and the primary element

Choose the primary element by the kind of requirement: a regular expression for a structured identifier, a keyword list or keyword dictionary for known terms, or one of the preconfigured functions[9]. A SIT can hold several patterns, each with different elements and confidence levels, which is how one SIT returns a high-confidence match when strong evidence is present and a lower-confidence match when little is. Custom SITs created in the portal are added to the Microsoft.SCCManaged.CustomRulePack rule package. A rule package[11] is the XML container for SIT rules, identified, like each rule in it, by a unique GUID (Microsoft's procedures also say rule pack, and rule collection for an exported set); Creating a custom SIT in Security & Compliance PowerShell shows its skeleton. A deterministic format, such as a document ID shaped like a timestamp, needs a regular expression rather than a trainable classifier, because the format can be written down exactly.

Confidence levels

The three levels have values: low is 65, medium is 75, and high is 85[3]. High confidence returns the fewest false positives but might return more false negatives; low or medium returns more false positives but few to zero false negatives. Each selection includes the levels above it: low returns low, medium, and high matches; medium returns medium and high; high returns only high. An instance count, also called the occurrence or match-count threshold, sets how many matches a policy needs, and Microsoft suggests high-confidence patterns with low instance counts (say five to 10) and low-confidence patterns with higher counts (20 or more). The count and the confidence level do different jobs: the level reflects the evidence behind each match, so raising the count never repairs an overbroad primary pattern.

Proximity needs the whole supporting value

Proximity counts characters, and a supporting element must fall entirely inside the window. In Microsoft's example a nine-digit primary element needs an AccountNumber, Name, or DateOfBirth value within 250 characters, and a window that extends into the Name value without including the entire value doesn't match[3]. Proximity and confidence are separate settings: picking a higher confidence level never widens the window, so evidence that legitimately sits farther away needs a larger proximity value, and a wider window is worth testing so that unrelated records nearby don't supply accidental corroboration.

Regular expression rules

Purview SITs use the Boost.RegEx 5.1.3 engine[9], and custom SIT regex patterns must follow three rules:

  • Don't use positional anchors such as ^ and $; scanning gives no guarantee where content starts or ends.
  • Use one primary capturing group, the only capturing group, with noncapturing groups (?:...) for any other grouping.
  • Put every match variant inside that single group with |; top-level alternation between separate capturing groups is blocked during validation.

A valid pattern: three ID formats as alternatives inside one capturing group (Microsoft's example)

(?:[\s,;])([A-Z]{2}[0-9]{3}|[A-Z]{2}[A-Z]{4}[0-9]|[A-Z]{2}[0-9]{5})(?:[\s,;])

How the parts decide a match

The figure below follows one candidate through the four pattern components in the table above: the primary element matches, supporting evidence is sought inside the proximity window, the evidence found sets the confidence level, and a policy keeps the match only if that level meets the policy's own confidence setting; otherwise the match is dropped.

A pattern's accuracy therefore rests on three separate controls: the primary element decides what is a candidate, proximity decides which nearby evidence counts, and the confidence level decides how much evidence a policy demands.

Primary element matchesSupporting evidenceinside proximity windowEvidence sets confidenceConfidence meetspolicy setting?YesNoMatch keptMatch dropped
How a SIT pattern turns a primary element match into a kept or dropped match through proximity, evidence, and confidence.

Refining what a SIT pattern matches

Once a pattern's parts exist, accuracy comes from refining them, and each refinement fixes a different kind of error: a vocabulary that is too small or changes often, an identifier with the right shape that is still invalid, a known value that should never match, or Chinese or Japanese mixed with single-byte text, which Purview's added word spacing splits.

Keyword lists and keyword dictionaries

Both supply vocabulary, as a primary or a supporting element. Keyword lists are limited in size[12] and, in SITs created with PowerShell, need XML edits to change. Keyword dictionaries are simpler to manage at larger scale and support any language; all dictionaries in a tenant share a combined limit of 1 MB after compression, close to one million characters. A changing vocabulary, such as a list of project code names, therefore belongs in a dictionary rather than in a regular expression that hard-codes every value. In the portal you add a dictionary while creating or editing a SIT pattern: upload a dictionary file in TXT or CSV format, choose from existing dictionaries, or enter keywords manually and name a new dictionary. In Security & Compliance PowerShell, put each keyword on a separate line in a text file saved with Unicode encoding, then import it:

Creating a keyword dictionary from a Unicode text file (Microsoft's procedure)

# Read the Unicode-encoded file (one keyword per line) as bytes
$fileData = [System.IO.File]::ReadAllBytes('<filename>')
# Create the dictionary from those bytes
New-DlpKeywordDictionary -Name <name> -Description <description> -FileData $fileData

Validators: is a well-shaped identifier valid?

A regular expression checks only shape; regular expression validators[13] test meaning:

  • The checksum validator multiplies each digit by a weight, applies a modulus, and compares the result with the check digit at a set position; give the check digit weight 0 if it isn't part of the calculation.
  • The date validator checks a date embedded in the identifier, in a chosen date format, starting at an offset; a date that occupies the first six digits has offset 0.
  • The Luhn check validator requires the match to pass the Luhn algorithm.
  • Function processors such as Func_India_Aadhar validate your own regular expression with the logic a built-in SIT requires.

A supporting keyword can't do a validator's job: an employee keyword next to a nine-digit number says nothing about whether its first six digits are a valid hire date.

Additional checks: drop known false positives

Additional checks[13] change which matches count without rewriting the primary pattern:

Additional check What it does
Exclude specific matches Ignores listed values, such as the test card number 4111111111111111
Starts or doesn't start with characters Requires or rejects given leading characters
Ends or doesn't end with characters Requires or rejects given trailing characters
Exclude duplicate characters Ignores matches whose digits are all the same, such as 111111
Include or exclude prefixes Requires or rejects keywords immediately before the match
Include or exclude suffixes Requires or rejects keywords immediately after the match

Prefix and suffix checks look only at text immediately next to the match, whereas a supporting element only has to fall inside the proximity window. To stop a known test value matching, exclude it; raising the confidence level doesn't help, because the test value can still appear beside convincing evidence.

Chinese or Japanese mixed with single-byte text

Purview adds a space between each word in languages that use double-byte characters and removes special characters such as punctuation[9], so a keyword or regex that mixes Chinese or Japanese with English or other single-byte characters needs two variants. To detect 机密的document, add both 机密的document and 机密的 document. Keep keywords that contain double-byte characters in a separate dictionary or keyword list from English-only keywords, and use string match rather than word match in such a keyword list, as Microsoft recommends.

Pick the refinement by the error you see: a large or changing vocabulary goes in a dictionary, invalid identifiers fall to a validator, known noise falls to an additional check, and mixed-language terms need both spacing variants.

Creating a custom SIT in Security & Compliance PowerShell

The portal covers most custom SITs, but Security & Compliance PowerShell offers more configuration options[3] and is the route for a SIT the portal can't copy, such as one with 10 text processors (see Classifier types and what each compares content against). Microsoft's walkthrough customizes the built-in Credit Card Number rule[11], and the same approach applies to other built-in SITs. You can't modify the default rules directly, so the procedure always produces a new SIT:

  1. Export the rule package. Run Get-DlpSensitiveInformationTypeRulePackage, store the result in a variable, and write its SerializedClassificationRuleCollection property to an XML file with [System.IO.File]::WriteAllBytes.
  2. Extract the rule. In the exported file, find the rule under <Rules> (the credit card rule references Func_credit_card) and paste it into the rule package template shown below.
  3. Give it new identities. Rule packages and rules are identified by unique GUIDs, so generate two: one for the RulePack id, and one to replace the built-in rule's Entity id. The Resource idRef under LocalizedStrings must equal the new Entity id, because that resource carries the rule's localized name and description.
  4. Edit the pattern, for example by removing the corroborative evidence, changing patternsProximity, or adding your own Keyword list.
  5. Save with Unicode encoding. Save the file as XML with Unicode encoding; the rule won't work if the file is saved with a different encoding. The template's XML declaration is encoding="utf-16".
  6. Import and verify. Run New-DlpSensitiveInformationTypeRulePackage -FileData ([System.IO.File]::ReadAllBytes('C:\custompath\MyNewRulePack.xml')), type Y to confirm, then run Get-DlpSensitiveInformationType to check the upload and the display name before you add the SIT to a policy.

Changing only the display name while keeping the exported Entity id doesn't produce a distinct SIT: the GUID, not the localized name, identifies the rule.

Skeleton of a custom rule package, abridged from Microsoft's template (quoted placeholder text marks values you supply; ... marks removed lines)

<?xml version="1.0" encoding="utf-16"?>
<RulePackage xmlns="https://schemas.microsoft.com/office/2011/mce">
  <RulePack id="new GUID for the package">
    ...
  </RulePack>
  <Rules>
    <Entity id="new GUID for the rule" patternsProximity="300" recommendedConfidence="85">
      <Pattern confidenceLevel="85">
        <IdMatch idRef="Func_credit_card" />
        ...
      </Pattern>
    </Entity>
    <LocalizedStrings>
      <Resource idRef="same GUID as the new Entity id">
        ...
      </Resource>
    </LocalizedStrings>
  </Rules>
</RulePackage>

The figure below lists the six steps in order.

Identity and encoding are the two details this procedure insists on: two new GUIDs with a Resource idRef that matches the new Entity id, and a file saved with Unicode encoding.

Export the rule packageExtract the ruleGive it new identitiesEdit the patternSave with Unicode encodingImport and verify
Six steps to create a custom SIT from an exported built-in rule in Security & Compliance PowerShell.

Testing and safely changing custom SITs

A custom SIT that saves without errors can still misbehave in production. This section covers how to test it, why results differ between workloads, how existing content picks up a change, and what to check before you edit or delete a classifier.

Test with sample files

You can test a SIT by uploading a sample file[3]; the results show the number of matches for each confidence level, and testing works for built-in SITs, custom SITs, trainable classifiers, and exact data match. Test files that should match and files that shouldn't. The tenant needs at least one Exchange Online license, or the Test SIT option is greyed out. A test evaluates only the file you upload: a successful uploaded-file test doesn't establish that historical files have the same classification results[14].

Validate in every workload

The same SIT can produce different results depending on where content is evaluated[3]: email in transit in Exchange, files at rest in SharePoint and OneDrive, and files on managed devices in Endpoint DLP. Content-extraction limits such as maximum file size and the number of characters scanned also affect detection. Validate a custom SIT in each workload where you plan to use it (for example a mail flow rule, a SharePoint or OneDrive policy, and an endpoint policy) and tune it against the false positives and false negatives you find there.

Endpoint DLP evaluates every SIT in the tenant

Endpoint DLP classifies files based on all SITs available in the tenant, including custom SITs, regardless of their use in any DLP policy[3]. An untuned custom SIT that matches many files therefore causes excessive classification traffic even when no policy references it. Microsoft's remedy is to remove unused SITs and redesign SITs that match most of your organization's files; deleting a SIT's policy reference leaves the SIT, and its classification traffic, in place.

Existing files keep their earlier results

SharePoint and OneDrive content is classified by the search crawler, so a new custom SIT is identified in existing content only after that content is recrawled[9]; crawling runs on a schedule, and you can manually request a recrawl of a site collection, list, or library. In the same way, changing a custom SIT definition doesn't by itself reclassify unchanged historical files[14], and restarting an auto-labeling simulation only reevaluates policy matches without reclassifying the underlying content. On-demand classification, covered in On-demand classification, classifies such files again within the scan's scope.

Before you edit or delete a classifier

Check authority and dependencies first. The Information Protection Admin role can create, edit, and delete DLP policies, sensitivity labels and their policies, and all classifier types[15]; it is assigned through role groups such as Information Protection Admins and Compliance Administrator, so managing classifiers doesn't require Global Administrator. Compliance Administrator names both a Microsoft Entra role and the Microsoft Purview role group mapped to it[16]: here, for on-demand scans, and for OCR it means the role group, while page access to the explorers uses the Microsoft Entra role. Information Protection Analyst and Information Protection Investigator have view-only access to all classifier types. On the Classifiers page you can see how many policies use a classifier and inspect those dependencies[4] before changing it. The page's Classified items count shows how many items were found with that classification, which says nothing about which policies would lose their detector.

Test with files, validate in each workload, keep the tenant's SIT inventory lean for endpoints, refresh historical results deliberately, and confirm both the role and the dependent policies before you edit or delete.

Document fingerprinting

Many organizations move sensitive information on standard forms: a patent template, an employee information form for Human Resources, a government or HIPAA compliance form. Document fingerprinting[8] takes such a form and creates a fingerprint SIT that detects documents created from the same template.

How a fingerprint is built

Everyone who fills in a form starts from the same original words, so the form's word pattern identifies its copies. You upload the blank form, and Purview converts its word pattern into a document fingerprint: a small Unicode XML file containing a hash value that represents the original text. Only the hash is stored, and the original document can't be reconstructed from it. In the portal, choose Create Fingerprint based SIT on the Sensitive info types page, enter a name and description, upload the template, and optionally adjust the requirements for each confidence level. The template has rules of its own:

  • Its extracted text must be between 256 and 204,800 characters.
  • Text in embedded documents isn't used, so supply templates without embedded documents.
  • Fingerprints are stored in a separate rule pack (a rule package of its own) with a 300 KB maximum size, which allows approximately 100 fingerprints per tenant.
  • Creating a new fingerprint from a template that already has a fingerprint isn't supported.

Partial or exact matching

With partial matching[8] you give each confidence level (low, medium, high) the percentage of text, between 30% and 90%, that a file must share with the fingerprint. With exact matching you set the high confidence level to Exact: only files with exactly the same text as the fingerprint are detected, and even a small deviation means no match. Use partial thresholds when completed or altered forms must be caught, and Exact when only unchanged copies count. A list of file hashes can't do the partial job, because filling in a form changes the file. In PowerShell, New-DlpSensitiveInformationType creates a fingerprint SIT and Set-DlpSensitiveInformationType edits one, with the same settings as parameters:

Creating a fingerprint SIT with partial-match thresholds (Microsoft's example)

# Read the blank template as bytes
$Employee_Form = ([System.IO.File]::ReadAllBytes('C:\My Documents\Contoso Customer Form.docx'))
# -ThresholdConfig sets the low, medium, and high match percentages; -IsExact $false keeps partial matching
New-DlpSensitiveInformationType -Name "Contoso Customer Confidential" -FileData $Employee_Form -ThresholdConfig @{low=40;medium=60;high=80} -IsExact $false -Description "Message contains Contoso customer information."

Licensing, file eligibility, and locations

E5 is the licensing line for managing fingerprints: Microsoft recommends that E5 customers update existing fingerprints to the full feature set, and E3 customers who don't upgrade can't modify existing fingerprints or create new ones after April 2023[8]. Existing fingerprints and the policies that use them should continue to work. To move an existing fingerprint to the newer features, edit its SIT and upload the same fingerprint file again[8]: that re-uploads the template into the existing fingerprint SIT, unlike creating a new fingerprint from a template that already has one, the unsupported case under How a fingerprint is built.

Fingerprint SITs work in DLP policies scoped to Exchange, SharePoint, OneDrive, Teams, and devices, and in auto-labeling for Exchange, SharePoint, and OneDrive[8]; devices also need Advanced classification scanning and protection turned on. Fingerprinting doesn't detect sensitive information in password-protected files, files that contain only images, documents that don't contain all the text from the original form, or files larger than 4 MB. The missing-text limitation is quoted as Microsoft's limitations list states it, while the same page documents partial matching from 30%; test altered forms against your thresholds rather than assume either reading. A fingerprint SIT can also be used in Exchange mail flow rules, but mail transport never sees a file copied to a USB drive; protecting that path needs a DLP policy scoped to devices.

The figure below shows the path from blank form to policy.

Fingerprint the blank form, set thresholds for filled-in copies, and confirm the license, the file's eligibility, and the target location before relying on the policy.

Upload the blank formWord pattern extractedHash stored asfingerprint SITDLP or auto-labeling policy
How document fingerprinting turns a blank form into a fingerprint SIT that a DLP or auto-labeling policy uses.

Exact data match: detecting your own records

A national ID SIT matches numbers that meet its pattern requirements. When only your own records count, such as the IDs of current customers, exact data match (EDM) based classification[17] creates a custom SIT that refers to exact values in a database of sensitive information. All EDM SITs are created from scratch[3]. The database can contain up to 100 million rows of data and can be refreshed daily, so the SIT stays current as employees, patients, and clients come and go, and matching compares one-way cryptographic hashes rather than clear-text values.

The three parts of an EDM SIT

  • Sensitive information source table: your data. Column headers are the field names, and each row is one record.
  • Schema: an XML file that names the DataStore and the fields, with a one-to-one mapping between schema field names and source table column names. It also marks which fields are searchable, which corroborative evidence fields require multi-token match mode, and which fields support configurable matches, such as ignoring delimiters and case. Field meaning comes from the schema, so a column-mapping error is fixed in the schema, never by renaming the SIT.
  • Rule package: the primary element (the data field used in exact lookup), the classification (the SIT match that triggers an EDM lookup), supporting elements, confidence levels, and proximity.

The primary field: discover first, then look up

EDM doesn't scan content for table values directly. Primary elements must be discoverable through an existing SIT[17]: for a U.S. social security number field you associate the U.S. Social Security Number (SSN) SIT. When that SIT finds a candidate, EDM compares its hash with the source table, then looks for the record's supporting fields within proximity, individually and in combination, and the supporting evidence found sets the confidence level as it does for any SIT. Multi-token matching lets supporting fields such as full names or street addresses match when associating them with a SIT isn't easy. The primary field's SIT matters: the new experience (one of the two portal ways to create EDM SITs, compared in New experience or classic experience below) warns when a primary field is mapped to a loosely defined SIT[18], which causes lookups on large numbers of unrelated strings, and when the primary field's values repeat across many rows, which can cause timeouts; both can result in missed detections and poor performance. Prefer a strongly defined SIT such as U.S. Social Security Number and a field with distinctive values, not a common surname.

New experience or classic experience

You can create EDM SITs in the new experience, the classic experience, or PowerShell. The new experience creates the schema and SIT together[18], one schema per SIT. It can infer the schema from a sample file with the same headers and 10 to 20 rows of representative data, which must not be sensitive because the sample isn't hashed. It generates high and medium confidence rules for each primary field and reports status from Data not yet uploaded to Indexing complete. A tenant can have at most 10 EDM schemas. Choose the classic experience when several EDM SITs must share one schema, when you need more than 10 EDM SITs, or when you need a custom schema name (the new experience names the schema after the SIT, for example PatientNumberschema). Schemas created in the classic experience or uploaded as XML with PowerShell aren't viewable or manageable in the new experience.

Where EDM SITs work

Service Locations
Microsoft Purview Data Loss Prevention SharePoint, OneDrive, Teams chat, Exchange Online, devices
Microsoft Defender for Cloud Apps SharePoint, OneDrive
Auto-labeling (service side) SharePoint, OneDrive, Exchange Online
Auto-labeling (client side) Word, Excel, PowerPoint, Exchange desktop clients

The same services table[17] also lists Customer Managed Key, eDiscovery, and Insider Risk Management locations.

The figure below traces how an EDM SIT evaluates content.

EDM narrows detection to your own records, but only after the primary field's SIT finds a candidate; the schema and the authoring experience then decide how many EDM SITs you can build on one table.

SIT finds candidate valueHash looked up in tableValue in yoursource table?NoYesNo EDM matchCheck supportingfields in proximityConfidence level set
How an EDM SIT evaluates content: a SIT finds a candidate, EDM looks up its hash, then checks supporting fields to set confidence.

Hashing and uploading EDM source data

Creating an EDM SIT is a multiphase process[18], and a SIT that exists isn't yet a working detector: the new experience reports status values such as Data not yet uploaded, Data upload complete, Indexing complete, Data upload failed, and Data indexing failed. Check that status before testing the SIT or relying on a policy that depends on it. This section covers the upload phase, done with the EDM Upload Agent[19].

Who may upload

The upload account is a Microsoft 365 work or school account that belongs to the EDM_DataUploaders security group; a Groups Administrator (the best least-privilege option) or a User Administrator creates the group and adds members. On the upload computer you authorize the agent from its installation folder with the /Authorize command and sign in with that account. Upload rights come from this group membership; publishing a sensitivity label to someone grants nothing here.

Validate before you hash

Download the schema with EdmUploadAgent.exe /SaveSchema, then validate the data file against it with EdmUploadAgent.exe /ValidateData /DataFile [data file] /Schema [schema file]. The agent expects CSV by default and also accepts tab- or pipe-separated data when you indicate the separator. The common failure is a mismatched number of columns, caused by commas or quote characters inside values; fix the export instead (surround such values with double quotes, or use the tab-delimited format, which Microsoft recommends as less susceptible to these issues), because a detection setting such as a confidence level can't correct a parsing error.

Two computers or one

The agent hashes the table with a salt value, added automatically unless you provide your own with /Salt, and uploads only the hashes; custom hashing isn't supported. Microsoft's best practice is the two-computer method, which keeps the clear-text table off any computer with a direct connection to the internet:

  1. On a computer in a secure environment, validate the data file and run /CreateHash, which outputs an EdmHash file and an EdmSalt file.
  2. Securely copy both files to a computer that can connect directly to your Microsoft 365 tenant. Both computers must run the same version of the EDM Upload Agent.
  3. On that computer, authorize the agent and run /UploadHash.
  4. Confirm with /GetDataStore (the data stores and when each was last updated) and /GetSession (the uploads to one store).

The single-computer method runs /UploadData, which hashes and uploads in one step, so the clear-text source table must reside on the computer connected to the tenant. The new experience's schema sample file is a separate upload that isn't hashed, which is why it must hold nonsensitive values. A spreadsheet protected with a document password is no substitute for any of this, because only the agent's two documented methods can be used to hash and upload the data.

The figure below shows the two-computer method.

The reference data must be uploaded and indexed before an EDM SIT can match it.

Secure computerValidate data fileCreate hash and saltCopy both filesUpload computersame agent versionAuthorize the agentUpload the hashCheck the data store
The two-computer EDM upload: hashing on a secure computer, then uploading from a computer connected to the Microsoft 365 tenant.

Trainable classifiers

Some content has no pattern to match: a legal agreement, a press release, a price quote. A trainable classifier[20] learns how to identify a type of content by looking at examples, which suits content that can't easily be identified with manual or automated pattern-matching methods. There are two kinds: pretrained classifiers, created by Microsoft and shown with the status Ready to use, and custom trainable classifiers, which you train when the pretrained ones don't cover your content. Document fingerprinting is no alternative for agreements written independently of each other, because without a shared template there is no word pattern to fingerprint.

Check the consumers and limits before training

Classifiers can be conditions for auto-labeling with sensitivity labels, auto-applying retention labels, DLP, and communication compliance, among other uses (insider risk management also considers trainable classifiers in images, as Turning on OCR for existing classifiers notes), but communication compliance supports Microsoft-provided trainable classifiers only[20], so a custom classifier can't meet a communication compliance requirement. Support for custom classifiers is limited to English, and classifiers only work with items that aren't encrypted.

Seed content

Seeding means a person selects two sets of samples[21]: at least 50 and up to 500 positive samples that strongly represent the category, and at least 150 and up to 1,500 negative samples that clearly don't belong. Positive samples alone aren't enough, because the negative set is what shows the classifier where the category ends. The seed rules:

  • Only text-based files are supported; image files and image-only PDFs can't be used.
  • Put the positive and negative samples in separate SharePoint folders, each holding only seed content, on a communication site or another SharePoint site type, but not in a Teams folder type or a OneDrive folder.
  • Allow at least an hour for a newly created seed site and folder to be indexed before creating the classifier.
  • The account you use must have access to both seed folders.
  • The classifier processes up to the 2,000 most recently created samples, by file creation timestamp, so keep the folders controlled: newer stray files can displace the samples you chose.

Train, review, publish

Microsoft's procedure[21] says to sign in "with either Compliance admin or Security admin role access"; Compliance Administrator and Security Administrator each exist as a Microsoft Entra role and as the Purview role group mapped to it[16] (see Before you edit or delete a classifier). Then select Create trainable classifier and point it at the positive and negative folders. Within 24 hours or less[21] it builds a prediction model, and its status moves from In progress to Training is complete and items have been tested. Review its predictions, sorting out true positives, true negatives, false positives, and false negatives; if they aren't accurate enough, add data and retrain before publishing, then choose Publish for use. By default only the user who created a custom classifier can train it and review its predictions.

Improving a classifier

An unpublished classifier is improved by reviewing the test results, adding data, and restarting the training. Retraining a published custom classifier isn't supported[20]: remove the classifier and start over with larger sample sets. Changing a policy's settings never retrains a classifier, because policy conditions don't rebuild the learned model.

Display in Content explorer (preview)

In preview, trainable classifiers in Content explorer automatically display the number of incidents found in SharePoint and Teams[20] without requiring any labeling. Despite the word incidents, this is a Content explorer display of what the classifiers found: a request to Microsoft Support disables the display of your sensitive data that isn't used in any labeling policies[20] within Content explorer. Scanning can also be disabled by request, but then sensitivity labeling and DLP policies that use those classifiers don't work, so review dependent policies before asking.

The figure below shows the custom classifier lifecycle, including the retraining loop that ends at publication.

Check the consumer first, seed balanced and controlled folders, review predictions before publishing, and remember that publishing ends retraining.

Seed the two foldersTrain the modelReview predictionsAccurateenough?NoAdd data,retrainYesPublish for usePublished: no retraining;remove and recreate
Custom trainable classifier lifecycle: seed, train, review, retrain while unpublished, publish, then remove and recreate to improve.

Data explorer and Content explorer

Once classifiers and labels are in use, the explorers answer one question: where is classified content right now? Data explorer[22] sits under Information Protection > Explorers, and Microsoft's Classifiers documentation[4] directs you to it to review items with classification matches and see where those items are located. Content explorer[7] sits under Data Lifecycle Management > Explorers, and Microsoft Learn titles its page Get started with Content Explorer (classic), marking it as the classic experience. Both show a current snapshot of the items that have a sensitivity label or a retention label or have been classified as a SIT, and both can show trainable classifier results too. Each lets you natively view the items summarized on the overview page[7], and the access and timing rules that follow hold for both unless a sentence names one.

Finding items

Type a label, classifier, or SIT name into the filter box or browse for it, select a data source or location, and drill down to the item. After you drill into a location, a Filter tool appears: for Exchange or Teams you search on the mailbox's full email address, and for SharePoint or OneDrive on site names, folders, and files (a full site name, a file name, the start of a file name, text after an underscore, or a file extension). Export creates a CSV file listing whatever the pane is focused on, which is not the same as a tenant-wide report. Content explorer can also save customized views.

Three layers of access

Access is split on purpose, because the explorers let you read the contents of scanned files and their permissions supersede permissions assigned locally to items:

What the user can do What grants it
Open the explorer page The Microsoft Entra Compliance Administrator, Security Administrator, or Compliance Data Administrator role (Data explorer also lists Global Administrator); Information Protection roles and role groups also apply
List items and their locations Data Explorer List viewer or Content Explorer List viewer role group (the data classification list viewer role)
View item contents, and item names in list view Data Explorer Content viewer or Content Explorer Content viewer role group (the data classification content viewer role)

Membership in the page-access roles doesn't allow you to view the list of items or the contents of the items[22], so the Microsoft Entra Compliance Administrator role alone doesn't let an investigator preview a document. The list viewer and content viewer role groups are independent and not cumulative; a user who must list locations and read files needs both. Both explorers support administrative units for scoping these roles, and Role management role holders assign the role group memberships.

Timing and encrypted items

Counts can take up to seven days to update, and 14 days for files in SharePoint[7], so a count taken right after a change isn't a completeness check. Document preview and download are disabled for encrypted documents, a product restriction that extra explorer permissions don't lift. Sensitivity labels must be enabled for files in SharePoint and OneDrive[23] for that data to surface (see Implement and manage sensitivity labels in Microsoft Purview), and encrypted sensitivity labels aren't surfaced on content explorer for SharePoint and OneDrive[22].

A snapshot, not a history

The explorers don't record who did what, or when. Activity explorer[24] provides a historical view of activities on labeled content, built from the Microsoft 365 unified audit logs, reports on up to 30 days of data, and includes a filter set for sensitivity labels applied, changed, or removed. A label publishing policy isn't an inventory either: its scope determines which users can see and apply the label[6], not which content carries it.

Use the explorers for what is classified where, grant list and content access separately, allow for update delays, and go to activity explorer for history.

Match and Not a Match feedback

Classifiers produce false positives and false negatives, and the feedback experience is how you confirm which matches are right before you change anything. In Data explorer, Content explorer, and the matched items pages for a SIT or trainable classifier, you can confirm whether a matched item is a true positive (Match) or a false positive (Not a match)[25] for SharePoint and OneDrive items, using the Contextual Summary tab where the classifier supports it; the DLP alerts page extends this to Exchange email. If you choose the wrong option, select Withdraw feedback.

Not every classifier supports this feedback:

Classifier Match and Not a Match
Built-in SIT Yes
Custom SIT Yes
Fingerprint SIT Yes
EDM SIT No
Named entity SIT No
Credential SIT No
Pretrained trainable classifier Yes
Custom trainable classifier Yes

Two limits apply: the experience is only available for items created or updated after it was enabled for the tenant, and DLP policies that use the SITs or trainable classifiers must already be deployed and have matched items.

Feedback records a verdict; it doesn't edit the classifier. You act on it by tuning, and Microsoft lists options such as increasing the thresholds of SITs found, pairing a low confidence level with a high instance count (or a higher confidence level with a low count), cloning built-in SITs to add conditions, excluding known prefixes, suffixes, or patterns (for example your organization's own phone numbers in email signatures), adding dictionary-based SITs as conditions, and, for named entity SITs like All Full Names, setting a higher instance count threshold such as 10 or 50. A custom trainable classifier that keeps failing is rebuilt rather than retrained once published, as Trainable classifiers explains. Raising a policy's instance count before reviewing disputed items changes how many detections trigger the policy, not whether those detections are correct.

Review the disputed matches first, then tune the SIT or rebuild the classifier; the verdicts themselves change no rule.

On-demand classification

Existing files keep the classification results they already have, as Testing and safely changing custom SITs explains. On-demand classification[26] identifies and classifies sensitive content in historical data stored in SharePoint, OneDrive, and endpoints: files that were never classified, files not classified or modified for a long time, and files that need updated classification.

What a scan changes

On-demand classification updates classification results, and matching policies determine any resulting actions. For SharePoint and OneDrive, updated results can trigger active policies, including policies that apply or remove sensitivity labels, so an auto-labeling policy you're testing can stay in simulation while other active policies act on the same files. On-demand classification isn't an auto-labeling simulation. For SharePoint and OneDrive, each scanned file is evaluated against Data Loss Prevention, Information Protection, Data Lifecycle Management, and Insider Risk Management policies.

Running a scan

  1. Review active policies that could act on the refreshed results.
  2. Create the scan. Go to Information Protection > Classifiers > On-demand classification (also available under Data loss prevention), select New scan, and define the name and description, the scope and locations, the classifiers to scan for, the file last-modified date range, and the file extensions. By default a scan includes all items created or modified within the past year, all supported file extensions, and every classifier configured in the tenant; you can choose up to 50 specific classifiers at a time.
  3. View the estimation to review items found and estimated cost, and use Edit scan to narrow or expand the scope.
  4. Start classification. Classification can start up to 30 days after estimation, though a shorter gap gives more accurate final counts and costs.

Permissions, cost, and limits

Running a scan requires membership in the Compliance Administrator role group; viewing classification results requires Content Explorer Content Viewer or Content Explorer List Viewer. The feature uses pay-as-you-go billing, per-user licensing, or both. Scan cost follows the total number of files scanned, not the number of classifiers, and EDM classifiers cause previously classified files to be rescanned, which raises cost unless you exclude the EDM classifier from the scope. If you select specific classifiers, only those classifiers have their results updated. For SharePoint and OneDrive a scan can process up to 150,000 locations and 100 million files, and Content explorer updates within seven days of scanning.

The figure below shows the scan sequence.

Review active policies, scope the dates and classifiers to the files you actually mean, check the estimate, and only then start classification.

Review active policiesCreate the scanView the estimationStart classificationMatching policies act
On-demand classification scan sequence for historical files, from reviewing active policies to the policies that act on refreshed results.

Turning on OCR for existing classifiers

A screenshot of a spreadsheet or a scanned ID card holds text that a text-based classifier can't read. Optical character recognition (OCR)[27] lets Purview scan images for sensitive information; it is an optional feature that you enable at the tenant level and then point at locations.

Billing comes first

  1. Azure pay-as-you-go subscription. The Global admin verifies that one exists for the tenant, or creates it.
  2. Estimate OCR charges with the OCR cost estimator.
  3. Microsoft Syntex billing. A Global or SharePoint admin follows Set up Microsoft Syntex billing in Azure to add a subscription for OCR. You don't need to set up Microsoft Syntex itself.
  4. Configure OCR. In the Microsoft Purview portal, select Settings > Optical character recognition (OCR), select the locations where you want to scan images, select the groups to include in or exclude from OCR scans, and select Done. OCR settings generally take effect about an hour after you turn them on.

Configuring OCR[27] requires membership in one of these role groups: Compliance Administrator, Compliance Data Administrator, Global Administrator, Information Protection, or Information Protection Admins. Microsoft's roles reference[15] lists Information Protection Admin, singular, as a role held by role groups including that last entry, Information Protection Admins; Before you edit or delete a classifier explains the distinction.

No new classifiers

You don't create separate classifiers for images. Once OCR is configured, existing SITs, EDM SITs, trainable classifiers, and fingerprint SITs scan images as well as documents and emails[27], so a second credit card SIT for screenshots is unnecessary. Only images uploaded after OCR is enabled are scanned, so turning it on gives no retroactive coverage of images already stored.

Locations and the solutions that act on image text

Location Supported solutions
Exchange Data loss prevention, auto-labeling policies, auto-apply retention label policies
SharePoint sites Data loss prevention, insider risk management, auto-apply retention label policies
OneDrive accounts Data loss prevention, auto-apply retention label policies
Teams chat and channel messages Data loss prevention, insider risk management
Devices Data loss prevention, insider risk management

In this locations table[27], auto-apply retention label policies support keywords and SITs, and insider risk management considers SITs and trainable classifiers present in images for risk scoring.

The figure below shows the four enablement steps and their result.

Set up billing, choose locations and groups, and your existing classifiers can then evaluate the text in eligible new images within that scope after the settings take effect.

Azure pay-as-you-gosubscriptionEstimate OCR chargesMicrosoft Syntex billingConfigure OCRlocations and groupsExisting classifiersscan new images
OCR enablement sequence, from the Azure pay-as-you-go subscription to existing classifiers scanning newly uploaded images.

OCR coverage by location

What OCR scans differs by location, so check the location's limits before you design a policy around image text. The limits below come from the OCR documentation[27].

Size, resolution, and format

Requirement Limit
File size, Exchange and Teams 20 MB maximum
File size, SharePoint, OneDrive, and Windows and macOS endpoints 50 MB maximum
Image resolution 50 × 50 pixels minimum, 16,000 × 16,000 pixels maximum
Text extracted The first 2 million characters

For formats, Exchange scans JPEG, JPG, PNG, BMP, TIFF, and scanned PDFs[27], plus embedded images in DOCX, PPTX, XLSX, RAR, TAR, ZIP, 7z, and hybrid PDFs, with a limit of 20 embedded images scanned per file. Teams and Windows and macOS endpoints scan JPEG, JPG, PNG, BMP, TIFF, and image-only PDFs. SharePoint and OneDrive support a much longer list that includes camera raw formats, HEIC and HEIF, scanned and hybrid PDFs, and embedded images in DOCX, PPTX, and XLSX.

Exchange: direction scope and policy tips

By default, OCR scans incoming email (from outside the organization), internal email, and outgoing email (sent outside the organization)[27]. To exclude incoming mail, change the OCR scope from All sender groups to Specific sender groups and specify the internal groups to scan. To scan only mail sent outside the organization, select the option under Advanced Setting (Only Exchange); then neither incoming mail nor internal communications are scanned. DLP policy tips aren't supported for images in Exchange. OCR scope is set in these OCR settings (locations, groups, and the Exchange option), not by a sensitivity label's publishing audience.

Windows and macOS endpoints

When OCR is on for devices, the devices send images to the cloud for scanning[27], subject to a default bandwidth limit of 1,024 MB of data per device per day. OCR stops scanning images once a device reaches that daily limit, so if coverage must continue you increase the bandwidth limit; adding SITs to a policy doesn't change it. If you exclude a path in the Endpoint DLP settings, OCR doesn't scan images in that folder either, and network settings must not obstruct OCR: a wildcard allowing blob.core.windows.net endpoints should be present.

Besides arriving before OCR was enabled, an image can fall outside OCR coverage through its size or format, the Exchange direction scope, or a device's bandwidth limit and excluded paths.

How OCR is charged

OCR bills by images scanned, not by the policies that use the results. Each image scanned counts as one transaction[27]: a standalone JPEG, JPG, PNG, BMP, or TIFF image is one transaction, and each page of a PDF file is charged separately, so a 10-page PDF counts as 10 separate scans. Each scanned image can be used in any number of policies across data loss prevention, insider risk management, auto-labeling, and records management at no extra charge.

Caching reduces repeat charges:

Location Caching
Exchange Small images such as logos and signatures are scanned and billed only once per unique image across all users of the tenant, for a moving window of five days
Endpoints A cache local to each device, kept for 30 days, storing only the classifiers identified on the image and the image hash
SharePoint and OneDrive No caching for standalone images; in embedded file types, images aren't scanned again if only text is updated

Before reusing a cached result, the service checks parameters such as the image stream hash and image size, and it scans the image again if any parameter doesn't match.

Budget per image and per PDF page; the number of policies that consume the results doesn't add cost.

Telling look-alike answers apart

Several classifiers and settings can look interchangeable for one requirement, so the table below separates each such pair by the mechanism that tells them apart. Each row below restates a rule taught earlier in this topic, with the section that explains it.

Requirement in the scenario Fits Tempting wrong answer, and why it fails
Only current customers' account numbers EDM SIT[17] (Exact data match: detecting your own records) A built-in ID SIT alone: a valid format doesn't prove membership in your table
Filled-in copies of the HR onboarding form Fingerprint SIT[8] (Document fingerprinting) EDM against an employee list: it matches supplied values, not the form's wording
Independently written contracts Trainable classifier[20] (Trainable classifiers) Document fingerprinting: there is no shared template to fingerprint
A fixed-format internal document ID Custom SIT with a regular expression[9] (Anatomy of a custom SIT pattern) A trainable classifier: a deterministic format needs no learned model
Names of people never seen before Named entity SIT[10] (Classifier types and what each compares content against) A keyword dictionary of known customers: it matches only the listed terms
Ignore a known test card number Exclude specific matches[13] (Refining what a SIT pattern matches) A higher confidence level: the test value can still appear with strong evidence
Corroborating evidence sits outside the window Larger proximity value[3] (Anatomy of a custom SIT pattern) A higher confidence level: it never widens the window
Improve a published custom classifier Remove and recreate it[20] (Trainable classifiers) Retraining it: unsupported after publication
Who removed a label last week Activity explorer[24] (Data explorer and Content explorer) Data explorer: it shows a current snapshot, not history
An investigator must preview documents Content viewer role group[22] (Data explorer and Content explorer) The Microsoft Entra Compliance Administrator role alone: it opens the page but lists no items
Screenshots that contain card numbers Enable OCR[27] (Turning on OCR for existing classifiers) A second SIT for images: OCR feeds existing classifiers
Device OCR stops after heavy use Raise the device bandwidth limit[27] (OCR coverage by location) Adding SITs: the daily bandwidth limit is unchanged

When two answers both sound right, ask what each one compares content against, or which permission or setting it actually changes; the right answer is the one whose mechanism matches the requirement.

Pattern-based SIT vs fingerprint SIT vs EDM SIT vs trainable classifier

CriterionPattern-based SITFingerprint SITEDM SITTrainable classifier
Compares content againstA pattern: regular expression, keyword list, keyword dictionary, or function, plus corroborating evidenceThe word pattern of a blank form or template you uploadHashed values from your own sensitive information source tableWhat it learned from example items
Fits a requirement likeCredit card or national ID numbers, person names, addresses, credentials, or your own ID formatFilled-in copies of a patent template or an HR formOnly the IDs of your current customers, patients, or employeesContracts, strategic plans, or other categories with no fixed format
What you supplyNothing for a built-in SIT; patterns and evidence for a custom SITA template with 256 to 204,800 characters of extracted textAn EDM schema and a source table of up to 100 million rows, hashed by the EDM Upload AgentNothing for a pretrained classifier; positive and negative samples in SharePoint for a custom one
How you change itBuilt-in SITs can't be edited but most can be copied; named entity SITs can be neither edited nor copiedEdit the SIT and upload the template againRefresh the source table, as often as dailyRetrain while unpublished; once published, remove it and create a new one
Match and Not a Match feedbackSupported for built-in and custom SITs; not for named entity or credential SITsSupportedNot supportedSupported for pretrained and custom classifiers

Decision tree

Only values in yourown records count?YesEDM SITNoItems share onestandard form?YesFingerprintSITNoCan a patterndescribe it?NoTrainableclassifierYesDoes a ready-madeSIT fit?YesBuilt-in,named entity,or credentialSITNoCustom SITcopy or from scratchText in images: enable OCR

Sharp facts the exam loves — give these one last read before exam day.

Cheat sheet

Sharp facts the exam loves — scan these before test day.

A classification taxonomy must express business sensitivity in terms users understand

A sensitivity taxonomy translates business and regulatory requirements into recognizable categories and handling expectations. Define label names and descriptions with business stakeholders, include concrete examples in tooltips, and test the terminology with the people who will classify content. Configure the protection required for each category after establishing what belongs in it.

Trap Using department names as the entire sensitivity hierarchy when the same department handles both public and restricted information, because ownership alone does not identify handling requirements.

4 questions test this
Recognizing sensitive information and enforcing its handling are separate configuration decisions

Microsoft Purview classifiers identify sensitive content, while sensitivity labels and policies determine how that content is classified and protected. A sensitivity label can deliberately identify a category without adding encryption or markings, allowing an organization to measure usage before introducing protection. Choose the detection requirement and the required handling action separately rather than treating a match as an automatic grant or restriction of access.

Trap Using a custom sensitive information type as the access-control policy, because a detector supplies classification evidence rather than a set of authorized readers.

5 questions test this
Classifier changes require management permission and review of dependent policies

Before editing or deleting a classifier, use an account with classifier-management permission and inspect the policies that depend on it. The Information Protection Admin role, available through Information Protection Admins, permits classifier creation, editing, and deletion without requiring Global Administrator; Information Protection Analyst and Investigator provide view-only classifier access rather than classifier-management authority. On the Classifiers page, inspect the policy-usage count and dependencies so you can assess the effect on consuming policies before changing their detector. The Classified items count measures detected items, not how many policies depend on the classifier.

Trap Using the Classified items count as the safety check before deleting a detector, because content prevalence does not reveal which configured policies still reference it.

3 questions test this
A supported built-in sensitive information type can be copied when its detection needs customization

When a Microsoft-provided pattern already meets the requirement, use its built-in sensitive information type to avoid maintaining a separate detector. For organization-specific changes, copy a supported built-in type and edit the custom copy, or define a custom type from scratch. Built-in definitions themselves are not editable, and the portal documents exceptions that cannot be copied, including several bundled identification types. Named-entity SITs cannot be edited or copied. For an otherwise copyable SIT with 10 text processors or restricted text processors, use PowerShell because the portal cannot copy it directly.

Trap Using a named-entity SIT as an editable template, because its definition supports neither copying nor editing.

6 questions test this
Bundled and unbundled named entities offer different detection breadth

Choose a bundled named-entity SIT such as All physical addresses for broad recognition, or an unbundled entity for a particular country, region, or class of terms. Named-entity sensitive information types recognize person names, physical addresses, and medical terms and conditions. Verify E5 licensing before choosing advanced named-entity detection. For a bundled SIT in Endpoint DLP, also verify that Advanced classification scanning and protection is enabled.

Trap Using a keyword dictionary of known customers for an open-ended requirement to identify previously unseen personal names, because a vocabulary list only matches its supplied terms.

4 questions test this
All credentials detects the credential family while individual members narrow the requirement

Select the All credentials sensitive information type when the requirement is to detect the full set of credential-scanning types available in the portal. Each member can also be used as a standalone SIT when only a particular credential type belongs in the detection scope. Credential-scanning SITs require an E5 license, so verify that entitlement before choosing this built-in detector family.

Trap Using All credentials for a rule limited to one credential type, because the bundle includes the other credential detectors and broadens the matching population.

5 questions test this
A custom sensitive information pattern combines a primary match with corroborating evidence

A custom sensitive information type contains one or more patterns, each defining a primary element, confidence level, character proximity, and any supporting elements. Use a regular expression for a structured identifier, a keyword list or dictionary for known terms, or an available function for supported detection logic. Supporting elements provide contextual corroboration, and multiple patterns let the same information type recognize different evidence combinations.

Trap Using a trainable classifier to implement an exact timestamp-shaped document ID, because learning a document category is unnecessary for a deterministic format requirement.

5 questions test this
Sensitive information confidence thresholds trade false positives against false negatives

Sensitive information types use low, medium, and high confidence values of 65, 75, and 85. Selecting low includes low, medium, and high matches; medium includes medium and high; high selects only high-confidence matches. Higher-confidence patterns generally require stronger corroboration and reduce false positives at the cost of potentially missing sensitive content, so choose confidence and occurrence thresholds together for the intended protection.

Trap Increasing a rule's occurrence threshold to repair an overbroad primary pattern, because the threshold changes how many detections are needed rather than improving each detection's evidence.

4 questions test this
Character proximity determines which supporting evidence belongs to a primary match

Character proximity defines how close supporting evidence must be to the primary element of a sensitive information pattern. The complete supporting value must fit within the configured proximity window; a window that reaches only the beginning of the value is insufficient. Set a narrow window for tightly formatted records and a wider one when legitimate context is farther away, validating that unrelated records do not supply accidental corroboration.

Trap Selecting a higher pattern confidence level to connect a distant customer name with an identifier, because confidence selection does not extend the character window.

6 questions test this
Custom sensitive information regex patterns require one primary capturing group

Microsoft Purview custom sensitive information regex patterns use one primary capturing group, with noncapturing groups for any additional grouping. Put alternative formats inside that single capturing group rather than using separate top-level captures. Avoid positional anchors such as ^ and $, because the content-scanning process does not guarantee a stable document start or end for matching.

4 questions test this
An XML-based custom SIT needs distinct identifiers and a Unicode rule-package import

When creating a custom SIT from an exported definition, give the RulePack and Entity new GUIDs and make each localized Resource idRef match the corresponding Entity id. In Security & Compliance PowerShell, export definitions with Get-DlpSensitiveInformationTypeRulePackage and write the SerializedClassificationRuleCollection bytes to an XML file before extracting and modifying the required rule. Save the custom rule XML with Unicode encoding matching its utf-16 declaration, then import its bytes with New-DlpSensitiveInformationTypeRulePackage -FileData ([System.IO.File]::ReadAllBytes('C:\Rules\Custom.xml')). Verify the resulting SIT and its display name with Get-DlpSensitiveInformationType before adding it to a consuming policy.

Trap Reusing the exported Entity GUID while changing only its display name, because the GUID identifies the rule independently of the localized name.

5 questions test this
Keyword dictionaries provide reusable vocabulary matching at larger scale than keyword lists

When the sensitive content is defined by a maintained vocabulary, use a keyword dictionary as a primary or supporting element in a custom sensitive information type. In the portal, create or edit the type and its pattern before uploading a TXT or CSV dictionary, reusing an existing dictionary, or entering terms. Dictionaries support any language and share a combined tenant limit of 1 MB after compression. For Security & Compliance PowerShell imports, save one keyword per line in a Unicode-encoded text file, read it with [System.IO.File]::ReadAllBytes, and pass those bytes to New-DlpKeywordDictionary -FileData.

Trap Using a regular expression that enumerates every changing project code, because a managed vocabulary is easier to update and reuse than embedding the entire inventory in pattern logic.

4 questions test this
Regex validators test whether a well-shaped identifier is actually valid

A regex validator supplements a format match with checks such as a checksum, Luhn validation, or a date embedded in the identifier. Configure checksum weights, modulus, and check-digit position for the identifier's algorithm, or choose the date format and the starting offset of its date substring. Use these validators when format alone accepts invalid values; a date beginning at the first character has offset 0.

Trap Adding a nearby employee keyword to validate an embedded hire date, because contextual evidence does not verify that the identifier contains an acceptable date.

3 questions test this
Additional checks refine a sensitive information match without replacing its primary pattern

Sensitive information type additional checks can exclude specified test values, reject repeated identical characters, constrain starting or ending characters, and include or exclude immediate prefixes and suffixes. Choose the check that expresses the known false-positive rule instead of weakening all detection. Prefix and suffix checks concern adjacent text, while a supporting element uses a proximity relationship.

Trap Raising the confidence requirement to remove a known test identifier, because the same test value can still appear with convincing contextual evidence.

4 questions test this
Custom sensitive information types require workload-specific testing and tuning

Testing a sensitive information type evaluates the uploaded file rather than refreshing historical classification: test representative matching and nonmatching files and review the number of matches at each confidence level; an Exchange Online license must exist in the tenant for the portal's SIT test option to be available. Also validate the custom detector in each consuming workload, such as Exchange, SharePoint or OneDrive, and endpoints, because extraction and processing limits can change results, and tune against representative false positives and false negatives there. Endpoint DLP evaluates all tenant SITs, including custom types unused by any DLP policy, so reduce excessive classification traffic by removing unused detectors or redesigning overbroad definitions rather than just removing their policy references. After changing a custom definition, unchanged SharePoint and OneDrive files do not automatically acquire updated classification results; use the documented recrawl or on-demand classification process as appropriate, then evaluate policy matches separately.

Trap Restarting an auto-labeling simulation as the mechanism for refreshing old classifier results, because simulation evaluates available classification rather than rescanning all file content.

3 questions test this
Mixed Chinese or Japanese patterns need spaced and unspaced variants

Define both spaced and unspaced keyword or regex variants when Chinese or Japanese text is combined with English or single-byte characters. Purview normalizes double-byte text by inserting spaces between words and removing special characters, so a keyword such as 机密的document needs both 机密的document and 机密的 document. Keep double-byte and mixed-language vocabulary in a separate dictionary or keyword list from English-only terms. For these keyword lists, prefer string matching over word matching as Microsoft recommends, rather than relying on English-style word boundaries.

Trap Using only the visually identical unspaced mixed-language keyword with word matching, because token normalization can insert a boundary absent from that literal form.

4 questions test this
Document fingerprinting identifies documents derived from a standard textual form

When employees fill in a standard business form, create a fingerprint-based sensitive information type by uploading the blank form or template. Purview derives a word-pattern fingerprint and stores its hash rather than the original document, so the template cannot be reconstructed from the stored value. The template must provide between 256 and 204,800 extracted text characters, and text inside embedded documents is not used to build the fingerprint. Fingerprint definitions share a separate rule pack with a 300 KB maximum size, supporting approximately 100 fingerprints per tenant. To adopt expanded matching capabilities for an existing fingerprint, edit its SIT and upload the same template again rather than creating another fingerprint from an already-used template, which is unsupported.

Trap Using exact data match against a list of employees to recognize the organization's blank HR form, because reference-record matching identifies supplied values rather than the form's stable textual structure.

4 questions test this
Fingerprint matching can tolerate partial similarity or require identical text

Fingerprint-based sensitive information types can assign low, medium, and high confidence thresholds for partial matching using text-match percentages from 30% to 90%. For a strict unchanged-document requirement, set the high-confidence match to Exact, which accepts only identical text and rejects even a small deviation. In Security & Compliance PowerShell, create the fingerprint SIT with New-DlpSensitiveInformationType or modify an existing one with Set-DlpSensitiveInformationType -Identity, passing the blank template's bytes from [System.IO.File]::ReadAllBytes to -FileData. Configure -ThresholdConfig @{low=40;medium=60;high=80} with -IsExact $false when completed or altered forms must still be recognized, or use -IsExact $true for identical-text matching.

Trap Using an exact file hash inventory to recognize filled-in versions of a template, because inserting field values changes file identity even when the form's shared wording remains.

4 questions test this
Fingerprint protection depends on licensing, readable content, and workload support

Verify E5 licensing before creating or modifying fingerprint SITs or adopting the expanded fingerprint feature set; existing E3 fingerprints and their policies can continue working without granting those management rights. Fingerprint SITs can be used by DLP for Exchange, SharePoint, OneDrive, Teams, and devices, and by auto-labeling for Exchange, SharePoint, and OneDrive. Document fingerprinting excludes password-protected files, image-only files, and files larger than 4 MB. Device use additionally requires Advanced classification scanning and protection, so validate both content eligibility and the destination workload before relying on a fingerprint policy.

Trap Using only an Exchange mail flow rule to protect local USB transfers with a fingerprint, because mail transport does not evaluate endpoint copy activities.

3 questions test this
Exact data match narrows detection to sensitive values in an organization's reference table

When generic identifiers produce too many unrelated matches, use an EDM sensitive information type to compare detected content against an authoritative structured dataset. EDM checks one-way cryptographic hashes of the supplied values and can use associated fields to strengthen recognition. The source database can contain up to 100 million rows and can be refreshed daily, allowing detection to follow changes in employees, customers, or patients. EDM supports Purview DLP in Exchange Online, SharePoint, OneDrive, Teams chat, and devices, but its documented Defender for Cloud Apps locations are SharePoint and OneDrive. For EDM auto-labeling, supported client-side locations are Word, Excel, PowerPoint, and Exchange desktop clients, while service-side locations are Exchange Online, SharePoint, and OneDrive.

Trap Using a generic national-ID detector alone for a requirement limited to current customers, because format recognition does not establish membership in the customer dataset.

4 questions test this
An EDM schema defines the reference fields and their matching behavior

The EDM XML schema identifies the DataStore and maps its field names one-to-one to the source table's column headers. It also identifies searchable fields, multi-token corroborative fields, and configurable matching behavior such as ignoring case or delimiters. Keep the exported schema aligned with the data used for hashing and uploading so that the service interprets each column as intended.

Trap Changing the display name of a sensitive information type to repair a source-column mapping error, because field interpretation comes from the EDM schema rather than the display label.

4 questions test this
An EDM primary field needs a detector before exact lookup can validate the record

An EDM rule associates its primary field with an existing sensitive information type that discovers candidate values, then performs exact lookup and checks nearby corroboration. Prefer a strongly defined primary detector and a field with discriminating values, because very broad matches or many repeated primary values increase lookup work and can miss detections through processing timeouts. For supporting values such as full names or addresses, configure multi-token matching when an appropriate pattern detector is impractical.

Trap Selecting a common surname as the primary lookup field to maximize the number of candidates, because frequent repeated values make exact lookup less selective and more expensive.

3 questions test this
The classic EDM experience is required when multiple sensitive information types must share a schema

Use the classic EDM experience when multiple SITs must share a schema, more than 10 SITs are needed, or custom schema naming is required. A tenant supports at most 10 EDM schemas, and the new experience creates a schema and SIT together with a one-to-one relationship. The new experience can infer structure from 10 to 20 representative nonsensitive sample rows. Classic-created and PowerShell-uploaded schemas are not manageable in the new experience.

Trap Creating a separate new-experience schema for every departmental rule over the same customer table, because each creation consumes another of the tenant's schema slots.

4 questions test this
EDM can separate sensitive-data hashing from the internet-connected upload computer

Use the EDM Upload Agent to hash source data with a salt before uploading it. For stronger separation, create the hash on a secure computer and copy the resulting hash and salt files to the upload computer; both computers must run the same Upload Agent version. The single-computer method requires the clear-text source table on the internet-connected machine, while a schema-inference sample file is a separate upload that must contain nonsensitive representative values.

Trap Uploading a spreadsheet encrypted with a document password as the EDM reference dataset, because the ingestion workflow requires the agent's schema-aware salted hashes.

5 questions test this
EDM data upload uses a dedicated security group and agent authorization

Add the Microsoft 365 work or school account used for EDM uploads to the EDM_DataUploaders security group, then authorize the Upload Agent with that account. Schema authoring, data ingestion, and policy use are separate phases, so creating the SIT is not evidence that its reference values have been uploaded and indexed. Review upload and indexing status before validating the resulting detector or activating dependent protection.

Trap Publishing a sensitivity label to the upload operator as the authorization for EDM ingestion, because label availability does not assign data-uploader rights.

4 questions test this
EDM source data must be validated against the exact schema before ingestion

Validate an EDM source file with EdmUploadAgent.exe /ValidateData using its /DataFile and /Schema values before hashing and uploading. CSV is the default input format; tab-separated and pipe-separated data are also supported when the delimiter is specified. Embedded delimiters and inconsistent quoting can change the parsed column count, so correct the export format rather than modifying detection confidence to compensate for an ingestion error.

4 questions test this
Trainable classifiers recognize content categories that are difficult to express as deterministic patterns

Use a trainable classifier when the meaning and structure of examples identify a category such as contracts or strategic plans. A Microsoft-provided pretrained classifier is ready to use without your own training, while a custom classifier requires representative organizational examples. Custom classifier language support is limited to English, and classifier processing requires unencrypted items. Check the consuming solution before investing in custom training: Communication Compliance accepts Microsoft-provided trainable classifiers only, so a custom classifier cannot satisfy that consumer's requirement.

Trap Using document fingerprinting for a collection of independently written legal agreements with no shared template, because a common form's text is not available as a stable fingerprint.

4 questions test this
Custom classifier training requires separate positive and negative seed collections

Prepare 50 to 500 positive samples and 150 to 1,500 negative samples for a custom trainable classifier, placing the collections in separate dedicated SharePoint folders. Use text-based supported files, including suitable Office files and text-bearing PDFs, and allow at least an hour for a newly created seed location to be indexed. The training account needs access to both folders; OneDrive folders, Teams folder types, images, and image-only PDFs are unsuitable seed locations or content. Processing uses up to the 2,000 most recently created samples by file creation timestamp, so keep the seed folders controlled to prevent unintended newer files from displacing the examples you selected.

Trap Using only highly representative positive examples to accelerate training, because the model also needs content that establishes what falls outside the category.

4 questions test this
Trainable-classifier lifecycle changes require validation and policy-impact review

Review a custom classifier's predictions and verify true positives, true negatives, false positives, and false negatives before publishing it for policy use. By default, the creator is the person who can train it and review its predictions. Improve an unpublished classifier by updating samples and restarting training; a published custom classifier must be removed and recreated to improve its accuracy because retraining it is unsupported. To suppress automatic Content explorer display of trainable-classifier detections not used in labeling policies, request the display change through Microsoft Support rather than disabling classifier scanning. Disabling scanning is a separate support request that stops sensitivity-labeling and DLP policies using those classifiers from working, so review dependent protection before requesting it.

Trap Editing a sensitivity label's confidence threshold to retrain a published classifier, because policy evaluation settings do not rebuild the learned model.

4 questions test this
Data explorer and Content explorer provide an inventory of classified and labeled items

Use Data explorer or Content explorer to locate items associated with sensitive information types, sensitivity labels, retention labels, and supported trainable classifiers. Start with the classifier or label, choose the data source, and drill into the repository to inspect matching items. Filter Exchange or Teams by a full mailbox address, and SharePoint or OneDrive by site, folder, or file information; CSV export reflects the current focus of the pane rather than an automatically complete tenant report. These views describe the classified-content inventory; use activity or audit records when the requirement is to establish who changed a label and when.

Trap Using a label publishing policy's target-user list as the inventory of labeled files, because publication records eligibility to apply a label rather than which content carries it.

4 questions test this
Explorer navigation, item listing, and content viewing require different permissions

An administrative role that exposes an explorer page does not by itself grant access to its item list or content. Data Explorer List viewer and Data Explorer Content viewer grant independent permissions through Data Classification List Viewer and Data Classification Content Viewer; Content explorer uses the corresponding Content Explorer role groups. Grant both capabilities when a user must list locations and read content, and apply administrative-unit restrictions when access must be scoped; content-view permission is also needed to reveal potentially sensitive item names.

Trap Assigning Compliance Administrator alone to an investigator who needs document previews, because portal administration and direct access to classified content are separate permissions.

4 questions test this
Explorer investigations must account for indexing delays and encrypted-content restrictions

Explorer counts can take up to seven days to update, and SharePoint file results can take up to fourteen days, so an immediate count is not a real-time completeness check. Preview and download are disabled for encrypted documents.

Trap Granting broader explorer content permissions to solve an encrypted document's unavailable preview, because the preview restriction is a product boundary rather than missing list access.

5 questions test this
Explorer match feedback identifies classification errors to guide detector tuning

In Content explorer, review a supported SIT or trainable-classifier match for a SharePoint or OneDrive item and record Match for a true positive or Not a Match for a false positive. Use the classifier's matched-items view or the item's available contextual summary to inspect the detection, and use Withdraw feedback to correct a mistaken submission. Apply the findings to detector tuning rather than treating a feedback selection as an immediate change to its rules. EDM, named-entity, and credential-scanning SITs do not support Match and Not a Match feedback, so check the classifier's feedback capability before choosing this workflow.

Trap Raising the policy's match-count threshold before reviewing disputed items, because changing how many detections trigger a policy does not establish whether those detections are correct.

5 questions test this
On-demand classification requires a scope and policy-impact review before execution

Before starting on-demand classification for SharePoint or OneDrive, review active policies that could act on refreshed results: apply-label or label-removal policies can change files even while the policy being investigated remains in simulation. Under Information Protection > Classifiers > On-demand classification, create a scan and select locations, classifiers, file extensions, and the file last-modified date range to include the intended historical files. Review View estimation and estimated charges, adjust scope if necessary, and only then select Start classification. Running the scan requires Compliance Administrator membership; inspecting classification results separately requires Content Explorer List Viewer or Content Explorer Content Viewer permissions for the intended view.

Trap Using one auto-labeling policy's simulation state as tenant-wide change isolation during an on-demand scan, because other active policies can act on the refreshed classification.

5 questions test this
OCR adds image text to existing Purview detection after billing and locations are configured

Enable OCR at tenant level after connecting an Azure pay-as-you-go subscription through Microsoft Syntex billing; deploying Microsoft Syntex itself is not required. In Microsoft Purview Settings, select Optical character recognition and configure the locations and included or excluded groups. Existing sensitive information types, EDM, trainable classifiers, and fingerprint SITs can then recognize extracted image content, with each workload limited to its supported solutions. Only images uploaded after OCR is enabled are scanned, so activation does not provide retrospective coverage of existing images.

Trap Creating a second credit-card SIT specifically for screenshots before enabling image scanning, because OCR supplies extracted text to existing detectors.

4 questions test this
OCR eligibility depends on workload-specific size, format, and image dimensions

OCR permits images up to 20 MB in Exchange and Teams and up to 50 MB in SharePoint, OneDrive, and Windows or macOS endpoints. Supported image dimensions range from 50 by 50 to 16,000 by 16,000 pixels, and extraction is limited to the first 2 million characters. Exchange supports selected embedded-image formats with at most 20 embedded images scanned per file, while Teams and endpoints support JPEG, JPG, PNG, BMP, TIFF, and image-only PDFs; check the location's format support before choosing the policy design.

4 questions test this
Exchange OCR scope controls scanning but does not provide image policy tips

Exchange OCR scans incoming, internal, and outgoing email by default, but Exchange DLP policy tips are not supported for images. Select Specific sender groups to limit the sender population, or the Exchange advanced option for only externally sent mail to exclude both incoming and internal traffic.

Trap Changing a sensitivity label's publishing audience to reduce Exchange OCR scanning, because label publication does not set the OCR sender or direction scope.

4 questions test this
Endpoint OCR depends on cloud connectivity, bandwidth, and file-path coverage

Windows and macOS endpoints send images to the cloud for OCR, with a default advanced-classification bandwidth limit of 1,024 MB per device per day. OCR stops scanning images after the device reaches that limit; increase it when the required coverage justifies the additional traffic. Endpoint DLP file-path exclusions also exclude those folders from OCR, and network controls must permit the required blob.core.windows.net endpoints.

Trap Adding another sensitive information type to a policy when image classification stops after heavy device use, because detector selection does not increase the device's cloud-scanning bandwidth.

4 questions test this
OCR charges per scanned image rather than per policy that uses its results

Each scanned standalone image is an OCR transaction, and each PDF page is charged separately. The same extracted result can be used by multiple supported Purview policies without an additional scan charge for each policy. Exchange caches eligible repeated small images across the tenant for a moving five-day window, and endpoints maintain a local thirty-day cache; standalone SharePoint and OneDrive images do not use that caching mechanism.

4 questions test this

References

  1. https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/sc-401
  2. https://learn.microsoft.com/en-us/purview/information-protection
  3. https://learn.microsoft.com/en-us/purview/sit-sensitive-information-type-learn-about
  4. https://learn.microsoft.com/en-us/purview/data-classification-overview
  5. https://learn.microsoft.com/en-us/purview/get-started-with-sensitivity-labels
  6. https://learn.microsoft.com/en-us/purview/sensitivity-labels
  7. https://learn.microsoft.com/en-us/purview/data-classification-content-explorer
  8. https://learn.microsoft.com/en-us/purview/sit-document-fingerprinting
  9. https://learn.microsoft.com/en-us/purview/sit-create-a-custom-sensitive-information-type
  10. https://learn.microsoft.com/en-us/purview/sit-named-entities-learn
  11. https://learn.microsoft.com/en-us/purview/sit-customize-a-built-in-sensitive-information-type
  12. https://learn.microsoft.com/en-us/purview/sit-create-a-keyword-dictionary
  13. https://learn.microsoft.com/en-us/purview/sit-regex-validators-additional-checks
  14. https://learn.microsoft.com/en-us/purview/auto-label-classification-simulation
  15. https://learn.microsoft.com/en-us/defender-office-365/scc-permissions
  16. https://learn.microsoft.com/en-us/purview/purview-permissions
  17. https://learn.microsoft.com/en-us/purview/sit-learn-about-exact-data-match-based-sits
  18. https://learn.microsoft.com/en-us/purview/sit-get-started-exact-data-match-based-sits-overview
  19. https://learn.microsoft.com/en-us/purview/sit-get-started-exact-data-match-hash-upload
  20. https://learn.microsoft.com/en-us/purview/trainable-classifiers-learn-about
  21. https://learn.microsoft.com/en-us/purview/trainable-classifiers-get-started-with
  22. https://learn.microsoft.com/en-us/purview/data-classification-data-explorer
  23. https://learn.microsoft.com/en-us/purview/sensitivity-labels-sharepoint-onedrive-files
  24. https://learn.microsoft.com/en-us/purview/data-classification-activity-explorer
  25. https://learn.microsoft.com/en-us/purview/data-classification-increase-accuracy
  26. https://learn.microsoft.com/en-us/purview/on-demand-classification
  27. https://learn.microsoft.com/en-us/purview/ocr-learn-about