Orchestrate processes
Choosing between Dataflow Gen2, a pipeline, and a notebook
Sales files land in a storage container every night, need deduplicating and conforming before anyone can query them, and a semantic model (the tabular data model that Power BI reports query) has to refresh once the load is clean. Three Fabric items can carry parts of that job, and choosing the wrong one is the most common way an orchestration design goes wrong before a single activity ever runs. Building that nightly job end to end takes four decisions, and making them is what this page equips you to do: which item does the work, what starts the run, how values reach it, and what happens on each step's success and failure paths.
A neighboring page answers a different question. Ingest and transform batch data weighs the transformation engines against each other: Dataflow Gen2, notebooks, KQL, and T-SQL. This page is about the items that run and coordinate the work, how a run gets started, and how the steps are sequenced once it does.
The three items
A data pipeline is a workflow made of activities, where an activity is one configured step on the pipeline canvas. Fabric sorts activities into three families: data movement, data transformation, and control flow. A pipeline can hold up to 120 activities by default, and that count includes activities nested inside containers such as ForEach (Activity overview[1]).
Dataflow Gen2 is the low-code transformation item, authored in Power Query[2]. It is the successor to the original Power BI dataflow, which Fabric now calls Dataflow Gen1, so a doc or a colleague saying just "dataflow" may mean either; this page always means Gen2. It reaches 150+ connectors, offers 300+ transformation functions, and writes to a fixed list of output destinations that currently covers Fabric Lakehouse tables and files, Fabric Warehouse, Fabric SQL database, Fabric KQL database, Azure SQL databases, Azure Data Explorer, Azure Data Lake Gen2, SharePoint files, Snowflake, and PostgreSQL (Differences between Dataflow Gen1 and Dataflow Gen2[3]). Since April 2026, every new Dataflow Gen2 item is created with CI/CD and Git integration support by default; existing items created without it keep working.
A Fabric notebook is the code-first item. It runs Python, Scala, SQL, or R on Apache Spark clusters managed by Fabric, which is why its transformation ceiling is set by the Spark and open-source libraries you can import rather than by a fixed function list.
Reading the choice off the work
The selection criteria come from the work, not from preference, and Microsoft publishes them as a property comparison (Fabric decision guide[4]).
When the transformation is standard reshaping that low-code users have to keep maintaining, Dataflow Gen2 is the fit. Dropping such a transformation into a notebook trades away that maintainability for Spark capability the job does not need, which is a real cost paid for no benefit.
When the work needs custom Python, Scala, SQL, or R, or needs distributed processing for very large files or a custom algorithm, use a notebook. This is also where transformations go once they exceed what Power Query can practically express.
When the requirement is to coordinate several of those pieces, with ordering, conditional paths, and retries, only a pipeline does that. Dataflow Gen2 transforms data and cannot orchestrate other items, so an answer that nominates it as the coordinator of several child pipelines is wrong on capability, not on style.
These are not exclusive choices, and in practice they are usually combined. A pipeline runs a Dataflow Gen2 through the Dataflow activity[5] and a notebook through the Notebook activity[6], so the common production shape is one pipeline conducting both. The rest of this page follows that shape: what starts a run, how values move through it, and how the steps are wired together.
Scheduling pipeline runs
Fabric pipeline schedules are bounded, never open ended. Every schedule carries a start date, an end date, a frequency, and a time zone (Run, schedule, or use events to trigger a pipeline[7]). There is no option for an open-ended schedule, so the documented way to keep a pipeline running long term is to set the end date far in the future, for example 01/01/2099 12:00 AM, and update or stop the schedule when the requirement changes. Leaving the end date blank is not an available configuration, and an answer that offers it is a distractor.
A single pipeline can carry up to 20 schedules, added one at a time, each with its own frequency and its own start and end times.
Fixed and interval-based schedules
There are two schedule types, and picking between them comes before configuring either one.
A fixed schedule runs at the times its frequency describes. It is the ordinary choice, and it is the only type you can enable, disable, or edit in place after saving.
An interval-based schedule, currently in public preview, configures fixed, non-overlapping intervals instead. Its distinguishing feature is that it surfaces two extra values in the pipeline expression builder under Trigger parameters, named Window start time and Window end time, so each run can bound its own query or file selection to the slice of time that run owns. Two limits travel with it. Time slice monitoring and backfill are not yet available, so missed windows are not replayed automatically. And an interval-based schedule cannot be enabled, disabled, or edited after it is saved: changing one means deleting it and creating a replacement.
Schedule types compared
| Property | Fixed schedule | Interval-based schedule (preview) |
|---|---|---|
| Window boundaries in expressions | Not provided | Window start time, Window end time |
| Edit, enable, or disable after saving | Yes | No, delete and re-create |
| Automatic backfill of missed windows | Not applicable | Not available |
Passing parameter values from a schedule
A schedule can supply values for the pipeline's parameters at each execution, in one of two forms. A Direct value is a static value typed into the schedule configuration. A Variable library reference pulls the value from a Variable library item instead, which keeps configuration central and simplifies promotion between environments.
The parameter names configured in the schedule must exactly match the names defined in the pipeline. Mismatched names are ignored at run time rather than raising an error, so a typo here produces a silently wrong run instead of a failed one.
Failure notifications and who may configure them
Email notifications for failed scheduled runs are set up by adding users or groups under Failure notifications in the schedule pane. Three boundaries matter more than the setup itself (Job Scheduler in Microsoft Fabric[8]).
Notifications are sent only for failures from scheduled runs, and are not sent for manually triggered runs. A manual test therefore produces no email regardless of who is listed.
Notifications are configured at the item level, and the settings apply across all schedules for that item rather than to the one schedule open in front of you. Recipients can be users or groups in the Microsoft Entra tenant, including B2B guest users; direct external email addresses are not supported.
To set up or modify a schedule, or to configure failure notifications, you need at least the Contributor role in the workspace or Write permission on the item. Users with only Viewer access can see existing schedules but cannot change them.
Two platform behaviors sit behind the scheduler and explain runs that stop without anyone touching them. A scheduler that hits consecutive failures is shut down into an auto-disabled state and must be restarted manually through the UI, API, or CI/CD process; the threshold varies by item and is typically 10 consecutive failed runs. Separately, schedules expire if a user does not sign in to Fabric for 90 consecutive days.
The through-line for this section: a schedule is a bounded, item-level object, not a property of a single run. Its time window, its parameter bindings, its notification recipients, and the permission needed to touch any of them all live with the item, which is why so many of its behaviors surprise people who expect per-run settings.
Event-based triggers and Activator rules
A requirement worded "start ingestion when the file arrives" is asking for an event trigger, and no schedule frequency satisfies it. A one-minute schedule polls rather than reacts, and it spends capacity on the hundreds of runs per day that find nothing. The distinction is what the exam tests here, so it is worth stating plainly: a schedule asks when, a trigger asks what happened.
Event triggers start a pipeline when file events, job events, or workspace events occur. They are built on two platform features rather than on the pipeline itself. Fabric eventstreams carry the events, and Activator alerts (written Data Activator in some docs; the same feature) evaluate the condition and take the action.
What the setup actually creates
Starting from the Trigger button on the pipeline's Home ribbon, the setup walks through the event source (OneLake events or Azure Blob Storage events, among the sources offered), the event types to listen for, and a Subject filter that narrows to a folder name, file name, file type, and container. Two objects come out of it. A new eventstream object is created in the workspace, and the trigger itself is saved as a Reflex item, which is the object type to look for in the workspace list when you later need to view, edit, or stop the trigger.
Activator rules start more than pipelines
The same Activator machinery reaches further than the pipeline you set it up from. An Activator rule can take an action that starts a job on a Fabric item, and the supported targets are Fabric pipelines, Dataflows, notebooks, Spark job definitions (a Spark application submitted as a batch job rather than run interactively), User Data Functions (custom business logic packaged as a callable function), Copy jobs (the wizard-driven data movement item), and Publish business event, which is in preview (Trigger Fabric items[9]). The action is configured in the Action section of the rule definition, and the target item is picked from the OneLake catalog, the workspace-wide item browser.
Passing parameter values to the activated item is in preview and carries one strict requirement: define the parameter name and type exactly as they are defined in the target Fabric item. Values are entered manually or selected as dynamic properties from the data source. Activator accepts string, Boolean, and number (float) parameters, and supports every parameter type that User Data Functions themselves support. One target does not participate at all, and it is a recurring exam detail: Copy jobs don't accept parameters.
So an event-driven design is really two decisions: which event should wake the work, and which item should do it. The eventstream and the Reflex item are plumbing that the setup creates for you; the Activator rule is the piece you actually author.
The figure below traces the storage-event chain from the event through to the pipeline run and the parameters that run receives.
Parameters, variables, and dynamic expressions
Two constructs hold values inside a pipeline and they are not interchangeable: a parameter's value is fixed for the whole run, while a variable exists precisely so that it can change during one.
Parameters pass external values into a pipeline. You define them on the Parameters tab of the pipeline editor canvas with a name, a data type, and a default value, and after a run starts, the value stays the same throughout that run (Parameters[10]). That immutability is what makes one pipeline reusable across many sources: the same activities, a different set of parameter values on each run.
Variables are the construct for state that has to move. A counter incremented inside a ForEach loop, a list built up with Append variable, a flag flipped by Set variable: each of those is a variable. Reaching for a parameter as a counter is a design error the run will not report as such, because the assignment simply has nowhere to land.
One naming collision is worth settling here, because four different things on this page are called parameters. Pipeline parameters are the ones defined on the pipeline itself, described above. Trigger parameters are supplied by the platform from a schedule window or a storage event. Base parameters are what a Notebook activity discovers from a notebook and passes into it. Dataflow parameters are the ones a Dataflow Gen2 exposes to its calling activity. They share a word and nothing else: different owner, different lifetime, different place to configure them.
Referencing values, and the type the reference returns
An expression can appear inside a string through string interpolation, where the expression is wrapped in @{ ... }. This is where a small syntax choice changes the data type an activity receives.
Expression forms and their return types, for myNumber = 42 and myString = foo
| Expression value | Result |
|---|---|
@pipeline().parameters.myString | Returns foo as a string |
@{pipeline().parameters.myString} | Returns foo as a string |
@pipeline().parameters.myNumber | Returns 42 as a number |
@{pipeline().parameters.myNumber} | Returns 42 as a string |
Answer is: @{pipeline().parameters.myNumber} | Returns the string Answer is: 42 |
The rule those rows encode is short. String interpolation always returns a string; a direct expression reference preserves the native type. So when an activity property requires a number, a Boolean, an array, or an object, reference the expression directly. Wrapping it in @{...} to be safe does the opposite of safe, because the property then receives text.
Two neighboring pieces of the same syntax are worth pinning down at first touch. To use a literal string that begins with @, escape it by typing @@, so "@@" yields a one-character string containing @. And inside a ForEach, @item() returns the current element of the collection being iterated: for the items [1, 2, 3] it returns 1 on the first iteration, 2 on the second, and 3 on the third (Expression language[11]).
System variables that describe the run itself
A set of pipeline-scope system variables can be referenced anywhere in the pipeline and answer questions about the current execution rather than about your data. @pipeline().RunId is the ID of this specific run, @pipeline().Pipeline its name, @pipeline().TriggerName and @pipeline().TriggerTime identify the trigger that invoked it and when it actually fired, and @pipeline()?.TriggeredByPipelineName names the calling pipeline when another pipeline started this run through an Invoke pipeline activity, evaluating to Null in every other case.
The ? in that last one is the same null-safe operator the trigger parameters need, for the same reason: the field is legitimately absent in some runs. Trigger-related date and time system variables, in both pipeline and trigger scopes, return Coordinated Universal Time in ISO 8601 format, for example 2017-06-01T22:20:00.4061448Z, so a comparison against a local-time value has to convert first.
Where a manual test run differs
The built-in trigger parameters from an event-based start deserve one more line here, because they interact with parameters and expressions directly. Data Factory parses the file name and folder path out of the storage event's Subject and Topic fields and exposes them in the expression builder, referenced as @pipeline()?.TriggerEvent?.FileName. During a manual test run there is no storage event, so those parameters are unset and return NULL. The null-safe ? syntax is required rather than stylistic: without it, an on-demand test fails on the expression rather than on the logic you meant to exercise.
Two habits carry this section: decide whether a value is fixed for the run (parameter) or changing during it (variable) before you write the expression, and check what type the reference returns before you hand it to an activity property.
Dependency conditions between activities
The arrow between two activities is not just sequence, it is a condition, and there are exactly four of them.
On success runs the downstream activity only when the upstream one succeeded. On failure runs it only when the upstream one failed. On completion runs it after the upstream activity either way. On skip runs it only when the upstream activity was skipped, which happens when that activity's own incoming condition was not met. Reading a pipeline means reading which of the four each arrow carries: two activities can sit adjacent on the canvas and never both run.
The figure below shows one activity's four outgoing paths.
Control flow activities
Branching and looping are activities in their own right, and picking among them is a question about the shape of the decision rather than about the data (Activity overview[1]).
If Condition branches on a condition that evaluates to true or false, running one set of activities for each result. Switch implements a switch expression, allowing multiple subsequent activities for each potential result of one expression, which is the right choice when there are more than two outcomes selected by a single value. Until implements a do-until loop, executing a set of activities repeatedly until the condition associated with it evaluates to true, and it accepts a timeout value. ForEach iterates over a collection and runs its child activities once per item. Filter applies a filter expression to an input array, which is how a collection is narrowed before a ForEach rather than by branching inside the loop.
A deactivated activity confirms that the condition, not the execution, chooses the path. A deactivated activity never actually runs, but you pick what to Mark activity as, and the branch follows that choice: marked Succeeded runs the On success or On completion branch, marked Failed runs On failure or On completion, and marked Skipped runs On skip. That also makes deactivation a clean way to comment out part of a pipeline during development without deleting it from the canvas.
ForEach parallelism, stated without ellipsis
ForEach concurrency is item-level, and the compressed version of this rule is where the mistakes come from. On the ForEach Settings tab, Sequential processes the items one at a time. Leave it unselected and Batch count caps how many items are processed at the same time. Either way, the child activities inside the ForEach are processed sequentially with respect to one another for each item (ForEach Activity[12]).
So clearing Sequential parallelizes items; it never parallelizes the steps within one item. Multiple items are processed in parallel, each of them running sequentially through the list of child activities. An answer that clears Sequential in order to make one item's child activities run simultaneously is wrong about which axis the setting controls.
Letting the data choose the path
Two activities exist to turn data into control flow instead of hard-coding it. Lookup reads or looks up a record, a table name, or a value from an external source, and its output can be referenced by succeeding activities. Get metadata retrieves metadata of data in a pipeline, such as whether an item exists, its structure, or its column count.
The pattern they compose into is the workhorse of parameterized orchestration: Lookup a configuration table, feed its rows into the ForEach Items property, reference the current row with @item() inside the child activities, and branch on a Get metadata result rather than encoding the list of sources into the pipeline. Adding a source then means adding a row, not editing a pipeline.
Per-activity failure and logging settings
This section covers the per-activity settings on an activity's General tab that decide how a step behaves when it fails and what it writes into the run logs. They are configured per activity, so each one is a local decision rather than a pipeline-wide policy.
Retries
Most pipeline activities support automated retries, so when one fails on a transient error such as a throttled API, a brief service outage, or a flapping dependency, retry settings handle it with no orchestration you have to build yourself. Select Enable retries, then set Retry to the number of attempts, a value between 1 and 1000 with a default of 1 (Activity retries[13]). Wrapping a Copy activity in a ForEach loop to emulate retries adds a construct the platform already provides, and it loses the back-off behavior described next.
Retry interval type chooses how the wait between attempts is calculated:
| Type | Behavior | Defaults |
|---|---|---|
| Fixed (default) | Waits the same number of seconds between every attempt | Retry interval 30 seconds |
| Increasing Delay | Exponential back-off: each retry waits a random interval drawn from a range that grows exponentially, up to a configurable maximum | Retry interval 30 seconds, Max retry interval 3600 seconds |
The range doubles with each attempt until its upper bound reaches Max retry interval, after which every remaining retry waits at that maximum. Drawing randomly inside each range is deliberate: it reduces the chance that concurrent pipeline retries collide on the same upstream system at the same moment.
By default an activity retries on any failure. Retry conditions, in preview, narrow that to specific errors so attempts are not spent on failures that will never resolve, such as an authentication error. A condition matches on one of three fields: Error message (the text content of the error), Failure type (the category, for example User error or System error), or Error code (the specific code returned, for example 429 for rate limiting). Conditions combine with And or Or. Conditional retries are supported for specific activity types, including Copy data, Notebook, Dataflow, and Stored procedure activities, so this is not a setting you will find on every activity.
One ordering detail catches people out: the retry interval runs before the condition is evaluated. With a one-hour retry interval and a condition that turns out not to match, the pipeline still waits the full hour before proceeding to the next activity or ending the run. Retry interval fields also do not accept dynamic expressions; their values must be static integers.
Timeouts
Timeout controls how long an activity may run before it is cut off. The default is 12 hours, the maximum is seven days, and the value uses the format D.HH:MM:SS.
One scope qualifier belongs with that number rather than after it. For the Dataflow activity, timeout and cancellation apply only to Dataflow Gen2 with CI/CD support. For runs of a Dataflow Gen2 without CI/CD support, timeouts are ignored and cancelling the pipeline does not stop the dataflow run.
Keeping values out of the logs
Two settings under Advanced properties control what run logging records. Secure input, when selected, keeps the activity's input from appearing in logs. Secure output does the same for its output.
Apply them to the activities whose parameters, credentials, or returned values would otherwise be readable by anyone with access to the run history. Because both are per-activity settings, securing one activity in a chain does not secure the activity that consumed its output: a secret passed from a secured Web activity into an unsecured Notebook activity is back in the logs at the second step.
Running notebooks from pipelines
A Notebook activity is how Spark code becomes one step of an orchestrated workflow. The pipeline supplies the scheduling, the dependency conditions, the retries, and the parameter values, and the notebook keeps doing the code-first processing it is good at (Notebook activity[6]).
Parameters come from the parameter cell, and only from there
Add the activity, choose an authentication method under Connection, then select the notebook from the Notebook dropdown. Auto-populate base parameters discovers the notebook's parameters and fills them into the activity during authoring, which keeps the pipeline aligned with the notebook's own logic instead of with a hand-maintained copy of it.
The discovery has one prerequisite that governs the whole behavior: the notebook must contain a parameter cell, and only variables defined in that cell are detected as parameters. Variables declared in ordinary cells are invisible to the activity no matter how they are written. When you select a notebook, the Base parameters section populates with names, types, and default values, which you can review and optionally override with different values for this pipeline. Selecting a different notebook refreshes the list, and a notebook with no parameter cell populates nothing at all.
Session tags need a workspace setting to do anything
Set a session tag on a Notebook activity to cut the startup time of a notebook job. The tag instructs Spark to reuse an existing session rather than creating a new one, any arbitrary string works as the value, and if no session exists with that tag, a new one is created using it.
The setting only takes effect if the workspace permits it. High concurrency mode for pipeline running multiple notebooks, found under the High concurrency mode Spark settings in Workspace settings, must be turned on. Identical session tags with that option left off produce no session reuse whatsoever, and nothing reports a problem: the notebooks simply each pay full Spark startup time. Workspace-level Spark configuration is covered on Configure Microsoft Fabric workspace settings.
One boundary belongs with the activity itself: the Notebook activity does not support calling notebooks across regions.
Notebook-driven orchestration with NotebookUtils
Not every chain of notebooks needs a pipeline wrapped around it. NotebookUtils is a built-in package for Fabric notebooks, available in PySpark, Scala, and SparkR notebooks and in Fabric pipelines, and designed for Spark 3.4 (Runtime v1.2) and above (NotebookUtils for Fabric[14]). It was renamed from MSSparkUtils; existing mssparkutils code remains backward compatible and does not break, but that namespace is slated for retirement, so new work uses notebookutils.
The module that matters for orchestration is notebookutils.notebook, which runs and chains notebooks together, including parallel execution with DAG (directed acyclic graph) support, cross-workspace references, and exit values that let one notebook return a result to whichever notebook called it. Running notebookutils.notebook.help() lists its methods from inside a notebook.
Choosing between the two orchestration surfaces follows the same logic as choosing the items themselves. A pipeline is the right home when the workflow spans different item types, needs conditional paths, or needs schedules and event triggers, none of which NotebookUtils provides. NotebookUtils is the right home when the whole workflow is notebooks and the person maintaining it works in code.
Invoke pipeline: parent and child pipelines
A pipeline that does everything on one canvas becomes unreadable and impossible to reuse. The Invoke pipeline activity is what lets a workflow be built instead as a small parent that sequences purpose-built children.
Two Invoke pipeline activities, and only one crosses workspaces
There are currently two Invoke pipeline activities, and the difference is not cosmetic (Invoke pipeline activity[15]). Invoke pipeline (Legacy) supports only Fabric pipelines in the same workspace as the parent pipeline, can only monitor the parent pipeline, and cannot invoke Azure Data Factory or Synapse pipelines. The current Invoke pipeline activity invokes pipelines across Fabric workspaces, invokes them from Azure Data Factory or Synapse, and monitors child pipelines.
The current activity requires a Connection object held in the Fabric credentials store, and its Authentication kind offers three options: organizational account, service principal, or workspace identity. You then choose the target workspace and the pipeline within it. Using a workspace identity adds two prerequisites: the tenant setting Service principals can call Fabric public APIs has to be enabled because it is off by default, and when the target pipeline lives in another workspace, the identity needs at least Contributor access to that workspace.
Wait on completion decides the parent's flow. Leave it on when a following parent activity consumes the child's output, because the parent would otherwise move on before the child produced anything. Turn it off when the child can run independently, in which case the invoked pipeline executes in parallel with the activities that follow it in the parent pipeline's execution flow.
The figure below shows the nesting this produces. A Lookup activity and a ForEach activity, both from the control flow section above, sit inside the parent pipeline; inside the loop, a Notebook activity reaches the notebook with its parameter cell and session tag from the previous section, and an Invoke pipeline activity reaches a child pipeline in any workspace through its connection. That nesting is the reusable shape: the parent decides what to iterate over, and each child stays a single-purpose item.
The Dataflow activity
Running a Dataflow Gen2 as a pipeline step is what turns a standalone low-code transformation into part of an orchestrated workflow, and the activity carries a few settings that exist nowhere else.
The Dataflow activity runs a Dataflow Gen2 from a pipeline: select an existing workspace and dataflow on the Settings tab (Dataflow activity[5]). Where the selected item is a Dataflow Gen2 with CI/CD support and public parameters mode enabled, a Dataflow parameters section appears listing the available parameters, their types, and their default values. Required parameters are marked with an asterisk, cannot be deleted from the grid, and must be passed for the dataflow to run; optional parameters can be selected and removed.
Two operational notes save time later. Running a Dataflow Gen2 (CI/CD) successfully requires Member or higher access to the workspace plus access to all connections the dataflow uses, and missing either produces run failures. And the error information shown in the Dataflow activity can be incomplete or partial, so troubleshooting starts in the Dataflow Gen2 (CI/CD) recent runs view where full logs are downloadable, not in the pipeline output pane. Diagnosing failures across item types is covered on Identify and resolve errors.
The pattern to remember: the activity starts the dataflow and reports a summary, while the dataflow item itself remains the place where the run is actually observable.
The Semantic model refresh activity
The last step of the nightly job from the opening section is the one that makes the loaded data visible to report readers, and it is an activity like any other.
The Semantic model refresh activity connects to a Power BI semantic model and refreshes it from a pipeline, which is what closes the loop on a nightly load (Semantic model refresh activity[16]). It works only with semantic models you create yourself, and the workspace needs Power BI Premium, Premium Per User, or Power BI Embedded capacity.
By default the activity runs a full refresh when triggered from the pipeline. You can narrow it by picking only the tables and the partitions to refresh instead of the whole model, and both selections accept pipeline expressions, so the scope itself can be parameterized per run.
The Advanced section holds the execution controls. Wait on completion is on by default, which makes the activity wait until the refresh finishes before the pipeline moves on. Max parallelism, Retry Count, and Semantic model refresh timeout tune how the refresh executes, and a commit mode choice decides whether the refresh commits all at once (Transactional) or in batches (Partial Batch).
With these two activities the orchestration surface is complete: a pipeline chooses and sequences the work, a schedule or an event starts it, parameters and expressions carry the values, dependency conditions and control flow shape the graph, and the Invoke pipeline, Notebook, Dataflow, and Semantic model refresh activities reach out to the other items.
Exam-pattern recognition
DP-700 tests orchestration mostly through short scenarios that hand you a requirement and four plausible-looking configurations. The stems repeat, and so do the distractors, so recognizing the shape is worth more than recalling any single setting.
"Processing must start when a file is uploaded." Any answer that sets a schedule is wrong regardless of how short the interval is. The requirement names an event, so the answer is an event-based trigger built on eventstreams and an Activator rule. The distractor is seductive because a one-minute schedule would eventually notice the file; it is wrong because it polls rather than reacts.
"A low-code team maintains this transformation" versus "the source is a 500 GB file." These two stems point in opposite directions from the same three-item menu. Maintainability by non-programmers selects Dataflow Gen2; volume and custom algorithms select a notebook. Picking the notebook for the first stem is technically capable and organizationally wrong, which is exactly the trade being tested.
"Several existing pipelines must run in order, with a failure path." Only a data pipeline coordinates other items. Dataflow Gen2 offered as the coordinator is the standard wrong answer, and it is wrong on capability rather than on preference.
"The value must change as the loop runs." Parameter versus variable. A parameter's value stays the same throughout the run; anything incremented, appended to, or flipped is a variable.
"The activity property needs an integer." Interpolation versus direct reference. @{...} returns a string, and the direct @ form preserves the native type. The distractor wraps the value in @{...} and describes it as the safer option.
"The next activity consumes the child pipeline's output." Wait on completion stays enabled. Disabling it is the trap, and the answer text usually justifies it as improving parallelism.
"The pipeline to call is in another workspace." The current Invoke pipeline activity with a Fabric connection. Invoke pipeline (Legacy) is workspace-local and is the near-miss answer.
"Notebooks in this pipeline should share a Spark session." Session tags and the workspace high concurrency setting. An answer that sets only the tags is incomplete, and incomplete is wrong.
"Notify the team when the test run fails." Schedule failure notifications never cover on-demand runs, and the recipient list is item-wide rather than per-schedule. Watch which of those two facts the question actually turns on.
"Only some tables in the model need refreshing." The Semantic model refresh activity's table and partition selection, not a full model refresh and not a separate pipeline.
"Retry only on throttling." A retry condition matching Error code 429, not an unconditional retry count, and not a ForEach wrapper around the activity.
"The pipeline must run indefinitely." A far-future end date, because there is no open-ended schedule. An answer offering a blank end date describes a configuration that does not exist.
One reading habit helps more than memorizing any single mapping. Identify which of three questions the stem is really asking: which item does the work, what starts the run, or how the steps are wired together. Distractors are usually correct answers to one of the other two.
Choosing the Fabric item for the work
| Decision factor | Dataflow Gen2 | Data pipeline | Notebook |
|---|---|---|---|
| Authoring surface | Power Query visual connectors and transformations | Activity canvas with control flow | Code in Python, Scala, SQL, or R |
| Primary job | Clean, combine, and reshape data | Sequence and coordinate other work | Distributed processing on Apache Spark |
| Strongest fit | Low-code transformation maintained by non-programmers | Recurring movement and multi-step orchestration | Very large files, custom algorithms, logic beyond Power Query |
| Coordinates other items | No | Yes, through Notebook, Dataflow, and Invoke pipeline activities | Yes, through NotebookUtils notebook chaining and DAGs |
| Runs inside a pipeline | Yes, as a Dataflow activity | Yes, as an Invoke pipeline activity | Yes, as a Notebook activity |
| Started by an Activator rule | Yes | Yes | Yes |
Decision tree
Sharp facts the exam loves — give these one last read before exam day.
Cheat sheet
Sharp facts the exam loves — scan these before test day.
- Dataflow Gen2 fits visual low-code ingestion and transformation
Choose Dataflow Gen2 when Power Query's visual connectors and transformations can clean, combine, and reshape data without custom code. It can load multiple supported destinations and can also run as an activity inside a larger pipeline.
Trap Choose a notebook solely because a standard Power Query transformation must be maintained by low-code users.
7 questions test this
- Wingtip Toys has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1. A cloud accounting service exposes a table of daily store transactions that is about 90 MB per month. Every mo
- Contoso, Ltd. has a Microsoft Fabric workspace named Workspace1 that contains a lakehouse named Lakehouse1. Three cloud databases hold customer records that use different column names, different data
- Woodgrove Bank has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1. A published Dataflow Gen2 applies a documented set of Power Query steps to regulatory reference data, and tw
- Tailwind Traders has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1. A data engineer standardizes supplier contact records with a series of Power Query steps that combine seve
- Northwind Traders has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1 and a connection to an Azure SQL database named SalesDB. A data engineer cleanses a customer table and a p
- Fabrikam, Inc. has a Microsoft Fabric workspace that contains a warehouse named DW1 and a published Dataflow Gen2 named DF1 that the finance team maintains in Power Query. A nightly workflow must run
- Litware, Inc. has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1. An on-premises Microsoft SQL Server database named Database1 holds monthly budget tables in a wide layout. An
- A data pipeline fits recurring movement and multi-step orchestration
Choose a Fabric data pipeline when the solution must coordinate copies, stored procedures, scripts, notebooks, dataflows, or conditional control flow as one workflow. Pipelines also provide connector and gateway-based movement for sources that are not directly suitable for Spark code.
Trap Use a Dataflow Gen2 item alone to coordinate several existing child pipelines with success and failure dependencies.
8 questions test this
- Trey Research has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1. A partner uploads an extract to a folder in OneLake at unpredictable times between 01:00 and 09:00. Three dep
- Relecloud has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1. A nightly load stages incoming orders in Lakehouse1. When the staged row count matches the count recorded by the
- Lucerne Publishing has a Microsoft Fabric workspace that contains a warehouse named DW1 and a Power BI semantic model that reads from DW1. Every night, sales files must be copied from an Azure Data La
- Alpine Ski House has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1 and a warehouse named DW1. A nightly workload loads revenue data into Lakehouse1 and then publishes it to D
- Wide World Importers has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1 and a warehouse named DW1. Every morning, order data must be copied from an Azure Blob Storage containe
- Adatum Corporation has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1. Every night, raw files must be landed in the bronze layer of Lakehouse1 from an Amazon S3 bucket, from a
- Proseware, Inc. has a Microsoft Fabric workspace that contains two existing pipelines. One pipeline lands raw files, and the other builds curated tables from those files. Both pipelines already work,
- Fabrikam, Inc. has a Microsoft Fabric workspace that contains a warehouse named DW1 and a published Dataflow Gen2 named DF1 that the finance team maintains in Power Query. A nightly workflow must run
- A notebook fits code-first distributed Spark processing
Choose a Fabric notebook when ingestion or transformation needs custom Python, Scala, SQL, or R logic and Apache Spark's distributed processing. This is a strong fit for very large files, custom algorithms, or transformations that exceed the practical expressiveness of Power Query.
Trap Choose Dataflow Gen2 for a 500-GB file solely to obtain Spark's highest-throughput parallel file processing.
7 questions test this
- Southridge Video has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1. Lakehouse1 holds one billion viewing events. A senior engineer has written a sessionization routine in Pyt
- Tailspin Toys has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1. A Dataflow Gen2 named DF_Clean has cleaned and combined order files for two years. The daily file volume has
- Lakeshore Retail has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1. Lakehouse1 stores three years of basket-level sales as Delta tables that total 12 TB. A cost-allocation ro
- Nod Publishers has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1. Lakehouse1 holds 2 TB of raw circulation data in Delta tables. A nightly job must first apply a parsing rout
- Blue Yonder Airlines has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1. Lakehouse1 holds several years of flight telemetry as Parquet files in OneLake. A rebuild must read th
- First Up Consultants has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1. A media partner delivers 300,000 audio recordings a day to the Files area of Lakehouse1. For each reco
- City Power & Light has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1. Lakehouse1 stores six years of meter readings as Delta tables that total 40 TB. Each night a job must jo
- A Notebook activity embeds Spark processing inside pipeline control flow
Use a Notebook activity when custom Spark code is required as one step of an orchestrated workflow. The pipeline supplies scheduling, dependencies, retries, and parameter passing, while the notebook remains responsible for its code-first data processing.
Trap Rewrite every pipeline activity into one notebook because pipelines cannot invoke Fabric notebooks.
8 questions test this
- Proseware, Inc. has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1, a warehouse named DW1, and a notebook named NB_Root. NB_Root uses NotebookUtils to run five child notebooks
- The Phone Company has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1 and a notebook named NB_Enrich. A team member runs NB_Enrich by hand each morning after the overnight file
- Woodgrove Bank has a Microsoft Fabric workspace that contains a data pipeline named PL_Recon and a notebook named NB_MatchPay. PL_Recon runs NB_MatchPay in a Notebook activity, and a Teams activity co
- Margie's Travel has a Microsoft Fabric workspace that contains a warehouse named DW1, a lakehouse named LH1, and a data pipeline named PL_Nightly. PL_Nightly copies booking files into DW1 and then run
- Best For You Organics has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1, a notebook named NB_Load, and a data pipeline named PL_Daily. NB_Load has had its own daily schedule
- Fincher Architects has a Microsoft Fabric workspace that contains a data pipeline named PL_Build. PL_Build copies survey files from an on-premises file share, runs two stored procedures in a warehouse
- Proseware, Inc. has a Microsoft Fabric workspace that contains a lakehouse named Lakehouse1, a warehouse named DW1, and a notebook named NB_Root. NB_Root uses NotebookUtils to run five child notebooks
- Contoso, Ltd. has a Microsoft Fabric workspace named Workspace1 that contains a lakehouse named Lakehouse1, a data pipeline named PL_OrderStage, and a notebook named NB_OrderShape. PL_OrderStage uses
- A fixed pipeline schedule requires start, end, frequency, and time zone
Fabric pipeline schedules are bounded rather than open ended, so configure both start and end dates along with frequency and time zone. A pipeline can have as many as 20 schedules, each with independent timing.
Trap Leave the end date blank to create a permanently open-ended Fabric pipeline schedule.
6 questions test this
- Northwind Traders has a Microsoft Fabric workspace named WS_Season that contains a data pipeline named PL_Holiday. PL_Holiday must load campaign sales into a lakehouse every two hours from 1 November
- Northwind Traders has a Microsoft Fabric workspace named WS_Rollout that contains a lakehouse and a data pipeline named PL_Onboard. PL_Onboard loads sales from newly opened stores into the lakehouse.
- Contoso, Ltd. has a Microsoft Fabric workspace named WS_Ops that contains a data pipeline named PL_Cutover. PL_Cutover loads a new order-management source into a lakehouse. The source system goes live
- Litware, Inc. has a Microsoft Fabric workspace named WS_Parallel that contains a lakehouse and a data pipeline named PL_Compare. PL_Compare loads a new billing source into the lakehouse while the lega
- You have a Microsoft Fabric workspace named Workspace1 that contains a data pipeline named PL_SalesLoad. PL_SalesLoad loads point-of-sale data into a lakehouse every night and has no planned decommiss
- Fabrikam, Inc. has a Microsoft Fabric workspace named WS_Supply that contains a data pipeline named PL_Replenish. A data engineer created a fixed schedule for PL_Replenish two weeks ago with a daily r
- Interval-based schedules expose nonoverlapping window boundaries to expressions
An interval-based schedule defines fixed nonoverlapping execution windows and supplies Window start time and Window end time as trigger parameters. Time-slice monitoring and backfill are not available for these interval schedules.
Trap Use an interval schedule because it automatically backfills every missed time window.
7 questions test this
- Northwind Traders has a Microsoft Fabric workspace named WS_Slice2 that contains a lakehouse, a data pipeline named PL_Slice and a Spark notebook named NB_Window. PL_Slice runs under an interval-based
- Fabrikam, Inc. has a Microsoft Fabric workspace named WS_Slice that contains a data pipeline named PL_Window. PL_Window runs on a fixed schedule every 30 minutes and has no other schedules. Its Copy d
- Adventure Works has a Microsoft Fabric workspace named WS_Recon2 that contains a lakehouse and a data pipeline named PL_Recon2. PL_Recon2 copies an operational table into the lakehouse once an hour. A
- Litware, Inc. has a Microsoft Fabric workspace that contains a data pipeline named PL_Hourly. PL_Hourly runs under an interval-based schedule with one-hour intervals. An auditor asks for evidence of w
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Events. PL_Events already has an interval-based schedule that runs it every 15 minutes. A Copy data activity in PL_Events u
- Litware, Inc. has a Microsoft Fabric workspace named WS_Late that contains a data pipeline named PL_Adjust. PL_Adjust loads sales adjustments from an operational database into a lakehouse and runs onc
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Sensor. You add an interval-based schedule to PL_Sensor with 20-minute intervals and save it. A data engineer then opens th
- Event triggers start pipelines in response to matching platform events
Use an event-based run when a pipeline must react to file, job, or workspace events rather than poll on a clock. Storage event setup uses Fabric eventstream and Activator capabilities and creates a Reflex item that holds the active trigger rule.
Trap Create a one-minute fixed schedule to satisfy a requirement that ingestion start specifically when a file arrives.
7 questions test this
- Litware, Inc. has a Microsoft Fabric workspace named Workspace1 that contains a lakehouse and a data pipeline named PL_Intake. Partner files land in the Files folder of the lakehouse at unpredictable
- You have a Microsoft Fabric workspace named Workspace1 that contains a data pipeline named PL_Ingest and a storage event trigger that starts PL_Ingest when a file lands in a monitored folder. The trig
- Contoso, Ltd. receives daily extracts from a partner into an existing Azure Blob Storage account that Contoso already owns. A Microsoft Fabric workspace contains a data pipeline named PL_Partner that
- Contoso, Ltd. has a Microsoft Fabric workspace named WS_Vault that contains a lakehouse and a data pipeline named PL_Purge. Users occasionally remove a source file from a monitored folder under Files
- Fabrikam, Inc. has a Microsoft Fabric workspace named WS_Assets that contains lakehouses, notebooks and data pipelines. A data pipeline named PL_Inventory refreshes an inventory table that lists every
- You have a Microsoft Fabric workspace that contains a lakehouse and a data pipeline named PL_Load. New files arrive in a OneLake folder at unpredictable times throughout the day. PL_Load must start as
- You have a Microsoft Fabric workspace that contains a Dataflow Gen2 query named DF_Bronze and a data pipeline named PL_Silver. DF_Bronze runs on its own schedule, and its duration varies from ten minu
- Storage trigger file and folder metadata can drive dynamic pipeline expressions
Data Factory derives built-in file-name and folder-path trigger parameters from a storage event and exposes them in the expression builder. Use null-safe pipeline references when manually testing because an on-demand run has no storage event metadata.
Trap Expect a manual pipeline run to populate the same event file-name parameter as a BlobCreated trigger.
6 questions test this
- Litware, Inc. has a Microsoft Fabric workspace named WS_Raw that contains a lakehouse and a data pipeline named PL_Batch2. A storage event trigger starts PL_Batch2 whenever a file is created in a moni
- Adventure Works has a Microsoft Fabric workspace named WS_Partner that contains a lakehouse and a data pipeline named PL_Route. A storage event trigger starts PL_Route when a file is created anywhere
- Litware, Inc. has a Microsoft Fabric workspace that contains a data pipeline named PL_Batch. A storage event trigger starts PL_Batch for each created blob, and a Copy data activity reads the trigger f
- Fabrikam, Inc. has a Microsoft Fabric workspace named WS_Route2 that contains a lakehouse and a data pipeline named PL_Sortie. A storage event trigger starts PL_Sortie for each file created in a monit
- Litware, Inc. has a Microsoft Fabric workspace named WS_Trace that contains a lakehouse and a data pipeline named PL_Trace. A storage event trigger starts PL_Trace for each file that a partner creates
- Litware, Inc. has a Microsoft Fabric workspace named WS_Enrich that contains a lakehouse, a data pipeline named PL_Enrich and a Spark notebook named NB_Enrich. A storage event trigger starts PL_Enrich
- Pipeline schedule failure notifications apply only to scheduled runs
Add users or groups to a pipeline schedule's Failure notifications when email is required after that scheduled run fails. Those schedule notifications are not sent for on-demand executions.
Trap Configure schedule failure notifications as the email mechanism for every manual test run.
4 questions test this
- Fabrikam, Inc. has a Microsoft Fabric workspace that contains a data pipeline named PL_Receipt. PL_Receipt is started by a storage event trigger for each partner file and by a fixed nightly schedule f
- Adventure Works has a Microsoft Fabric workspace named WS_Test that contains a data pipeline named PL_Verify. A data engineer added the support group under Failure notifications while configuring PL_V
- Contoso, Ltd. has a Microsoft Fabric workspace named WS_Fin that contains a data pipeline named PL_Post and a Dataflow Gen2 item named DF_Rates. PL_Post runs on a fixed nightly schedule and already li
- Fabrikam, Inc. has a Microsoft Fabric workspace named WS_Relay that contains a lakehouse and a data pipeline named PL_Relay. PL_Relay runs on a fixed nightly schedule, and the on-call group is listed
- Managing schedules and failure notifications requires write-level access
To create or modify an item schedule or configure its failure notifications, a principal needs at least workspace Contributor or Write permission on the item. Viewer-only access permits inspecting existing schedules but not changing them.
6 questions test this
- Fabrikam, Inc. has a Microsoft Fabric workspace named Workspace1 in which three data engineers hold the Member role so that they can maintain pipeline schedules and failure notifications. A governance
- Adventure Works has a Microsoft Fabric workspace named Workspace1 that contains several scheduled data pipelines. Each quarter an internal auditor must confirm the recurrence, the start and end dates,
- Litware, Inc. has a Microsoft Fabric workspace named Workspace1 whose access is granted through Microsoft Entra ID security groups. A compliance rule states that reporting analysts must not change any
- Contoso, Ltd. has five Microsoft Fabric workspaces that each contain scheduled data pipelines. A duty manager must review, across all five workspaces, which scheduled items have failure notifications
- Litware, Inc. runs a release process that uses a Microsoft Entra ID service principal to create and update data pipeline schedules in a Fabric workspace named Ops1 through the Fabric Job Scheduler API
- Adventure Works has a Microsoft Fabric workspace that contains a data pipeline named PL_Assay. A support engineer who holds the Viewer role in that workspace opens the pipeline's Schedule pane during
- Failure notification recipients apply across every schedule for an item
Configure scheduled-run failure notification recipients at the Fabric item level. That recipient configuration applies to all schedules on the item rather than to only one selected schedule.
- Activator rules start supported Fabric jobs and pass matching typed parameters
An Activator rule can start a pipeline, Dataflow, notebook, Spark job definition, user data function, or copy job when its condition is met. For supported parameterized items, define the Activator parameter name and type exactly as in the target and pass manual or dynamic values; copy jobs do not accept parameters.
- Pipeline parameters provide run inputs that remain fixed during execution
Define typed parameters to reuse one pipeline with different external values on separate runs. A parameter value is immutable for the duration of a run, while variables are the appropriate construct for state that must change as activities execute.
Trap Use a pipeline parameter as a counter that a ForEach activity increments during the run.
5 questions test this
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Ingest. PL_Ingest is started by a parent pipeline that supplies a parameter named SourceFolder. Partway through a run, a Ge
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Poll. PL_Poll calls a REST endpoint that reports whether an upstream export has finished, and the pipeline must keep callin
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Archive. PL_Archive uses a ForEach activity with Sequential selected to walk a list of files, and each iteration copies one
- Contoso, Ltd. has a Microsoft Fabric workspace named Workspace1 that contains twelve source-specific ingestion folders in a lakehouse. A data pipeline named PL_Child copies one folder into the bronze
- You have a Microsoft Fabric workspace named Workspace1 that contains a lakehouse named Lakehouse1 and a data pipeline named PL_Load. PL_Load begins with a Lookup activity that reads a control table an
- String interpolation converts a pipeline expression result to text
An expression referenced directly can preserve its native type, but an expression wrapped in @{...} interpolation always returns a string. Use direct expression form when an activity property requires a number, Boolean, array, or object rather than text.
Trap Wrap an integer parameter in string interpolation to guarantee that the activity receives an integer.
5 questions test this
- You have a Microsoft Fabric workspace that contains a notebook named NB_Load and a data pipeline named PL_Run. The parameter cell of NB_Load declares a variable named BatchSize that the notebook uses
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Gate. PL_Gate defines an integer parameter named RowThreshold, and an If Condition activity compares the row count returned
- You have a Microsoft Fabric workspace that contains a notebook named NB_Load and a data pipeline named PL_Run. The parameter cell of NB_Load declares a variable named BatchSize that the notebook uses
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Notify. When PL_Notify finishes, a Teams activity must post a message that reads Load finished for run, followed by the ide
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Fan. PL_Fan defines a parameter named FileList whose type is array and whose value is supplied at run time. A ForEach activ
- Notebook activity parameter discovery reads the notebook parameter cell
A Notebook activity can auto-populate names, types, and defaults only for variables defined in the selected notebook's parameter cell. Pipeline authors can override those base values, and selecting a different notebook refreshes the discovered list.
Trap Expect the activity to auto-populate every variable declared in arbitrary notebook cells.
3 questions test this
- You have a Microsoft Fabric workspace that contains a notebook named NB_Merge and a data pipeline named PL_Merge. The parameter cell of NB_Merge defines a variable named target_table. In the Notebook
- Litware, Inc. has a Microsoft Fabric workspace that contains a notebook named NB_Score whose parameter cell defines a variable named lookback_days with a default value suited to development work. A da
- You have a Microsoft Fabric workspace that contains a notebook named NB_Silver and a data pipeline named PL_Silver. NB_Silver declares the variables source_table, target_table, and load_mode in three
- Notebook session tags reuse Spark only when pipeline high concurrency is enabled
Assign the same session tag to compatible Notebook activities to reuse an existing Spark session and reduce startup time. The workspace setting for high concurrency on pipelines running multiple notebooks must be enabled; otherwise the tag does not provide session reuse.
Trap Set identical session tags while leaving high concurrency disabled and expect all notebook activities to share Spark.
4 questions test this
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Batch with five Notebook activities. A developer set the same session tag in the advanced settings of all five Notebook act
- You are a data engineer who holds the Contributor role in a Microsoft Fabric workspace. You build a data pipeline that runs four Notebook activities, and you want the four notebooks to reuse a single
- Contoso, Ltd. has a Microsoft Fabric workspace that contains a data pipeline named PL_Medallion with eight Notebook activities. All eight notebooks are run by the same user, use the same default lakeh
- You have a Microsoft Fabric workspace in which high concurrency for pipelines running multiple notebooks is enabled. A data pipeline named PL_Load runs six Notebook activities. Four of them load bronz
- Invoke pipeline (Legacy) can wait for a child or let parent execution continue
Configure the Invoke pipeline (Legacy) activity to wait on completion when downstream parent activities depend on the child pipeline's completed output. Clear wait on completion when the child can execute independently: the invoked pipeline then runs in parallel with the activities that follow in the parent flow. Current documentation binds this wait/continue choice to the LEGACY activity's settings only; the modern Invoke pipeline activity's documented settings (Type, Connection, Workspace, Pipeline) carry no wait toggle.
Trap Clearing wait on completion on the Legacy activity when the next parent activity must consume the child pipeline's completed output.
- ForEach controls parallelism across items while preserving each item's child order
ForEach evaluates an item collection and exposes the current element through @item(). Select Sequential to process items one at a time; otherwise Batch count limits concurrent items, while the configured child activities still run sequentially for each individual item.
Trap Clear Sequential to make all child activities for one item execute simultaneously.
3 questions test this
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Sync. PL_Sync must process 30 store extracts. For each store, a Copy data activity must land the extract in a lakehouse and
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Retail. PL_Retail uses a ForEach activity that has the Sequential option cleared and Batch count set to three. The Items li
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Regions. PL_Regions uses a ForEach activity whose Items list contains twelve region codes. In the ForEach settings, a devel
- Activity retries handle transient failures before control moves to a failure path
Configure retry count and interval on the failing activity when intermittent connectivity or throttling should be retried without custom orchestration. For supported activity types, optional retry conditions can restrict retries by error code, message, or failure type so permanent authentication errors do not consume unnecessary attempts.
Trap Add a ForEach loop around a Copy activity solely to retry a transient connection failure.
3 questions test this
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Sensor. PL_Sensor calls a metered API through a Copy data activity. The API returns error code 429 when the account exceeds
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Stage. PL_Stage uses a Copy data activity that reads from a partner REST endpoint. Roughly once a week the activity fails w
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Guard. A Notebook activity in PL_Guard has retries enabled with a retry interval of 3600 seconds and a retry condition that
- NotebookUtils orchestrates reusable notebook chains and DAGs
Use the NotebookUtils notebook module to run and chain notebooks and receive exit values from them. It also supports parallel DAG execution and cross-workspace notebook references when notebook-driven orchestration is required.
3 questions test this
- You have a Microsoft Fabric workspace that contains twelve PySpark notebooks that build silver tables. Nine of the notebooks are independent of each other, and three must wait for specific upstream no
- You have a Microsoft Fabric workspace that contains notebooks named NB_Extract and NB_Refine. NB_Extract determines the batch identifier that must be processed, and NB_Refine needs that identifier bef
- You have two Microsoft Fabric workspaces named WS_Core and WS_Finance. A PySpark notebook in WS_Core must call a notebook that the finance team owns and maintains in WS_Finance, and it must receive th
- Pipeline dependency conditions select the required upstream outcome
Use On success or On failure when a downstream activity must run only for that respective upstream result, On completion when it must run after the upstream activity regardless of outcome, and On skip when it must run only if the upstream activity is skipped.
5 questions test this
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Harvest. PL_Harvest copies files into a staging folder in a lakehouse and then runs a Stored procedure activity that loads
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Route. A Script activity in PL_Route is deactivated during a source system freeze, and its Mark activity as setting is set
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Split. In PL_Split, a Stored procedure activity that publishes gold tables is connected to an upstream Notebook activity wi
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Harvest. PL_Harvest copies files into a staging folder in a lakehouse and then runs a Stored procedure activity that loads
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Audit. PL_Audit ends with a Copy data activity that writes a reconciliation extract to a lakehouse. When that copy fails, a
- Choose pipeline control flow by branch shape and loop condition
Use If Condition for true and false branches, Switch for multiple branches selected by one expression, and Until to repeat nested activities until its condition becomes true.
4 questions test this
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Verify. A Lookup activity in PL_Verify returns a row count, and an expression compares that count with an expected value an
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Pager. A Web activity in PL_Pager reads one page of records from a vendor API, and the response reports whether more pages
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Dispatch. PL_Dispatch receives a string parameter named MarketCode whose value is one of five market codes. Each market cod
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Triage. PL_Triage uses a Switch activity whose expression returns a document type that is read from a control table. Cases
- Lookup and Get Metadata outputs drive data-dependent pipeline control flow
Use Lookup to return configuration rows or values and Get Metadata to return properties such as existence, structure, or column count. Reference their outputs as dynamic content in succeeding activities to drive branches and loops instead of hard-coding orchestration decisions.
- Secure input and output prevent activity values from appearing in logs
Enable Secure input to suppress an activity's input from pipeline logs and Secure output to suppress its output. Apply the corresponding setting wherever sensitive parameters, credentials, or returned values would otherwise be exposed in run logging.
4 questions test this
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Token. A Web activity in PL_Token calls an identity endpoint and receives a bearer token in the response, and later activit
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Payroll. PL_Payroll contains six activities, and four of them pass or return values that a governance policy classifies as
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Token. A Web activity in PL_Token calls an identity endpoint and receives a bearer token in the response, and later activit
- You have a Microsoft Fabric workspace that contains a data pipeline named PL_Vault. A Web activity in PL_Vault retrieves a database password and already has Secure output selected. A Notebook activity
- The current Invoke pipeline activity uses a connection for cross-workspace calls
Use the current Invoke pipeline activity, not Invoke pipeline (Legacy), to call a Fabric pipeline in another workspace. Its Fabric connection can authenticate with an organizational account, service principal, or workspace identity; the legacy activity is limited to the parent pipeline's workspace.
3 questions test this
- Northwind Traders has Microsoft Fabric workspaces named WS_Pulse and WS_Quarry. A data pipeline named PL_Relay in WS_Pulse must run a child pipeline named PL_Sift that the ingestion team owns and main
- Trey Research has a Microsoft Fabric workspace named WS_Studio that contains a parent pipeline named PL_Anchor and a child pipeline named PL_Feeder. PL_Anchor runs PL_Feeder with an Invoke pipeline (L
- Proseware, Inc. runs two Microsoft Fabric workspaces named WS_Trellis and WS_Fathom. A data pipeline named PL_Cadence in WS_Trellis uses an Invoke pipeline activity to run a child pipeline named PL_Wi
- Pipeline schedules bind exact parameter names to static or library values
A schedule can pass either a direct static value or a Variable Library reference to each pipeline parameter at run time. The schedule's parameter name must exactly match the pipeline definition; a mismatched name is ignored.
3 questions test this
- Wide World Importers has a Microsoft Fabric workspace that contains a data pipeline named PL_Territory. PL_Territory defines a string parameter named TerritoryCode and loads the files of one territory
- Northwind Traders has a Microsoft Fabric workspace that contains a data pipeline named PL_Consign. PL_Consign builds its source folder path from a partner batch code that a developer types into the Co
- Alpine Ski House has Microsoft Fabric workspaces named WS_Build and WS_Live. A data pipeline named PL_Source defines a parameter named SourceRoot, and a schedule supplies the value of that parameter a
- Semantic model refresh activity controls both refresh scope and execution behavior
A Semantic model refresh activity can target selected tables or partitions when the pipeline should process only part of a model. Configure its retry count, maximum parallelism, and refresh timeout to control execution behavior for that scoped refresh.
References
- Activity overview
- What is Power Query?
- Differences between Dataflow Gen1 and Dataflow Gen2
- Fabric decision guide: copy activity, dataflow, Eventstream, or Spark
- Dataflow activity
- Notebook activity
- Run, schedule, or use events to trigger a pipeline
- Job Scheduler in Microsoft Fabric
- Trigger Fabric items
- Parameters
- Expression language
- ForEach Activity
- Activity Retries in Microsoft Fabric Data Factory
- NotebookUtils (former MSSparkUtils) for Fabric
- Invoke pipeline activity
- Semantic model refresh activity in Data Factory for Microsoft Fabric