Image and video generation on Microsoft Foundry

A marketing team sends three requests in one morning: a product shot on a transparent background, that same shot with the logo swapped out, and an eight-second clip for a social feed. On Microsoft Foundry those are three different operations against two different model families, and choosing the wrong one is the most common way an image or video feature stalls in review. The sibling page on multimodal understanding runs in the opposite direction, taking pixels you already have and producing captions, answers, and extracted fields; the page on responsible AI for multimodal content owns classifying and filtering visual material. This page produces the pixels and the frames, and owns the controls that shape them.

Both model families are reached the way every other model in a Foundry project is reached, by naming a deployment against a project or resource endpoint, so the new material here is the operations and the parameters sitting behind that call rather than the plumbing around it. This page ends where a build decision ends: with one named operation and the parameter that makes it satisfy the requirement.

The image models you can actually deploy

Start from what is deployable, because that answer changed recently. The dall-e-3 model was retired on 4 March 2026, is no longer available for new deployments, and existing deployments are non-functional^[1]. A new Foundry image workload therefore targets the GPT-image family, which the model catalog lists as gpt-image-1, gpt-image-1-mini, gpt-image-1.5, and gpt-image-2^[2]. Within that family the split is about cost and fidelity rather than about which API you call: gpt-image-1-mini is the cheap, fast option and is the one member that does not support input_fidelity, the control that decides how closely an edit reproduces the source's look and faces, covered in the editing section below; gpt-image-2 accepts arbitrary output resolutions in multiples of 16 pixels up to a 3,840-pixel long edge instead of the three fixed sizes the gpt-image-1 series offers.

GPT-image is one image-generation family among several sold on Foundry, not the only one. The catalog also carries FLUX models from Black Forest Labs^[2], some of which answer on the same image generations and image edits paths, and the MAI-Image family from Microsoft AI, among others. This page works in the GPT-image family throughout, because that is the family the image operations, the Sora 2 companion documentation, and the Agent Service image generation tool are all documented against.

Video is a separate family with a separate surface

Video generation models are listed separately in the same catalog as sora and sora-2^[2]. Sora 2 is the current one, and it uses the Azure OpenAI v1 API with the same structure as the OpenAI Sora 2 schema^[3], reached through client.videos. The older job-based surface documented on the same page, /openai/v1/video/generations/jobs with width, height, and n_seconds, is a different set of parameter names for a different model generation; when a question names Sora 2, the parameters are size and seconds.

Starting material picks the operation

With the models settled, one question routes every request on this page: what do you already have? Four operation names carry the whole objective, and this page uses these forms throughout: image generations and image edits for stills (the REST paths /images/generations and /images/edits, called from Python as client.images.generate and client.images.edit), and videos.create and videos.remix for clips. A prompt on its own goes to image generations, or to videos.create when the output is a clip. A picture you already own goes to image edits. A single still that should open a clip rides in as the input_reference on a fresh videos.create. A clip Sora 2 has already produced goes to videos.remix. Figure 1 lays those four answers side by side; each branch names the operation and the one thing it preserves from the input.

Image generations and image edits are separate operations with genuinely different payload shapes, which is worth internalising before any code is written. Image generations posts a JSON body. Image edits posts a multipart/form-data request that carries the source image file and, optionally, a mask file, because the image and mask travel as files rather than as JSON string fields^[1]. Attempting to pass an existing image into the generations operation as a JSON field has no supported form.

The takeaway to carry into the rest of this page: the medium (image or video) decides the model family, and the starting material decides the operation. Everything after this is the mechanics of one of those four boxes.

The shortest useful description of an image generation call is that it is synchronous and it hands back bytes. There is no job to poll and no link to fetch later: the response body carries the picture inline as base64 in data[0].b64_json, and GPT-image-series models always return base64-encoded images, so the response_format parameter is not supported^[1]. An application that reads a url field from data[0] is written against the older DALL-E response shape and raises a key error against a GPT-image deployment. The figure below, Figure 2, traces the whole path, including the optional streaming detour.

A generation call, end to end

This listing is the concrete form of the leftmost branch of Figure 1: a prompt goes in, bytes come out, and the application does the saving. model names a deployment created in your Foundry project, not a catalog model identifier, which is why repointing a deployment at a newer model version changes behaviour without touching this code.

import base64
from openai import OpenAI
from azure.identity import DefaultAzureCredential, get_bearer_token_provider

token_provider = get_bearer_token_provider(
    DefaultAzureCredential(), "https://ai.azure.com/.default"
)
client = OpenAI(
    base_url="https://<resource-name>.openai.azure.com/openai/v1/",
    api_key=token_provider,
)

result = client.images.generate(
    model="gpt-image-1",          # a DEPLOYMENT name, not a catalog model id
    prompt="A red ceramic mug, plain studio background, product photography",
    size="1024x1024",
    quality="high",
    output_format="png",          # png is what makes the next line work
    background="transparent",
    n=1,
)

# No url field exists here: decode the bytes and persist them yourself.
with open("mug.png", "wb") as f:
    f.write(base64.b64decode(result.data[0].b64_json))
# ... retry and error handling omitted

The controls that shape the output

Four parameters in that listing do the shaping, and two of them are coupled to a third. size on the gpt-image-1 series is one of 1024x1024, 1024x1536, or 1536x1024, with square images generating fastest. quality is low, medium, or high. n requests between 1 and 10 images in a single call.

The coupling is around output format, and it is where a plausible-looking configuration goes quietly wrong. background: transparent yields real transparency only when output_format is png; requested alongside a JPEG it produces an opaque background rather than an error, so the failure shows up as a white box behind a product cut-out rather than as an exception. output_compression, an integer from 0 to 100, is the mirror image of that rule: it applies to lossy output and is ignored for PNG. On the image operations the available formats are png and jpeg, so compression there means JPEG; the Agent Service tool covered later also offers webp, and compression covers both lossy formats on that surface.

Streaming trades render passes for perceived wait

Setting stream to true together with partial_images (1 to 3) makes the model emit progressively refined previews before the final render, which is worth doing in an interactive UI and worth skipping in a batch pipeline. Be precise about what it buys: partial images shorten the time before the user sees something, not the total render time and not the cost of the render, which is why the documentation frames streaming as a way to improve perceived latency^[4]. The genuine time-and-cost lever is quality, since image generation typically takes 10 to 30 seconds depending on the model, size, and quality settings^[1].

So: one synchronous call, base64 bytes back, and a small cluster of parameters where output_format is the one that decides whether the others take effect.

One line governs every edit: only a mask limits what the model is allowed to repaint. Sending just an image and a prompt to the image edits operation is a supported prompt-driven modification, but with no mask the model may re-render the entire frame, so naming one object in the prompt does not freeze the rest of the composition. That is fine for a mood change across a whole picture and unacceptable for a legally approved product shot where only the label may move. The figure below, Figure 3, puts the two paths side by side.

What a mask actually is

A mask is a PNG file, the same type as the main image input, and it carries the edit region in its alpha channel rather than in its colours. The documentation is exact: the mask defines the area you want the model to edit using fully transparent pixels (alpha of zero) in those areas, must be a PNG file, and must have the same dimensions as the input image^[1]. Two consequences follow. Painting the edit region in solid white or solid black does nothing useful, because those are opaque pixels and opaque means frozen; a mask drawn that way inverts the intended edit area. And a mask whose dimensions have drifted from the source no longer satisfies the same-dimensions requirement, so mask generation belongs next to whatever produced the source image, at the same pixel dimensions.

An edit request, and why it is a form and not JSON

Unlike generation, editing is a multipart/form-data request, because the source image and the mask travel as files. The image file is a PNG or JPG under 50 MB^[1]; mask is the PNG described above; prompt describes the change you want inside the transparent region.

curl -X POST "https://<resource-name>.openai.azure.com/openai/deployments/<deployment>/images/edits?api-version=2025-04-01-preview" \
  -H "Authorization: Bearer $TOKEN" \
  -F "image=@shot.png" \
  -F "mask=@mask.png" \
  -F "prompt=Replace the logo on the mug with a plain white circle" \
  -F "model=gpt-image-1" \
  -F "input_fidelity=high" \
  -F "size=1024x1024"
# ... other optional form fields omitted

The mask form field is what turns this from a whole-frame rewrite into a bounded edit, and input_fidelity is the separate dial discussed next. Drop the mask line and the request still succeeds; it just stops being a guarantee.

input_fidelity is about likeness, not about region

These two controls are easy to conflate and do different jobs. The mask answers where the model may paint. The input_fidelity parameter answers how hard the model works to match the input, controlling how much effort goes into preserving the style and features, especially facial features, of the input images. The model-capability table records one gap that matters for planning: gpt-image-1-mini does not support input fidelity control^[1], so the cheapest member of the family is the wrong choice for a headshot-retouch workflow where likeness has to survive. Choosing mini to cut the cost of a face-preserving pipeline removes the control the pipeline depends on.

Put together: mask for boundaries, input_fidelity for likeness, and neither one is implied by a carefully worded prompt.

Video does not behave like image generation in the one way that matters most to an application's design: the create call does not return a video. It returns a job. The object it hands back is called a Video even though no video exists yet, and this page calls that object the job from here on. Code that expects playable bytes back from videos.create gets that Video with a status of queued and progress at 0, which is the single most common early mistake against this API. Figure 4 traces the states a job passes through and the two things that happen after it completes.

Five endpoints, one lifecycle

The Sora 2 API provides five endpoints: create video, get video status, download video, list videos, and delete video^[3]. The first three make up the working lifecycle. Create starts a render job from a prompt, with optional reference input or a remix id. Get status retrieves the job's current state and progress percentage; the expected states are queued, in_progress, completed, and failed. Download fetches the finished MP4, and only once the job has completed.

Create, poll, download

This listing is the video counterpart of the image call in the previous section, and the shape difference is the point: three steps rather than one, with a wait in the middle. client.videos.retrieve re-reads the job by its id, and download_content is the separate call that returns the bytes.

import time
# ... client built exactly as in the image listing above, from the same v1 base URL

video = client.videos.create(
    model="sora-2",       # the Sora 2 deployment name in your project
    prompt="A slow dolly shot across a rain-soaked city street at night",
    seconds="8",          # 4, 8 or 12 only; the default is 4
    size="1280x720",      # 720x1280 (default) or 1280x720
)

while video.status not in ("completed", "failed"):
    time.sleep(20)
    video = client.videos.retrieve(video.id)   # progress climbs toward 100

if video.status == "completed":
    content = client.videos.download_content(video.id, variant="video")
    content.write_to_file("clip.mp4")          # store it now, see retention below
# ... failure handling omitted

The shape parameters are an allow-list

The two format parameters in that listing accept a fixed set of values rather than a range. seconds takes 4, 8, or 12 and defaults to 4^[3], and size is portrait 720x1280, which is the default, or landscape 1280x720. That size is the video parameter and shares only its name with the image size from the previous section: the two accept completely different values, and a 1024x1024 here is not a smaller video, it is a rejected request. A width and height combination the model does not support comes back as a 400 Bad Request with a dimension error^[3], not as a clip quietly snapped to the nearest legal shape, so passing a bespoke aspect ratio or a 30-second duration fails the request outright.

Audio is part of the render rather than a later step. Sora 2 supports audio generation in its output videos^[3], and dialogue and effects are produced natively during generation. No operation lays a music bed or a narration track over an already-finished clip, so an audio change means generating again.

Restrictions the API enforces before your filter sees anything

Separately from the content filter configured on the deployment, the Sora 2 API enforces several content restrictions^[3]: only content suitable for audiences under 18, rejection of copyrighted characters and copyrighted music, refusal to generate real people including public figures, and rejection of input images containing human faces. These are properties of the model surface, not severity thresholds you tune, which is why a failing promotional-video job whose reference photo shows a person is neither a quota problem nor a filter-threshold problem. Configurable severity thresholds, custom categories, and provenance metadata for generated media are the subject of the responsible AI page for multimodal content.

Jobs expire, so the pipeline downloads

Generated videos are not durable storage. Jobs are available for up to 24 hours after they are created, after which a new job must be created to generate the video again^[3], and a completed Video object carries an expires_at timestamp that reflects that window. A production pipeline therefore downloads the MP4 as soon as the status reaches completed and stores it in its own blob storage; linking end users straight at a service-side video id gives them a link that breaks within a day.

Three things to carry out of this section: video generation is a job you poll rather than a call that returns bytes, the shape parameters are a short allow-list rather than a range, and the finished MP4 is yours to store because the job is not.

Two Sora 2 features look like ways to edit a video and only one of them is. Remix takes something Sora 2 already produced and adjusts it. input_reference takes a still picture and starts a completely new render from it. Reaching for input_reference to tweak an approved clip produces a fresh clip that resembles nothing about the original except the first frame, which is the trap Figure 5 is drawn to prevent.

Remix keeps the take you already approved

Remix is the operation for the sentence "the client loves it, but make the jacket blue". By referencing the id of a previously completed generation and supplying an updated prompt, the system maintains the original video's framework, scene transitions, and visual layout while implementing the requested changes^[3]. The returned Video object records where it came from in remixed_from_video_id, which is what lets a review pipeline trace a delivered clip back to the approved take.

The call is the same client as the create-and-poll listing in the previous section, with one method swapped and the source clip named by id instead of described in a prompt.

video = client.videos.remix(
    video_id="<previous_video_id>",   # a COMPLETED generation, not a still image
    prompt="Shift the color palette to teal, sand, and rust, with a warm backlight.",
)
# The response carries remixed_from_video_id pointing at the source clip.

The alternative, resubmitting the original prompt with an edit appended, starts a fresh render, so camera move, lighting, and staging drift on every run. Remix exists precisely to stop that drift.

One adjustment per remix, one video out

Two limits shape how remix is used in practice. The first is guidance rather than a hard error: limit modifications to one clearly articulated adjustment, because narrow, precise edits retain greater fidelity to the source material and minimise the likelihood of visual defects^[3]. Batching a reviewer's whole change list into one remix prompt to save a round trip degrades fidelity and tends to return a clip that no longer matches the approved scene; the cheaper path is a short chain of single-change remixes.

The second is structural. Remix accepts exactly one source video id and returns exactly one video. Sora 2 offers no stitching, trimming, or timeline-assembly operation at all, so assembling several generated clips into a continuous piece is an external editing step in whatever video editor the team already uses. There is no multi-clip concatenation call waiting to be discovered.

input_reference anchors a first frame, nothing more

The input_reference parameter transforms an existing image, and the resolution of the source image and the final video must match^[3], with the supported values being 720x1280 and 1280x720. It accepts a single still in JPEG, PNG, or WEBP form and uses it as the visual anchor for the opening frame of a new generation. Everything after that first frame is generated from the prompt.

That resolution rule is stricter than it first reads, because it is an equality rather than an aspect-ratio check: a 1920x1080 hero image is landscape and still does not match a 1280x720 request. Resize the still to the exact target size before the call.

The distinction to carry into an exam question is short. If the input is a video id, it is a remix and the framework survives. If the input is a still, it is a new render and only the first frame is anchored.

Everything so far assumed your code decides when to make a picture. Inside an agent, the model decides, and the plumbing changes shape: the image generation tool in Foundry Agent Service generates images from text prompts in conversations and multistep workflows, and the agent's orchestrator model, the chat model that reads the conversation and decides when a tool should run, routes the request. Two pieces of setup are easy to get wrong and produce the same symptom, so the figure below, Figure 6, shows the whole request path with both of them marked.

Two deployments and one header

The tool needs two model deployments in the same Foundry project: a compatible orchestrator model for the agent, for example gpt-4o, and an image generation model deployment, gpt-image-1^[4]. The documented compatible orchestrators are gpt-4o, gpt-4o-mini, gpt-4.1, gpt-4.1-mini, gpt-4.1-nano, o3, and the gpt-5 series. Deploying only the image model leaves nothing to orchestrate the call.

The second piece is a routing header. Every Responses call must carry x-ms-oai-image-generation-deployment naming the image deployment; the troubleshooting guidance for a failing tool is to verify that the header is present on the Responses request and matches your image generation deployment name^[4]. Naming the orchestrator model in that header routes the request to the wrong deployment.

The listing below puts both pieces in one place: the agent definition that attaches the tool, and the Responses call that carries the header.

# ... project and openai clients built from the Foundry project endpoint; imports omitted

agent = project.agents.create_version(
    agent_name="agent-image-generation",
    definition=PromptAgentDefinition(
        model="gpt-4.1-mini",                    # the orchestrator deployment
        instructions="Generate images based on user prompts.",
        tools=[ImageGenTool(model="gpt-image-1", quality="low", size="1024x1024")],
    ),
)

response = openai.responses.create(
    input="Generate an image of a sunset over a mountain lake.",
    extra_headers={"x-ms-oai-image-generation-deployment": "gpt-image-1"},
    extra_body={"agent_reference": {"name": agent.name, "type": "agent_reference"}},
)
# ... agent clean-up omitted

ImageGenTool is where the image parameters live in this path, and x-ms-oai-image-generation-deployment in extra_headers is the routing header named above. The model argument on PromptAgentDefinition is the orchestrator; the model argument inside ImageGenTool is the image deployment. Keeping those two names distinct in the project avoids the misconfiguration where the agent appears to use the wrong deployment.

Reading the result out of the response

The picture does not arrive in the assistant's prose. When the tool runs, the response output contains an item whose type is image_generation_call and whose result field contains base64-encoded image data^[4]; the assistant's text message is a sibling item in the same output array. That gives a precise diagnostic: if you see only text output and no image_generation_call item, the request was never routed to image generation, which points back at the missing deployment or the missing header rather than at a broken tool.

Extracting the picture is therefore a filter over response.output on that item type, not a read of the text reply:

# ... base64 imported as in the first listing on this page
image_data = [o.result for o in response.output if o.type == "image_generation_call"]
if image_data:
    with open("lake.png", "wb") as f:
        f.write(base64.b64decode(image_data[0]))

The optional parameters, including region-limited editing

The tool carries its own optional parameters, set when the tool is created: size (1024x1024, 1024x1536, 1536x1024, or auto), quality (low, medium, high, or auto), background (transparent, opaque, or auto), output_format (png, webp, or jpeg), output_compression for the two lossy formats, moderation (auto or low), partial_images for streaming, and input_image_mask, an optional mask for repainting one region of an existing picture (inpainting), supplied as an image_url (base64) or a file_id^[4]. That last one is the mask from the image edits section wearing an agent-tool name, and it matters architecturally: a bounded edit is available inside the conversation, so a user asking for a touch-up does not force the application out of the agent and onto the raw image edits operation.

When the tool is worth it

The documentation names exactly two advantages of the tool over the Azure OpenAI image API: streaming, to display partial image outputs during generation and improve perceived latency, and flexible inputs, to accept image file ids in addition to raw image bytes. Both are conversational benefits. A headless batch renderer gets nothing from either and pays the two-deployment setup, so that workload belongs on the image generations operation from the previous sections.

Questions on this objective rarely ask what a service is. They describe a workflow, hide one constraint in it, and ask which operation or parameter satisfies that constraint. The reliable technique is to read for the starting material first, exactly as Figure 1 sets it out, then read for the constraint.

Stem fingerprints and the answer they point at

What the stem says	What it is testing	The answer
"the untouched parts of the approved shot must survive"	mask semantics	image edits with a same-size PNG mask whose transparent pixels cover the edit region
"the code reads data[0].url and fails"	image response shape	GPT-image deployments return base64 in b64_json; there is no URL variant
"the cut-out has a white box behind it"	format coupling	background: transparent needs output_format: png
"the create call returns nothing playable"	video is asynchronous	poll the video id until completed, then download the MP4
"the client asks for a 30-second clip"	Sora 2 duration allow-list	seconds accepts 4, 8, or 12 only
"keep the camera move and staging, change the jacket"	remix versus regeneration	videos.remix with the source video id and one adjustment
"a hero still should open the clip"	input reference	input_reference at exactly the requested size; this is a new render
"the reference photo shows a person"	Sora 2 restrictions	the API rejects input images with human faces; no filter setting changes it
"the video link stopped working overnight"	job retention	jobs live about 24 hours; download and host the MP4 yourself
"the agent replies with text and no picture"	agent tool wiring	missing gpt-image-1 deployment or missing x-ms-oai-image-generation-deployment header

Why the usual distractors are wrong

DALL-E 3 appears in options because it is the image model people remember and because its automatic prompt rewriting was distinctive. It is retired and undeployable, so it cannot be the answer to a build question. gpt-image-1-mini appears as the cost-saving option in scenarios that turn on likeness; it does not support input_fidelity, which is precisely the control those scenarios need. A prompt-only edit appears wherever a mask is called for; it is a real feature, but it permits a whole-frame re-render, so it never satisfies a stem that promises the rest of the composition is untouched.

On the video side, three distractors recur. An "add soundtrack" or dub operation is offered for finished clips; audio is generated natively during the render and no such operation exists. A concatenation or timeline-assembly call is offered for multi-clip pieces; remix takes one source video and returns one video, and joining clips is external work. And input_reference is offered as the way to adjust an approved clip; it seeds a new generation from a still, so remix is the operation that preserves the approved take.

One more habit is worth building for this domain. When a stem mixes generation with safety wording, decide which page it belongs to before picking: shaping output is this page, and classifying or filtering output is the responsible AI page for multimodal content. A question about severity thresholds on a generated image is not answered by any parameter described here, with the one exception of the agent tool's own moderation setting, which sits alongside the deployment's filter rather than replacing it. The official AI-103 skills outline^[5] keeps those two responsibilities in separate objectives for the same reason.