AI Video API Integration: Async Production Engineering

TL;DR

A production AI video API integration is workflow orchestration, not a long HTTP request. Persist the job before dispatch, separate the logical job from billable provider attempts, model ambiguous submissions explicitly, combine signed webhooks with polling, and copy temporary outputs into storage you control. Reliability comes from state invariants and reconciliation, not from choosing a supposedly "best" model.

Table of Contents

• Key Takeaways
• Start With the User Contract
• Treat Provider Capabilities as Versioned Evidence
• Define Job Attempt Event and Asset Records
• Use a Monotonic State Machine
• Submit Without Creating Duplicate Generations
• Combine Signed Webhooks With Polling
• Retry by Failure Class
• Take Ownership of Generated Assets
• Control Quotas and Cost Per Usable Output
• Measure the Control Plane
• Run the Go Reference Implementation
• Build a Release Gate
• FAQ
• Summary

Key Takeaways

• Return a local job ID immediately. Video generation belongs behind a 202 Accepted contract, a queue, and a status resource.
• Persist before calling a provider. A database transaction should create the job, reserve budget, create attempt 1, and enqueue an outbox event.
• Model unknown submission outcomes. A client timeout can occur after the provider accepted a paid job; blind retrying can create two videos.
• Treat callbacks as hints, not sole truth. Verify the raw body, deduplicate event IDs, update state monotonically, and poll stale work for reconciliation.
• Do not expose temporary provider URLs as your asset layer. Copy, validate, checksum, and retain the result under your own lifecycle policy.
• Measure cost per ready output. API-call cost hides retries, policy failures, abandoned work, and asset-ingestion failures.

This article focuses on the control plane behind an AI video generation API. For model-quality experiments, use the separate AI video evaluation protocol. The two decisions are related but not interchangeable: a model can win a blind review and still be operationally unsuitable for a specific account, region, or reliability target.

Start With the User Contract

The public API should promise a durable workflow, not mirror any one provider's schema. A useful contract accepts a logical request through an API gateway, returns a locally generated job ID, and lets the caller retrieve a stable status resource.

A create response should contain:

The create endpoint should return after the local transaction commits. It should not wait for generation, and it should not return a provider URL. This boundary gives the application control over authentication, cancellation semantics, tenant quotas, retention, and user-visible errors.

Define three terms before implementing the workflow:

• A job is the user's logical request. It survives retries and provider changes.
• An attempt is one potentially billable provider submission.
• An asset is a validated copy of an output stored under your control.

Keeping these identities separate prevents a common accounting error: counting retries as new user jobs or treating provider success as successful delivery.

Treat Provider Capabilities as Versioned Evidence

Provider adapters should translate current service contracts, while routing policy should read a versioned capability registry. Product names, model aliases, limits, regions, prices, and preview status change too quickly to belong in application branches.

Current first-party documentation illustrates four different surfaces:

This table is not a ranking. It proves that the stable abstraction is an asynchronous attempt with a remote identifier, not a hard-coded "draft provider" and "final provider." Route only after checking the exact account, endpoint, model, region, input mode, duration, resolution, moderation behavior, concurrency, and price effective at submission time.

A capability registry should therefore record:

Store the selected snapshot on every attempt. Otherwise, an investigation months later cannot distinguish a model change from an application regression.

Define Job Attempt Event and Asset Records

The minimum durable model needs five records because each record answers a different operational question.

Add database constraints for the invariants, not just indexes for query speed:

1. (tenant_id, client_idempotency_key) is unique for the logical job.
2. (job_id, attempt_number) is unique.
3. (provider, provider_idempotency_key) is unique when the provider accepts such a key.
4. (provider, remote_id) is unique after the provider identity becomes known.
5. (provider, event_id) is unique in the event inbox.
6. A job cannot become ready without a committed asset row.

Sensitive inputs need a separate policy. Store a prompt hash for deduplication and auditing, but do not place full prompts, source media URLs, signed output URLs, or credentials in metrics and ordinary logs. If reproducing output requires retaining a prompt or source asset, encrypt it, scope access by tenant, and attach a deletion schedule.

Use a Monotonic State Machine

A monotonic state machine prevents delayed callbacks and concurrent workers from moving a job backward. Provider-specific statuses should be normalized at the adapter boundary and applied through one compare-and-set transition function.

The key distinction is succeeded versus ready. succeeded means the provider reports an output. ready means your system has validated and stored that output. A user-facing success counter should use ready.

Apply these transition rules:

• Terminal states do not regress when a delayed running event arrives.
• The database update includes the expected version or previous-state set.
• Duplicate events return success without repeating effects.
• Unknown provider states are stored for investigation but do not invent a local transition.
• Cancellation is a request until the provider confirms it; local cancellation does not prove that compute or billing stopped.

Submit Without Creating Duplicate Generations

The safest submission flow commits local intent before network I/O and treats a timeout as an epistemic problem. You do not know whether the remote side accepted the request.

Use this sequence:

1. Validate policy, media, tenant concurrency, and a conservative cost ceiling.
2. In one database transaction, create the job and attempt, reserve budget, and add a dispatch_requested outbox row.
3. Let a worker claim that outbox row with a lease.
4. Submit with the attempt's stable provider idempotency key or external task ID.
5. Persist the remote ID and pricing snapshot if the response succeeds.
6. If the connection fails after request transmission, move to submit_unknown.
7. Reconcile by the same provider key before creating another attempt.

The outbox closes the crash window between database commit and queue publication. It does not make an external provider call exactly once. The achievable design is at-least-once delivery with idempotent local effects and provider-assisted deduplication.

The submit_unknown branch deserves its own operations policy:

Changing the key during a retry creates a new attempt. Reusing one key across different prompts corrupts identity. Generate it from immutable attempt identity, not from a worker delivery count.

Combine Signed Webhooks With Polling

Webhooks minimize completion latency, while polling restores correctness when delivery fails. Both must call the same transition code.

For providers following the Standard Webhooks specification, verify the signature over webhook-id.webhook-timestamp.raw_body. Read the raw bytes before JSON parsing, reject timestamps outside a short tolerance, compare signatures in constant time, support overlapping keys during rotation, and enforce a unique event ID.

The receiver should do only bounded work before returning 2xx:

1. Enforce request size and content type.
2. Verify timestamp and signature against the raw body.
3. Insert the event into a durable inbox under a unique constraint.
4. Enqueue processing transactionally.
5. Return success.

Do not download a video, call another provider, or run quality evaluation in the webhook request. Google's webhook guidance notes that non-2xx deliveries can be retried for up to 24 hours. A slow handler amplifies duplicate traffic during an incident.

Polling should be scheduled data, not a fixed loop in a web process. Store next_reconcile_at and increase the interval while the remote task remains active. Poll immediately after a webhook gap, after worker recovery, and before declaring a task stuck. Add jitter so a deployment or provider recovery does not synchronize the fleet.

Retry by Failure Class

A retry policy must answer whether the previous operation was accepted, whether a retry can change the result, and whether another attempt is still within budget.

RFC 9110 defines Retry-After as either delay seconds or an HTTP date. Parse both forms, cap them against the job deadline, and then randomize the worker's wake-up time. A retry budget should limit attempts, cumulative delay, and estimated spend.

Provider fallback is a new semantic attempt, not a transport retry. Another model can change composition, moderation, duration, or audio. Require product policy to say whether that change is acceptable, expose it in provenance, and reserve a new budget amount.

Take Ownership of Generated Assets

A provider success response is not durable delivery because output URLs often expire. OpenAI documents temporary video download links lasting up to one hour, while Kling documents result-file cleanup after 30 days. Copy the output promptly and independently of user traffic.

The ingestion worker should:

1. Accept only a URL emitted by the trusted adapter for the expected remote task.
2. Require HTTPS and route downloads through a restricted egress client.
3. Allowlist expected hosts or validate every redirect and resolved destination to prevent SSRF.
4. Stream with byte and time limits instead of buffering the entire file.
5. Check the declared and detected media type, dimensions, duration, and policy limits.
6. Compute a cryptographic checksum while streaming.
7. Write to a temporary object key, verify the completed upload, then promote it.
8. Commit the asset record and ready transition atomically.

The asset record should preserve provider, model identity, attempt ID, prompt compiler version, source-media hashes, policy decision, checksum, and any provenance credential that survives your media pipeline. Generative AI output still needs ordinary media security, rights review, and retention controls.

Do not rely on a signed provider URL as a permanent database field. It can leak through logs and support tools, expire before the user opens it, or grant broader access than intended. Return an authenticated application URL backed by your own object store.

Control Quotas and Cost Per Usable Output

Cost control starts before dispatch and ends after the asset becomes ready. Static "cheap model first" advice is not enough because a draft may be a poor predictor of the final renderer, and extra attempts can cost more than direct generation.

Use a two-phase ledger:

1. Reserve a conservative ceiling using the capability, duration, resolution, and current price snapshot.
2. Release or settle the reservation when the provider returns billable usage or the attempt reaches a documented terminal condition.
3. Reconcile delayed usage records without changing historical price snapshots.

Enforce at least four independent limits:

Measure economics at the outcome boundary:

cost per ready video = sum(actual cost of all attempts) / count(ready jobs)
cost per ready second = sum(actual cost of all attempts) / sum(duration of ready assets)
effective yield = ready jobs / billable attempts

Segment these metrics by provider, model, input mode, duration, resolution, tenant tier, and prompt compiler version. Do not publish a savings claim until a controlled workload compares the complete attempt ledger, review acceptance, and delivered assets against a baseline.

Measure the Control Plane

An operational dashboard should reveal where a job stopped, not merely whether the provider endpoint returned errors.

Track these measures:

• queue age from accepted to dispatch claim
• submission latency and submit_unknown rate
• provider acceptance, running, terminal failure, and policy rejection rates
• completion latency by capability and provider, including percentiles
• webhook signature failures, duplicates, age, and processing lag
• polling reconciliation discoveries and stale-job count
• asset download, validation, and durable-ingestion failure rates
• reserved versus settled spend and cost per ready second
• user cancellation, abandonment, and manual-review volume

Trace one logical job across API, outbox, worker, provider attempt, webhook, poller, ingestion, and notification. Include job ID, attempt ID, provider, model snapshot, remote ID, state version, and event ID. Exclude prompts, credentials, signed URLs, and source media.

Define service-level objectives around outcomes you control. For example, measure the share of eligible accepted jobs that reach ready before a capability-specific deadline. Keep provider generation time separate from queue delay and ingestion time so incidents have an owner.

Run the Go Reference Implementation

The following standard-library package implements the two critical control-plane paths: recovery from an ambiguous submission and idempotent, monotonic provider events. It also verifies Standard Webhooks against the exact raw body and a bounded timestamp.

The in-memory maps make the example easy to compile, not production durable. Replace them with transactions, unique constraints, an outbox, and an event inbox in your database.

A production provider adapter must classify every network error that occurs after request bytes may have left the process as ErrOutcomeUnknown. Only a provable pre-send failure or an explicit provider rejection may follow the ordinary failure branch.

Create controller.go:

package videoworkflow

import (
	"context"
	"crypto/hmac"
	"crypto/sha256"
	"crypto/subtle"
	"encoding/base64"
	"errors"
	"net/http"
	"strconv"
	"strings"
	"sync"
	"time"
)

var (
	ErrOutcomeUnknown = errors.New("provider may have accepted the request")
	ErrRemoteNotFound = errors.New("provider job not found")
	ErrInvalidState   = errors.New("invalid state transition")
)

type State string

const (
	StateAccepted      State = "accepted"
	StateDispatching   State = "dispatching"
	StateSubmitUnknown State = "submit_unknown"
	StateSubmitted     State = "submitted"
	StateRunning       State = "running"
	StateSucceeded     State = "succeeded"
	StateReady         State = "ready"
	StateFailed        State = "failed"
)

var stateOrder = map[State]int{
	StateAccepted:      10,
	StateDispatching:   20,
	StateSubmitUnknown: 25,
	StateSubmitted:     30,
	StateRunning:       40,
	StateSucceeded:     50,
	StateReady:         60,
	StateFailed:        60,
}

type Job struct {
	ID             string
	IdempotencyKey string
	RemoteID       string
	State          State
	Version        int64
	Failure        string
}

type Provider interface {
	Submit(context.Context, string) (string, error)
	LookupByIdempotencyKey(context.Context, string) (string, bool, error)
}

type Controller struct {
	mu          sync.Mutex
	provider    Provider
	jobs        map[string]Job
	jobByRemote map[string]string
	seenEvents  map[string]struct{}
}

func NewController(provider Provider) *Controller {
	return &Controller{
		provider:    provider,
		jobs:        make(map[string]Job),
		jobByRemote: make(map[string]string),
		seenEvents:  make(map[string]struct{}),
	}
}

func (c *Controller) Create(jobID string) Job {
	c.mu.Lock()
	defer c.mu.Unlock()
	if job, ok := c.jobs[jobID]; ok {
		return job
	}

	job := Job{
		ID:             jobID,
		IdempotencyKey: "video:" + jobID + ":attempt:1",
		State:          StateAccepted,
		Version:        1,
	}
	c.jobs[jobID] = job
	return job
}

func (c *Controller) Dispatch(ctx context.Context, jobID string) error {
	c.mu.Lock()
	job, ok := c.jobs[jobID]
	if !ok || job.State != StateAccepted {
		c.mu.Unlock()
		return ErrInvalidState
	}
	job.State = StateDispatching
	job.Version++
	c.jobs[jobID] = job
	c.mu.Unlock()

	remoteID, err := c.provider.Submit(ctx, job.IdempotencyKey)

	c.mu.Lock()
	defer c.mu.Unlock()
	job = c.jobs[jobID]
	if errors.Is(err, ErrOutcomeUnknown) {
		job.State = StateSubmitUnknown
		job.Failure = err.Error()
		job.Version++
		c.jobs[jobID] = job
		return err
	}
	if err != nil {
		job.State = StateFailed
		job.Failure = err.Error()
		job.Version++
		c.jobs[jobID] = job
		return err
	}

	c.bindRemote(&job, remoteID)
	c.jobs[jobID] = job
	return nil
}

func (c *Controller) ReconcileUnknownSubmit(
	ctx context.Context,
	jobID string,
) error {
	c.mu.Lock()
	job, ok := c.jobs[jobID]
	c.mu.Unlock()
	if !ok || job.State != StateSubmitUnknown {
		return ErrInvalidState
	}

	remoteID, found, err := c.provider.LookupByIdempotencyKey(
		ctx,
		job.IdempotencyKey,
	)
	if err != nil {
		return err
	}
	if !found {
		return ErrRemoteNotFound
	}

	c.mu.Lock()
	defer c.mu.Unlock()
	job = c.jobs[jobID]
	if job.State != StateSubmitUnknown {
		return nil
	}
	c.bindRemote(&job, remoteID)
	c.jobs[jobID] = job
	return nil
}

func (c *Controller) bindRemote(job *Job, remoteID string) {
	job.RemoteID = remoteID
	job.State = StateSubmitted
	job.Failure = ""
	job.Version++
	c.jobByRemote[remoteID] = job.ID
}

func (c *Controller) ApplyProviderEvent(
	eventID string,
	remoteID string,
	incoming State,
) error {
	if incoming != StateRunning &&
		incoming != StateSucceeded &&
		incoming != StateFailed {
		return ErrInvalidState
	}

	c.mu.Lock()
	defer c.mu.Unlock()
	jobID, ok := c.jobByRemote[remoteID]
	if !ok {
		return ErrRemoteNotFound
	}
	if _, duplicate := c.seenEvents[eventID]; duplicate {
		return nil
	}
	c.seenEvents[eventID] = struct{}{}

	job := c.jobs[jobID]
	if job.State == StateReady ||
		job.State == StateSucceeded ||
		job.State == StateFailed {
		return nil
	}
	if stateOrder[incoming] <= stateOrder[job.State] {
		return nil
	}
	job.State = incoming
	job.Version++
	c.jobs[jobID] = job
	return nil
}

func VerifyStandardWebhook(
	secret string,
	header http.Header,
	body []byte,
	now time.Time,
	tolerance time.Duration,
) error {
	messageID := header.Get("webhook-id")
	timestamp := header.Get("webhook-timestamp")
	signatures := header.Get("webhook-signature")
	if messageID == "" || timestamp == "" || signatures == "" {
		return errors.New("missing webhook signature headers")
	}

	seconds, err := strconv.ParseInt(timestamp, 10, 64)
	if err != nil {
		return errors.New("invalid webhook timestamp")
	}
	delta := now.Sub(time.Unix(seconds, 0))
	if delta < -tolerance || delta > tolerance {
		return errors.New("webhook timestamp outside tolerance")
	}

	encodedSecret := strings.TrimPrefix(secret, "whsec_")
	key, err := base64.StdEncoding.DecodeString(encodedSecret)
	if err != nil {
		return errors.New("invalid webhook secret")
	}

	signed := make([]byte, 0, len(messageID)+len(timestamp)+len(body)+2)
	signed = append(signed, messageID...)
	signed = append(signed, '.')
	signed = append(signed, timestamp...)
	signed = append(signed, '.')
	signed = append(signed, body...)

	mac := hmac.New(sha256.New, key)
	_, _ = mac.Write(signed)
	expected := mac.Sum(nil)

	for _, candidate := range strings.Fields(signatures) {
		versionAndValue := strings.SplitN(candidate, ",", 2)
		if len(versionAndValue) != 2 || versionAndValue[0] != "v1" {
			continue
		}
		actual, decodeErr := base64.StdEncoding.DecodeString(versionAndValue[1])
		if decodeErr == nil &&
			subtle.ConstantTimeCompare(actual, expected) == 1 {
			return nil
		}
	}
	return errors.New("webhook signature mismatch")
}

The reference package was exercised with fault-injection tests for an accepted-but-timed-out submit, a delayed state event, a modified body, and a stale signature. The sample deliberately omits provider HTTP payloads because each adapter must be generated from its current official contract, not copied from a fictional universal endpoint.

Build a Release Gate

A production release should fail if fault injection breaks an invariant. Test adapters against recorded, redacted provider fixtures and exercise the complete workflow in a non-production account.

Also verify the user experience. Refreshing the page must not create another job. A resumed session must see the same state. Error messages should distinguish invalid input, policy rejection, capacity delay, generation failure, and asset-delivery failure without exposing provider secrets.

FAQ

How should an AI video API be integrated into a production application?

Create a local job and return its ID before invoking a provider. A worker dispatches an attempt from a transactional outbox, stores the remote identity, and advances the job through a monotonic state machine. Signed webhooks provide fast updates, scheduled polling repairs missing events, and an ingestion worker copies the result into durable storage. This architecture decouples your user contract from provider-specific APIs.

What should the system do when submission times out?

Assume the outcome is unknown if request bytes may have reached the provider. Keep the same attempt and provider idempotency key, then query by that key or external task ID. Do not generate a new key merely because a worker retried. If the provider offers no reliable lookup, quarantine the attempt for bounded investigation rather than risk duplicate output and cost.

Should video generation use webhooks or polling?

Use both through one transition function. A webhook reduces completion latency but can be delayed, replayed, or lost. Polling is slower and consumes quota, but it repairs gaps and verifies stale jobs. Deduplicate webhook IDs, make state updates monotonic, and schedule polling with backoff and jitter.

How do you prevent duplicate jobs and duplicate billing?

Deduplicate the client request with a tenant-scoped key, identify each provider attempt with a stable key, and enforce both with database uniqueness. Store callback IDs in an inbox and make downstream effects idempotent. Most importantly, distinguish a known rejection from an unknown submission outcome; only the former is immediately safe to retry as a new attempt.

When is a generated video ready for the user?

It is ready only after the output is copied to storage you control and its validation record commits. Check origin, redirects, media type, byte limit, duration, dimensions, checksum, policy result, and provenance metadata. A provider's succeeded status only starts this delivery phase.

Summary

Reliable AI video generation depends on a provider-neutral control plane. Persist local intent before network calls, separate jobs from attempts, represent uncertainty explicitly, reconcile webhooks with polling, and make every state change idempotent and monotonic. Then secure the asset path, settle actual attempt costs, and operate against outcome-level SLOs.

This architecture also applies when the media model changes. The adjacent multimodal pipeline engineering guide explains input validation and model-independent adapters, while the Text-to-Video glossary covers the generation concept itself.

Primary Sources and Related Learning

• Google Gemini API video generation
• Google Gemini API webhooks
• OpenAI Videos API guide
• Kling image-to-video API reference
• Runway API usage guide
• Runway production go-live guide
• Standard Webhooks specification
• RFC 9110 Retry-After semantics
• AI Video Generation: Veo vs Sora vs Kling
• AI Video Evaluation Protocol