The Complete Guide to Enterprise Video Processing Architecture

Why Video Architecture Is Different

Video processing breaks every assumption from traditional web architecture:

The Scale Problem

Metric	Web Request	Image Processing	Video Processing
Payload size	10 KB	5 MB	500 MB - 10 GB
Processing time	50 ms	500 ms	5-60 minutes
Compute cost	$0.000001	$0.001	$0.10 - $2.00
Memory required	50 MB	500 MB	8-32 GB
Hardware	CPU	CPU or GPU	GPU required

A single 4K video enhancement job uses more resources than 10,000 typical web requests. Your architecture must be designed around this reality from day one.

The Reliability Problem

When a web request fails, retry in milliseconds. When a video job fails 30 minutes in:

• You've lost 30 minutes of GPU time
• The customer is frustrated
• Retry means another 30 minutes
• Error messages may not explain what went wrong

Video systems need checkpointing, partial results, and intelligent retry strategies.

The Economics Problem

GPU infrastructure is expensive:

• On-premise: $10K-$40K per GPU server, plus maintenance
• Cloud: $1-$4/hour for comparable GPU instances
• Utilization: Idle GPUs cost as much as busy ones

Architecture decisions directly impact cost. The wrong choices can make video processing unprofitable.

The Reference Architecture

Enterprise video processing systems share a common architecture:

┌───────────────────────────────────────────────────────────────────────┐
│                           CLIENT LAYER                                 │
│  Web Dashboard  │  Mobile App  │  API Clients  │  Integrations        │
└────────────────────────────────┬──────────────────────────────────────┘
                                 │
                                 ▼
┌───────────────────────────────────────────────────────────────────────┐
│                           API LAYER                                    │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  │
│  │   Upload    │  │    Jobs     │  │   Status    │  │  Download   │  │
│  │   Service   │  │   Service   │  │   Service   │  │   Service   │  │
│  └─────────────┘  └─────────────┘  └─────────────┘  └─────────────┘  │
└────────────────────────────────┬──────────────────────────────────────┘
                                 │
                    ┌────────────┼────────────┐
                    ▼            ▼            ▼
              ┌──────────┐ ┌──────────┐ ┌──────────┐
              │  Queue   │ │ Database │ │ Storage  │
              │ (Redis)  │ │(Postgres)│ │   (S3)   │
              └────┬─────┘ └──────────┘ └──────────┘
                   │
                   ▼
┌───────────────────────────────────────────────────────────────────────┐
│                         WORKER LAYER                                   │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  │
│  │  GPU Worker │  │  GPU Worker │  │  GPU Worker │  │  GPU Worker │  │
│  │   (A10G)    │  │   (A10G)    │  │   (L40S)    │  │   (L40S)    │  │
│  └─────────────┘  └─────────────┘  └─────────────┘  └─────────────┘  │
└───────────────────────────────────────────────────────────────────────┘

Layer Responsibilities

• Client Layer: Authentication, file upload, status polling/webhooks, result download
• API Layer: Request validation, job creation, rate limiting, authentication
• Queue: Job distribution, priority management, retry handling
• Worker Layer: GPU processing, model inference, frame extraction/encoding
• Storage: Input staging, output delivery, signed URL generation

Pipeline Architecture

Video pipelines process video through sequential stages. The architecture of these stages determines performance, reliability, and maintainability.

The Ingest Stage

Getting video into the system:

• Chunked upload: Large files upload in parts for resumability
• URL fetch: Download from customer storage (S3, GCS, signed URLs)
• Streaming: Process without downloading entirely (advanced)

The Processing Stage

Where the actual work happens:

• Decode: Extract frames from container format
• Process: Apply AI models to frames/audio
• Encode: Repackage into output format

The Delivery Stage

Getting results to customers:

• Signed URLs: Time-limited download links
• Webhooks: Notify when processing completes
• Streaming: Progressive download during processing

Each stage can scale independently. More upload capacity doesn't require more GPUs.

GPU Infrastructure Decisions

GPU infrastructure is the biggest architectural decision. Build vs. buy, on-premise vs. cloud, which GPUs to use.

GPU Options for Video AI

GPU	VRAM	Use Case	Cloud Cost
NVIDIA T4	16 GB	Light inference, development	~$0.50/hr
NVIDIA A10G	24 GB	Production inference	~$1.00/hr
NVIDIA L40S	48 GB	High-resolution, complex models	~$2.50/hr
NVIDIA A100	40/80 GB	Training, batch processing	~$3.00/hr
NVIDIA H100	80 GB	Latest models, maximum throughput	~$4.00/hr

Cloud vs. On-Premise

Cloud advantages:

• Scale to zero when idle
• No upfront capital
• Access to latest GPUs
• Geographic distribution

On-premise advantages:

• Lower cost at high utilization (>70%)
• Data sovereignty guarantees
• No vendor lock-in
• Predictable capacity

The hybrid approach: On-premise for baseline load, cloud for bursts.

Scaling Video APIs

Scaling video APIs requires different strategies than scaling web applications.

The Queue-Based Pattern

Video APIs should be async by default:

1. Client submits job via API
2. API validates and queues job immediately
3. API returns job ID (response time: 100ms)
4. Worker picks up job from queue
5. Worker processes video (minutes)
6. Worker updates status, triggers webhook
7. Client retrieves result via signed URL

This decouples API response time from processing time.

Scaling Dimensions

• API layer: Horizontal scaling behind load balancer
• Queue: Redis cluster or managed queue service
• Workers: Horizontal scaling based on queue depth
• Storage: Object storage scales automatically

Capacity Planning

Work backwards from requirements:

Target: 10,000 jobs/day
Average processing time: 5 minutes
Jobs per GPU per hour: 12
Jobs per GPU per day: 288
GPUs needed: 10,000 / 288 = 35 GPUs (at 100% utilization)
With 70% target utilization: 50 GPUs

Microservices for Video

Microservices architecture for video processing has distinct patterns.

Service Decomposition

Split by function, not by feature:

• Upload service: Handles chunked uploads, URL fetching
• Job service: Creates and tracks jobs, manages workflow
• Worker service: Runs on GPU, processes video
• Delivery service: Generates signed URLs, manages output lifecycle
• Webhook service: Sends notifications, handles retries

Communication Patterns

• Sync: API layer to database (job creation)
• Async: API layer to workers (via queue)
• Event-driven: Job completion triggers webhook

What Not to Microservice

Avoid over-decomposition:

• Don't split the video processing pipeline into separate services
• Don't microservice configuration management
• Don't create services for single-use functions

Enterprise API Design

Enterprise-grade APIs require patterns beyond CRUD.

Idempotency

Every POST must be idempotent:

POST /v1/jobs
X-Idempotency-Key: customer-job-12345

# Retry with same key = same result, not duplicate job

Webhooks

Robust webhook implementation:

• Signed payloads (HMAC)
• Exponential backoff retries
• Delivery status tracking
• Fallback to polling

Pagination and Filtering

For job listing and history:

GET /v1/jobs?status=completed&since=2026-01-01&limit=100&cursor=abc123

Versioning

URL versioning for breaking changes:

/v1/jobs  # Current
/v2/jobs  # Breaking changes

Rate Limiting

Tiered rate limits with clear headers:

X-RateLimit-Limit: 100
X-RateLimit-Remaining: 95
X-RateLimit-Reset: 1625097600

Distributed Processing

Distributed processing parallelizes work across multiple GPUs or machines.

When to Distribute

• Long videos: Split into segments, process in parallel, merge
• Multiple operations: Pipeline stages on different GPUs
• High resolution: Tile-based processing for 8K+

Coordination Patterns

• Scatter-gather: Split work, assign to workers, collect results
• Pipeline: Each worker handles one stage, passes to next
• MapReduce: Map operations to workers, reduce results

Challenges

• Segment boundaries: Artifacts at split points
• Coordination overhead: Management complexity
• Partial failure: What if one segment fails?

BetterVideo handles distribution internally — you submit one job, we parallelize as needed.

Reliability Patterns

Reliability is critical for video systems where jobs take minutes and failures are expensive.

Checkpointing

Save progress at intervals:

• Every N frames processed, checkpoint state
• On failure, resume from last checkpoint
• Reduces wasted work from minutes to seconds

Health Checks

Monitor worker health:

• GPU memory utilization
• Processing throughput
• Error rates
• Queue depth per worker

Graceful Degradation

When capacity is constrained:

• Queue priority (paid tiers first)
• Reject new jobs with retry-after header
• Fallback to lower-quality processing

Disaster Recovery

• Multi-region deployment
• Cross-region job migration
• Automated failover

Cost Optimization

Cost optimization determines whether video processing is profitable.

GPU Utilization

The #1 cost lever:

• Target: 70%+ utilization
• Problem: Idle GPUs cost as much as busy ones
• Solutions: Autoscaling, spot instances, serverless GPU

Serverless GPU (Modal, RunPod, etc.)

Pay only for processing time:

• Container spins up in seconds
• Billed per second of compute
• No idle cost
• Cold start overhead (5-30 seconds)

Spot/Preemptible Instances

60-90% discount with interruption risk:

• Good for batch processing
• Requires checkpointing
• Not for SLA-bound workloads

Right-Sizing

Don't over-provision:

• A10G for standard enhancement (not A100)
• Profile memory usage, pick appropriate VRAM
• Test throughput to find optimal instance

BetterVideo's Approach

We run on Modal's serverless GPU infrastructure:

• Scale to zero when idle
• Cold start under 5 seconds
• A10G for standard, L40S for high-resolution
• No infrastructure management

Architecture Decision Framework

Use this framework when making architectural decisions:

Build vs. Buy

• Build if: Video is your core product, you have GPU expertise, you need custom processing
• Buy if: Video is a feature, you want to focus on your differentiator, you need compliance

Sync vs. Async

• Async always for jobs over 5 seconds
• Sync only for metadata operations

Monolith vs. Microservices

• Start monolith, extract services as needed
• Extract: upload, webhook, worker
• Don't extract: job state, configuration

Cloud vs. On-Premise

• Cloud default for flexibility
• On-premise only at >70% utilization or strict data residency
• Hybrid for baseline + burst