The model worked. The infrastructure was the problem.

The application needed to remain available even when no video was being generated.

However, the pipeline depended on GPU-intensive AI models. Running those models on the same always-on infrastructure would couple application availability with expensive GPU resources.

I separated the application/API layer from the GPU inference layer.

The application handles requests, workflow state, and storage, while GPU workers are responsible for the actual AI inference.

This gives the two parts of the system different scaling and lifecycle characteristics.

The API does not need a GPU simply because the AI workload does. GPU infrastructure can be scaled according to video-generation demand rather than application traffic.

This introduces additional infrastructure boundaries and makes communication between the application and inference layer more explicit.

Job state, failures, and outputs need to be handled across services.

The deployed architecture physically separates AWS application/storage infrastructure from RunPod Serverless GPU inference.

The AI pipeline runs independently from the application server instead of requiring the API itself to host the GPU models.

Video generation is not a continuous workload.

There can be periods where no videos are being generated, followed by periods with multiple GPU-intensive jobs.

Keeping a GPU instance running continuously would mean paying for compute even when the system is idle.

I used RunPod Serverless for the GPU workloads so that inference capacity could be provisioned around actual jobs.

The infrastructure model better matches the workload:

This separates application availability from GPU availability.

On-demand execution introduces startup overhead and makes cold-start behavior part of the user experience.

The system has to tolerate the fact that GPU capacity may not always be immediately available.

The production deployment uses RunPod Serverless for the GPU-intensive stages rather than keeping a dedicated GPU machine continuously active — GPU compute follows workload demand instead of running permanently.

Generating a lecture video can take significantly longer than a normal HTTP request.

Keeping an API connection open while TTS, video generation, and lipsync execute would make the API responsible for a long-running computation — creating timeout, reliability, and resource-management problems.

I treated video generation as a job rather than a synchronous API operation.

The API accepts the request and the generation pipeline processes the work independently.

This gives the application a clear separation between the request lifecycle and the generation lifecycle.

The user doesn't need to keep an HTTP request alive for the entire duration of GPU inference.

Asynchronous processing adds state management. The application needs to know whether a job is:

1. queued→
2. running→
3. completed→
4. failed

It also requires handling retries and partial failures explicitly.

The deployed system represents video generation as a processing workflow rather than executing the pipeline inside the API request. The API and GPU execution have independent lifecycles.

The final lecture video was not produced by one model.

Speech generation, teacher/video generation, visual generation, and synchronization were handled by different components. Treating them as one opaque inference process would make failures hard to identify and components hard to replace.

I kept the generation process as a sequence of explicit stages.

1. Input→
2. TTS→
3. Teacher Video→
4. Visual / Slides→
5. Lipsync→
6. Final Video

Each stage has a different computational characteristic and can evolve independently.

A model can be replaced without requiring the entire application architecture to change.

More stages mean more intermediate data, more orchestration, and more potential failure points.

The system becomes more complex than a single inference endpoint.

The deployed application integrates VietTTS, MimicTalk, and Stable Diffusion-based processing as separate parts of the workflow — an orchestration of several AI workloads rather than a single model endpoint.

Generated videos and intermediate artifacts are much larger and longer-lived than the GPU processes that create them.

If outputs were tightly coupled to the GPU execution environment, they could become difficult to access once the compute process finished.

I treated generated media as persistent application data rather than something owned by the GPU worker.

The compute layer produces the artifact; storage owns its lifecycle.

This allows GPU workers to remain temporary.

A worker can execute a job, produce the output, and disappear without becoming the permanent home of the generated video.

The system needs explicit upload/download paths and additional data movement between compute and storage.

Large video files also make transfer time and storage management part of the system design.

The architecture separates GPU execution from persistent video storage and application access — generated artifacts survive independently of the GPU environment.