AI Reel Creator

A Python pipeline that transforms an audio file (song, poem, recitation, or any spoken/musical content) and its .srt subtitle file into a video reel.

The pipeline uses AI models of your choice (tested with Ollama Cloud) to select which audio file to work with based on your instructions, then picks the most relevant segment to feature. Both decisions are logged to a database for traceability and to avoid redundant work. From there, it generates a shot-by-shot image plan describing the visuals for each moment, uses an AI image generation model (I used Stable Diffusion via the AUTOMATIC1111 WebUI, hosted on cloud) to render those images, and stitches everything together into a synced reel with burned-in subtitles.

below are some reels created using this pipeline

Planned improvements:

Generating the .srt transcript directly from the audio, removing the need to supply one manually

Replacing static images with generated video clips or animated stills, giving each reel real motion instead of a slideshow feel

a character-lock system so that when a poem references a specific person, that character's appearance stays the same across every reel generated from that poem

note: the project isn't finished yet