TikTok Agent Control
An AI agent that runs a TikTok account's content: it reads the numbers, decides what to make, builds the video and posts it.
Part of the same system as Agentic Video Editing. Two halves of one system. This side watches the account and decides what to make; the editing agent builds the clip inside After Effects and hands back a project to review.
- Role
- Solo: architecture, agents, dashboard
- Year
- 2026
- Type
- AI & Agents
- Status
- Prototype
$ cat brief.md# In short
- The problem
- Running a content account means reading the analytics, deciding what to make next and then making it. Those are usually three different tools, and none of them remembers the account.
- What I did
- Solo: the architecture, the agent and its memory, the video pipeline, the background worker and the dashboard.
- Result
- A prototype where one agent per account reads real numbers, plans the next clip, builds it and posts it at the account's best time, through a real Android phone running TikTok.
- Prototype
- Runs locally with a background worker; read-only TikTok API access
- Auto-posting
- Through a USB-connected Android phone; opt-in, with a dry-run mode that stops before Post
$ stats --summary# In numbers
- 1
- agent per account, with memory of only that account
- 2
- posts a day on Autopilot by default, at fixed slots
- 20
- integrations: sources, media, voice and rendering
- 0
- taps on the phone: the video is opened, captioned and posted by the agent
$ ./demo --play# Watch it run
$ ls desktop/# On desktop
$ cat README.md# Overview
A control room where each TikTok account has its own agent. Mine is called Nova. It knows only its account: the analytics, the posts that worked and failed, and the content rules you set. It reads that data, writes an honest analysis of what's working, decides what to make next and produces the clip.
You can ask for a video in the chat or switch on Autopilot, which posts at fixed daily slots, by default twice a day. Finished videos reach me in Telegram with Approve and Reject buttons. TikTok's API won't let an unaudited app post, so publishing goes through the TikTok app on a real Android phone plugged into the PC: at the slot the video is copied over, opened in TikTok's editor, captioned with its hashtags and posted, with no one touching the phone.
$ tree architecture/# How it fits together
Dashboard
Next.js app
Built solo
Accounts, audience charts, videos, agent chat, clips and autopilot.
Agent & worker
Nova (per account)
Built solo
Claude Opus 5 / Sonnet 5 with the account's profile, memory and live data.
Worker
Built solo
Sync, analysis, sourcing, clipping and rendering as retryable jobs.
SQLite + Drizzle
Accounts, snapshots, posts, jobs, chats and memory.
Media
yt-dlp + ffmpeg
Download sources and cut them.
Remotion
Renders finished 1080×1920 clips.
After Effects agent
My other project
Builds the edit inside After Effects for review.
Publishing
Phone driver
Built solo
Python + uiautomator2 over adb: opens the video in TikTok, writes the caption, taps Post.
Android phone
Plugged into the PC over USB, logged in to the account.
$ cat flow.log# How it flows
- 1.
Sync the account · Worker
Followers, views, likes and per-video stats are pulled into snapshots.
- 2.
Read the numbers · Claude
Nova writes what's actually working and what to make next, from live data only.
- 3.
Find the source · yt-dlp · DVIDS
A long interview or public-domain footage that tells the chosen story.
- 4.
Cut the clip · ffmpeg · Remotion · AE
Captions become sentences, the moment is picked, then it's cut, captioned and rendered.
- 5.
Post it · Android phone
At the slot, the worker opens the clip in TikTok on the phone, writes the caption and hashtags and taps Post. A late clip goes out as soon as it's ready.
$ git log --author=amar# What I built
- One agent per account with layered memory: a binding content profile, standing directives distilled from chat (say "stop using trending audio" once and it holds), and live data.
- Account analysis written by Claude from real numbers: which topics and lengths carry the account, when posting works, and what is dragging the median down.
- Autopilot that posts at fixed daily slots (by default 12:00 and 19:00 in the account's timezone), or at the account's measured best hours.
- Telegram as the owner's channel: warnings, milestones and finished videos arrive there with Approve and Reject buttons, and a rejected video can be revised from my notes.
- Long-form 16:9 videos for YouTube, made chapter by chapter from researched sources, with narration, sourced pictures, picture credits, chapter timestamps and resume after a stop.
- Comment reading and reply drafting, plus 18-second teaser versions of finished videos.
- Automatic publishing through a real phone: the worker copies the finished video over adb, opens it in TikTok, types the caption and hashtags and taps Post. Buttons are found by their text with uiautomator2, not by screen position, so it works on any screen size.
- Video production pipeline: find a source (long YouTube interviews, or public-domain footage from DVIDS and the Internet Archive), turn captions into sentences, pick the moment, then cut and caption it.
- Two render paths: Remotion for finished clips, or a bridge to my After Effects agent that leaves an edit in an AE project for a human to review.
- Rights-first safety rails: footage starts as unverified, music comes only from a folder the owner filled, and phone posting is opt-in, with a dry run that stops on the Post screen.
- Background worker where every step is a row in a jobs table with retries and exponential backoff, so one failed download never stalls the pipeline.
- Dashboard with follower and view trends, per-video stats, the agent chat and a live log of what the agent is doing and why.
$ cat fixed-bugs.md# Challenges I solved
An agent that makes up numbers is worse than no agent.
Nova may only quote numbers from live data and has to say when something is missing. Its analysis is rebuilt from fresh snapshots.
Downloading and rendering video is slow and often fails.
Each step is a job with retries and exponential backoff, run by a separate worker so the dashboard stays fast.
Remotion needs React 18, but the app runs on React 19.
Remotion lives in its own workspace and is driven through its command line: settings go in as JSON and a finished mp4 comes out.
Footage rights: archive.org mirrors a lot of copyrighted material.
Sources are limited to verifiable public-domain collections, clips start as unverified, and music only comes from a folder the owner filled.
Podcast captions arrive as one long stream of words.
They're grouped into sentences first, so cuts, captions and effects land on real sentence boundaries.
TikTok's API won't publish for an app that hasn't passed its audit.
The video is posted the way a person would: through the TikTok app on an Android phone, driven over adb. Pop-ups are dismissed as they appear, every run ends with a screenshot, and a failed post retries with backoff.
$ cat learned.txt# What I learned
- Designing an agent with memory that the user can steer in plain language.
- Building long-running pipelines that recover from failures on their own.
- Treating rights and safety as product requirements, not an afterthought.
$ cat package.json# Stack
Next.js · TypeScript · Claude · SQLite · Drizzle · ffmpeg · yt-dlp · Remotion · Zod · Python · adb · uiautomator2