To sum it up in one sentence: Throw a 1-hour video into it, and it willlocal machineAfter running the speech recognition, hand the subtitles with timestamps to the big model to pick out the clips worth posting.
You drag twice on the timeline and export dozens of slices that can be sent directly with one click – no video or audio is uploaded in the whole process.
↑ Actual running screenshots: left player, right “preview screen”, middle timeline (subtitle track + slice track), bottom subtitle/slice/export panel.
The material is a 9 minute and 59 second interview, and AI has cut out 7 candidate clips.
The project has been open source:
All screenshots in this article are from the real operation of v0.3.4 (real projects, real recognition results, real export), without any synthesis or editing.
1. Why do this?
People who make content have probably gone through this process: a live broadcast replay/an interview recording/a class recording. It takes a whole day to cut it into 10 short videos.
What really takes time is not the operation of the editing software;Drag the progress bar back and forth to find “which sentence is worth cutting out separately”。
Taken apart, this thing is actually very mechanical:
- Convert speech into subtitles with timestamps (who said what when)
- Read the subtitles to determine which paragraphs are self-contained, informative, and catch people’s attention at the beginning
- Cut out the video according to the start and end time and export it into a movie
Step 2 is what the language model is best at, and steps 1 and 3 are tool chain issues. Since all three segments can be run locally, there is no reason to upload the user’s original video to any server.
So there is VClip: An Electron desktop application that recognizes, slices, and exports all locally. The only external call isOptionalText-level large model API
(You can also change it to local Ollama, so there will be no outbound requests at all).
2. What exactly can it do (implemented functions)
Let’s break down the functions first, and then talk about how technology supports these functions.
↑ The home page is a list of projects. Each card is directly marked with “Recognized subtitles/7 slices/Duration/Creation time”. Click on it to see the workbench.
1. Local speech recognition (offline)
- Built-in FunASR system model:SenseVoice Small(Chinese/English/Japanese/Korean/Cantonese, with token-level timestamp),Paraformer(Chinese and small editions)
- Silero VAD does voice activity detection. It first cuts out the “someone talking” segments and then recognizes them segment by segment – this not only saves computing power, but also avoids recognizing silence as hallucinated text.
- Identification runs on an independent
utilityProcessHere, the interface can still drag the timeline and change settings during 1-hour video recognition. - Models can be downloaded with one click on the settings page, or can be preset into the installation package off-screen.
2. AI analysis slicing + regular slicing to find out the details
- Feed the subtitles into character blocks (6000 words per block) to the large model and ask it to return structured JSON: title, summary, start and end time, rating, recommendation reason
- Services compatible with all OpenAI protocols: DeepSeek / Tongyi Qianwen / Silicon-based Flow / Ollama…
- You can publish videos without API Key: The built-in rule engine aggregates candidate slices based on subtitle semantic boundaries, pauses, and word count.
- When AI returns an empty array or fails to parse, it will automatically downgrade to regular slicing – ensuring that “if there are subtitles, there will be slicing” and you will not be left staring at an empty list.
↑ The 7 candidates cut out: title, start and end time, one-sentence summary, recommendation reason, rating bar. The “AI” mark in the upper right corner indicates that the source is a large model rather than a rule engine.
The “Export” switch on the right side of the card controls whether to participate in the export. The start and end times can be changed manually or directly dragged on the timeline.
3. Timeline: can be dragged, wiped, and hovered to view the screen
- Subtitle track + slice track dual tracks; slices can be dragged as a whole, drag the edge to change the start and end points, double-click to switch whether to export, and changes are saved in real time
- Click on the blank space to locate, and the video screen is linked in real time; toolbar support
90/1:30/01:02:03Three types of timecode jumps - Shortcut key: space to play and pause,
←/→±1 second (Shift±5 seconds),,/.frame by frame - Hover the mouse to display a thumbnail of the moment. + time code + the original text of the subtitles at that time, and draw a time scale line that is strictly aligned with the thumbnail
- independentPreview screenPanel, you can switch “Follow the mouse/Follow the playhead”. After selecting the slice, you can “Set as starting point/Set as end point” with one click.
4. One-click export (this step has the most details)
- Precise mode: Re-encoded (libx264/veryfast/CRF 20), the picture is accurately aligned to milliseconds
- Quick mode：
-c copyStream copying is completed in seconds without loss of image quality. - Export content is optional: “Single Slice” and “Slice Collection” are two independent switches – only slices, only collections, or both.
- Slice collection: Combine the selected slices into one in order
00_切片合集.mp4;When importing only the collection, each slice only exists in the temporary subdirectory as a splicing intermediate product. After the whole assembly is completed, there will be only one file in the output directory. - The entire subtitle can be exported to SRT/text with time code/JSON, and UTF-8 BOM can be written uniformly (Windows Notepad, VLC, and PotPlayer will not be garbled)
5. Packaging and desensitization
- electron-builder: Windows NSIS installation package +
win-unpackedGreen directory (copy and run), macOS produces DMG - Package built-inTwo desensitization gates：Scan it before entering the bag.
app.asarAfter generation, scan it once and hit the API Key to directly interrupt the build. - API Key only exists in the user data directory
settings.jsonnever enter the package, never enter the warehouse, and never enter the log (the log is only masked)
3. Technology stack
| layer | Selection | illustrate |
|---|---|---|
| desktop container | Electron 33 | need utilityProcess(running ASR), custom privileged protocol (media streaming), native Node extension (ONNX Runtime) |
| Build | electron-vite 2.3（Vite 5.4） | Main process/preload/rendering process three build configurations, HMR during development period |
| render layer | React 18.3 + Ant Design 5.21 + TypeScript 5.6 | Forms, tables, and sliders use antd directly; the timeline is a handwritten SVG/div layout. |
| speech recognition | sherpa-onnx-node 1.10(ONNX Runtime N-API binding) | Pure native binding, no need for Python or additional services; comes with Silero VAD package |
| Model | SenseVoice Small / Paraformer(FunASR series, ONNX int8) | int8 quantization, SenseVoice is about 240MB, Paraformer-small is about 82MB |
| Media processing | FFmpeg（ffmpeg-staticcollected when packing resources/ffmpeg-bin） |
Cut segments, combine sets, probe duration, frame extraction |
| large model | Any OpenAI compatible HTTP interface | Only send text (subtitles), not video; can point to local Ollama |
| Pack | electron-builder 25 | NSIS + dir, incl. beforePack / afterPack hook |
| test | Self-developed e2e + CDP probe | The flagship with 44 assertions is back and can run without adjusting LLM/ASR (no money required) |
Why sherpa-onnx instead of “start a Python service”: The Python runtime stuffed into desktop software is a bottomless pit (version, path, false positives from anti-virus software).sherpa-onnx-node It is the N-API binding of ONNX Runtime,npm install Once installed, it’s just one .node document,new sherpa.OfflineRecognizer(...) Direct adjustment, VAD also has ready-made packages. The price is to package it from app.asar insideasarUnpack out (native modules cannot be loaded from asar).
4. Six implementation details that really take time
The previous function descriptions all sound quite smooth, but behind each one there is a point that “if you don’t write it this way, it’s bad.” This part is the focus of this article.
1. media:// Protocol: Must implement HTTP Range/206 yourself
This was the most expensive lesson of the entire project.
The most intuitive way to write videos in Electron is to register a custom protocol, and then:
// ❌ 这样写，视频能播，但进度条拖不动
protocol.handle('media', (req) => net.fetch(pathToFileURL(filePath).toString()))
The phenomenon is very confusing: the video can be played normally, but of seekable only [0, 0]，Drag progress bar, timeline positioning,
All hover previews are invalid, and only the first few seconds of buffering can be played.。
reason:net.fetch(file://) There is neither in the returned response Content-Lengthnor Accept-Ranges，
request header Range And was thrown away. If the browser cannot get this information, it thinks that the resource cannot be accessed randomly.
So the protocol must implement Range semantics itself:
protocol.registerSchemesAsPrivileged([
{ scheme: 'media', privileges: { standard: true, secure: true, supportFetchAPI: true, stream: true } }
])
protocol.handle('media', async (request) => {
const { size } = await fs.promises.stat(filePath)
const range = parseRange(request.headers.get('Range'), size) // bytes=start-end
const baseHeaders = { 'Content-Type': mime, 'Accept-Ranges': 'bytes' }
if (!range) {
// 普通请求：给了 Content-Length 才能算出 seekable
return new Response(createReadStream(filePath), { status: 200, headers: { ...baseHeaders, 'Content-Length': String(size) } })
}
if (range.unsatisfiable) {
return new Response(null, { status: 416, headers: { ...baseHeaders, 'Content-Range': `bytes */${size}` } })
}
return new Response(createReadStream(filePath, { start: range.start, end: range.end }), {
status: 206,
headers: { ...baseHeaders, 'Content-Range': `bytes ${range.start}-${range.end}/${size}` }
})
})
Acceptance criteria (also one of the regression tests):
await fetch('media://file/D%3A/a.mp4', { headers: { Range: 'bytes=0-99' } }) // → 206
document.querySelector('video').seekable.end(0) === video.duration // → true
Conclusion: Custom protocols involving media streaming,200 / 206 / 416 three piece set +
Content-Length+Accept-RangesNo one can be missing。
The “can be played” that only returns 200 is an illusion.
2. Recognize that the UI cannot be stuck:utilityProcess + VAD segmentation
ASR is CPU-intensive. Running it in the main process will freeze the entire application, and running it in the rendering process will grab the UI thread. Electron 33’s utilityProcess
The correct answer: Node environment, independent process, through postMessage Communication failure does not affect the main process.
const child = utilityProcess.fork(path.join(__dirname, 'asr-worker.js'), [], { serviceName: 'vclip-asr' })
child.postMessage({ type: 'start', audioPath, modelDir, numThreads })
// worker 侧 postMessage({ percent, stage }) → 主进程转发到渲染进程的进度条
The process in the worker is ffmpeg 抽 16k 单声道 wav → 读成 Float32Array → VAD 切段 → 逐段 OfflineRecognizer：
const vadConfig = {
sileroVad: {
model: vadPath,
threshold: 0.5, // 语音判定阈值
minSpeechDuration: 0.3, // 短于 0.3 秒的"语音"当噪声丢掉
minSilenceDuration: 0.4, // 静音 0.4 秒以上才切断
maxSpeechDuration: 15, // 单段上限，避免一口气 3 分钟不切
windowSize: 512
},
sampleRate, numThreads: 1, provider: 'cpu'
}
const vad = new sherpa.Vad(vadConfig)
// 逐 512 采样点喂进去，vad.front() 取出一段语音 → 单独识别 → 按窗口偏移拼回全局时间轴
This has three advantages: silent segments are not sent to the model (saving time and less hallucinations), long audio does not need to be entered into the memory recognizer at once, and progress can be reported by segments.
3. The timestamp is hidden: Paraformer does not return the token time at all
There is an easy pitfall here:Paraformer series does not return token-level timestampsgiven only a paragraph of text.
If you directly regard it as “each sentence starts from 0 seconds”, the subtitle time will be completely wrong, and the alignment of subsequent slicing, hover preview, and export will be completely broken.
There are two layers in the code:
hasUsableTimestamps(timestamps, duration)——After identifying, first check whether the timestamp is monotonic, whether it falls within the duration, and whether it is all 0- Go when unavailable
buildSubtitlesProportional()——The total duration is allocated to each token in proportion to the number of characters, and then combined with the VAD segment boundary for correction and fragment merging
Therefore, the advice in the product is:Precise sentence-by-sentence timingJust use SenseVoice Small (supports token timestamp),
Paraformer is more suitable for scenarios where “just text and fast is enough”. This difference is also stated in the FAQ of the README – not hidden.
4. LLM slicing pipeline: blocking, constraint, verification, and bottom-up
It’s not feasible to feed an hour of subtitles (tens of thousands of words) to the model at once: the context exceeds the limit, the cost is high, and the model will also start to “summarize the full text” instead of “select fragments.”
So break it into chunks first:
const MAX_CHARS_PER_BLOCK = 6000 // 按字符量切块，块之间保留时间边界
export function chunkSegments(segments) { /* ... */ }
Each block is requested individually and requests are returnedPure JSON array(No markdown code blocks, no explanation text), then normalizeClips()
Do a strict check: the time must be within the block range, the start and end cannot be reversed, the duration must be within the threshold, and any missing fields will be discarded.
The duration threshold isAdaptable to the total length of the material– This one was made by stepping on a pit. In the early version, “slice for at least 15 seconds” was hardcoded.
As a result, all 10-second short video candidates (such as a spoken word broadcast) were discarded, and the user saw “Analysis completed, 0 slices”.
Now the prompt word will first tell the model the total length of the material:
- Short material (< 1 minute): slices can be 8 seconds ~ full film duration, "Better short than empty"
- Regular material: 45 seconds to 8 minutes is best, short video format is 1 to 4 minutes is best
andReturning an empty array is explicitly prohibited：
## 时长与数量要求
- **禁止返回空数组 []。**
- 若确实找不到亮点，就挑选语义最完整、时长最长、最连贯的段落作为切片；
- 只有在"字幕为空"这种极端情况下才允许返回空数组。
Add another layer of engineering cover:analyzeProject() When the number of slices obtained is 0 or JSON parsing fails, directly call analyzeByRules()，
Candidates are aggregated according to the semantic boundaries of subtitles (end-of-sentence punctuation, long pauses, topic switches).
Only when these three layers (prompt word constraints → result verification → rule back-up) are stacked can we dare to promise users that “if there are subtitles, there will be slices.”
5. Hover thumbnail: solo seek + rAF merge
“When the mouse slides over the timeline, the picture at that moment is displayed” sounds simple, but it is a performance trap in implementation:
mouse mousemove It can be triggered hundreds of times per second. If every time video.currentTime = tthe decoder will accumulate a bunch of seek requests,
The screen starts to twitch and the CPU is maxed out.
ScrubVideo The solution isSingle-flight + rAF merge：
// 同一时刻只允许一个 seek 在途；期间的新的目标时间只覆盖 targetRef
// seeked 之后再由 requestAnimationFrame 取最新目标 → 天然去抖，不堆积
Two other real details:
- seek to just right
durationAt this point, some decoders will stop in place and not trigger.seekedevent – to be clamped toduration - εand add a timeout - If you are dragging a slice when hovering, the preview must give way (otherwise the dragging feel will be slowed down by the preview’s seek)
↑ The mouse is parked at 04:11: the floating layer is the picture at that moment (1280×720, decoded) + time code + the original text of the subtitle at that time;
The “Preview Screen” on the right switches to “Follow the Mouse”, showing the same frame. The blue blocks are subtitles and the red blocks are slices. You can tell at a glance which segments have been selected.
6. FFmpeg export: why there are two modes and how to put together the collection
Precise mode(recoded):
ffmpeg -y -ss -t -i in.mp4 \
-c:v libx264 -preset veryfast -crf 20 \
-c:a aac -b:a 128k -movflags +faststart out.mp4
-ss and -t put them all -i Before(Input side positioning), only with re-encoding can frame-level accuracy be achieved; if placed on the output side, the entire segment will be decoded first and then cut.
Slow and prone to bias.
Quick mode(stream copy):
ffmpeg -y -ss -t -i in.mp4 -c copy -avoid_negative_ts make_zero out.mp4
Completed in seconds and lossless, the price is that the starting point will be pulled to the nearest key frame (the difference between before and after is at most one GOP).-avoid_negative_ts make_zero
This is to zero out the common negative timestamps (caused by B frames) in stream copying, otherwise some players will swallow up the first few hundred milliseconds and misalign the audio and video.
Accurate mode intentionally does not produce any subtitles(Neither plug-in SRT nor burn into the screen).
This is not a missed feature: the time base output after re-encoding is not exactly the same as the original subtitles, and the plug-in SRT will be overall offset and misaligned with the picture.
If you want subtitles, use the entire subtitles to export, or switch to fast mode (the timestamp is consistent with the original film) and check “Additional generation of plug-in SRT”.
Slice collectionUse concat demuxer:
# list.txt 里每行 file '<路径>'，路径要先统一成 / 再转义单引号，否则 Windows 路径容易炸
# 生成逻辑：`file '${p.replace(/\\/g, '/').replace(/'/g, "'\\''")}'`
ffmpeg -f concat -safe 0 -i list.txt -c copy 00_切片合集.mp4
Try in quick mode first -c copyonce the fragment parameters are inconsistent (different resolution/frame rate/timebase),Automatic downgrade and re-encoding。
The failure of the collection does not affect the single slice, and is only prompted with a label in the result area.
There is an invariant that must be maintained:Collection duration ≡ sum of independent slice durations(Not “sum of requested ranges” – stream copy becomes longer due to keyframes).features.js There is an assertion specifically comparing this difference (< 0.35s), because it may break once the export logic is changed.
The export panel looks like this:
↑ Export mode (accurate/fast), slicing subtitle strategy (default “do not generate any subtitles”, the reason is written directly next to the option),
The exported content (single slice/slice collection/attached clips.json) are independent of each other and can be combined in any way.
↑ In the fast mode, 7 slices are imported in a few seconds. The real output directory is given in the result area. Click “Open Directory” to access directly.
Note that this combination is “only import a single slice” (the collection one is unchecked), so 7 independent MP4s are produced.
5. Security: Block the issue of “Key being included in the installation package” from the process
The most common way for Electron applications to overturn is: developer local settings.json There is a real API Key in it,
I accidentally brought it in during a certain packaging, and then the installation package was passed to the group.
What VClip does isNo room for “remember to clean”directly use hooks to block the packaging process:
| gate | Location | do what |
|---|---|---|
| Before packaging | beforePack → scan-secrets.js |
Scan files that are about to be included in the package (out/**、package.json、build/ffmpeg-bin), hit sk-…、Bearer …private key block, cloud vendor certificate, literal key assignment → throw an error and interrupt the build |
| After delivery | afterPack |
Scan the generated app.asarand check whether mixed in settings.json / .env / *.pem |
Three things that come with it:
- Packaging whitelist：
fileswrite onlyout/**(compiled product). Source code, scripts, documentation,.envNever enter the bag——
It is not “exclude sensitive files”, but “only allow these files to be released”. In the opposite direction, the risk of missed configuration is an order of magnitude smaller. - log mask:Only print on hit
sk-abc…456(len=35)never write the complete key to the log - Run-time desensitization: When the LLM interface reports an error, go through the key traces in the response body first.
redactSecrets()Then display to the interface
The settings page explains the “life cycle” of the key by the way – fill it in here only once, and the landing is %APPDATA%\VClip\settings.json，
Read when the application starts and write when saving, without touching the installation directory:
↑ The above is the large model configuration (API address/model/Key/Temperature), and the following is the local ASR model management, with volume and “downloaded” status.
This screenshot comes from a clean configuration directory, so the API Key is empty – also from the side: the installation package itself does not contain any Key.
There is another detail worth mentioning:This scanner will scan itself. I wrote a fake Key in the test script sk-SYNTHETIC-... used for
Controlled experiment to “prove that the application only reads keys from userData”, results npm run scan:secrets During the full warehouse audit, I hit myself.
Finally, it was changed to string splicing generation (the values are exactly the same and the behavior remains unchanged), so that the audit can stably produce 0 hits.
When your security check starts blocking your test code, it’s really working.
The self-check list before release has also been made into a script:
npm run scan:secrets # 全仓审计硬编码密钥
npm run verify:asar # 校验已生成的 app.asar：结构 + 密钥 + 代码标记
npm run verify:dist # 从安装包内层解出 app.asar，用本机真实密钥的 5 种编码变体做字面量比对
6. List of pitfalls (for people who want to do similar projects)
| pit | Phenomenon | solution |
|---|---|---|
| Custom protocols do not have Range | The video can be played, but seekable only [0,0]can’t drag |
Return by yourself 200/206/416 + Content-Length + Accept-Ranges |
| Native module into asar | sherpa-onnx-node Loading failed |
asarUnpack: "**/node_modules/sherpa-onnx-*/**" |
| VAD audio slice parameter transfer | napi newspaper External buffers are not allowed |
vad.front() Must be passed falsetake a copy instead of the external memory buffer |
| ASR runs in the main process | The entire interface freezes during recognition | utilityProcess.fork() Independent process + progress message |
| Paraformer no token time | The subtitles are all 0 seconds or crowded together | Timestamp availability check + character proportional allocation + VAD boundary correction |
| Slicing duration hardcoded | Short video “Analysis completed but 0 slices” | The threshold is adaptive according to the total length of the material + Prompt words prohibit empty arrays + Rules are in place |
| Precision mode external subtitles | Overall subtitle offset and screen misalignment | Precise mode does not produce subtitles; if you want subtitles, go to fast mode or export as a whole |
| Hover preview crazy seek | As soon as the mouse slides, the screen twitches and the CPU becomes full. | Solo seek + rAF merge + disable preview when dragging |
For collection -c copy Splicing |
Fails directly when parameters are inconsistent | Automatic downgrade and re-encoding on failure without blocking a single slice |
| electron-builder first package | Cannot create symbolic link(macOS symbolic link in winCodeSign) |
Turn on Windows developer mode, or preset the cache directory/specify it explicitly ELECTRON_BUILDER_CACHE |
| Batch deletion of language packs | Matched electronLanguages Instead, the packaging is interrupted by the security policy. |
It’s simply not worthy, it just takes up 20MB more. |
7. What is and is not in the warehouse
This project is on GitHubOnly release source codeof:
NoSomething (intentional):
- ❌ Windows installation package (
VClip Setup x.y.z.exe） - ❌ Installation-free green package (
win-unpacked/） - ❌ Speech recognition model weight (320MB+)
They are all native build products,.gitignore inside release/、models/、build/ffmpeg-bin/ Exclude directly.
If you want to use the finished product, just build it yourself – for projects with Electron + native modules, this is the least likely to cause problems.
(You don’t need to bet on other people’s CPU instruction sets, and you don’t need to bet on the anti-virus software’s attitude toward unsigned exes).
Install from source (Windows)
git clone .git
cd videoclip
# 国内先配镜像，否则 Electron 二进制（100MB+）大概率下载卡死
npm config set registry
export ELECTRON_MIRROR=
export ELECTRON_BUILDER_BINARIES_MIRROR=
npm install
npm run dev # 开发模式；首次进「设置 → 本地语音识别模型」下载模型
npm run dist:win # 打包：release// 下出安装包 + win-unpacked 绿色目录
Environment requirements: Node.js ≥ 18 (20/22 LTS recommended), FFmpeg + ffprobe (automatically collected during packaging, or manually put in build/ffmpeg-bin/）、
ASR model (download in-app, or manually put in models/<模型id>/), disk ≥ 5GB.
The complete steps (including FFmpeg placement, model directory structure, packaging pitfalls, data directory, and troubleshooting) are atdocs/INSTALL.md。
8. Who is this design suitable for and who is not suitable for it?
Suitable: Batch slicing of dozens to hundreds of short and medium-sized videos; interviews, courses, and live broadcast playback that do not want to transfer the original videos to the cloud;
Teams that already have an OpenAI compatible interface (or native Ollama).
Not suitable: Scenes that require “real editing” capabilities such as multi-track timelines, transitions, and color correction – VClip only does “slicing” and does not do post-production.
What I want to do next: Vertical screen reconstruction (automatic cropping of the main body), batch generation of title/cover copy by slices, export queue breakpoint resume,
and put features.js That set of assertions is then extended to cover the boundaries of the derived combination.
9. Write at the end
The most valuable code in this project is not the AI-related parts, but the few places that “seem to just adjust the API”:
A custom protocol response header, a seek throttling, a duration threshold adaptation, and a scanning gate before packaging.
What they have in common is:It won’t report an error when you make a mistake, it will just say “it looks like it works” ——The video can be played but cannot be dragged, the subtitles are exported but misplaced,
The analysis is completed but not sliced, and the packaging is successful but the key is included. This type of problem can only be solved by “writing down the invariants first, and then nailing them with tests.”
If this content helps you avoid even one of these pitfalls, then the project will be worth it.
