The second and third languages are generally more expensive options—that is, re-shooting the video or settling for a dub, where the voice and the mouth don't necessarily match up. Vidnoz provides two utilities for browsers to address both parts of the issue. This guide explains when to use each and what to get ready in advance, as well as common pitfalls that lead to poor generations.
Two Tools, Two Starting Points
The first question you must ask is what you already own between the two.
- The talking photo AI tool allows you to get a presenter without any camera, if you can provide a photo and a text.
- The lip sync AI tool is used to re-match the speaker's mouth to a new audio track if you already have a video and a new voice track.
There is no software to install and both are free to try out first. In this workflow, you convert a photo into a presenter.This workflow transforms a photo into a presenter.
First, create the image to be used.
The root cause of most issues is the source picture. The tool requires a clear face that is well-lit, neutral expression, and a closed mouth. Faces must be accompanied by nothing that covers the face (sunglasses, hand, heavy shadow, etc.) and all files must be less than 10MB. A passport-type picture taken in front of a window will typically look better than a dramatic, angled portrait.
You can also upload from a text description and create an image, or choose from presets avatars. The page indicates that cartoon and animal graphics can be used, provided that a face is identifiable.
Create for people who can't see.
Talking photo scripts are short and to the point. The free flow is limited to a max of one minute which is suitable for a greeting, product announcement or lesson opener. Divide lengthy ideas into multiple short clips rather than a single long talk.
Select voice and delivery
The voice library contains over 2000 voices, which can be narrowed down by language, gender, emotion and scenario. You can adjust the speed, pitch and volume of the speech, and even preview it before generating, saving credits and time. For those wanting to record using their own voice, you can record directly in the browser, upload a file or clone your voice.
Pick the look
There are two kinds of motion settings: light body movements and full-body motion. The portrait can be transformed into looks like cartoon or painting style with optional style filters and subtitles can be turned on. The green screen, transparent or white background exports are helpful if you are wanting to put the presenter in a larger edit as it leaves only the avatar.
In this second workflow, the text is added to an existing video.This second workflow is used to add text to an already existing video.
Use clean source footage to start.Use clean source footage as a start.
The lip sync AI works best if the speaker's face can be seen. Upload a video up to 500MB in size in the following formats: MP4, MOV, M4V or WEBM. You will be required to verify that the video is yours.
Ensure that there is a backup recording of the audio.
The new track can be MP3, WAV, M4A or MP4, up to 50MB. Localization quality is determined in the sound part. Speak in a native language or with a good quality voice and try to use a similar speed as the original text. It is more difficult to match length differences if they are large.
Generate and review
Click generate and see the output. On the free tier, only the first 60 seconds of a clip will be processed, so try out a clip of some length before you make a full-length clip. It is assumed that the speaker's expression and body language stay as recorded and the mouth follows the new language.
A simple checklist for localizing.
1. Edit the video source to a short, concise portion that is easy to understand.
2. Record or make the translated sound file and play it twice.
3. Upload both and create a test clip.
4. Practice pronunciation of names, numbers and longer words.
Only then, process the full video or upgrade for longer files.
What to Watch Out For
Both work on a person's face, but are different in that one is likeness and the other is consent. Use your own pictures and video or pictures and video that you have explicit permission to use.
Disclosure. When using AI-generated presenter in public-facing content, mention that the presenter is AI-generated. It is a way to gain trust and prevent confusion.
Unsupported subjects. Animals and cartoon characters might not be supported, according to the lip sync page. For that tool, use real people and realistic avatars.
Normally there are free tier ceilings, length caps and free daily allowances. If you must work on a deadline, plan for them.
Who Gets the Most Value
Teachers and Course creators can create short narrated introductions, and translate existing lessons. There are several ways that small businesses can produce product explainers without the services of a voice actor or presenter for multiple languages. A single video clip can be tweaked for different regions on social media. In all cases, the shortest and least complex inputs will produce the best outcomes, not the longest and most complicated projects.
Final Thoughts
These tools are not intended to replace an expert video production for serious campaigns, but they will eliminate a lot of hassle during regular video production. Always begin with a small number of photos, a brief script, and a short test clip. If the output fits your requirements, build up slowly so as to scale; always preview the end product video of the target language before it goes public.
1 thought on “How to Localize Short Videos and Animate Portraits: A Practical Guide to Vidnoz’s Photo and Lip Sync Tools”