EbSynth

Overview of EbSynth

EbSynth is a software that allows users to transform video by painting over a single frame. It uses neural networks and optical flow algorithms to synthesize stylized animations from normal footage. EbSynth takes a reference video and one or more "keyframe" images as input. The keyframes contain the desired artistic style painted by the user. EbSynth then transfers the style from the keyframes to the entire video while preserving the motion and contents of the original footage.

Key Features

• Transform video into animation, paintings, sketches, etc. by providing artistic keyframes
• Preserve motion and content of original video
• Add more keyframes to improve quality over time
• Support for textures, strokes, transparency, and more
• Fast processing powered by neural networks
• Free beta version available

Use Cases

Here are some common use cases for EbSynth:

• Convert normal videos into animated cartoons or paintings
• Create stylistic VFX like painted fire, lightning, smoke, etc.
• Add a unified artistic look and feel to a video scene
• Integrate 2D animation with live action footage
• Change the mood and aesthetics of a video with different art styles
• Speed up production by reducing amount of drawing/painting needed

Workflow

The general workflow for using EbSynth is:

1. Import reference video file
2. Import one keyframe image painted in desired art style
3. Adjust settings like mapping and de-flicker
4. Click synthesize to generate output video
5. Review results and paint additional keyframes where needed
6. Export final stylized video

Advanced settings allow adjusting parameters like diversity and GPU usage. Multiple keyframes can be blended for smoother results. The software supports common image and video formats like PNG, JPG, MP4, etc.

Tips for Best Results

Follow these tips when working with EbSynth:

• Use keyframes with same resolution as reference video
• Start with diffuse lighting when filming reference footage
• Add keyframes on problem frames appearing in initial results
• Use visually rich textures and colors for better tracking
• Increase mapping for stylistic coherence, decrease for more variation
• Adjust de-flicker higher for temporal stability, lower for independent frame styles

Exporting and Sharing

EbSynth exports image sequences which can be assembled into videos using editing software like:

• Adobe After Effects
• Adobe Photoshop
• Blender

Finished videos can be shared on sites like YouTube, Vimeo, Instagram, etc. Images can be shared on art communities and portfolio sites.

Community and Support

Resources for learning more about EbSynth:

• Video tutorials on EbSynth YouTube channel
• Tips and troubleshooting on EbSynth FAQ page
• Example artworks in EbSynth Discord community
• Updates and announcements via EbSynth Twitter

Paid professional support will be offered in the future. For now, beta users can seek help on Discord or by emailing the EbSynth team.

Developers and Technology

EbSynth was created by a team of AI researchers and engineers at Adobe Research:

• Scott Cohen
• Bei Xiao
• Zeyu Jin
• Gautham Mandyam
• Matt Levine
• Corneliu Florea
• Roger Blum

It combines several key technologies:

• Neural style transfer - stylize images using deep learning
• Optical flow - track movement between video frames
• PatchMatch - efficiently match content between keyframes and video
• Motion processing - handle complex motions like occlusion

These allow transforming video in a wide variety of artistic styles quickly and automatically.

Licensing and Availability

The EbSynth beta is available for free download on ebsynth.com. It can be used free for commercial and non-commercial purposes.

A paid professional version is under development. It will add features like:

• Support for common video formats
• GPU accelerated processing
• Advanced motion controls
• Resolution independence
• Timeline previews

This will allow production level workflows. Pricing and availability is to be announced.