Add live face capture to avatars in real-time.
Record, review, edit facecap or export to Blender.
Compatible with all Apple ARKit blendshapes.
AR head tracker (tabletop head tracking) mode.
Embedded Files
blendshape_retargeting.mp4A new Blendshape Retargeting menu has been added that allows animators to use custom blendshapes on any avatar with any blendshape names, even while facecap is connected and running you can adjust the mapping! Then save the retargeting maps as a .json file so you can reuse the mapping later.
Note blendshape nams order does not matter when using the retargeting since the mapping is a purely blendshape names-to-name mapping.
It should be possible to quickly apply the saved retargeting files on any number of avatars that share the same blendshape names!!
Download the Sidekick Addon for Blender:
Download the Sidekick Addon for Blender:
Download the LATEST Sidekick Addon For Blender v9.5.4c includes the new Blendshape Re-Targeting menu
Or download the legacy Sidekick Addon For Blender version v7.0.4
Recording Facecapture
Recording Facecapture
When the Sidekick addon is installed in Blender you can record facecaputre from a IPhone using the Sidekick app to record face capture directly on avatars in Blender.
The "Sync Audio" checkbox allows recording facecap aligned with audio clips in Blender's Timeline.
It can be challenging to start speaking exactly as the audio clip begins, which may cause noticeable lip-sync issues initially,
This video shows how lip-sync issues can be fixed using the Graph Editor.
SidekickAudioSync.mp4Install Instructions For Blender
Install Instructions For Blender
Installing the SidekickBlenderAddon.py add-on script is the same as any Blender Addon, simply by selecting the .py from the install menu in Blender.
Activate the add-on in Blender by opening Preferences>Add-Ons and selecting the SidekickBlenderAddon.py that was downloaded above.
Ensure the checkbox is enabled for the APS Sidekick Addon.
Locate the menu bar for the Sidekick app that includes the connectivity, retargeting and recording options.
More Information
Truedepth sensor required.
iPhone X or higher is required when installing the APS Sidekick Face Capture App
Sidekick App
Sidekick App
App for iOS devices
Vive Pro Eye
Vive Pro Eye
Vive Face Tracking
See Also:
iOS Sidekick facecap with UE live-link plugin:
iOS Sidekick facecap with UE live-link plugin:
Unreal Engine live production.
See Also:
AR Sidekick video recording:
AR Sidekick video recording:
Blender3D live action camera.
Google Sites
Report abuse