VTuber Room

VTuber Final.mp4

Live streamers and VTuber can create custom scenes with the screen alignment and avatar spawn points setup for transparent overlay mode so the avatar is always located at one corner of the desktop screen. A custom desk model can be used in place of the desk model in this example.

Using the APS SDK's includes example scenes creators can quickly adapted the "VTuber Desk" example scene to their own custom scenes. This tutorial will focus on the "VTuber Desk" example scene.

I used a background video from the voice converter tutorial to create this.
You can simply use a .mp4 on the desktop as the background, Or even youtubes!!

Tying everything together into a short production, this video was rushed but shows many of the new features and capabilities.

VTuber Desk.mp4

The Stream Deck was added and can be accessed from the Streams panel. These act as external controller panels and can be opened in a web browser or even on a tablet!

VTuber mode is a feature that allows players to easily create live-streams using custom avatars on the desktop overlays. When the VTuber mode is enabled avatars and props appear as transparent overlays superimposed on the PC desktop.
A VTuber room is a custom scene which has been setup and includes a VTuberDesktopCamera (a camera and screen) prefab and at least one spawn point. The VTuberDesktopCamera prefab makes it very simple to create a custom scene where the avatar will always align to the spot on the streamer's desktop whenever the room is loaded in VR. In the following example rooms the first shows a very simple "standing only" room with three spawn points, the second room includes a desk mesh and shows a "seated" style VTuber room.
Multiplayer is supported and allows for any number of other players to join a single VTuber room and create collaborative streams. More about multiplayer VTuber mode further down.

Scene Controls

Desktop View - The PC desktop in VR
VTuber Mode - Transparent Avatar Overlay
Display - Change desktop on vtuber screen
Reset - Places the player at the desk
Lock Hand - Avatar hand(s) stay in place
Invisible - Stop rendering on desktop
Share Desktop - Multiplayers can see desktop
Quality - Image compression amount

A new VTuber remote was also just added.

Look for this in the props list. This allows controlling the vtuber scene, avatar transitions, background media and avatar visibility when holding the controller.

The vtuber stream panel just got overhauled and now includes a bunch of new options:

Mouse enabled mouse control on the main desktop.
Media play or pause active media (simultates keyboard FN play button).
Invisible hides the main avatar from the desktop screen

Desktop View is simply a mesh plane for the computer monitor model on the desk, it displays the user's PC desktop to the VR player in the scene.
VTuber Mode button enables the transparent overlay mode so that avatars will appear as transparent overlays on the desktop. Clicking the button again disables the vtuber mode making the game window visible on the desktop.
Display buttons allow selecting the active display that is shown on the vtuber screen (the large panel in the scene).
Reset resets the main play to the desk, facing forward and directly over the spawn point.
Lock Hand buttons allow pinning the avatars hands in the scene, this is useful when wanting the avatar's hands to be always over the keyboard model to avoid tracking errors that would cause the avatar to twist when controller visibility is lost.
Invisible allows hiding everything from the main PC desktop when in transparent overlay mode. When enabled all avatars and props will not be visible on stream.
Share Desktop is only for multiplayer and only available for host of the room. When this option is enabled the vtuber screen panel will be shared with all other players in the room, they will see the same vtuber screen as the host but with reduced resolution and only four frames per second update rate, but should be enough that other players are spatially aware of what's on the desktop and can interact during stream!
Quality sets the compression amount of the shared desktop stream when using multiplayer, lower values may result in better performance at a cost of image clarity for the other clients.

Multiple Monitor Support

This allows optionally assigning each of the monitors to a unique virtual texture to emulate dual-head and even triple-head displays in VR.

When adding a SceneItemDesignator to meshes you can specify a Monitor Screen Number.

You can change the display number of the monitor texture

Vtuber Room

The APS SDK includes example scene templates for creating VTuber Rooms with desktop view panel and the player's spawn points.

The VTuberDesktopCamera prefab has been added.

The VTuberControls prefab has been added.

Three SpawnPoint prefabs have been added.

"Standing" Example

Download and import the APS SDK into a new Unity project (2019.4.40f1) and locate the APS_SDK>Example Scenes folder and open the VTuberRoom scene.

Screen Position Target

The screen target is the point where the VTuber screen aligns to, it's the projected point from the VTuberDesktopCamera that the screen will intersect. The target should be placed about one meter in front of where the player will stand.

Setting The Target

Screen panel can also be placed behind spawn points if you'd like the desktop panel to be behind the player when streaming.

Align The Screen Height

The screen panel should be aligned so that the bottom corner of the panel is just above the ground and in front of the spawn points.

Camera Distance (FOV)

The camera target should be far away from the player to produce a "long-shot" effect, in the example scenes the camera is 60 meters away from the player and a field of view (FOV) of only 6.5 degrees.

Adding Spawn Points

Multiple Spawn Points

Any number of spawn points can be added, they are selected at random. Multiple spawn points are useful for multiplayer rooms, while only one spawn pont is recommended for seated rooms.

The SceneItemDesignator is located in the APS SDK, adding this component to any gameobject in the scene and setting it as "Spawn Point" and the avatar will appear at this location. There is also a SpawnPoint prefab in the SDK.

Building The Scene

Then build the scene asset to a .scene file. First save the scene, Then open the APS_SDK menu and locate the Scene Asset Builder.

Click on Build Current Scene to create the scene file.

Camera Sliding

A important feature is the ability to move the desk from one corner to the other quickly while streaming. This can be accomplished by "sliding" the view from left to right by holding the middle mouse button and dragging side to side over the game window (do this only while transparent mode is off).

The way it works is the VTuberDesktopCamera we added to the scene earlier acts as a regular scene camera except a few of the constraint options in the SceneItemDesignator component have been enabled that allow the user to only drag the camera sideways, and not rotate the camera!

Check back soon for more updates!

Thanks for reading ❤️

Google Sites

Report abuse