Usage in Blender
Download iOS app and PC app
Download apps from following url
YouTube Tutorial
How to use
1. Press the «Open Advanced Setting» button on the PC version of iFacialMocap.
2. Switch Software Name to «Blender». Then switch the Blender version.
3. Press the Addon button.
4. Open the sample scene (iFacialMocapTestObj.blend).
5. Install iFacialMocap_to_Blender.py.
6. Press the «N» key on your keyboard to open the «iFacialMocap Panel» at the top right of the Blender viewport.
7. Press the «waiting for connection» button.
8. Start the smartphone app and prepare.
9. Press the «Connect to Blender» button.
10. The connection is now complete.
11. When you long press the screen of the SmartPhone App, the 3DCG character’s face will face forward.
12. If you create up to 52 shapeKeys according to the sample scene naming rules, you can move the original character.
If you want to move the shape keys of multiple objects
By specifying the name of the group that contains the object, you can search for and move the shape keys of all objects, such as children and grandchildren, within the group hierarchy.
(As the number of ShapeKey nodes increases, the operation becomes heavier, so if you want to make the operation faster, it is better to combine the meshes as much as possible and use it)
How to use «Bake Recorded Animation «
If you bake while animating a character in Blender in real time, the processing load of Blender will increase and the motion may shift.
Press the “REC” button in the iOS app to record, then use the “Bake Recorded Animation” button in the PC app to receive and bake the motion in non-real time for better baking.You can bake slowly but accurately.
To bake a recorded animation, follow these steps:
1. Record the animation on your iPhone.
Then stop recording.
2. Press the «waiting for connection» button in Blender.
3. Press the «Bake Recorded Animation» button on the PC side
Do not press the «Connect to Blender» button.
Источник
Update Info
2021/02/21
・ IOS version (Version 1.1.5) released
You can now change the language in the app.
The lookForward button is now accepted as an external command.
2021/01/21
・ PC version (Version 1.12) released
Fixed a bug in the Blender add-on.
2021/01/10
・ IOS version (Version 1.13) released
We can now use a value greater than or equal to 1.0 for the maximum blendshape value.
・ PC version (Version 1.11) released
We can now use a value of 1.0 or higher (100 or higher for Unity) for the maximum value (max) of blendshape.
Updated the contents of the Blender add-on a little.
Fixed a bug inside the Unity script.
・PC version (Version 1.10)
Fixed a problem that the delay became large in Version 1.09
・PC version (Version 1.09) ＆ iOS version (Version1.1.2) released
Fixed a minor bug around the connection of Bake Recorded Animation. Added a mode for automatic IP address detection. Improved operating performance on Macs and laptops.
・PC version (Version 1.08) released
Version 2.9x add-ons for Blender are available. Also, the add-on has been updated a bit, and until now it was not possible to put it into the non-standby state without pressing the Esc key, but it can now be changed to the non-standby state from the button operation.
・IOS version (Version 1.08) released
You may be able to avoid the problem by connecting to the smartphone via TCP instead of UDP.
In addition, a function to adjust the BlendSHape weight and a function to remove noise have been added.
・PC version (Version 1.07) released
An item called «iOS Connection Type» has been added to address connection problems in some environments.
・PC version (Version 1.06) released
Fixed an issue where the settings in StandAlone_Unity mode could not be saved.
Fixed an issue where the light was backlit in StandAlone_Unity mode. Also, when right-clicking on the screen, a menu is displayed so that you can adjust the angle and intensity of the light.
We have made minor adjustments to the behavior on the Mac.
Mobile version (Version 1.0.7) is now available.
When recording animation in REC mode, audio is now recorded. Audio files are saved in a file folder and can be sent to a PC by email or other means.
Mobile version (Version 1.0.6) is now available.
Head movement values are now sent.
We changed the way the head rotates slightly.
2020/06/21
The PC version (Version 1.05β) is now available.
・Added a mode called StandAlone_Unity and added support for reading VRM models. You can now use a single script for Unity.
・ The script used in Unity was previously divided into multiple scripts, but now it’s a single script. You can handle it in the same way as in Maya and Blender mode by specifying a name, and you can use it in the same way as in Maya and Blender mode. It is also compatible with the previous individual attaching scripts. It is now compatible with the previous individually attached scripts.
・You can now enter a hot key. You can specify a blendshape that you don’t want to move while pressing a key. You can specify how many seconds after a key is pressed to max out the weight, or how many seconds after the key is released.
・It’s only for Windows, but it has an automatic update notification function. When it starts after a new version has been released, an update notification is displayed, and when you accept it, it will be automatic. The new version of the DL will begin.
・ You can now save the settings one by one for each software. You will be able to see that. Also, the buttons such as the Save Setting have been fixed at the bottom.
・The default settings of the Spine and Neck axis in Unity did not match with Humanoid’s. So, I fixed it to match with Humanoid.
・The performance of Maya has decreased after version 1.03 to 1.04. The problem was fixed.
・ Fixed a problem where the Save Setting did not work properly on Mac.
2020/04/05
・ SmartPhone version (Version 1.0.3) & PC version (Version 1.04) released
Before ver1.03, if the baking process was animated inside Maya or Blender, the processing became heavy and it was difficult to bake correctly. To solve the problem, we have added a function to record animation in the smartphone app and receive the animation slowly.
In Blender, after loading the latest version of the add-on, pressing «waiting for connection» and then pressing the «Bake Recorded Animation» button, the animation recorded in the smartphone is baked on Blender. In Maya, if you press the «Bake Recorded Animation» button after pressing the «wait connection» button, baking is performed in the same way.
Also, it is now possible to export animated face objects in FBX Ascii format. For software such as Blender that does not support FBX Ascii, export to FBX2012, use the FBX Converter provided by Autodesk for free, convert it to FBX binary format, and then read it.
The processing in Blender is a heavy problem, but it happened before ver1.03 because animation was poured at 60 FPS. From ver1.04, the initial value in Blender version was set to 20FPS and the operation was a little lighter.
・ SmartPhone version (Version 1.0.2) released
In version 1.02, it can be used with mosaic applied.
The function to turn the character to the front can now be used with the button.
Added a function to zero the weights around the eyes and around the mouth at once.
・ PC version (Version 1.0.3) released
Added the function to link the movement of the neck (Neck) and the spine (Spine) with the movement of the head.
In Maya and Blender, enter the bone name in the GUI and it will work.
You can enter multiple object names separated by commas.
Unity works by assigning scripts to the neck and spine.
The interlocking ratio can be changed by the slider for each axis.
Added eyebrow mesh to Blender and Unity sample scenes.
Reduce the number of polygons in Blender sample scene.
A scroll bar has been added to the GUI.
BlendShape and ShapeKey names can be changed individually.
Example) «eyeBlinkLeft»-> «eyeBlink_L»
PC version (Version 1.02) released
The amount of movement of the eyeball can now be adjusted by individual weight values up, down, left, and right.
Until Version 1.01, the weight of each BlendShape and ShapeKey ranged from 0 to 1.
From Version1.02, the weight range has been extended from 0 to 100.
If the weight of the expression that is difficult to respond is set to 1.0 or more, the sensitivity of the reaction will increase.
-«3DCG Software IP Address» and «3DCG Software Port» are provided.
By changing this IP address, you can transfer data received from Windows to Mac etc.
-An item called debugMode has been added.
When debugMode is turned ON, an error message will be displayed on the screen.
To be released in January 2020
PC version (Version1.01) released
[Changes in Blender]
In Version1.00, there was a problem that only one ShapeKey node to be moved could be specified.
In Version1.01, if one group of objects is specified, all the ShapeKey nodes of the objects in that group will be searched and moved.
Commentary video about the update on YouTube (Japanese Language Only)
[Changes in Unity]
In Version 1.00, UnityRecieve_iFacialMocap.cs can be attached to only one object, and there was a problem that BlendShape of multiple objects could not be moved.
In Verison1.01, if you attach UnityRecieve_iFacialMocap.cs only once to a node that contains multiple objects, you will be able to move the BlendShape of objects that are children or grandchildren of that node.
Commentary video about the update on YouTube(Japanese Language Only)
[Changes in Maya]
There is no particular change for Maya.
2020/1/14
The smartphone application version (Version 1.0.1) has been released.
The English version of the homepage URL was incorrectly displayed on the AppStore.
2020/1/13
The smartphone application version (Version 1.0.0) & PC version (Version 1.00) has been released.
Источник
iFacialMocap 4+
FacialMocap & Send to PC
Yasushi Emoto
Designed for iPad
-
- #118 in Productivity
-
- 3.0 • 5 Ratings
-
- £6.99
Screenshots
Description
Motion capture of facial expressions & send to PC
iFacialMocap allows you to capture facial expressions in an iOS app and communicate in real time with 3DCG software on your PC. You can also do an FBX export by email.PC-side software allows you to import VRM models.
To export FBX, turn on the REC mode and record. And when you press the mail button, the version selection screen of FBX appears, and when you press the OK button, the FBX file is saved in the File folder and the mailer starts.
Detailed usage is available at the following URL:
https://ifacialmocap.com/
This app can create face animations by communicating with Maya, Unity and Blender in real time. In addition, animations created in real time can be baked.
This app can load VRM avatars within the iOS app. In addition, this app is linked with VMagicMirror on the PC and VRM related apps such as Luppet and HANA_APP.
To use, first start this app, then start «iFacialMocap to SoftWare app» on your PC and enter your iPhone’s IP address.
(Donwload from PC use :https://ifacialmocap.com/download/
)
Next, create up to 52 morph targets (BlendShape or ShapeKey) according to the naming convention.
Finally, use 3DCG software to load the script / add-on and press the connection button.
Very accurate facial motion capture can be performed very easily.
This app is only available for models equipped with FaceID. The camera image and resulting depth data are only used to send motion capture data to the user’s PC. That data is not stored in the iPhone. To get depth data from Apple’s API, access to the camera is required.
This application is only used for communication with PCs within the same network and does not send information to third parties.
This app can only be used on iOS devices equipped with FaceID.
Supported devices are as follows：
・iPhone
iPhone 11 Pro Max
iPhone 11 Pro
iPhone 11
iPhone XS Max
iPhone XS
iPhone XR
iPhone X
・iPad
iPad Pro 12.9-inch (4th generation)
iPad Pro 12.9-inch (3rd generation)
iPad Pro 11-inch (2nd generation)
iPad Pro 11-inch
* Face ID is not installed in the new iPhone SE 2nd generation released in 2020, but you can use iFacial Mocap by updating to iOS 14 or later. Not limited to iPhone SE 2nd generation, iFacial Mocap can be used if it has an A12 or higher chip and has been updated to iOS14.
Источник
