-
Notifications
You must be signed in to change notification settings - Fork 1
/
msLipSyncReadme.txt
196 lines (172 loc) · 7.1 KB
/
msLipSyncReadme.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
Files
=====
tool\msLipSync.lua - the main code for the lipSync script
utility\msDialog.lua - dialog utility routines
utility\msHelper.lua - debugging routines
utility\msPhonemes.lua - routines for breaking text into phonemes
utility\lipSync.txt - bone information for lip sync
msLipSyncReadme.txt - this file
Installation
============
- Copy msLipSync.lua into the script\tool\ directory of your choice.
- Copy the utility\*.* files into the script\utility directory
It's recommended that you copy these files to the script directory in your user
content location.
Functionality
=============
The script allows the user to specify a phrase in a dialog box, as well
as a starting and ending frame. It will then create keyframes for the
character, matching the phrase entered by the user.
Usage
=====
This video shows the basics of using the script (original)
https://www.youtube.com/watch?v=kwfEie-RSe8&list=PLMDdsoe58WVPCRcjRs8RUitA5_fvsBnir
A video showing the updates for version 2.0 is at
https://youtu.be/uBFCzb6awGE
The script is named Lip Sync. In order to select the script you
must select either a switch layer, that controls the standard mouth
shapes, or a bone layer that controls mouth movements. If you are
not on one of those layers the script will not be enabled.
After you have selected a mouth switch layer or bone layer, the Lip Sync
tool will be enabled. Next, move the timeline marker to the starting point
of the audio you want to lip sync. Now, move the cursor to the workspace
and drag until the timeline marker is moved to the place where you want
the lip sync to end. (Note: You need to drag in the workspace area, because
the scripting does not allow you to drag within the timeline ares.)
When you have finished dragging the mouse, let the mouse button up and
a dialog will come up. The start frame and end frame will automatically
be populated by your mouse drag. Next enter the text for the audio.
The dialog allows you to select Phonetic spelling and Debug.
When you select ok the script will add key frames to the layer you
have selected. It will not delete existing key frames.
Intention of the Script
=======================
While the script does an excellent job spacing out phonemes, the
intention is not specifically to get a perfect match. Rather the
intention is to get the proper sequence of phonemes in the correct
order and close to the correct location. You can then tweak the
speech by dragging key frames as normal.
Consonants, Vowels and Held Tones
=================================
The script recognizes that, in general, many consonants cannot be held.
For example, in the word, "cat", "c" and "t" cannot really be held a
long time. By default, those kinds of letters are only given one
frame. The script does allow you to specify the length of the
consonant sounds.
Vowels, on the other hand can easily be held for a long time. As
a result the script will assign all the short consonants the specified
short time, subtract them from the total number of frames and then divide the
result by the number of vowels. This creates a phoneme spacing that
matches the speech in most cases.
If your speech has certain sounds
that are held inordinantly long you simply need to list those sounds
multiple times. For example "the faaat cat" would hold the "a" sound,
in "fat", three times longer than the other vowels in the phrase. The
script also recognizes that certain consonants, like "n" and "m" can
be held, so they act like vowels.
Non-letters
===========
Punctuation and numbers should not cause the script to crash, but the
symbols will be unrecognized and will, in most cases, result in an
"etc" phoneme.
Phonetic or not
===============
The script does a fair job at converting basic English phrases into the
appropriate phonemes, but using phonetic spelling matches the phonemes
exactly to what you want. You can even use it to match foreign language
by and non-language, just by selecting whatever phonetic symbol you
want. The exact phonetic mapping is given at the end of this file.
Debug
=====
You can select Debug if something goes wrong in the lip sync. With
Debug selected, statements will be printed to hopefully help fix the
issue. Note, since I do not earn my income from the script I cannot
support problems that might occur, but hopefully the Debug option can
help if you encounter a problem.
Switch Layers
=============
If you are using switch layers for lip sync, it needs to have the
standard mouth shapes. It can have other shapes, and they can be in
any order, but it must have the shapes AI, E, L, FV, etc, MBP, O, U,
WQ, rest. Note that capitalization does matter.
Bone Layers
===========
If you are using bones to control your mouth, the dialog will show a
selection box where you will choose the appropriate bone map to use
with your character. You can modify the file Utility\lipSync.txt to
lip sync any bones controlling you character's speech.
Bone Maps
=========
A bone map tells the script what angles to set for which bones in order
to produce the mouth look you want for any of the standard mouth
phonemes. You can have any number of bones controlling your mouth. Bone
maps that come with this script include Scarlett (used with Scarlett Riggs),
Bigsby, Dragon, Gorilla, and Gruille.
Creating Your Own Bone Maps
===========================
Creating your own bone maps is easy, but it follows very specific
rules. Failure to follow the rules will cause the script to break.
The easiest way to create a bone map is to copy an existing one and
modify it. You don't add a new file, you just insert another map into
the same file. The format of the map is
mapName numberOfBones
boneName1 boneName2 ... boneNameN
phoneme boneAngle1 boneAngle2 ... boneAngleN
Rules of the Bone Maps
======================
- The last line in the file must be the word "end"
- mapName and boneNames must be a single word
- the mapName can be anything, but the boneNames must match real bones
in the character
- / is allowed in names, but other characters might cause the script to
break
- capitalization matters for bone names
- you need to include bone angles for each phoneme
- capitalization matters for phoneme names
- bone angles need to be in degrees
A Word of Advice About Creating Your Own Maps
=============================================
Before you create your own bone map, study the ones that come with the
script. Next create a very simple map with a single bone. If a bone map
doesn't seem to work, try to simplify it and test with a very simple
phrase.
Phonetic Spelling
=================
The following list shows the phonetic letter and the phoneme that will
be created
a = AI
b = MBP
c = etc
d = etc
e = E
f = FV
g = etc
h = etc
i = E
j = etc
k = etc
l = L
m = MBP
n = etc
o = O
p = MBP
q = WQ
r = etc
s = etc
t = etc
u = U
v = FV
w = WQ
x = etc
y = etc
z = etc
A = AI
E = E
I = AI
U = U
O = O
R = O
- = rest
Notice that in many cases the long and short vowel have the same mouth
phoneme. That just makes it easier to specify the spelling. Notice
also, that you may specify "-" so signify the "rest" mouth position.