PDA

View Full Version : Math for skin transfer?


SuperRune
11-29-2010, 11:38 AM
Hi there,

I am writing a script to transfer skin data from one mesh to another, and I hope the script can be used to transfer other data in the future (such as UVs). The math is causing me a headache though, I immediately start bumping into problems that are above my level (which is basic linear algebra).

Here's what I'm going to do:

- From a selected skin modifier, the script will save out the weight data as a text file. There will be one file per bone, with the position and weighting for each vertex.

- For the character that is to receive the new weighting, the file data will be read into arrays, and the script will interpolate new values based on the old data.

Now, the interpolation is what's causing me a headache. I don't want to just pick the nearest vertex and use its weight. I want to interpolate it properly, so that I can transfer between different topologies. After doodling on paper and prototyping in maxscript for two days, I see that point cloud interpolations are quite complex to solve. I hope someone here have ideas for a simple and good enough way of solving it.

Searches on Google suggests doing a full delaunay polygonization and go from there, but that's way beyond what I can do. Right now I'm using Shepards method, but it's not very accurate with points that are far away from the sampled data.

Does anyone here have some suggestions for a quick and dirty way of doing point cloud interpolation?

Here's a screengrab of my attempts at interpolation. The data is at left, two rows of five white and black points. In the middle is nearest neighbor, at the right is Shepards method. Notice how Shepards starts to fade to black above the white points, and to white under the black points - something that would have been a major problem had these been skin weights.

MatanH
11-29-2010, 03:30 PM
why don't you simply use the skinUitilities for that?
can you post the code for the Shepards method test?

eek
11-29-2010, 04:05 PM
why don't you simply use the skinUitilities for that?
can you post the code for the Shepards method test?

I concour,

Walk through each vertex, (selected or not) formating its id. Walk through its bones with:
skinOps.getVertexWeightCount <skin> <vertex>

appending the id of each bone(s) that the vertex uses to an array:
skinOp.getVertexWeightBoneID <skin> <vertex> <index>

and its weight for that vertex to an array:
skinOp.getVertexWeight <skin> <vertex> <index>


You'll end up with something like so:
vertex <array of bone ids> <array of weights>

1 #(1,2,3,4) #(.25,.25,.25,.25)


- As for doing it over different topologies id store the vertex position as well, and then do a closest match on the different mesh.

In any system you need a baseline reference to essentially find the skeleton for the mesh area/vertex etc..

If the models look the same but have different topologies - a better approach maybe skin wrap. Basically on the driven mesh find the closest 4 vertexes on the target mesh and find the average.

SuperRune
11-29-2010, 07:29 PM
I've actually never seen Skin Utilities before! The "match by face" option was greyed out when I tested it, so copying from one topology to another didn't work. I just got the default bind on the mesh.

Charles, those were the procedures I were planning on using. But since I mostly will be doing this across very different topologies, it's the interpolation math itself I need input on. Nearest neighbor will be my failsafe, but I would really really like to have proper interpolation in here.

Getting the average of the four closest vertices will not work, because those can be grouped together on one side of the point, and vertices that should be influencing from the other side will then be ignored.

Matan, here's part of my interpolation test script that have the code for the Shepard method. Just copy paste this into a script window to see the result.

function testPolate =
(
-- Set up some test values and a small bitmap to play with
testDataCoords = #( point2 5.0 18.0, point2 10.0 15.0, point2 20.0 13.0, point2 30.0 11.0, point2 45.0 11.0, \
point2 5.0 22.0, point2 10.0 22.0, point2 20.0 20.0, point2 30.0 19.0, point2 45.0 18.0 )
testDataValues = #(color 256 256 256, color 256 256 256, color 256 256 256, color 256 256 256, color 256 256 256, \
color 0 0 0, color 0 0 0, color 0 0 0, color 0 0 0, color 0 0 0, color 0 0 0)

-- Shepard's method
testMap3 = bitmap 50 50 hdr:true

for x = 0 to 49 do
(
for y = 0 to 49 do
(
local setColor = color 255 0 0
local thisCoord = point2 x y

local val1 = color 0 0 0
local val2 = 0.0

for c = 1 to testDataCoords.count do
(
local newDist = distance thisCoord testDataCoords[c]

local wi = pow (1 / newDist) 8

val1 = val1 + testDataValues[c] * wi
val2 = val2 + wi
)

setColor = val1 / val2
setPixels testMap3 thisCoord #(setColor*.75)
)
)

-- Draw original pixels on the maps
for c = 1 to testDataCoords.count do
(
setPixels testMap3 testDataCoords[c] #(testDataValues[c])
)

-- Display our result!
display testMap3
)

testPolate()

I'm now trying a new method where I pick the closest vertex in three cone subdivisions around the point, and then doing a linear interpolation between the resulting three vertices. I'll try that in 2D first, and see if I can extrapolate it out to 3D.

eek
11-30-2010, 04:19 AM
How different are the meshes and their topologies from one another? Are we talking big proportional changes?

SuperRune
11-30-2010, 07:14 AM
The proportions are more or less identical, the first skin weight I'm transferring is to the same set of bones. The topology is very different though, it has been subdivided one level and then remodeled.

SuperRune
11-30-2010, 09:47 AM
I seem to have some success with a hybrid of Shepards method and a subdivision method I have been thinking about.

Instead of searching for the nearest vertices, I'm picking the nearest inside three segments around the point. This result in everything from one to three hit points, which I then use Shepards method to weight. It's very crude, but I think the results are sufficient. At least it's better than just using nearest neighbor.

Here are the results from a black/white dataset and a debug coloured dataset:

PEN
11-30-2010, 12:07 PM
Use Skin Utilities as suggested or even Skin Wrap and then Convert To Skin.

SuperRune
11-30-2010, 12:42 PM
Converting a Skin Wrap does give quite good results on my test model. As I mentioned earlier, Skin Utilities doesn't work at all. I'll stick with Skin Wrap a while (converted though, pure Wrap is too slow), and continue with the script if Skin Wrap doesn't handle the more complex models.

I was halfway on the final script though, felt like a shame to "give up" :)

IkerCLoN
11-30-2010, 01:08 PM
SkinWrap is a life saver, but sometimes it behaves very weird. I found it useless in some situations where you can't make it work properly (it does not recognize some geometry on the model, and I swear to God the model was OK).

If I were you and had some spare time, Rune, I'd go ahead with the script ;)

SuperRune
11-30-2010, 01:21 PM
If I were you and had some spare time, Rune, I'd go ahead with the script ;)

Thanks Iker, I will try! I was planning on releasing it when it's done, that way everybody's happy :)

MatanH
11-30-2010, 02:17 PM
Hi,

I had some fun with your code and rearranged it in an interactive dialog.
press the left mouse button anywhere on the map to add a white point
and press then right mouse button to add a black point.

I added another variable named 'thresh' to the equation to help deal with the issue you had with everything becoming gray (averaged) when far away from any point.

I also added the 'trunk' variable to un contrast the interpolated points so that the original points will become visible.

I still need to think of a way to calculate the correct thresh value instead of inputing it by hand..


(
global roll_Test
try (destroyDialog roll_Test) catch ()
rollout roll_Test "Shepard's method test"
(
-- Local Structs
-----------------------------------------

struct s_Points (coord, val)

-- Local Variable Declerations
------------------------------------------

local w = 200
local h = 100
local testMap = bitmap w h hdr:true color:red

local Pts = #()

-- User Interface
------------------------------------------

imgTag imtTest bitmap:testMap align:#center transparent:blue
spinner spnTrunk "trunk:" range:[0, 1, 0.1] scale:0.1
spinner spnP "p:" range:[0.01, 100, 3] scale:1
spinner spnThresh "thresh:" range:[0, 100, 15] scale:1

-- Functions
------------------------------------------

fn getShortestDist Pts x =
(
local lastDist = distance Pts[1].coord x
for i = 2 to Pts.count do (
local newDist = distance Pts[i].coord x
if newDist < lastDist then
lastDist = newDist
)
lastDist
)

fn testPolate w h Pts p thresh trunk =
(
for y = 0 to h - 1 do (
local colors = #()

if Pts.count > 0 then (
for x = 0 to w - 1 do (
local setColor = color 255 0 0
local thisCoord = point2 x y

local val1 = color 0 0 0
local val2 = 0.0

local shortestDist = getShortestDist Pts thisCoord

for i = 1 to Pts.count do (
local newDist = distance thisCoord Pts[i].coord
newDist -= shortestDist - thresh

local wi = 1.0 / (pow newDist p)

val1 += Pts[i].val * wi
val2 += wi
)

setColor = val1 / val2
setColor = (1 - trunk) * setColor + 0.5 * trunk * white
append colors setColor
)

setPixels testMap [0,y] colors
)
)

for i = 1 to pts.count do
setPixels testMap Pts[i].coord #(Pts[i].val)

imtTest.bitmap = testMap
)

fn redrawMap =
(
testPolate w h Pts spnP.value spnThresh.value spnTrunk.value
)

fn openDialog =
(
createDialog roll_Test width:210
)

fn init =
(
redrawMap()
)

fn done =
(
-- cleanup code
gc light:true
)

-- Event Handlers
------------------------------------------

on imtTest lbuttonup pos flags do (
append Pts (s_Points coord:pos val:white)
redrawMap()
)
on imtTest rbuttonup pos flags do (
append Pts (s_Points coord:pos val:black)
redrawMap()
)
on spnP changed arg do redrawMap()
on spnThresh changed arg do redrawMap()
on spnTrunk changed arg do redrawMap()
on roll_Test open do init()
on roll_Test close do done()

)

roll_Test.openDialog()
)

Matan.

SuperRune
11-30-2010, 02:50 PM
Fun stuff, Matan!!

I'll see if I can put my new method into your script! It's got a couple of parameters that I think would be very interesting to interact with!

PEN
11-30-2010, 03:40 PM
Skin Utilities works great, you must just be doing something wrong with it.
Skin Wrap is a bit easier how ever but I have run into the very odd case where it does something strange.

MatanH
12-01-2010, 07:34 AM
I had some more fun with the ui, don't know why..
anyhow here it is:

(
global roll_Test
try (destroyDialog roll_Test) catch ()
rollout roll_Test "Shepard's method test"
(
-- Local Structs
-----------------------------------------

struct s_Points (coord, val)

-- Local Variable Declerations
------------------------------------------

local w = 200
local h = 100
local rad = 3
local testMap = bitmap w h hdr:true color:red

local Pts = #()

-- User Interface
------------------------------------------

progressBar pbrProg width:w height:10 align:#center
imgTag imtTest bitmap:testMap align:#center transparent:blue
button btnClear "Clear" width:w align:#center
radioButtons rbnMode "" labels:#("Add", "Edit")
spinner spnTrunk "trunk:" range:[0, 1, 0.3] scale:0.1
spinner spnP "p:" range:[0.01, 100, 3] scale:1
spinner spnThresh "thresh:" range:[0, 100, 15] scale:1

-- Functions
------------------------------------------

fn getClosestPt x =
(
local lastPoint = Pts[1]
local lastDist = distance lastPoint.coord x

for i = 2 to Pts.count do (
local newDist = distance Pts[i].coord x
if newDist < lastDist then (
lastDist = newDist
lastPoint = Pts[i]
)
)

lastPoint
)

fn getShortestDist x =
(
distance x (getClosestPt x).coord
)

fn movePointTo x =
(
if Pts.count > 0 then
(getClosestPt x).coord = x
)

fn delPoint x =
(
if Pts.count > 0 then
deleteItem Pts (findItem Pts (getClosestPt x))
)

fn setPt Pt rad mark:green =
(
for y = amax 0 (Pt.coord.y - rad) to amin (h - 1) (Pt.coord.y + rad) do (
local colors = #()
local startPt = undefined

for x = amax 0 (Pt.coord.x - rad) to amin (w - 1) (Pt.coord.x + rad) do (
local dist = distance Pt.coord [x,y]
if dist <= rad then (
local val = Pt.val
if dist < rad * 0.4 then
val = mark
append colors val
if startPt == undefined then
startPt = [x,y]
)
)

if startPt != undefined then
setPixels testMap startPt colors
)
)

fn testPolate w h Pts p thresh trunk =
(
if Pts.count > 0 then (
pbrProg.value = 0

for y = 0 to h - 1 do (
local colors = #()

for x = 0 to w - 1 do (
local setColor = color 255 0 0
local thisCoord = point2 x y

local val1 = color 0 0 0
local val2 = 0.0

local shortestDist = getShortestDist thisCoord

for i = 1 to Pts.count do (
local newDist = distance thisCoord Pts[i].coord
newDist += thresh - shortestDist

local wi = 1.0 / (pow newDist p)

val1 += Pts[i].val * wi
val2 += wi
)

setColor = val1 / val2
setColor = (1 - trunk) * setColor + 0.5 * trunk * white
append colors setColor
)

setPixels testMap [0,y] colors
pbrProg.value = 100.0 * y / (h - 1)
)
)

for Pt in Pts do
setPt Pt rad

imtTest.bitmap = testMap
)

fn redrawMap =
(
testPolate w h Pts spnP.value spnThresh.value spnTrunk.value
)

fn openDialog =
(
createDialog roll_Test width:(w + 10)
)

fn init =
(
Pts = #()
testMap = bitmap w h hdr:true color:red
imtTest.bitmap = testMap
)

fn done =
(
-- cleanup code
gc light:true
)

-- Event Handlers
------------------------------------------

on imtTest lbuttondown pos flags do (
case rbnMode.state of (
1: (
local Pt = s_Points coord:pos val:white
setPt Pt rad mark:red
append Pts Pt
)
2: setPt (getClosestPt pos) rad mark:red
)

imtTest.bitmap = testMap
)
on imtTest rbuttondown pos flags do (
case rbnMode.state of (
1: (
local Pt = s_Points coord:pos val:black
setPt Pt rad mark:red
append Pts Pt
)
2: setPt (getClosestPt pos) rad mark:red
)

imtTest.bitmap = testMap
)
on imtTest lbuttonup pos flags do (
case rbnMode.state of (
1: ()
2: movePointTo pos
)
redrawMap()
)
on imtTest rbuttonup pos flags do (
case rbnMode.state of (
1: ()
2: delPoint pos
)
redrawMap()
)
on btnClear pressed do init()
on spnP changed arg do redrawMap()
on spnThresh changed arg do redrawMap()
on spnTrunk changed arg do redrawMap()
on roll_Test open do init()
on roll_Test close do done()

)

roll_Test.openDialog()
)

now you can edit the points by selecting edit modeand then
left click will mode the closest point to the clicked position
right click will delete the closest point.

I also made the points bigger

Matan.

SuperRune
12-01-2010, 03:07 PM
Great stuff, Matan. This is really fun to play with :)

MatanH
12-01-2010, 03:42 PM
I added a checkbox next to the thresh parameter so that it will be clear how it effects then result.

(
global roll_Test
try (destroyDialog roll_Test) catch ()
rollout roll_Test "Shepard's method test"
(
-- Local Structs
-----------------------------------------

struct s_Points (coord, val)

-- Local Variable Declerations
------------------------------------------

local w = 200
local h = 100
local rad = 3
local testMap = bitmap w h hdr:true color:red

local Pts = #()

-- User Interface
------------------------------------------

progressBar pbrProg width:w height:10 align:#center
imgTag imtTest bitmap:testMap align:#center transparent:blue
button btnClear "Clear" width:w align:#center
radioButtons rbnMode "" labels:#("Add", "Edit")
spinner spnTrunk "trunk:" range:[0, 1, 0.3] scale:0.1
spinner spnP "p:" range:[0.01, 100, 3] scale:1
spinner spnThresh "thresh:" range:[0, 100, 40] scale:1
checkBox cbxUseThresh "" align:#left offset:[0,-22] checked:true

-- Functions
------------------------------------------

fn getClosestPt x =
(
local lastPoint = Pts[1]
local lastDist = distance lastPoint.coord x

for i = 2 to Pts.count do (
local newDist = distance Pts[i].coord x
if newDist < lastDist then (
lastDist = newDist
lastPoint = Pts[i]
)
)

lastPoint
)

fn getShortestDist x =
(
distance x (getClosestPt x).coord
)

fn movePointTo x =
(
if Pts.count > 0 then
(getClosestPt x).coord = x
)

fn delPoint x =
(
if Pts.count > 0 then
deleteItem Pts (findItem Pts (getClosestPt x))
)

fn setPt Pt rad mark:green =
(
for y = amax 0 (Pt.coord.y - rad) to amin (h - 1) (Pt.coord.y + rad) do (
local colors = #()
local startPt = undefined

for x = amax 0 (Pt.coord.x - rad) to amin (w - 1) (Pt.coord.x + rad) do (
local dist = distance Pt.coord [x,y]
if dist <= rad then (
local val = Pt.val
if dist < rad * 0.4 then
val = mark
append colors val
if startPt == undefined then
startPt = [x,y]
)
)

if startPt != undefined then
setPixels testMap startPt colors
)
)

fn testPolate Pts p thresh trunk =
(
if Pts.count == 0 then
testMap = bitmap w h hdr:true color:red
else (
pbrProg.value = 0

for y = 0 to h - 1 do (
local colors = #()

for x = 0 to w - 1 do (
local setColor = color 255 0 0
local thisCoord = point2 x y

local val1 = color 0 0 0
local val2 = 0.0

local shortestDist = getShortestDist thisCoord

for i = 1 to Pts.count do (
local newDist = distance thisCoord Pts[i].coord
if cbxUseThresh.checked then
newDist += thresh - shortestDist

local wi = 1.0 / (pow newDist p)

val1 += Pts[i].val * wi
val2 += wi
)

setColor = val1 / val2
setColor = (1 - trunk) * setColor + 0.5 * trunk * white
append colors setColor
)

setPixels testMap [0,y] colors
pbrProg.value = 100.0 * y / (h - 1)
)

for Pt in Pts do
setPt Pt rad
)

imtTest.bitmap = testMap
)

fn redrawMap =
(
testPolate Pts spnP.value spnThresh.value spnTrunk.value
)

fn openDialog =
(
createDialog roll_Test width:(w + 10)
)

fn init =
(
Pts = #()
redrawMap()
)

fn done =
(
-- cleanup code
gc light:true
)

-- Event Handlers
------------------------------------------

on imtTest lbuttondown pos flags do (
case rbnMode.state of (
1: (
local Pt = s_Points coord:pos val:white
setPt Pt rad mark:red
append Pts Pt
)
2: (
if Pts.count > 0 then
setPt (getClosestPt pos) rad mark:red
)
)

imtTest.bitmap = testMap
)
on imtTest rbuttondown pos flags do (
case rbnMode.state of (
1: (
local Pt = s_Points coord:pos val:black
setPt Pt rad mark:red
append Pts Pt
)
2: (
if Pts.count > 0 then
setPt (getClosestPt pos) rad mark:red
)
)

imtTest.bitmap = testMap
)
on imtTest lbuttonup pos flags do (
case rbnMode.state of (
1: ()
2: movePointTo pos
)
redrawMap()
)
on imtTest rbuttonup pos flags do (
case rbnMode.state of (
1: ()
2: delPoint pos
)
redrawMap()
)
on btnClear pressed do init()
on spnP changed arg do redrawMap()
on spnThresh changed arg do redrawMap()
on spnTrunk changed arg do redrawMap()
on cbxUseThresh changed arg do redrawMap()
on roll_Test open do init()
on roll_Test close do done()

)

roll_Test.openDialog()
)

denisT
12-02-2010, 12:11 AM
what the"Shepards method" is? can anyone give me a link to the method's math explanation?

MatanH
12-02-2010, 04:57 AM
http://en.wikipedia.org/wiki/Inverse_distance_weighting

CGTalk Moderation
12-02-2010, 04:57 AM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.