Object rotates after reset xform - Inverse matrix error?


I have an issue where the inverse matrix function seems to cause a problem with reset xform. The object will rotate when the reset xform is collapsed.

I’ve created a test case based on one of the matrices that seems to be problematic. I’m creating thousands of meshes using this basic technique and it works without issue 98% of the time. This is the matrix of one of the problem cases. The matrix is created by crossing two orthogonal edges of a polygon. The matrix isn’t perfect, but the error seems to be tiny. I’ve tried to orhogonalize the matrix, but it makes no difference.

The error in rotation after reset xform is huge compared to whatever error was in the matrix. Is there a way to fix this?


thebox = box size:100
thebox.name = "box"

theboxortho = copy thebox
theboxortho.name = "box ortho"

thematrix = matrix3 [-0.000001525883,1.000000000000,0.000000000000] [0.000000000000,0.000000000000,1.000000000000] [1.000000000000,0.000001525883,0.000000000000] [0,0,0]

debugpoint = point size:25 box:false axistripod:true cross:false centermarker:false
debugpoint.transform = thematrix
debugpoint.wirecolor = green
debugpoint.name = "point - thematrix"

thematrixortho = thematrix
orthogonalize thematrixortho

format "thematrix       = %\n" (thematrix)
format "thematrix ortho = %\n" (thematrixortho)

format "(thematrix == themtarixortho) = %\n" (thematrix == thematrixortho)

thematrixinverse = inverseHighPrecision thematrix
thematrixorthoinverse = inverseHighPrecision thematrixortho

thebox.transform = thematrixinverse
theboxortho.transform = thematrixorthoinverse

-- this works fine, so it's the inverse of the matrix that becomes a problem with reset xform
--thebox.transform = thematrix
--theboxortho.transform = thematrixortho

theboxprereset = copy thebox
theboxprereset.name = "box pre reset xform"
theboxorthoprereset = copy theboxortho
theboxorthoprereset.name = "box ortho pre reset xform"

resetxform thebox
collapsestack thebox

resetxform theboxortho
collapsestack theboxortho


I don’t understand the goal of all these transformations.
lets start from beginning.
what do you want to do with the matrix which slightly rotated (original thematrix)?
and what is the final goal?


This is an isolated example which illustrates the error. This error only pops up about 2% of the time and there isn’t a clear pattern as to why. This matrix is one that I was able to capture as being a problem case.

The tool where I’m using this technique optimizes geometry by combining faces. I do this by creating a shape around the perimeter of the faces and converting it to an edit poly. The issue is that max only correctly polygonalizes shapes in the viewport in which they were created, or in the case of a programatically created shape, in the XY plane.

This is the basic technique, which you were a great help with:


If you run that technique on a mesh with many faces in arbitrary positions and orientations, you will get the error I’ve isolated above eventually and consistently.

I have tried many combinations of constructing my own matrix to avoid using matrixfromnormal, thinking that might be the issue, but it’s not.

The question is: why does resetxform cause the object to rotate so much? The matrix has been orthogonalized. It’s not a perfect matrix, but the error is tiny, in the 6th decimal place. Why does that cause a rotation error of several degrees?


as I can see here and in Area you only need to extrude shape in direction of its normal…

probably everything can be done cleaner and easier, but here is a plan:

fn make_poly_face_tm node face pos:#center = 
	vv = polyop.getfaceverts node face
	v0 = polyop.getvert node vv[1]
	v1 = polyop.getvert node vv[2]
	v2 = polyop.getvert node vv[vv.count]
	front = normalize (v1 - v0)
	side = normalize (v2 - v0)
	up = normalize (cross front side)
	side = normalize (cross up front)
	pos = case pos of
		#center: polyop.getfacecenter node face
		 #first: v0
		default: pos as point3
	tm = matrix3 front side up pos

fn get_poly_spline node face transform:on = 
	ee = polyop.getedgesusingface node face	
	polyop.createShape node ee name:("pshape_" + face as string)
	shape = shapes[shapes.count]
	if transform do
		p = shape.center
		tm = make_poly_face_tm node face
		xtm = tm * inverse shape.transform
		shape.transform *= inverse tm
		worldAlignPivot shape
		CenterPivot shape
		resetXform shape
		converttosplineshape shape
		shape.transform = tm
		shape.center = p

fn extrude_shape shape amount convert:on = 
	addmodifier shape (extrude amount:amount)
	if convert do converttosplineshape shape

fn extrude_polyshape node: face: amount:10 = 
	if node == unsupplied do node = selection[1]
	if iskindof node Editable_Poly do
		if face == unsupplied do 
			if node.selectedfaces.count > 0 do face = node.selectedfaces[1].index
		if iskindof face Number do
			sh = get_poly_spline node face
			extrude_shape sh amount

so make a poly object, select a face your need, and run:



Thank you for your response! I have tried your script on a box and it works fine. On a sphere, there are problems:

Anyway, I already have that part of my script working well.

The issue I’m having is with the transform technique that you showed me a while back. If you run it on enough cases(for example: lots of spheres with many faces and random positions), you will get a case like the one in the top of this thread, and the new polygon won’t align with the original one. You will get large gaps where the new polygons don’t fit like the original one did. It looks like this latest sample you provided uses the same technique to get a matrix. That problem vector from my test case above was created in the exact same way as your sample–by crossing two edges, though I only use edges that are perfectly perpendicular. I think on a sphere they are not, which is why that sample doesn’t work on a sphere. I have also tried creating a transform with 2 of three vectors created with cross product, which should be perfectly orthogonal. That also eventually will become a problem for resetxform.

That technique only seems to work 98% of the time, because I believe there is an error in resetxform. That’s the only issue I can’t solve. I will try doing the worldalignpivot and centerpivot; those are steps I have not tried.

I will try your latest code on the polygons which created the problem transform above and see if I get the same result as my code.

Thank you for your input.


If you don’t need the node to be aligned with the face normal, but extruded in the direction of the face normal you could also convert the shape to poly and extrude the face/s.

Or simply detach the faces and extrude them.


As I can see in your example, the problem may be due to float precision, as not all the components of the matrix are normalized.

If so, you could try to implement a custom routine to normalize the vectors using doubles and then manually create the matrix.


I actually don’t need extrusion at all. That was just a nice visual way to demonstrate that shape objects don’t convert to polygons correctly unless they’re built on the xy plane(at least in script). If the extrusion isn’t perpendicular to the face, converting a complex shape to a polygon isn’t going to work correctly either–it will create all kinds of overlapping and inverted faces. That’s why the shape needs to be transformed to the xy plane, converted to a polygon, and then transformed back to the original position. I don’t care what the orientation of the object is, just that they line up perfectly.

I’ll look into this.


I tried this, but I think it’s a dead end. Max will just round the number as soon as it’s cast into the matrix and it ends up being the same.


Using the inverse transform and resetxform was too unreliable.

The solution I came up with is:

  1. make a copy of the spline
  2. transform it to the XY plane
  3. resetxform on it (now it will extrude/polygonize correctly)
  4. copy the vertex positions from the original, untransformed spline to the transformed one
  5. convert to edit poly
  6. delete the original spline