[SDK] SubObjectHitTest and MNMesh

[Serejah]

Hi.

I made a function to hit test subobjects under the cursor, but for unknown reason it behaves very strange on poly mesh, but only in MNedge and MNFace mode. As if it only do hit tests for the last subobject in mesh.
I mean only the last edge and last face return a value. But MNVert mode works as expected and every vertex near the mouse cursor returns a value. Is there anything else I need to do to make it work?
I looked at sdk sources but couldn’t find anything that I would miss…

MNEdge mode test:

Here’s the code I use to hit test editable mesh nodes.
The only difference between MNMesh version is that IObject casts to TriObject instead of IPolyObject.

public static void GetMeshSubObjHit( int x, int y )
{
	var ip = GlobalInterface.Instance;

	var node = ip.COREInterface.GetSelNode( 0 );
	var obj = node.EvalWorldState( ip.COREInterface.Time, true ).Obj;
	var mesh = ((ITriObject)obj).Mesh_; // this is the only difference between mesh and poly methods

	IHitRegion     hr = ip.HitRegion.Create();
	ISubObjHitList hl = ip.SubObjHitList.Create();            
	IIPoint2       pt = ip.CPInitPos_.Add();
	pt.X = x;
	pt.Y = y;

	int crossing        = 1;
	int POINT_RGN       = 0x0001;
	uint GW_PICK        = 0x0001000;
	uint GW_ILLUM       = 0x0000002;
	uint SUBHIT_EDGES = 1 << 26;

	ip.MakeHitRegion( hr, POINT_RGN, crossing, 4, pt );

	var gw = ip.COREInterface.ActiveViewExp.Gw;
	gw.RndLimits = (gw.RndLimits | GW_PICK) & ~GW_ILLUM;
	gw.ClearHitCode();
	gw.SetHitRegion( hr );
	gw.Transform = node.GetObjectTM( ip.COREInterface.Time, ip.Interval.Create() );


	bool has_hit = mesh.SubObjectHitTest( gw, gw.Material, hr, SUBHIT_EDGES, hl, 1 );

	if ( !has_hit ) return;


	if ( gw.CheckHitCode )
		debugPrint( ">> hit " );                       
	
	debugPrint( $"edge index:{hl.First.Index + 1}" );

}

upd
I just found out that it is possible to get the subobject index using ip->COREInterface->SubObHitTest, but unfortunately it requires that the object must be selected and the subobjectlevel should be active to get edge/face indexes.

[Serejah]

Ok, here we go again.
Can somebody explain why is it also not working? Only the last edge in poly mesh gets hit.
ps. tested on x2014

def_visible_primitive(GetPolySubObjHit, "GetPolySubObjHit");
Value* GetPolySubObjHit_cf(Value** arg_list, int count )
{
	// <node>inode, <int>x, <int>y
	check_arg_count(GetPolySubObjHit, 3, count);

	ReferenceTarget* owner = NULL;
	INode* inode = arg_list[0]->to_node();
	MNMesh* poly = _get_polyForValue(arg_list[0], MESH_READ_ACCESS, &owner, _T("Expected Editable Poly as first argument"));

	if (!poly) return &undefined;

	Interval valid;
	int savedLimits, crossing = 0;
	Interface* ip = GetCOREInterface();
	
	GraphicsWindow *gw = ip->GetActiveViewExp().getGW();
	HitRegion hr;

	IPoint2 p( arg_list[1]->to_int(), arg_list[2]->to_int() );
			
	MakeHitRegion( hr, POINT_RGN, crossing, 4, &p );
	gw->setHitRegion(&hr);
	Matrix3 mat = inode->GetObjectTM( ip->GetTime() );
	gw->setTransform(mat);	
	gw->setRndLimits(((savedLimits = gw->getRndLimits()) | GW_PICK) & ~GW_ILLUM);
	gw->clearHitCode();

	SubObjHitList hitList;

	int result = poly->SubObjectHitTest( gw, gw->getMaterial(), &hr, SUBHIT_MNEDGES, hitList );

	gw->setRndLimits(savedLimits);

	if ( !result ) return &undefined;

	float index = static_cast<float>(hitList.First()->index + 1);
	return new Point3Value( index, hitList.First()->dist, 0.0f );
}

mxs

try (destroydialog X ) catch ()
rollout X "SubObjHit" width:220 (

	checkbutton ply "poly" width:90 across:2
	checkbutton msh "mesh" width:90 checked:true 
	timer t interval:50 active:true
	
	on t tick do
	(		
		if selection.count == 0 do return ok
		clearListener()

		if ply.checked and isKindOf $ Editable_Poly do
		(
			local result = GetPolySubObjHit $ mouse.pos.x mouse.pos.y
                        format ">> %\n" result
		)
		
		if msh.checked and isKindOf $ Editable_mesh do
		(
			-- GetMeshSubObjHit $ mouse.pos.x mouse.pos.y
		)			
	)

	on ply changed state do
	(
		msh.checked = not state
	)
	
	on msh changed state do
	(
		ply.checked = not state
	)
)
createDialog X

[Klvnk]

your script doesn’t match the c++ function (no node)

[Serejah]

Thanks, fixed.
But the problem isn’t in mxs code.

[Serejah]

Had to call SubObjectHitTest on trimesh to make it work as expected.
Is there any workaround to get the correct poly edge without so much hassle?

      // ...
      //int result = poly->SubObjectHitTest( gw, gw->getMaterial(), &hr, SUBHIT_MNEDGES, hitList );
	
	Mesh tri;
	poly->OutToTri( tri );
	int result = tri.SubObjectHitTest(gw,gw->getMaterial(),&hr,SUBHIT_EDGES,hitList);

	gw->setRndLimits(savedLimits);

	if ( !result ) return &undefined;
	
	int idx = hitList.First()->index;
	int edge_index_in_face = idx % 3;
	int face_index = idx / 3;

	int v1 = tri.faces[ face_index ].v[edge_index_in_face];
	int v2 = tri.faces[ face_index ].v[(edge_index_in_face+1)%3];	

	int* vedg =  poly->vedg[v1].Addr(0);
	int poly_edge_index = -1;
	for ( int i = 0; i < poly->vedg[ v1 ].Count(); i++)
	{
		if ( poly->e[ vedg[i] ].v1 == v1 && poly->e[ vedg[i] ].v2 == v2 || poly->e[ vedg[i] ].v1 == v2 && poly->e[ vedg[i] ].v2 == v1 )
		{
			poly_edge_index = vedg[i];
			break;
		}
	}
		
	return new Point4Value( 
		static_cast<float>(poly_edge_index + 1),
		static_cast<float>(v1 + 1),
		static_cast<float>(v2 + 1),
		hitList.First()->dist 
	);
}

[Klvnk]

I know it’s no help but i gave up developing for MNMesh in the sdk as it was too flakey and unreliable.

[Serejah]

Found working alternative. Requires subobject level to be active in order to work.

if (hit = (dotNetClass "Autodesk.Max.GlobalInterface").Instance.CoreInterface.ActiveViewExp.SubObjHitList.First) != undefined then 1 + hit.HitInfo else 0
 

.
upd
to make MNMesh .SubObjectHitTest work correctly it is enough to check that hitlist.first isn’t NULL and get the info about the hit from there.
How could I miss that

// polyedit.cpp
//finally try to hit faces...
	if(testTris)
	{

		faceRes = pMesh->SubObjectHitTest(gw, gw->getMaterial(), &hr, hitFlag|SUBHIT_MNFACES, faceHitList);
		if(faceHitList.First())
		{
			hitMade = TRUE;
			faceClosest = ::GetClosestHitRec(faceHitList);
		}
	}