View Full Version : In-game model problem

11-30-2006, 05:53 AM
Hey all,

This might seem like a rather stupid question, probably one that's already been asked numerous times before, but i did search, and nothing i read seems to help. There's two parts to this thread, first is a question, and the second is a problem i've encountered which leads me to ask the question in the first place.

What are some of the required things to check for when it comes to low poly models for in-game use. I know i've read about the evils of non-planar faces, and penetrating geometry in game engines, but what else can cause problems with a model?
I've already gone through the "Game modeling - Rules of the trade" thread.

Ok, here's the problem i've encountered. I ramble on a bit:

I'm incharge of building a creature model (along with normal, colour and spec map) for a mod group to use in their game engine . I created the model in maya - as i always do - layed out the UVs using headus' UVLayout tool and some uv tweaking again in maya. Checked the model using maya's Polygon>cleanup tool and checked for any non-planar faces. A whole bunch of faces came up. I manually fixed any major non-planars before hand so i assumed maya was just being fuzzy (some of the other poly game models i've looked at by other artists also show up as having nonplanar faces, according to maya).
I exported the monster and imported it into Zbrush for detailing. It's around here were some of the problems began. Stored the cage as a morph target, subdivided a couple of times and began detailing. All looked good and decided to generate a test normal map to see how everything was going (first time using normal maps). Zmapper gave me some odd artifacts on the tangent normal map so i used zbrush's tools>normalmap>Create NormalMap tool and got some clean results. Send that off to the guy in charge of code (normal map and model in .tga and obj format respectively) to see how it'll look (he had to convert the .obj into a .smd first).
Some weird issues arose on the model's surface with the existing normal map. Odd black squares were present at different angles of light.

He told me he'd never experienced any problems with any of the (doom3) normal maps he'd tested on the engine. Naturally we both assumed it was something to do with zbrush and the normal map so i got into some investigating and i came up with others who'd experience issues with zbrush and normal mapping. I asked one of the team's other modellers to see what he used (since i was told he too used zbrush and his normal maps "worked"). He gave me a link to a free normal software tool called xNormals ( (it generates normal maps using a lowpoly and highpoly mesh). I tested it out and generated a new normal map with relative ease. Send that off to be tested again, but still the same problem occured, not so bad, or apparent, but still it was there at certain angles.
Naturally i kept trying different settings and testing different normal maps but none of them seemed to solve the problem. Ultimately the coder told me that it's my model.

After the initial shock wore off, i opened the model in maya to investigate. The only problem i could think of was the (very slightly) non-planar faces that maya had bitched about earlier. So, i used the cleanup tool and used maya to fix them (it just divided the quads into triangles). Used the cleaup tool again to search for any other problems (non-planar, faces with holes, concave faces etc), nothing, mesh was clean. So i exported it again (.obj) and send it off to be tested (the original maps still worked fine on the clean model).
Despite all this, there were still the same problems ingame. No change had occured. BY this stage i'm completely stumped.

I have not idea what's wrong, whether it's the model, the engine, or the maps.
If it is the model then there's something i'm over looking, which lead me to my question (at the start).

The game engine being used it called Darkplaces. I'm using maya 6.5 and the latest version of Zbrush.

11-30-2006, 06:16 AM
One of the things we do at work before I rig up a model is to export the finished mesh as an .obj with everything unchecked except for smoothing. We then open up a new scene and reimport the .obj model into this scene. I rig THIS model because .obj import in a fresh scene ensures that there is nothing in the HyperGraph except for a single "Mesh" node. No extra junk nodes Maya tends to leave behind.

As far as problems with the mesh, the .obj import will also reveal things like negative scale on geometry, something that will screw you up later on. Those areas that have negative scale will show up as black polygons.

The other thing that may be screwing you up are the normals. Maya's way of doing Harden/Soften normals can be a pain and it's certainly done better in Max's Smoothing Groups, however, you can work around this by reseting all the edges to be "all hard" then "all soft", then put back your particular hard edges you want. Now, if you plan on bringing it into Zbrush, don't mess with the normals yet.

Turn on "Backface culling" and check that the faces are all pointing in the right direction.

Artifacting in the Normal Map can be a couple of things, but one thing that will screw it up are vertices that are on top of each other. A good tool to use to check this it the .fbx exporter. If you export the mesh as .fbx, it'll tell you if verticles are unwelded. Another thing to check is the "ray distance" in the Advanced settings in Zmapper. This is the length variable that determines how far from the lowpoly surface Zbrush will reach out for the high res detail. The problem with subdiving a lowres "game" mesh is that volume tends to "thin out" at higher subdivions due to the lack of polys that are "holding" the form of certain areas like an arm or leg. Often times, this is why artists "rebuild" the lowpoly mesh around the highpoly one. If the lowpoly and highpoly mesh's are too far away from each other in certain areas, you won't get a decent normal map no matter how much you mess with it.

That's all I got for now. :)


11-30-2006, 06:30 AM
Thanks for the reply podman,

I'll try unchecking those options on export. I remember on the first export send to the coder had problems with smoothing (basically lack of it). If i remember correctly he told me it was something he did by mistake when converting it, never brought it up again so i'm guessing that's not the issue.
I tried opening the original exported .obj and did find some junk in there.
Never heard of negative scale before, can you explain or provide a link to somewhere that explains it?
I made sure all the normals are lined up from the word go.

Yeh, i thought that was why the artifacts were showing up, but by this stage i'm pretty sure the normal map is not the problem.

I'll try your suggestions, Cheers ;)

11-30-2006, 06:32 AM
What I don't understand is your programmer friend using doom 3 normal maps, they can only be used in the doom 3 engine as I understand.

If you can provide some screen grabs :)

11-30-2006, 06:34 AM
What I don't understand is your programmer friend using doom 3 normal maps, they can only be used in the doom 3 engine as I understand.

If you can provide some screen grabs :)
Well i think that's what he said, i don't understand it either. Screen shots of the problem? I'll ask the coder if that's Ok just in case.

11-30-2006, 07:40 AM
These are the last screenshots taken ingame, apart from the crappy color map and seam problems you can see some dark squares around sections of the model.
It's a little more apparent in this angle ^. You can see areas where the shadow isn't consistend, it just stops (like where the leg and foot meet - funnily enough thats where there's a seam).
Supposedly they get worse at different angles, although i haven't seen it first hand.

11-30-2006, 07:53 AM
Don't know if this might help or not but this is an earlier in-game test (before normal mapping):
You can see black squares on the model for some reason, there was also no smoothing. The coder told me that he did something wrong on converting the .obj and that fixed the smoothing issue.

11-30-2006, 01:51 PM
hmmm. That certainly looks like a "normals" issue to me. I've gotten crap like that in UnrealEngine 2. The issue for me was that I was using the "Average Normals" command on rows of faces(Trying to simulate "smoothing groups" at the time.) Lots of game engines don't like this Maya command. I would export the whole mesh as .obj with everything "unchecked", including smoothing and normals. Then import into a new scene(the model should be completly faceted.) and marquee select all edges and do a full Normals>All Soft. From there, do your hard edges and when done, delete history. That's what eventually fixed it for me. Only select "edges" when invoking the normals soften/harden command. Seems to do the trick.

One other thing, you might want to try and "triangulate" the mesh in Maya before export. Sometimes "ingame" engine's triangulation is bad and you've got to do it yourself.


12-01-2006, 10:48 AM
Thanks podman,
Sounds a little odd but i'll give it a shot. I'll check-in with how it goes.

12-02-2006, 06:07 AM
Ok sorry for the delay but i seem to have stumbled onto a problem trying out the_podman's suggestion.
I exported the .obj mesh with all options on OFF (including smoothing and normals). Imported this faceted clean version into maya again selected all edges (and edges only) and applied allsoft. Then i selected the edges i wanted to be hard and applied normals >all hard, then exported it again as a .obj with smoothing ON.
Problem is on opening/importing the smoothed (.obj) version this appears:
All the normals are lined up so it's not a normal reversal thing. Just to make sure i hadn't accidently selected some unwanted edges i broke the proccess into these steps:

1) Imported clean faceted .obj model, selected all edges and applied normals>allsoft. Exported as "model(smoothed)01.obj" with only smoothing turned ON.
2) Imported "model(smoothed)01.obj" into a new scene. Everything looks fine, no problems like in the image above. Selected only a few of the edges that had to be hardend (claw nails) and applied normals>allhard. Exported as "model(smoothed)02.obj" with smoothing turned ON.
3) Imported "model(smoothed)02.obj" into new scene. Problems appear as in the image above. I'm not quite sure why

I'm 100% sure i'm not selecting unwanted edges by accident.
The original version i worked on (where i made all the necessary edges allhard in one go) only had one visible face with this problem. The step by step version where only portions of the allhardening were done has larger portions affect.
Any ideas?

12-02-2006, 06:33 AM
Nevermind, i fixed that problem. Now to re-uv map (i made some alterations to the model thinking it was something else) and draw up a quick normal map in zbrush before testing it again.

12-04-2006, 05:50 AM
Ok, so i tried pod_man's suggestion but all that seems to happen now is the whole model goes black.
Coder says: "all the faces are inverted inside". Not too sure what that means.
It only happens with real-time lighting. Not sure if that's a clue.
I couldn't get a concrete answer on whether or not the same problem was there without the normal map (and real-time lighting). By this stage - as one can understand - the coder doesn't want to do anymore tests on this model seeing as it's not likely too go into the demo.
I did use Zbrush to generate the last test normal map which in retrospect may of been a mistake (should've used "xNormals"), but i guess there's not much that can be done about that now.

Still i have no idea what's causing this problem, whether it be the model, normal map or something else. I have a suspicion it's the normal map.

12-04-2006, 08:26 AM
hmmm. I'm pretty much tapped out of ideas. The fix I gave you is pretty much the cleanest way I know of to fix "problem" meshes. Have you tried the .fbx export to see if errors are generated?

The only other thing I can think of is to import into Max(if you've got access to it) and try exporting from there. Not sure if that will do much, though.
Occassionally, I ran into stuff like the faces on your last post. Whenever I get those, I usually delete them and reappend them. One of Maya's strengths as a modeler is also one of it's weakness' and that's the ability to create really strange polygons.

Do you have multiple UV sets?

12-04-2006, 08:45 AM
Hey pod_man, thanks again,
I'm starting to wonder wether i followed your instructions correctly. To get rid of the odd faces i imported the clean faceted .obj version and saved it as a new maya scene before re-exporting as a .ojb with smoothing. Didn't think that extra step would do anything bad. Still don't think it would affect it.

I tried exporting the model as a .fbx and got this:

[WARNING] Geometry conversion - When 'Export hard edges' is activated, the vertices on exported geometry are duplicated so that the correct number of normals can be defined. If you want to keep the geometry with the original number of vertices, disable this option.

I'm tryingto get my hands on the latest 3ds max, hopefully that will solve any problems with the mesh (if it is the mesh).

12-04-2006, 08:25 PM
I thought you solved your problem.
Ok I don't use Maya however long time ago I had that kind of problem too. Everything looked good but in HL2 source I had similar problems. The problem was in normals (not the normal maps).Try to unify them in Maya ( this is how it's done in Max) then export it as usually you do and it should work.
Hope it helps.

______________________ (

12-05-2006, 03:00 PM
Hey Vlad,
Unify normals? Not too sure what that does or what the maya equivalent is. Haven't been able to find anything similar in the normals menu yet.
All the normals on the model are facing the same (out) direction.

-edit: Ok, did some research, sounds like "unify normals" does the same thing as "conform" in maya. I made sure previously that all the normals are facing the same direction (out).

12-05-2006, 06:49 PM
Yeah, I believe we had already tried to "reset" those normals. That is why I was suggesting to bring the .obj into another package like Max or even try to bring it into Wings3d which is free to download and export from there. Reason being is that Maya may be writing the .obj in a weird way(different softwares write the .obj slighty different from each other.) and that by exporting from another package may resolve the problem. The .fbx error you got is normal and you can ignore that. Motionbuilder uses a realtime game engine for it's OpenGL and it's just the way the engine simulates hard edges.

12-06-2006, 06:07 AM
I got my hands on part of the engine to do my own testing on so now i can be more thorough. I just need some software that converts to smd format (outside of maya would be good). Downloaded one ( maya (v4-7) but have yet to get it to work in v8 (i originally used v6.5 but that seems to have died after installing v8).

I've been using headus UVLayout to export the UVmapped version but i haven't tried using it to export a smoothed UVmapped version (not even sure if it preserved smoothing). I got wing3D too, so i can try using that to export a version of the smoothed-uvmapped model. Still i need to be be able to convert it to hl2 format (outside of maya would be good).:curious:

12-06-2006, 06:59 AM
Heh, I'm sort of completly rusty with Maya :)

there you go

______________________ (

12-06-2006, 09:31 PM
Worst problem ever :). I get things like this every once in a while. First thing I would suggest is merging all verts on .0001. Then go to display options(in the attrib editor) and turn off double sided then start looking for any duped faces by deleting a face here and a face there. If you don't find any, then make a cube in your scene, combine it with your model, delete history and run extract on your model. This usually fixes all my normal problems if exporting as an .obj doesn't do it.

Good luck with the maya voodoo:)


12-06-2006, 11:22 PM
I have had a few weird modelling problems like what you've got here. In the past a sure fire way to fix it for me was to delete the faces in the problem areas (for instances where your inexplicable hard edges were after smoothing) and rebuild them. It's annoying to put those faces back into the UV layout, but not too problematic as the areas are usually very small. More often than not I find that the problem was multiple faces occupying exactly the same space (still not sure how I've ever done that), but sometimes it's just...something.

I don't have a really good grasp on what your problem is exactly, but you can almost never go wrong in completely rebuilding areas that have problems.

12-07-2006, 01:05 AM
Just remembered. Try to aplpy a checker map and scale it up a bit to see whether you have a correct layout. If not select those polygons break them in your UV-map modifier, place the vertices in the correct order and weld it back.

______________________ (

12-07-2006, 06:04 AM
Thanks guys,
Lots of possible solutions there, i'll be sure to try each of them.
Problem right now however is that i can't convert .obj's into hl2 format to do any testing with the engine.
I can't seem to get the maya one working and i haven't been able to find a standalone application that converts to smd.

-edit:Seeing as the newer version of the model didn't fix anything, i've reverted back to it to do testing. I applied a checkered map as Vlad suggested. No layout problems, just some bad seams. :rolleyes:

12-13-2006, 12:10 AM
Did you try combining it with a cube and then separating it yet? It makes a new shape node for the object (I believe) and *usually* fixes my normal problems.

12-13-2006, 03:31 AM
Hey tofugorilla,.
I always merge vertices & merger multiple edges after combining both halfs of the mesh.
Edit Poly>Extract-ing the new combined mesh seperates all the faces into individual parts. Not sure if that's what you were suggesting.

When i first encountered the faces in the problem area (during the smoothing process, prior to re-uvmapping after finishing modifications to the mesh) i went back and rebuilt the faces in question. Despite this the same problem kept coming up during smoothing.
I was able to get around the problem by importing the clean unsmoothed .obj and saving it as a new clean maya scene to do the smoothing then reexporting it with smoothing (no other junk is exported). That stoppe the face smoothing problems in those areas.

Like i said before, currently i'm unable to convert to hl2 format through maya (plugin doesn't seem to be working) to do any testing on the engine, so i'm waiting to get my hands on the latest 3Ds max to do it through there. Unfortunately that won't be for another week or so. :(

12-13-2006, 08:29 AM
No, it's not what I was saying :) I know it sounds bazaar but try following these steps:

1) Make a poly cube in the scene
2) Select the bad poly mesh and then the poly cube
3) Polygons>Combine
4) Delete history
5) Polygon>Extract (w/ "Separate Extracted Faces" checked)
6) Select bad mesh and delete history
7) Edit Polygons>Normals>Set to face

I know, it's weird and it makes little sense, but it's saved me at least 50 models and countless hours of remodeling.


12-17-2006, 08:25 AM
No, it's not what I was saying :) I know it sounds bazaar but try following these steps:

1) Make a poly cube in the scene
2) Select the bad poly mesh and then the poly cube
3) Polygons>Combine
4) Delete history
5) Polygon>Extract (w/ "Separate Extracted Faces" checked)
6) Select bad mesh and delete history
7) Edit Polygons>Normals>Set to face

I know, it's weird and it makes little sense, but it's saved me at least 50 models and countless hours of remodeling.

-TOk, i followed your instructions (Sorry for the delay), however step 5 (with "Seperate Extracted Faces" checked) seperates all the faces on the combined (character + cube) mesh. You didn't mention combining all the extracted faces again (afterwards) so i'm guessing this isn't what's meant to happen.

I've downloaded 3Dsmax 9 (trial) and downloaded the 3dsmax SMD exporter ( from here. ( However, it seems that that exporter isn't written for v9. :banghead: (i know i know i should've checked). Anyone know if there's a v9 (32bit) compatible SMD exporter? Guess i'll have to start looking for versions 6 or 7.

CGTalk Moderation
12-17-2006, 08:25 AM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.