Realflow-like liquids with nParticles

[alexx]

hmm. to me nParticles used for intercolliding water seems slow.
meshing even slower.

that could be because of the missing multi threading for sure :)

cheers

alex

[KidderD]

From what I've played with so far, the liquid particles should have the self collide off (faster) but also play with the nucleus force fields, there is a self attract, that makes a huge difference. As well with everything Maya , there is a lot of tweaking. radius scales, mesh tri size.

[alexx]

with self collide off you can not e.g. "fill" a glass.
sure it is faster with that turned off.

since it is the threads title i am comparing it to realflow so far :)

cheers

alex

[elvis75k]

Well, here it is, i knew it.. the missing multi threading is a bad news.. yes.
I hope is not that slow, i mean is slow? how much?!
Compared to the fluid bomb-a-tomb presets on a 32 bit machine?! It's hard to do a benchmark without tweaking i know but, how many days you need to post some cool video (or still)?
The self attract wow.. i can't stay away longer from this release!!

Also, can anybody see if texturing the output mesh, maybe based on the age/speed is easy to do or need some mel script? I'm getting too much curious, forgive me!

[KidderD]

with self collide off you can not e.g. "fill" a glass.
sure it is faster with that turned off.

since it is the threads title i am comparing it to realflow so far :)

cheers

alex

I'm pretty sure I filled. The SPH solver uses the incompressibility parameter to allow the filling of objects. Now, I still wasn't able to get the look I would like to make it fill like water, but this is nParticle 1.0, and I'm sure there are many tricks I don't know about.

[alexx]

i am quite sure i checked that.
without the self collide you can not fill.

you can get the water-flow look which is nice, but no filling.

cheers

alex

about the speed.. hmm. i just did some small tests. the simulation time i would say is about 1/4 of realflow, the meshing time 1/10 of realflow.

please note that this is 100% subjective impression from my side :)

and yes: it is nParticles 1.0 :) i know.

[KidderD]

Here is a screen, with self collide off. Now, I didn't get enough, particles to fill, but it appears to be stacking without self collide. Which you would think the SPH solver would be handling this.

[Tobbe]

If you have Liquid simulation enabled they do collide, but not at the particle "surface" as with self collide.

/Tobbe

[Ronald]

Selfcollision is definetly off. the selfcollision you need for the liquid is controlled via the fluid attributes (incompressibility, rest density and liquid radius). still, liquid sims seem to be kinda odd and slow (I am used to realflow, which does a pretty good job - then again, this is already at version 4). And yes, meshing is even slower and the fluidy quality of the mesh is not very convincing - it does look very much like a simple blobby representation with no options for things like compression and surface tension (motion streak parameter seems to make little cylindrical blobbies based on speed). Also, it's not possible to cache the liquid mesh which makes it a lot more difficult to preview.

i don't mean to bash the liquid simulation (or nparticles in general), but from a production point of view, this is something I could only use for a very limited amount of tasks.

I hope the maya dev team gets the chance (from AD) to bring nparticles to production level.

r

[Aikiman]

Can we have some screen shots please?! Showing some basic settings and times would be good in shaded mode maybe one with a water shader on it too. They really need to multi thread this stuff, its a bit of a disappointment it appears to be so slow, I was hoping they would speed up nCloth too but it looks as though that is still single thread, im getting old just talking about it!

[alexx]

Here is a screen, with self collide off. Now, I didn't get enough, particles to fill, but it appears to be stacking without self collide. Which you would think the SPH solver would be handling this.

stop emission, wait several frames and check your result with a meshing.

it wont fill up for sure. it will only look very very strange :)

cheers

alex

[chrisbyk]

I just tried a small water test, and everything was fine in the particle simulation. Then I cached the simulation by creating a new nCache. No problem. The next step was to convert the particles into polygons, and tweak the look a little bit. Fine, but then the simulation becomes very slow again, so I tried to "Geometry Cache" the polySurfaceShape, wich seems to work fine too. I finally tried to playback the whole test, but my geometry cache is messed up, and I keep getting an error saying ".../.../.../polySurfaceShape1` may have an invalid cache description or invalid, missing or empty cache file. // .
Any idea about how to proceed correctly?

[Tobbe]

I'm not sure you can cache a surface with changing topology... might be the problem.

Tobbe

[chrisbyk]

You might be right Tobbe. I thought that the "one file per frame" in the cache options would do the trick. If not, is there another way to do it?

[Bonedaddy]

Been playing around with it for a half hour or so, initial thoughts:

Holy crap is it slow. The single-threading on the simulation hurt, but was doable. The meshing, on the other hand, is so slow as to be unusable. Given that meshing is also single-threaded in RealFlow, I am wondering what the holdup is. I worry that it's using some sort of brute-force method to detect nearest neighbors for meshing, as opposed to caching it somehow.
The MOST damning thing about this is that they provided no smart methods to cache animated meshes, eliminating the possibility of motion blur, and rendering all meshing functionally useless (except for particle renders). The problem is that the topology changes on a frame-by-frame basis, and there's no way to determine how fast one vertex is moving, since it may not even be there on the next frame. That's why the geometry caching doesn't work.
The mesh object has no UVs, at least by default. I don't expect there will be any good way to get UVs on this, aside from hijacking some PP parameters, so I didn't poke around too much.
That said, the actual simulation does look nice, and it interacts with the rest of the stuff in Maya with the agility one would expect. It certainly isn't useless, it just isn't nearly as useful as it should be.
I've been thinking over workarounds, and here's what I've come up with:

You could sim your nParticles in Maya, then convert them to Realflow's BIN format and mesh them there, or run it through one of Realflow's rendertime solutions, like the MR or Renderman Toolkits. This gets motion blur, is cached, far more controllable, and looks nicer. The advantage of doing the sim inside of Maya as opposed to RF is to have it play nice with everything else in Maya (although the particle count won't get nearly as high).
Problem with this method is you need to write your own converter. It's not that hard, I did it for RealFlow's BIN and Maya's PDC formats at my last job.
However, I'm having a devil of a time finding an SDK on the .MC (Maya Cache) format. I found the documentation on the XML created along with it, but nothing on MC. I may be able to reverse-engineer it to come up with a RF-Maya converter script, but I'd seriously rather not. Am I thick? I just can't find it.

Final verdict: Meshing will -maybe- useful for some minor effects (sculpture?). The nParticle stuff is useful on multiple levels, meshing less so.

[Bonedaddy]

Okay, I think I figured out the nParticle file SDK, and could probably write a .MC->Realflow BIN (and back) converter, if the need is there. If anyone's curious, the new nParticle cache files are basically using the Geometry Cache file format, so just search for geometryCacheFile.h in the docs and you should be able to figure it out with the help of a hex editor.

[azshall]

i don't mean to bash the liquid simulation (or nparticles in general), but from a production point of view, this is something I could only use for a very limited amount of tasks


Reading through the posts... Knowing that there is other solutions (ie, realflow glu3d, etc) obviously... I laugh, ... I mean, of course this is the first iteration of a new feature added to Maya, liquids and meshing. However, come on... Cut them some slack. Did you ever have to use the god awful blobby particles? To me, the liquids in Maya are a god send! Anytime I had to do ANYTHING "fluidy" that couldn't be done with fluids due to budgets or just development time I cringed.

They may be slow, ...They may not be perfect... But at this point, anything like this is WAY BETTER than blobbies ever wished it could be. I for one am thrilled that Maya can do this natively and now I don't have to look to other solutions so quickly...

[Bonedaddy]

They may be slow, ...They may not be perfect... But at this point, anything like this is WAY BETTER than blobbies ever wished it could be. I for one am thrilled that Maya can do this natively and now I don't have to look to other solutions so quickly...

Right. Like I said, the liquid simulation is useful in many, many ways. The meshing itself, however, is almost totally useless from a production standpoint (unless I am missing something here).

[elvis75k]

.. and could probably write a .MC->Realflow BIN (and back) converter, if the need is there...

Very welcome if you get the time :)

[alexx]

... (unless I am missing something here).

if you missed it, i did so as well :)

azshall:
well... yes and no.
you can do some small stuff, but as it comes to a real job where real money is payed for i would think you still can not use it.

maybe in maya 2010 or 2011, but not now.
it is more like a tech demo currently.

btw. did anyone notice the thick clouds preset?
duncan seems to have taken care of it after the big thread here:
http://forums.cgsociety.org/showthread.php?f=86&t=155001

cheers

alex

[thomwickes]

i'm still on 2008 - could you show us a render of it?

[Bonedaddy]

Okay, so I got some quick meshing tests done.

I ran a (unfortunately pretty ugly looking) sim, just to get a decent number of particles to stress-test the meshing functions. I then meshed it in both Maya and RealFlow (using a python script to convert .mc to .bin -- more on that later). Here's the results:


http://www.jasonporath.com/uploads/cgtalk/maya_meshing.jpg

http://www.jasonporath.com/uploads/cgtalk/rf_meshing.jpg

This test isn't meant to make good-looking meshes -- I'm confident both can turn out fairly decent meshes, although RF has a lot more control over how to tweak it. That isn't my concern here. I aimed to test:

* Meshing speed
* Mesh density (with roughly equivalent detail)

And in both of those, RF is the clear victor. I mean, yeah, Maya's a version 1, but still -- 16 seconds versus 191 seconds. That's almost 12x as fast. Plus, RealFlow's meshing gets you:

* Motion blur.
* Distributed meshing
* Motion blur.
* Sensible geometry caching
* Motion blur.
* Simplified method of doing distributed renders
* Did I mention motion blur?

As for the nitty-gritty:

The machine I did this on is an octo-core 2.8 Ghz Mac Pro, 10 GB ram, all done under OSX. Neither RF or Maya at any step in this process used more than 1 processor. I didn't monitor RAM usage.

The sim itself had:

* Self-collisions on
* Incompressibility 0.5
* Viscosity 0.01
* Rest density 2
* Liquid radius .05
* Floor: 0.9 bounce, 0.1 friction, 0 sticky

Maya's mesh settings:

Threshhold: 0.05
Blobby radius scale: 0.95
Motion streak: 1
Mesh triangle size: 0.13
Max triangle resolution: 100
Mesh method: Acutetetrahedra

RF's mesh settings:

Radius: 0.1
Blend factor: 95
Polygon size: 0.13
Mesh method: Metaballs
Filter: .1/0/32
Optimize: Curvature/6/2/1



The python script I used to do the conversion is something I am planning to release in a bit, once I get it polished up. It's called Rosetta, and it's basically a framework for doing conversions between any particle file format out there. Right now it handles back-and-forth with BIN, PDC, and MC. MC's kind of a pain of a file format, though, and it's taking me awhile to slough through it.

Eventually, I want to support:

* Houdini
* RIB
* MI (although I'm not sure this would be even useful, given MR's render setup)

The conversion isn't perfect, because each format has its own eccentricities -- BIN has a lot of fluid-specific information (nearest neighbors, pressure, etc), and Maya's MC contains all the information it can about the entire dynamics network that made the particles in question (although it doesn't seem to record animated values, rendering it of limited utility). But it works pretty well so far.

[Aikiman]

Mesh quality between Maya and RF look very similar. Im guessing the particle sim time is a lot quicker in RF also? Thanks for the speed tests BTW :)

[Bonedaddy]

Mesh quality between Maya and RF look very similar. Im guessing the particle sim time is a lot quicker in RF also? Thanks for the speed tests BTW :)

I couldn't get a 1-to-1 test on sim time going, but yeah -- RF would completely smoke Maya in that department. Not only is it faster and more memory-efficient in general, but it is multi-threaded (provided you have enough licenses).

[Tobbe]

I'm a noob when it comes to fluid simulation, but in what way makes realflow meshing motion-blur friendly? The mesh topology must change there too ?

Tobbe

[alexx]

nice testing Bonedaddy.

but you write: "sim time 22 minutes" on both images.
does that mean the simulation time was identical? that would mean maya is incredible much faster since it is not multi-threaded?

cheers

alex

about the motion blur: yes of course the meshes topology changes as well.
but realflow stores the speed information (which it extracts from the particles before meshing) in the meshes as well and the mesh loader in maya is able to render the motion blur then.
should not be so hard to implement in nParticles as well and i guess it will be in in the next update!

[Tobbe]

Ok alexx, so the realflow mesh isn't rendered as a standard mesh then, ok!

[Bonedaddy]

nice testing Bonedaddy.

but you write: "sim time 22 minutes" on both images.
does that mean the simulation time was identical? that would mean maya is incredible much faster since it is not multi-threaded?


They're both the exact same sim. I simmed it in Maya, and then converted the Maya particle files into the RF format for mesh testing. I just wanted to provide some info on how long the sim took.

[Bonedaddy]

Ok alexx, so the realflow mesh isn't rendered as a standard mesh then, ok!

Pretty much. It is a standard Maya mesh and behaves like one, but the mesh has an input from the RealflowMesher node, which updates its topology depending on time. The RF mesh format stores velocity information in each vertex, so if you go to a subframe, the RFMesher node will use that info and move that vertex an appropriate amount in the velocity vector's direction -- thus creating motion blur.

[Duncan]

I agree that we need to improve the nParticle meshing... this was just a first pass.

In terms of speed check out the quad mesh method. Use it with about 2 mesh smooth iterations. Also make motionStreak 0.0, as it can really make the meshing slower(although values around 0.3 can sometimes work instead of using motion blur, which is not yet supported) In a quick test I found quad meshing to be about 12 times faster than the acute tetrahedra meshing(which is the slowest method currently I think). The nice thing about the quad meshing is that one can do the meshing at a lower resolution then do a poly smooth to further refine the mesh(quad meshes smooth better than triangle ones). By using smooth on the mesh node the nParticle meshSmoothingIterations can be relatively low, which also makes it a little faster.

Also note that motion blur is supported for the direct blobby raytrace in Mental Ray. (The blobby raytracing is probably better for things like spray and droplets while the meshing will be better for things like water in a cup. Also it doesn't use as much memory as meshes.)

Duncan

[Bonedaddy]

Thanks, Duncan. I turned off motionStreak and followed your suggestions. I did two versions, smoothed and unsmoothed:
http://jasonporath.com/uploads/cgtalk/maya_meshing_quad_unsmoothed.jpg

http://jasonporath.com/uploads/cgtalk/maya_meshing_quad_smoothed.jpg

A lot faster, but obviously some issues with the edges -- which I'd suspect would extend to splashes, foam, or any sort of low-density, stringy fluid simulation. Still, very promising.

Sorry if any of this has come off as overly harsh -- I realize it's a version 1, and that you guys had a lot on your plates for nucleus already. The nParticle meshing is almost there, it just needs another push, to get the caching/rendering set up better. Maybe next version. :)

As for the MR rendertime motion blur, that is good news, but having a precalculated static mesh can be beneficial in many different scenarios. Still, useful!

[Duncan]

I said to use 2 with the smoothing on the quad mesh with the notion that one would do an additional poly smooth on the mesh (i.e. hit the 3 key). If you don't do this then you need to increase the smooth iterations to around 4 or so. The idea behind the additional poly smooth is that you can use a larger meshTriangleSize setting, which should make things much faster. (the smooth on the nParticle node has nothing to do with poly smooth, but is a relaxation of the initial grid derived mesh onto the actual particle isosurface) The nice thing is that the quad mesh method works well with the poly smooth (one gets Catmull Clark surfaces), while a smoothed triangulated mesh will have poorer interpolation and irregular triangles.

One would never use zero smooth on the quad mesh unless it was a special effect.

Duncan

[Duncan]

Looking at your post again it seems that you in fact did an additional poly smooth( judging from the triangle counts ). I would suggest either increasing the blobby radius scale or lowering the triangle size to get rid of the artifacts. Also increasing the smooth iterations on the nParticle node to 3 or 4 might help.

Duncan

[Bonedaddy]

I said to use 2 with the smoothing on the quad mesh with the notion that one would do an additional poly smooth on the mesh (i.e. hit the 3 key). If you don't do this then you need to increase the smooth iterations to around 4 or so. The idea behind the additional poly smooth is that you can use a larger meshTriangleSize setting, which should make things much faster. (the smooth on the nParticle node has nothing to do with poly smooth, but is a relaxation of the initial grid derived mesh onto the actual particle isosurface) The nice thing is that the quad mesh method works well with the poly smooth (one gets Catmull Clark surfaces), while a smoothed triangulated mesh will have poorer interpolation and irregular triangles.

One would never use zero smooth on the quad mesh unless it was a special effect.

Duncan

A ha. I did not know about this -- I picked up Maya around version 3 or 4, and, at least when I was learning, the 1-2-3 keys only really did anything with NURBS. I didn't even think to try them with polygon surfaces. Thank you, that looks much nicer. It doesn't seem like much of a speed hit, either.

The terminology is a bit confusing, though. You have the 1-2-3 poly smooth, you have the Mesh->Smooth command, and you have the nParticle mesh smoothing iterations. Lot of controls.

As for the zero smooth, I posted that merely to show what the effect of the mesh smoothing iterations were for the lurkers in this thread.

[Duncan]

The 3 key poly smooth is a new feature (I believe new to 2009). There are now smoothing options on the mesh node instead of needing a separate smooth node. It is much more efficient and you can set it so that it only smooths at render time.

Duncan

[thomwickes]

Digital tutors have releases a new training dvd for nparticles:
http://www.digitaltutors.com/store/product.php?productid=3568&cat=42&page=1

Think i might give this one a miss though to be honest.

[Aikiman]

Digital tutors have releases a new training dvd for nparticles:
http://www.digitaltutors.com/store/product.php?productid=3568&cat=42&page=1

Think i might give this one a miss though to be honest.

Yeah saw that, nParticles appear to be pretty self explanatory I assume.

[elvis75k]

Good news.. NextLimit plugin for maya 2009 avaiable for download on their website.

[Duncan]

I just completed a tutorial for pouring water with nParticles. You can check it out on my blog:
http://area.autodesk.com/index.php/blogs_duncan/blog_detail/pouring_water_with_nparticles/

Duncan

[SePu]

Thanks Duncan !

[Helgi]

Thanks Duncan for the tutorial!

It still looks to me that Real Flow is the most viable options for water sim in a production
... for now. Can't wait to see the next upgrade.

[pollart]

thanks Duncan

[Aneks]

Hi Duncan,

Just check out the post in the Area. Nice one ! I am still trying to get more of a hands on with nParticles so this is very useful.

Commented there but thought I might repeat it here. It would be really great to see some more info about some of the examples you showed at SIGGRAPH, especially the good/elastic examples as the meshing seemed very fast?!

Was this all done using convert nParticles to mesh ?

[Duncan]

The interactive fluid demos I did were all with particle to poly. I used the quad meshing method with the smooth iterations at 2 or 3. I also did a further poly smooth by hitting the 3 key. It is important to make the triangle size value as large as accepable for speed. One other point is that it helps if the particles are more or less together: if there any particle that are far away( for example a drop that fell out of a cup with no ground to stop it) then the voxel grid walked by the mesher gets much larger. (at some point we will hopefully fix this by using an octtree) The mucus example was done by applying a component to component constaint to the particles, using max distance and glue strength.

Duncan

[Aneks]

thanks Duncan.

So much to learn, but it's nice to be really excitied about a Maya tool again !

In terms of component nConstraint it seems very similar to the old Softbody springs. Which I remember with mixed emotion. I guess setting it to rubber band will allow for some elasticity.....

When you did the example of chopping up the nParticle mesh, roughly, what where you constriant settings ?

[Phlok]

Well...er...maybe it's a stupid question but the meshing process doessn't work at all for me.
Here's the steps I took:

1. Created a liquid simulation, using nParticles, rendered as BlobbySurfaces (Water presest in the nParticles menu)

2. After I had some particles on my screen, I did modify -> convert -> nParticle to polygons (yes I did select my particles before)

3. in the script editor it said particleToPoly

4. In the outliner a new object appeared "polySurface1"

5. nothing else happend. My maya didn't even get quite slow so I could conclude on some process happen in the background.

I've been trying that for a few times with no success whatsoever.

Did I miss something??

[Duncan]

It depends on the current settings on your particle system: radius, threshold, and the mesh attributes like triangle size. Just lower the triangle size and you should start to see something.

Duncan

[CGTalk Moderation]

This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.