PDA

View Full Version : Particle Cache and (Almost) Real-time Volume Rendering


zsmoke
11-03-2009, 12:57 PM
http://i37.tinypic.com/v2zlmo.jpgHello,

Think it is time to show the plug-in I've been working on for a while. It is particle-based hardware volume rendering. The idea is saving particles to custom cache data format, along with pre-computed lighting info. When the cache is imported, all particles are draw inside Maya OpenGL viewport as overlaying textured planes, creating the vision of a volume. The aim is to render cloud and smoke at high resolution within seconds. Taking advantage of HDR image, Spherical Harmonics, OpenGL Shading Language, artists may tweak parameters of render and watch the result without waiting for hours. It happens in near real-time. It could not beat the accuracy and beauty of those software volume render, but maybe it is worth trying. Anyway, it is still fun to work on.

The title image is a screenshot of Maya viewport. About 40 thousands of particles under HDR evironment.

In case you are interested, take a look at the blog entries about the progress:

http://cid-a6597afda81373ba.spaces.live.com/

And the gallery of tests:

http://cid-a6597afda81373ba.skydrive.live.com/browse.aspx/.res/a6597afda81373ba!351?ct=photos (http://cid-a6597afda81373ba.skydrive.live.com/browse.aspx/.res/a6597afda81373ba%21351?ct=photos)

It is still at its early stage of development. Will update more info about it later.

stooch
11-05-2009, 05:09 AM
very cool. ill test this for you if you want. in production :)

azshall
11-05-2009, 06:02 AM
god damn. that shit is dope... i'd love to play around with that :D

zsmoke
11-05-2009, 06:11 AM
I did play around with it over night. It is an obsessive waste of time:) You guys will be able to try it when it is ready.

swampjesus
11-05-2009, 10:12 AM
Great that you are back - i was wondering where did you disappear years ago :)

Wixman
11-05-2009, 02:48 PM
this is sick. nice job, man.

andymoorer
11-05-2009, 04:01 PM
Exciting stuff, keep at it! :D And thanks for the updates.

JulianJohnson
11-06-2009, 03:08 AM
Looking good Dr well done. i'm Looking forward to seeing more tests

oceanq
11-06-2009, 07:24 AM
ZJ?

Are you the guy who wrote the Voxel render 4years ago?

zsmoke
11-06-2009, 10:13 AM
Yes, I revived the Voxel started years before. Have no idea so much time has been lost.

Voxel is based on 3D texture, which can never catch the detail of a large scene. One should write particles to 3D texture and show in a stand alone. Result is bad. This is particle-based, and integrated into Maya. Also takes advantage of Spherical Harmonics method I used in the PRT tool, to get image based lighting. Light the volume interactively with an HDR image inside Maya.

Not much to update for now. I have to fix some problem with animation first.

SePu
11-06-2009, 04:38 PM
Looks very promising ! really looking forward to play with it ....
great work ZSmoke ! keep it up .

zsmoke
11-07-2009, 10:00 AM
It is making progress. No problem to playblast animation at full quality. A few more attributes to support and off-screen render to images, then we are done.

raylistic
11-07-2009, 11:31 AM
Nice, how did you do these? In API?

JulianJohnson
11-07-2009, 04:30 PM
Where's the link to the quicktime file Dr ?

zsmoke
11-07-2009, 05:34 PM
That is a screenshot of some fly-through animation test. Just to check out if the cloud looks correct. Not very interesting indeed.

stooch
11-07-2009, 06:15 PM
how is the motion blur handled? is there motion blur support?

zsmoke
11-09-2009, 11:17 AM
Motion blur will be added later. The idea is to displace the slices along velocity vector transformed on-screen. Test shows larger slice count is needed. Of course more complex particle dicing method.

Aikiman
11-09-2009, 06:01 PM
Did you write the plugin on a mac?! I noticed a screenshot you use a macbook thats all. Hope this is a plugin that macusers wont miss out on :)

zsmoke
11-10-2009, 06:00 AM
Yes, most of the developing working is on mac, because I love XCode and the whole user interface. I also work on Windows 32-bit and Linux 64-bit, the code differs slightly on each operation system. It is easy to compile the plug-in on a different os. Still have no idea about 64-bit Windows, but no worry for now.

zsmoke
11-11-2009, 09:21 PM
Bump to say it is very close to public beta. It is working on windows 32-bit, and it saves images out as expected. Will prepare the documentation and post it somewhere after more tests.

http://cid-a6597afda81373ba.skydrive.live.com/self.aspx/GPU/volwin.jpg#resId/A6597AFDA81373BA!398 (http://cid-a6597afda81373ba.skydrive.live.com/self.aspx/GPU/volwin.jpg#resId/A6597AFDA81373BA%21398)

zsmoke
11-12-2009, 05:02 AM
Linux fans should cry. This tool cannot work on Linux because I still haven't any luck using OpenGL Extensions within Maya plug-ins. Calling those functions always crash Maya instantly.

Same report here:

http://nccastaff.bournemouth.ac.uk/jmacey/RobTheBloke/www/mayaapi.html

Anyone has same experience, or any idea about a fix?

stooch
11-12-2009, 05:38 AM
very exciting stuff. cant wait to play wiht this!

zsmoke
11-12-2009, 02:03 PM
http://cid-a6597afda81373ba.skydrive.live.com/self.aspx/documentations/zsmokedoc.pdf

Some early doc is available. Still lacking of fancy demo images, but enough to show how to use.

Public Beta for 2008ext2 2009 2010 win32 is posted here:

http://cid-a6597afda81373ba.skydrive.live.com/self.aspx/documentations/particleCache%5E_public%5E_Nov132009.zip

And it is on creativecrash:

http://www.creativecrash.com/maya/downloads/scripts-plugins/c/zsmoke

noizFACTORY
11-12-2009, 05:13 PM
Wow, this is awesome news! Too bad though we are stuck in 8.5 at work. Thanks for sharing this Zhang.

-Sachin

arkangel-fx
11-12-2009, 05:20 PM
Downloading! This really deserves a try!

Well, didnt work for me :(
"Specified Module could not be loaded" when trying to load particleCache.mll

zsmoke
11-13-2009, 04:25 AM
Likely it cannot find some dll needed. Have you copied all files in /bin folder of the package to C:/windows/system32 ? If you did, I should have missed something. Also to mention the 2008 version is for 2008ext2, not 2008, different API version. Let me know there is problems.

And some report about crash on loading. Seems low opengl version could crash maya (http://cid-a6597afda81373ba.spaces.live.com/blog/cns!A6597AFDA81373BA!408.entry)

Recommend to update to latest drive from nvidia site. GeForce 6600 should work.

chippey
11-13-2009, 06:34 AM
Any chance of a Mac version?

ChrisBore
11-13-2009, 06:35 AM
I'm not getting any errors bit no particle cache menu in ztools (just the HDR viewer).

The two plugins are visible in the plugin manager, but the particle cache plugin refuses to load up.

Maya 2009 (32 bit on 64 bit vista)

Take it easy,

chris

tokanohanna
11-13-2009, 06:57 AM
Zsmoke, you are the man. Looking forward to trying this beta version.

zsmoke
11-13-2009, 07:20 AM
Any chance of a Mac version?

This is originally written on my macbook, so it is easy. But only possible work with 2008 ext2 on OS X 10.5.6+ and Intel CPU, don't think will work on ppc.

The two plugins are visible in the plugin manager, but the particle cache plugin refuses to load up.

Maya 2009 (32 bit on 64 bit vista)

dll shows up in plug-in manager does not mean it is working, should load it. It is compiled on 32-bit windows xp, no guarantee it will work on 64-bit operation systems. I need to find a way to compile on 64-bit windows first.

Also it might because of missing library, need those dlls copied to C:/windows/system32. I am not sure if I have all needed dlls included in the bin folder, because I don't have a clean setup. So anybody successfully loaded the both plug-ins?

zhang

bazuka
11-13-2009, 07:58 AM
yo z

i can compile for x64 if u want...

cheers

zsmoke
11-13-2009, 01:22 PM
http://cid-a6597afda81373ba.spaces.live.com/blog/cns!A6597AFDA81373BA!411.entry

Today updated to 0.5.0

Damned highend3d link falls to someone else's ownership, cannot update or delete it.

chippey
11-14-2009, 02:46 AM
This is originally written on my macbook, so it is easy. But only possible work with 2008 ext2 on OS X 10.5.6+ and Intel CPU, don't think will work on ppc.

zhang

Cool! I'd love to try it out on my Mac. If you like, I can compile it for 2009 on intel.

zsmoke
11-14-2009, 02:15 PM
Mac lovers can get the tool at:

http://cid-a6597afda81373ba.skydrive.live.com/browse.aspx/documentations

That .bundle files are for 2008ext2, compiled on Intel Mac OS X 10.5.8, using the same MEL scripts for windows version. Need OpenEXR installed on your system.

viki164
11-14-2009, 06:02 PM
Hey Zsmoke!

So I tried something here but not quite impressed with output :(.
I get a flat shading here with hdri lighting but no self shadowing!
is there any way to get selfshadowing details?? & also how one controls the opacity?
does it takes any input from particles for shading?

have a look at my attempt

thnxs anyway!
Vik

viki164
11-14-2009, 06:39 PM
So after playing with the shading parameters I got some detailing which looks cool for non-dynamic structures.

http://www.youtube.com/watch?v=CRgKbRiWSXI

a small simple test for some reason youtube is not uploading proper length of my videos. Its gets stuck most of the time especially in the end :(

let me know of what more I can get with these plugin.
thnxs much z!

best
Vik

noizFACTORY
11-14-2009, 06:48 PM
Vikas, maybe you need to increase the density while baking the particles. In the docs, I see that its the density attribute that controls how much the particles occlude each other. And maybe your key camera needs to be in a better position or something.

Haven't played with this yet so all assumptions are based on the doc that I've read.

-Sachin

chippey
11-15-2009, 10:06 AM
Mac lovers can get the tool at:

http://cid-a6597afda81373ba.skydrive.live.com/browse.aspx/documentations

That .bundle files are for 2008ext2, compiled on Intel Mac OS X 10.5.8, using the same MEL scripts for windows version. Need OpenEXR installed on your system.

Brilliant! Can't wait to try it out! :)

zsmoke
11-15-2009, 02:07 PM
Hey Zsmoke!

So I tried something here but not quite impressed with output :(.
I get a flat shading here with hdri lighting but no self shadowing!
is there any way to get selfshadowing details?? & also how one controls the opacity?
does it takes any input from particles for shading?

have a look at my attempt

thnxs anyway!
Vik

Thanks for Vik's feedback and time to post the test. Glad to know it is loaded.

May I see your cache in low mode? When the cache is loaded, maya will create a locator under a transform, that is what drawing the volume in OpenGL. Select the transform, find pmapViz1 in AE or CE, set Draw Type to 1 will draw low mode, that is a lot of boxes without volume effect and a lot faster to interact. There will be lighting preview in low mode too. Change KKey KBack, rotating the HDR Viewer or change its exposure will change the lighting in realtime. You have activated IBL, right? Maybe the exposure of HDR image is too high, too bright to view any color.

Working on some tutorial now, has more images to make it easier to artists. Not just lines of text.

zhang

zsmoke
11-15-2009, 04:01 PM
http://cid-a6597afda81373ba.spaces.live.com/blog/cns!A6597AFDA81373BA!438.entry (http://cid-a6597afda81373ba.spaces.live.com/blog/cns%21A6597AFDA81373BA%21438.entry)

Some tutorial with more images, showing how to change the look of volume.

Current particle dynamic is still crappy. Not working with dynamics for a long time. Going to revive my GPU Fluid Field, will be interesting.

http://cid-a6597afda81373ba.skydrive.live.com/browse.aspx/documentations

Also put some HDR environment maps I am using along with the files.

Aikiman
11-15-2009, 06:44 PM
Cool! I'd love to try it out on my Mac. If you like, I can compile it for 2009 on intel.

Can you...please?

zsmoke
11-24-2009, 09:21 PM
A bit cool down these days. Testing my tools on window 64bit for now. Maya 2010 x64 people can grab a copy here:

http://cid-a6597afda81373ba.skydrive.live.com/self.aspx/documentations/particlecache2010x64.mll

zsmoke
11-27-2009, 06:23 PM
http://i48.tinypic.com/dqnvc.jpg

This thing is still hot. I have GPU fluid integrated. Still having issues to work out, but we are close.

arkangel-fx
11-30-2009, 04:16 PM
Wow thats awesome. Its great to see this stuff finally being implemented not only by the big studios :D

zsmoke
12-22-2009, 06:22 AM
http://i46.tinypic.com/1e041t.jpg

Have something to update at last. Almost completely re-write the tool in weeks past. It is not really PARTICLE cache now. It is fluid cache from zFluid, another plug-in. And Octree + billboard directly rendering works pretty well. Shadowing is based on nVidia CUDA SDK smoke particle example. Maya feedback is totally realtime. These tests are lacking of many controls like color and noise, but the base of rendering is working. Moving the light position can interactively change the look of shadow.

More to come.

viki164
12-22-2009, 01:52 PM
hey Z!
I think now u chose the best route Man. these seems to have a good potential. I always love GPU acclerated stuff. Hope to see more updates on this. Let me know If I wuld be of any help to u like in doing stress tests in my happy hours after work :).
by any chance can U accelerate maya fluid`s speed in GPU mode?? hope its not a stupid question. really appreciate ur efforts here & U have very good artistic & devlopment vision.
thnxs much & Merry X-mas
Best
Vik

tokanohanna
12-22-2009, 05:38 PM
Zsmoke,

Every time I load a cache, It looks wonderful. Once I rewind to frame 1.......crash. Very unfortunate. Would like to try this out, but keeps crashing. Any workarounds?

Edit: Never mind I figured it out. Start frame was not set to start frame of cache. All good now.

zsmoke
12-23-2009, 11:26 AM
In case Maya crashes on specific frame, esp frame 1, check if there are cache file (.pmap) exists at that frame. Cache export will skip zero particle count, no particle at frame 1 no cache. Load cache start from frame 2. There should be some validate check on cache loading, hope I can fix that on next release. Just be careful not to load no cache for now.

http://i45.tinypic.com/33or7a1.jpg


Add noise and a lot controls today.

For GPU-boost Maya Fluid, I think that is not possible. Maya Fluid has no interface that you can re-code it. It is pure CPU procedural, maybe. And Maya Fluid 3D is way too slow to do any serious work. Simulation is slow, rendering is slow. Oh, I am too impatient about that. I have my own 3D fluid plug-in, zFluid, based on Cg and OpenGL. The restriction of GPU is the highest grid resolution is limited by display card memory. I tested it on a GeForce 9600 GT 512 Mb box, running simulation at 256 x 128 x 256 grid at more than 1 fps. Feel happy about that so far.

zFluid will be released to public along with zSmoke next version. Then you will have some idea about how GPU fluid works.

viki164
12-23-2009, 12:16 PM
that sounds super cool Man!
It seems very promising though, I`ll be happy to promote ur plugin to my studio & use it further for future shows.
thnx
Vik

Aikiman
12-24-2009, 06:31 AM
Cool! I'd love to try it out on my Mac. If you like, I can compile it for 2009 on intel.

chippey, can you still make this happen? Ima hangin out.

chippey
12-24-2009, 07:59 AM
chippey, can you still make this happen? Ima hangin out.

I'm more than willing give my go at compiling this for 2009 on osx :) However, it's really more up to zsmoke. I'm guessing he may want to sell this plugin eventually, so he may not want to give the source code out to people.

Cheers!

zsmoke
12-25-2009, 09:12 AM
http://i49.tinypic.com/24goo7k.jpg

Add the function that any poly mesh can work as fluid source. zFluid and zSmoke is close to release.

Not sure about what to do with this tool. I spent a lot time on this, certainly won't let people get it for nothing.

arjan_meerten
12-25-2009, 09:57 PM
Wow that looks really great!
Does it use it's own solver to simulate? What kind of noise do you use?
Is it possible to add variation in the softness of the fluids like in real clouds? Like some parts of a cloud are very dense and have only a hard edge and some parts are really transparent and soft.

zsmoke
12-25-2009, 10:17 PM
It use zFluid for simulation, GPU-based 3D fluid solver a lot faster than built-in one.

The noise is based on GL image3D and fractal by GLSL, there are some note about how the noise is done on my blog. Still not quite like the noise.

Can change over all opacity when drawing the volume. Density field is cached by solver. Definitely its variation is controlled by 3D fluid simulations. Also how fast the mass will loss after each step will change the density. More density if the fluid is blocked by some obstacle.

I tend to use higher density to show shadow in cloud. There can be more variation there more complex simulation is needed. Maybe need a better solver to diffuse the fluid.

http://cid-a6597afda81373ba.skydrive.live.com/self.aspx/documentations/cloudguy%5E_dec2609.avi

Some test about a guy walking as a cloud. Quite interesting.

zsmoke
12-28-2009, 07:04 AM
Decide to release zFluid plug-in to public. You can get it from:

http://cid-a6597afda81373ba.skydrive.live.com/self.aspx/documentations/zFluid151.zip

Check out the screenshot:

http://i47.tinypic.com/qzhgex.jpg

Pretty easy to use. One-click to create a demo setup.

Current release is for Maya 2010 windows x64 and Maya 2008 mac intel x86

Will add doc when I have time.

Aikiman
12-28-2009, 08:04 AM
Decide to release zFluid plug-in to public. You can get it from:

http://cid-a6597afda81373ba.skydrive.live.com/self.aspx/documentations/zFluid151.zip

Check out the screenshot:

http://i47.tinypic.com/qzhgex.jpg

Pretty easy to use. One-click to create a demo setup.

Current release is for Maya 2010 windows x64 and Maya 2008 mac intel x86

Will add doc when I have time.

zsmoke, I believe you just made a lot of Maya fluid users very happy ;) Im am so looking forward to giving this a test drive. Can this be compiled for Mac intel Maya 2009?

zsmoke
12-28-2009, 08:59 AM
Oh, let me find Maya 2009 os x first.

arkangel-fx
12-28-2009, 12:46 PM
zhang, good job with the solver, pretty fast indeed. Just tryied it at work with a quadro fx 450(i believe) darn old card and it still solved a 64*64*64 grid at 3fps! Should i cache the particles with zsmoke and use it to visualize it also? right now im on 2010 x64 so im connecting youre solver to a maya fluid, pretty neet :D

noizFACTORY
12-28-2009, 02:22 PM
Thanks for the release zsmoke. Excited to try it out tomorrow at work!

bazuka
12-28-2009, 02:31 PM
yo Z can u plz make a build for maya 2009 win x64?

10x and great work

zsmoke
12-29-2009, 07:33 AM
Thanks for all the feedback. Something should be clarified here.

About the grid resolution. Default is 64 x 64 x 64, definitely it can be higher. The limit is how much display card memory you have. Old hardware has 128 mb, it is safe to use 64 * 128 * 64 or so. My macbook has GeForce 9400 256 mb, it can carry 128 * 128 * 128. I tested it on a desktop with GeForce 9600 512 Mb, it can run at 128 * 256 * 128. You see all numbers are power-of-two. Non-power-of-two grid resolution should be working on those new hardware, but not tested yet, so it blocks to use non-power-of-two grid resolution.

About the memory, zFluid needs a lot memory to run 128 * 256 * 128 grid, I will recommend to have a 2G RAM desktop to do anything serious.

About the speed, it is a lot faster than Maya fluid indeed. Because speed is affected by a lot things, and I never tested Maya 3D fluid with grid higher than 128, it is hard to know exactly how many times faster it is. If you have a lot heavy poly mesh as source or obstacle, simulation will slow down. I have to use a 50K triangle mesh as source for a 128 * 256 * 128 grid, it is down to 0.2 fps, not good. So use low mesh as source and obstacles. No need to have mush details.

About the source and obstacles, zFluid can use poly mesh as source to emit fluid into the solver or obstacle to block the fluid. To do that, select meshes, shift-select the particle, choose menu attach obstacle or object as source. Connection will be made.

About preview, there is a shoTex attrib of the grid, 0 is no texture. If you set to 1, it will display a slice of density field when you playback. Change slice Id can change where to display the slice. That is the only way you can preview the result for now. Of course moving of the particle is some sort of viz too.

About rendering, zFluid is just a solver, need to save out cache files per-frame, and render with zSmoke. Current public zSmoke does NOT support that kind of cache. Please wait for a new release.

About the doc, it is quite a large piece. zFliud is not that simple to explain howto. Hope I can release some notes soon.

Support for other maya version on windows and mac should wait.

azshall
01-14-2010, 08:39 PM
Decide to release zFluid plug-in to public. You can get it from:

http://cid-a6597afda81373ba.skydrive.live.com/self.aspx/documentations/zFluid151.zip

Check out the screenshot:

http://i47.tinypic.com/qzhgex.jpg

Pretty easy to use. One-click to create a demo setup.

Current release is for Maya 2010 windows x64 and Maya 2008 mac intel x86

Will add doc when I have time.

Is there a possibility of a PC Maya 2008 x32 version??? :) I would love to test this out!

thomwickes
01-14-2010, 09:19 PM
omg seth, maya 2008 is SO last year

azshall
01-14-2010, 09:29 PM
omg seth, maya 2008 is SO last year

hahahahaha, yeah ... well ... when your employer won't shell out the loot to upgrade us we're stuck with sticks and stones to play with :)

zsmoke
01-15-2010, 05:14 PM
Just updated zFluid to 1.7.3 with doc. zSmoke rendering, all versions on 32-bit windows, 2009 on mac still need wait.

azshall
01-15-2010, 05:38 PM
Just updated zFluid to 1.7.3 with doc. zSmoke rendering, all versions on 32-bit windows, 2009 on mac still need wait.

i only see x64 for PC .. Is there a way to use the x86.bundle on PC or is that Mac only? .. Forgive me for being dumb, I've never really used Maya a Mac :)

zsmoke
01-16-2010, 12:05 PM
Now it supports 2009 on intel x86 mac, i checked the API version is 200904, so it is sp1. It works as expected so far.

Edit:

Report about cannot load Cg library on Mac, think need to install Cg Toolkit downloaded from nVidia.

arkangel-fx
01-21-2010, 12:12 AM
i only see x64 for PC .. Is there a way to use the x86.bundle on PC or is that Mac only? .. Forgive me for being dumb, I've never really used Maya a Mac :)

Any news about a 32bit version for 2009 or 2010?

Aikiman
01-21-2010, 03:44 AM
Now it supports 2009 on intel x86 mac, i checked the API version is 200904, so it is sp1. It works as expected so far.
Edit:
Report about cannot load Cg library on Mac, think need to install Cg Toolkit downloaded from nVidia.

Im geting loading errors as well - OSX intel Maya 2009.

zsmoke
01-21-2010, 06:51 PM
You should install Cg.framework and build OpenEXR on your mac. Cg is easy, Building OpenEXR is not quite easy for artists. Someone got it working by copying those .dylib files to exact path.

Anyway, zFluid now supports win32 2009 and 2010. Since most people still using 32-bit windows now, hope that will get more feedback soon.

arkangel-fx
01-21-2010, 10:35 PM
You should install Cg.framework and build OpenEXR on your mac. Cg is easy, Building OpenEXR is not quite easy for artists. Someone got it working by copying those .dylib files to exact path.

Anyway, zFluid now supports win32 2009 and 2010. Since most people still using 32-bit windows now, hope that will get more feedback soon.

so the x64.mll in the zfluid zip are w32 really?

zsmoke
01-22-2010, 08:31 AM
x64 is 64-bit, x86 is 32-bit

KidderD
01-22-2010, 02:31 PM
Hey zsmoke,

thanks for letting us test this beauty plugin out for you. I hope you get the feedback you need to inspire yourself further. I've had two different experiences with this, one, where it seemed, like I could change the resolution to whatever, and the sim would stay the same, and the second, really painful one was, I think I had it set at 256^3, and I couldn't even get enough get my mouse back to do anything. So, I'm not sure. What specifically would you like us to test? It seems to be pretty fast at what it does, but it is quite limited. I am also not totally sure I'm using it correctly, I can't get the shotex to work, I'm basically only able to advect (I think) particles through a fluid volume.

Nonetheless, impressive work, and very comforting to know that Maya is capable of these kinds of things. Oh, and it's also a lot of fun to play with collission objects in interactive dynamics mode.

zsmoke
01-23-2010, 10:57 AM
Thanks for your screenshot, I see the triangle used as emitter object, that is from old 1.5.1, which parameter update including shotex is broken, and vorticity confinement has wrong effect. Both is fixed in 1.7.3. Think you should update your zFluid to 1.7.3

If you create a demo set from the menu, it will bring a basic setting with one emitter inside 64^3 grid. One may get more complex simulation by connecting more emitters to particle system, emitting from moving objects, some obstacles, increasing the grid resolution.

About the resolution, it depends on your display memory and system ram. I never get 256^3 because no memory to allocate the opengl textures. You can get nothing be freezed screen if windows is using virtual memory on this. But lower grid like 128 256 128 should be enough for most simulations.

I want people to use zFluid to create complex particle simulation that cannot be achieved in traditional ways. Make sure it is stable to use in complex situation. Feel safe to render the particles for now, I will update zSmoke to deal with the zFluid cache rendering soon. Also it is fluid simulation a few times faster then Maya fluid. Of course it has quite limited controls for now. The work is yet to finish.

And you don't need that dense mesh for obstacles, it slows down simulation heavily. The idea is using low mesh, breaking complex shape into multiple convex simpler parts, triangulate them and attach as fluid obstacles.

Some screenshots here

http://www.creativecrash.com/maya/downloads/scripts-plugins/dynamics/c/zfluid

1,7,3 should work like that.

fxabhi
01-23-2010, 04:55 PM
zhang jian, this is awesome stuff...i just asked for a new solver yesterday i guess :)... wish granted...i have to yet try this for that matter... if you can just help me out with this cause am real slow about what you have done so here it goes i have a dell precision laptop with 4gb ram and quadro fx 1600 card and its a core 2 duo... 64 bit windows os but maya 2009 32 bit...... when i work with fluids in maya i fail to have a resolution of anything above 100 100 100 ... it crashes if i go 120...so firstly would this be a good hardware to test this on...and secondly is there any way you can put up some video maybe as to how to use this properly and effectively...the way you would have intended this to be used...

thanks and cheers :bowdown:

zsmoke
01-23-2010, 07:32 PM
No need highend hardware to run zFluid. The code is mainly written on my Macbook, it is an entry level laptop. Video card is GeForce 9400M, so it only has 256 Mb shared memory. I can still run 128 x 128 x 64 grid with acceptable fps.

If you have latest card with 1G memory, you may set grid higher than 128 x 128 x 256, but it is not tested, I think that is a limit of grid you can allocate on hardware. That is different from Maya fluid, which lives in RAM.

You may notice zFluid need grid rez set to power-of-two, 32, 64, 128, that is due to old hardware does not allow non-power-of-two textures. That is waived on new hardware. I will test with non-power-of-two grid on next update. Should be helpful if it can set to 150 or something.

About crash, it is using some OpenGL extension, if your display driver is not up-to-date, it could crash because of low OpenGL core version. Downlaod and install latest drivers from nVidia should fix that. It should not crash if you just try the demo setup.

About how to use it. Think it is necessary to post some tutorial, video one, when I have time on that. Check out doc from site first.

Did some test, my macbook can run 128 x 128 x128 grid at 1.1 fps, think that is the limit if you have only 256 mb display memory.

Remember, one should playback from start to update any change of grid resolution and size.

This is proof:

http://cid-a6597afda81373ba.skydrive.live.com/self.aspx/Fluid/Snapshot%202010-01-24%2005-11-25.jpg

YourDaftPunk
01-23-2010, 08:50 PM
256x256x256 solves at about 7 seconds a frame on my nvidia gtx 975. That's 16,777,216 voxels so roughly 2.4 million voxels per second. That's insane!

Nice work!

Aikiman
01-25-2010, 07:41 AM
You should install Cg.framework and build OpenEXR on your mac. Cg is easy, Building OpenEXR is not quite easy for artists. Someone got it working by copying those .dylib files to exact path.

Anyway, zFluid now supports win32 2009 and 2010. Since most people still using 32-bit windows now, hope that will get more feedback soon.

Anyone here successfully installed on an intel mac? Im having problems with the openEXR stuff, as it appears there is the IlmBase 1.0.1 source code to build first before the openEXR, this is from the homepage openEXR.

zsmoke
01-25-2010, 03:44 PM
Glad to know it can go to that high. triple 256 means eight 4K textures reading and writing scores of times every frame. Incredible! But think that is the limit. 3D fluid simulation requires quite a lot resources.

And about OpenEXR on Mac, I heard about cannot compile on Snow Leopard, but copying dylib from older os can fix that. One can get the libs I am using. Compiled on 10.5.6 long time ago. I am still using 10.5.8 for now, so cannot test on 10.6.

Get it from:

http://cid-a6597afda81373ba.skydrive.live.com/self.aspx/documentations/openexr1.6.1%5E_precompiled%5E_intelx86.zip

Copy all files to /usr/local/lib should be fine.

Aikiman
01-25-2010, 05:39 PM
Glad to know it can go to that high. triple 256 means eight 4K textures reading and writing scores of times every frame. Incredible! But think that is the limit. 3D fluid simulation requires quite a lot resources.
And about OpenEXR on Mac, I heard about cannot compile on Snow Leopard, but copying dylib from older os can fix that. One can get the libs I am using. Compiled on 10.5.6 long time ago. I am still using 10.5.8 for now, so cannot test on 10.6.
Get it from:
http://cid-a6597afda81373ba.skydrive.live.com/self.aspx/documentations/openexr1.6.1%5E_precompiled%5E_intelx86.zip
Copy all files to /usr/local/lib should be fine.

That worked, thanks a bunch!

Aikiman
01-25-2010, 07:07 PM
Zsmoke, increasing the resolution on the viz node appears to scale the transform on the fluid node, is this correct? At playback it appears to have increased the res too because it becomes slower, it just seems a little quirky to scale the transform also.

Apart from that everything else appears to be working well. BTW it is necessary to set your project otherwise the caching wont work as expected.

Cheers.

zsmoke
01-26-2010, 08:04 AM
Good to know it is working on your side!

Increase grid resolution WILL enlarge size of the box, boundary of simulation, because more grids are there. CANNOT scale the box, there is a parameter named Grid Size, default is 1, use smaller grid size will correct that. 1 for 64^3, then 0.5 for 128^3. Playback to see the update.

Of course increasing grid resolution WILL slow down the simulation. More grid, more calculation, more details, more time.

Save Cache is meaningless for now, it dumps 2 files into /data folder each frame. Can only render that with new zSmoke, which is not finished yet.

manymany
02-02-2010, 04:33 PM
hi im newbie in your plugin (experienced in CG industry though)

i tried out a bit of your software and it's of great potential .

im however i bit lost on the workflow and relationship between zFluid and Zsmoke
does it mean we sim using Zfluid, cache the particles, then Zsmoke read in the cache and do the rendering(viewport)?
it's definitely very fast you are great.
how is it compare to the functionality of maya fluid and fumefx in general?

another question is, in some page in your blog i always saw a pixelated fluid render in a 2D plane representing the 3D particles of zFluid,

https://1hz37q.bay.livefilestore.com/y1mvMzGO_s9W_gIl5e7_FOlRy71huW7i1mQzNmMHKWl53srjaLr3YwMCz3TPgnUvkU8dDTJKRRNsraLiI9rfUYefVLiBnadXri8D6-N3RMEyfidC4sY6V_Kq6MfFsckyTYPRsNmVq7omKp3A0eD5X_3dQ/sttptc.jpg

how is it done? How and when do we use this "2D plane mode" compare to the Zsmoke 3D fluid rendering? is it a faster preview render of fluid method?

thanks for the big effort and beautiful science!

zsmoke
02-03-2010, 01:55 AM
Thanks manymany for the feedback. It is a bit cool down these days.

First is about zSmoke, some volume rendering plug-in. Lots of overlapping semi-transparent textures planes. Position of those planes are controlled by particles. That is why this thread called Particle Cache.

Then zFluid. That is a different thing. A fluid 3D solver on GPU. It does have some, of course not all, functions of Maya Fluid. It updates field properties, like density and velocity, every frame. You see those particles? They are used to inject density and velocity into the field, to impulse the solver to move. In fact, grid-based fluid solver can work WITHOUT particles. Why still need particles? Well, that is quite straight forward way to visualize density field in 3D.

There is not easy to render a volume, zFluid does show a slice of flied, that is what you call "2D plane mode", that is no 2D, but a slice in 3D. And that is for display purpose only, giving some idea about how the density looks like at that slice.

Maya Fluid does both simulation and rendering. zFluid does simulation only. That is too complex to do both on GPU. I want to separate the render function in zSmoke. It will still draw planes, not on particles but on density field saved by zFluid.

Since most of my energy is spent on 3delight stuff these days. Those who want to render zFluid cache via zSmoke should wait for a while.

ruchitinfushion
02-26-2010, 03:47 PM
Hello zsmoke, in Ztools saving & loading is working fine but when i try to render nothing is comes up...So,plz tell me wht to do.Thnx

Als
04-18-2010, 10:34 AM
Hi Zhang,
this is such a great tool, I would like to contribute if that would help to get x64 versions out.
Do you have some paypal donate account?


Als

Als
04-18-2010, 04:10 PM
Hi,
I'm trying to cache zfluid particles, but they don't save.
Any thought on that, or any tools for rendering them directly to archives for 3Delight?
Tried bin exporter from realflow, and that works fine.

Thanks


Als

zsmoke
04-19-2010, 03:00 PM
Hi,

Sorry not being around for a while. I have to do some work on 3delight. Now I am preparing a major update of zFluid. Let it be zFluid Pro, which fixed a few issues with current zFluid, and added a few more features:



Multi-Particle System: fluid field can move more than one particle systems.
Field cache: run simulation once to save the field, velocity per cell mainly, when field is cached, 3D fluid simulation can be skipped. Simply read the result back can speed up playback greatly.
Multipass Particle Cache: playback to save out particles. Each time will be a pass. Because field is cached, large movement, position of swirls, remains same. But individual particles varies each pass due to noise output force. If Maya can do 500K particles during a pass, save out 10 passes will accumulate to 5 million. This will replace the Maya PDC files.
RenderMan Rendering: saved particle files will be loaded by RenderMan procedural primitive to create points for render. This will be added to Anemone ( another plug-in for 3delight shading/rendering management ) and render with 3delight.
Still testing those new features. Future development will include some custom rendering tool, but it will not be available on next release. zFluid Pro will not be available for free, but I will update the public demo with basic functions.

About 64-bit build: I can build 64 bit on Windows now, will be no problem supporting multiple version of Maya on both 32 bit and 64 bit Windows.

About zSmoke: no development about the sprite-based particle rendering tool so far. Overlapping textured quads in OpenGL still lacks of detail when particles are moving fast. Need more sophisticated rendering tech!

zsmoke
04-19-2010, 03:15 PM
http://i42.tinypic.com/30ti44y.jpg

http://i42.tinypic.com/ix49zd.jpg

Thanks Olivier Junquet for sharing his test animation. Basically save particles to RIB and render in PRMan with ReadArchive. Lots, lots of points with motion blur and deep shadow. This is done by old zFluid WITHOUT multipass particle cache.

I will post more tests.

KidderD
04-19-2010, 04:37 PM
Multipass Particle Cache: playback to save out particles. Each time will be a pass. Because field is cached, large movement, position of swirls, remains same. But individual particles varies each pass due to noise output force. If Maya can do 500K particles during a pass, save out 10 passes will accumulate to 5 million. This will replace the Maya PDC files.



This feature of it sounds Krakatoaish. Nice!

Aikiman
04-19-2010, 07:57 PM
Awesome stuff zsmoke, Im liking where you are going with zFluid. Look forward to more updates and a release!

Als
04-19-2010, 08:21 PM
I can pay through paypal, pal.

Thanks


Als

Als
04-20-2010, 07:37 PM
Hi,

Sorry not being around for a while. I have to do some work on 3delight. Now I am preparing a major update of zFluid. Let it be zFluid Pro, which fixed a few issues with current zFluid, and added a few more features:





Multi-Particle System: fluid field can move more than one particle systems.
Field cache: run simulation once to save the field, velocity per cell mainly, when field is cached, 3D fluid simulation can be skipped. Simply read the result back can speed up playback greatly.
Multipass Particle Cache: playback to save out particles. Each time will be a pass. Because field is cached, large movement, position of swirls, remains same. But individual particles varies each pass due to noise output force. If Maya can do 500K particles during a pass, save out 10 passes will accumulate to 5 million. This will replace the Maya PDC files.
RenderMan Rendering: saved particle files will be loaded by RenderMan procedural primitive to create points for render. This will be added to Anemone ( another plug-in for 3delight shading/rendering management ) and render with 3delight.
Still testing those new features. Future development will include some custom rendering tool, but it will not be available on next release. zFluid Pro will not be available for free, but I will update the public demo with basic functions.

About 64-bit build: I can build 64 bit on Windows now, will be no problem supporting multiple version of Maya on both 32 bit and 64 bit Windows.

About zSmoke: no development about the sprite-based particle rendering tool so far. Overlapping textured quads in OpenGL still lacks of detail when particles are moving fast. Need more sophisticated rendering tech!


Hi,


this is all great news. I'm was serious about being your first customer, please contact me!!!

Cache doesn't work with native maya caching. But works with realflow export.
Also I'm geting the square artifact on particles when using big number of particles.

Thanks




Al

zsmoke
04-20-2010, 09:49 PM
Testing with one fluid moving more-than-one particle systems. It seems working so far:

http://cid-a6597afda81373ba.skydrive.live.com/self.aspx/tmp/multipsys.mov

That video is captured in real-time. You can see two particles mixing each other.

Maya native PDC doesn't work, because the field won't update.

What is "square artifact on particles when using big number of particles"? During simulation or rendering? Using realflow export? What renderer are you testing with? So far I only test it with RenderMan. Krakatoa users may try Maya-To-Krakatoa Exporter:

http://code.google.com/p/m2k/

But I haven't tested that.

JulianJohnson
04-20-2010, 10:07 PM
well done Dr it's looking really good. looking forward to trying it soon :thumbsup:

Als
04-21-2010, 05:18 AM
Testing with one fluid moving more-than-one particle systems. It seems working so far.

How do you set that one up? Can one do it now, or this is your current version? I have to try it.


What is "square artifact on particles when using big number of particles"? During simulation or rendering? Using realflow export? What renderer are you testing with?

But I haven't tested that.

I'm using 3Delight.
The artefacts are same as when maya moves particles with fluids.
Set beyonce to zero instead of 2, and crack up number of particles, it is very obvious,
both during simulation and rendering. I've rendered without motion blur, just a test render,
so without caching. I tried to cache with realflow export bin, and that seems to be working.
I can not show the render, because it's for the job I'm on at the moment.
I'll try to set one up with different emitter to illustrate (when I grab some time)

Thanks


Als

zsmoke
04-22-2010, 08:54 AM
http://www.youtube.com/watch?v=8fPq-rE3e50

I have uploaded tutorial video about some basic zFluid operations and a 400K points test. A bit slow on my laptop, but still makes sense. It shows the latest version of the plug-in, a few improvement from the last update. And the feature of attach a field to two particle systems is not available with the current release.

Now most of the solver is done, I can focus on the rendering side.

Aikiman
04-22-2010, 09:56 PM
Will this work with auto-resize fluid in 2011?

zsmoke
04-29-2010, 04:11 PM
Still have no idea what "auto-resize fluid" means:( Will find out.

http://i41.tinypic.com/4q3djn.jpg

larger image (http://public.bay.livefilestore.com/y1p5M38lu4ojD4wUVHDiTq6zD2_XuMRObQ1wo6lif0nPDxmDrntMCm1B1txu9sO7eA7KtCVj3vxau92cUAuDQUVcA/Snapshot%202010-04-29%2023-18-26.jpg)

Just worked out the DSO, Viz, RIBGen to render particle cache saved by zFluid in 3delight. About 1.2 million points are saved in 4 passes and aggregated and rendered with deep shadow and motion blur. Just a quite test. Will do heavier tests tomorrow.

tkdmatt
04-30-2010, 01:56 AM
I'm having a lot of fun with this in Maya 2010 x64, and would like to render out the particles. However the particle caching doesn't seem to be working properly, I tried caching the zFluid then caching the particles, but the particles just appeared to move as they usually would.

Is this not supported yet or am I missing something?

zsmoke
05-01-2010, 10:28 AM
Maya PDC command simply does not update the field as expected. I have worked out Field Cache as a solution. So those vortex will end up in same places each playback. Here is the showoff:

http://www.youtube.com/watch?v=zhu5B69sUEM

During first 3 playback, flow changes a lot. After saving Field Cache, it does not change, because using the same (velocity) field read back. Of course position of individual particles varies slightly.

Field Cache does not allow jumping to arbitrary frame like PDC does. So it is not a replacement. Particle Cache will do that. I will capture another video about that.

zFluid Pro is also ready now. Anyone want to have a pre-release test? Please contact me via

zhangmdev@gmail.com

zsmoke
05-02-2010, 07:54 PM
http://www.youtube.com/watch?v=h8W9lEgXT7k

Video about saving and loading particle cache. That setup already reused existing Particle Cache. What created after load back is not particle, it is a custom locator showing the data.

Next will be example rendering the cache in 3delight.

zsmoke
05-04-2010, 05:19 PM
http://i41.tinypic.com/eu0o3q.jpg

Everything seems working now. Did some surreal test today. Uploading a video about how to setup rendering now. Going to render animation at higher resolution to see how it moved.

azshall
05-04-2010, 05:30 PM
http://i41.tinypic.com/eu0o3q.jpg

Everything seems working now. Did some surreal test today. Uploading a video about how to setup rendering now. Going to render animation at higher resolution to see how it moved.

Amazing. How many particles in total are you simulating and rendering?

zsmoke
05-04-2010, 06:02 PM
It is from a 150 frame run, about 1.2 million particles simulated at the end. Particle Cache saved in 6 passes. So there are maximum over 7 million points. Of course at the middle there are less.

http://www.youtube.com/watch?v=SNUoS4AyGnw

That is the video I captured about how to setup the render. You can see how easy it is!

This is the final stop. I am working on the doc and demo animation. zFluid Pro is ready for business now.

zsmoke
05-05-2010, 02:58 PM
http://www.youtube.com/watch?v=VawnwbKBog8


this is the rendered animation in HD.

KidderD
05-05-2010, 03:54 PM
Amazing fidelity! Just beautiful. Great work. Curious to see the price and what other applications this could be used for, and if I ever have a need for crazy wispiness like this.. Wow! Can your particle cache be used to add detail to a nParticle SPH sim? Or is it strictly through your gpu fluid solver? Or are there future plans for such a thing, I'm sure collisions are a completely different story, but just curious.

Congratulations

zsmoke
05-05-2010, 04:48 PM
This is good to simulate tiny particles floating in current, i.e. smoke in air, and ink in water. Splashing water droplets is not good for this i am afraid.

I have no follow-on plan so far, all depends on how this turns out to be. I am not an artist, but I am sure people can dig out a lot interesting stuff with such a tool.

More info:

http://cid-a6597afda81373ba.spaces.live.com/blog/cns!A6597AFDA81373BA!1094.entry (http://cid-a6597afda81373ba.spaces.live.com/blog/cns%21A6597AFDA81373BA%211094.entry)

CGTalk Moderation
05-05-2010, 04:48 PM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.