Hello,
From a thread “Where to start with BIG projects?”
(I made a new thread because it’s a different topic)
Originally posted by playmesumch00ns
[B]Maya uses lazy evalutation implemented by a “pull” method using the concept of dirty plugs.i.e. every plug on a node has a dirty flag (actually it has several different dirty flags, but that’s not important now)
[/B]
What are the several different dirty flags?
I’m interested in the deep inside of the dependency graph.
The way it works is as follows:
You have nodes in maya, each with input and output [plugs] values (attributes).
Say we have a hierarchy with nodes A, B, C and D. A feeds value X into B and B feeds value Y into C aswell as Z into D.
Hope you can picture that mentally. As each connected node depends on its input plugin to calculate its output value (in this case), you need to know when a plug is “dirty”. If for example A changed its output value (dirty), indicating to Maya that B needs to get a new value from A and recalculate its output values (Y and Z). Obviously then C and D need to recalculate their values too (as B’s output plug is dirty). Each time a this is calculated, the relevant plugs are set to “clean”.
However, assume that only B changed its internal value somehow. Now with the dirty plugs Maya will only need to recompute B, C and D, as A is not affected by B’s output, and A’s output value is simply requested, not recomputed.
This helps save processing power by only recalculating in those nodes that actually need to be recalculated (and only those routines that require the dirty values). In a large DAG this can save quite some processor power.
Hope that answered your question and that it was clear enough.
Hi,
There seems to have two types of dirty flags.
MEL command “isDirty” has options
-d/datablock
(Check the datablock entry for the plug)
and
-c/connection (default)
(Check the connection of the plug)
“isDirty -c ***” and “isDirty -d ***” often return different value. I don’t understand how
-c option differ from -d. I thought playmesumch00ns’s comment is something to do with this difference.
Sorry, wasn’t aware of that discrepency. Where I assume the difference lies is that the datablock for a node is where its internal attributes are stored. Whereas the connection is the link between two nodes and their attributes.
If a datablock is dirty, then that attribute’s dirty flag will propagate to the next node in the DAG, causing the connection to become dirty.
That’s how I understand it anyways. A node can have clean datablocks, but may have a dirty connection from a node with a dirty datablock. The word lazy comes from the fact that if you require a value from a node at the end of the graph (leaf) then you would only propagate back as far as the last dirty node. Say in the eample above: A, B, C, D. You wanted to recompute C and all other nodes were clean. C would simply just take its stored value from B and use that, as B isn;t dirty there is no need to recalculate that. Had B’s value been set as dirty, B would have to recompute its value, which also means checking that it has A’s latest value… and so on…
It’s basically a pull approach, pulling data out of the tree from the leaves, rather than pushing it through the tree to the leaves. It saves processing time where it’s not needed…
Originally posted by Jhavna
Sorry, wasn’t aware of that discrepency. Where I assume the difference lies is that the datablock for a node is where its internal attributes are stored. Whereas the connection is the link between two nodes and their attributes.
If a datablock is dirty, then that attribute’s dirty flag will propagate to the next node in the DAG, causing the connection to become dirty.
Oh, thanks for the help, Jhavna.
I tested again and got a curious result.
The test I did is like this:
1)Make a test plug-in that has two nodes “.input” and “.output”
“.input” affects “.output”
2)Create three test nodes A, B, C.
Connect A.output to B.input and
connect B.output to C.input.
3)setAttr A.input 1 then
getAttr A.output
getAttr B.input
getAttr B.output
getAttr C.input
getAttr A.output
4)See how dirty plugs are set/reset during 3)
Though I’m not 100% sure, the result implies “isDirty -d” returns
the state of dirty flags and “isDirty -c” returns the state of propagation bits.
i.e.
“isDirty -d” is “If the plug needs to be updated”,
and
“isDirty -c” is “When the plug gets dirty, if the downstream nodes needs to be checked to propagate dirty”.
This is the raw data I got
here
A.output 0 1
means
the result of “isDirty -d A.output” is 0 and
the result of “isDirty -c A.output” is 1
after “setAttr A.input 1”
A.input 0 0
A.output 1 1
B.input 1 1
B.output 1 1
C.input 1 1
C.output 1 0
after “getAttr A.output”
-d -c
A.input 0 0
A.output 0 1
B.input 1 1
B.output 1 1
C.input 1 1
C.output 1 0
after “getAttr B.input”
-d -c
A.input 0 0
A.output 0 0
B.input 0 0
B.output 1 1
C.input 1 1
C.output 1 0
after “getAttr B.output”
-d -c
A.input 0 0
A.output 0 0
B.input 0 0
B.output 0 1
C.input 1 1
C.output 1 0
after “getAttr C.input”
-d -c
A.input 0 0
A.output 0 0
B.input 0 0
B.output 0 0
C.input 0 0
C.output 1 0
after “getAttr C.output”
-d -c
A.input 0 0
A.output 0 0
B.input 0 0
B.output 0 0
C.input 0 0
C.output 0 0
That looks fine.
Let me try to explain (thanks for the handy example ;))
(if you set a value, it will always be clean in the node as that’s where it has been changed)
after “setAttr A.input 1”
A.input 0 0 (no inputs, so both are clean)
A.output 1 1 (out of a is dirty, which is connected, so propagate. The value is dirty as out is affected by in value)
B.input 1 1
B.output 1 1
C.input 1 1
C.output 1 0 (no out connection, so no need to propagate, but value is dirty)
after “getAttr A.output”
-d -c
A.input 0 0
A.output 0 1 (only output is dirty so propagate, value inside is set, so it stays clean)
B.input 1 1
B.output 1 1
C.input 1 1
C.output 1 0
after “getAttr B.input”
-d -c
A.input 0 0
A.output 0 0 (A is not affected by B so all A is clean)
B.input 0 0
B.output 1 1
C.input 1 1
C.output 1 0
.
.
.
The rest is similar to above… you get the picture…
.
.
.
after “getAttr C.output”
-d -c
A.input 0 0
A.output 0 0
B.input 0 0
B.output 0 0
C.input 0 0
C.output 0 0 (as C is not connected, no need to propagate)
Hope that’s an OK description for you…
This thread is helping me to get even more aquainted with all this… thanks 
Hi, Jhavna
I think I am getting closer to the final understanding
but my empty head refuses working:surprised .
In my understanding, when isDirty -c flag is set,
downstream dirty propagation is skipped to save CPU time.
So it should only be set when downstream connected nodes are ALL already dirty.
but I found a case it is set when one of the downstream node is NOT dirty.
In such case, when an upstream node gets dirty, dirty propagation should be
skipped and dirty states of the downstream nodes should not be changed.
But the dirty propagation was not skipped when I tested. The dirty
propagation is what the dependency graph should behave, but I can’t understand
why it is propagated properly, ignoring the isDirty -c flag.
This is the detail.
http://hohehohe2.hp.infoseek.co.jp/dirtyprop.html
Your comments on my test data sounds
isDirty -c flag is set when there is a dirty downstream nodes, not
ALL of the downstream nodes are dirty.
Is that right? If so, how does Maya use the flag?
Koichi
Unfortunately the image on your site is referencing an image on the local hardrive, so I can’t see it…
But lemme try to use an example
say we have the following
A is root. B and C are connected to A and D and E are connected to C.
Now, I update the value in A. A will not be dirty (no need to recalculate) as it generated the value. Downstream connections to B and C are set to dirty as B and C both require an up-to-date value from A in order to produce their outputs.
B has no downstream connections and thus needs not propagate its dirty flag. However the output datablock will be dirty because it requires recalculation in order to provide the current data.
C doe shave downstream connections and these are flagged dirty to propagate the changes to D and E, who will each behave like B above.
Now assume we just change C.
A will not be affected by C changing as it is upstream from C. B will not be affected either as it is not connected to C at all (and A does not need updating). C’s input value is not dirty as it generated that itself (or it was set).
C does have downstream connections that will need to be updated and thse are set to dirty, as are both D and E nodes, as they rely on C’s output in order to produce their output.
Let me know when your picture works so I can have a look at what you mean. Can’t really visualise what you meant in your post…
EDIT: Just saw your image. What is happening is that when you add D, B is already dirty, and D will pull out B’s output datablock (set to clean (0)) and there will be no need to re-calculate D as it has B’s current value and was added after B’d propagation flag was set.
Then, when you update A again, You set the dirty propagation in motion and it behaves as it should…
Hope that answers your question…
Originally posted by Jhavna
Unfortunately the image on your site is referencing an image on the local hardrive, so I can’t see it…
But lemme try to use an example
…
Let me know when your picture works so I can have a look at what you mean. Can’t really visualise what you meant in your post…
GAAAAA! sorry. I’ve updated it. please don’t tell it to my mum.
I haven’t read your comment. I’ll post again when I read through.
Hi, again
Well, I guess you have a clear picture.
but anyway please let me explain how my poor brain is tangled up, a bit more logically. Sorry for the long comment.
a) Maya’s dirty flag saves CPU time for multiple node evaluation as you pointed out first.
b) If an upstream node(plug) is changed, all its downstream nodes(plugs) are set to dirty.
c) If there are a thousand of downstream nodes, CPU time for b) can’t be ignored. So Maya employs another flag called “propagation bit”. It is used to prevent multiple dirty propagation b).
(According to “The inside Maya architecture” DVD)
d) I couldn’t understand how a propagation bit is used. I thought I would do the next way during process b), instead of using propagation bit.
e) When I saw your second comment in this thread, I thought propagation bit is more efficient than my way in the next case.
Because when the plug gets dirty, Maya has to look at the dirty state of one thousand downstream plugs. If A.out has its own propagation bit indicating “all downstreams of A.out are already dirty”,
Maya doesn’t have to look at the dirty state of one thousand downstream plugs.
d) So I did the first test (A - B - C). The result was satisfactory to me. Even if an output plug (e.g. B.output) value is getAttred (so cleaned), it’s isDirty -c is not cleared. If isDirty -c returns its propagation bit, it means that “B.output’s downstreams are still all dirty, so Maya doesn’t have to set them dirty again when B.output gets dirty again”.
e) I couldn’t understand your comments on my raw data (I confess).
f) So I did the second test. What if I create another node(D) and connect it to B?
g) I tested (A - B - C,D). The result was not satisfactory to me. Here I’m lost.
If you know where I’m first lost, please let me know.
Koichi
Originally posted by hohehohe2
[B]
f) So I did the second test. What if I create another node(D) and connect it to B?g) I tested (A - B - C,D). The result was not satisfactory to me. Here I’m lost.
If you know where I’m first lost, please let me know.
Koichi [/B]
I guess you didn’t see my edit from above
Just saw your image. What is happening is that when you add D, B is already dirty, and D will pull out B’s output datablock (set to clean (0)) and there will be no need to re-calculate D as it has B’s current value and was added after B’s propagation flag was set.
Then, when you update A again, You set the dirty propagation in motion and it behaves as it should…
Hope that answers your question…
But anyway… I am starting to doubt if I am correct. But you have to understand that there’s a push-pull approach in Maya too. Push is, as you rightly say, where a upstream node (nearer the root) propagates it’s flag to the next downstream nodes and so forth.
The pull approach is when you request a value from a downstream node. The node will either give you its value straight away, or, if it’s dirty, pull the last clean value along the tree, evaluating it on the way as it goes through each node. Say if you were to ask D for its value. If D’s value is dirty, then it will go to its upstream node for the latest value… if that is also dirty that node will go to its parent and so on. If D’s parent is clean, its parent will simply give D the value and D will recalculate and give you its output value.
Hence the DAG paths will only have to be evaluated to the point where the next clean node is.
In an example say you have a transform matrix, a scale and a deform node attached to a geom node (in that order, with the matrix being root)
Now if you change the matrix, the change in the matrix’s values will have to be propagated right through to the geom node in oder for you to see the updated geometry correctly. If you change the deform node, the matrix and scale remain unaffected.
Say however you changed the matrix and didn’t update the viewport. Then the marix would be dirty and the mesh would be as it was before you changed the matrix node. If you then ask Maya to update the scene, maya will go to the geom and see that it has a dirty input. So the geom node will ask its parent (the deform) for its current value (as the geom only knows that its parent is dirty). The geom sees that its input is drity and requests an update from the scale node and so on.
If the same was done by simply changing the deform node, the geom node would see its input is dirty and request the deform node’s value. The deform node sees its input is clean (no upstream nodes have been changed) and then gives its output value without the need of recomputing it.
This saves time as only values that need recomputing are recomputed.
In your example number 2 you add D after B has been changed. As you see B is set to (0, 1) indicating its datablock (internally stored data) is clean, but C is still dirty and hasn’t been updated.
This means that D will get B’s most current and up-to-date value so no recalcualtion is needed on that node link. If you asked C for its out value, it would see that it needs to get a current value from B and request it.
Later, when you update A again, D becomes dirty because B has changed its value (1, 1) and therefore D will need to get B’s updated value.
God, I hope I am making sense… It’s starting to make my brain hurt now too
The basic rule in my experience is that Maya will do everything it possibly can to avoid updating. This is good if you’re a user, but bad if you’re a developer and want to see what effect your code has…
Usually you can force a pull just by calling getAttr on an attribute that has the proper attributeAffects relationship set up.
Originally posted by Jhavna
I guess you didn’t see my edit from above
Oh, sorry. I did see your edit, but couldn’t understand, so
I tried to explain what’s in my head.
there will be no need to re-calculate D as it has B’s current value and was added after B’s propagation flag was set.
This is where I don’t understand. Is propagation flag used? How?
In your example number 2 you add D after B has been changed. As you see B is set to (0, 1) indicating its datablock (internally stored data) is clean, but C is still dirty and hasn’t been updated.
This means that D will get B’s most current and up-to-date value so no recalcualtion is needed on that node link. If you asked C for its out value, it would see that it needs to get a current value from B and request it.
Here again, how is propagation flag used?
If I look at only dirty flag (i.e. isDirty -d), it
satisfies Maya’s push-pull approach perfectly.
All my tests indicates that a propagation flag (i.e. isDirty -c) is set when
there is a dirty downstream node connected. And it’s probably what you tells me.
But if propagation flags are used like my previous post, it must be set when
“ALL” the connected downstream nodes are set dirty, not just
there’s one dirty downstream node connected. Or Maya has to check all the
connected downstream nodes anyway to see if they are already dirty, during dirty propagation.
Does it make sense?
Originally posted by playmesumch00ns
[B]The basic rule in my experience is that Maya will do everything it possibly can to avoid updating. This is good if you’re a user, but bad if you’re a developer and want to see what effect your code has…Usually you can force a pull just by calling getAttr on an attribute that has the proper attributeAffects relationship set up. [/B]
Hello playmesumch00ns.
Yes, we must be careful when we make a little bit complecated node and connection.
By the way is this topic about what you say “actually it has several different dirty flags”?
There’s a dirty bit on the data block. If the data is dirty, then when it is requested by value, it will force a compute.
Each plug has a dirty bit which indicates whether the data coming into it is valid, and it also has a connection dirty bit which will be set when the data upstream has changed but the connection has not yet been traversed.
I think my 0.8g brain worked, finally.
To make things clearer I will call
datablock dirty — Ddirty
connection dirty — Cdirty
When Maya propagates Ddirty from upstream to downstream, it really doesn’t propagates Ddirty at a plug,
instead Maya set the plugs connection to dirty (Cdirty), indicating “Ddirty needs to be propagated through this connection to
downwards, but hasn’t been done yet”.
When Maya finishes traversing the plugs upstream (may not be accurate, depending on Maya’s dirty traversing implementation), it looks at its Cdirty and if it’s set, Maya starts traversing downstream to propagate Ddirty.
So Ddirty and Cdirty are used just the same way, but Ddirty is used to save nodes multiple calculation, and Cdirty is used to save multiple dirty propagation.
Is this O.K?
:buttrock: By jove, I think he’s got it :buttrock:
Yes, that’s the idea…
Hopefully this thread will serve others as a source of enlightement too 
Originally posted by Jhavna
[B]:buttrock: By jove, I think he’s got it :buttrock:Yes, that’s the idea…
Hopefully this thread will serve others as a source of enlightement too
[/B]
[B]
Ahhhh!!!
thank you, Jhavna, playmesumch00ns.
I really really appreciate for all your help![/B]
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.