PDA

View Full Version : Get attribute outside of Compute()?


VesuvianPrime
03-10-2011, 04:24 PM
Hi

Currently I'm using dataBlock.inputValue() stuff to get values inside my node. Is there a way to get attribute values without using the dataBlock?

Keilun
03-10-2011, 05:57 PM
Yup, use the MPlug class for retrieving attribute data outside of a compute method().

VesuvianPrime
03-10-2011, 07:31 PM
Thank you very much!

flaiver
03-11-2011, 07:45 AM
Keilun is absolutly right. But using this indicates bad design. Make very sure, that there is no alternative way to use the datablock, because its much faster and also: its made for it!

Robert Bateman
03-11-2011, 09:50 AM
Keilun is absolutly right. But using this indicates bad design. Make very sure, that there is no alternative way to use the datablock, because its much faster and also: its made for it!

citation needed. You may well be right (So if you have sources, I'd love to see them!), but....

I fail to see how one type of pointer (MPlug) is any slower than another kind (MDataHandle). I fail to see how using MPlug for it's intended purpose indicates a bad design.
I fail to see how deliberately forcing a compute step, inturn forcing maya to re-package up the node data into an MDataBlock, so the you can access a single variable via an MDataHandle is somehow more preferable to simply using MPlug.

flaiver
03-11-2011, 10:10 AM
Hey Rob

you are right ... the access of the MPlug might or should be at least a compareable speed as the MDataHandle. But what I had have in mind was the "creation" of the MPlug itself. E.g. if your are using findPlug, which is quite a expensive function in my opinion.

I see no reason to use a MPlug if you just want to access static members - that I want to say mentioning bad design. Yes - there are reasons e.g. if you want to access world vertex position of mesh input etc.or also using dynamic members but in my experience, this happens very rarely.

But please let me know, if your experience differs in this points.

Best

flaiver

Robert Bateman
03-11-2011, 12:30 PM
you are right ... the access of the MPlug might or should be at least a compareable speed as the MDataHandle. But what I had have in mind was the "creation" of the MPlug itself. E.g. if your are using findPlug, which is quite a expensive function in my opinion.

If you pass in text string to findPlug() then maybe, however if you are extracting the plug using it's MObject, it is impossible for this to be faster or slower than MDataHandle.


I see no reason to use a MPlug if you just want to access static members - that I want to say mentioning bad design.

The only static members in Maya are typically the shared attribute descriptions (The MObjects constructed during the initialisation of the plug-in), which are static to prevent duplication of the attribute's meta data (long name, short name, is keyable etc). These act as nothing more than quick lookups (effectively pointer offsets) to the data blocks of the node instances. i.e. MPlug gets the base address of the node instance, and an offset is provided by the static MObject attribute description. Armed with both of those, maya can do a direct pointer dereference to get to the data. Doing anything else would be insane!

or also using dynamic members but in my experience, this happens very rarely.

It happens all the time.....


class FooNode : public MPxNode
{
public:

// The usual process carried out in initialise()
static MObject m_staticAttr;

// declare the attribute description common to all instances
static void addStaticFoo()
{
MFnNumericAttribute fn;
m_staticAttr = fn.create("staticFoo", "sf", MFnNumericData::kFloat);
addAttribute(m_staticAttr);
}

// grabbing the attr value from the MPlug of a node instance....
// Note: this cannot be a static function!
float getStaticFoo()
{
MPlug plug(thisMObject(), m_staticAttr); //< note usage!
return plug.asFloat();
}

// grabbing the same attr value within compute isn't too different....
float getFooFromDataBlock(MDataBlock& block)
{
MDataHandle handle = block.inputValue(m_staticAttr);
return handle.asFloat();
}

//-------------------------------------------------------------------------------
// Now the same process with a dynamic attribute
//
// This is a member variable, not a static variable!
MObject m_dynamicAttr;

void addDynamicFoo()
{
// construction is the same
MFnNumericAttribute fnAttr;
m_dynamicAttr = fnAttr.create("dynamicFoo", "df", MFnNumericData::kFloat);

// but adding the attr has to be done with the function set
MFnDependencyNode fnNode(thisMObject());
fnNode.addAttribute(m_dynamicAttr);
}

// this isn't any different to the static version....
float getDynamicFoo()
{
MPlug plug(thisMObject(), m_dynamicAttr); //< note usage!
return plug.asFloat();
}

//-------------------------------------------------------------------------------
// as an absolutely final one....
float getDynamicFooUsingFindPlug()
{
// yes, construction of this is expensive....
MFnDependencyNode fnNode( thisMObject() );

// but this is no more expensive than the previous method....
MPlug plug = fnNode.findPlug( m_dynamicAttr );
return plug.asFloat();
}
};

flaiver
03-11-2011, 12:51 PM
Thats interesting. Thank you for sharing your little foo example. To be honest, I am a little bit confused. If there is no advantage to use MDataBlock instead of using MPlug, why to implement two interfaces to access any data inside a node.
If you are right, and I am sure you are, there would be just little advantage for the MDataBlock implementation. Instead Autodesk have to do maintenance for two interfaces which doing more or less the same but you can use MPlug also outside of the compute function.

VesuvianPrime
03-11-2011, 12:57 PM
I'll probably get flamed again for using cmds buuuut

In Python, how would I use an MPlug attribute to connectAttr()?

Robert Bateman
03-11-2011, 01:56 PM
In Python, how would I use an MPlug attribute to connectAttr()?

Use MDgModifier (and MDagModifier for parenting changes)

Thats interesting. Thank you for sharing your little foo example. To be honest, I am a little bit confused. If there is no advantage to use MDataBlock instead of using MPlug, why to implement two interfaces to access any data inside a node.

This following is mostly educated guess work...... but I reckon it won't deviate much from this description.....

Anyhow. There are advantages, but speed is not one of them....

Take something simple like a file texture node (worth having a look at it's attributes!). It contains some image data, and also some input attributes that determine which UV coord to sample (which is returned at the output color). When you are using multi-threaded rendering, multiple threads will want to call this, but with differing UV values that expect differing results. So maya can't simply use the node data for this, because you'll have some nasty race conditions within the compute function (the UV coord, output colour, etc).

So, Maya *MUST* make a copy of this data block for each rendering thread! However.... making a copy of *all* the file texture data is not required. Infact, you only need to duplicate the attributes that directly affect the output (setup with attributeAffects in the initialise). So AFAIK, the MDataHandles can refer to data on the node itself, temporary input values for rendering, or can point further upstream to the result of a previous node's computation.

Copying data about tends to be an extremely expensive operation within DG's. Copying the result of node A, into the input attributes of nodes B/C/D, quickly starts eating CPU cycles. I'm fairly certain that the input data handles are therefore going to point to data on input nodes, and not data on the node that's being computed.

I think it's therefore reasonable to assume that the MDataHandles are marginally slower than MPlugs, but we're probably only talking about an additional switch statement at a guess..... At least, ..... that's the case for raw data access anyway.

If the attribute being queried is flagged as dirty, it will cause an evaluation of the attribute within the DG (something MDataHandle does not do). Therefore, if you were to access an MPlug within a compute function, and that happened to spiral off a DG evaluation, you could end up with never ending recursion until maya crashes.

If you are right, and I am sure you are, there would be just little advantage for the MDataBlock implementation. Instead Autodesk have to do maintenance for two interfaces which doing more or less the same but you can use MPlug also outside of the compute function.

Aside from threading, the only other reason for the existance of MDataBlock / MDataHandle is to easily let you schedule the order in which nodes need to be computed within the DG (in a way that doesn't spiral off recursive loops accidentally....)

Keilun
03-11-2011, 02:40 PM
That's my understanding of it as well, with the added explanation that this was a means to control memory usage.

The other aspect is that the data block offers a lower level interface to DG data than MPlug does. When querying MPlug for attribute data, it will lead to DG evaluations where necessary. For the MDataBlock you can request the memory allocated for an attribute's data without triggering computes (ie. the difference between MDataBlock::inputValue()/outputValue()). When calling inputValue you are always guaranteed valid data (possible evals triggered), while outputValue is only providing a handle to the memory with the understanding that it will only be used to write to. This is one of the reasons you don't want to use MPlug in a compute. While the DG is quite tolerant of this behavior, it's good form to use the data block directly here.

To me they're two interfaces to retrieving the same data, but are built with different contexts in mind. One is an external access to DG data (MPlug) and the other internal to a DG compute.

Robert Bateman
03-11-2011, 03:39 PM
The other aspect is that the data block offers a lower level interface to DG data than MPlug does.

I think they are at about the same level personally....

When querying MPlug for attribute data, it will lead to DG evaluations where necessary. For the MDataBlock you can request the memory allocated for an attribute's data without triggering computes (ie. the difference between MDataBlock::inputValue()/outputValue()).

Almost, but not quite.

Whether you query a value via MDataHandle, or MPlug, is entirely irrelevant really. The exact same number of CPU cycles must have been expended before that value is returned to you. Ok (bare with me!), let's say you call MPlug::asFloat(). It triggers a DG compute, which inturn determines the attributes that need computing, and then farms each compute step off to a different CPU core. Each step upsteam of the attribute is computed, and eventually the computed value is returned to you.

When using MDataHandle, it immediately returns a single value (without spawning a compute). Since the value is correct at that point, it implies that the upstream nodes *must* have been previously computed. Since this is within a compute step, something must have triggered that compute step, and the only thing able to do so, is MPlug.

So there really is zero difference computationally between the two! ;)


When calling inputValue you are always guaranteed valid data (possible evals triggered), while outputValue is only providing a handle to the memory with the understanding that it will only be used to write to. This is one of the reasons you don't want to use MPlug in a compute.

Which boils down to 1 simple reason: you must not let two threads write to the same data at the same time.


While the DG is quite tolerant of this behavior, it's good form to use the data block directly here.

It's anything but tolerant of this behaviour! When people are animating/modelling within maya, it doesn't appear that the DG evaluation is spread across CPU cores in the same way (It looks to me as though some individual nodes use threads in a jobswarm/parallel-for style, but seperate compute funcs do not appear to be called at the same time). So if you abuse the threading mechanism, it won't cause too much damage because Maya's running single threaded anyway.

If however you fire some scenes off to the render farm, suddenly, BANG! Maya starts using threading in anger, and those little race conditions suddenly rear their ugly little heads.... !

Keilun
03-11-2011, 04:03 PM
Almost, but not quite.

Whether you query a value via MDataHandle, or MPlug, is entirely irrelevant really. The exact same number of CPU cycles must have been expended before that value is returned to you. Ok (bare with me!), let's say you call MPlug::asFloat(). It triggers a DG compute, which inturn determines the attributes that need computing, and then farms each compute step off to a different CPU core. Each step upsteam of the attribute is computed, and eventually the computed value is returned to you.

When using MDataHandle, it immediately returns a single value (without spawning a compute). Since the value is correct at that point, it implies that the upstream nodes *must* have been previously computed. Since this is within a compute step, something must have triggered that compute step, and the only thing able to do so, is MPlug.

So there really is zero difference computationally between the two! ;)

A couple points:

1. By the time you have a MDataHandle, the evaluation has already occurred. Calling MDataBlock::inputValue triggers the extra computes to generate the handle. But yes, on the whole, for inputValues MPlug and the corresponding evals from MDataBlock are equivalent.

2. I was actually pointing out the difference between invoking the MDataBlock inputValue() / outputValue() methods. Calling MDataBlock outputValue will not try to guarantee valid data in that memory block. Instead it will just return you a reference to that block for write. There is no equivalent for that through MPlug. Yes MPlug doesn't need that, but that's what my point is - it's meant for a different context. Through my eyes that context is external access to DG data.


Which boils down to 1 simple reason: you must not let two threads write to the same data at the same time.

Hm, I was fairly certain DG evals were not multi-threaded yet. Various nodes thread their computes internally for faster fluid solves for instance, but on the whole I don't believe the DG eval fires off separate threads yet to handle isolated DG chains. Last I heard they're working on it, but I can imagine it's a tough problem to isolate DG "islands" of appropriate size for threading speedup.


It's anything but tolerant of this behaviour! When people are animating/modelling within maya, it doesn't appear that the DG evaluation is spread across CPU cores in the same way (It looks to me as though some individual nodes use threads in a jobswarm/parallel-for style, but seperate compute funcs do not appear to be called at the same time). So if you abuse the threading mechanism, it won't cause too much damage because Maya's running single threaded anyway.

If however you fire some scenes off to the render farm, suddenly, BANG! Maya starts using threading in anger, and those little race conditions suddenly rear their ugly little heads.... !

I was using the term tolerant pretty loosely, in that in simple scenarios, use of MPlug within a compute will not cause harm and will work (which I think ultimately winds up biting new Maya coders in the butt later on when they reach the less tolerant categories). I was actually somewhat surprised to see the OP ask for the other way around as usually most people are familiar with MPlug first and refrain from learning the data block when they move to nodes.

Kevin Picott once told me that there is a category of operations that can be done and that the DG will handle gracefully, but is still deemed illegal. This is one of those scenarios. Thats what I meant by tolerant. Intolerant in my books is never allow it to work at all even if you could handle it.

VesuvianPrime
03-12-2011, 12:17 PM
Back to the mDGModifier bit, when is the correct time in a node's creation to make the connection?

ticket01
03-12-2011, 01:03 PM
Hm, I was fairly certain DG evals were not multi-threaded yet. Various nodes thread their computes internally for faster fluid solves for instance, but on the whole I don't believe the DG eval fires off separate threads yet to handle isolated DG chains. Last I heard they're working on it, but I can imagine it's a tough problem to isolate DG "islands" of appropriate size for threading speedup.


The islands wouldn't be a problem since they could be identified in linear time when making node connections. You simply have to keep track of them. Since the DG cannot be altered during evaluation or animation the islands are static and could therefore be evaluated in separate threads.

Keilun
03-14-2011, 01:16 PM
Back to the mDGModifier bit, when is the correct time in a node's creation to make the connection?

Ideally you would have either a MEL script or command to set this up, either at node creation time or as a separate process if the node already exists. Although I think it's possible to do it in the MPxNode::postConstructor method as well, but I've never tried it myself so I could be wrong about that.

Keilun
03-14-2011, 01:20 PM
The islands wouldn't be a problem since they could be identified in linear time when making node connections. You simply have to keep track of them. Since the DG cannot be altered during evaluation or animation the islands are static and could therefore be evaluated in separate threads.

We could make lots of assumptions here and all it would amount to is speculation. Clearly there's a good reason the DG has not been multi-threaded yet, despite the dev team's efforts in the area. I know they've tried to do it, but had to scale it back due to complications.

ticket01
03-14-2011, 01:41 PM
Ideally you would have either a MEL script or command to set this up, either at node creation time or as a separate process if the node already exists. Although I think it's possible to do it in the MPxNode::postConstructor method as well, but I've never tried it myself so I could be wrong about that.

Using the postConstructor is a bad idea. This might work when the node is created but if you save this scene and re-load it, the node gets created and and since the connection is already set up and stored as such in the scene it would at least result in an error telling you that there's already a connection in place.

Worst case is that you create the connection on node creation but delete the connected node and try to re-load this scene. This would fail for sure.

Keilun
03-14-2011, 02:04 PM
Using the postConstructor is a bad idea. This might work when the node is created but if you save this scene and re-load it, the node gets created and and since the connection is already set up and stored as such in the scene it would at least result in an error telling you that there's already a connection in place.

Worst case is that you create the connection on node creation but delete the connected node and try to re-load this scene. This would fail for sure.

I agree it's not a good idea, but the thinking was you would disable the code during load scene. It's still an awkward workflow - you'd lose undo capability of MDGModifier as well. I'd recommend the script/command approach as that's how most things are done.

Robert Bateman
03-14-2011, 02:16 PM
We could make lots of assumptions here and all it would amount to is speculation. Clearly there's a good reason the DG has not been multi-threaded yet, despite the dev team's efforts in the area. I know they've tried to do it, but had to scale it back due to complications.

Hmm. So effectively you are saying that when I render a scene using 12 threads, Maya duplicates *all* of the data within the DG 12 times?? If that is true as you claim, that is truly appalling.

Seriously. Multi-threading this stuff is not rocket science - and is in fact trivial (as ticket01 has mentioned). Having worked on (and designed) a few commercial animation packages myself (that do multi-thread their computation), I seriously can't believe that the guys at Autodesk haven't been able to figure out a solution to this problem in over 15 years?

If you've got any links or resources that detail this, I'd be extremely interested in seeing them. If what you say is true, it would directly challenge the validity of some of Maya's patent claims - and that would make my day job significantly easier!!

Keilun
03-14-2011, 04:02 PM
Rendering is a different case. I can't say I'm as familiar with how the software rendering side works. I do know it's multi-threaded, but it works in an atypical DG fashion. From what I recall, when rendering it's not using typical DG data evaluation but rather evaluates on a per sample basis.

I think one of the big issues they're tackling is the subset of internal nodes that are not strictly DG legal in terms of data flow. Nodes that browse up the chain for data, etc. I imagine a lot of legacy issues are probably biting them, but like I said, only the devs themselves know the real issues and I'm fairly certain apart from select individual nodes and rendering, the DG is a single-threaded beast.

CGTalk Moderation
03-14-2011, 04:02 PM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.