PDA

View Full Version : new compositor - very early stages


buttachunk
11-13-2006, 06:44 PM
hey all,

looking to cut together a compositing app, which will be free, and most likely will have an advanced commercial version added later. it will be entirely based in OpenGL-- the UI, as well as the processing-- to increase speed dramatically over AE/Shake/etc.. The project codename for the app is; emergence, but that may change. the compositor will be node-based.

I already have much of the code written for processing blurs, blooms, upscaling, image loading, etc., and a base for the UI already implemented. may need a bit of advise about how to cut it all together (haven't made any freestanding apps in many years, this will be my first in OpenGL).

already have code for OpenEXR handling (read and write), and 8 bit, 16 float and 32 float RGBa happening. also already have HDR images able to be loaded with basic gamma correction in the basic app core, but am having problems with the control implementation. i have clocked the ability to load 211 frames of 16bit float images per second on my athlon x2 3800+ in the base app.

as this is my first real app, I'm wondering how many people in this forum would be interested in possibly helping out to get it started.

buttachunk
11-13-2006, 10:13 PM
no replies yet ?

here is a mockup of the GUI thus far (subject to change alot);

http://img.photobucket.com/albums/v251/buttachunk/emergencemockupcopy.jpg


it is very simple-- designed to take the GUI out of the artist's way. it's monochromatic focus is to draw the attention more to the images rather than the 'overall picture' perceived by the artist. it may include some python scripting capabilities in the pro version, but for now I'm focusing on building the app to work basically first, and to be streamlined and fast as hell.

emergence will be cross platform, due to its OpenGL / C++ core.

Maelcum
11-14-2006, 11:55 AM
Hi.

I've been playing around with the very same idea for a while but I thought it's too large task for one person only.
It's good to see that someone has guts to get into it :-)

Please let me know if I can help You in any way!

M

billrobertson42
11-14-2006, 03:23 PM
I'm curious.

What is a compositing app? What would a user do with it?

Thanks!

buttachunk
11-14-2006, 03:33 PM
Bill,


http://en.wikipedia.org/wiki/Compositing

Used for combining/layering images. Some are written in a 3d environment. After Effects, Shake, Nuke, Digital Fusion, Combustion, Flame, Flint, Inferno, Toxic, etc., are all commercial examples.

TAVO
11-14-2006, 03:57 PM
well, it looks very nice, i like the UI, i think you are right about the draw the attention to the work itself. Do you have a date to realease an alpha o beta stage ? Im not an advanced user, i use After effects but im starting with Fusion 5 and sounds that this app may work like this. Well i hope you kepp developing this app., ill be watching your thread to see how it goes. Good luck. :thumbsup:

billrobertson42
11-14-2006, 07:02 PM
Thanks for the link.

buttachunk
11-14-2006, 08:46 PM
Jesus, thanks for your reply. I'm currently exploring OpenGL more to make this compositor run as fast as possible. It won't have every function of some other compositing apps, which become slow and cumbersome at times-- this will be very streamlined and efficient, and as fast as possible.

I want the GUI to basically frame images like artwork-- like a painting,, not like a speadsheet, like many other compositors do. Also, in terms of the processing, it will be very artist-friendly-- like a moving canvas, with some unique tools to create a very different type of flow for compositing. I realistically hope to have a free alpha version available in December of this year.


Bill, you're welcome.

scrimski
11-14-2006, 10:31 PM
Interesting. No idea about programming but I would like to test it.

I would like to see controls rather below the preview window, which needs a possiblity the split up in two pictures(for better comparison) or the controls on one monitor and the preview on the other.

Subscribed

liquidik
11-14-2006, 11:08 PM
Ohhh gosh, I like to play from time to time with code, and I'm starting with OpenGl too. Can you point me to some paper or information on the same subject (make a compo application) ?

BTW your interface looks great to me!

Gian

buttachunk
11-15-2006, 02:33 AM
scrimski, thank you. yes, I eventually want the controls to be a transparent floating window. unfortunately, GLUT does not support this (as far as I know).

liquidik, thank you. yes, I am trying to make it very artist friendly-- like a moving canvas. many of the tools will be implemented differently than other apps.

rendermaniac
11-15-2006, 10:05 AM
If you are doing this with OpenGL are you going to do the actual operations using Cg (or equivilant GLSL etc) language? It would be really good if you had the option to write your own nodes.

I guess there would have to be another file to define the interface too.

I think the biggest hurdle with a node based system is getting your DAG working and actually connecting them together rather than the interface.

Looks nice though so far!

Simon

playmesumch00ns
11-15-2006, 10:29 AM
As simon said, writing a compositor is easy. It's writing the DG/caching stuff that's hard.

Since you're going to be leveraging the GPU for compositing operations, be careful how "flashy" you make the GUI: you don't want to waste fp cycles drawing buttons and sliders.

I'd start sketching up your node interface soon as well, and how that interfaces to the viewing window. You don't seem to have left anywhere for it to go in your mockup :)

You mentioned having pyhon support in the "pro" version. I'd strongly advise writing as much as possible in python and only writing the slow bits in C++. Most of the gui logic could quite happily live in python for example.

Good luck!

UrbanFuturistic
11-15-2006, 10:50 AM
What kind of help are you looking for? I can certainly use my C-Fu to dump GLUT and swap over to SDL. So long as you keep everything technically within the Window Manager window (or full screen) I can already think of a couple of ways floating semi-transparent windows and controls could be implemented with p-buffers or SDL surfaces.

Is this going to have audio support or is it purely visual?

Also, what dev platform are you using? Visual Studio, CBuilder, MinGW, Dev-C++?

djwarder
11-15-2006, 11:03 AM
Hey! Just a thought, but I remember starting a thread on 'node-based' programming for a 3d app ages ago, and there's lots of stuff in there that might be helpful ...

http://forums.cgsociety.org/showthread.php?t=176040

Enjoy!

H3ro
11-15-2006, 04:33 PM
Interesting project, looking forward to see how it will develop.

:)

buttachunk
11-15-2006, 04:49 PM
Thank you very much everyone !

Render; "It would be really good if you had the option to write your own nodes". Yes, they are GLSL currently, will be adding more Cg with the pro version (adding 3d camera, mappable geometry, 3d tracker and such). I will release the the app here (and elsewhere), in binary and source beginning soon (alpha, and later beta) to gain feedback. I will be creating an online repository for shaders/nodes during beta. "getting your DAG working and actually connecting them together". Yes, this is my bottleneck right now, and where I need quite a bit of help to implement properly.

Playmesum; "writing a compositor is easy. It's writing the DG/caching stuff that's hard". I agree. Currently, I have 3 classes for nodes; 1. creators (file in, perlin noise gen., particles, etc.) 2. shaders (bicubic filtering, blurs, gamma, paint, etc.) 3. layer (with simplish photoshop-style A/B overlay modes). These have all been simply tested by themselves, though not integrated yet. To gain max. speed, I am forgoing pBuffer callbacks to system RAM. Caching will simply exist in the form of writing nodes to disk (right click in node view), so no RAM caching will take place. The current flow is; Image in(from disk)>scene file input>downstream nodes (shaders)>render to screen. In the DAG, I will be creating 2 states for the nodes: view and edit. The user can happily view one node while editing another. "You don't seem to have left anywhere for it to go in your mockup" In the upper left corner of the image window, there are 3 buttons; image, node, graph. These will be selectable with numeric keypad (1,2,3), or by simply clicking on the buttons. The graph editor will also have a smaller image-viewing window. I don't want to have them all scrunched together for a variety of reasons. The primary reason is to have the image viewing area be more like a moving artist's canvas, free of spreadsheet-like clutter. "Most of the gui logic could quite happily live in python for example" Will look into this today-- have written more Python than C++ lately, but have an extensive library of C++ code, and not yet sure exactly how to integrate the Python-- will look into it. Cheers !

Odubtaig; "What kind of help are you looking for?" Feedback/suggestions are extremely valuable. I will be posting many more questions, and code, in the coming weeks. If others would like to help with coding/integration, I willl gladly add their names in the app's credits (if desired). This could be beneficial in many ways, including giving the app more accessible and open architecture. "I can certainly use my C-Fu to dump GLUT and swap over to SDL." Excellent-- would like to forgo GLUT for a variety of reasons, probably during beta phase. "Is this going to have audio support or is it purely visual?" My focus with this is purely visual throughout beta. If you want to add audio once the code is available, that would be very appreciated as well. "what dev platform are you using?" Visual Studio now,, looking at Novell's Mono for cross-platform.

DLWarder; "I remember starting a thread on 'node-based' programming for a 3d app ages ago". Thank you for the link ! Will be reviewing that info extensively.


Thanks again everyone !

T-bat
11-15-2006, 06:03 PM
Hi,
I'm writing node based compositor, too, but I'm not as far as you are. Mine is basic, but it's working except for viewer:sad: I'm writing it in hope of getting a job in the industry and it's a known fact that you automatically get a job when you write one of these, right?:D:scream:


And your compositior is looking good so far:thumbsup:

buttachunk
11-15-2006, 08:50 PM
H3ro and T-bat, thank you !

Update; looking into the Boost libraries, particularly graphviz and python for the node structure handling and for C++/Python integration.

Cheers !

UrbanFuturistic
11-15-2006, 10:21 PM
SDL is entirely cross-platform with Win/Linux/OS X. It's originally intended for more games/multimedia type purposes but a lot can be done with it, a lot more than GLUT anyway.

With something that relies so much on performance, I'm loath to go with something quite so OOP as Mono/C#, I'd prefer to stick with something a little more low level.

SDL very much negates the need for the Windows API compatability of Mono and adds the advantage of being Mac compatible. If you stick with OpenGL, SDL and standard C/C++ then cross compatability won't be too much of an issue. Visual Studio might be a sticking point as I only have a student license but then programming with SDL is a little different from your usual Windows programming, no winmain() for starters. Of course, once you get into cross-platform programming of this style, you're going to lose half the advantages of Visual Studio anyway.

The only real problem I've found with Xplatform OpenGL is doing anything past OpenGL 1.2 in Windows, all those extra bit to legitimise proper OpenGL commands are a PITA. :banghead:

buttachunk
11-15-2006, 11:04 PM
SDL is entirely cross-platform with Win/Linux/OS X. It's originally intended for more games/multimedia type purposes but a lot can be done with it, a lot more than GLUT anyway.

Sorry, the info I found about SDL at the time of writing my last reponse was a bit vague. Will definately look into it more. Will I have to go with Java or .Net though ?


I'd prefer to stick with something a little more low level.... Of course, once you get into cross-platform programming of this style, you're going to lose half the advantages of Visual Studio anyway.

Yeah, just looking multiplatform without spending a fortune. EDIT; Has anyone tried netbeans ?


The only real problem I've found with Xplatform OpenGL is doing anything past OpenGL 1.2 in Windows, all those extra bit to legitimise proper OpenGL commands are a PITA. :banghead:

Exactly-- must write different implementations for pBuffers and such in windows vs. POSIX.


Cheers !

buttachunk
11-16-2006, 05:54 AM
Update;

Rendermania, I decided to implement Cg sooner after thinking about it a bit,, not hard to add and opens up the app a bit more.

Thinking about messing with Qt right now to make life easier planning the app. May be able to create some node-based objects in there,, which could expedite the process quite a bit. The project would have to go GPL if I use it to create any of the app, though (no commercial license).

playmesumch00ns
11-16-2006, 10:42 AM
Re: "The only caching I'm going to have is right-click save on nodes...", what happens when all your images don't fit in memory? What happens when you try to composite 2 2k float images together and they don't fit on the GPU?

Your concept of having three node types seems a little odd to me. Presumably they're all subclassed from the same node. The only difference is the number of inputs they have. And then even the file in nodes have inputs for things like the filename.

Before you get started on the flashy UI, i'd get it working in script form first. Then you can concentrate on functionality and stability

UrbanFuturistic
11-16-2006, 11:06 AM
Sorry, the info I found about SDL at the time of writing my last reponse was a bit vague. Will definately look into it more. Will I have to go with Java or .Net though ?Nope, it's entirely C based, I've been using Visual Studio and GCC with handwritten makefiles.

buttachunk
11-16-2006, 04:05 PM
Playmesum;

"what happens when all your images don't fit in memory?"

this app will be optimized for fast cards w/ 256mb-1gb of onboard memory.


"What happens when you try to composite 2 2k float images together and they don't fit on the GPU?"

You're right-- system performance could be slowed by large images with alot of processing on them. Those trees can be written to disk. I may include a -write to ram- caching system later, but it will not be directly implemented into the flow of the app, as it will slow performance when not needed.


Also, sorry for the vague wording in my last response. I'll try to explain this better-- I have a rough simulator built for testing texture performance/PBO. I've simulated the following;

1. glTextureSub Image()
2. a PBO Image pusher
3. a double PBO, using two pixel object buffers.

I've found double PBO to be the fastest. Then simulated caching back to system RAM and found that appreciable (approx 50% when using glReadPixels, and 20-25% for asynchronous readback) decreased speed (on a humble system) due to the bottleneck of having to send the info back to system RAM and back to the GPU for display.

The two trouble spots for performance without RAM caching are; 1. playback will be slowed with added processing/layers, and 2. enabling the paint tools to save the frames (a later, optional -write to RAM- can be implemented for previewing, then -write to disk-) .

Also, about the different nodes-- correct, the 'creator' nodes have no input in the node flow-- some are image in nodes, and some will create geometry, etc.. I have yet to fully optimize the app's flow, but sorry for the confusing wording anyway..

So, where I'm at right now with the app is;
-must optimize the flow
-must design and implement the node-tree system
-implement the GUI
-EDIT; also thinking about a plugin-maker as a seperate app, for others to be able to add GLSL/Cg shaders as plugins, to semi-automate the implementation of additional needed coding

I've noticed several people with much more programming experience attempt a node-based compositor and stop,,, I don't want to get lost writing the backend or handcoding extensive GUI functions. Just trying to get it to alpha as soon as possible so people here and elsewhere can dig through it and hopefully make some changes/additions.

buttachunk
11-16-2006, 04:21 PM
Odub,

SDL sounds like an excellent option during beta once the app is up-and-running, but I don't want to get myself lost with a switch-over-- need to keep this as simple as I can until I have a working foundation. I've been concerned about the windows-centricness of Visual Studio, as I may be switching development of the app over to linux once the app is in alpha.

buttachunk
11-18-2006, 11:57 AM
Questions for everyone;

-what features would you like to be scriptable ?
-in python ?

any other features anyone would like to see in a compositor ?

playmesumch00ns
11-20-2006, 10:54 AM
I'd say just glue the whole thing together in python. Write the important stuff, like your GL library in c++, and your base node and graph classes, then derive everything else in python. Since all the script will be doing is handling the ui, setting up framebuffers and calling GL shaders you won't have to worry about speed.

This means every part of it will be customisable... it'll also make developing it a lot quicker too.

buttachunk
11-20-2006, 11:28 PM
Playmesum,

cool.

I'm thinking ahead to a final render engine for export since I'm not all that happy with OGL filtering for final renders... As Cg has many similarities to Renderman, I'm looking at the possibility of integrating CGKit for Python and Rman, as this would give the system an incredible amount of scriptability as well as add an excellent/fast/flexible/customizable render engine. This is subject to change, but I am reviewing the possibility extensively.

rendermaniac
11-21-2006, 03:54 PM
Playmesum,

cool.

I'm thinking ahead to a final render engine for export since I'm not all that happy with OGL filtering for final renders... As Cg has many similarities to Renderman, I'm looking at the possibility of integrating CGKit for Python and Rman, as this would give the system an incredible amount of scriptability as well as add an excellent/fast/flexible/customizable render engine. This is subject to change, but I am reviewing the possibility extensively.

It would be very nice to have another visual shader editor - the problem is the same as a compositor (it could easily be argued it is the same problem - and a hard one too).

An OpenGL preview would be especially nice as the only one I know that does this right now is Houdini.

One thing that could be a problem is that most RenderMan renderers expect their textures in prioritory formats. Usually these are tiled TIFF files or based off them - just most other software only ever bothers with scanline TIFFs. This extra conversion step could be a problem.

I think Gelato and Mental Ray(?) may let you write your own texture import plugins. Neither is RenderMan complient.

You may also be able to optimize more if you keep everything 2d - although saying that it seems that most compositing packages get some sort of 3d projection system eventually.

Simon

PS I agree with Anders about having it set up as a python API with a GUI on top - makes it very flexible.

rendermaniac
11-21-2006, 10:56 PM
Actually thinking about it, you probably could support other file formats by using a shadeop dso. However you'd then need to deal with filtering/memory etc yourself - which is why most people don't bother. At least prman 13's SIMD dso's let youdo this more efficiently than before. You'd also need to compile it for each renderer that you want to support.

Simon

buttachunk
11-22-2006, 05:16 AM
Odub,,
I finally got a good look at SDL-- very attractive. May be going in that direction for further development. :)


Anders and Simon,,,
Agreed-- I'm knee-deep in Python at the moment. I have scripts ready for camera shake and the like,, trying to figure out how to include Python scriptability into the GLSL/Cg shaders, while simultaneously making them animatable in the graph editor, and yet keeping them user-writable and editable...

note; the Rman compliance wouldn't be for the 1st gen. of the compositor,,, definately down the road a bit. may be including a proprietary Rman-compliant renderer with the app at that time...

playmesumch00ns
11-22-2006, 09:57 AM
I don't think having a seperate final renderer from what you're using in the gui is a good idea. It means you have to have 2 different render paths. Trying to retarget a 3d renderer to doing 2d images just for the sake of having a shading language is a bit of a waste: after all a well-constructed dag is a visual programming language. If you have a good set of base components you can do almost anything you want, and you can always expose a node that lets you put in custom fragment shaders.

I don't understand your concerns about OGL filtering. Surely you don't want to do any filtering at all? There's a chapter in GPU Gems II by the people who developed Apple's Motion that includes some information on making sure the GPU did predictable pixel lookups, perhaps that's what you mean?

pmsc.

buttachunk
11-22-2006, 03:13 PM
Playmesum,


the app is eventually going to have a very robust 3d system, with .obj import, lights, 3d camera, 3d tracker, HDRI import (and painting), deformable meshes, etc.. I already have code for this stuff at least half-way written, just not implemented yet. I have 32x32 bicubic filtering ready to go in GLSL,, which should be more than adequate for 2d,,, will need more options for more advanced 3d stuff later. This possibility is being considered due to several requests to make the app able to render on a renderfarm without OpenGL hardware... Again, this is a long way down the road, not version 1...

For Version 1, I have code now for creating nodes, defining edges and noodles, etc.. I have yet to complete the design of the plugin architecture to include the needed gui elements, graph editability, and Python scriptability. Also, still need to create and implement the graph editor...

buttachunk
11-22-2006, 04:13 PM
also, regarding videocards, I did a bit of research about inexpensive cards and found a card with 512m of ram, memory bandwidth of 10.7 Gigs/sec, fill rate of 2.8 billion pixels per second, and an HDTV output for $115 USD (!). The 256mb ram version is $80. 256mb will likely be the minimum requirement for doing HD with the compositor. These cards are SLI capable (as the compositor will be).

I would like to take advantage of the HDTV outputs on these cards for previewing composites without additional hardware.

bernieLomax
11-23-2006, 03:07 PM
hi everyone, i kinda stumbled across this thread and i have to say that i dont have a clue about graphics programming. But I think the most commom while developing a program inside the opensource community is this:
it is made by a programmer for a programmer.
Please dont take me wrong. This is problem but most of all a challeng to be mended throug the software development pipeline.
Most of the compositng software as similiar, and im speaking only in terms of th GUI, the rest i dont know much about. So here is my suggestion forget about the compositing gui and think about the opossite. For instance how come you have to manage a node system which i think is one of the best inventions ever, why wont you have those nodes in a system kinda similar to MACOSX tiger Dashboard or expos - its a great user interface.
Also think that we use both hands to work and not only the right one. try to make somthing similiar like mayas hotbox for menus like color correction and filter like that.
bare in mind that these are only suggestions and i dont even know how to make them work for you. like i said im not a programmer only an end-user.
About your software is it going to be opensource?
i hope.
hugs bern

buttachunk
11-23-2006, 03:19 PM
Bernie,

yes, I am considering several keyboard shortcuts for accessing the image, node, and graph windows, as well as play, stop, go the beginning, etc., so it can be a 2 handed process.

also, I mentioned previously that if I use Qt for creating the program, I will be bound to a GPL license with the software. Not sure if I'm going to use Qt yet... I would actually like to make it open source for the first few releases, then offer updates for free. I will also be making a commercial compositor after with some of the same components (maybe compatible), with additional features.

rendermaniac
11-25-2006, 05:59 PM
Not to confuse matters, but you might want to consider wxwidgets too http://www.wxwidgets.org/

Simon

buttachunk
11-27-2006, 02:32 PM
Render,

yes I have d/l'd wxwidgets to look at as well. also currently checking out freemat, the freeware equivolent to matlab for some additional processing (it includes a compiler for apps/processors made within matlab/freemat).

I'm hoping to have a serious portion of the code posted by mid-december.

buttachunk
12-05-2006, 08:59 AM
sorry for the delay,, just bought an intel mac pro. still working on the app development. OSX has some very interesting development tools-- XCode, an OpenGL shader builder/tester, Quartz Composer, which appearantly can be used to create FXPlug plugins for Motion and Final Cut Pro, etc..

has anyone seen conduit ? sheesh ! $150 for an openGL node-based compositor in motion and final cut ! I've heard it's a bit noodly to get around in though...

AlexeiPuzikov
12-07-2006, 11:33 PM
Hello,

Extremerly interested in your compositing endevour. Basicly thinking of starting my own.

And - don't listed to people that claim that DAG traversing is hard. I implemented such in ShaderMan during 2 evenings (actually, during a weekend). Just good planning, a probably a lil bit of OOP - but you're on C, so - good planning.

Please keep us informed, and you can count on me regarding the testing and publicizing.

Alexei Puzikov
renderman.ru - co-creator, admin
dream.com.ua - ShaderMan and other stuff
[unnamed studio in Canada] - pipeline


has anyone seen conduit ? sheesh ! $150 for an openGL node-based compositor in motion and final cut ! I've heard it's a bit noodly to get around in though...

buttachunk
12-11-2006, 07:41 PM
Alexei,

thank you for your kind words. Shaderman is certainly an inspiration for this program.

The date for posting code is being moved back, approximately a month (mid January) due to switching platforms, and a commercial project in the works.

AlexeiPuzikov
01-30-2007, 11:47 PM
The date for posting code is being moved back, approximately a month (mid January) due to switching platforms, and a commercial project in the works.

Any news since then? Wonderful new MacOS X Universal Build Alpha? :)

scorpion007
02-06-2007, 02:42 AM
If you're going to open the source, why not register it as a project on SF.net or similar? They provide source control (CVS, SVN) so that will encourage more developers to contribute patches, etc.

(I do hope you're currently using version control too.)

playmesumch00ns
02-06-2007, 10:50 AM
scorpion007's right. Stick it on the subversion server on sf.net. It'll make your life a LOT easier if you're not using version control already, plus then you know that your code is safe, even if something nasty happens to your machine :)

gamedeveloper
02-06-2007, 05:26 PM
Thought this website might be a good resource for you:

http://www.designinginteractions.com/

Especially as you begin to refine the interface/workflow.

CGTalk Moderation
02-06-2007, 05:26 PM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.