New Digital Painting Technology

[DarkTure]

That looks pretty cool :)

Good luck


edit: Just checked the bunny movie and it looks great!

[-Vormav-]

Looks very cool. I'd want to see how fast it responds at higher resolutions...or would the way that the simulation is handled make it mostly resolution-independent?
Anyway, I thought the video was pretty impressive.

[Nelson Chu]

Thanks for your comments. :)

would the way that the simulation is handled make it mostly resolution-independent?

The ink simulation is resolution dependent, at least for now. The whole paper is processed even when parts of it are dry. The simulation is done on the GPU, and currently it's not too easy to do some very high resolution simulation yet. But anyway, I hope in a year or two, most digital painters would be equipped with a good enough GPU.

[kraal]

i will say that i think the paint brush on the screen needs to go.

Is that all it simulates is the chinese brush ink effect? If so it does a good job but i see that as just being novel and a 'neat trick' but not really something to run out and buy.... oh well that is just my opinion I still use pencil and paper when i want a pencil sketch. so i dont see a digital app for one effect promicing. plus some of the articfacts emulated i switched to digital to get away from :)

just one artist's opinion

[Nemoid]

as a traditional artist , i'd say nothing for now can really substitute paper and pencil, ink and other real stuff. for example : comic nbook artists keep using pencil and ink , then scan pages and use computer for coloring, because its easier to color with it than doing it by hand. but the base work is traditional.there's a different feel : the surface of paper, the sweetness to use a brush on it, or a pencil ,these are tactile that are hardly substituted by a plasitic pen on a screen. reults can be the same one , tho. its a matter of habits.

BUT, in the future there will be good things. right now just using a Cintiq device and apps like this one would be fine. the same happens with painter and photoshop. i'm always inmterested in these new technologies for painting and drawing, maybe also combining traditional with digital.

so every new app is welcome !

[Crazy Max]

Nelson

I tryed emailing you at the address on your homepage but my email bounces back. :sad:
Is there an easy way to get in touch with you?

Thanks

[Blur1]

Hi, just wanted to say, looks GREAT, the paintbrush looks cool, I like the way the brush gives you feedback on A) direction, B) pressure. Not a gimmick if you ask me. Please release for OS X too.

[PavelPetrenko]

Very cool. Indeed!

[el_diablo]

Very nice! I love to see interesting uses of current graphics hardware. I already read through some ink simulation papers few years back, but yours seems to be very balanced in complexity vs. speed. Also nice to see that tilt function of wacom really works if software uses it correctly. I never been able to get anything interesting in corel painter or photoshop using tilt. Whats the upper limit on hardware textures nowdays anyway? 4096? Might limit the usefullnes of the approach but prolly in some cases it wont. Also, i assume your approach might be usefull for simulating other painting methods, like oils or acrylic.

Anyways if you make a product of it, let us know!

[Nelson Chu]

First, thanks for all responded.

@Crazy Max: Sorry, I was afraid of spams, so I added some string inbetween my email address displayed in my homepage. Actually I still get lots of spams everyday, so well, I just changed the display of my address to normal to make it easy for people who need to contact me. I do use a spam filter, so if you don't get a response, please mail again.

@kraal: Currently I focus on bringing Eastern art tool/materials to the digital domain. Why? Because for Western tools/materials, at least we have nice apps like Painter or ArtRage, but I'm afraid tools for Eastern style that could really do the job is virutally not existing. Like Blur1 said, the brush gives you feedback on what is really being painted - control of line quality. And, I'd stress that line quality is very important in Eastern art.:)

@Nemoid: Yes, I know what you mean. I use real pencil and paper too.:)

@Vormav: The simulation is resolution dependent, at least for now. It's done on the GPU, and I hope in a year or two, most digital painters would be equipped with a fast enough GPU. The CPU is already used for brush simulation.

Also nice to see that tilt function of wacom really works if software uses it correctly.
I'm still waiting for the new 6D wacom art pen to be available here in Hong Kong.:D

Whats the upper limit on hardware textures nowdays anyway? 4096? Might limit the usefullnes of the approach but prolly in some cases it wont.
It's possible to use tiles of texture to do the simulation, although it's not easy and slow to run. Anyway, I believe in the future, graphics hardware technology would just get better and better. :)

Also, i assume your approach might be usefull for simulating other painting methods, like oils or acrylic.

hm... I'd say it should be not too far from Western watercolor. But for oil/acrylic, I think it depends on whether you really want the 3D quality of the media as in the impasto style.

[instinct-vfx]

Hey there Chu....

Your name does look familiar from the GLScene NG....good to see your progress, very nice stuff, keep em coming :)

Thorsten

[ThomasMahler]

Whoa, this is some kickass stuff!

I could also imagine something like that for digital sculpting - That'd be awesome :)

Very cool, I'd love to try this out!

[Fluckrat]

That looks really nice - great work!

Does anyone know what happened to DAB? it was shown at Siggraph in 2001 as a research project and then din't do much else after.... Looked to me to be the best digital painting solution ever created....

Check it out: http://www.cs.unc.edu/Research/geom/dab/
I highly recommend downloading the big movie.

Sorry if I've in any way hijacked the thread....

[Nemoid]

however i have to say that from the videos, the behaviour of ink on paper seems to be very well done, and very similar to reality. great job indeed.

also, viewing the brush on the screen is very funny.

question is : will us be able to do illustrations even with acrilic behaviour or other kinda media, (oils, pastels.. )for more polished kinda illustrations or is this related mainly to ink/watercolor ?
will us be able to change brush type?(more squared, for example or thinner and much other ) guess yes,because many apps have this even if in a less interactive way.
but from videos we see other things so far.

[el_diablo]

As far as i can see from a quick rundown of the paper, the distribution and behavior of paint is mostly geared towards thin media as ink or watercolor(LBE model). To model thick media as oils prolly different mixing functions should be implemented also there is no depth/impasto data afaik so that should be added first as Mr. Chu already said.

However, there is a part of their work i find more interesting, and that is the brush dynamics model which look pretty awesome. You can see the realistic beahivor of the brush on some closeups in the videos. This part of the work is prolly decoupled from the paint behavior model somewhat so that would work even with more simplistic model.

Anyways, I hope this thing gets developed more into a commercial product or something.

As I gather from the paper the fluid sim is run on 512 square vector map? I thought maybe you could do a mipmapping of sorts, multiresolution map depending on a current zoom level up to the canvas base resolution or even beyond to sort out the resolution problem.

[specialbrew]

Absolutely stunning - although as other boarders have noted, the Western application of this precise simulation may be limited, both the brush dynamics and the complete brilliance of the inclusion of the onscreen brush 'avatar' itself is inspired... looking at this, you suddenly release the rut that most standard paint packages have gotten themselves into, whereas this looks like a whole new paradigm shift.

I wish you guys all the best for Siggraph - I imagine you may get a fair amount of interest...

[Nelson Chu]

Again, thanks to those who responded.

@Thorsten: Glad to meet you here too. :)

@Fluckrat: You can check out the first author Bill Baxter's homepage. He did some more work on paint media simulation after 2001.

@Nemoid: It's relatively easy to add dry media like pastel. Also, more squared brush should be no problem. Oil is another story. :)

You can see the realistic beahivor of the brush on some closeups in the videos.
Thanks. Actally I'd say it's still not good enough for Chinese calligraphy, and I'm trying to improve it.

Anyways, I hope this thing gets developed more into a commercial product or something.
Me too. Let's see what happen after SIGGRAPH.

... even beyond to sort out the resolution problem.
hm... not sure to understand. 'Zoom level'? You mean using mipmapping for viewing the canvas?

@specialbrew: thank you. Glad that quite some of you think that the brush itself is good. Although it's the ink simulation that got us a SIGGRAPH paper, the brush part is not less important, if not more, in my opinion.

[el_diablo]

hm... not sure to understand. 'Zoom level'? You mean using mipmapping for viewing the canvas?

Actually i thought something in the lines of representing your canvas in hierahical manner depending on the zoom level when (re)constructing the vector field map (or color map, or any other 2d entity) after the stroke. My thoughts on the matter of multiresolution bitmaps are dated back to 1994 or so when i first thought of painting application that utilizes the z-buffer (same way zbrush does nowdays) but mine was 2-sided so you could actually do 3d transformations of the 2d image afterwards. That brought the resampling problem, however I did some thinking/research into representing 2d bitmap as a tree of somewhat intersecting multiresolution planes with dynamic sampling. Detail should be reconstructed (oversampling) for display with methods similar to the way you trim your boundary (iso-island/surface generation or similar). The tree could be collapsed at will and discretion of the user. The whole paradigm follows (in idea) h-spline or h-nurbs work.

I actually never had the time to take this any further than some ideas on paper and such, so I dont know the viability of the approach.

[Nelson Chu]

I see. So you're referring to the underlying representation, instead of the speed of processing for high resolution output as Vormav mentioned. Actually my current implementation uses a shader applied on the brush tuft geometry to do the deposition; some kind of isosourfacing is already there so it's already somehow resolution independent. But still the LBE simulation needs to be done at a specific resolution. I think ultimately I'd have all the stroke paths, user commands, brush loadings, etc saved, and re-generate the output at a user-specified resolution from the saved data.

[f97ao]

Holy Crap!
That is the most promising technique I have seen since first reading of Global Illumination in 3d art.

I have wanted a 3d brush for years inside a painting program. Myself I mostly paint with oil collor, so it would be awesome if you could do something with that. Some other ideas you may think of:
- Ability to have a paper that is not flat, but in 3d. Basically a 3d object. Would be very interersting.
- To have the painting strokes as vektors, that you can later on change. For example if you have made an S you can later control this vektor and make it into another letter. This would mean you have to history of all strokes.
- Make it possible to contorl 2 styluses at once. For example the 4dmouse could be used to instantly control the color by draging it up and down, or perhaps the size of the brush by moving the scroll wheel while still painting. This would really advance painting a ton.
- Make it possible to have a second object, for example something to mask with, so you can paint and have a plastic mask hiding sections.
- Air brushing would be very good with this technique and doesn't seem that far away from what you are doing. Remember masking is very important with airbrushing. Also make it easy to instantly switch between masks.
- make sure you multi thread, since you need lots and lots of cpu. Or well you use the GPU, but a good idea would probably be to use both.

Well, just some ideas. I really believe the future in 2d and even to some extent in 3d art is with similar tools as these. I'm sure several would be interesting in having a tool that can only do a few things, for example airbrush and water color, just as long as it does it way better than everything else. Just like you seem to do.

Good luck!

/Andreas

[kiaran]

Just a couple comments:

- Very cool work! :)
- I think this would make a very nice photoshop plugin.
- To me, I've always found the biggest difference between traditional media and digital painting is the fact that your pen and your canvas are in a different place digital painting. If you could draw directly on a screen (like a tablet PC), I think alot of artist would really dig it.

[noisewar]

Very nice stuff Nelson. Could the next step be capturing brush strokes as vector references for cel-shading 3D animation? I'd love to see a whole animation painted out in asian calligraphy strokes, moody rain soaking in the canvas of the setting itself, trees growing with brush strokes, expressions changed with absorbency changes, things like that. I've seen some watercolor animations that were beautiful, but faced many constraints not being of the digital medium. This could be a huge step!

[Nelson Chu]

Could the next step be capturing brush strokes as vector references for cel-shading 3D animation?
Probably. Actually I'm also thinking about rendering oriental fonts. The current True-Type Font technology is good for outline fonts, but not so suitable for brush-written characters.

If you could draw directly on a screen (like a tablet PC), I think alot of artist would really dig it.
I have yet to try this. A question arises: how should I display the 3D shape of the virtual brush, when there's already a stylus on top of the screen?

- Make it possible to have a second object, for example something to mask with, so you can paint and have a plastic mask hiding sections.
Yeah, general mask is already on my to-do list. :)

[Nelson Chu]

We should be able to demo our paint system prototype at SIGGRAPH;

hm... maybe not. I recently learn that there's no demo space for technical paper authors to demo their systems in SIGGRAPH this year...

Nelson

[Nelson Chu]

Update: We manage to get a laptop that can run MoXi (although not as fast as on a desktop) and I'll bring that with me to SIGGRAPH. I've also added support for the 6D art pen.

Nelson

[CGTalk Moderation]

This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.