Viewport Grabbing (yes, again)

lo1 · 2011-09-08 13:09:26 UTC

Hi Guys

I know this issue has been discussed on a number of threads before, but this specific issue I could not find any mention of.
I am using the following function to grab the viewport DIB:

fn getViewportDIB w h =
(	
	local originalSize = getViewSize()
	local vpAspect = originalSize[1]/(originalSize[2] as float)
	local reAspect = w as float / h
	local dispSafe = displaySafeFrames
	displaySafeFrames = on
	local cropBox, dibW, dibH
	if vpAspect/reAspect < 1 then
	(
		dibW = w
		dibH = w / vpAspect		
		cropBox = box2 0 ((dibH-h)/2.) w h
	)
	else
	(
		dibW = h * vpAspect
		dibH = h
		cropBox = box2 ((dibW-w)/2.) 0 w h
	)
	gw.SetPos 0 0 dibW dibH
	forceCompleteRedraw()
	local dib = gw.GetViewportDIB()
	local result = Bitmap w h			
	displaySafeFrames = dispSafe
	pasteBitmap dib result cropBox [0,0] type:#paste
	close dib
	free dib
	StatusPanel.visible = not StatusPanel.visible
	StatusPanel.visible = not StatusPanel.visible
	result
)

display (getViewportDib renderWidth renderHeight)

The problem is viewport backgrounds. The background does not seem to heed the call of gw.SetPos and remains the same size that it was in the original viewport.

I’ve tried InvalidateAllBackgrounds() and various redraw methods, but no luck. Any ideas?

spacefrog · 2011-09-08 13:18:27 UTC

gw.Setpos ( and some other gw related calls) are broken in Nitrous/Max2012 and will probably never return working …
In my PowerPreview script i use Windows messages and user32.dll calls ( SetWindowPos etc…) , using runtime compiled dotNet code ( thx to Denis for this method btw. ). That way i set the nitrous viewport’s size to my liking…

lo1 · 2011-09-08 13:25:04 UTC

Yes, I realize the question is not relevant for max2012.
That’s interesting though, I tried doing it with SetWindowPos before trying gw.setPos and it didn’t seem to work at all, but then again I was trying it in max2009. I’ll give it another try, thanks.

PiXeL_MoNKeY · 2011-09-08 14:58:27 UTC

When making gw calls are you putting them in Viewport Redraw Callback as outlined in the 2012 Maxscript help?

In addition, with the introduction of the Nitrous Graphics Manager in 3ds Max 2012, any MAXScript Graphics Window (gw.) Viewport Drawing Methods must be wrapped in a Viewport Redraw Callback function to produce any results due to the progressive refinement nature of the drawing process.
-Eric

lo1 · 2011-09-08 15:40:47 UTC

First let me get this working in max2009, then I’ll worry about Nitrous

lo1 · 2011-09-08 15:52:02 UTC

@spacefrog: btw, I noticed in another thread you are looking for a way to grab a viewport with alpha. I’ve implemented a convenient method for this, allow me to share it:

I think it is better than the ‘set all objects to white color’ method, because it supports transparent materials on objects or otherwise semi-transparent pixels.

assuming dibFn is a function which returns a viewport DIB of custom size:


local dib = dibFn renderWidth renderHeight
local colA = getVPortBGColor()
local bgOn = viewport.DispBkgImage
setVPortBGColor black
viewport.DispBkgImage = off
local dibA = dibFn renderWidth renderHeight
setVPortBGColor white
local dibB = dibFn renderWidth renderHeight
setVPortBGColor colA
viewport.DispBkgImage = bgOn				
local w = dib.Width
for y = 0 to dibA.height-1 do
(
	local row = getPixels dib [0,y] w
	local rowA = getPixels dibA [0,y] w
	local rowB = getPixels dibB [0,y] w
	for x = 1 to w do
	(
		row[x].alpha = 255 - (rowB[x].value - rowA[x].value)
	)
	setPixels dib [0,y] row
)
close dibB
free dibB
close dibA
free dibA
dib

lo1 · 2011-09-09 11:04:33 UTC

Tried with the setwindowpos approach again, but it doesn’t seem to have any effect in max2009. If it try to call forceCompleteRedraw() after using it, I get a system exception and all the viewports are screwed up for the entire max session.

I’ve tried practically all reasonable WM messages, but the only thing that seems to actually affect the returned bitmap size is gw.setPos. Must be something I’m missing…

spacefrog · 2011-09-09 12:50:06 UTC

what windowhandle do you use ?
“Viewpanel” is the parent of each individual viewport ( plus the splitter bars and the viewport menus which are of class “Label”) -
UPDATE maximize vour viewport and use the ViewPanel" handle to resize your viewport !!!, i mistakenly wrote you have to use the D3D views handle, which is plain wrong … sorry for the mistake

Winspector (Download) or Spy++ from Visual Studio does a great job checking all about existing windows handles and windows messages etc…

Note: both tools are 32 bit only , so they only can capture windows messages of 32 bit applications, getting window handle infos works on 64 bit applications though

this is the call i use in powerpreview, note that xpos/ypos are relative to the parent window (viewpanel in that case). Of course you have to define the SWP_* flags according to the values in the header files - see the list below



SetWindowPos( hWnd, 0 , xpos, ypos, width, height, SWP_NOMOVE | SWP_NOZORDER | SWP_FRAMECHANGED )

/* Flags for SetWindowPos
SWP_ASYNCWINDOWPOS=0x4000
SWP_DEFERERASE=0x2000
SWP_DRAWFRAME=0x0020
SWP_FRAMECHANGED=0x0020
SWP_HIDEWINDOW=0x0080
SWP_NOACTIVATE = 0x0010
SWP_NOCOPYBITS=0x0100
SWP_NOMOVE=0x0002
SWP_NOOWNERZORDER=0x0200
SWP_NOREDRAW=0x0008
SWP_NOREPOSITION=0x0200
SWP_NOSENDCHANGING=0x0400
SWP_NOSIZE=0x0001
SWP_NOZORDER=0x0004
SWP_SHOWWINDOW=0x0040
*/

lo1 · 2011-09-09 14:07:36 UTC

spacefrog:

this is the call i use in powerpreview, note that xpos/ypos are relative to the parent window (viewpanel in that case). Of course you have to define the SWP_* flags according to the values in the header files - see the list below
SetWindowPos( hWnd, 0 , xpos, ypos, width, height, SWP_NOMOVE | SWP_NOZORDER | SWP_FRAMECHANGED )

Thank you very much for your help. This specific bitwise combination doesn’t give me a system exception but it also doesn’t seem to have any effect. I tried running the PowerPreview alpha you posted in another thread I found and it does seem to work on max2009 very well (except some harmless exceptions being raised) so you have proved what I am trying is possible in max2009, I just need to find out what I’m doing wrong.

p.s. - if you are using SWP_NOMOVE, I believe xpos and ypos are meaningless as they are ignored by the function.

I have attached my sloppy test code, perhaps you would immediately see what I’m forgetting or doing wrong.

thanks again

(
source = "using System;
"
source += "using System.Runtime.InteropServices;
"
source += "using System.Text;
"
source += "class assembly
"
source += "{
"
source += " [DllImport(\"user32.dll\")]
"
source += " public static extern bool SetWindowPos(IntPtr hWnd, int hWndArg, int Left, int Top, int Width, int Height, int hWndFlags);
"
source += " [DllImport(\"user32.dll\")]
"
source += "	static extern bool GetWindowRect(IntPtr hWnd, out RECT rect);
"
source += "	public struct RECT
"
source += "	{
"
source += "	 public int Left;
"
source += "	 public int Top;
"
source += "	 public int Right;
"
source += "	 public int Bottom;
"
source += "	}
"
source += "	public int[] getWindowRect(IntPtr hWnd)
"
source += "	{
"
source += "	 RECT rect;
"
source += "	 if ( GetWindowRect(hWnd, out rect) )
"
source += "	 {
"
source += "	 return new int[] { rect.Left, rect.Top, rect.Right - rect.Left, rect.Bottom - rect.Top };
"
source += "	 }
"
source += "	 return null;
"
source += "	}
"
source += "}
"

csharpProvider = dotnetobject "Microsoft.CSharp.CSharpCodeProvider"
compilerParams = dotnetobject "System.CodeDom.Compiler.CompilerParameters"

compilerParams.GenerateInMemory = on
compilerResults = csharpProvider.CompileAssemblyFromSource compilerParams #(source)

global assembly = compilerResults.CompiledAssembly.createInstance "assembly"

global targetHwnd
global targetHwndInt
global targetPosition = mouse.screenpos - mouse.pos
for w in (for w in (windows.getChildrenHWND #max) where  matchpattern w[4] pattern:"d3d*" collect w[1]) do --get the right viewport
(
	local wPtr = (dotnetobject "system.intptr" w)
	local r = assembly.getWindowRect wPtr
	if r[1] == targetPosition[1] and r[2] == targetPosition[2] do 
	(
		targethwnd = wPtr
		targetHwndInt = w
	)
)
assembly.setwindowpos targethwnd 0 0 0 1200 900 0x0026 --- 0x0026 is the bitwise combination of swp_nomove swp_nozorder and swp_drawframe
windows.sendmessage targetHwndInt 0x000F 0 0 --WM_PAINT maybe?
forcecompleteredraw()
display (gw.getViewportDib()) -- returns dib in original size.
--display (viewport.getViewportDib()) -- return different result, but also not correct.

statuspanel.visible = not statuspanel.visible -- return the viewport sizes to normal
statuspanel.visible = not statuspanel.visible -- return the viewport sizes to normal
)

denisT · 2011-09-09 14:50:16 UTC

it seems like you need a big gun.

you are scaling wrong window. max doesn’t like it and crashes.
scale whole ViewPanel


ViewHwnd = for w in (windows.getChildrenHWND #max) where w[4] == "ViewPanel" do exit with w[1]

if act = (viewport.numViews != 1) do max tool maximize
setwindowpos ViewHwnd 0 0 0 2000 1900 0
completeRedraw()
display (gw.GetViewportDIB())
hwnd = windows.getmaxhwnd()
WM_EXITSIZEMOVE = 0x232
windows.sendmessage hwnd WM_EXITSIZEMOVE 0 0 -- force to update views client area
if act do max tool maximize
completeRedraw()

how to make it in hidden mode see i my other posts.

lo1 · 2011-09-09 14:57:41 UTC

Once again, Denis to the rescue.
This is exactly what I was looking for.
Unfortunately, it is not returning to a ‘normal’ state after the operation despite the window messages but that is easily solvable by other methods now.
Thanks so much. One day I will buy you that beer.

My only question is if it’s alright to use the technique in my tool?

denisT · 2011-09-09 15:14:42 UTC

it has to… but you can send the fullredraw message to the max window

that’s free to use. you can give a credit if you want.

denisT · 2011-09-09 15:19:29 UTC

actually the maximizing the view is not necessary. simple calculations give you the right size for the main view panel.

spacefrog · 2011-09-09 22:08:21 UTC

ahh - Denis’s method is right , i got my memory wrong how i did the resizing myself:
In fact i do it the same way as he posted

i switch the viewport in question to maximized ( so it’s the single subwindow of “ViewPanel” )
i do a SetWindowPos call to the “ViewPanel” window !! - not the D3d subwindow

sorry to not check my own sourcecode before posting

lo1 · 2011-09-10 05:41:29 UTC

followed that advice and it works great, thanks.

lo1 · 2011-09-10 11:04:50 UTC

The only thing that worked for me is a WM_SIZE message:

windows.sendMessage maxHwnd 0x0005 0 (maxSize[2]*65536+maxSize[1])

maxSize variable was retrieved using a GetClientRect call.

edit: actually it seems the numbers are not important, max will ignore them.

windows.sendMessage maxHwnd 0x0005 0 0

chib · 2012-01-16 00:23:48 UTC

(about nitrous and gw.setpos) It doesn’t work either…sigh

CGTalk_Moderation · 2012-01-16 00:23:49 UTC

This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.