Unit Testing Framework maxtest

#1

Happy New Year.

The following might be interesting for developers who wish to write unit tests for their MAXScript code.

Backing up my code with unit tests gets more and more important to me.
Roughly two month ago, I started developing a unit testing framework for MAXScript.
It is inspired by cppunit and Pythons unittest framework.
It currently runs over 600 tests, growing.
Using it on a daily basis, gives me confidence to make it available to a wider audience.

Features:
[ul]
[li]discovers tests
[/li][li]distinguishes between error, failed and passed tests
[/li][li]reports filepath & line number of a failed test
[/li][li]supports test suites with setUp and tearDown
[/li][li]comprehensive test results outputted to the listener
[/li][li]current asserts are: equals, notEquals, assertTrue, assertFalse, raises, notRaises
[/li][/ul]

It is written for Max2011 and doesn’t support the new Assert methods of Max2012.

Yesterday I made maxtest open source.
It is hosted under Google Code within the 3ds-max-dev project. A repository created by Christopher Diggins from Autodesk.

Try the example.ms file, it will run the test inside test_example.ms file.
Documentation and more examples are coming soon.

My goal is to maintain and add features to the framework and collaborate with the community.

Greets,
Pixelbaker

Maxtest, unit testing framework
#2

I added documentation for maxtest.
It should cover the basics and get you going.

Expect continues improvement and extension throughout the next weeks.

Have a good week.

#3

This is pretty interesting - i’m looking into unittests for python. Test driven development is pretty neat.

With your code my understanding is your basically passing the function and arguments to a stringstream and executing it within a try, catch - throwing the error found in the listener?

Anyway ill try and test out this framework sometime,

#4

Hi eek,

I recently set up a continuous integration system at work. I am using Hudson for this.
It runs all the tests, currently 1892, with the use of maxtest each hour if someone committed to the trunk.

ofer_z and I created two more Presenters.
One is the InteractivePresenter which gives you a nice GUI for the test results and opens up the function or line, where an error or failed assert occured.
The second is the JUnitTestResultPresenter, which outputs test results to xml in the JUnit test result format. That’s a format which Hudson understands and uses for statistic evaluations, etc.

For the brave: the branches, in which both presenters are developed:

InteractivePresenter branch
JUnitTestResultPresenter branch

I didn’t find the time to bring them both into the trunk yet.

#5

That is pretty neat! -

So is hudson firing the maxscripts to the tests, returning the results etc?? What im asking is this similiar to pythons unittest runner/discover?

#6

maxtest is similar to Pythons unittest runner/discover. That was one source of inspiration, when I developed it.

Hudson on the other end is just a giant automatic way to run Maxtest.

At the current state Hudson+Maxtest is a massive patchwork, which needs a lot more to mature.

[ul]
[li]First Hudson queries each hour the trunk and checks if new changes arrived.
[/li][li]If so, it gets the changes. (checkout/update/switch)
[/li][li]Then Hudson creates and saves a maxscript, which by using maxtest discovers, runs tests, outputs the results in a special xml format to disk and finally quitMax #noPrompt.
[/li][li]Next Hudson tells Max to start and to execute this script.
[/li][li]Maxtest does its thing and at the end Max closes.
[/li][li]If everything went fine, Hudson grabs the result xml file and reads it if everything went fine.
[/li][li]Otherwise an email is fired, and hell breaks loose.
[/li][/ul]

There are more details to the setup, especially for our pipeline.

But I guess I went too far off. Your initial question was more about how maxtest is working inside, wasn’t it?

#7

Yes i guess, i was just intrigued by the whole setup. I’m guessing commits are for some subversion - perforce etc. I’m looking into Shotgun atm, which has similar sorts of triggers and callback mechanisms. Its process management with a layer of abstraction, e.g a task can have a flag set by a user which internal can be ‘listened’ to via a callback mechanism and initiate new tasks etc…

It can listen to any flag, a text menu being changed, a checkbox ticked etc…

So a process could in theory, initiate a bunch of unit tests which if passed allow that process to move to the next step. Its more process management than tools release though.

Im guessing Hudson is calling the maxTests on scene data id assume? Or i guess are the tests just for value testing and not if eg. material x exists on object b?

An awesome approval mechanism i saw was called ‘QualityGuard’ - id love to implement it in python/maxscript:

http://www.martintomoya.com/Site/tools_info.html

Hell, all of this guys tools are amazing!

#8

We have currently three categories for our tests.
[ul]
[li]Unittests
[/li][li]Integrationtests
[/li][li]Performancetests
[/li][/ul]

The tests range from simple function return value evaluation, over how fast does a function operate on x nodes to does tool Y start.
Tests should never be depended on production data.

Checking scene data is another topic. We have a completely different system for that, which is similar to “Quality GuAard”.
Cause for systems like this, you often want to provide a fix or help the user to fix it manually.

If that’s what you are aiming at, then maxtest is not your framework of choice. maxtest supports you for writing and running tests to make sure individual units of source code work as expected.

#9

This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.