LZO decompress/compress data


#1

Hi, I have a problem, I’m trying to add to my sript a function that reads a lzo compressed data. Unfortunately, i don’t know very well C++ and i don’t know how to use the library into maxscript.

Someone can help me ?


#2

You can use this in your maxscript if I’m correct. (.net).

http://lzo-net.sourceforge.net/

I cannot help you with the dotnet stuff - since I do not use it myself.
But I figured this would be a easier way then to implement it in c++ and then expose to maxscript as plugin function.


#3

max2011 test:
dotnet.loadAssembly @“C:\Documents and Settings\User\Desktop\lzonet_1.0.1\lib\lzo.dll”
– Runtime error: dotNet runtime exception: Invalid directory on URL.

this mean that i must re-compiled dll with max SDK… but i don’t know how do…


#4

uhmm… but this work:
lzo = dotnet.loadAssembly @“C:\Documents and Settings\User\Desktop\lzonet_1.0.1\lib\Simplicit.Net.Lzo.dll”
showProperties lzo
showMethods lzo

.CodeBase : <System.String>, read-only
.EntryPoint : <System.Reflection.MethodInfo>, read-only
.EscapedCodeBase : <System.String>, read-only
.Evidence : <System.Security.Policy.Evidence>, read-only
.FullName : <System.String>, read-only
.GlobalAssemblyCache : <System.Boolean>, read-only
.HostContext : <System.Int64>, read-only
.ImageRuntimeVersion : <System.String>, read-only
.Location : <System.String>, read-only
.ManifestModule : <System.Reflection.Module>, read-only
.ReflectionOnly : <System.Boolean>, read-only

.<System.Object>CreateInstance <System.String>typeName
.<System.Object>CreateInstance <System.String>typeName <System.Boolean>ignoreCase
.<System.Object>CreateInstance <System.String>typeName <System.Boolean>ignoreCase <System.Reflection.BindingFlags>bindingAttr <System.Reflection.Binder>binder <System.Object[]>args <System.Globalization.CultureInfo>culture <System.Object[]>activationAttributes
.[static]<System.String>CreateQualifiedName <System.String>assemblyName <System.String>typeName
.<System.Boolean>Equals <System.Object>o
.[static]<System.Boolean>Equals <System.Object>objA <System.Object>objB
.[static]<System.Reflection.Assembly>GetAssembly <System.Type>type
.[static]<System.Reflection.Assembly>GetCallingAssembly()
.<System.Object[]>GetCustomAttributes <System.Boolean>inherit
.<System.Object[]>GetCustomAttributes <System.Type>attributeType <System.Boolean>inherit
.[static]<System.Reflection.Assembly>GetEntryAssembly()
.[static]<System.Reflection.Assembly>GetExecutingAssembly()
.<System.Type[]>GetExportedTypes()
.<System.IO.FileStream>GetFile <System.String>name
.<System.IO.FileStream[]>GetFiles()
.<System.IO.FileStream[]>GetFiles <System.Boolean>getResourceModules
.<System.Int32>GetHashCode()
.<System.Reflection.Module[]>GetLoadedModules()
.<System.Reflection.Module[]>GetLoadedModules <System.Boolean>getResourceModules
.<System.Reflection.ManifestResourceInfo>GetManifestResourceInfo <System.String>resourceName
.<System.String[]>GetManifestResourceNames()
.<System.IO.Stream>GetManifestResourceStream <System.String>name
.<System.IO.Stream>GetManifestResourceStream <System.Type>type <System.String>name
.<System.Reflection.Module>GetModule <System.String>name
.<System.Reflection.Module[]>GetModules()
.<System.Reflection.Module[]>GetModules <System.Boolean>getResourceModules
.<System.Reflection.AssemblyName>GetName()
.<System.Reflection.AssemblyName>GetName <System.Boolean>copiedName
.GetObjectData <System.Runtime.Serialization.SerializationInfo>info <System.Runtime.Serialization.StreamingContext>context
.<System.Reflection.AssemblyName[]>GetReferencedAssemblies()
.<System.Reflection.Assembly>GetSatelliteAssembly <System.Globalization.CultureInfo>culture
.<System.Reflection.Assembly>GetSatelliteAssembly <System.Globalization.CultureInfo>culture <System.Version>version
.<System.Type>GetType()
.<System.Type>GetType <System.String>name
.<System.Type>GetType <System.String>name <System.Boolean>throwOnError
.<System.Type>GetType <System.String>name <System.Boolean>throwOnError <System.Boolean>ignoreCase
.<System.Type[]>GetTypes()
.<System.Boolean>IsDefined <System.Type>attributeType <System.Boolean>inherit
.[static]<System.Reflection.Assembly>Load <System.String>assemblyString
.[static]<System.Reflection.Assembly>Load <System.Byte[]>rawAssembly
.[static]<System.Reflection.Assembly>Load <System.Reflection.AssemblyName>assemblyRef
.[static]<System.Reflection.Assembly>Load <System.String>assemblyString <System.Security.Policy.Evidence>assemblySecurity
.[static]<System.Reflection.Assembly>Load <System.Byte[]>rawAssembly <System.Byte[]>rawSymbolStore
.[static]<System.Reflection.Assembly>Load <System.Reflection.AssemblyName>assemblyRef <System.Security.Policy.Evidence>assemblySecurity
.[static]<System.Reflection.Assembly>Load <System.Byte[]>rawAssembly <System.Byte[]>rawSymbolStore <System.Security.Policy.Evidence>securityEvidence
.[static]<System.Reflection.Assembly>LoadFile <System.String>path
.[static]<System.Reflection.Assembly>LoadFile <System.String>path <System.Security.Policy.Evidence>securityEvidence
.[static]<System.Reflection.Assembly>LoadFrom <System.String>assemblyFile
.[static]<System.Reflection.Assembly>LoadFrom <System.String>assemblyFile <System.Security.Policy.Evidence>securityEvidence
.[static]<System.Reflection.Assembly>LoadFrom <System.String>assemblyFile <System.Security.Policy.Evidence>securityEvidence <System.Byte[]>hashValue <System.Configuration.Assemblies.AssemblyHashAlgorithm>hashAlgorithm
.<System.Reflection.Module>LoadModule <System.String>moduleName <System.Byte[]>rawModule
.<System.Reflection.Module>LoadModule <System.String>moduleName <System.Byte[]>rawModule <System.Byte[]>rawSymbolStore
.[static]<System.Reflection.Assembly>LoadWithPartialName <System.String>partialName
.[static]<System.Reflection.Assembly>LoadWithPartialName <System.String>partialName <System.Security.Policy.Evidence>securityEvidence
.[static]<System.Boolean>ReferenceEquals <System.Object>objA <System.Object>objB
.[static]<System.Reflection.Assembly>ReflectionOnlyLoad <System.Byte[]>rawAssembly
.[static]<System.Reflection.Assembly>ReflectionOnlyLoad <System.String>assemblyString
.[static]<System.Reflection.Assembly>ReflectionOnlyLoadFrom <System.String>assemblyFile
.<System.String>ToString()


#5

according to the docs, what you need is the LZOCompressor class:


lzocompressor = lzo.createInstance "LZOCompressor"
compressedByteArray = lzocompressor.compress yourByteArray

#6

thanks, i never used dotnet in maxscript. But don’t work, it return undefined
i found in Visual Studio project the class what i must instanced “public class LZOCompressor {…”
but lzocompressor = undefined…???


#7

i recompiled in Framework 2.0 3.0 3.5 but the same issue … i’m confused


#8

sorry, you also need to include the namespace


lzocompressor = lzo.createInstance "Simplicit.Net.LZO.LZOCompressor"
compressedByteArray = lzocompressor.compress yourByteArray

#9

ok, now load but how work the dll ? i write

>lzo = dotnet.loadAssembly “D:\Documents and Settings\Admin\Desktop\lzonet_1.0.1\lib\Simplicit.Net.Lzo.dll”
>lzocompressor = lzo.createInstance “Simplicit.Net.Lzo.LZOCompressor”
>compressedByteArray = lzocompressor.compress #()
and return
>#(17, 0, 0, 0, 0, 0, 0)
but if i write
>compressedByteArray = lzocompressor.compress #(1,2,3,4)
write a strange error…


#10

… uhm… i must use Encoding.Default.GetBytes ?


#11
(dotnetclass "system.text.encoding").default.getbytes "my string"

>> #(109, 121, 32, 115, 116, 114, 105, 110, 103)


#12

an error not in english but in windows language (this is strange) tell me
– Runtime error: dotNet runtime exception: Cannot widen from source type to target type either because the source type is a not a primitive type or the conversion cannot be accomplished.
I resolved creating a dotnet bytes array, i test applying a value>255 and now work.
But “bytes” is a dotnetObjects and “compressed_bytes” is a max’s array.
How can i simplify the creation of a byte array (max use a int64 array ?) because my script read data stream from a file with:
bytes = for i=1 to DATASIZE collect readByte fin #unsigned

clearlistener()
(
SIZEOF = 100
bytes = dotNetObject "System.Byte[]" SIZEOF
for idx=0 to SIZEOF-1 do bytes.SetValue (dotNetObject "System.Byte" (random 0 455)) idx
	
lzo = dotnet.loadAssembly ((getDir #archives) +"/m2tw_scripts/lib/Simplicit.Net.Lzo.dll")
lzocompressor = lzo.createInstance "Simplicit.Net.Lzo.LZOCompressor"
--((dotNetClass "System.Text.Encoding").default.getbytes "m string")
compressed_bytes = lzocompressor.compress bytes

format "original : %
compressed : %
" (for idx=0 to SIZEOF-1 collect bytes.GetValue idx) compressed_bytes
)


#13

this is a example and work, but it can done better ?


clearlistener()
try
(
	filename = getOpenFileName types:"all types (*.*)|*.*" caption:("LZO compression")
	if filename!=undefined then
	(
		fin = fopen filename "rb"
		SIZEOF = getFileSize filename
		
		bytes = dotNetObject "System.Byte[]" SIZEOF
		for idx=0 to SIZEOF-1 do bytes.SetValue (dotNetObject "System.Byte" (readByte fin #unsigned)) idx
		
		fclose fin
	
		lzo = dotnet.loadAssembly ((getDir #archives) +"/m2tw_scripts/lib/Simplicit.Net.Lzo.dll")
		lzocompressor = lzo.createInstance "Simplicit.Net.Lzo.LZOCompressor"
		compressed_bytes = lzocompressor.compress bytes
		--format "original : %
compressed : %
" (for idx=0 to SIZEOF-1 collect bytes.GetValue idx) compressed_bytes
		
		filename = getSaveFileName types:"all types (*.*)|*.*"caption:("LZO compression save")
		if filename!=undefined then
		(
			fout = fopen filename "wb"
			for b in compressed_bytes do writeByte fout b #unsigned
			fclose fout
		)
		
	)
)
catch ("ERROR")


#14

i don’t undertand this error:


			SIZEOF = decompressed.count
			format ">> run lzo compression with sizeof:%
" SIZEOF
			BytesArray = dotnet.ValueToDotNetObject decompressed (dotNetClass "System.byte[]")
			lzo = dotnet.loadAssembly (LibraryDir +"/Simplicit.Net.Lzo.dll")
			lzocompressor = lzo.CreateInstance "Simplicit.Net.Lzo.LZOCompressor"
			compressed = lzocompressor.Decompress BytesArray


>> MAXScript Rollout Handler Exception: -- Runtime error: dotNet runtime exception: Arithmetic operation resulted in an overflow. <<



#15

in max2011 there aren’t issue but in max9 i can’t build a Byte[] array:

in max9 this work:
arr = dotNetObject “System.Int32[]” 100
arr.set 0 (dotNetObject “System.Byte” 255)

but this not work:
arr = dotNetObject “System.Byte[]” 100
arr.set 0 (dotNetObject “System.Byte” 255)

because <dotNetObject “System.Byte” 255> return a int32… how is possible ? someone can help me a re-compiled a new Simpl icit.Net.Lzo.dll that accept a
dotNetObject “System.Int32[]” ?


#16

to do this i re-compiled the dll:

	--BytesArray = dotnet.ValueToDotNetObject compressed (dotNetClass "System.Byte[]")
 	BytesArray = dotNetObject "System.Int32[]" SIZEOF
 	for i=0 to SIZEOF-1 do BytesArray.Set (dotNetObject "System.Int32" i) (dotNetObject "System.Byte" compressed[i+1])
 	
 	
 	lzo = dotnet.loadAssembly ((getDir #archives) +"/m2tw_scripts/lib/Simplicit.Net.Lzo.dll")
 	lzocompressor = lzo.createInstance "Simplicit.Net.Lzo.LZOCompressor"
 	decompressed_bytes = lzocompressor.Compress BytesArray

in C# i write:

...
 		public byte[] Compress(int[] src_int) {
 
 			byte[] src = new byte[src_int.Length];
 			for (int i = 0; i < src_int.Length; i++)
 			{
 				src[i] = (byte)(src_int[i] & 0xFF);
 			}
 ...

the new dll in the attachemtns


#17

I wonder how you ever make this woking… maybe you test it on 32-bit Max?
I constantly get .Net exception:

An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)

and that means that those lzo.dll is built for x86 only. Did you get similar runtime error?


#18

this script was tested in windowsXP32 bit with max9 32bit, but this weekend i will test it on windows7 64bit with max2011 32 and 64 bit.
The dll was compiled with Framework 2.0 (for max9 but work also for max2011)

max9 have a issue using “system.byte[]” becasue it convert always in “system.int32[]” so the lzo.dll don’t work… in max2011 there are the new function “dotnet.ValueToDotNetObject <max array of integer> (dotNetClass “System.Byte[]”)” but for big file (example 500000bytes) the heapsize increase a lot because max must build an array with 500.000 integers so i add a new function to let the dll read the file, but the output is always a big array and the file must be closed when reading.

in the attachment there are the new c# code, you can recompiled if you want or add new functions


#19

when you decompres to convert some bytes array into float or integer you can do it:

convFN = dotnetClass "System.BitConverter"
for i=1 to SIZEOF by 4 collect
(
bytes =#(data[i],data[i+1],data[i+2],data[i+3])
floatvalue = conv.ToSingle (dotnet.ValueToDotNetObject bytes (dotNetClass "System.Byte[]")) 0
--intvalue = conv.ToInt32 (dotnet.ValueToDotNetObject bytes  (dotNetClass "System.Byte[]")) 0
)

#20

Well, that make sense, I’m on XP x64 so I changed the target platform form AnyCPU to x86 in my vs project, and now (after re-comp.) it work in 32-bit Max, but am too write a methods to work directly with files in C#, as the Max convert .net bytes to integers anyway and this kill whole goodness (: but still no way to use the new wrapper with that lzo.dll in 64-bit Max.

I see no hope to run your assembly on x64, but just in case I will test it later on.
Meanwhile I attach my LZOCompressor.cs