PDA

View Full Version : Byte Order Mark (BOM) in Unicode files causing errors on fileIn


davestewart
12-02-2008, 12:10 PM
That's it really. Any ideas how to stop this, or do I need to raise a bug report?

ZeBoxx2
12-03-2008, 03:28 AM
Is it only the BOM? If it is.. could you detect it (binary read), and then decide to read in (and write out) the file yourself and run from there?

Sounds like a bug anyway, although I'm sure there's not many unicode (UTF-8) scripts out there :)

davestewart
12-03-2008, 11:53 AM
Hey Richard,
Crikey, sounds like a pain, but an interesting workaround if it works! Thanks for the tip.
Anyone know where I submit a bug report?
Cheers
Dave

davestewart
12-03-2008, 06:32 PM
Crikey, now I'm also finding that unicode characters are throwing off the highlighting of offending lines of code!!

If I use my comment headings (http://forums.cgsociety.org/showthread.php?f=98&t=695768&highlight=comment) tool with full block characters, when max throws an error, the line that is highlighted is WAY off!

A short test reveals that the more block characters you add, the worse it gets:

rollout roTest "Test"
(
-- ████████████████████████████ -- the more █ characters you add to this line, the worse the offset becomes

function test = -- this line will get highlighted on error!
(
someFunction() -- but it should be this line
)

on roTest open do test()
)

createdialog roTest 300 300

I know Comment Headings is a bit gimmicky in some ways, but a) it makes navigating large scripts a total breeze and b) that really isn't good from a "this product works well" point of view.

Argh!!!

ZeBoxx2
12-03-2008, 07:12 PM
well that sucks :\ I figured if it was going to do something it'd be a full-on script error (perhaps only in certain max localizations), but what you're mentioning is pretty crazy... that'll make debugging hell.

Sadly, none of the standard ASCII chars have the same graphical appeal

davestewart
12-03-2008, 07:24 PM
Yeah, I'm pretty pissed actually!

It's been SO much easier whizzing up and down 1000 or so lines of script with big ol' words in them, but with #s or @s they do kinda blend in with the backgorund when you're dragging the scrollbar.

Buggeration.

ZeBoxx2
12-04-2008, 01:17 AM
It's been SO much easier whizzing up and down 1000 or so lines of script with big ol' words in them

I can't make heads nor tails of how on earth I would configure the SCiTE bits and pieces to do a custom syntax highlighting... otherwise I'd suggest that maybe you can write your header comments in a specific form and have the syntax highlighter deal with that form by increasing the font size.

/me looks around for somebody who knows how on Earth that would be done, if at all
All I've found is keywords12 which is reserved for user-defined key words to syntax highlight.. but I haven't spotted whether you can specify wildcards/regex in those, or whether you can add, say, keywords13 (a quick test shows that does zilch, but I might be missing a reference somewhere that keywords 12 does - creating a style.MAXScript.24 certainly wasn't it)

Edit: P.S. How on Earth!?

davestewart
12-04-2008, 01:21 AM
I'd suggest that maybe you can write your header comments in a specific form and have the syntax highlighter deal with that form by increasing the font size.

Now that is a good idea! I've submitted a report to Autodesk anyway. Maybe something will come of it!

davestewart
12-04-2008, 01:26 AM
I think it has something to do with the language-specific "lexer"s, which I assume are written in C++. A bit out of my league...

davestewart
12-04-2008, 01:46 AM
Well that was an interesting road...

So I can change the size of comments, for example adding this to the user options file...

font.comment=font:Courier New,size:50

Gives me massive comments, but all lines go to 50 as well! Not so readable. And a little searching for "SCITE line-height" brings me to this (http://www.scintilla.org/ScintillaDoc.html):
All lines of text in Scintilla are the same height, and this height is calculated from the largest font in any current style. This restriction is for performance; if lines differed in height then calculations involving positioning of text would require the text to be styled first.
So there wo go. Good thoughts though Richard!

ZeBoxx2
12-04-2008, 01:53 AM
yeah, I had poked at adding..

style.MAXScript.2=$(colour.code.comment.line),Comic Sans MS,size:18,eolfilled


To the user properties.. so that all -- comments would be big, while /* */ would be the same as usual.. but alas.. no luck with the lineheight.. was actually looking for a way around that, but no-go, I guess.

Back to the drawing board... which is looking pretty full of stuff that has been wildly crossed through in frustration :)

ZeBoxx2
12-04-2008, 02:05 AM
somebody here (hi PM!) suggested adding the unicode character to chars.alpha in the MXS_Editor.properties file.. I have no idea what chars.alpha actually does other than getting re-used in maxscript.properties' word.characters.eek=$(yikes)-' which in turn is referenced by the lexer..

PiXeL_MoNKeY
12-04-2008, 02:20 AM
Further googling, and it appears that any non-alphanumeric characters you want to use should be placed in the chars.accented line of MXS_editor.properties file. So try adding your unicode stuff there and see if it resolves anything.

*Found in this .properties (http://74.125.45.132/search?q=cache:lWofFuDn1QoJ%3Cimg%20src=%22images/smilies/tongue.gif%22%20border=%220%22%20alt=%22%22%20title=%22stick%20out%20tongue%22%20smilieid=%226%22%20class=%22inlineimg%22%20/%3Eine.fm/LearnToProgram/SciTEGlobal.properties+scintilla+%23+Define+values+for+use+in+the+imported+properties+files&hl=en&ct=clnk&cd=6&gl=us&client=firefox-a) file kindly converted to html by google.

-Eric

Edit: Further digging in the MXS_Editor.properties and found that there is a section of commented out unicode stuff in the # Internationalisation section:# Unicode
#code.page=65001
#character.set=204But I have no idea if this will solve anything or not.

davestewart
12-04-2008, 02:23 AM
Ah, a nice idea, but ce ne pas marche!

Still get the BOM error, and the highlighting is screwy :S

-- Error occurred during fileIn in <File:E:\05 - Scripting\MaxScript\projects\For Scripters\Inspector Gadget\iterators 02.ms>
-- Error occurred in anonymous codeblock; filename: E:\05 - Scripting\MaxScript\projects\For Scripters\Inspector Gadget\iterators 02.ms; position: 3
-- Syntax error: at bad, expected <factor>
-- In line: 

Right - am off to bed. Many thanks for the efforts! I think I'm going to let this one gracefully slip away for now :)

CGTalk Moderation
12-04-2008, 02:23 AM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.