LC #42 Pipers Alley

View Full Version : Byte Order Mark (BOM) in Unicode files causing errors on fileIn

12-02-2008, 01:10 PM
That's it really. Any ideas how to stop this, or do I need to raise a bug report?

12-03-2008, 04:28 AM
Is it only the BOM? If it is.. could you detect it (binary read), and then decide to read in (and write out) the file yourself and run from there?

Sounds like a bug anyway, although I'm sure there's not many unicode (UTF-8) scripts out there :)

12-03-2008, 12:53 PM
Hey Richard,
Crikey, sounds like a pain, but an interesting workaround if it works! Thanks for the tip.
Anyone know where I submit a bug report?

12-03-2008, 07:32 PM
Crikey, now I'm also finding that unicode characters are throwing off the highlighting of offending lines of code!!

If I use my comment headings ( tool with full block characters, when max throws an error, the line that is highlighted is WAY off!

A short test reveals that the more block characters you add, the worse it gets:

rollout roTest "Test"
-- ████████████████████████████ -- the more █ characters you add to this line, the worse the offset becomes

function test = -- this line will get highlighted on error!
someFunction() -- but it should be this line

on roTest open do test()

createdialog roTest 300 300

I know Comment Headings is a bit gimmicky in some ways, but a) it makes navigating large scripts a total breeze and b) that really isn't good from a "this product works well" point of view.


12-03-2008, 08:12 PM
well that sucks :\ I figured if it was going to do something it'd be a full-on script error (perhaps only in certain max localizations), but what you're mentioning is pretty crazy... that'll make debugging hell.

Sadly, none of the standard ASCII chars have the same graphical appeal

12-03-2008, 08:24 PM
Yeah, I'm pretty pissed actually!

It's been SO much easier whizzing up and down 1000 or so lines of script with big ol' words in them, but with #s or @s they do kinda blend in with the backgorund when you're dragging the scrollbar.


12-04-2008, 02:17 AM
It's been SO much easier whizzing up and down 1000 or so lines of script with big ol' words in them

I can't make heads nor tails of how on earth I would configure the SCiTE bits and pieces to do a custom syntax highlighting... otherwise I'd suggest that maybe you can write your header comments in a specific form and have the syntax highlighter deal with that form by increasing the font size.

/me looks around for somebody who knows how on Earth that would be done, if at all
All I've found is keywords12 which is reserved for user-defined key words to syntax highlight.. but I haven't spotted whether you can specify wildcards/regex in those, or whether you can add, say, keywords13 (a quick test shows that does zilch, but I might be missing a reference somewhere that keywords 12 does - creating a style.MAXScript.24 certainly wasn't it)

Edit: P.S. How on Earth!?

12-04-2008, 02:21 AM
I'd suggest that maybe you can write your header comments in a specific form and have the syntax highlighter deal with that form by increasing the font size.

Now that is a good idea! I've submitted a report to Autodesk anyway. Maybe something will come of it!

12-04-2008, 02:26 AM
I think it has something to do with the language-specific "lexer"s, which I assume are written in C++. A bit out of my league...

12-04-2008, 02:46 AM
Well that was an interesting road...

So I can change the size of comments, for example adding this to the user options file...

font.comment=font:Courier New,size:50

Gives me massive comments, but all lines go to 50 as well! Not so readable. And a little searching for "SCITE line-height" brings me to this (
All lines of text in Scintilla are the same height, and this height is calculated from the largest font in any current style. This restriction is for performance; if lines differed in height then calculations involving positioning of text would require the text to be styled first.
So there wo go. Good thoughts though Richard!

12-04-2008, 02:53 AM
yeah, I had poked at adding..

style.MAXScript.2=$(colour.code.comment.line),Comic Sans MS,size:18,eolfilled

To the user properties.. so that all -- comments would be big, while /* */ would be the same as usual.. but alas.. no luck with the lineheight.. was actually looking for a way around that, but no-go, I guess.

Back to the drawing board... which is looking pretty full of stuff that has been wildly crossed through in frustration :)

12-04-2008, 03:05 AM
somebody here (hi PM!) suggested adding the unicode character to chars.alpha in the file.. I have no idea what chars.alpha actually does other than getting re-used in' word.characters.eek=$(yikes)-' which in turn is referenced by the lexer..

12-04-2008, 03:20 AM
Further googling, and it appears that any non-alphanumeric characters you want to use should be placed in the chars.accented line of file. So try adding your unicode stuff there and see if it resolves anything.

*Found in this .properties ( file kindly converted to html by google.


Edit: Further digging in the and found that there is a section of commented out unicode stuff in the # Internationalisation section:# Unicode
#character.set=204But I have no idea if this will solve anything or not.

12-04-2008, 03:23 AM
Ah, a nice idea, but ce ne pas marche!

Still get the BOM error, and the highlighting is screwy :S

-- Error occurred during fileIn in <File:E:\05 - Scripting\MaxScript\projects\For Scripters\Inspector Gadget\iterators>
-- Error occurred in anonymous codeblock; filename: E:\05 - Scripting\MaxScript\projects\For Scripters\Inspector Gadget\iterators; position: 3
-- Syntax error: at bad, expected <factor>
-- In line: 

Right - am off to bed. Many thanks for the efforts! I think I'm going to let this one gracefully slip away for now :)

CGTalk Moderation
12-04-2008, 03:23 AM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.