What does TokenType mean?

[williamy]

After digged into the details of the source code, i realised this project is amazingly powerful.
I am confident that I understood most of the code i read, but currently i am stucked at the configuration file of the SyntAnalyzer.

Can you please help to explain the idea of TokenRule & BlockRule in the SyntAnalyzer?

e.g.
TokenRule properties:

FRegExpr: TecRegExpr;
FTokenType: Integer;
FOnMatchToken: TOnMatchToken;
FColumnTo: Integer;
FColumnFrom: Integer;

config in text:
item
DisplayName = 'Code block start'
StyleName = 'Code border'
TokenType = 9
Expression = '<\?(php)?'
ColumnFrom = 0
ColumnTo = 0

By matching the StyleName, i think the StyleName is used to links up with the StyleFormat. But I really dont understand the popuse of TokenType, ColumnFrom, ColumnTo.

Can you please help to elaborate? Thanks!

The BlockRule is more complex. and I am really trying to understand how it works. but failed to find any document/information about it. is it possible to create a wiki page for this 2 classes?

[uvviewsoft]

>project is amazingly powerful.

thank you.

>DisplayName = 'Code block start'
StyleName = 'Code border'
TokenType = 9

>But I really dont understand the popuse of TokenType, ColumnFrom, ColumnTo.

Tokentype is index in the TokenTypes stringlist. here its 9. so we have stringlist with at least 10 items, it is in the Lexer Config dialog in SynWrite UI, and here we use item#9. we assign item#9 to this style.

ColumnFrom = 0
ColumnTo = 0

these are almost not used at all. if they are set, the parser will find only text starting at column ColumnFrom until column ColumnTo.

[uvviewsoft]

>The BlockRule is more complex. and I am really trying to understand how it works.

see some example lexer. find BLockRule in LCF file. for ex

Code: Select all

 BlockRules = <
    item
      DisplayName = 'Id registers'
      StyleName = 'Id registers'
      BlockType = btTagDetect
      ConditionList = <
        item
          TagList.Strings = (
            'ah'
            'al'
            'ax'
            'bh'
            'bl'
            'bp'
            'bx'
            'ch'
            'cl'
            'cs'
            'cx'
            'dh'
            'di'
            'ds'
            'dx'

here we use BLockRule with kind "tag detecter" which will find keywords for this rule.
BLockRule can have other kinds. e.g. "range start", "range end".
you can see my OpenOffice help files in the readme/tutorials folder.