Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Karramarro]

Hello,

My name is Javier and I am a former Amiga user.
Several years ago I discovered AROS and thought that it could be interesting to contribute but had no much time to end any project until now.

This summer remembered the XAMOS project and a thread about programming languages I read from the aros-exec forum talking about several lacks on XAMOS (expecially the "FOR TO STEP NEXT" control structure) and when tried to add this feature found that it has neither lexical nor syntax analyzer.
Thought that it could be good to remember the times I was on the university, dust off the Dragon Book and create a lexical and syntax analyzer to use with XAMOS.

Well, at this moment, the analyzer can parse several AMOS structures, tell if a simple program is syntactically correct or not and return a syntax tree.

I have followed the "AMOS Professional User guide" by Mel Croucher and Stephen Hill to try to figure out the grammar of the language but I have a question and have not found help on the book.

Well, the first one is about the functions as it seems to be several types of functions:

1.- The functionname (parameter,parameter...) where the (parameter,parameter...) part is optional and the number of parameters are optional
This kind of function is, lexically, an identifier and it's parsed OK.

2.- The functionname parameter parameter,parameter,... as in the following code from page 123:

Do
Cls : Wind Open Rnd(99)+1,1,1,25,5,1
Print "Window number ";Windon : Wait Key
Loop

Where "Wind Open Rnd(99)+1,1,1,25,5,1" is a function of this kind. with two identifiers, and,as the function "locate ,number" also is valid, the parameter list can have empty parameters.

I have thought that a grammar for this kind of functions could be

FUNCTION := IDENTIFIER {IDENTIFIER} {EXPRESSION {','{EXPRESSION}}*}

But, should the first parameter (the Open on the example) be treated as an expression?
Could there be more parameters before the parameter list?

Thank you for your help.

[Mia]

Hello,

This summer remembered the XAMOS project and a thread about programming languages I read from the aros-exec forum talking about several lacks on XAMOS (expecially the "FOR TO STEP NEXT" control structure) and when tried to add this feature found that it has neither lexical nor syntax analyzer.
Thought that it could be good to remember the times I was on the university, dust off the Dragon Book and create a lexical and syntax analyzer to use with XAMOS.

Where "Wind Open Rnd(99)+1,1,1,25,5,1"

FUNCTION := IDENTIFIER {IDENTIFIER} {EXPRESSION {','{EXPRESSION}}*}

I saw that someone is attemping a lexer in python, maybe they have something useful

One thing I have noticed is that the each command does seem to have the ability to have it's own syntax with some comands of the like Command N,X,Y To X,Y

and then other commands which you would think need the "To" but don't use it.

and some commands are 3 words with spaces between them - I see this in the extensions.

[Sidewinder]

1.- The functionname (parameter,parameter...) where the (parameter,parameter...) part is optional and the number of parameters are optional
This kind of function is, lexically, an identifier and it's parsed OK.

2.- The functionname parameter parameter,parameter,... as in the following code from page 123:

Do
Cls : Wind Open Rnd(99)+1,1,1,25,5,1
Print "Window number ";Windon : Wait Key
Loop

Where "Wind Open Rnd(99)+1,1,1,25,5,1" is a function of this kind. with two identifiers, and,as the function "locate ,number" also is valid, the parameter list can have empty parameters.

I have thought that a grammar for this kind of functions could be

FUNCTION := IDENTIFIER {IDENTIFIER} {EXPRESSION {','{EXPRESSION}}*}

But, should the first parameter (the Open on the example) be treated as an expression?
Could there be more parameters before the parameter list?

I would point out that the two "function types" you show above are actually different in a very fundamental way, namely #1 returns a value and #2 does not return a value.

#1 is what would be considered a function in normal BASIC syntax.  Functions are expressions.  In your Wind Open example you see the function Rnd(99)--it returns a random integer value between 0 and 99.  In a grammar functions would fall under expressions:

expression := {INTEGER} | {FUNCTION}
integer := [0-9]*
function := "Rnd" '(' expression ')'

#2 is not a function because it does not return a value.  This means that it cannot be used in an expression.  You could categorize it as a procedure or subroutine, but those are usually user defined.  Language defined procedures are usually called commands.  In the grammar commands are typically statements:

statement := {COMMAND} | {ASSIGNMENT} | {COMMENT}
command := "Wind Open" {EXPRESSION} ',' {EXPRESSION} ',' {EXPRESSION} ',' {EXPRESSION} ',' {EXPRESSION} ',' {EXPRESSION}

In regard to Mia's comment about some commands that use a "To" token, that could look something like:

command := wind_open_command | line_command
wind_open_command := "Wind Open" {EXPRESSION} ',' {EXPRESSION} ',' {EXPRESSION} ',' {EXPRESSION} ',' {EXPRESSION} ',' {EXPRESSION}
line_command := "Line" {EXPRESSION} ',' {EXPRESSION} "To" {EXPRESSION} ',' {EXPRESSION}

I've spent my share of time writing a grammar for AMOS BASIC (unfortunately for a project that never was completed) so I understand the desire to make very general grammar rules, but I've found that if the rules are too general, then the parser will spend more time later trying to match specific constructs using string compare instructions (read very slow) or other methods to disambiguate the grammar.  It is often easier (and faster) to let the grammar do the disambiguation, although it does create larger grammars.

Good luck on your project!

[Sidewinder]

Also, creating optional expressions is useful in the AMOS language and can be done using a new expression rule:

optional_expression := /* empty */ | {EXPRESSION}

This can then be used in say, the Bob command:

bob_command := "Bob" {EXPRESSION} ',' {OPTIONAL_EXPRESSION} ',' {OPTIONAL_EXPRESSION} ',' {OPTIONAL_EXPRESSION}

In this example, the first argument to the bob command is required (it is the bob id number), but the other three (the X and Y coordinate, and image number) are optional.  The commas, however, are required.  This allows parsing of statements like:

Bob 1,X+2,,

Which would move bob 1 two pixels to the right of X, but leave the Y coord and the image unchanged.

[Karramarro]

Thank you for your help.

I knew that the problem I had had something to do with the names (I called them "function types"): functions, procedures and commands.

My grammar is quite simple, and of course it should be improved, at this time it looks like:

PROGRAM -> SENTENCE_SEQUENCE

SENTENCE_SEQUENCE ->    SENTENCE {':'|'\n' {SENTENCE_SEQUENCE}}   

SENTENCE ->    IF_SENTENCE
       |ASSIGNEMENT
        |WHILE
        |FOR
        |REPEAT
        |DO
        |EXIT
       |ON
       |PROCEDUREDEF
       |GOTO
       |GOSUB
       |DIM
       |PRINT
        |LABEL

IF_SENTENCE -> 'if' EXPRESION '\n' SENTENCE_SEQUENCE 'end if'
       |'if' EXPRESION '\n' SENTENCE_SEQUENCE 'else' SENTENCE_SEQUENCE  ('end if'|'endif')


ASSIGNEMENT -> FACTOR '=' EXPRESION

WHILE ->    'while' EXPRESION '\n' SENTENCE_SEQUENCE 'wend'

FOR ->    'for' ASIGNEMENT 'to' VALUE {'step' VALUE} '\n' SENTENCE_SEQUENCE 'next' IDENTIFIER

REPEAT ->     'repeat' '\n' SENTENCE_SEQUENCE 'until' EXPRESION
 
DO ->     'do' '\n' SENTENCE_SEQUENCE 'loop'

EXIT ->       'exit' {IF EXPRESION{, VALUE}|VALUE}

ON ->       'on' EXPRESSION ('proc'|'goto'|'gosub') IDENTIFIER {,IDENTIFIER}*

PROCEDUREDEF ->   'procedure' IDENTIFIER {'[' IDENTIFIER {',' IDENTIFIER}*']'}* SENTENCE-SEQUENCE 'end'  'proc'{'['expression']'}

GOTO ->       'goto' IDENTIFIER

GOSUB ->       'gosub' IDENTIFIER

DIM ->  IDENTIFIER '('INTEGER{',' INTEGER}* ')' {',' IDENTIFIER '('INTEGER{',' INTEGER}* ')'}*

PRINT ->  'print' EXPRESSION {';' EXPRESSION}*

EXPRESSION -> EXPRESSION_PLUS {opcomp EXPRESSION}
 opcomp => '='|'<'|'<>'|'<='|'>'|'>='

EXPRESSION_PLUS -> TERM {opadd EXPRESSION_PLUS}
 opadd => '+'|'-'|'or'

TERM ->  FACTOR {opmul TERM}
opmul => '/'|'*'|'and'

FACTOR -> EXPONENT {'^' FACTOR}

EXPONENT --> opadd FACTOR | final
 
FINAL ->     '(' EXPRESSION ')'
       | VALUE
       | VAR

VAR ->  IDENTIFIER {'('EXPRESSION {',' EXPRESSION }* ')'}*

I dont know if every command has to have it's own grammar, it's true that comparing strings is slow but thought that a more general grammar could let the language be more easily developed. New commands could be added without changing the parser. Perhaps I should think about it again.

I'm afraid that my Amos Basic experience is very low, I have never heard about the "Y To X" expression as a parameter.
They only appears on commands?

[SamuraiCrow]

Yes, it only happens in commands to use x to y.

[Sidewinder]

That's a really good grammar you have so far!  Using the terms of your grammar, what I termed "commands" would be additional "sentences".

Just a couple of notes, there is a third form of the "If" clause that uses Then:  If X>0 Then Y=0 Else Y=1
That's always a fun one to handle.   

Also, you've hard coded new line codes into the grammar.  This could cause problems for people who do not like white space.  Something like the loop below would not be recognized:

Do : X = X + 1 : Print X : Loop

Also, the Print statement can also have a comma separating the values (a comma tells AMOS to insert white space between the values when they are printed and a semicolon tells AMOS to not insert the white space).  I realize this is a trivial item, but I'm not sure how detailed you want to make your parser.


You certainly don't need to make every command have its own grammar rule.  I mostly did that for clarity in my response.  You can absolutely make general rules that match a whole bunch of commands.  I chose not to for my parser because I didn't want to do the string compare in the actions.  It was a personal choice, really.  If you're more comfortable sorting things out in the actions, please don't let me dissuade you.

Making the grammar extensible (able to recognize new instructions) is a hard problem even if you keep the grammar relatively generic.  There will need to be hooks that call new actions for the new commands.  I'm not exactly sure how AMOS did this with its extension system, but one solution (the one I was looking at) was to create a separate grammar/parser for each extension.  If a command was not recognized by the main parser, it would call the extension parsers, one after the other, until a match was found or control returned to the main parser.  I don't know if that makes sense for the project you're working on, but it's one idea.  If you have other ideas, I'd be glad to hear them.  I spent a long time looking for a good solution to that problem.

The "To" delimiter isn't used very often, mainly in drawing routines like Draw, Box, Clip, etc.  Usually it's used to define the two end points on a line, or the dimensions of a box.  It's just one of those "gottcha" things that AMOS throws in to keep us on our toes.   

[Karramarro]

So, the print command grammar changes to:

PRINT ->  'print' EXPRESSION {( ';' | ',' ) EXPRESSION}*

And yes, it's true, the line:
Do : X = X + 1 : Print X : Loop

Throws a syntax error:

Syntax Error on 1,4:
Do : X = X + 1 : Print X : Loop
----^
Unexpected Token Type
EXPECTED 19, RECEIVED 15

Where the 19 is the token code for the End Of Line (EOL) and the 15 is the code for the colon.
This error codes will be improved and it won't show this meaningless codes. It is on the todo list.

In a previous version there was no difference between them but as there seems to be a diference when using REM and ' ' ' I included the difference.
The change will take little time: check for both of them and then match the token that is.

I have checked the " If X>0 Then Y=0 Else Y=1" structure and it is parsed ok, it seems to be a documentation error as I remember having "lots of fun" with all the if structures.

The expression 'to' experssion thing seems easy:

TOEXPRESSION -> EXPRESSION {'to' EXPRESSION}

That will only be used on the commands.

Now, I have to find a grammar for the commands and how to tell what statements are commands, what are assignments (easy as they have a '='token) and what are just prodecures without accepting something like this:

commandname$(1,3,5+b) commandparameter#("foo",3) 1,4 to 10,,"HELLO"
That could be accepted with a simple grammar:
COMMAND -> VAR {VAR} {{TOEXPRESSION} ','}* {TOEXPRESSION}

Well, perhaps it could be lexically accepted and throw the error on run time... Don't know.

Thank you for your help.

[Sidewinder]

Regarding the difference between REM and ':  the ' is only used at the beginning of a line.  It can have white space in front of it, but that comment will be the only thing on the line.

I'm glad the If...Then...Else construct is working.  That can be a tricky one.

It looks like you've got a handle on this.  Let me know if I can be of any further help.