Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[MadAngus]

The discussion to-date has highlighted some specific areas as follows:

Code review:
To facilitate a common development environment whereby everyone is working from common knowledge, the entire source code should be reviewed, cleaned up, documented, modularized, and updated (Known bugs fixed).
    Sidewinder

This reflects the statements and initial suggestion by bruceuncle as to what the initial milestones should be.

In software enhancement terms:

Stage1:	AMOS Pro V2.0 as released and fully documented.  This is the "still point" from which development can proceed.  The documentation is well on its way and is databased.  So I can pull sets of bugs and weird behaviour out with simple queries.  That's the groundwork for the next stage.
Stage2:	AMOS Pro V2.1 with bugs fixed and updated docs.  Nothing changed by way of additional functionality.  This is the stable base to build the next stage upon.

    bruceuncle

As it will take some time to review the code the Stage 1 milestone could continue as it is at the moment with the documentation set (currently being written) and version 2.0 being released together as the initial resource kit.

However for the Stage 2 milestone a two part goal could be the primary focus.

a.) The actual code is reviewed and documented.

The documentation would most likely be in the form of comments for each block/line of code. It would be beneficial if these comments were not overly cryptic in their meaning and written as if talking to a laymen. It Can be difficult to understand someone else's logic when it is not simplified for general understanding.

Also someone has to pull these comments out into developer/contributor documentation.

If possible the code should be modeled. As AMOS isn't an object model architecture, it's would be difficult to model it using UML, however it may be possible to produce block diagrams showing function dependencies (other functions etc.), additional models could also be produced were appropriate and relevant. Basically use UML techniques. How does it all fit together, definitely requires further feasibility analysis/thought.

b.) Code updated (Known bugs fixed)

For this a comprehensive list of known bugs will be required.
This would require that all sources of information, AmigaCoding, forums, mailing lists, bruceuncles work etc. be trawled and be collated. A job for some lucky volunteer(s). I did volunteer for this didn't I, sigh!

With this bug documentation at hand, contributors would have a basis for discussion as to which libraries/bugs to prioritise and delegate.


Optimisation:
Lots of comments on this.

Here are some comments (not in any predetermined order) to capture the theme of optimising and it's considerations. There are more in the thread.

If there's any hope of making the compiler optimize code better let me know.  That was always one of the shortcomings of Amos code.  I have heard that AmosPro does bit-shift optimization of multiplies but not much else.

    SamuraiCrow

Where do you stand with regards to extension commands that duplicate (but are more efficient) AMOS commands?

...Wouldn't you love to not need to scour through optimisation 'tips' and just get on with writing your programs? For example, I am constantly frustrated by using GOSUBs where i could/should be using nice tidy procedures... sometimes only 1 or 2 lines long!

    Hungry Horace

In order to get it to be as fast as, say... SAS/C, it would need to be totally reengineered to use register-passing.  All extensions would need to be reengineered to take those changes into account as well.

    SamuraiCrow

Putting in 100 hours work to gain 1 millisecond performance is a waste of resources

    MadAngus

maintain support for current code in AMOS Pro, AMOS The Creator, Easy AMOS, Supports both the 1.3 and 2.x1 extension formats

    bruceuncle

All original commands should remain for compatibility to older code, if they could be optimised without braking compatibility, I do not see why not.

    MadAngus

So from the comments in the thread we surmise this into two categories

a.)What to optimise.
This really comes down to analysis of two types, Visual perspective and Profiling.

With visual perceptive (qualitative analysis) it is simple a case of comparing one technique to another and seeing if the difference can be detected visually while the program is running.

Profiling would most likely be the preferred method whereby measures of space (memory) or time complexity of program, the usage of particular instructions, or frequency and duration of function calls can be analysed and turn show the weakest areas of the program. Any qualitative analysis could also be verified by this technique.
Wikipedia is my friend

In additional the benefit of any optimisation should justify the workload that it entails.

b.) Compatibility
I doubt we would be very popular if any optimisations implemented would render all past code redundant requiring the rewrite of said code or extensions to work in the new distribution.

Unless compatibility considerations are completely disregarded, re-engineering the code using alternative techniques may not be the way to go.


AGA support and other features:
AGA support, who doesn't want it!

However, to attempt to begin implementing AGA enhancements too early in the development cycle may lead to problems fixing old code while new code is being injected. Such as changes to original core code to allow integration of the new AGA code whilst trying to fix an old bug that related to that original core code.

As has been said, trying to dictate what people work on during the course of the development is absolutely out of the question for this project, there will be none of the "We know best" or "we won't implement that because we don't want to work on it/agree with it/like it" attitudes in this project.

So how to accommodate the desires for AGA support and other features (Including optimised code)?

We have been provided with the solution to that problem, Extensions thank you Francois.

This would allow anyone who wishes to develop alternative optimised code or features and role them into an extension (or even an alternative AMOSPro), and at a later date and if feasible role them into the core code. So if anyone decides to work on other things outside of this projects boundaries, know that you will be welcome to submit your work for integration at some point, it would also be appreciated if you would submit your work.

With all documentation being made available to everyone, your projects could follow an inline development process, i.e. as new fixes are applied to the core code you could in turn test your code against the changes. Would make implementation at a later date simpler.

AMOS Pro interpreter and compiler in C

I guess now I can shelve my plans to write a new AMOS Pro interpreter and compiler in C.

    Sidewinder
In regards to the above, I wouldn't say that, you could develop a new AMOS Pro interpreter and compiler in C and make it compatible with old amos code or you could work on an optimised AMOS Pro interpreter and compiler and submit it for integration into the core code. The options are yours to choose.

Here are a few areas that could come under the inline development environment. (Actually their on my wishlist)
AGA support.
TOME Resourced and redeveloped.
Alternative interface.
EasyAMOS tutor for AMOSPro.


Other questions raised:

Which code repo to use, if any? SVN, GIT, HG, CVS
Discuss pros and cons and preference. Maybe a poll.

Language used, stick with 68k assy or port to c.
In the first instance it may be best to stick with the current code language (68k assy) during stage 1 and 2. After these stages a port to c debate could be started.

Could any of the xAmos code be used.
Should development of implementations be designed to be portable so that xAmos can make use of them.
Simple answer, no, as the workload required to port to and from xAMOS would be at the detriment to the actual AMOSPro project.


Project management:
I think I'm running out of my Daily Man Word Count!

Based on the docs I attached on the bottom of this post -> Here, I'll draw something up and submit it. You may want to download these and provide comments/criticisms etc.

Other things to think on:
Project Website
Feature voting system
'Official' versioning system

Plagiarism aside what do you think?


Finally -

Wheres our local hero Spellcoder, we would really appreciate yours and Lonewolf10's opinions, especially with the amount of work that has been done on AmigaCoding and your exquisite knowledge.

[bruceuncle]

Thanks for such a comprehensive overview MadAngus.     And thanks to everyone providing comments and suggestions - keep them coming.

So how to accommodate the desires for AGA support and other features (Including optimised code)?

Step1	Collate all the responses and requests.  MadAngus already doing a good job with this.  I think we need a separate repository for all this stuff as there is useful commentary going on in other threads.  We'll lose it if we don't centralise it soon.
Step2	Agree the deliverable model - AMOS Pro Vx.x
	Isn't the final model just this?: AMOS Pro Core   Launcher   Editor   Monitor   Compiler   LIBS:amos.library   AMOS private specialised library set (helpers) - All *.Libs not prefixed AMOSPro   AMOS public library set (the language itself) - All *.Libs prefixed AMOSPro All other functionality is implemented through Accessories written in AMOS itself (if you see my point  ) This provides the boundary line for what is 'in' the project and what is 'out'.  The 'scope' of the project.
Step3	Document the above and that's Stage 1 complete.
Step4	Using the same scope:  fix bugs;  fix 'unusual' behaviour but only where it will not break existing code;  optimise where possible without breaking the model.  Document all the changes. And I would definitely rule out register passing for parameters here.  Not just for the effort involved in changing the entire model, but because it's not that overhead that slows AMOS down (see end of this post).  And, as I've previously stated in this forum, Francois went to a lot of trouble to allow easy integration of machine-code into an AMOS program.  Where AMOS is too slow for you, that's the time to learn a bit of assembler and speed it up yourself.  If it's a specific set of functionality, maybe even write an extension.
Step4	Document the above and that's Stage 2 complete.

The above is a lot of work  , so I won't speculate any further.  There are many routes that could be taken after that work is complete to get to an AMOS Pro V3.x.  I would hope that we'll have the experience after the above load of work to know which is best at that time.

But the point MadAngus made is very pertinent:

We have been provided with the solution to that problem, Extensions thank you Francois.

All the more reason to maintain a stable core for the product.  While we're doing Stage 1 and Stage 2, we don't want to do anything to upset any parallel development going on in Extensions.  People can then have confidence that their own efforts will be stable and useable in the future.  (Preferably without having to re-compile against a changed set of system equates ).
In the meantime, we're information mining.  So any references people feel would be useful, from this and any other forums, please let us know.  We also don't want to re-invent the wheel  as there's a lot to do already!

Note on performance in AMOS.
Any compiler has to make compromises.  It doesn't 'know' what the programmer intended, so it has to treat all high-level instructions as equal.  It may attempt to 'recognise' some common programming constructs and choose to use a different method to gain a little speed.  But the extent of that functionality is limited by cost versus gain.
The 'hit' on performance is a cost-per-instruction in the compiler.  (And much more obvious in an interpreter where it includes operator tokenising and look-ups - all small, but significant, 'time wasters' when compared to assembler code.).
So, if an instruction is very powerful and can do a lot of work, it can appear to be very fast.  All its code is machine-code dedicated to doing that complex task as fast as possible.  For example, Copy Start,Finish To Destination is fast because its cost-per-instruction is pretty negligible compared to what it does. 
At the other end of the spectrum, Rol.l Number,BinaryValue is only a few machine-code instructions.  The overhead of its cost-per-instruction hit is much higher compared to what it does.  Bear in mind that this includes fetching the BinaryValue from the variable storage.  Tiny but significant as, if you programmed it yourself in assembler, that Rol.l would be in context and would probably only involve registers.
Don't get the wrong impression.  I'm all in favour of doing whatever we can to get the best possible performance out of compiled (and interpreted!) AMOS.  I just want people to realise that there are obvious limits and to see what those limits are.

So I'll keep pushing my own agenda    "If AMOS isn't fast enough, do it in assembler and integrate it!" 

Use AMOS to do the grunt work setting up an application.  Then optimise any necessary bits with integrated assembler.  If you're getting that close to performance issues in an app, the step to assembler is pretty easy.  You're probably already doing some rough pseudo-assembler with Peek, Poke, Deek, Doke, Leek, Loke, Rol.l, Rol.w, Rol.b, Bchg, Btst, Bset, etc, etc anyway.

[MadAngus]

Devils Advocate:

Note on performance in AMOS...

Yea, I know I don't know what I'm talking about, more's the fun. Folks if you disagree with bruceuncle's agenda then profile/document your ideas and prove that they should be implemented. Evidence based.

Now I'm going to jump into the bunker before bruceuncle lobs a grenade my way.

[MadAngus]

Yea, I know I don't know what I'm talking about.

After doing some reading of documents and the Dragon book, I've been thinking about the above comment I made.

I have serious doubts about my suitability as any kind of project manager. This is obviously due to my lack of knowledge of 68000 assembler, the AMOS architecture and compiler/interpreter development in general.

The possibility of a new AMOSPro release has happened far sooner than I expected. I believed that this would take place around the end of next year giving me a year to do some intense studying in relation to this project.

With the likelihood of the source being available within the next couple of months there is no possibility that i could be upto speed to do this role justice or keep up with the technical discussions that are no doubt going to take place.

I'm going to withdraw myself from this role and step back, I'll continue with the general documentation side of things in the AMOS Manuals / AMOS resource kit so I'll still involved in that sense.

Instead I am going to focus on the learning side of things with the aim of researching and developing some of the wishlist items that I have, i.e. 'Alternative interface', 'EasyAMOS tutor for AMOSPro', 'TOME Resourced and redeveloped' and 'AGA support'.

As these are side projects I can progress on them at my leisure as my knowledge increases.

Not to fear, If i think I have something to contribute during the development I will jump right in.

[bruceuncle]

I wouldn't worry too much about the timescale MadAngus.  It will probably take around twelve months before anyone understands the V2.x sources.  This is a complex system and I don't think anyone will be able to jump right in and make immediate value-judgements on where to go with it in terms of change.

I'm assuming (always a bad mistake - "All projects fail on assumptions.") that the Stage 1 and Stage 2 approach will be the necessary grounding in AMOS Pro architecture that we'll all need before we can move it into V3.x in any meaningful way.  And I can't see that happening inside of that timescale.  I've done enough 'looking around' in V2.x to see that it does differ significantly from the public V1.3 sources currently available.  So even someone steeped in that knowledge is going to find it 'challenging'.  Add in the French-English translations required to understand some of the comments and it gets time-consuming to say the least.  It's one thing to generate complex source whilst in the middle of programming something.  It's another thing entirely to understand that source years later without any access to the thoughts and processes that went into the design.

Anyway.  I'd vote for MadAngus to retain project management of the documentary stages.  He does a great job with it and has more energy than I've seen around lately.  When it gets to the actual change-management stages and understanding the code/architecture/intent required for that task, I'm pretty sure another year will have passed and maybe MadAngus might feel different then?

And we need someone to collate all these requests right now, don't we?

In the meantime, when the sources are available, if anyone wants to jump right in and play with stuff, then there's no reason not to.  Just bear in mind that, to me at least, the major objective of the project is to make available an 'approved' AMOS Pro software set that people can rely on.  That's all.  And I think that fits very well with the objectives of the AMOS Pro Resource Kit Project that MadAngus has so heavily promoted and pushed along.

PS.  I'm the world's worst project manager.  It took me years to achieve that happy state.  I'm happy to stay that way. 

[Hungry Horace]

Great news! I've just today received permission from Francois to have Pietro release the sourcecode .

Just going back to the all-important element of this that it is all too easy to lose track of.... any news on the actual source release?

[SamuraiCrow]

MadAngus:
The Dragon Book is a hard book to get through.  I wish I had a copy though.

If time is a concern for AMOS 3.0, I'd suggest looking into some compiler middleware such as LLVM to take care of the optimization.  Sidewinder had expressed an interest in making an AROS 68k backend for LLVM.  Since he made the x86 LLVM cross compiler for AROS, he's probably the most qualified person to talk to on the subject.  I think the hardest part would be producing Amiga Hunk-style binaries instead of the ELF binaries used by AROS.

BruceUncle and Thread:
The problem with trying to update the AMOS compiler is that it generates non-optimal Assembly code internally.  All of the function/procedure calls use stack-based parameter passing.  Register loading optimization is not a trivial task to get encumbered with Assembly code writing.  I think the best way to fix it is to look into another open-source compiler with more extensive optimization techniques regardless of the language it's written in.

If AMOS 3.0 generated C source code or some other intermediate code like LLVM bitcode, we could use the optimizer of that compiler to get around the shortcomings of AMOS' primitive code generator.  The main obstacle to this is that advanced compilers require more memory to operate than what the average Amiga has.

Optimizing assemblers such as VAsm or DevPac only do a "peephole" optimization technique while a macro-optimizing compiler can do much more advanced rearrangement of code including branch-elimination, register-loading optimization, function inlining, an occasional loop-unroll, and others.

Of course this will have nothing to do with the 2.1 release of AMOS but I propose that the 2.1 release be the last version of AMOS to try to compile on a 1 Meg system.  This will give the compiler room to operate when generating much more optimal code.

[bruceuncle]

Of course this will have nothing to do with the 2.1 release of AMOS but I propose that the 2.1 release be the last version of AMOS to try to compile on a 1 Meg system.

Agreed.  And thanks for the feedback.

I'm pretty convinced that we need to get to V2.1 before we decide how to get to V3.x.  There are some problems with AMOS Pro that are nothing to do with performance - compiler crashes, inconsistent limits, 'odd' behaviour, other bugs, etc.  I think we need to fully understand how to fix all that within the current architecture before throwing it open to any approaches that attempt to fix performance issues by changing the architecture.

That gives us the stable base on which to build.  I don't think it would be much fun trying to both fix bugs and radically change the architecture at the same time.  We've also got to understand what effort would be involved in any post-V2.1 developments.  We may be in a non-commercial environment but I see no point in spending an excessive amount of time to shave milliseconds here and there.  That effort would be far better spent on rationalising extra functionality.  And encouraging integration of machine-code in an AMOS app where speed becomes an isssue.  I'm definitely not saying we don't so any optimisation (it would be hard to resist  ) but we need to keep it in perspective.

My own wish would be an ability to add in-line assembler to AMOS.  Something simple along the lines of Asm On and Asm Off instructions with everything between being DevPac syntax (for source compatibility with the AMOS includes).  And using the existing Areg and Dreg instructions to communicate.  Not as a core language feature, but as an Extension.  I remember seeing something mentioned about an AMOS assembler way, way back in time.  Anyone know if it went anywhere?

There's a lot of interest in AGA support.  Could those interested define what aspects of AGA they want to use?  I've only just started looking at what's involved with AGA (more precious time stolen when I should be documenting, but all work and no play... etc.)  Came across this page which looks interesting:
http://www.mways.co.uk/amiga/howtocode/text/aga.php

I'm new to today's Amiga environment.  So I may well come across stuff that's old hat to some people in this forum.  Please let me know 

[MadAngus]

I'll just have classify it as a challenge, time to slay the Dragon.

It seems to me from reading these posts there are two ideologies developing indicating that v2.1 AMOSpro could get forked into two architectures. Which is not a bad thing.

Quite possibly as follows.

Note: User code: The code that a user developes for their own project (not the internal code of AMOSPro).

AMOSpro Classic:
AMOSpro as is whereby bug fixes are implemented and any optimisations/additions that provide a distinct performance gain or user/developer benefit are integrated, AGA support for example. Of course optimisations/additions should be compatible with the 2.1 model and user code.

AMOSpro (Add working title here):
A complete new architecture developed (for the Amiga) based on the current model (even loosely). It would be sensible to add a constraint to this in that it supports, compiles and and is backward compatible to any legacy user code from AMOSPro classic.

For me project management would stop at the AMOSPro Classic architecture and enhancements. For a complete new Amiga AMOSPro architecture someone else can take up the project management reins.

This thread is a holding place for the discussion of hacking, updates and future versions of AMOSPro for Amiga 68k.

I would like to remind everyone that this thread is for discussing any AMOSPro ideologies, architectures, potential enhancements etc, etc, as I suspect many of them will cross over.

When it comes time to focus on the development within a specific framework a new thread will be created for that purpose with this one being used as a reference point.

Nothing is fixed here, consider this a think tank.

[MadAngus]

On a little side note, I would like to highlight that my referral to Dimac in a previous post is a martial art referenced in the book The Book Of Ultimate Truths by Robert Rankin.

Author of:
The Sprouts That Ate Christmas.
The Occult World Of XML And Norman.
The Haberdashery Book Of Weaponry.
and other such books that may or may may not exist.

[bruceuncle]

For anyone interested in 68000 optimisation, I've attached the Instruction Execution Times pages from Motorola's 68000 manual.

Makes interesting reading if you've never tried working out how many clock cycles your code is taking.  Take particlar note of the cost of interrupts - all the internal calling from one relocatable-code-block to another in the *.Libs files uses the FLINE interrupt.  It's the cost involved for having a small executable when the code's compiled.

[Edit by MadAngus 9/10/12] Nothing serious. Changed the attached file name, filled the spaces with underscores. The download system truncates the name to the first found space and leaves everything else out including the extension. It's the same file.

[SamuraiCrow]

Keep in mind that AMOS was based on the earlier STOS Basic for the Atari ST.  AMOS may still contain evidence of Atari ST-isms as a result.  I think those FLINE interrupts may be one of them.

[bruceuncle]

I think those FLINE interrupts may be one of them.

Trouble is, they're what's used to make each 'instruction chunk' independantly relocatable.  So the compiler can 'cherry pick' just the code it needs for the program it's compiling.  Which saved a lot of space as the usual alternative is to plonk the whole library into the compiled executable.  Also allows a 'scatter load' into available memory instead of having to have a contiguous slab available for a full library load.

It's what I refer to as 'the existing architecture' as it's pretty fundamental to the way V2.x works.

Any ideas as to how else to implement relocatable 'instruction chunks'?

If the core AMOS Pro libraries were re-written (not a pleasant task!) to resemble traditional LVO_ libraries, you'd get quite a reduction in the 'per instruction' overheads at the expense of needing more memory.  The big downside of that approach is that all existing extensions would be affected and require re-writing!

Maybe a hybrid approach would work:  AMOS Pro 'core' libraries re-written but the Interpreter and Compiler still able to handle the existing FLINE Extension libraries.  This 'two-faced' approach seems to have been used to enable both AMOS Pro V2.x and V1.3 libraries to still both work despite different formats and 'instruction numbers' for the tokens (deduced from a bit of exploration, so I may be wrong).

[SamuraiCrow]

If using a standard linker and object files are not going to work easily, then what are the other options?  I don't think many people are running on 1 meg systems any more if additional memory is required.

--edit--
Are you referring to the compiler only or the entire system?

--edit2--
Sidewinder and I discussed a few weeks ago that there might be the possibility of using another extension format in AmosPro 3+ that gets around the extension slot limitation of the AmosPro 2.x so that the older format could be used only with a backward-compatibility "harness" to get it to work.

[Hungry Horace]

Great news! I've just today received permission from Francois to have Pietro release the sourcecode .

Just going back to the all-important element of this that it is all too easy to lose track of.... any news on the actual source release?

errr....  any news? at all?