Ultimate Amiga

Network Boards => AMOS Professional Re-Development => AMOS Factory => General Discussion => Topic started by: MadAngus on December 07, 2012, 08:15:41 PM

Title: Source code versioning system and repo
Post by: MadAngus on December 07, 2012, 08:15:41 PM
Alrighty folks :

Now that the source has been released it is now time to get it into a source repository and versioning system. This will help alleviate the potential confusion that multiple development streams can have without some root repository.

The two main repositories are Google and SourceForge, there are however others, also which versioning system to use SVN, GIT, HG, CVS. A decision on this should made ASAP.

For the release/build versioning I would recommend a system already defined rather than expending the time and effort creating a new one. One example is the Eclipse.org system defined in Eclipse.org Version Numbering (http://wiki.eclipse.org/Version_Numbering) document.

Whatever repository the code is placed in a project name is required. This name should cover the ClassicAMOS development and the eventual AMOS 3.x branch. Think of one!

Only the core developers will have write access to the repository with everyone else given anonymous read only access.

Q:
Who will create the repository and be the repository manager/maintainer. (Not me)

@ghizzo
A written licence would be preferred for use in the repository and to be distributed with the source. It could be based on the BSD licence modifyed to specify that the code and compiled binaries can only be used for non-commercial purposes and include a copyright statement specifying the copyright holder. This licence could be called the
AMOS Public Licence (AMOSPL).

What do you think?

NotE: For the core developers and the repository manager/maintainer
All source modules from the original release should have a header added stating the licence and copyright holder. This will ensure there in no confusion between the original code modules and any additional code modules.
Title: Re: Source code versioning system and repo
Post by: bruceuncle on December 08, 2012, 10:21:21 AM
Good thinking. 8)

My biggest worry is the mayhem that may ensue from multiple developers working on AMOS.  It's not exactly well partitioned.  The library format and 'relocatable building bricks' do give some natural partitioning, but there is close coupling between a lot of the 'bits'.

Quote
Stage 1:  AMOS Pro V2.0 as released and fully documented
Stage 2:  AMOS Pro V2.1 with bugs fixed and updated docs
Stage 3:  AMOS Pro V2.2 with approved extensions and updated docs
Stage 4:  AMOS Pro V3.0 for AGA and fully documented

For V2.0.x versions, would suggest an approach of:
I'm probably stating the bl**ding obvious above but it's essential that we all appreciate what the environment will be.

Any suggestions welcome as usual.

I'm unsure whether any work could proceed on the above before the docs phase V2.0.x is complete.  Maybe they could run in parallel if anyone's keen to get their hand dirty.   ;)
Title: Re: Source code versioning system and repo
Post by: Lonewolf10 on January 05, 2013, 12:47:33 AM
Quote from: bruceuncle on December 08, 2012, 10:21:21 AM
My biggest worry is the mayhem that may ensue from multiple developers working on AMOS.  It's not exactly well partitioned.  The library format and 'relocatable building bricks' do give some natural partitioning, but there is close coupling between a lot of the 'bits'.

It is going to be difficult to fix/add everything that has been mentioned in "the plan" without multiple people working on it. I think everyone should have their own source files, comment out (with ; or a *) uneeded code and stick all new code within "asterisk fields", e.g.:

Code: [Select]
   dc.b "Original code here..."
*****\
   dc.b "New code here..."
*****/
;   dc.b "Commented out code"

I used string fields just to make it all clear ;)
I only recently started using what I have dubbed as "asterisk fields" whilst debugging my current AMOS project (Tile Master). It makes seeing original code and new code easier to spot.

Once something has been fixed/added, it can then be added to the master code (a separate set of files only available to one person?). That would avoid possible confusion with multiple people working with the same files.
Title: Re: Source code versioning system and repo
Post by: bruceuncle on January 05, 2013, 01:54:09 PM
Quote from: Lonewolf10 on January 05, 2013, 12:47:33 AM
It is going to be difficult to fix/add everything that has been mentioned in "the plan" without multiple people working on it. I think everyone should have their own source files, comment out (with ; or a *) uneeded code and stick all new code within "asterisk fields", e.g.:

Code: [Select]
   dc.b "Original code here..."
*****\
   dc.b "New code here..."
*****/
;   dc.b "Commented out code"

I used string fields just to make it all clear ;)
I only recently started using what I have dubbed as "asterisk fields" whilst debugging my current AMOS project (Tile Master). It makes seeing original code and new code easier to spot.

Once something has been fixed/added, it can then be added to the master code (a separate set of files only available to one person?). That would avoid possible confusion with multiple people working with the same files.

It's going to be a tricky probelm to control.  I like the idea of using a consistent 'marker' to highlight changes with 'original code' commented out.  I use that technique a lot in VBA and VB.NET (wash my mouth out!).

BUT, before we get that far, we've got to arrive at a set of sources that will actually assemble into AMOS Pro V2.0. 

I mentioned here:
http://www.ultimateamiga.co.uk/index.php/topic,9520.msg44972.html#msg44972 (http://www.ultimateamiga.co.uk/index.php/topic,9520.msg44972.html#msg44972)
that at least +w.s is corrupt.  Looks like a fix is half-way programmed in it (?).

So I've self-delegated the task of reconstructing it from the amiga.library load file in conjunction with the +w.s source.  And translating the Frech comments as I go.  It's a lot easier than it sounds using Resource and the symbol files I posted here:
http://www.ultimateamiga.co.uk/index.php/topic,9604.msg44953.html#msg44953
Resource tells me I'm 10% of the way through it.

Has anyone else looked at the other sources yet for completeness?

Other bits of interest:

So that's all the other stuff that's going on in conjunction with these sources.

I need a few days to re-group.  Then I'll post what I think would be a suitable plan of attack.  The first step will most likely be a matching of source files to the AMOS Pro file set...
Title: Re: Source code versioning system and repo
Post by: Lonewolf10 on January 05, 2013, 11:59:33 PM
Quote from: bruceuncle on January 05, 2013, 01:54:09 PM
Has anyone else looked at the other sources yet for completeness?

No :(
I glanced at some of the files after downloading the source, but have been tied up with other stuff ever since. I'm unlikely to have free time for a while, but will try to have a look at the "+w.s" file.
Title: Re: Source code versioning system and repo
Post by: Lonewolf10 on January 07, 2013, 08:28:02 PM
Quote from: bruceuncle on January 05, 2013, 01:54:09 PM
I mentioned here:
http://www.ultimateamiga.co.uk/index.php/topic,9520.msg44972.html#msg44972 (http://www.ultimateamiga.co.uk/index.php/topic,9520.msg44972.html#msg44972)
that at least +w.s is corrupt.  Looks like a fix is half-way programmed in it (?).

It doesn't look corrupt to me. I tried assembling it and found these files

Code: [Select]
Include "layers_lib.i"
Include "graphics_lib.i"

weren't with the other files, but I did find them in the includes files that came with DevPac 3.18 :)

However, some of the variables used to define some other variables in the WB OS includes files are missing, and those are what appears to be generating the errors when assembling.
The problem with the way the code is written is that sometimes code references/jumps to code found in other files and can make it very hard to follow.

Can I suggest that you add the Includes files into your references (e.g File "A" requires files "B","C" etc.) and perhaps build a tree so it's easy to see which files are required to attempt to reassemble the sourcecode?
Title: Re: Source code versioning system and repo
Post by: bruceuncle on January 08, 2013, 06:42:15 AM
Quote from: Lonewolf10 on January 07, 2013, 08:28:02 PM
Quote from: bruceuncle on January 05, 2013, 01:54:09 PM
I mentioned here:
http://www.ultimateamiga.co.uk/index.php/topic,9520.msg44972.html#msg44972 (http://www.ultimateamiga.co.uk/index.php/topic,9520.msg44972.html#msg44972)
that at least +w.s is corrupt.  Looks like a fix is half-way programmed in it (?).

It doesn't look corrupt to me. I tried assembling it and found these files

Code: [Select]
Include "layers_lib.i"
Include "graphics_lib.i"

weren't with the other files, but I did find them in the includes files that came with DevPac 3.18 :)

However, some of the variables used to define some other variables in the WB OS includes files are missing, and those are what appears to be generating the errors when assembling.
The problem with the way the code is written is that sometimes code references/jumps to code found in other files and can make it very hard to follow.

Can I suggest that you add the Includes files into your references (e.g File "A" requires files "B","C" etc.) and perhaps build a tree so it's easy to see which files are required to attempt to reassemble the sourcecode?

Thanks for the excellent feedback Lonewolf10.  So far, I've only actually assembled the equates files to derive symbols for Resource to use.  This stuff is coming from chasing a few AMOS Basic instructions up for the Reference Manual.  I found this particular problem checking whether AMOS Memory Banks are cleared when they're created (they are).  So, accepted that there are symbols and labels all over the place.  That's not what I'm referring to.

Examples from the +w.s file:

Line 9,983 onwards is part of the jump table for system functions in amiga.library:

   bra   ChipMM         ;55-SyChip:    Reserve CHIP
   bra   FastMM         ;56-SyFast:    Reserve FAST

You won't find those symbols as destination labels anywhere in the sources nor the Amiga Includes directory and sub-directories.

A little further on, you'll find their 'replacements' at line 10,020:

   bra   WMemFast      ;92-MemFast      
   bra   WMemChip      ;93-MemChip      

There are a few other labels in that jump table that seem to be both 'legacy' and 'current'.

Exploring a bit further, the only mention of these 'legacy' labels elsewhere is in the Monitor (+monitor.s) where they're used as parameters to a macro.  So you see a somewhat different subroutine call table that look like this at line 905:

   bsr   TT_SPatch   ChipMM      ;55- Reserve CHIP
   bsr   TT_SPatch   FastMM      ;56- Reserve FAST

That macro is at line 953:

;   Patch systeme normal...
; ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TT_SPatch
   move.l   a0,-(sp)
   move.l   4(sp),a0
   sub.l   T_SyVect(a5),a0
   subq.l   #4,a0
   add.l   Old_SyVect(pc),a0
   move.l   a0,4(sp)
   move.l   (sp)+,a0
   rts

I haven't gone any further than that yet as I'm overloaded to blazes with other stuff right at the moment (violin screeches in background  ;D).  But it looks suspiciously like a remapping of jump tables.  At first I thought Pre-Pro and post-Pro perhaps?  But more likley would be Monitor and Interpreter maybe.  (And, as a relative newcomer returning to 68000 assembler after 20 years, I like the cute way of getting at the macro parameter...  8).)

Anyhow.  Surely +w.s won't assemble unless those symbols are resolved?  And a few others in that jump table - mainly to do with memory management.  So what I'm doing with a few percent of my head's multi-processor time is reconstructing the +w.s source from the amiga.library load file (V2.0 - straight out the box) to see what they resolve to when I get that  far.

I've also already plonked all the sources into that database I mentioned earlier in this thread:

Quote from: bruceuncle
Other bits of interest:
•I've databased the sources in Access 2010.  That process at least proved to me that all the sources are syntactically correct.  (The +w.s problem results from missing labels, not syntax errors.)

That will let me build that 'tree' you mentioned.  Time is the biggest problem at the moment though.  I've got to get some other stuff completed and published before I chase all these other bits up.

If you want to have a go at the task yourself, drop me a PM and I can get a copy of that database to you.  What we need are:

If you do manage to get some load files out of them and want to compare them with the original load files, I've also got a powerful differencing tool that does the job if you don't have anything yourself.  Just send 'em back to me.  Unfortunately, it's Windows platform and it's part of a commercial package I've used for years and, although it's been abandonware for a while now, it's still copyright.  So unless I can get permissions from the copyright owners, I can't exactly give it away.

As you've probably gathered by now, I'm insisting that this all gets databased.  It provides an organised storage for what we're doing.  It's the initial code repository until we come up with something better.  And I've many, many years of working magic with MS Access (wash my mouth out, etc, etc) from Version 1.0 through to 2010.  So I can pull out just about any query results we may need for this project that way rather having to deal with text source files.  As a side benefit, if anyone wants to volunteer a bit of French-to-English translation talent, all the comments have already been isolated in that database...

The Resource user-defined symbol files I mentioned above, and a bit more info are here http://www.ultimateamiga.co.uk/index.php/topic,9604.msg44953.html#msg44953 (http://www.ultimateamiga.co.uk/index.php/topic,9604.msg44953.html#msg44953)

I'm off to try assembling +w.s to see if I'm fooling myself or not.  But I thought I'd better put you in the picture first  ;).
Title: Re: Source code versioning system and repo
Post by: bruceuncle on January 08, 2013, 06:52:02 AM
I should have mentioned in that previous post that there's a personal agenda in this Resource reconstruction of amiga.library.

I've always found the best way to get to 'know' a piece of software is to pour over the sources.  Using Resource forces me to do this in an intense way that means I'm getting a good grounding in how it all works and hangs together.  With amiga.library being the interface to both the Amiga libraries and direct hardware access (tut, tut, but faaast) it's the logical one to start with.  It's also forcing me to translate all the French comments as I go.  And, although my French is pretty pathetic schoolboy standard, the context in the code usually makes it a lot easier.
Title: Re: Source code versioning system and repo
Post by: bruceuncle on January 08, 2013, 09:09:53 AM
Quote from: bruceuncle
I'm off to try assembling +w.s to see if I'm fooling myself or not.  But I thought I'd better put you in the picture first

I just tried that after kicking the includes into shape.  The layers_lib.i and graphics_lib.i are a bit of a mystery as I can't find them anywhere.  However, leaving them out completely allows some checking by GenAm.

More of a mystery to a novice like me is all the errors then thrown up for bit manipulation instructions on address destinations.  The 68000 only allows bits 0 - 7 unless the destination is a register.  But this source has quite a few that break that.  E.g. at line 275:

ColRW:   bsr   BlitWait
   btst   #13,DmaConR(a6)
   rts

That's one that's hard-coded in the source which makes the example obvious.  Most of the others are bit-number symbols in +equ.s (BitBobs, etc).  I also had to turn on case insensitive as some symbols don't follow case sensitivity rules, which is a bit of a worry.

As I said, I'm to be considered a novice in 68000 assembler and using DevPac.  So, am I missing something?  Help!!
Title: Re: Source code versioning system and repo
Post by: bruceuncle on January 08, 2013, 11:19:07 PM
Quote from: bruceuncle on January 08, 2013, 09:09:53 AM
Quote from: bruceuncle
I'm off to try assembling +w.s to see if I'm fooling myself or not.  But I thought I'd better put you in the picture first

I just tried that after kicking the includes into shape.  The layers_lib.i and graphics_lib.i are a bit of a mystery as I can't find them anywhere.  However, leaving them out completely allows some checking by GenAm.

More of a mystery to a novice like me is all the errors then thrown up for bit manipulation instructions on address destinations.  The 68000 only allows bits 0 - 7 unless the destination is a register.  But this source has quite a few that break that.  E.g. at line 275:

ColRW:   bsr   BlitWait
   btst   #13,DmaConR(a6)
   rts

That's one that's hard-coded in the source which makes the example obvious.  Most of the others are bit-number symbols in +equ.s (BitBobs, etc).  I also had to turn on case insensitive as some symbols don't follow case sensitivity rules, which is a bit of a worry.

As I said, I'm to be considered a novice in 68000 assembler and using DevPac.  So, am I missing something?  Help!!

"All projects fail because of assumptions!"

I should nail that to my forehead perhaps ;D .  The wild goose chase I've been on about missing symbols is because I assumed that the AMOS team were using case-sensitive symbols and labels.  Not so.  Which is a bit of a worry.

More concerning is that 'illegal' bit number referencing for memory location destinations.  The 680xx bibles quite clearly state that only values of 0 thru 7 are allowed unless the destination is a register.  So I examined the structure of the instruction in that example I quoted in the amiga.library load file:

ColRW:   bsr   BlitWait
   btst   #13,DmaConR(a6)
   rts

The result in hex is:  $082E 000D 0002

Translating that back to assembler opcodes and adressing modes is as follows:

$082E = %0000 1000 0010 1110

Rearranged as an opcode:

Bit Number:   15 14 13 12 11 10  9  8  7  6  5  4  3  2  1  0
Value:         0  0  0  0  1  0  0  0  0  0  1  0  1  1  1  0
Meaning:      <----------btst-------------> <-Effective Addr>
                                            <-Mode-> <-Reg--> 

Mode:     %101 = "Address register indirect with displacement".
Register: %110 = A6.


The two words following the opcode are  $000D 0002
These give the bit-number on which to operate ($000D) and the displacement offset for the addressing mode ($0002)

So the bit number is 13 (illegal!) and the displacement is 2.

Despite the bit-number being out of the 'legal' range of 0 thru 7, the 68000 seems to accept this as OK!  :o

So how did they manage to assemble that source?

I looked at the GenAm file supplied with the sources and it's V3.2.  Mine's V3.18.  Time to make a new instance of DevPac to test out my suspicions.  Sure enough, after the usual messing about getting the includes recognised in a new location, it compiled fine.  Incidentally, those graphics_lib.i and layers_lib.i files were missing from my V3.18 includes.  So I substituted a genuine WB 2.0 set and there they were.

The resulting load file doesn't match the AMOS Pro amiga.library 'out-of-the-box' file.  But a quick differencing showed these all seem to be relative address offset differences.  I haven't checked yet (here I go again) so I'm assuming that's just because I haven't set an optimal offsets option somewhere in DevPac.  So branches without a specific offset size (e.g. just bra  instead of bra.s, etc) aren't being optimised to the same values as in the load file.  Checking that would take some time so I'm shelving that investigation until later.

Conclusion (and this is critical guys!)

You need the version of GenAm that comes with the sources to assemble any of this stuff as that version has been hacked to assemble those bit instructions with 'illegal' bit-numbers.

And does anyone have any idea what the files GenAm2 and GenIm2 do?

Title: Re: Source code versioning system and repo
Post by: bruceuncle on January 09, 2013, 12:30:53 AM
Quote from: bruceuncle
I looked at the GenAm file supplied with the sources and it's V3.2.  Mine's V3.18. 

Scrub that.  That's the file version reported by DOpus.  It's actually version 3.02 - the original version, unhacked and untouched.  So that must have been a bug in V3.02 of GenAm.

Next questions.  Looking back at the source from V1.23, these operations were originally done using registers for the bit operations (along with absolute addresses, etc).  So the bit numbers were 'legal' as bits 0 thru 31 can be specified for a register destination.  In converting that to fully relocatable code, did the AMOS team just use the original bit numbers and not realise that their assembler would swallow them without protest?  Although the processor doesn't protest, is it a legal operation (does the correct bit respond)?  Is this a source of some AMOS bugs?  What am I doing chasing all this up when I should be doing other stuff?   ::)  Watch this space...
Title: Re: Source code versioning system and repo
Post by: bruceuncle on January 09, 2013, 02:32:20 AM
Just can't keep away from this thread ::) .

When the processor encounters one of these 'illegal' bit-number values, it wraps it into the range 0 thru 7 of whatever's specified in the bit-number word.  Note it wraps it (modulo 8 ) .  It does not mask it.  It then uses the result on the byte pointed to by the destination effective address.

So, in the example I've been using so far, that bit-number of 13 ($D) modulo 8 results in 5.  The bit-numbers are counted from zero so specifying bit-number 13 for a btst operation would actually refer to bit number 5 and give the result $20 (%0010 0000).  As it follows a blitter operation, the bit to be tested probably was bit-number 13 (DMAB_BLTNZERO in hardware/dmabits.i) and not bit-number 5 (DMAB_SPRITE in hardware/dmabits.i) which is what it actually does.

Getting back to the AMOS sources in general, as far as I can tell so far, other bit-numbers affected range from 8 thru 15 (in +equ.s):

*************** Test control bits  
BitControl:   equ    8
BitMenu:      equ    9
BitJump:      equ    10
BitEvery:     equ    11
BitEcrans:    equ    12
BitBobs:      equ    13
BitSprites:   equ    14
BitVBL:       equ    15

Which will yield 'legal' values of 0 thru 7.  The memory address referred to is a word at address T_Actualise(a5) .  So any bit operations on that address will be performed on the high byte.  Which may not be what was wanted. 

Interesting.   ;)

Now I really will get on with some other stuff.
Title: Re: Source code versioning system and repo
Post by: Lonewolf10 on January 09, 2013, 10:11:38 PM
Quote from: bruceuncle on January 08, 2013, 11:19:07 PM
"All projects fail because of assumptions!"

I should nail that to my forehead perhaps ;D .  The wild goose chase I've been on about missing symbols is because I assumed that the AMOS team were using case-sensitive symbols and labels.  Not so.  Which is a bit of a worry.

I should have spotted that. When I assembled my amiga demo (in 2011 and 2012) I always turned case-sensitivety off, as it's too easy to mistype a label in terms of upper/lower case.

Quote from: bruceuncle on January 08, 2013, 11:19:07 PM
I looked at the GenAm file supplied with the sources and it's V3.2.  Mine's V3.18.  Time to make a new instance of DevPac to test out my suspicions.  Sure enough, after the usual messing about getting the includes recognised in a new location, it compiled fine.  Incidentally, those graphics_lib.i and layers_lib.i files were missing from my V3.18 includes.  So I substituted a genuine WB 2.0 set and there they were.

The resulting load file doesn't match the AMOS Pro amiga.library 'out-of-the-box' file.  But a quick differencing showed these all seem to be relative address offset differences.  I haven't checked yet (here I go again) so I'm assuming that's just because I haven't set an optimal offsets option somewhere in DevPac.

Don't assume anything! ;)

Does Pietro (Ghizzo) remember whether the sources are before or after the fixes he did?

Quote from: bruceuncle on January 08, 2013, 11:19:07 PM
Conclusion (and this is critical guys!)

You need the version of GenAm that comes with the sources to assemble any of this stuff as that version has been hacked to assemble those bit instructions with 'illegal' bit-numbers.

And does anyone have any idea what the files GenAm2 and GenIm2 do?

I'm not sure that GenAm has been hacked. I have seen plenty of codes before that also test for illegal bits.
GenAm2 and GenIm2 are part of the files from DevPac 2. I will try to look up what they do as my memory is hazzy right now (down to the fact I'm damn sleepy!) and I haven't used DevPac2 for a few years now.
Title: Re: Source code versioning system and repo
Post by: bruceuncle on January 10, 2013, 12:35:18 AM
Quote from: Lonewolf10
I'm not sure that GenAm has been hacked.

No it hasn't.  My apologies if that post mislead anyone.  It's just GenAm V3.02, not the 3.20 that DOpus reported (see a couple of posts back in this thread where I'd just realised what I was doing wrong).  And that version will let these bit-numbers through without an error.

Quote from: bruceuncle
So, in the example I've been using so far, that bit-number of 13 ($D) modulo 8 results in 5.  The bit-numbers are counted from zero so specifying bit-number 13 for a btst operation would actually refer to bit number 5 and give the result $20 (%0010 0000).  As it follows a blitter operation, the bit to be tested probably was bit-number 13 (DMAB_BLTNZERO in hardware/dmabits.i) and not bit-number 5 (DMAB_SPRITE in hardware/dmabits.i) which is what it actually does.

Those 'illegal' bit-numbers are a bit more of a problem.  Not so much the values used, as the processor will always do the modulo/mask to get them in the correct range.  The problem is that the destination address is sometimes a word, not a byte.  So the processor will hit the high byte of the word when the low byte may be what was intended.

In most cases that won't matter too much because the result will be consistent if the equates are used each time.  And bit 13 is bit 5 for the high byte of a word address, so no damage done in theory if it's just data storage memory.  However, in practice, the usage is scrappy and that hardware address example I gave points out the problem.

It's worth scanning through the 1.23 and 1.3 sources to see what they were doing there.  They used data registers for the hardware address bit-number operations back then.  So they hit the correct bits.  Obviously, hitting the wrong bits for hardware addresses could be a disaster at worst or an intermittent bug at best.

I've attatched a quick grep across the sources for a symbol called T_Actualise to illustrate the sort of things to look out for.  T_Actualise is a negative offset from A5 used to store a few flags.  It's defined as a word size.  You'll see that someone programming the editor realised the mistake and subtracts 8 from the flag values  :D.  But there's also a lot of simple testing of the byte at that address.  Again, that's ok as it will be the high byte of the word so no damage done.  There's some word-sized movement back and forth to data registers which looks like it's just preserving the value.  Again, no serious damage.  But you can see the potential if the data register then has low byte bits set and tested while it's sitting in a data register...

It's just something to bear in mind as we review these sources with an eye on bugs.  And it's why I'll continue the Resourcing of amiga.library.  So I can see if there is any real difference (other than branch offset sizes) between the source and the load file.  But don't expect that to be completed any time soon as there's too much else to be completed before I seriously get into these sources (hysterical laughter as the men in white coats chase me again while I ponder the word 'understatement' ;D ).  I'd always thought semi-retirement would give me more time but it just seems to keep filling up  :).

Postscript:  I'll try to stop assuming.   8)  Please regard these posts as "thinking out loud".    :)  And they're in the wrong thread anyway. 

@MadAngus, can you be persuaded to move them to the Source Code Reviews and Comments thread here:
http://www.ultimateamiga.co.uk/index.php/topic,9604.0.html (http://www.ultimateamiga.co.uk/index.php/topic,9604.0.html)
when you've got some time?  (You can borrow my men in white coats if yours are getting a bit worn out.  They come with a lifetime supply of dried frog pills and a complimentary set of steak knives...  ;) )
Title: Re: Source code versioning system and repo
Post by: Lonewolf10 on January 10, 2013, 08:31:39 PM
Quote from: Lonewolf10 on January 09, 2013, 10:11:38 PM
Quote from: bruceuncle on January 08, 2013, 11:19:07 PM
And does anyone have any idea what the files GenAm2 and GenIm2 do?

GenAm2 and GenIm2 are part of the files from DevPac 2. I will try to look up what they do as my memory is hazzy right now (down to the fact I'm damn sleepy!) and I haven't used DevPac2 for a few years now.


Ok, just looked them up. In the version of DevPac 2 I have they have the following sizes:

GenAm2 - 23,684 bytes (23.1K)
GenIm2 - 29,104 bytes (28.4K)


GenAm2 is a newer version of GenAm. I suspect GenAm came with a version of DevPac 1. GenAm2 is an executable and is an ASM editor and assembler in one. GenIm2 is also an executable and is some sort of command line tool - possibly used to link files together? I have never used GenIm2 before... I didn't use MonAm2 (or MonAm) either - debugging software part of DevPac 2 (or DevPac 1).
Title: Re: Source code versioning system and repo
Post by: bruceuncle on January 12, 2013, 10:37:18 AM
Quote from: Lonewolf10 on January 10, 2013, 08:31:39 PM
GenAm2 is a newer version of GenAm. I suspect GenAm came with a version of DevPac 1. GenAm2 is an executable and is an ASM editor and assembler in one. GenIm2 is also an executable and is some sort of command line tool - possibly used to link files together? I have never used GenIm2 before... I didn't use MonAm2 (or MonAm) either - debugging software part of DevPac 2 (or DevPac 1).

Thanks for that.   8) It makes sense that the AMOS team used something like GenAm2 as it's got a much smaller footprint than GenAm (around 65k).  Which would have been a help assembling big stuff on small systems.  Because I'm using WINUAE and can 'make' a powerhouse A4000 to order, I tend to forget what it was like programming on a 'real' A500 and A2000 ;).  But I've also set up an A2000 system, initially for nostalgia (that's what 'my' Amiga was) but now also useful for checking performance for any new stuff I'm writing.

I'd seen GenIm mentioned in the Equates section of AMOS Pro.  I'ts mentioned in a comment in AMOS_Pro\Productivity1\Equates\Make_Equates.AMOS:
"' Name of the .LST file, produced by GenIm "
and in ...\Equates.Doc:
"* You should use Genam2 (or 3), in fact the assembler, Genim2."
if that's a clue.

I know very little about Devpac as I'd only bought it just before my Amiga 2000 was retired years ago.  The only stuff I did with it was one simple procedure to embed in an AMOS 1.3 program (which is what prompted the purchase).  I was scared stiff (at the time) of working in assembler on the Amiga.   ::)  I'm using a dowloaded disc set and dowloaded docs at the moment.  Which isn't all that informative!  But I'm getting the hang of it now.  ;)

Have you noticed the programs drip   and flush   in some of the scripts?  I lurve the names but would also like to know what they did if anyone's come across them?  They sound a bit too much like a couple of in-house programs from the names so we may never know...

The more I peek (terrible pun!) into the sources, the more confident I am that we can pull all this together without too much trouble.  But it's gonna take a loooong time.  Helluva a lot to do.
Title: Re: Source code versioning system and repo
Post by: Lonewolf10 on January 12, 2013, 08:30:29 PM
Quote from: bruceuncle on January 12, 2013, 10:37:18 AM
I'd seen GenIm mentioned in the Equates section of AMOS Pro.  I'ts mentioned in a comment in AMOS_Pro\Productivity1\Equates\Make_Equates.AMOS:
"' Name of the .LST file, produced by GenIm "
and in ...\Equates.Doc:
"* You should use Genam2 (or 3), in fact the assembler, Genim2."
if that's a clue.

Ok, thanks. I'll see if I can do some more digging...

Quote from: bruceuncle on January 12, 2013, 10:37:18 AM
I know very little about Devpac as I'd only bought it just before my Amiga 2000 was retired years ago.  The only stuff I did with it was one simple procedure to embed in an AMOS 1.3 program (which is what prompted the purchase).  I was scared stiff (at the time) of working in assembler on the Amiga.   ::)  I'm using a dowloaded disc set and dowloaded docs at the moment.  Which isn't all that informative!  But I'm getting the hang of it now.  ;)

It's not that hard coding asm. I find it harder trying to use the OS via ASM, which is why my assembly written software will always close down the OS. Not to mention it's great fun having full control of the machine ;)

Quote from: bruceuncle on January 12, 2013, 10:37:18 AM
Have you noticed the programs drip   and flush   in some of the scripts?  I lurve the names but would also like to know what they did if anyone's come across them?  They sound a bit too much like a couple of in-house programs from the names so we may never know...

Yes, there is one script which goes something like:

Code: [Select]
flush
drip
amos pro
flush
drip

Flush probably just ensures the system is cleared of anything that may have been running previously. Thus ensuring that no false bugs are found when testing AMOS Pro - some programs can crash when run because a previous program contained dodgy code.
Drip on the other hand is a mystery :(
Title: Re: Source code versioning system and repo
Post by: bruceuncle on January 13, 2013, 04:46:53 AM
Quote from: Lonewolf10 on January 12, 2013, 08:30:29 PM
Quote from: bruceuncle on January 12, 2013, 10:37:18 AM
I know very little about Devpac as I'd only bought it just before my Amiga 2000 was retired years ago.  The only stuff I did with it was one simple procedure to embed in an AMOS 1.3 program (which is what prompted the purchase).  I was scared stiff (at the time) of working in assembler on the Amiga.   ::)  I'm using a dowloaded disc set and dowloaded docs at the moment.  Which isn't all that informative!  But I'm getting the hang of it now.  ;)

It's not that hard coding asm. I find it harder trying to use the OS via ASM, which is why my assembly written software will always close down the OS. Not to mention it's great fun having full control of the machine ;)

Drip on the other hand is a mystery :(

Yes, it was the OS that put me off back then.  A kind work colleague gifted me his full RKM set for an A1000 that he'd owned when he heard I'd just bought an A2000.  That's what did the damage!  ;D  It's the only bit of Amiga antiquity I've still got (if you can call around 10Kg of manuals a 'bit'!).  For some reason I hung onto them when my A2000 passed away in a flood at a friends place.  They're way out of date in the details compared to the WB3.x set.  But the basics haven't changed all that much and I've found them very useful now I understand what they're telling me.  And I prefer reading real books to reading online ones as my back problems make reading stuff on the PC a very real pain.  I tracked down a set of WB3 autodocs so I've got the up-to-date info too.  (Up-to-date seems a stange phrase for an Amiga in 2013  :))

The only ASM I'd dabbled with previously were 6800 on a home-built 'Dream 6800' that no-one's heard of these days.  That was hand assembly with lots of graph paper!  I've still got the hardware in a cupboard.  It had a massive 64 x 32 pixel b&w display  ;D

Then it was a 6809 in an 'Hitachi Peach', which is probably also something no-one's heard of these days.  I did lots in ASM with that as it had a fairly simple Basic-and-OS-in-ROM and a 'hackers guide' (= its RKM equivalent) was published and available whilst it was still a 'current' machine.

But that was all 'back then'.  I ain't afraid of nuffin' now!  ;)  And I'm loving the 68000 - a simple set of instructions but a magnificent set of addressing modes.  So easy and so logical compared to the Intel offerings of today.

Quote from: Lonewolf10
Flush probably just ensures the system is cleared of anything that may have been running previously. Thus ensuring that no false bugs are found when testing AMOS Pro - some programs can crash when run because a previous program contained dodgy code.
Drip on the other hand is a mystery

That makes a lot of sense.  One can lose a lot of RAM to leaks crashing AMOS programs!  When I started programming AMOS again, I quickly learnt that it's essential to put a call to a 'clean up' procedure at the start of a program that uses a lot of memory banks as well as at the end.  An error or break can leave a lot of unreleased memory lying around.

Thanks for all the info.   8)
Title: Re: Source code versioning system and repo
Post by: james666 on May 06, 2013, 12:15:34 PM
Quote from: bruceuncle on January 05, 2013, 01:54:09 PM
BUT, before we get that far, we've got to arrive at a set of sources that will actually assemble into AMOS Pro V2.0. 

I have successfully recreated AmosPro.lib and the AmosPro executable by assembling +lib.s and +b.s respectively in Devpac 3.  (w.s appears to be an old, pre-2.0 version of the lib and can be ignored.)  The only changes I had to make were to fix the illegal bit instructions by subtracting 8 from the operand. For example, btst #x,DmaConR(a6) would be changed to btst #x-8,DmaConR(a6).  It seems that the illegal instructions aren't flagged in some versions of Devpac (or is there an error override option somewhere?), which would explain how this rather sloppy code managed to slip through the net.  As when creating an extension, the amospro.lib code needs to be assembled as an executable without any debug symbols.

My output files are not byte-exact copies of the originals but they seem to work fine as 1:1 replacements in my WinUAE AmosPro 2.0 installation.  AmosPro.lib is 101500 bytes vs 98184 bytes original.  AmosPro is 71652 bytes vs 22624 bytes original.

It's intriguing to read some of the documentation.  Seems that they were serious about doing the AGA version in early 93.  There's even a demo AMOS program to show an "AGA detected" message.  There's also talk of a "PCOS" windows port.  I wonder why Europress pulled the plug.
Title: Re: Source code versioning system and repo
Post by: james666 on May 07, 2013, 09:16:50 PM
A correction to my earlier post...  The +w.s file is of course the source for the amos.library, a rather critical part of the system.  The good news is that after a couple of minor adjustments, this one also assembles into a working binary.  The changes are as follows:

1. The include lines at the start need to be modified to point to the "include" directory.  exec/types.i needs to be added so that the STRUCTURE type is defined.  Non-standard names were used for gfx.i and layers.i.  These are also fixed: 

Quote
IncDir   "dh0:devpac/work/amospro sources/includes/"
Include "exec/types.i"
Include "graphics/rastport.i"
Include   "graphics/clip.i"
Include "graphics/layers.i"
Include "graphics/text.i"
Include "devices/input.i"
Include "devices/inputevent.i"
Include "exec/interrupts.i"
Include "exec/io.i"
Include "hardware/intbits.i"
Include "graphics/layers.i"
Include "graphics/gfx.i"

2. The source doesn't contain the LVO offsets for certain graphics and layers library calls.  I just looked them up in "Mapping the Amiga" and appended these lines to +libsoffsets.s:

Quote
_LVOInitBitmap      EQU   -$186
_LVONewLayerInfo   EQU   -$90
_LVOCreateUpfrontLayer   EQU   -$24
_LVOSetAPen      EQU   -$156
_LVOSetBPen      EQU   -$15C
_LVOSetDrMd      EQU   -$162
_LVOSetFont      EQU   -$42
_LVOInstallClipRegion   EQU   -$AE
_LVODisposeRegion   EQU   -$216
_LVODeleteLayer      EQU   -$5A
_LVODisposeLayerInfo   EQU   -$96
_LVONewRegion      EQU   -$204
_LVOOrRectRegion   EQU   -$1FE
_LVOOpenFont      EQU   -$48
_LVOCloseFont      EQU   -$4E
_LVOOwnBlitter      EQU   -$1C8
_LVODisownBlitter   EQU   -$1CE
_LVOWaitBlit      EQU   -$E4
_LVOText      EQU   -$3C
_LVOWaitTOF      EQU   -$10E

3.  You need to comment out the following debug code at c. line 9342.  I found that my WinUAE "Amiga" froze when Amos was started otherwise:

Quote
;IFNE Debug=2                            Si debug
;bclr   #WFlag_LoadView,T_WFlags(a5)   AMIGA-A normal...
;ENDC

(Interestingly, the immediately preceding code runs a check for the AA chipset.)

4. I changed all the "illegal" bit instructions as described in my first post.  This may not be needed for some assemblers.

5. Devpac needs to be set to non-case sensitive mode.

+w.s then assembles into a binary that can be used as a direct replacement for libs:amos.library.  It's 47320 bytes - slightly larger than the original. 

To summarise, we have working source code for the guts of AmosPro 2.0:

+lib.s  assembles into a working amospro.lib
+b.s assembles into a working amospro loader
+w.s assembles into a working amos.library
Title: Re: Source code versioning system and repo
Post by: Hungry Horace on May 07, 2013, 09:53:47 PM
Whilst I don't think I will be doing any modifications to the Amos Pro myself, this is still great news!

Well done on getting the initial compile working... Bit strange that the Amos pro exe should be 3 times bigger though? I assume its been checked that the original doesn't have some kind of compression on it?
Title: Re: Source code versioning system and repo
Post by: james666 on May 07, 2013, 10:27:51 PM
Quote from: Hungry Horace on May 07, 2013, 09:53:47 PM
Bit strange that the Amos pro exe should be 3 times bigger though? I assume its been checked that the original doesn't have some kind of compression on it?

Yes, it is a bit odd.  The code repository contains an bject file called "(b.o)" about 100k in size.  Maybe this was combined with a cruncher to generate the official amospro executable.
Title: Re: Source code versioning system and repo
Post by: bruceuncle on May 08, 2013, 12:02:05 AM
Quote from: james666
The only changes I had to make were to fix the illegal bit instructions by subtracting 8 from the operand. For example, btst #x,DmaConR(a6) would be changed to btst #x-8,DmaConR(a6).  It seems that the illegal instructions aren't flagged in some versions of Devpac (or is there an error override option somewhere?), which would explain how this rather sloppy code managed to slip through the net.

The earlier versions of Genam will let this slip through without complaint.  If you use the version included in the source distribution, it will assemble without complaint.  You can just substitute that Genam executable into your Devpac3 installation (back up the original of course!) and you've virtually got the environment they used.

This btst, bclr, bset problem is mostly self-correcting.  See this post earlier in this thread for the gory details:  http://www.ultimateamiga.co.uk/index.php/topic,9600.msg45034.html#msg45034

However, I did notice one case in +w.s where it does matter.  I haven't got the details handy as I'm up to my ears in something else at the moment.  It's where the hardware registers are tested to see if a blit is completed.  The wrong bit is tested due to the Genam bug, so partial blits are the result.  I can see the effect in the AMOS Read Text instruction where the title bar for the window is missing chunks from some bit planes.  It also crops up occasionally when using Unpack to paste Resource Bank Images onto a screen - missing chunks.  Not easy to repeat as timing will vary and most of the time the missed 'wait for blit to complete has no effect.

Quote
My output files are not byte-exact copies of the originals but they seem to work fine as 1:1 replacements in my WinUAE AmosPro 2.0 installation.  AmosPro.lib is 101500 bytes vs 98184 bytes original.  AmosPro is 71652 bytes vs 22624 bytes original.

I got the same results.  It's mostly caused by the assembler not optimising bcc branches so it's defaulting to the word or long offset format.  Byte-by-byte comparisons of the original executable and the assembled one for amos.library shows all the differences to be bcc branches in the long offset format.

That explains your AMOSPro,Lib difference, but the AMOSPro difference is a bit too big for that explanation!  (as HungryHorace just pointed out  ;) )

Work in progress

DBL Editor

Still have the DBL Editor and assorted AMOS tools to release on an unsuspecting public.  I've had some serious delays in this due to ill-health and an unexpected bereavement in my wife's family.  The app itself is finished and the AmigaGuide docs that go with it are just about complete.  All the other bits in the package listed below are complete excepting the AMOS Program Detokeniser which I'm releasing as 'work in progress'.

Why all this concentration on DBL?  Well, the entire Editor, Monitor and most of the Accessories use it.  So I thought we'd better get an understanding of it  ;) .  And I needed to get my hands nice and dirty in AMOS and Assembler.

This is a package of bits and pieces:

DBL Editor    large AMOS program
DBL Editor.guide     extensive guide to the above
DBL Editor Menu Maker    AMOS program for an SDK for DBL Editor
DBL Editor Data Maker    AMOS program for an SDK for DBL Editor
DBL Editor Tutorials and files    these go with the AmigaGuide docs
DBL Editor asm source    used as an embedded proc in DBL Editor
*Comment Stripper Accessory    AMOS Accessory to strip comments
**AMOS Program Detokeniser    lists the AMOS tokens and parameters alongside the original source for an AMOS program
Resource Bank Message Lister    AMOS program
AMOS Editor Message Lister    AMOS program
Assorted commented DBL sources    includes the Default Resource Bank progs

*    This just reduces the size of an AMOS program.  If a program won't compile, it can save a lot of space (around 64k for the DBL Editor) for systems tight on memory.

**  This is very much a work in progress for an AMOS Variable, Label and Proc Xref Accessory.  In its current format, it just lists the tokens, parameters and texts along with the original source text.  But all the definitions are in there ready for the XRef Accessory to follow.  May be useful for anyone investigating tokenisation and the AMOS program file format.

I'm not giving a release date for the above as it was supposed to have happened last Xmas!  But too many things in my private life have buggered that all up.  Let's just say 'real soon now' and leave it at that.   ;)

Source Investigations

I've printed out the sources and been browsing them for a while now.  It's time I corrected one of the pre-source-availability assumptions that I'd made about the way AMOS Pro works:

The way that relocatable calls within and between libraries uses the AFLINE 680xx instructions followed by one or two words of data.  I'd assumed that this was implemented as an interrupt vector.   Wrong, wrong, wrong!  If you browse +b.s (the loader) and its included file, +verif.s, you'll see that it does a full relocation of these instructions as it loads the libraries.  So the AFLINE instruction and its following one or two words of data are completely replaced by relocated jumps.  There's also a note in the source that proudly proclaims that these are replaced by bcc instructions in the compiler.  But I haven't looked at the compiler yet so I can't confirm that.

Currently I'm working on a full understanding of the tokenisation used by AMOS Pro.  The file +verif.s gives most of the detail.

I suggest we start to collate what we're each discovering.  Anyone got any ideas as to how we're going to accomplish that?  I can start a personal repository for info but one in the forum would be much better.

The following tables are very much work-in-progress so take it as unverified until I've completed them.  A bit long-winded, but the sort of stuff we need to do to understand how the jigsaw pieces fit together.

Source to Executable Xref:

Directory   File   Source
AMOS Pro   AMOSPro   +b.s, +verif.s, +Internal_Jumps.s
AMOS Pro   Compiler_Shell   Compiler_Shell.AMOS (compiled)
AMOS Pro\APSystem   AMOSPro.Lib   +lib.s, +Toktab_Verif.Bin, +Dialog_Funcs.Bin
AMOS Pro\APSystem   AMOSPro_Compact.Lib   +compact.s
AMOS Pro\APSystem   AMOSPro_Compiler.Lib   +compext.s
AMOS Pro\APSystem   AMOSPro_Editor   +edit.s
AMOS Pro\APSystem   AMOSPro_IOPorts.Lib   +io_ports.s
AMOS Pro\APSystem   AMOSPro_Monitor   +monitor.s
AMOS Pro\APSystem   AMOSPro_Music.Lib   +music.s
AMOS Pro\APSystem   AMOSPro_Request.Lib   +request.s
AMOS Pro\APSystem   APCmp   +apcomp.s ? (not verified)
AMOS Pro\APSystem   Compiler.Lib   +clib.s
AMOS Pro\APSystem   Header_Backstart.Lib   +header_backstart.s
AMOS Pro\APSystem   Header_CLI.Lib   +header.s
Libs:   amos.library   +w.s, +WFont.Bin, ReqPic.Bin, +AMOSPro_Mouse.Abk

Source cross-reference (*.s files only, i.e. no incbin files listed yet):

This is a bit too large to comfortably fit in the post.  So it's attached as a *.csv file.
Title: Re: Source code versioning system and repo
Post by: Sidewinder on May 09, 2013, 08:59:33 PM
This is excellent news james.  I admit is had me excited to get it working myself and I'd very much like to see your changes if you can share them.  To that end I've set up a .git repository with the AMOSPro sources over at bitbucket.org.  This is where I plan to store my personal modifications to the source code, but if any of the other developers would like commit access, just let me know and I'll grant it to you (although the limit for the free account is 5 active developers).  But since the repository is public, everyone should have read access.  I've also set up a wiki and bug tracking services if those could prove useful.

https://bitbucket.org/mness1978/amospro (https://bitbucket.org/mness1978/amospro)


Quote from: james666 on May 06, 2013, 12:15:34 PM
I have successfully recreated AmosPro.lib and the AmosPro executable by assembling +lib.s and +b.s respectively in Devpac 3.  (w.s appears to be an old, pre-2.0 version of the lib and can be ignored.)  The only changes I had to make were to fix the illegal bit instructions by subtracting 8 from the operand. For example, btst #x,DmaConR(a6) would be changed to btst #x-8,DmaConR(a6).  It seems that the illegal instructions aren't flagged in some versions of Devpac (or is there an error override option somewhere?), which would explain how this rather sloppy code managed to slip through the net.  As when creating an extension, the amospro.lib code needs to be assembled as an executable without any debug symbols.

My output files are not byte-exact copies of the originals but they seem to work fine as 1:1 replacements in my WinUAE AmosPro 2.0 installation.  AmosPro.lib is 101500 bytes vs 98184 bytes original.  AmosPro is 71652 bytes vs 22624 bytes original.

It's intriguing to read some of the documentation.  Seems that they were serious about doing the AGA version in early 93.  There's even a demo AMOS program to show an "AGA detected" message.  There's also talk of a "PCOS" windows port.  I wonder why Europress pulled the plug.
Title: Re: Source code versioning system and repo
Post by: bruceuncle on May 10, 2013, 02:09:46 AM
Quote from: Sidewinder
To that end I've set up a .git repository with the AMOSPro sources over at bitbucket.org.  This is where I plan to store my personal modifications to the source code, but if any of the other developers would like commit access, just let me know and I'll grant it to you (although the limit for the free account is 5 active developers).  But since the repository is public, everyone should have read access.  I've also set up a wiki and bug tracking services if those could prove useful.

https://bitbucket.org/mness1978/amospro

This is going to be an ongoing problem as more people start making bug-fixes and enhancements.  We'll end up with umpteen versions of AMOS Pro!

My original concept was to collate changes as they are made with the objective of engineering a composite release incorporating them all.

The more I see of the sources, the more worrying this becomes as there's some tight-coupling between the loader/interpreter (+b.s and +verif.s), amos.library (+w.s) and AMOSPro.Lib (+lib.s).  This means that a change in one can radically affect the others if it's related to the token tables.

Simple fixes for incorrect logic within an instruction should be ok though.

Anyway, we need somewhere to stick this stuff so we can sort through it.  It's great to see changes being highlighted in the source as that will make that task easier.  Thanks james666.

Quote from: james666
1. The include lines at the start need to be modified to point to the "include" directory.  exec/types.i needs to be added so that the STRUCTURE type is defined.  Non-standard names were used for gfx.i and layers.i.  These are also fixed: 
Quote from: james666
2. The source doesn't contain the LVO offsets for certain graphics and layers library calls.  I just looked them up in "Mapping the Amiga" and appended these lines to +libsoffsets.s:
They're not non-standard names.  Just copy layers_lib.i and graphics_lib.i from the Devpac includes into the AMOS includes in the sources distribution.  That solves both problems.  Why they haven't got them in the includes layers and graphics directories is a mystery.

It's also worth setting up an alternative Devpac environment using the Genam executable from the sources distro.  Just slot it in as a replacement.  It will glide over the illegal bit manipulation instructions without reporting an error.  I know that sounds blasphemous, but bear in mind that we need to be able to reproduce amos.library exactly as the original to be sure we haven't got a modified version.  We know that bug exists so can address it later when we've proved we've actually got unmodified sources.

And it does cause a problem at least in one point in the code when it's used against the hardware registers - see +w.s at line 277.  Exactly which bit in DMACONR did they intend to issue a read to?  BZERO or SPREN?

For similar bit operations on label offsets from a5, it's self-correcting.  Which is probably why they didn't notice it.  But the hardware registers are different as I understand that they should always be referenced using word-sized data.  Can anyone confirm that as I'm still a novice as far as Amiga hardware's concerned?
Title: Re: Source code versioning system and repo
Post by: james666 on May 10, 2013, 04:26:10 PM
I just noticed that the +b.s loader file contains some conditional debugging includes.  If you open the debug.s file and change the DEBUG flag to 0, +b.s will assemble to a slimmed down 22624 byte executable - exactly the same size as the original.  Furthermore, if DEBUG=0 there is no need to edit out line 9342 of +w.s.

So let's recap.  To create working amospro, amos.library and amospro.lib binaries while making minimal changes to the original code archive we need to do the following:

1. As Bruceuncle suggests, use the provided version of genam2 to circumvent the illegal bit instruction errors.

2. Change the DEBUG flag to 0 in debug.s, as must have been the case when the final production version of AmosPro was assembled.

3. +b.s will assemble without changes to create a replacement Amospro executable.

4. +lib.s will assemble without changes to create a replacement AmosPro.lib

5. Copy & paste my above list of _LVO equates to +libsoffsets.s.  As Bruceuncle suggests, copy the standard Devpac layers_lib.i and graphics_lib.i to the includes directory.  +w.s will then assemble without changes to create a replacement amos.library so long as you enable Devpac's case-insensitive symbols option.
Title: Re: Source code versioning system and repo
Post by: james666 on May 10, 2013, 05:53:00 PM
Quote from: bruceuncle on May 10, 2013, 02:09:46 AM
And it does cause a problem at least in one point in the code when it's used against the hardware registers - see +w.s at line 277.  Exactly which bit in DMACONR did they intend to issue a read to?  BZERO or SPREN?

For similar bit operations on label offsets from a5, it's self-correcting.  Which is probably why they didn't notice it.  But the hardware registers are different as I understand that they should always be referenced using word-sized data.  Can anyone confirm that as I'm still a novice as far as Amiga hardware's concerned?

The BTST, BSET, BCLR and BCHG instructions always operate on a byte when the destination is a memory address, which a custom chip register is as far as the CPU is concerned.  If you want to manipulate bits 0-31 of a word or long word you have to load it into one of d0-d7 first. 

btst   #13,DmaConR(a6) does exactly what one would expect.  It tests bit 13 of a word address, which is the same thing as testing bit 5 of the high byte.  I can see why a programmer would prefer to write it that way than btst #5, DmaConR(a6), which does exactly the same thing but is less intuitive.  The problem arises when you really want to test bit 5 of the lower byte and naively type a btst instruction that tests the high byte instead.     
Title: Re: Source code versioning system and repo
Post by: james666 on May 11, 2013, 11:02:24 AM
Reading the Devpac manual, it turns out that you can enable case-insensitive mode and disable the illegal bit instruction errors by inserting "OPT    NOCASE" and "OPT    NOCHKBIT" respectively at the start of the source code. 
Title: Re: Source code versioning system and repo
Post by: bruceuncle on May 13, 2013, 02:11:12 AM
Quote from: james666
I just noticed that the +b.s loader file contains some conditional debugging includes.  If you open the debug.s file and change the DEBUG flag to 0, +b.s will assemble to a slimmed down 22624 byte executable - exactly the same size as the original.  Furthermore, if DEBUG=0 there is no need to edit out line 9342 of +w.s.

Eureka!   8)

I completely missed that debug.s reference.  Now amos.library assembles exactly byte-for-byte.

I'd even resorted to using the original script file they used.  All the files in the sources distribution starting with a  (from a3d  through aw) are scripts for assembling/compiling/whatever the sources in their original environment.  They use Genim2 in some cases (with the only option being case-independent labels) Genam in others.  Plus a few that look like custom-built tools eg. Library_Digest which is compiled from a source in the Dot_AMOS directory (though the two compiled instances don't match!).  They make interesting reading (I haven't been through them all myself yet).

Quote
So let's recap.  To create working amospro, amos.library and amospro.lib binaries while making minimal changes to the original code archive we need to do the following:
~~~~~~~
5. Copy & paste my above list of _LVO equates to +libsoffsets.s
~~~~~~~
I found I didn't need these as they're already defined in layers_lib.i and graphics_lib.i - with those includes they shouldn't be needed.

Source Version Control
Quote from: bruceuncle
Quote from: Sidewinder
To that end I've set up a .git repository with the AMOSPro sources over at bitbucket.org.  This is where I plan to store my personal modifications to the source code, but if any of the other developers would like commit access, just let me know and I'll grant it to you (although the limit for the free account is 5 active developers).  But since the repository is public, everyone should have read access.  I've also set up a wiki and bug tracking services if those could prove useful.

https://bitbucket.org/mness1978/amospro
This is going to be an ongoing problem as more people start making bug-fixes and enhancements.  We'll end up with umpteen versions of AMOS Pro!

I've attached an XML export from the database I'm using to control the sources.  I used XML to try keeping it as generic as possible.  But I know stuff-all about XML, I just relied on the export to do it right!  Each file contains both the schema and data.  This zips up reasonably small to 2.8Mb but expands to over 80Mb, (I always said XML was a waste of space  ;D No brickbats!  It's a joke.  It's a very weak joke!)  If that format's no use to anyone, let me know and I'll see what I can do for other formats (SQL Server data-definition SQL and bcp for the data?)

The database has the sources broken into label/instruction/operands/(comment) fields by source file, by source line.  I've keyed the lines so as to leave up to 1,000 lines of insertion between any two lines (should be more than enough!).  And there're fields for added-flag/date/version and removed-flag/date/version with an overall flag for inclusion in a source extract.  To explain: you'd be able to pull any source at any version just by setting this flag based on a version number and/or date.

Each source file has similar flags.  To kick off the data for these, they're all marked as added on 06/01/2001 (the date on the sources) and version 2.000.000.  Yeah, yeah, I know that's overkill for the version number but I always like to leave plenty of elbow room for projects like this.   ;)

Cross-Referencing

I'm about to use the database to create a labels XRef.  Gotta parse them out of the operands field first.  It also required me to do a recursive check on the includes and incbin files so I know what to include (literally) when generating the cross-references.

That threw up some interesting info on what's missing (see attached PDF).  The only ones I'm concerned about are the pp/powerpacker_lib.i and pp/ppbase.i files.  We've solved the other Devpac standard library includes and the ones for +comp.s are (I think?) irrelevant as that file only seems to be there for comparison with the +apcomp.s replacement for V2.0 (both assembled to apcmp).
Title: Re: Source code versioning system and repo
Post by: james666 on May 13, 2013, 10:10:06 AM
We are also missing the source for Editor_DBL.bin, which is inserted (via Insert_DBL.AMOS) into AMOSPro_Editor_Resource.Abk.  I couldn't get your DBL editor to read it from a memory bank but the plain text is visible in the binary so I think we can reconstruct it. 
Title: Re: Source code versioning system and repo
Post by: bruceuncle on May 14, 2013, 12:39:54 AM
Quote from: james666 on May 13, 2013, 10:10:06 AM
We are also missing the source for Editor_DBL.bin, which is inserted (via Insert_DBL.AMOS) into AMOSPro_Editor_Resource.Abk.  I couldn't get your DBL editor to read it from a memory bank but the plain text is visible in the binary so I think we can reconstruct it. 

The format for Editor_DBL.bin is simple:

Start:   word   2   Number of DBL progs
   long   Prog1-Start   Offset to prog 1   
   long   Prog2-Start   Offset to prog 2   
Prog1:   word   length of prog 1      
   string   prog 1 text (null-terminated)      
Prog2:   word   length of prog 2      
   string   prog 2 text (null-terminated)      
           

However, it's all redundant.  I would speculate that Insert_DBL.AMOS was an early prototype for ResourceBankMaker.AMOS which will load and save DBL in plain text files.  My DBL Editor does the same thing.

I've attached the two AMOSPro_Editor_Resource.Abk 'source' files in a zip.

These will both extract from the bank okay using my DBL Editor.  But you will have a problem using Expand... on the second program.  It's got Tab characters in it which the parser will throw out as 'illegal'.  I just saved it and edited it in a hex editor.  (There's a useful Hex Editor by Jean-Baptiste Bolcato in the sources distribution.  He seems to have done all the work on storing DBL in Resource Banks and authored the Resource Bank Maker accessory.  It's in the JBB_progs directory.)

DBL_Editor.AMOS V1.38 coming soon:

Missing Files in General

Personally, I'm not regarding any files as missing if they can be extracted back to a 'source' without too much trouble:

Some of the accessories need fixing and/or rewriting.  So not too bothered (famous last words ::) ).  As long as we have the sources for the core executables, libraries and config files, I think we'd have the complete kit.