Programming tutorial: Part 15–Memory and segments

Atari Lynx programming tutorial series:

In part 12 we covered memory in the Lynx for the first time. By now you may have run into memory limitations while building your Lynx games. Admitted, 64KB of RAM is not really much, especially considering that considerable amounts of the memory are required for the Lynx hardware.

Atari Lynx memory layout

The layout of the Lynx’s memory varies over time. At startup, there is nothing really loaded and the memory layout resembles this:


The green areas that are required by the hardware meaning both Suzy and Mikey. The Mikey 65SC02 requires zero page memory and a stack at $0000 and $0100 respectively. It’s yours to use, but only through zero page addressing and variables, plus by Push and Pull instructions that manipulate the stack. You cannot use these 512 bytes for any other purpose. This is the main reason that most programs get loaded at $0200, which is exactly after the stack’s memory. We already saw that $FC00 to $FCFF contains memory mapped registers for Suzy, and $FD00 to $FDFF likewise for Mikey. $FE00 to $FFF7 is the boot ROM area that is used for booting the Lynx.

After initialization of the Lynx hardware and the C and TGI libraries from the CC65 toolset the memory looks like this:


As you can see in orange, there are three new memory areas:

  • C stack
    The C library uses a stack of its own. This stack is usually 2KB large and is needed for more complex pieces of code (e.g. recursive functions).
  • Video buffers
    The TGI library initializes the video driver and will automatically allocate two buffer for video and do double buffering to avoid screen tearing and other weird effects during updates. One buffer requires 160*102 = 16320 pixels. Since each pixel can hold 16 colors and requires only 4 bits to hold the pen index, the actual number of bytes is 8160 (or $1FE0 in hex). With two required buffers, that’s quite a lot of memory.

Excessive direct access

One thing may have struck you as odd: how come we can use the area from $FC00 to $FDFF where Suzy and Mikey’s hardware mapped registers are. Wouldn’t there be a conflict between the registers and the RAM address space? Would we have to do a lot of memory mapping tricks to make it work? Luckily, we do not have to reserve that area and no mapping of the memory is required. That would be way too complicated. No, the good thing is that the LCD panel gets its data directly from RAM memory… always. This feature is called DMA for Direct Memory Access. So, the video display will always read RAM, no matter where the video buffers are located.


That’s why it is better to overlay it on top of an area that is otherwise less (easily) useable. Hence, the Suzy and Mikey address spaces. We can simply leave it at the regular hardware space, so we can access the special registers. The RAM access is not needed, unless we want to draw directly into the video buffers. That is pretty unlikely.

The same will hold true for the collision buffer, should you want to use that. It will take another 8160 bytes and can be located anywhere. You probably want to lay it right before the first video buffer. That’s where TGI will place it if you use the tgi_setcollisionbuffer(1); call.

With the C-stack and the video buffers in place you are around 18 KB poorer in memory, 26 KB for a collision buffer as well. The bottom line is that in most cases (no collision detection) you can spend your memory from $0200 to $0B838.

Configure my memory

Let’s take a look at how this translates to the CC65 suite. The programs and games we write consist of C and assembler code. The cc65.exe compiles the C code and generates assembler code from it. The assembler code (from C and your own) gets assembled by ca65.exe. We end up with a couple of object modules that need to be tied together by the linker. The object modules do not have exact addresses for memory just yet. It uses placeholders to be flexible in the actual allocation in memory. The linker ld65.exe performs the connection of the modules and the choice of final memory locations based on the configuration of your memory areas as indicated in a configuration file.

Each specific area in memory has a few characteristics:

  • Start address
    The area is located from a specific address up in memory space. As an example, take the video buffer that has its start address at $C038.
  • Area size
    Each area is of a particular size. Sticking with the same example, the video buffers are both 8160 bytes in size.
  • Type of memory
    Some memory areas are read or write or read/write. In the Lynx we mostly deal with read/write memory, because everything memory is located in the 64KB of RAM. Other systems have memory mapped cartridges that are ROM, i.e. read-only.

The linker uses configuration files to tie the individual parts of your program or game together. A configuration file holds information on the memory area and segments. The ld65 linker has built-in configurations for each of the known targets. The lynx has 4 built-in configurations:

  1. lynx
    The default configuration that will have the MEMORY section like above. It adds a small boot loader and a required directory, plus a LNX header so it can run in Handy. The ROM image without the LNX header can be burned to an EEPROM or Flashcard and will produce a working cartridge.
  2. lynx-bll
    This configuration creates a BLL header to the output file, so it can be uploaded via a PC to ComLynx cable using any of the cartridges that allow BLL uploads (e.g. SIMIS and Championship Rally).
  3. lynx-coll
    Essentially the same configuration as the default one. It claims an additional $1FE0 of memory for the collision buffer, before the first video buffer.
  4. lynx-uploader
    This configuration adds a special uploader area right before the first video buffer. The useable RAM area is reduced by a full $100 (supposedly because of alignment?). I believe this configuration file does not function correctly.

Shown below is a fragment of the default configuration that is used by the linker ld65.exe for the lynx target in case you did not specify your own configuration.

MEMORY {
  ZP:     file = “”, define = yes, start = $0000, size = $0100;
  HEADER: file = %O,               start = $0000, size = $0040;
  BOOT:   file = %O, start = $0200, size = __STARTOFDIRECTORY__;
  DIR:    file = %O,               start = $0000, size = 8;
  RAM:    file = %O, define = yes, start = $0200,
          size = $BE38 – __STACKSIZE__;
}

You can see what a particular configuration is by running:

ld65.exe –dump-config lynx

where the bold item is the configuration name. The source file for the default lynx configuration is called lynx.cfg. You can find it in your CC65 folder under the source code for ld65, presumably C:\Program Files\CC65\src\ld65\cfg. This configuration is compiled as part of ld65.exe, so changing the file has no effect. The other three configuration files are located in the same folder.

Focus on the bold items in the MEMORY section for now. You should be able to recognize some of the numbers. Zero page (ZP) runs from $0000 to $00FF, for a total size of $0100. The user available RAM area starts at $0200 as we saw earlier and runs until $B837. The size is computed as follows:

VIDEOBUFFER1_START – STACKSIZE – RAM_START
= $C038-$0200-STACKSIZE = $BE38 – $0800 = $B638 (46648 bytes)

The default configuration file does not give you the origin of the numbers, just the correct (resulting) ones. The constants come from the symbols section of the same configuration file:

SYMBOLS {
  __STACKSIZE__: type = weak, value = $0800; # 2k stack
  __STARTOFDIRECTORY__: type = weak, value = $00CB;
  __BLOCKSIZE__: type = weak, value = 1024; # cart block size
  __EXEHDR__:    type = import;
  __BOOTLDR__:   type = import;
  __DEFDIR__:    type = import;
}

The stack size and start of directory are defined constants in the symbols section. These values can be used to define your memory areas and make them more flexible and less hardcoded. You could change the C stack size and make it bigger or smaller for your needs. All it takes is adjusting the value attribute of the __STACKSIZE__ symbol.

There are a couple of things in the memory and symbols section that do not make sense right now. We will get to them in time. For now, suffice to say that HEADER, BOOT and DIR are areas for respectively the Handy emulator’s LNX file header, the encrypted boot loader on the cartridge and the directory with file entries on the cartridge.

Define it for me

Notice how some of the memory areas use an attribute called define with a value of yes and no. Each memory area that has a define=yes will make the linker emit two values. For an area called AREA51 it will emit __AREA51_START__ and __AREA51_SIZE__ corresponding to the start address and size of the memory.

Other pieces of code may rely on these values to allow for a flexible layout of memory and the code that is tied to the memory layout. An example is the implementation of the C stack that depends on the location and size of the RAM memory area. We already saw that the __STACKSIZE__ is a constant in the symbols. But the implementation also relies on the final physical location. It uses the value of __RAM_START__ to indicate the start address. Later, the linker will emit this value because a memory area called RAM is defined. When linking together, all puzzle pieces fit together.

In a while you will see how the linker emitted values for these defines on memory areas can be very useful. For one, they allow you to create the file entries in the directory structure that each larger game cartridge will have.

Dividing in segments

The source code items you create get compiled, assembled and assigned to the memory areas. There’s another dimension to all this. The source code consists of elements such as executable code, variables and static data, that have a different behavior and memory requirements. Similar elements are combined and group together into memory segments of a certain type. In general this allows for the protection of memory and programs residing in it. There are four segment types in C source code:

  1. Code
    Regular executable code. Normally, this cannot be altered and it resides in a read-only memory segment. If code is self-modifying it cannot reside in this segment.
  2. Data
    Refers to data that can be altered. This data comes from the global and static variables that you declare and initialize with values. These values are combined in the data segment. They can be found in the object module as binary values that get copied by the loader at the memory location of the variables, initializing them to the values you gave them. After that they can be altered, because the memory is for variables (after all).
  3. Read-only data
    Data, but this is not meant to be altered. It is reference data from constant valued variables (marked as const). Some examples of read-only data are binary data for images and music, and text strings containing messages.
  4. Bss (Block Start by Symbol)
    This is data that is uninitialized. It will have a zero or null value. There is no need for the object module to contain this data, just the location in memory. A simple routine can initialize the values, because it will be zero anyway.

That’s a lot of theory, so a real example with code might illustrate this a bit. Take the following code sample:

unsigned char a;
char b = 42;
const char text[] = “Hello, World!”;

void example()
{
  int x, y = 1337;
}

The compiler will generate the following assembler code for this (showing relevant fragments):

.segment    “DATA”
_b :
  .byte  $2A

.segment    “RODATA”
_text :
  .byte  $48, $65, $6C, $6C, $6F, $2C, $20, $57, $6F, $72, $6C, $64, $21, $00

.segment    “BSS”
_a : .res    1, $00

; ———————————————————– –
; void __near__ example(void)
; ———————————————————– –

.segment    “CODE”
.proc    _example : near

.segment    “BSS”
L0035 :
  .res    2, $00
L0036 :
  .res    2, $00

.segment    “CODE”
;
; int x, y = 1337;
;
  ldx     #$05
  lda     #$39
  sta     L0036
  stx     L0036 + 1
;
; }
;
  rts

.endproc

Hopefully you can make some sense of the transition from C to 6502 assembler. Notice how the segments for the various types of code and variables is declared. It uses the .segment keyword combined with the quoted segment name. Since b was initialized it is placed in the data segment with its initialization value. Likewise, a was not initialized and can reside in the bss segment. The constant value for text is listed as the hex ASCII values in the read-only data segment. The code segment is used for the implementation of example.

What may come as a surprise is that the initializer for y inside the example function is placed in the bss segment just like x. The reason is that y needs to be initialized every call to example(), so it is not sufficient to have the value 1337 in the data segment. Instead it is placed in the method itself and y is simply placed in bss, to save size in the binary image for the object module.

Choosing your segments

The names for the segments we just saw might seem arbitrary, but nothing is further from the truth. You chose them when you compiled your C source code. You don’t remember? Well, that’s because we never really discussed this. I will take you back to one of the first tutorials where we looked at the MAKE files for our projects. Here’s an excerpt from the lynxcc65.mak file:

CODE_SEGMENT=CODE
DATA_SEGMENT=DATA
RODATA_SEGMENT=RODATA
BSS_SEGMENT=BSS

SEGMENTS=–code-name $(CODE_SEGMENT) \
  –rodata-name $(RODATA_SEGMENT) \
  –bss-name $(BSS_SEGMENT) \
  –data-name $(DATA_SEGMENT)

# Rule for making a *.o file out of a *.c file
.c.o:
  $(CC) -o $(*).s $(SEGMENTS) $(CFLAGS) $<
  $(AS) -o $@ $(AFLAGS) $(*).s

The MAKE file defined some macros for the 4 segments and gave them the names of CODE, DATA, RODATA and BSS. It might have been anything you liked, although changing this will force some adjustments in other places as well. The inference rule for .o files for object modules from C source code shows that the cc65.exe compiler takes the arguments –code-name, –rodata-name, –bss-name and –data-name to define the segments names used in the compilation to assembler code. This will make the compiler emit the .segment “DATA” and similar pieces of assembler code we saw in the earlier fragment.

Every time you call the compiler cc65 you are free to pass different segment names. This allows you to choose your segment names for all C files that are compiled by that single command. As the SEGMENTS macro is like a global variable and the inference rule will apply to all times the rule is triggered, it is a bit fairer to say that it applies to every C file that is affected by your make file and thus your entire project as it currently is organized.

If you want a more fine grained control over the segments you have a few options:

1. Create more MAKE files

Each MAKE file will hold its own SEGMENTS redefinition. The lynxcc65.mak file is included by every MAKE file, and defines the SEGMENTS macro first. If you add your own (re)definition in your MAKE file (say fonts.mak), it will overrule the previous definition with your new one. A separate MAKE file can be triggered by calling:

cd fonts && $(MAKE) $* /f fonts.mak

assuming you have placed the items build by the fonts.mak MAKE file into a relative subfolder fonts.
The fonts.mak file should hold a new definition like this:

SEGMENTS=–code-name FONTS_CODE \
  –rodata-name FONTS_RODATA \
  —bss-name FONTS_BSS \
  —data-name FONTS_DATA

2. Include #pragma statements in your C source code

The cc65.exe compiler recognizes the following #pragma statements: Adding this at the top of your C file will make sure that all code inside that C file is compiled into the specified segment names. It could be used midway through the code, but that would mean that some code gets compiled into the default SEGMENTS defined segment names, and the rest in the #pragma ones.

#pragma data-name (“FONTS_DATA”)
#pragma rodata-name (“FONTS_RODATA”)
#pragma code-name (“FONTS_CODE”)
#pragma bss-name (“FONTS_BSS”)

It is even possible to push the current (old) name for a segment onto a sort of stack with the push keyword. It seems unlikely that you will need this control any time soon.

So far we discussed how you can control the segments for C code. In case you are writing assembler code yourself, you will need to specify the segments for the various types of code and variables yourself. You will use the .segment keyword, just like in the compiler generated assembler code, to do so. As a matter of fact, you were already using it without knowing it.

# Rule fore making a *.o file out of a *.bmp file
.bmp.o:
  $(SPRPCK) -t6 -p2 $<
  $(ECHO).global _$(*B) > $*.s
  $(ECHO).segment “$(RODATA_SEGMENT)” >> $*.s
  $(ECHO) _$(*B) : .incbin “$*.spr” >> $*.s
  $(AS) -t lynx -o $@ $(AFLAGS) $*.s
  $(RM) $*.s
  $(RM) $*.pal
  $(RM) $*.spr

When a bitmap file was used to create the read-only SCB data for a sprite, it used an inference rule that generates a new assembler file containing the line

.segment “RODATA”

or whatever the read-only data segment is called by the RODATA_SEGMENT macro at that point in time. For example, when we did the robots.bmp file this gave the following robots.s assembler file:

.global _robot
.segment “RODATA”
_robot:
  .incbin “robot.spr”

It might require a different inference rule or redefinition of RODATA_SEGMENT to place your sprite data in an other segment.

Segments and areas

At this point you are probably wondering what all these segments and memory areas are all about. And maybe even how the two are related like I hinted at when I mentioned another dimension to memory. Get ready for it, here it comes.

Individual segments are assigned to a memory area. As memory areas can hold various types of memory, like read/write for RAM or read-only for ROM, certain segments should go into compatible memory areas. E.g., code segments can come from ROM, but data should always be assigned to RAM, as it requires read/write memory.

Typically (for the Lynx) related segments are assigned to the same memory area. The Lynx only has RAM memory to work with. Admitted, the cartridges are like ROM, but it is accessed as a sequential stream that needs to be copied into RAM before it can be used.

The linker can work its magic for each of the segments that are assigned to memory areas. It can allow segments to be assigned to overlapping memory areas. This way we can have code and data in the same memory space at different times. By loading the required code and data at the appropriate time it will enable us to fit more code into our already constrained memory space.

The linker configuration file has a section for the segments and their mapping to memory areas. It tells the linker what type of code or data is in each segment and where to load the segment into memory. Here is a fragment from the default lynx configuration:

SEGMENTS {
  EXEHDR: load = HEADER, type = ro;
  BOOTLDR: load = BOOT, type = ro;
  DIRECTORY:load = DIR, type = ro;
  STARTUP: load = RAM, type = ro, define = yes;
  LOWCODE: load = RAM, type = ro, optional = yes;
  INIT: load = RAM, type = ro, define = yes, optional = yes;
  CODE: load = RAM, type = ro, define = yes;
  RODATA: load = RAM, type = ro, define = yes;
  DATA: load = RAM, type = rw, define = yes;
  BSS: load = RAM, type = bss, define = yes;
  ZEROPAGE: load = ZP, type = zp;

  EXTZP: load = ZP, type = zp, optional = yes;
  APPZP: load = ZP, type = zp, optional = yes;
}

The bolded items are the segments we have encountered so far. CODE and RODATA are read-only segments as indicated by the type=ro attribute. DATA is read-write and BSS is of type bss, like you would expect. Each of these segments gets loaded into the RAM memory area. That much makes sense, as the RAM segment is currently the only user memory,

There are a few other segments (ZEROPAGE, EXTZP, APPZP) defined that are used by the compiler for zero page variables. The segments at the top EXEHDR, BOOTLDR and DIRECTORY are for creating a binary image that you can run in Handy. The STARTUP, LOWCODE and INIT segments are for the C runtime to put stuff that needs to be in potentially special areas. Consider these a given for now.

Also, notice the fact that some segments are marked as optional, where others have the define=yes attribute. The former means that the segment might not actually be there and some optimizations can be done. The latter will make the linker emit values for the section, similar to what it does for define=yes in memory areas. For a segment named FONTS_DATA the linker creates values __FONTS_DATA_SIZE__ and for FONTS_CODE two values __FONTS_CODE_LOAD__ and __FONTS_CODE_SIZE__. The values will come in useful at a later time.

Some rules of engagement

You might think about renaming some of the areas and segments. Be careful though, because some things simply need to be present and named according to presets. As an example, the C runtime library depends on the RAM memory area to be present. It assumes that the C stack is located directly after the RAM area. It uses the generated values for __RAM_START__ and __RAM_SIZE__.

Next time

This was a pretty deep and theoretical part in the tutorial. It covered a lot of ground that was more computer science related and less specific for the Lynx. Nevertheless, it was necessary to tackle this, because a lot of other Lynx and CC65 specifics are related to it either directly or indirectly. Next time we will continue our investigation of segments and look into loading code and data into memory from cartridges. Till then.

Advertisements
This entry was posted in Tutorial. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s