Wednesday, December 23, 2015

RC 2016/01 Update: A change and prepping for the challenge

It's a little more than a week before the start of my challenge, but over two weeks since I announced it.  I haven't been idle in that time, however.  I've been thinking a lot about the project, which has led me to some changes and issues.

Changing the Challenge

I've decided to make a change to my challenge.  Instead of developing my dBase for the IIgs, I will be developing it for the IIe/IIc - or, more specifically, for ProDos 8 instead of GS/OS.  There are several reasons for this, but the biggest is that I really don't want to exclude using my IIc.  I love my IIgs and it is the machine that I have use regularly, but the IIc was the first Apple II machine I purchased, and I still like it a lot, even though it's packed away.  I just really don't want to write a system that would preclude my using it on that machine.

Another reason I want to do it for ProDos 8 is because the emulators are just better.  I've used all the emulators for the IIgs that I'm aware of (kegs, gsport, sweet16), and honestly they pale beside the IIe emulators like AppleWin and Virtual II.  More specifically, if I did the project for the IIgs I had planned to use ORCA/C, which doesn't seem to work as well on the emulators.  It compiles C code just fine, but it has odd graphical issues in the editor when I move the mouse or the text cursor.  It makes it very difficult to use.  These issues don't occur on my real IIgs, but it's consistent across all the emulators, so I'm not really sure where the issue is, but in any case I really don't want to debug into the emulators or ORCA/C to try to figure it out.

As a result of my switch, I will be calling my project kBase instead of kBaseGS.

Development System

I have switched to using CC65 for my development.  For those who are not familiar with this tool, it's a full C compiler, assembler, and linker targeting many of the 6502-based machines.  As a modern tool, it runs on all the modern OS's - Windows, Linux, and Mac OS X - but generates binaries that can be executed on the old 8-bit machines or their emulators.  It includes most of the common C libraries, each of which has been recoded for each of the target platforms.  So you can write code that is reasonably portable between the Apple II, C64, and Atari platforms, for example.  That's not my intent, as demonstrated by the fact that I'll be using 80-column text mode on the Apple II, which doesn't exist on the C64 or Atari systems.  

Toward that end, I've already started doing some development - not on kBase itself, but on a CARDIAC simulator that will give me a good chance to learn some of what I need to know.  If you're not familiar with CARDIAC, you should really check it out.  It's a simple machine with 10 instructions, 100 memory addresses, and an accumulator as it's only register.  It's a base-10 system rather than binary, and each memory address can hold from 0 to 999.  It was produced by Bell Systems in the late 1960's to help high school students learn how computers work. 
Because it is so simple, it's something that can be knocked out very quickly, and you can see the program running to the right.  You can't really make it out too clearly, I'm afraid.  The 5 columns of numbers on the left are all 100 memory addresses, and on the right side is the program counter and the accumulator.  At the very bottom is the input (numbers only) and the output.

For me, the biggest benefit of this program (other than just the fun of writing it) is playing around with the 80-column mode and seeing what I can do with the text output routines that CC65 contains.  It has a version of the CONIO library included, so it does do quite a bit.  Next I want to work with the file input/output routines, volumes and directories, etc., which again will be necessary for kBase.

Incidentally, the pictures above do show A2 CARDIAC running on my IIgs.  I did my development on my iMac using Atom as my editor, and then used AppleCommander to transfer the binary to a floppy disk image that I tested using the Virtual II emulator.  I then used ADTPro to transfer the disk image to my IIgs so that I could load it off of a floppy.  The build process is managed using Make, which builds my binary, links in whatever libraries I need, and copies the binary to a disk image.  I handle loading it up in Virtual II manually.  If you're at all curious, feel free to check out the project on GitHub, and I'd love any feedback or suggestions.

dBase III issues

As I've been exploring dBase III Plus more, I have found some issues.  The biggest is the DATE data type, which stores years as a 2-digit number.  For 1985, this was fine, but of course all of us remember the Y2K discombobulation.  And given that we're now living in the 21st century, I'd much rather use a 4-digit year than 2-digit.  But I also want to maintain as much compatibility with dBase III as possible.  I'm considering doing a sort of add-on field.  That is, when you add a DATE field to your database, the system would automatically add a 2-digit numeric right after to represent the century.  In kBase, this field would be hidden and just treated as part of the DATE field.  In other xBase systems, this would then just appear as an additional numeric field, but hopefully the name of the field would be enough to suggest what it's purpose was.  I'll have to think about this, and maybe I'll come up with another option.

Saturday, December 5, 2015

Retrochallenge 2016/01: kBaseGS - A dBase III Plus clone for the Apple IIgs

The Challenge

I'm going to write an xBase system, named kBaseGS, that will be compatible with dBase III Plus (the standard early dBase product, and the one that most early xBase products - including FoxPro - emulate).  My eventual goal is to write a fairly complete xBase system that is capable of running full programs, but there's no way I can achieve that within the time span of this challenge, so I'm going to have more modest goals.  By January 31st, I hope to support the following features:

  • A system that allows interaction through a command prompt.  It may be text-based, or it may use a graphical command prompt; I haven't decided yet.  
  • Create a table (DBF file) that is readable by another dBase III Plus compatible system using standard dBase commands
  • Read, sort, and filter tables using standard dBase commands.  Because I won't be adding indexes (see stretch goals below), performance will not be a consideration here.  I will assume that this system will only handle a relatively small number of records in the area of <1000.
  • Support only the CHAR, NUMERIC, DATE, and LOGICAL data types.  This excludes only MEMO from what was available in dBase III Plus, which uses a second file that doesn't seem to be well documented.
  • Allow inserting, updating, and deleting records using standard dBase commands.  This will be functionally similar to dBase's facilities, though the interface for data entry may look different.
  • Build a reporting engine that will support only a subset of the dBase reporting functionality, enough to display a basic list with a header and/or footer and output this to the printer.
This certainly seems like more than enough work for one month, given that this is being done only in my free time (i.e., after the kids go to bed), but I do have two things going for me.  First, the family will be gone for a week in January, which I'm already planning to take off from work, so I'll have 7 or 8 days with no distractions (other than the tv, computer games...well, you get the idea).  Second, I'll be writing this using the ORCA/C compiler, not in assembler, so development should go faster.  C is a faster language for development than assembler to begin with, and on top of that, while I can happily do assembler, I'm just far more comfortable in C.  I've used it for so many years on a number of different platforms, so much so that I think I feel more comfortable in that than in Java, which is what I've been using professionally for over a decade.

So with that in mind, and being a naturally optimistic person (okay, not really, but certainly a naturally delusional person), I have two stretch goals:
  • Implement an indexing system.  Ideally this would use the dBase's NDX files, so I would just be able to read in an index generated by dBase III Plus, but this is poorly documented, so that may not really be feasible.  Failing that, I'd create my own indexing system with it's own file format, a KBX file.
  • Implement SET RELATION that allows linking two tables based on a common key.

As I indicated, I'll be doing this in C, so by the end of the challenge, I will post the code on GitHub, along with whatever else is needed to build and use the system - documentation, build files, etc.

What I'll be Doing Until January 1st

Obviously I won't be writing code for kBase.

But I have a good bit of prep work to do.  First, I have to learn about dBase III Plus.  As I talk about below, my background in the xBase world is FoxPro and even that was over 20 years ago.  I've ordered the dBase III Plus Handbook, which I'll use as my primary guide, and I've installed dBase III Plus on an MS-DOS 6.22 VirtualBox instance that I have setup (along with Doom - just couldn't install DOS without Doom!), so I have a system to experiment with.  And I'll do some research into the dBase file formats as well as any other dBase details that might be necessary to my project.

I also have a good bit to learn about Apple IIgs development.  I've had the system for a few months, but frankly it's only been really usable the past few weeks when I changed from using the composite output to the RGB-to-VGA output.  Under composite, GSOS looked like crap, but now it looks quite good.  I've been reading some books, Programming the Apple IIgs by Mark Andrews, and the ORCA/C manual.  But I haven't actually written anything more extensive than a "hello world" program yet, so I'll spend some time getting acquainted with the IIgs as a developer and actually sit down a put together a more substantive program.  I have a few small projects in mind.

So there it is, my project for RC 2016/01!

If you're curious as to what led me to this project, read on...

The Long-Winded Explanation for Why I'm Doing This

Originally I had "The Challenge" section at the end, but this turned into a much longer explanation than I had expected, so I decided to move this part to the bottom for those who don't really want to know why I decided to do this.  

Reliving the "bad" old days of FoxPro

My first job as a computer professional was at a small consulting firm in a small town.  When I was there in the early 90's, it had less than a dozen people, including the owner/managers, and we serviced small businesses with whatever IT needs they had - hardware and software.  It wasn't the best place to work, but I did learn a lot that I wouldn't have at most other places, especially in the area of PC hardware and networking.  For programming, I learned probably more about what not to do instead of what to do.  I had been a self-taught hobbyist up to that point using BASIC, C/C++, and Pascal, but the company used an xBase clone called FoxPro for most of their custom software.  I worked there for roughly 18 months, and I never went back to FoxPro.  That was more than 20 years ago, and I honestly don't remember that much about FoxPro - other than I thought it was mostly a bad language.  So despite several people that I respect who say good things about xBase languages, I have happily stayed away from the xBase world for the 20+ years since then.

Enter the Kaypro

But then I got a Kaypro.  Or, more precisely, as of this writing, I've ordered a Kaypro - but it hasn't actually gotten here yet.  But I started reading up on it a bit, and I noticed that among the CP/M software that I was getting with it was a copy of dBase II.  Now, bearing in mind that I still have no love for xBase systems, I decided that it might be a bit of a nice stroll down memory lane to do a project in dBase, which would also help me in learning the Kaypro and CP/M.  So I decided to do a dBase II application to inventory my retro collection and other electronic stuff.

There was just one wrinkle: I really wanted the ability to print out a report, but I didn't have a printer for the Kaypro.  Now, of course, I could buy one, but I really don't want to.  I'd already spent enough on the Kaypro itself and some other sundry items (e.g., more software, serial cables, a book), but more importantly I just don't have the room.  I have a "man cave" where I can do my retro stuff, but despite it being mine, I really share it with the kids - twin 4-year old girls and a 7-year old boy.  This leaves me with only enough room to setup one complete system, with a little leftover space for a "secondary" system if I'm so inclined, i.e., I can have this second system setup and hooked up to the monitor, but it will be cramped, and I can't have much in the way of accessories setup.  And right now my main system is an Apple IIgs.

The Apple IIgs

Over the past few months I haven't been blogging much because I've been pretty busy, but I have not been inactive in my retro activities.  These activities have mostly centered around the Apple II line of computers, and in particular the IIgs.  When the Apple II was actually available, it was too expensive for my family, so I had a Commodore 64.  My only experience with Apple II's back then was playing on one a little bit at a friend's house, and in my senior year at high school when they had replaced all their TRS-80's with Apple IIe's.  But honestly I can't really remember wanting an Apple II.  I remember wanting - but not being able to afford - other computers back then, but not Apple's.  They just seemed pretty limited compared to my C64, and to my teenage eyes they looked kind of ugly.  So when I got into retro computing a year or so ago, the Apple II was no where near the top of my list.

But then I saw an eBay auction for an Apple IIc, which I talked about in two earlier posts: My Latest Retro Acquisition and ADTPro and the Merlin assembler.

I had been happy with the IIc, but then I saw a cheap auction for a IIgs base system - no monitor, keyboard, floppy drives - just the base ROM 01 unit and the original Apple memory expansion card.  I bought an ADB keyboard and mouse and hooked it up to my monitor using the composite out and I had a functioning unit with 512k ram.  I added a 3.5" drive, a Classic IDE for Apple II card (an IDE interface card for use with hard drives or, as I use it, an IDE-to-Compact-Flash adapter), and a 4 MB memory expansion card, so now I'm running GSOS.  More recently, I ditched the composite cable and ran the RGB output into a GBS-8220 to convert it to VGA (using the instructions found here that were easy to follow), which improved my video display tremendously.  And I've ordered a CFFA 3000 and an Uthernet II and am anxiously awaiting for both of them to arrive.  And only about two weeks ago, I got an ImageWriter II dot matrix printer that works very nicely.  And, finally, I ordered the Opus II ByteWorks software set, which includes among other software a IIgs assembler and C compiler.

So I've really gotten hooked on the IIgs and the Apple II line in general.  It's a good system with impressive expansion possibilities, and I think it nicely straddles the fence between retro 8-bit systems and more modern systems.  It's still an understandable system that you can effectively program in assembler, but you can see in it the beginnings of a lot of the aspects of modern software and hardware that we use today.

So why am I going on about the IIgs when my project ostensibly involves dBase II on the Kaypro?  Well, as I talked about earlier, my idea for a project was to do an inventory application in dBase, but since I don't have a printer for my Kaypro and don't want to get one, I decided not to use dBase II or the Kaypro.  I would like to do this on my Apple IIgs, where I do have a nice printer, but there is no xBase platform for the IIgs that I could find.  Of course, there are other database applications, but I really want to do this on an xBase system.  Thus the decision to write my own.