RX50 Catalogue approach

This describes the approach to making a catalogue for my RX50 disk collection. Readership is public.

As well as retrieving and archiving data from RX50s, I also wanted to make an index to the collection so I needed to think about what form that would take and the practicalities of making it efficiently. Here's my approach to that problem.

Requirements

Have easy access to something that will tell me what's in my RX50 collection and which physical disks I need to find to make use of them. That something is called a Catalogue in the remainder of this list.
It would be nice to have a directory listing of disk contents available in the catalogue entry for the disk.
The catalogue should be electronic.
The catalogue should be accessible from more than one system.
The catalogue must be updatable over time as I investigate disks and possibly update disk contents or acquire new disks.
The process of investigating and cataloguing an RX50 should be routine and achievable from more than one RX50-reading system. (Because of PRO 350 FILES-11 format disks etc.)

Options

The requirements suggest that I want some sort of database with a well defined record structure. In principle, a card index or a paper notebook meets most of the requirements, but it would be laborious to maintain and not easy to transform or merge with other data. Hence requirement 3.

WSSIndex

At this point, it is worth mentioning that there is a Rainbow application called WSSINDEX^[1]. Its purpose is to maintain a database of disks and their contents. Should I use this?

WSSIndex very conveniently allows a fairly rapid cataloguing of disks. Feed them into the drive, press one key and the database is extended with information about the volume label and the file contents. But the process of adding one's own comments to the catalogue is via a cumbersome menu driven interface and I doubt it would work well with FILES-11 disks.

The database itself is in a binary file whose structure I'd have to reverse engineer to export to something more general purpose. I'd probably do that if I had an existing WSSIndex catalogue, but it seems unreasonable to impose that on myself now.

It would have been nicely retro to use this tool, but it was not to be.

Psion Series3 database

I actually already have a sort of electronic card index of media in the form of a database on a Psion Series3.

A Series3 RX50 database entry from 1994.

Look! It even has some RX50 entries in it dated 1994-06-04. It would definitely be in the spirit of Retrochallenge to keep this going.

Although the Series3 is handily portable, I have reservations about using that alone:

It isn't convenient to transcribe the RX50 contents listings to the Psion (Requirement 2);
Requirement 4 (Multiple system access) isn't met.

I care about requirement 4 because I know the vulnerability of keeping data that's not well duplicated or is in an arcane format.

But I do have a number of Psion databases which I'd like to liberate in a similar way so I'm motivated to go for a solution that allows continued practical use of my Psion machines while making the data available more widely.

Implementation

A good way to make data usable in the long term is to ensure it exists in or is routinely serialised to the form of plain text.

A good way to meet the immediate goals of the project (ie actually reading and cataloguing disks rather than just thinking about doing so) would be to note the data in a text file as I go along and see what the practical schema turns out to be later.

So my solution is to maintain text on the Rainbow as I read the disks and to find ways of transforming that data to Psion databases and other forms later. In the meantime, the plain text is easily searchable with a plain text editor.

The file is in YAML^[2] format, using a schema that is very similar to the one I used in the Psion database.

The editor I am using to maintain this file is SEDT^[3], which I installed on the Rainbow years ago because my fingers know how to use VMS EDT and its descendents. SEDT is powerful enough that I can use keyboard macros to make new disk records and populate them by reading the RX50 directory without having to leave the editing session for each new disk. Remembering how to use the more advanced features of this editor was another little twinge of pleasure in this Retrochallenge.

Extensions to the original schema I used on the Psion allow me to assign a unique ID to each disk and to specify the ID of its storage box in the record.

Yes, I already have storage boxes recorded in another database. If you are getting the impression from this that I tend to take configuration management practises to ludicrous extremes in my life, well, I suppose that's an understandable perspective.

In some future Retrochallenge, if I haven't done it already, I will meet the goal of import/export/merge between YAML data and Psion databases. I imagine there'll be some retro-entertainment involved in doing that and it has been too long since I last fired up the Psion SIBO SDK...

References

1 WSSIndex: Entry in the Rainbow-100 archive.
2 YAML.ORG: YAML is a human-readable data serialisation format. The YAML web page is actually presented in YAML format to give an illustration of what it looks like. I used to construct files very similar to this for my own purposes long before I found out that YAML existed. I imagine others will have done the same. It seems like a very natural format to use for editing structured data.
3 SEDT: Entry in the Rainbow-100 archive. SEDT comes with very comprehensive documentation and has some powerful features. It is now freely available as open source and there are implementations for a number of systems. Portability really meant something in those days, not just different flavours of Unix and Microsoft Windows {sigh, mutter}.

Updates

This document is maintained by Pute. Comments should be addressed to PNJ at XQWV dot ORG dot UK.

PNJ, 2014-01-31. From notes made as I decided what to do.