omer bar-or · The Plan Askew · How The OpenOffice.org Services Database is Created

So, that title is probably a bit more grand than I can actually deliver, but I have spent the past week attempting to figure out how to add services to OpenOffice.org (such as integration with the Mozilla Address Book, which [you might remember] isn't working properly in the Aqua build of OOo. Though I have not yet figured out why it isn't working, I have made a good deal of progress. I now have the Mozilla Address Book drivers installed and breaking, which is (I suppose) a step up from uninstalled, though my current status has them breaking the entirety of OOo, which is (one might say) not an ideal solution.

That aside, however, the point of this post is to walk through the steps that OOo takes to compile the correct services, link to them, and eventually run them.

Before we begin: what is a service? Well, that's actually pretty complicated... by which I mean, "I don't know." But, the short answer is that a service is something that is stored in a file called "services.rdb" that comes with the OOo package (in OS X, it is stored in the application package along with the executable that runs OOo: soffice.bin). The file holds information about all of the other external files that OOo will need to do all of he complex things it can do, or at least a large subset of them. The subset of which we are concerned are the driver libraries for address books, namely the Mozilla Address Book driver libraries, which weren't being added to the file. The files in question are: libmozab2.dylib and libmozabdrv2.dylib

That said, here is the process that I have seen:

1. The files that hold almost all of the information that will eventually go into services.rdb have an extension of "scp" and are stored (mostly, if not entirely) in a set of directories in scp2/source/. The ones that concern us are: scp2/source/ooo/file_library_ooo.scp and scp2/source/ooo/shortcut_ooo.scp. (And, I'm not yet sure that the latter is important.) The former holds information about address book (and other) driver libraries, and the latter holds information about shortcut files that are basically links to the libraries. Inside the former, we find the information about the two drivers that interest us.
2. The files all have directives that look a lot like preprocessing directives. They mostly (as far as I can tell) determine what gets pushed to the next level and what doesn't. When scp2 is built, the files go through a program called pre2par and the results are placed in files with a "par" extension in one of three folders in scp2/unxmacxi.pro/par/. Here, already, the Mozilla Address Book information was not being passed because it failed one of the directives. If it had, it would now be in scp2/unxmacxi.pro/par/osl/file_library_ooo.par and scp2/unxmacxi.pro/par/osl/shortcut_ooo.par. At the same time, scp2 also makes a complete file, with information from all files in the osl directory (I believe) called "setup_osl.ins"
3. When scp2 is delivered, a file called "d.lst," located in "scp2/prj/" tells the deliver script to move the par files and setp_osl.ins to "solver/680/unxmacxi.pro/bin/osl/" (Note: a d.lst file exists in a prj folder in all of the OOo modules I've seen, and I suspect that it always tells the deliver script where to move the important files in a module so that they can be used by the modules that depend upon them and also so that they can be put into the final package.)<li>The par files are used throughout the build process to determine which libraries are needed in the final package and which are not. (So, before I edited file_library_ooo.pre, the mozab drivers were not showing up in the "OpenOffice.org 2.3.app" package.)

4. On the other hand, the "setup_osl.ins" file is used by a script called "make_installer.pl" when the solver module is built. "make_installer.pl" is the script that creates services.rdb. make_installer.pl uses a whole bunch of modules located in "solenv/bin/modules/installer/" - the script and all of the modules all make use of the same global module (globals.pm) that basically holds all of the variables that everything shares. These variables are called with <x>installer::globals::<variablename>, where <x> is the Perl variable type ($ for scalars, @ for arrays, &c.) and <variablename> is the name of the variable (e.g., $installer::globals::debug is a boolean for whether we are debugging or not).

   Note (only for those curious about make_installer.pl): make_installer.pl is basically a list of function calls to external functions in the Perl modules in "solenv/bin/modules/installer/" It sets several global variables from the environment variables and then starts collecting information from files (such as setup_osl.ins) and setting more variables based on them. By line 788, when it runs the function to create services.rdb, only two other lines has actually modified files other than log files. And, these were in the previous few lines, in which the script unzipped files marked "ARCHIVE" in the setup_osl.ins file, and copied files marked "SCP_REPLACE" to a temporary location and replaced variables in the copied files (denoted by ${<variablename>} with the variables values as held by the script. Otherwise, the main goal of make_installer.pl has so far been the creation of variables based on the contents setup_osl.ins, and manipulating of those variables based on other variables in the script (e.g., replacing strings with ${<variablename>} with a variable in a hashmap that goes by that name). Then, to create the services.rdb file, the script uses another external program called regcomp. I don't know exactly how this program works, but it seems to somehow store the contents of the variable $filesinproductlanguageresolvedarrayref into a binary file...

   In order to understand how make_installer.pl works, I kept a running list of all of the variables that it uses until it creates services.rdb (on line 788). It is located here. That file also contains the exact call to make_installer.pl that was made during the build of solver. Some of the variables were too long to manageably include in the file directly, so there are some lines that read: "SEE: <somefile>" - the files that you will need are: d2.ins, files.setup_osl.ins, scpactions.setup_osl.ins, links.setup_osl.ins, profiles.setup_osl.ins, profileitems.setup_osl.ins, m2.ins, d2.ins, and f6.ins. (My naming scheme for files followed a progression: 1) <type>.setup_osl.ins, 2 trough n) <first_letter_of_type><number>.ins. So, a file having to do with files would follow: files.setup_osl.ins, f2.ins, f3.ins, f4.ins, &c.) The files went through so many stages because certain variables, especially those having to do with setup_osl.ins, are manipulated quite a bit. If you are interested, I built Perlscripts to go from each file to the next: fix_directories.pl, fix_files.pl, fix_files2.pl, fix_files3.pl, fix_files4.pl, and fix_modules.pl. The files that the particular script expects and the one that it outputs are included at the beginning of the script. Some also expect the contents of variables stored in the main variables file and are also noted in the beginning. I made these scripts so that I could experiment with what make_installer.pl did without actually doing any installation. Finally, if you are interested, here is a tarball containing all of the files and scripts (including intermediary files).

5. Once services.rdb is created, everything else is built, and the program is ready to run.
6. Upon running, the contents of services.rdb are placed in an XMultiServiceFactory (if I remember correctly, since, for some reason, I can't find right now). And, they wait there for the various services in OpenOffice.org to need them.
7. When we start the "Address Data Source" wizard (File->Wizards->Address Data Source...), OOo populates the list of your options based on certain preprocessing variables (e.g., is the Mozilla address book enabled?). See: the function OAddressBookSourcePilot::implCreateDataSource in extensions/source/abpilot/abspilot.cxx
8. When we select an option from this list, a URL is returned as a unique identifier linking the choice to the corresponding driver. (implCreateDataSource here calls functions located in extensions/source/abpilot/datasourcehandling.cxx). If we select Thunderbird, for example, the function createNewThunderbird is called and gives us the URL "sdbc:address:thunderbird"
9. A request is made for all Base drivers (including MySQL, Evolution, &c.), and each one is queried to see if its URL matches the one requested. All of the drivers are held in an OPoolCollection (see: connectivity/source/cpool/OPoolCollection.cxx), and the function to get a particular driver is "getDriverByURL." If successful, it returns the driver we requested.
10. The first time we do this, the pool has yet to be filled, so a new OPoolCollection is created and is bootstrapped (i.e., filled). It is given an XMultiServiceFactory in its constructor and creates an XDriverManager and and XDriverAccess based on the string "com.sun.star.sdbc.DriverManager" - the XDriverAccess is, in the future, used for every URL query. The factory holds all of the services (and we have seen it before), and when asked to create a manager for "com.sun.star.sdbc.DriverManager," it returns a driver manager that can access all drivers with sdbc URLs (e.g., sdbc:address:thunderbird).
11. Once we have the driver, it handles all interaction with the external source (e.g., Thunderbird). As to how it does this, well, that is where I have gotten stuck now. More updates to come!