omer bar-or · The Plan Askew · GSoC 2007

So, that title is probably a bit more grand than I can actually deliver, but I have spent the past week attempting to figure out how to add services to OpenOffice.org (such as integration with the Mozilla Address Book, which [you might remember] isn't working properly in the Aqua build of OOo. Though I have not yet figured out why it isn't working, I have made a good deal of progress. I now have the Mozilla Address Book drivers installed and breaking, which is (I suppose) a step up from uninstalled, though my current status has them breaking the entirety of OOo, which is (one might say) not an ideal solution.

That aside, however, the point of this post is to walk through the steps that OOo takes to compile the correct services, link to them, and eventually run them.

Before we begin: what is a service? Well, that's actually pretty complicated... by which I mean, "I don't know." But, the short answer is that a service is something that is stored in a file called "services.rdb" that comes with the OOo package (in OS X, it is stored in the application package along with the executable that runs OOo: soffice.bin). The file holds information about all of the other external files that OOo will need to do all of he complex things it can do, or at least a large subset of them. The subset of which we are concerned are the driver libraries for address books, namely the Mozilla Address Book driver libraries, which weren't being added to the file. The files in question are: libmozab2.dylib and libmozabdrv2.dylib

That said, here is the process that I have seen:

1. The files that hold almost all of the information that will eventually go into services.rdb have an extension of "scp" and are stored (mostly, if not entirely) in a set of directories in scp2/source/. The ones that concern us are: scp2/source/ooo/file_library_ooo.scp and scp2/source/ooo/shortcut_ooo.scp. (And, I'm not yet sure that the latter is important.) The former holds information about address book (and other) driver libraries, and the latter holds information about shortcut files that are basically links to the libraries. Inside the former, we find the information about the two drivers that interest us.
2. The files all have directives that look a lot like preprocessing directives. They mostly (as far as I can tell) determine what gets pushed to the next level and what doesn't. When scp2 is built, the files go through a program called pre2par and the results are placed in files with a "par" extension in one of three folders in scp2/unxmacxi.pro/par/. Here, already, the Mozilla Address Book information was not being passed because it failed one of the directives. If it had, it would now be in scp2/unxmacxi.pro/par/osl/file_library_ooo.par and scp2/unxmacxi.pro/par/osl/shortcut_ooo.par. At the same time, scp2 also makes a complete file, with information from all files in the osl directory (I believe) called "setup_osl.ins"
3. When scp2 is delivered, a file called "d.lst," located in "scp2/prj/" tells the deliver script to move the par files and setp_osl.ins to "solver/680/unxmacxi.pro/bin/osl/" (Note: a d.lst file exists in a prj folder in all of the OOo modules I've seen, and I suspect that it always tells the deliver script where to move the important files in a module so that they can be used by the modules that depend upon them and also so that they can be put into the final package.)<li>The par files are used throughout the build process to determine which libraries are needed in the final package and which are not. (So, before I edited file_library_ooo.pre, the mozab drivers were not showing up in the "OpenOffice.org 2.3.app" package.)

4. On the other hand, the "setup_osl.ins" file is used by a script called "make_installer.pl" when the solver module is built. "make_installer.pl" is the script that creates services.rdb. make_installer.pl uses a whole bunch of modules located in "solenv/bin/modules/installer/" - the script and all of the modules all make use of the same global module (globals.pm) that basically holds all of the variables that everything shares. These variables are called with <x>installer::globals::<variablename>, where <x> is the Perl variable type ($ for scalars, @ for arrays, &c.) and <variablename> is the name of the variable (e.g., $installer::globals::debug is a boolean for whether we are debugging or not).

   Note (only for those curious about make_installer.pl): make_installer.pl is basically a list of function calls to external functions in the Perl modules in "solenv/bin/modules/installer/" It sets several global variables from the environment variables and then starts collecting information from files (such as setup_osl.ins) and setting more variables based on them. By line 788, when it runs the function to create services.rdb, only two other lines has actually modified files other than log files. And, these were in the previous few lines, in which the script unzipped files marked "ARCHIVE" in the setup_osl.ins file, and copied files marked "SCP_REPLACE" to a temporary location and replaced variables in the copied files (denoted by ${<variablename>} with the variables values as held by the script. Otherwise, the main goal of make_installer.pl has so far been the creation of variables based on the contents setup_osl.ins, and manipulating of those variables based on other variables in the script (e.g., replacing strings with ${<variablename>} with a variable in a hashmap that goes by that name). Then, to create the services.rdb file, the script uses another external program called regcomp. I don't know exactly how this program works, but it seems to somehow store the contents of the variable $filesinproductlanguageresolvedarrayref into a binary file...

   In order to understand how make_installer.pl works, I kept a running list of all of the variables that it uses until it creates services.rdb (on line 788). It is located here. That file also contains the exact call to make_installer.pl that was made during the build of solver. Some of the variables were too long to manageably include in the file directly, so there are some lines that read: "SEE: <somefile>" - the files that you will need are: d2.ins, files.setup_osl.ins, scpactions.setup_osl.ins, links.setup_osl.ins, profiles.setup_osl.ins, profileitems.setup_osl.ins, m2.ins, d2.ins, and f6.ins. (My naming scheme for files followed a progression: 1) <type>.setup_osl.ins, 2 trough n) <first_letter_of_type><number>.ins. So, a file having to do with files would follow: files.setup_osl.ins, f2.ins, f3.ins, f4.ins, &c.) The files went through so many stages because certain variables, especially those having to do with setup_osl.ins, are manipulated quite a bit. If you are interested, I built Perlscripts to go from each file to the next: fix_directories.pl, fix_files.pl, fix_files2.pl, fix_files3.pl, fix_files4.pl, and fix_modules.pl. The files that the particular script expects and the one that it outputs are included at the beginning of the script. Some also expect the contents of variables stored in the main variables file and are also noted in the beginning. I made these scripts so that I could experiment with what make_installer.pl did without actually doing any installation. Finally, if you are interested, here is a tarball containing all of the files and scripts (including intermediary files).

5. Once services.rdb is created, everything else is built, and the program is ready to run.
6. Upon running, the contents of services.rdb are placed in an XMultiServiceFactory (if I remember correctly, since, for some reason, I can't find right now). And, they wait there for the various services in OpenOffice.org to need them.
7. When we start the "Address Data Source" wizard (File->Wizards->Address Data Source...), OOo populates the list of your options based on certain preprocessing variables (e.g., is the Mozilla address book enabled?). See: the function OAddressBookSourcePilot::implCreateDataSource in extensions/source/abpilot/abspilot.cxx
8. When we select an option from this list, a URL is returned as a unique identifier linking the choice to the corresponding driver. (implCreateDataSource here calls functions located in extensions/source/abpilot/datasourcehandling.cxx). If we select Thunderbird, for example, the function createNewThunderbird is called and gives us the URL "sdbc:address:thunderbird"
9. A request is made for all Base drivers (including MySQL, Evolution, &c.), and each one is queried to see if its URL matches the one requested. All of the drivers are held in an OPoolCollection (see: connectivity/source/cpool/OPoolCollection.cxx), and the function to get a particular driver is "getDriverByURL." If successful, it returns the driver we requested.
10. The first time we do this, the pool has yet to be filled, so a new OPoolCollection is created and is bootstrapped (i.e., filled). It is given an XMultiServiceFactory in its constructor and creates an XDriverManager and and XDriverAccess based on the string "com.sun.star.sdbc.DriverManager" - the XDriverAccess is, in the future, used for every URL query. The factory holds all of the services (and we have seen it before), and when asked to create a manager for "com.sun.star.sdbc.DriverManager," it returns a driver manager that can access all drivers with sdbc URLs (e.g., sdbc:address:thunderbird).
11. Once we have the driver, it handles all interaction with the external source (e.g., Thunderbird). As to how it does this, well, that is where I have gotten stuck now. More updates to come!

So, after two days of one issue or another (see below), I finally got a successful build of OpenOffice.org for Aqua on Wednesday morning. Upon seeing the build, after doing a mental jump for joy, I promptly decided to rebuild yet again, and not because I wanted another five hours of not really being able to work. It turns out that two of my configuration options (I stole them almost word-for-word from the configuration described on the OOo's Aqua Build wiki page) were actually disabling the Mozilla address book, which is what my project is supposed to emulate. The configuration options are (or seem to me to be, as far as I understand how the program works) : --disable-mozab and --disable-mozilla (disable Mozilla Address Book and Disable Mozilla, respectively). I gave the parenthetical caveat because I'm not sure that --disable-mozilla is actually relevant based on OOo's description of configuration options, yet as far as I can tell the Mozilla Address Book is tied to Mozilla itself being enabled. I will investigate this more as the summer continues to see if I can better understand what the different configure options do. Note: after removing --disable-mozilla, the build failed. The build then required two things (only one of which is explicitly mentioned):

• mozilla-source-1.7.5.tar.bz2, to be placed in moz/download (available here, as described by the file mozilla-source.txt, which is also in moz/download)
• libidl2 -- I got this using fink (syntax: fink install libidl2)

The other main peculiarity that came up while I was trying to build is that, as far as I can tell, the OpenOffice.org code will not successfully compile using Java 6, only Java 5. I call this a peculiarity because Java 6 is supposed to be backwards compatible with Java 5, but apparently the Mac OS X version, since it is fairly new, might not be.

After my final build (to date), with the --disable-mozab and --disable-mozilla options removed, I was able to see a few options in the "Address Data Source" wizard (File->Wizards->Address Data Source...). The options available are, "Mozilla / Netscape," "Thunderbird," "LDAP address data," and "Other external data source." See it for yourself! Unfortunately, from here, things weren't quite so straight-forward. Although I removed the --disable-mozab and --disable-mozilla configuration options, when attempting to access the Mozilla or Thunderbird address books, I receive the following error:

I decided, before asking on the IRC channel or either of my mentors what this error means, to learn some more about it, under the assumption that investigating this error would lead me to the right areas of the OpenOffice.org code for integrating the OS X Address Book. It was a lucky guess because it led my almost directly to four directories that look quite promising:

1. extensions/source/abpilot --> handles the address book wizard
2. dbaccess/source/core/dataaccess --> handles data access using the driver provided by cpool
3. connectivity/source/cpool --> stores a pool of drivers and their proxies, fails if a driver does not exist
4. connectivity/source/drivers -> holds all of the drivers available for OOo (or at least for OOo database integration)

I've spent the remaining time since building trying to figure out what the various files/methods/classes in each of these directories is doing, mostly by following what happens from when I click on "next" using "Mozilla / Netscape" until I get the error shown above. I will report my progress as I have it!

Finally, during any off-time I had with OpenOffice.org, I started looking up some OS X Address Book API references. The most helpful of these references has been "A Look Inside Address Book" by Michael Bearn. Based on that article, as far as I can tell, the API works through Cocoa (rather than Carbon), which led me to "Getting Started with Cocoa" and, more specifically, the "What is Cocoa?" tutorial to which it links. Since "Getting Started with Cocoa" suggests reading up on Cocoa "before you write a line of code," and I am still rather early in the summer, I figured that I would do my best to do a lot of reading before starting to code, which is opposite my usual strategy. I will, as always, let you know my progress.