Changes:

### Net::LDAP 0.1.1 / 2010-03-18
* Fixing a critical problem with sockets.

### Net::LDAP 0.1.0 / 2010-03-17
* Small fixes throughout, more to come.
* Ruby 1.9 support added.
* Ruby 1.8.6 and below support removed. If we can figure out a compatible way
to reintroduce this, we will.
* New maintainers, new project repository location. Please see the README.txt.

*
*

Changes:

### Net::LDAP 0.1.0 / 2010-03-08
* Small fixes throughout, more to come.
* Ruby 1.9 support added.
* Ruby 1.8.6 and below support removed. If we can figure out a compatible way
to reintroduce this, we will.
* New maintainers, new project repository location. Please see the README.txt.

### Net::LDAP 0.0.5 / 2009-03-xx
* 13 minor enhancements:
* Added Net::LDAP::Entry#to_ldif
* Supported rootDSE searches with a new API.
* Added [preliminary (still undocumented) support for SASL authentication.
* Supported several constructs from the server side of the LDAP protocol.
* Added a “consuming” String#read_ber! method.
* Added some support for SNMP data-handling.
* Belatedly added a patch contributed by Kouhei Sutou last October.
The patch adds start_tls support.
* Added Net::LDAP#search_subschema_entry
* Added Net::LDAP::Filter#parse_ber, which constructs Net::LDAP::Filter
objects directly from BER objects that represent search filters in
LDAP SearchRequest packets.
* Added Net::LDAP::Filter#execute, which allows arbitrary processing
based on LDAP filters.
* Changed Net::LDAP::Entry so it can be marshalled and unmarshalled.
Thanks to an anonymous feature requester who only left the name
“Jammy.”
* Added support for binary values in Net::LDAP::Entry LDIF conversions
and marshalling.
* Migrated to ‘hoe’ as the new project droid.
* 14 bugs fixed:
* Silenced some annoying warnings in filter.rb. Thanks to “barjunk”
for pointing this out.
* Some fairly extensive performance optimizations in the BER parser.
* Fixed a bug in Net::LDAP::Entry::from_single_ldif_string noticed by
Matthias Tarasiewicz.
* Removed an erroneous LdapError value, noticed by Kouhei Sutou.
* Supported attributes containing blanks (cn=Babs Jensen) to
Filter#construct. Suggested by an anonymous Rubyforge user.
* Added missing syntactic support for Filter ANDs, NOTs and a few other
things.
* Extended support for server-reported error messages. This was provisionally
added to Net::LDAP#add, and eventually will be added to other methods.
* Fixed bug in Net::LDAP#bind. We were ignoring the passed-in auth parm.
Thanks to Kouhei Sutou for spotting it.
* Patched filter syntax to support octal \XX codes. Thanks to Kouhei Sutou
for the patch.
* Applied an additional patch from Kouhei.
* Allowed comma in filter strings, suggested by Kouhei.
* 04Sep07, Changed four error classes to inherit from StandardError rather
Exception, in order to be friendlier to irb. Suggested by Kouhei.
* Ensure connections are closed. Thanks to Kristian Meier.
* Minor bug fixes here and there.

*
*

The first panel, held on the 31^st of March, was about participating in the Google Summer of Code and my experiences as a mentor. With me on the panel were Diego Novillo who works at Google on GCC; Blake Winton (@bwinton) who mentored for DrProject; Nelson Ko who mentored for Tiki Wiki; and Behdad Esfahbod (@behdadesfahbod)who was in different years a student for, a mentor for, and an organizer for the GNOME Foundation. It was a lively and informative panel, and I think a lot of interesting things were said.

The second panel is what FOSSLC calls a “Leaders panel discussion”, taking place next Wednesday, the 23^rd of April. I’m going to be participating with some fascinating people (not everyone is yet listed on the page) and I’m looking forward to it. The questions haven’t been decided yet, but I think that there will be interesting things said.

One thing which I mentioned in the last review for MacGourmet is that it failed to import my data from Yum 2.7.4 due to a change in the XML format by the new owners of Yum. This has, as promised, been fixed. I imported my database without problems.

SousChef, I’m happy to report, can now import from MacGourmet databases. It’s not quite the direct import that I’d like, but I have now imported my Yum database into SousChef via MacGourmet.

This leaves Yum 3.0, which I have purchased. These, to me, are the best three recipe management programs in the Mac world. I know some people love YummySoup!, but as I said in my last review, I’ve never been able to warm to YummySoup!, which is too bad because it looks nice otherwise.

I’m going to be living with these recipe programs for a while and do some serious evaluation of all three. This will take a while to do, because we’re preparing to move house at the end of May and we’ve already started packing. Bon Appetit!

lshal: symbol lookup error: /usr/lib/libgobject-2.0.so.0: undefined symbol: g_regex_unref

This helped me find a reference on slackware-italia.com that helped me solve my problem: my ld.so.conf.d had no reference to /usr/lib, only /usr/local/lib. I added it and all is well.

MIME::Types for Ruby allows for the identification of a file’s likely MIME content type based on the file’s filename extension, or provides a list of extensions associated with a MIME content type, if known.

• http://mime-types.rubyforge.org/
• http://rubyforge.org/projects/mime-types
• http://github.com/halostatue/mime-types/tree/master

MIME::Types for Ruby originally based on and synchronized with the Perl version by Mark Overmeer, copyright 2001 – 2009. As of version 1.15, the data format for the MIME::Type list has changed and synchronization, if it happens, will be sporadic at best. The preferred source of input for a MIME::Type is the IANA registered list.

Copyright: 2002 – 2009, Austin Ziegler; based on prior work copyright Mark Overmeer.

Licence Notes

MIME::Types is available under Ruby’s disjunctive licence with the GNU GPL or the Perl Artistic licence. See the file Licence.txt in the package for full details.

Requirements and Installation

MIME::Types has been tested with Ruby 1.8.6, Ruby 1.8.7, Ruby 1.9.1, JRuby 1.1.6 (in Ruby 1.8 mode), and MacRuby 0.3.

MIME::Types can be installed with:

% ruby setup.rb

Alternatively, you can use the RubyGems version of MIME::Types available as mime-types-1.16.gem from the usual sources.

MIME::Types 1.16

• Made compatible with Ruby 1.8.6, 1.8.7, and 1.9.1.
• Switched to the ‘hoe’ gem system and added a lot of build-time tools.
• Updated the MIME types to the list based on the values in the Perl library version 1.27. Also updated based on external source information and bug reports.
• This is the last planned version of MIME::Types 1.x. Work will be starting soon on MIME::Types 2.x with richer data querying mechanisms and support for external data sources.

In a word, hope.

I do not mean this flippantly, since it was one of the major themes of President-elect Obama’s campaign. Rather, I see this inauguration as an opportunity for America to heal divisions both recent and decades deep. When I was growing up as a child of an Air Force chaplain, I learned what most American schoolchildren learn: that America is the land of equality and opportunity, and that we are a shining beacon of hope and a model for the rest of the world to follow.

By time I finished college in the middle of the Clinton Presidency, I knew that this was as often false as it was true, and I felt it becoming less true as the years wore on with the bitter partisan fighting instead of paying attention to the needs of Americans and the world.

Ten and a half years ago I met my now-wife, who is Canadian. I found it very easy to emigrate: I was disillusioned with America and saw little hope that things would get any better. I could not completely leave America behind—as the place of my birth and where my parents and brother still live, it has a deep grip on my heart. I have watched over the last ten years with deep sadness and anger as my fears of the deep divisions in America were realized. When America quickly fell to partisan bickering and bullying after 9/11, I knew that I needed to be involved in my local political process and soon after took Canadian citizenship.

Yet every two years, I make sure that my voter registration is still valid and request an absentee ballot. I avoid voting on local issues because I am not affected by them, but I try to make sure that I am aware of who is running for national office and figure out the best choice that I can make. I have voted for a hope at healing in America three times in that last three Presidential elections; this election, it seems the rest of America voted with me.

The wounds in the American psyche are deep; there are entrenched interests who would not see them healed, but I have great hope that this Presidency will be the start of the healing process. I grew up with a vision of America as a better place; under President Obama, I have hope that it will begin to be so once again.

According to the rules that they put together, I am eligible for this contest since I am an American citizen, even though I reside in Canada. I hope to win, of course, but it also felt good to write this down. I’m going to leave comments open on this, but will be moderating comments much more heavily than normal. If you don’t have anything nice to say about this in a comment, either don’t say it at all or say it on your own blog.

It’s time to declutter the house. One of the things I want to get rid of are all the recipe magazines and loose recipes that I have. To do this, I need to keep the recipes that I like or want to try. I need a recipe management program. I currently use Yum 2.7.4, which is good, but not great. I decided to seriously evaluate the various recipe management programs available for the Mac. There’s a number of them out there, each with different strengths. I’m going to be evaluating these programs on the following criteria:

• Browsing: How easy is it to find a recipe without searching?
• Reading: How easy is it to read the recipes, once entered? A “kitchen mode” is nice, but absolutely unnecessary in my kitchen—my iMac is in the office. So, I’ll either be reading the recipe to get a sense of the way to make it (and often just a reminder), or I’ll be printing the recipe.
• Printing: How good does the output look? How much fiddling do I have to do to get multiple recipes on a single page, if they’re small enough? How much fiddling do I have to do to get a longer recipe on a single page? Sure, I can use the n-up feature on my printer, but that shrinks the whole page down really small.
• Searching: How good is the recipe search?
• Adding/Editing: While I will be reading, searching and printing recipes most often, I’m going to be most annoyed at a program while adding a recipe by hand.
• Import: I’ve been using Yum (see below) for a while, and while I’m not unhappy with it, it has some major annoyances on recipe entry that keep me looking. Whatever I switch to is going to have to handle importing my Yum database one way or another, without screwing it up. Additionally, since I’ll be looking at the latest version of Yum, I want to see the external import techniques (from text or the Web).
• Extras: What extras exist for the program? Can I easily transfer a recipe to my iPhone or iPod Touch? Does it have an extensive ingredient list with nutrition data to give a rough approximation of the caloric content? These things aren’t going to be deal breakers, but if everything else works, they could be deal makers.

There are more programs available than I am reviewing here. A number of these programs presented problems early enough in the review process that I didn’t think it was worth spending any more time on them.

One that I wish had been better was Measuring Cup. It has some really interesting ideas including sub-recipes and not distinguishing view and edit modes. Unfortunately, it doesn’t have any import facility to speak of, and the controls on the lists are non-standard and finicky. It’s worth looking at if you’re just starting your recipe collection. I’m not.

Connoisseur 1.2

Only Connoisseur 1.2 is available for direct download; there is a beta version referred to, but you must contact the developer for this information.

• Browsing: A Connoisseur has fairly good browsing. It’s laid out much like iTunes with a “sources” list on the left and three filtering criteria on the main panel under the toolbar: Cuisine, Course, and Ingredient. This will make it easy to find recipes based on the ingredients you have at hand and the meal you’re serving.
  You can great new “groups” (recipe collections/folders) and smart groups (groups with search criteria) pretty easily.
• Reading: C+ After browsing to the recipe, or searching for it, you double click on the recipe in the main list to display it¹ in a smaller (but resizable) display window. The window isn’t bad, but it’s too loose: the recipe metadata (such as cook time, yield, etc.) takes up far too much space; blank fields are shown; ingredient lines are 1¼ linespacing, which is too much.
  On the up-side, there’s a nice “Cooking View” that fills the screen with the recipe. The directions are too widely spaced, but this is a much better “view” than the default windowed view. As a nice touch, Connoisseur will also read the recipe steps to you.
• Printing: F Everything that’s wrong with the display window is wrong with the print function, and it has additional sins of commission. The linespacing per field appears to be about 1½ and the ingredients are still 1¼. The empty fields don’t show here, but the amount of space taken up by the metadata eats up half of the first page. Worse, the metadata is incomplete: it’s cropped if it goes over a certain (small) width. The directions for a “2-Minute Fudge” included with Connoisseur don’t begin until page two—and the heading “Directions” is on page one. The directions are cropped if they’re over a certain (small) width.
• Searching: F ⌘F and ⌘⌥F don’t work for search. As far as I can tell, aside from the filtering mechanisms mentioned in browse, there’s no way to search for a recipe.
• Adding/Editing: F I’m not sure where to begin with the problems presented by Connoisseur’s add/edit recipe sheet. When adding or editing a recipe, you are presented with a modal sheet on the main program window. If you’re copying information from another recipe, you have to first have displayed that recipe in a separate window before you start editing the current one. The recipe sheet is tabbed and has far too many controls on it (at the bottom of the sheet is a help question mark, previous, next, a “cooked” checkbox, cancel, and save. The recipe scaling feature looks impressive, but I’m not sure why it’s part of the recipe editing—this is something that Connoisseur should be able to figure out automatically, on the fly.
  The ingredients and directions are on separate tabs, requiring flipping tabs to make sure you’re correctly using all of the ingredients as you’re writing the recipe down (as someone who creates recipes from time to time, this matters). Ingredient and direction entry is more complex than it should be, relying on +/- buttons on each and every ingredient and step.
• Import: B Will not import directly from the Yum database format, but recognized the Yum XML format reasonably well. The import itself was fairly pedestrian, although my imported recipes had stupid errors, such as: “2. 1. In a small bowl…” There’s some level of intelligence that should be applied here. There’s no import from a webpage, and the import from text is a little hit-and-miss (in my test of a copied recipe from the New York Times, the instructions were inserted as ingredients, possibly because they were numbered).
• Extras: There’s a nice set of default recipes included. You can add a recipe to a shopping list and export it to an iPod as a list, or to the Palm program “HandyShopper” and “SplashShopper” (which would be useful if I used a Palm device anymore). There’s an online component for shared recipes and you can download others’ shared recipes, too. It looks like there’s a moderation process to the recipes, though.

This is not a program that I would recommend to anyone at this point. It looks pretty, and the filtering mechanism is superb, but I don’t think that this is a usable program.

Cookware Deluxe 3.2

• Browsing: B- Cookware Deluxe starts in a recipe view, requiring explicit switching to a browser. The browse interface is a filtering interface that’s rather busy and confusing. The display has three panels. The recipes are in the main panel on the right, and there is a switch (”First/Any”, defaulting to “First”), followed by the alphabet, “All”, and a heart symbol. Clicking on a letter will filter the display of recipes based on the presence of that letter at the beginning of a word in the recipe name; if the switch is set to “first”, only the first word is considered. Clicking on “All” clears the filters and clicking on the heart selects “collected” recipes (those that carry from version to version).
  The upper panel on the left has five tabs that act as filters: Cookbook, Ingredient, Region, Course, Bev (the alcoholic beverage one would have with the recipe, generally limited to wines and some beers). These filters are not co-operative filters; they work independently (one cannot select an ingredient of “Asparagus” and a beverage of “Pinot Grigio” to filter for recipes that call for both). The lower panel are for saved search templates (loading a saved search goes to the Find window) and menu sets (collecting several recipes that are intended to go together as a meal). Clicking on “All” above the menu list removes any of the filters applied from the left panel, too.
  Cookware Deluxe also has a planner view where meals and menus can be planned in advance. There’s both a monthly calendar and a weekly calendar view (called “Planner” and “Details”, confusingly). There is no option to make Monday the beginning of the week.
• Reading: The recipe display panel is fairly decent, divided into two panels (a large upper panel shaped like a recipe card and a lower tabbed panel). The recipe card is divided into three columns. The first column is a quarter of the entire card and contains the name of the recipe, a tabbed box for photographs of the recipe, and other important metadata (cooking time, oven pre-heat temperature, difficulty).
  The second column is the ingredient column and also takes up a quarter of the recipe card. The third column has a small toolbar (about ¹/₁₀ of the vertical space), a description area (about ²/₁₀ of the vertical space), and the directions. The toolbar over the directions provide a checkbox (”Multi-Print”), four buttons (Tools, Custom Print Layout, User Prefs, and Large Print Display), another checkbox (”Collected Recipe”, see above), and a bronze badge with a check mark inside for checking spelling.
  The font in all of these sections is small, but there are two icons of interest for this: in the lower left hand of the card is an icon that looks like a two item bar graph. Clicking on the taller bar zooms the interface larger; the shorter bar zooms it smaller (and it will go smaller than the default). In the toolbar over the directions column the last button will display the current recipe in a window with a much larger font for viewing from a distance. It’s not as good as Connoisseur’s full screen display, but it works (it also only shows the ingredients and directions.
  The bottom tabbed panel allows for category information (cookbooks, course, main ingredient, region, beverage, source, recipes it requires, and recipes that work well with it); additional source information, Weight Watchers™ points, and nutritional data (unformatted and not calculated on purpose); and notes and substitutions.
• Printing: A- Very strong. Includes the ability to customize where each section of a recipe, although the sections themselves include very weak formatting. The “Multi-print” selection (in recipes and the browse list) allows you to print multiple recipes on a single page (if they fit).
• Searching: Has relatively strong search capabilities, but the layout is pretty confusing.
• Adding/Editing: B- Recipes can only be created from the “Recipe” view; not from the “Browser” view. The strengths of the recipe view are present, but each of the fields is unformatted, leaving it to the user to know the appropriate abbreviations and names of each ingredient. This makes for easy cut and paste, but explains why the “main ingredient” must be entered separately; the ingredients are not stored individually, but in aggregate. Note that I did not attempt drag and drop adding, but I would expect it to work well given the lack of formatting in the fields.
• Import: C- Only supports importing past CookWare recipes, recipe sets, and MasterCook recipes. This is found under the application menu, not a “File” menu.
• Extras: Supports an iPod cookbook or recipe (presumably just a text note). Supports QuickTime movies for each recipe. Has a Windows version.

This is another program that I can’t recommend. There are some nice features, but this program is written on top of FileMaker and it feels like it. The layout is crowded and hard to read; if there’s been thought given to making this program easier to use, it seems to have been hampered by FileMaker forms options.

MacGourmet 2.3

MacGourmet and MacGourmet Deluxe are essentially the same program. MacGourmet Deluxe includes all of the available MacGourmet plug-ins (”Cookbook”, “Mealplan”, and “Nutrition”) and is a better value than buying the plug-ins individually. Because the plug-ins are extra for MacGourmet, I will only cover them in the Extras section of the review.

• Browsing: B+ The interface for MacGourmet looks like the Apple Mail or NetNewsWire interface with sources and folders on the left, a recipe index on the top right and recipe details on the bottom right. There are two views for the recipe index: a standard tabular view and a simplified image, name, source, and rating view. By default, the simplified view only shows two recipes at a time; the standard view shows seven.
• Reading: A The recipe layout is beautiful. The ingredients are in a formatted box with the source, yield, pictures, and other metadata on the right side of the recipe². The directions are below the ingredients. There’s a very readable chef view; as a nice touch, if multiple recipes are selected, the chef view is a tabbed interface, one tab per recipe. In the standard view, only one recipe can be displayed at a time; multiple recipes can be displayed in separate chef view windows as well.
• Printing: A Standard printing facilities are superb and support printing on Avery 3×5 and 4×6 cards as well as several other templates.
• Searching: A Simple but effective rule-based “cupboard” searching (like Spotlight) with ⌘F. ⌘⌥F provides access to the quick search box, which is very effective. Explicit note, recipe, and wine note searches are also available.
• Adding/Editing: B- Some of the problems seen in Connoisseur’s add/edit screen are here to a lesser degree. By default the recipe add/edit screen isn’t a modal sheet (but it can be configured as such), but it is a modal dialog that blocks the main view. Otherwise, it takes an iTunes approach to editing: multiple tabs for Info, Ingredients, Directions, Preparation, Notes, Picture, and Nutrition data (which can be calculated from the ingredient list). The ingredients list is pretty easy to use, but there are two columns of checkboxes (”D” and “M”) which aren’t explained in the help documents. “D” marks the ingredient line as a descriptive title separating distinct portions of the ingredient list; I haven’t figured out what “M” does. The other sections are unformatted text entry.
• Import: A Importing is strengthened by the clippings section to turn clipboard results into recipes relatively quickly. Featured recipes from MacGourmet.com and Amazon.com are also automatically importable through the “Featured” section. It supports importing from Yum, but only imported 70 of the 116 recipes that I have in Yum; I’m not sure if this is a demo limitation or not.
• Extras: There’s a shopping list, a generic note list, and wine notes. Easily supports publishing to standalone sites or weblogs (the ability to use a weblog publisher like MarsEdit would be nice).
  The Cookbook Builder plug-in is a fairly nice idea and works reasonably well (drag and drop recipes into the appropriate places), although I don’t think it’s something that I would use often, if at all. The meal planner works similarly but is a little finicky as to where things should be dragged (if I drag the “breakfast” meal into the day area, even if it’s not on the day row, the “breakfast” meal should land in the right spot; don’t make me care about your hierarchies when adding—you just need to do the right thing). It’s nice that the meal planner can easily become a shopping list. The nutritional plug-in is a little confusing as to how it works at first (you need to edit the recipe to calculate the nutritional value for it based on ingredients), but it does a very nice job of calculating once it’s clear.
• Update: Michael Dupuis, the developer of MacGourmet, indicated that the Yum import failure with MacGourmet is related to an XML format change in the Yum format, not a limitation in the demo. This will be fixed in the next update of MacGourmet. I discovered that I had forgotten to put a grade on the Import feature. He also mentioned that the “D” and “M” checkboxes in recipes are explained on pages 23–24 of the user guide as “Divider” (descriptive title, as I surmised) and “Main Ingredient”. Finally, the post editor allows you to choose an extern blog posting tool such as MarsEdit by choosing it under the “Post using” menu.

This is a great program. There’s enough here that I can possibly see replacing Yum with MacGourmet. I suspect that although I don’t see myself using the cookbook builder, I would consider using the meal planner and the nutrition calculator, so I might go with MacGourmet Deluxe.

SousChef 1.0.1

• Browsing: A- The app is divided into three panels: a source list, a recipe list, and a recipe display. It looks very much like the Address Book application provided with Mac OS X. The source list contains a master recipe library, a search results entry, recent imports, and then user-added folders and collections. Folders can only contain collections; collections contain recipes. Collections are associative; recipes can belong to more than one collection at a time. Deleting a recipe from a collection only deletes it from the collection; it does not appear that there is a way to delete a recipe from the recipe library permanently except using the recipe library view.
• Reading: A The reading pane is clear, well laid out, and easy to read. This is not as flexible as the display in MacGourmet, but I am pleased to say that it’s easily readable. The full-screen “Cook” view is the best I’ve seen in any of these programs: it will not only speak your recipe (which Connoisseur also does), but you can also turn on voice recognition so that you can tell it you want the next instruction to be read. Drop down menus in the ingredients list let you search for other recipes with this ingredient with ease.
• Printing: B- The print layout is functional and well laid out, but you can’t print more than one recipe at a time.
• Searching: A+ Excellent search on ingredients, name, category, or cuisine. Simultaneously searches local and “cloud” recipes.
• Adding/Editing: A- Heavy reminders of Address Book here; ingredients and directions are added and removed with the red “-” and green “+” circles. Can’t drag and drop ingredients, and the green “+” isn’t always visible on entry. Sub-recipes (the crust for a cheesecake) are handled manually (possibly poorly). Directions are automatically numbered. Notes are automatically bulleted.
• Import: B Can only import one recipe at a time essentially from the clipboard or a single text file. Does not handle heavily formatted recipes well. No support for Yum importing.
• Extras: Blogging, although how it works is not clear without a licence. “Cloud” recipe sharing of recipes in yours and others cookbooks.

This is another program that I really like. I’m not happy about the state of import for multiple recipes—I have an extensive collection that I want to import already. Conversion utilities would be very useful here. I’d also like to see print improved some, or at least some sort of iPhone integration.

Yum 3.0

Yum was recently acquired by “Dare to be Creative” and has been turned into a shareware program as of Yum 3.0. I’m currently using Yum 2.7.4 which is no longer supported. I’m reasonably happy with Yum 2.7.4. This review is based on the trial version of Yum 3.0.

• Browsing: A- Like several other recipe programs, Yum uses a 3 column layout. In this case, it’s category, recipe list, and then the recipe itself. Categories are associative (and in Yum 2.7.4, able to be managed in the “categories drawer”; this appears to be read-only in Yum 3). A nice touch carried over from Yum 2.7 is that when you select a recipe, all of the categories to which it belongs are highlighted (similar to Address Book).
• Reading: A- Simple, well-laid out. Ingredients are always on the left; the method (directions) are always on the right. There’s a full-screen mode, but the default configuration misses the point (it doesn’t increase the font size at all)—you need to go into the Format|Manage Layouts and change the “step-by-step” layout to “Step-by-Step” to be meaningful.
• Printing: A- Layouts can be edited (it’s not completely clear how they work) and multiple recipes can be printed on a single page if they fit. Layout is completely customizable (but I have never done so).
• Searching: B+ It works well enough, but there’s no keyboard shortcut.
• Adding/Editing: B+ Fairly good adding and editing. It’s been improved from 2.7 (which allowed the method to be formatted rich text). Automatically converts fractions to their proper display form. Has a “paste ingredients” for quick addition from a copied list. Scaling is done in here and not elsewhere.
• Import: Imports from Yum, MasterCook, and XML only. Doesn’t have a single recipe import mode (although “paste ingredients” helps with that).
• Extras: Now comes with a shopping list mode.

This is a fair update to a good recipe manager. I’m not sure that it’s worth the shareware cost, when others that offer more features are just a few dollars more. However, I am excited to see that Yum has been acquired and is under active development again; I would not be surprised to see Yum become a viable competitor to YummySoup!, MacGourmet, and SousChef moving forward.

YummySoup! 1.6.9.5

I’ve tried YummySoup! a few times and never been quite convinced by it.

• Browsing: A Like MacGourmet, this is somewhat reminiscent of Mail or NetNewsWire; there’s a source list on the left and recipes on the right. There’s both the “My Library” and the “Online Library” view for viewing and sharing recipes online. The default recipe index (top right panel) is the image browser. This isn’t quite Recipe CoverFlow, but it’s pretty damned close. This view alone argues in favour of taking pictures of your masterpieces. There’s also a standard tabular view which is clear and readable. There are both groups (folders) and smart groups (live search folders); groups are associative (recipes can exist in more than one group at a time). Recipes can be removed from a group or from the library.
• Reading: B+ Functional and pleasant, but it still puts metadata at the top and too prominent. The first thing I care about when looking at a recipe is the ingredients. I don’t care about the source, difficulty, ethnicity or anything else. The full screen view is well done, but not as nice as SousChef (and, to be honest, I don’t care about the recipe picture in full screen mode).
• Printing: C+ Only one recipe can be printed at a time. Standard print format.
• Searching: B No keyboard shortcut. It works well from the search box, and the smart groups really are smart.
• Adding/Editing: A- A modal sheet for editing, but everything is on one pane (no tabs!). Easily make new ingredient groups; good auto-fill values. Bullet (●), ℃ and ℉ buttons. It gets nearly everything right (modal?).
• Import: B+ Doesn’t support importing from Yum; I could import Connoisseur or MacGourmet recipes if I wanted to. Importing from a web site could not be easier (and works very similarly to SousChef; YummySoup! had it first, though).
• Extras: An easy to use grocery list; a liquor cabinet tracker. Online publishing (via email to HungrySeacow) and downloading.

I’m still undecided about what to think about YummySoup!. I like what it has, but it has some weaknesses that I’m not fond of. I don’t think that it’s as good as MacGourmet or SousChef.

The Verdict

Tonight, the verdict is to change nothing—I’m not convinced that the alternatives are worth the price today (including the new Yum 3.0), and the stronger contenders (MacGourmet, SousChef, YummySoup!, Yum 3.0) have serious flaws with how I need to use a recipe management program. If I were forced to make a choice, I think that MacGourmet Deluxe would be the winner, but I’m not sure that the expense is worth the time and effort it would take me to switch. I really want to like SousChef, but it’s not quite there yet for me.

He will be missed by the community.

_why has some of the best commentary on this.

There’s already talk of a permanent addition to Ruby in his honour, $ABOUT.ts (”ts” was his email address), and Ruby Central is considering naming something (possibly a grant) in his honour.

My parents tell me that they have wireless access in a campground more often than they don’t. Of the campgrounds we went to this summer, exactly three had any Internet access at all, and only one was reachable from where we were parked.

Intrepid road warriors, we kept up with our email and other web browsing most days through the iPhone. Even in Cape Breton on the Cabot Trail, we had decent EDGE data signal through the Rogers network³. For eighteen of our twenty-two days away from Toronto, the iPhone was our only meaningful Internet access. It performed like a champ. We even got tickets to see Neil Young at the Air Canada Centre in December while rolling down the road.

My parents also use Microsoft Streets & Trips with an attached GPS for navigation. More than once, the iPhone with its GPS and Internet connection (and Google Maps) gave us better directions than Streets & Trips. There was one notable incident where Streets & Trips put us in the Atlantic, but Google Maps on the iPhone gave us the right directions⁴

The iPhone is too small to be practical for extended use as your sole access to the Internet. It is an excellent adjunct to a standard laptop or other computer and I don’t regret the purchase or contract at all. I have some ideas on how a good web tablet might work, based on my use of the iPhone and a Tablet PC, but I’m going to let them percolate a bit before I publish them.

Unsurprisingly, Aristotle is probably wrong, as is the FSF. Their own history with respect to running applications on non-GNU GPLed operating systems suggests this. I’m going to briefly address Aristotle’s comments in regards to the FSF’s definition of “free software”¹.

The freedom to study how the program works, and adapt it to your needs (freedom 1). Access to the source code is a precondition for this.

Study? Yes. Adapt? Technically, with the source code, yes, except, without a developer key from Apple the most useful thing I can do with the changed source code is print it out and use that as… toilet paper.

On many Unixes, compilers were an added cost. This still remains true on some platforms (and gcc isn’t an acceptable replacement on some platforms). Without access to a compiler, the most useful thing you can do with the changed source code is, as Aristotle says, toilet paper. Note as well that there’s still the “Operating System exception” in the use of the GNU GPL. Specifically, I can use any services or libraries provided by the operating system without infecting the operating system or the GNU GPLed software that I’m compiling.

You may as well complain that the development toolchain requires that you purchase a Macintosh computer to write an iPhone application, since there’s no version for Windows or Linux. (Nor is there likely to ever be.) I see the requirement for a developer key as analogous to the requirement of a compiler and easily falling under the “Operating System exception” for execution purposes.

The freedom to redistribute copies so you can help your neighbor (freedom 2).

As long as my neighbour is the App Store, I suppose.

You can still do this, either through ad hoc distribution or through Apple’s own system. You can also redistribute the changed source without limitation. The GNU GPL says nothing about requiring that distributions be binary distributions.

Being free to do these things means (among other things) that you do not have to ask or pay for permission [emphasis mine].
You should also have the freedom to make modifications and use them privately in your own work or play, without even mentioning that they exist. If you do publish your changes, you should not be required to notify anyone in particular, or in any particular way [ditto].

Uhm, except Apple, right? Right?

This is where Aristotle has gone completely off the rails with respect to iPhone software development.

As a license agreement, the GNU GPL does not impose—nay, cannot impose—requirements on third parties. This is partially the source of the “Operating System exception”; you cannot force IBM to release the source of AIX just because you build, run, and/or distribute a GNU GPLed piece of software on it. (If you want to write well-accepted software for AIX, you’re not going to use gcc; you’re going to use IBM’s xlC, which is really expensive.)

I can, without limitation, make modifications to the WordPress iPhone application and use it privately with my developer key. I don’t have to tell anyone that I’m doing this, at all. (My iPod Touch still has some demo programs on it.) I can also, without limitation, post my modified source (which, again, is all the GNU GPL gives a damn about) anywhere I want without telling anyone else anything about it.

What I can’t do, without going through the App Store, is publish a compiled binary for other people to use without having to effectively buy the compiler.

But none² of my rights under the GNU GPL have been violated. Period. Because the GNU GPL just cares about the software in the abstract, not in a specific compiled form.

I guess those silly hippies at the Free Software Foundation were right.

I guess those silly hippies forgot to read their own licence and recognise their own history when it comes to their licence. You and I might prefer a more open App Store with fewer restrictions on what software can run and how it could be distributed, but nothing about iPhone software development violates the GNU GPL, as much as people would love to pretend that it does.

I didn’t.

When they offered the $30/6 GB data plan, I started looking at my options and on Friday, I ordered an iPhone (16 GB, Black) with the $30/6 GB data plan, combined with my existing 300 minute plan totalling out at $75. I don’t have Visual Voicemail or text messages with this, but I can add Visual Voicemail and a few other features for $15 per month, so for $90 a month I’m getting a pretty good plan—and I don’t have to give up my long distance option.

I may shift my voice plan around a bit since the plan that I’m on has been grandfathered in and includes voicemail already (which I won’t need with Visual Voicemail) and I’m pretty sure that I can save some money doing that.

Even better, since I ordered my iPhone on-line, it’ll arrive sometime next week (without having to stand in line) and it’s going to be $48 dollars cheaper than it would have been because I had 48 “Fido Dollars” in rewards.

Go me.

Now, Rogers has the iPhone. And the best data rates that they’ve come up with are absolutely insane — with no unlimited data and that we should feel that their plans are “High Value”.

Colour me not interested. Here’s the text of my feedback to Fido:

Thank you for showing that Rogers is both stupid and greedy. I had planned on getting an iPhone in two weeks. Now I will NOT get an iPhone because of your absolutely insane data pricing. You call it “High Value” pricing; I call it “highway robbery” pricing. When AT&T has a better price plan than you, it strongly suggests that you’re out of touch. Let me know when your prices return to Earth with an unlimited 3G plan and I’ll happily upgrade to the iPhone.

I should have pointed out, but did not, that I have no problem with the idea that I have to spend $75 or more per month—I’m already spending a lot of money. But I want value for that money, and Rogers plans are anything but value to the consumer. They’re a pure rip-off.

That said, there’s a petition on-line that I refuse to sign, mostly because I refuse to have my name associated with some of the disgusting things (including wishing Ted Rogers bodily harm). I may think that Ted Rogers is a greedy pinhead, but we should act civilized, people. Grow up.

What really happens is that the economy grows more vigorously when you lower tax rates. It is beyond the reach of economic science to explain precisely why that happens, but it does.

Dani Rodrik calls this shift faith-based economics. I agree.

(Via Dani Rodrik’s weblog.)

(Via Laura Fitton of Pistachio Consulting.)

So, as most of the gaming world knows, E. Gary Gygax died today.

This begins John C. Welch’s paean to the more famous of the creators of Dungeons and Dragons. I haven’t read a lot of them, but I really liked John’s and really think it’s worth reading. Wil Wheaton also has some really cool things to say.

Like many other geeks, I grew up playing D&D. I don’t remember exactly when I first played, but I know that like Wil, it helped me survive with a few of the other geeks at First Baptist Church School in Charleston, South Carolina. It gave me a ready-made group of friends when we moved to San Antonio, Texas and I was starting at Judson High School. I remember long hours of play with Travis, Randy, and Von. We explored not only D&D, but various other games including Star Frontiers.

D&D was one of the things that helped me survive at Boston University where I knew no one (the other was an early exposure to the Internet of 1989, where IRC channels were numbered, and I played my first MUD—largely based on a mix of D&D and Adventure). Without D&D, there would be no World of Warcraft.

My favourite module to run as a DM has always been Ravenloft. At Boston University, I played with a group of Monty Haulers (one of them had three Wands of Wonder, fully charged, ok?) and they were quite cocky. They hadn’t met someone who could match them in using the rules. I boosted Strahd—a vampire as nasty as they come, and the main villain of the module—to their power. He was powerful enough to cast Anti-Magic Shell. This spell, for those of you not familiar with D&D, nullifies all magic in a radius of 14 feet for 60 minutes and vampires can only be damaged by silver or magic weapons of a certain strength (+2, I think). What was left unsaid in the rules, and I made a call on, was that vampires weren’t powered by magic.

Strahd attacked them early on and devastated them by casting Anti-Magic Shell, leaving them with no way to damage him. How did they drive him off? Through an innovative use of their combined magical items. The wizard cast Tensor’s Floating Disk and then used a Ring of Telekinesis to invert it. They poured five or six bottles of holy water into the inverted disk and then used telekinesis to float the disk over Strahd’s head. They dispelled the disk and dumped the holy water on his head.

I ran Ravenloft twice more with different groups, and always had a lot of fun. I played more than D&D, getting into Cyberpunk and various super hero games (I always thought Villains and Vigilantes was the best), and kept playing until just over ten years ago. But it was an important part of my life, and something that I won’t forget. And, as John said:

Goodbye Gary, and thanks. My life wouldn’t have been the same without you.

Lemurs are beautiful and deeply threatened creatures that are found only in Madagascar. Mike Lee of Delicious Monster is organizing something called Club Thievey in an attempt to encourage people to donate to the Madagascar Fauna Group.

This is a worthy cause. While I still haven’t decided whether I’m participating in Laptop Giving or I’m making a last-minute donation for 2007 to Médecins Sans Frontières (Doctors without Borders), I have donated to MFG; my wife did so earlier this year after reading Mike’s excellent Dinosaur Ranch. It’s the right thing to do, and I want to see Mike’s goal of 100 members in the troop reached.

When Craig Hockenberry released Twitterrific 3 as either ad-supported or a $15 licence, I paid it immediately. According to one “Captain Marc” over at Odelbee, there’s a “hack” to disable the ads from Twitterrific, who apparently thinks that I’m a bit of an idiot for paying for Twitterrific. (Interestingly, between comment #9 and #21, Marc went from thinking that Twitterrific was a “POS” and a pretty good app.) “Captain Marc” is wrong: the switch from free to free+ads or paid wasn’t without warning; this was stated up front on the download page of Twitterrific.

He’s further wrong: Eudora was simply a tool for receiving and delivering … content, yet it sold for quite a long time. Very few of the applications that I’ve bought for the Mac since I switched eighteen months ago have I had any qualms about buying after buying them, and Twitterrific is one of the apps that I use every day, multiple times a day. There are features I’d like to see, certainly, but it’s a damned good product and I’m proud to have paid for it.There’s plenty of software that I wouldn’t buy at full price; sometimes I’ve waited for bundles (and my unwillingness to pay full price has generally been justified by the lack of use the product ends up seeing). More often, though, I just stop using the product. If I use it, though, I pay for it. No ifs, ands, or buts about it. And so should you, “Captain Marc.” You certainly shouldn’t be posting hacks or alleged hacks.

More on this from Seth Dillingham and Justin Miller.

The Twitterrific icon in this post is copied from the IconFactory web site. It belongs to IconFactory and has been used with permission. 

CTGradient contains an incredible diversity of built-in gradients, gradient styles… For demonstration purposes, all these features are excellent. For production, this is a nightmare.

It gets no better at the end:

…The documentation already shows you how to draw gradients, yet the number of applications using CTGradient – the whole 1300 lines of it – is astonishing.Please: When you use other people’s code, don’t put it in without a thought. Go through it, understand it, and optimize it for your specific need. For the better performance and reduced RAM usage, computers will thank you.

Quite legitimately, some Mac developers spoke out about this. Hoare’s famous dictum Premature optimization is the root of all evil was pulled out early. It’s completely applicable here by any measure. This story would have ended here, and I wouldn’t be writing now, had it not been for a series of bile-filled nonsensical articles posted at Rixstep, starting with one calling Mac developers (such as Daniel Jalkut) objecting to this inflammatory article as the Landed Gentry of Mac Development™.

Which Optimization?

I’ve been developing software for a long time. While I haven’t done any Objective C programming yet, there’s nothing in Ankur’s article which is unique to Objective C. When developing software, you make it work, make it right, make it fast. In that order. The goal is to ship your software, not have it languishing in interminable development. Mac developers aren’t the only ones who will use third-party source without looking for optimizations (or further optimizations, as the case may be); Unix and Windows developers do this, too. Any developer worth their salt does this, because the first goal is to make it work. At work, when we see performance and memory issues, we don’t start digging through third-party code. We don’t even start looking at our own code. We start looking at profiled performance data. Then, and only then, do we start to make something fast.So, if the first goal of software development is make it work, what’s the first optimization you should do? You should optimize your developers’ time toward shipping the softare. Window background gradients aren’t the type of thing that will sell more copies of an application, but their absence may prevent some sales because the application doesn’t look “tasty” enough. So, you start looking at how to provide gradients. You see that Core Graphics supports it, and you start digging in the documentation and you see that it’s going to cost you lots of extra time to learn the CG gradient support. And then you have to debug what you’ve written. No sensible unit testing here; this is visual inspection. If you can add a single file that reduces your time to implement a background gradient—which you don’t really care about yourself but you know it’ll cost you sales not to have it—then you’re going to be better off entirely. Scott Stevenson of Theocacao stated it much better in 2006:

Ah. Now doesn’t that feel better? It’s not that the class does anything that is otherwise impossible, it’s just a lot cleaner because all of the goofy callbacks and whatnot are moved into their own code space. In other words, you have more free time to work on the actual application.

CTGradient is (almost) all about developer optimization. It also helps you deal with the second optimization: optimizing for correctness. It’s code that someone else has written and debugged. You know that other people who develop Mac apps are using it, and no one is screaming loudly about bugs in the code, so you feel pretty comfortable with it. So, you know that by using this drop-in library, you are not only getting this negative feature done, you’re getting it done right. At this point, you can forget about the getting it done fast, because no one has complained about gradient performance at this point, because you’re not yet done shipping.Make it work; make it right; make it fast. Optimize for developer time first (and this includes good design), and then worry about the rest when you need to.

More Code Optimization = Better Developer?

Things really went off the rails with this discussion when Rixstep posted the “Landed Gentry” article. He’s posted further bilious articles attacking some of the developers involved in the discussion over CTGradient optimization, but I’m not considering those articles for this discussion, since they’re pure bile and add nothing meaningful to the discussion (that is, they’re still based on false optimizations that I’ll address in a moment). In the “Landed Genry” article, Rixstep asks:

Who would you rather have engineer your software? Someone who’s as conscientious as Ankur Kothari? Or someone who squirms and attacks and insinuates and does the absolute utmost to avoid the actual question? You the user/paying customer can decide. Wander over to Ankur Kothari’s article on CTGradient and see who’s objecting. At least you’ll now know what the Landed Gentry of Mac Development™ think of you.

Rixstep presents a false dichotomy here. Ankur isn’t particularly conscientious in his article; there are specializations presented as optimizations, and some optimizations aren’t geared toward the most important parts of development anyway. Daniel Jalkut, on the other hand, is very conscientious toward his most important target: his customers’ time in using the program. I’d be very surprised if anyone was saying that MarsEdit is too slow because of gradient display.Optimizing on parts of code that don’t matter doesn’t make you a better developer. Optimizing the right parts of code at the right time do. If I were hiring Ankur and Daniel, I’d have to watch over Ankur more to ensure that he wasn’t working on stuff that doesn’t matter to the customer. Like optimizing gradient code without actually knowing that it was a performance bottleneck. Ankur does a decent job of making sure that the resulting code meets the minimum required needs of the job (which, if you’re developing from scratch is a good thing, after all, YAGNI). He outright says that one should spend time going through third-party code that (probably) isn’t relevant to the primary mission of your softwasre, and therefore doesn’t help toward “make it work.”In reality? They’d probably both work out as great developers. But Ankur cares no more about a user’s software experience than Daniel does, despite the insinuations of Rixstep. I do question Ankur’s judgement on the optimizations made and how they were reached, or at least how they were explained.

The Axis of Optimization

Let’s play along for a moment, though. Assume we have already determined that there’s a measurable performance problem and we don’t have any currently outstanding feature requests or other problems that are higher priority than this performance problem. Assume further that we’ve profiled our program and we’ve determined that yes, our source of program slowdowns is CTGradient.There’s a legitimate question about whether Ankur’s effort was really optimization or specialization. When he says Firstly, let’s remove the methods we know for sure we won’t need…, the discerning reader should be asking why we don’t need those methods. Ankur doesn’t explain how he reached this conclusion. This means that the developer who is doing their job right won’t immediately think of eviscerating the entire CTGradient library, but rather measuring the performance characteristics of the library.Ankur made a mistake in his approach. Yes, simpler code is usually easier to maintain, debug, and will usually perform better. But simpler code does not necessarily mean better or more performant code. A bubble sort is simpler than quicksort; there’s no way that it’s faster. One may as well look at binary size for comparison. This isn’t to say that one shouldn’t strive for smaller binaries; the larger your binary, the more likely it is that you’ll cause something else the user uses to swap out to disk, which would be bad. Can CTGradient actively contribute to this? It’s not distributed as a Framework, so most projects compile it directly into their code.CTGradient’s 1,172 lines of code in six source files (as determined by Sloccount) doesn’t even come close to adding meaningful binary code bloat. I compiled CTGradient from the latest available SVN checkout and Ankur’s “Lean Gradient” project. I compiled both as Universal using the default settings and compared the final binary size (from Contents/MacOS/) and the intermediate object file sizes. Since the projects are structured differently, it’s not a completely fair comparison. Ankur’s binary does one gradient and results in a binary size of 42,992 bytes; CTGradient does a lot of different types of gradients and results in a binary size of 78,776 bytes, a difference of a mere 35,784 more bytes of Universal code. A lot of the binary size differences is overhead, though. CTGradient creates three object files per architecture: CTGradient.o (55,964 bytes), CTGradientView.o (14,160 bytes), and main.o (976 bytes), leaving an overhead slack of 7,676 bytes. optGrad just has main.o (10,912 bytes), leaving an overhead slack of 32,080 bytes. So, CTGradient.o isn’t going to add to your binary size overhead in any meaningful way.What about memory use? Ankur posted two follow-up comments that suggested that CTGradient adds between three and ten megabyes of memory use to a program over and above his approach. That suggests that there might be room for a CTGradientLite, but making that would require extensive profiling to determine the parts that could be excised prior to doing so. And, of course, if all you need is a single gradient like Ankur did, and have the time and mathematical knowledge to do what Ankur did, by means do so. Ankur’s five minutes, though, might be five hours for you—so make sure that you don’t have something more important you should be working on.

The Unwashed Messes

Ankur can perhaps be forgiven for the mistakes he made, as other posts by Rixstep suggest that he’s quite young. He hasn’t yet had to learn the personal interaction lessons that most of us have to learn, and that most of us do learn. There’s a few who don’t, and Rixstep appears to be one of these developers who resisted learning about how to behave toward other people throughout his career. He stepped into this discussion “defending” Ankur from the “Landed Gentry”, who were simply asking the same questions that any good software developer would. He’s gone further recently into personal attacks against two of the more vocal questioners. Reading further on Rixstep’s site, anything that is selling better than his software seems to result in that anything being called the “Landed Gentry” of something. He thinks that he’s defeating dragons, but in reality he’s tilting at windmills just as usefully as Don Quixote did.Rixstep is supposedly in the business of selling software. The content and tone of his posts have managed to do the opposite. I had recently considered purchasing some of his software; I know now that to do so would have been a mistake, because he doesn’t treat his peers well. It’s impossible for him to treat his customers well with that sort of attitude. It doesn’t take much to be generous in spirit to people, at least in starting. It doesn’t take much to realize that there are better ways to deal with conflict than has been done.There is a difference between taking a hard-line, hard-nosed approach to something and defending it on technical grounds and actively attacking someone without basis. Rixstep’s attacks, starting with the “Landed Gentry” article, are entirely without merit. Ankur didn’t optimize CTGradient; he made a specialization. This is valid sometimes, but in reality something like window gradients isn’t central to your application’s purpose, and to spend more than an hour or two dealing with them would be wholly inappropriate. The good developer knows that it’s better to ship than to spend all your time optimizing someone else’s code or rewriting it yourself.

Where does Ruby Fit into This?

The astute reader would have noticed that I filed this as a Ruby article as well. While I haven’t mentioned Ruby until this point in the article, the parallels here are obvious. Rixstep and Ankur both say that the dozens of applications that use CTGradient are abusing their users because it uses too much code, too much space, too much memory, and it’s too slow. Aside from the memory use, the claims have either been exposed as false already or haven’t been supported with data. As Scott Stevenson’s quote suggests, CTGradient optimizes for developer time.So does Ruby. I can get more written with Ruby than any other language that I use. It won’t be the fastest software, it might be a little more memory intensive, but I will get it written—and written correctly—faster. And that, given that shipping is the goal, is important. When Rixstep rejects CTGradient without providing a more optimized yet equivalent functionality alternative, he rejects any developer-side optimizations in favour of hand coding everything, including using more expressive languages.I reject that notion. And so should you.

I really wish more politicians believed this and started investing in more and better options for getting around. One of America’s greatest presidents was a conservative—in the absolute most accurate sense: he was a conservationist. That man was Theodore Roosevelt. Too bad that most modern conservatives aren’t after conserving anything.

(I’m not fond of most liberal-environmentalists, either, like Greenpeace. Frankly, I’m a futurist—I believe that technology will make a difference, and that nuclear is vital to the future.)

(Via kottke.org.)

A couple of minutes more digging yielded an older magazine, though, by five months.

But I kept looking. Long before my appointment was ready, I found something twenty-four years old:

But right under that one was one that was even older: Woman’s Day from February 1982. Twenty-five years old, almost twenty-six.

I don’t have a problem with identifiable pseudonyms, like Fake Steve Jobs or Mini-Microsoft or even why the lucky stiff. Those are all people who can be identified. I do have a problem with anonymous cowards, as Slashdot calls them.

If you want to comment on my blog in the future, you must provide either a valid email address that I can verify personally (and I will) or a valid URL to your own blog or public profile page that identifies you in a way that people can see what you’re about—whether you’re a real person or a pseudonymous person. If you don’t do one of the above, your comment will be deleted with no notice.

I also won’t allow abusive comments, toward me or toward others. If you feel that I have been abusive in a post, please post a comment. I will act on it. (See the discussion in last year’s posts with Ola Bini; we worked out our differences.)

My readers, whomever they may be, will be able to know that even if I accept a pseudonym, I have at least been able to find someone who will own up to the words posted here.

Comments on this post are closed.

The ConservativesReform Party, on the other hand, are a bunch of rednecks who would love to see it brought back. So, they come up with excuses as to why they’re backpedaling on forty years of precedent. They sound as lame as the Republicans do south of us.

It’s really too bad that Dion’s Liberals are too lily-livered to come up with real positions that the ConservativesReform Party can’t respond to without sounding like the rednecks they are. Or that real Progressive Conservatives can’t stand up and take their party back from Stephen Harper’s rednecks.

This is obviously old. The cut of the model’s blouse is not recent—I’m one of the least fashion-observant people in existence, but even I can recognise this. More obvious is the model’s hair style. Yeah, there’s a few people who have similar hair styles in 2007, but it’s very uncommon, and never on magazine covers at this point. So, when was this thing published?

Eighteen years ago.

I expect to find old magazines in a medical office. It’s a common thing these days, right? But eighteen years? Isn’t there a statute of limitations for keeping around old magazines?

This is not to say that I haven’t had to debug; it’s to say that where tests haven’t sufficed in helping me specify behaviour correctly, Kernel#p has helped me find what did go wrong so that I could fix the problem and add a test (where appropriate; PDF::Writer, for example, has no tests).

So I think that Giles Bowkett is right, even though I’d say that debuggers aren’t harmful; they’re pointless in a pointerless language.

I find that I mostly use the debugger in C/C++/C# for stack traces in any case. It’s a bit less painful with a good IDE, but depending on the nature of the problem that I’m debugging (especially one in a tight loop), I will usually add print outputs to a log file and debug based on those logs. Yes, even when there’s pointers involved (because it’s usually a binary search approach where I figure out where a pointer went bad over a large run).

Windows lost a huge chunk of the nerd market. Nerd switchers, in and of themselves, don’t constitute a significant enough number of people to account for anything other than a tiny blip in Apple’s Mac sales. But nerds are the people who recommend computers to friends and families; it seems inarguable that there are an awful lot of nerds recommending Macs today who weren’t five years ago.

I bought my first Mac (15″ MBP) in August 2006. In some ways, I bought it two months too early; in others, I bought just at the right time (I needed a new home laptop). In that time, I have convinced my parents to ditch their two Windows computers for a Mac—my MBP, when I upgrade early next year; I have told my wife that her next computer when this one dies will be a Mac and I have recommended Macs to three other people. There’s very little that one can do on Windows that one can’t do on the Mac; the only compelling reason for most people is games.

If you’re a big player of FPS games, you want a Windows PC. If you must always have the latest and greatest video card, you want a Windows PC.

Anyone else? You want a Mac. No, Apple doesn’t offer a $400 Mac. If that’s your budgetary limit, you *still* want a Mac, but you can’t afford one. Look at one of the Linux vendors. Because you’re not going to run a decent version of Vista on that $400 PC. And you’ll need to replace it sooner than you would an equivalent Mac. My rule of thumb for non-Mac hardware has been that you’ll get about 12 – 14 months of meaningful use per $500 you spend. I suspect that it’s 18 – 24 months of meaningful use per $500 you spend on a Mac. By meaningful use I mean before you start noticing memory and graphics limitations requiring significant hardware upgrades to keep up to date.

I’m not going back. I don’t know that I’ll stay with the Mac “forever”, but at this point, it’s the best computer investment that I’ve made so far. Aside from that first computer (and it was my dad’s investment then), that started me on the road I’m on.

The wrong question is the comparison between Vanilla Ice and Soulja Boy. Vanilla Ice was low rent back in the day — his second album was a live version of the first album, and his “superstardom” lasted about a year. Soulja Boy’s rap may be better than Vanilla Ice’s rap, but it’d be better to compare Coldplay or Oasis (people with more than one album to their name) to The Beatles or Pink Floyd when asking whether the singles price on iTunes is good, or whether they should be variably priced.

Radiohead may have hit on the right way to handle variable pricing, but that only works when you know you’re sending the majority of your money directly to the band. Most people don’t want to make the soul-suckers at the labels any richer than they need to be.

John replied last night with:

interesting — i read that differently. soulja boy is currently listed as one of the top 10 downloaded songs on itunes, which gave me the impression that his point was about demand. (right now, lots of people want that new hit single, and far fewer people want the old single that now sounds like crap.)

so i thought it wasn’t about which musician was “better” — i don’t especially like either of those songs, personally — but about which musician’s product was currently more “in demand.”

The record execs want you to think that it’s about demand, but it’s not. Demand pricing makes sense when there’s scarcity and you have a lock on the market. Newer music is anything but scarce. You can hardly go anywhere without hearing it as a ringtone, an advertising jingle, or just on the radio. I listen to a “classic rock” format station, yet they’re playing new music from Neil Young and Kim Mitchell, and they consider U2 (up until All That You Can’t Leave Behind, at least) classic rock. So music has the opposite problem of scarcity; there’s a glut.

The radio brings up another important point—as much as record execs would love to pretend otherwise, they’re not competing like you do in other markets. They’re competing against free. There’s plenty of songs out there that I like, but that I don’t like enough to even consider buying even at 99¢—not when I can hear them periodically on the radio. And if I wanted them, but they were more expensive than my lowest willing price, I can usually find them somewhere online. (I don’t; but I have used AllofMP3 before. I already pay the CRIA something every time I buy blank media, so I have the right to get songs that others share that way.)

There is one other theory the record execs could be using here, which is the concept that one values something one has to pay for. Therefore, one values something more that one has to pay more for. If this had been possible when Vanilla Ice was in his (short-lived) heyday, people may have been willing to pay $3.00 for “Ice Ice Baby”, but how many people would have regretted it later and deemed themselves fools? (The song has not aged well.) So the value of a song isn’t really derived from how hot it is, but its longevity. And even then, you’re still competing against a model that treats songs as essentially valueless (radio or satellite radio).

So, demand pricing doesn’t work well if there’s no scarcity (or artificial scarcity). We also know that subscription pricing for portable devices doesn’t work. (More accurately, it doesn’t work for portable devices where you control the content. Satellite radios are portable, but you don’t control the content.) Subscription pricing works when it gives you a menu of things to choose from (cable television, satellite radios) but the value is provided by the menu, not the individual pieces that make up the menu. (The subscription price is too low per person for the individual songs or television shows to be worth much at all. In aggregate, they can make money, but individually they’re fractional pennies value.)

What does work? That’s where we get back to the Radiohead experiment. They asked people how much they value their music. Some folks bought it for the absolute minimum; others paid much more. Most paid about the same price that they’d pay on iTunes. But you have to have an environment where that will work, and I don’t think that the Radiohead experiment is generally applicable for the entire catalogue of some bands’ music. (That is to say that I think that it’s a decent short-term strategy when the music is new, but I don’t think it’s sustainable for more than a few months.)

I don’t agree completely with Apple’s iTMS policy (you can have some songs album-only, but you can’t have an album-only album), as I’d love to see album-oriented rock make a return. (But I’d also love to see sound-bite politics go away.) I think, however, that the simplified pricing models make a lot more sense than pretending that something is more valuable because more people want it, when there’s no scarcity involved.

Souljaboy may be hot. Will he be next year? Or will he be dropped like Vanilla Ice?

…Whomever you believe, the fact remains that content providers have been pushing Apple to loosen up its pricing restrictions, and Apple has refused.

Regardless of demand, each song on iTunes costs 99 cents, each television program $1.99, and each feature-length movie $9.99. Most consumers are likely to agree that iTunes’ pricing seems illogical; there’s no obvious reason that Vanilla Ice’s ‘Ninja Rap 2′ should cost the same as the newest tracks from Soulja Boy. For content owners, this is more than illogical: It’s bad for business.

The question isn’t about Vanilla Ice. The question is about Led Zeppelin, Pink Floyd, The Rolling Stones, The Who, and the Beatles. There’s no reason whatsoever that newer, “hotter”, music should be more expensive than legendary music by these bands. Is the latest Justin Timberlake even remotely as good as “Behind Blue Eyes” or “Paint It Black” or even “Yellow Submarine”?

The usual standard presented by the recording industry morons is variable pricing based on the freshness and popularity of the music, where the more popular and fresh the music is, the more expensive it is. This works in a scarcity model, but music isn’t scarce. Music’s value increases the more that it’s heard and the more people internalize it. But if you’re going to price Soulja Boy’s latest tracks more than Harrison’s masterpiece “While My Guitar Gently Weeps”, you’re implying that Soulja Boy’s music has more value than the older, and better, music.

Why does Apple stick with fixed pricing? Market analysts generally say that this is because iTunes sales are a means to an end, where the end is selling iPods. As such, Apple’s interest is ensuring that desirable content for the iPod costs as little as possible.

John Gruber addressed this quite nicely in September. I’m not trying to say that Apple is perfect (far from it!), but I would assume the obvious: multiple price points confuse people unnecessarily. Those market analysts, of course, are wrong. Ivan Askwith gets this one right:

The more likely explanation, however, lies with the company’s obsession with simplicity. ITunes has been a huge success because it’s easy to use, and (at least for now) has the most digital content of any online store. Apple’s refusal to budge on pricing indicates it’s prepared to defend simplicity at the expense of selection.

That target of simplicity is important: the harder it is for people to play the media they want on the devices they want, the less they’ll buy the one they need less. (And, so far, the iPod is the device people overwhelmingly want. People will forgo on-line purchases if the rest of the device is easy enough to use. I’ve bought a total of four things from iTMS, and two of those were audiobooks.)

Simplicity matters. Amazon works, despite the pricing differences, precisely because it makes the shopping and shipping simple. Amazon’s music store works well because it integrates with iTunes, too.

What’s a fad, though, is how these networks work. I think that the people who are dealing with “open” social networking have the right approach. Facebook will have to adapt to input from outside (opening up the inputs), as it recently did with allowing applications the ability to set a user’s status, as with Twitter. Facebook has already said that it’s going to open certain things to the outside on an opt-in basis; if it can nail the user interface (and that’s a big if, given how many people stick with the standard privacy settings), then we’ll have an even bigger when as the network effect not only deals with people, but with sites.

Social networks as networks are fads; the value they provide, though, is real. The challenges that remain are better classification of friends and a reduction in the amount of effort it takes to allow users to segregate information between groups of friends. That last is important. I’m on Facebook. If I were looking for a new job (I’m not), I might talk about it on Facebook. There’s a problem, though: my boss and some of my coworkers are on Facebook and are friends with me. The moment I started talking about it publicly, they’d know something was up and it might make for bad relations at work (at a minimum).

Now, I’m pretty open, but if I wanted to do something like that, I would want an easy way of drawing a circle around them and saying: they can’t see these updates.

Before it’s too late.

Could be interesting. I’ll see.

Personally, I don’t care if Mac OS Rumours is right or not; I don’t listen to them. The idea of this move makes no sense anyway given that the Mini was just given an update in August. What interests me are the comment threads on TUAW and Digg.

Take, for example, the first comment on TUAW, asking where the mid-range headless with upgradeable components is. Or this one from Digg. What about this one calling for a Mini with an upgradeable video card?

I gotta wonder whether these people are nuts or stupid. Actually, I don’t really have to wonder at all. Like it or not, Apple is about control. You’re never going to get a consumer-level Mac with much of an upgradeable anything. They’re not trying to compete with Dell, so they don’t need a minitower model for consumers. The Apple model distribution is simple—and that’s on purpose. The pricing is roughly accurate: you spend more, you get more. Contrast that with Dell, where spending more doesn’t always get you more—it gets you different. (It always took me a couple of hours to figure out the best base-model machine to configure from at Dell for the best bang for the buck. It doesn’t take long at all with Apple.)

Insanity is colloquially considered to be trying to do the same thing repeatedly while expecting different results. You can keep wishing that Apple will cater to you and not to what will actually make them money, but it won’t do you any good. So, keep wishing. You’ll keep me laughing at you.

(Via Daring Fireball.)

“Never before in the history of content has the hardware been more valuable than the software…You think about the VCR or the video cassette—the video cassette always had more value than the VCR that you shoved it into. Apple has been able to turn that model on its head.”
Edgar Bronfman, Jr. (Chairman, Warner Music Group), in Like Amazon’s DRM-Free Music Downloads? Thank Apple

Edgar Bronfman is an idiot who simply doesn’t get it. Apple hasn’t made the hardware more valuable, it’s made the collection more valuable. It used to be that you weren’t able to carry the majority of your media collection with you; now, that collection represents a valuable investment. Not the hardware.

Moron.

I have, however, been thinking about the “list” seats versus geographically-based seats (”ridings”) and come to the conclusion that the problem isn’t list seats, but ridings. My office-place is an interesting example in this problem, in that the office is in nominally in Oakville (it’s at Winston Churchill and Dundas, right at the very border between Oakville and Mississauga), but very few people live in Oakville. A lot of people live west of the office, in Hamilton, St. Catharines, or Burlington; a lot of people live in Mississauga proper. Some, like myself, live in Toronto. I spend nearly ⅓ of my life away from where I live, and of the ⅔ that I’m at home, roughly ⅓ of it is spent sleeping.

So while I care about my neighbourhood’s representation in parliament (both federally and provincially), I also care about the places where I work and (at a minimum) the transit corridor I travel to and from work every day. My concerns are less about where I live than how I get to and from work, which means that regional transit policies matter to me. I can’t effectively and efficiently use a transit system to get to work. (The local Go station is ~10 minutes from my house, the train doesn’t run that often and even less often coming home, and the nearest station is still ~15 minutes from work by bus; my total commute by car is under 35 minutes).

I care about how municipalities (who should be caring more about the local rights and responsibilities) are being run over roughshod by the OMB and developers and the province itself. Mike Harris did more damage to Toronto’s infrastructure through forced amalgamation and downloading than anyone else. He also reduced our democratic representation by cutting the number of city councillors from 57 to 44—the City of Toronto web site says this was adopted by City Council, but I recall reporting at the time was that the adoption was at electoral gunpoint, just like the almagamation itself.

In other words, 90% of my concerns aren’t limited to my relatively small geographic region of the Parkdale-High Park riding, and the concerns I do have about Parkdale-High Park should be addressed through City Council and my local councillor rather than my provincial representation.

Ultimately, I don’t know that I care whether my representation is regional, and even think that regional representation may be the oddity in today’s world. As such, I can only end up supporting MMP. It may not be perfect, and I may regret supporting it in the future, but I don’t believe that being held hostage to the past in this case is a good thing.

Update: Reading a bit more, I have found another point that puts me in favour of MMP. The claim is that regional ridings represent the will of the people. This is only partially true, in that there are plenty of examples of parties parachuting in “star” candidates. I think Ken Dryden was this way for the federal Liberal party; while the people of his riding ended up voting for him, he did not have to win his party’s nomination for the seat.

Let me be perfectly clear: I disagree with Andrew Coyne on most of what he says and writes. I don’t believe the way he believes. He believes that Mike Harris was good for Ontario (when exactly the opposite has proven true). But he is absolutely right when he says:

A Tory government…of Ontario would look much the same under Mr. Tory as under Dalton McGuinty. It would do much the same things, at much the same cost, with much the same results.

Oh, Mr. Tory would fiddle at the margins — cut a tax or two, expand funding to a few thousand kids in religious schools — issues that both leaders would like you to think show the vast gulf between them. But they’re not kidding anyone. Whoever wins, the forecast is for McGuintory governments, as far as the eye can see.…

[The current voting system creates most of what we find in] Canadian politics — viciously partisan, yet unspeakably trivial; much ado about nothing much. McGuintoryism, in short.

I think that John Tory would be bad for the province (he keeps pretending that he’s like Bill Davis, but he’s much more like Mike Harris), but that doesn’t mean that I think that Howard Hampton or Dalton McGuinty are good for the province. Something has to change; MMP may not be the best change, but we have to try.

(Via Spacing Votes.)

All about the girl who came to stay?
She’s the kind of girl you want so much
It makes you sorry
Still you don’t regret a single day.
Ah girl! Ffff…Girl!

With Lennon’s “Girl” begins Julie Taymore’s latest film, Across the Universe. The film is visually rich, musically gifted, and utterly confused. Nonetheless, I really enjoyed it.

This movie is divisive, make no mistake. Rotten Tomatoes marks it “Rotten” as of the date of this post, as shown below. Only 51% of the critical reviews are positive. Look at those reviews, though, and you’ll see strong feelings for or against the film.

Disclaimer: I went to see this movie because my brother’s in it, as a dancer (when Max goes into the induction centre and is stripped to his boxers, Kenneth is the first soldier to Max’s right, and dances behind him; later, he is two beds to the left of Max). I also didn’t pay anything to see it (due to some complimentary—or compensatory—tickets provided by the theatre for a screening of Pirates of the Carribean that was interrupted by a 5-year-old).

I was prepared to hate the film, because going in you know it will take every possible cliché that it can. Despite a very slow start and an often muddled middle with the various characters flitting in and out of the film, the film managed to keep me entertained for more than two hours, and came across as relevant today, at least to me.

This is art, but it’s not high art. It’s candy on the order of Moulin Rouge!, but perhaps not as well executed in the end. Still, if you enjoy the Beatles, enjoy in-jokes, and are prepared to accept a caricature of the 60s (but what movie isn’t a caricature of an era), you will probably enjoy this film.

Update: I have (hopefully) fixed the comment-posting issue.

There are plenty of reasons to dislike PHP (although all of my criticisms have to be aimed at PHP4 right now, as I haven’t done anything with PHP5), but it is a useful general purpose language. It has different philosophies than Ruby, and I think that in general it isn’t nearly as good a language as either Ruby or Python. But it is just fine for a lot of people and projects.

Derek’s planned migration to Rails was probably doomed from the beginning for several reasons:

1. Derek chose the technology for the wrong reasons. He chose it partially based on the hype of Rails, but he envisioned it as a silver bullet that would magically make his application better just because it’s in Rails. Rails has advantages (not least of which is its language, Ruby), but it has drawbacks and weaknesses, too.
2. Rails didn’t fit Derek’s application model for CD Baby, and Derek’s application model is more important than the technology to be used, since it represents a business he understands well. Rails requires that your application fits its model, not the other way around. As Derek says:
   

   I hired one of the best Rails programmers…Jeremy [Kemper] could not have been more amazing, twisting the deep inner guts of Rails to make it do things it was never intended to do. But at every step, it seemed our needs clashed with Rails’ preferences.

3. He ignored his existing experts for the new technology. Neither he nor his employees knew Ruby aside, perhaps, from playing around with it. This wasn’t a technology that was deemed to be appropriate from experience; this was a technology deemed appropriate by management (sorry Derek, you might still be getting your hands dirty with code, but you’re still management).
   

   I’ve wanted to introduce Ruby in my workplace for several years, but aside from a number of compile-time helper scripts, it hasn’t been appropriate to do so. First, our product isn’t in Ruby’s sweet spot. Second, none of my co-workers know how to deal with it or its development environment. Even when Ruby is a superior choice, it’s hard to introduce it, because I’m the only Ruby expert at work.

4. Derek approached the project as a whole-environment ground-up rewrite with a One Big Day deployment, without considering ways to phase it in over time. It’s almost always possible to find interface points where you can replace one broken piece at a time. Ultimately, this is what the Rails folks wouldshould tell you anyway: replace one area at a time, each with a different codebase. Interface them as REST-ful services. Don’t make them depend on a single database schema.

Derek is wrong, though: language matters. A “better” language may not be the right choice for your environment, but its lessons will help you immensely. Without the Ruby and Rails experience, Derek would not have known to apply the lessons Ruby teaches to his PHP rewrite. I find that I write both C++ and C# with strong Ruby flavours these days. My boss, who has only used Ruby casually, has come to prefer the Ruby method naming convention (like_this) and a number of Rubyisms that I’ve introduced.

Ultimately, though, Derek’s Rails rewrite failed because he made a number of classic technology management mistakes. Not because of Ruby, Rails, or PHP. Because of management decisions made for the wrong reasons. Fortunately, he recognised this and was willing to reverse an expensive decision to fit his business better.

• Spacing.ca — MMP Primer Part I
• Spacing.ca — MMP Primer Part II
• Spacing.ca – Web Resources for MMP

I don’t think that it’s a bad system, but I am concerned about the reduction in the number of geographically-bound seats (ridings) from 107 to 90; I’d rather see us have 120 geographically-bound seats and an additional 29 “list” seats. I haven’t really decided which way I’m going to vote on it, but I am edging in favour of it.

Still, it’s interesting stuff and if you’re an Ontarian, you really need to vote on this.

Updated 2007.09.24: Added the link to the Spacing web resources for MMP. It’s important to note that MMP isn’t just a random proposal, but something strongly considered by fellow Ontarians.

Joel doesn’t present FogBugz as a bug tracking tool, but as a communications tool. FogBugz is definitely an example of opinionated software, and works better when you adapt your work practices to it rather than trying to make it adapt to your work practices. After the demo, I asked Joel how he felt FogBugz and Basecamp work together or differ. His response suggests that there’s some similarities in how the two tools work, although he sees limitations in how Basecamp handles tasks and schedule changes. (Disclaimer: I have not used either FogBugz or Basecamp at this point.)

The software development lifecycle as presented by FogBugz is presented in its five major modules:

• Vision and design (wiki)
• Implementation (project management)
• Release planning (evidence based scheduling)
• QA (bug tracking)
• Support (email and discussion)

Of all of the features that I saw, the wiki seemed the least useful, although the WYSIWYG editor for it was pretty impressive. The value in the wiki is not the editing space that it provides, but that it provides it in the same place as you track your implementation, releases, QA, and support.

The other three pieces are all based around case management. A case in FogBugz can be a feature, a bug, or even a development task. Cases have time estimates and work times that are used in calculating estimated ship dates. FogBugz uses R-squared analysis of a developer’s estimates and actual work time to determine their reliability of estimates, and then runs Monte Carlo simulations to determine when the last feature or bug required to be fixed will likely be completed based on current task assignment, giving an estimated ship date. It’s all very impressive. (Estimates and work times aren’t immune to gaming, but that’s more of a social problem than a technological problem.)

Case entry and management is insanely simple. There are no required fields (which will, of course, bother some people), but it makes adding cases (which are tasks, bugs, and even support requests) dead simple. There’s no implicit dependency tracking, but you can easily link two cases together simply by adding a note (e.g., “waiting for case 72″) and FogBugz automatically provides a link between them. One clones bugs the same way. There’s good SCM integration into FogBugz (including for Perforce), and there’s even a VisualStudio plug-in for task/case management.

The demo was useful to see, but the hard part will be in seeing exactly how it would integrate into work’s development cycle. It’s not that expensive, so it may be worth getting a few licenses (or trying a few months of the hosted version) so that we can see whether it would work for us.

I don’t really want to give up PDF::Writer or its ancillary libraries, but I owe it to the community to find a maintainer who will be more responsive and put more effort into it than I have. There are some issues to sort out regarding the pieces of the projects, and some changes that I would like to see someone implement. There are basically three projects:

1. Transaction::Simple. This has improved significantly since the last release, but I have not yet implemented the Ruby 1.9 Marshal.load block trick that Matz implemented for me after a minimal case for a #become-like behaviour was shown.
2. color-tools: This involves someone else, because after some preliminary discussion last year, I moved the color-tools codebase to the Color project on RubyForge and restructured things to be a bit smarter. There’s also some work that should happen hear regarding colour profiles, but that’s manageable. I basically want to work with the current owner of the Color project to hand this entire project off to someone who is interested in the math behind colours, and has the expertise to do something with it.
3. PDF::Writer. The big one. I need someone to take care of this. No, the community needs someone to take care of this. I’m more than willing to share some thoughts about the code, but it’s a bit of a mess, and there are better ways to do what I did. The code can’t support the ultimate goal of reading.

None of this means that I’m giving up on Ruby, or on PDF generation; if I find time, you may yet find a different set of PDF tools from me in the future. But PDF::Writer is here and it needs someone to help maintain it.

Could that be you? Comments open on this post for interested parties to let me know. Be warned: I’m not just handing off PDF::Writer. I’m going to be looking at code samples; wanting patches. I need to know that you’re going to give the care the PDF::Writer needs and deserves before I hand off the virtual keys to the project.

I’m going to be playing with some new themes for WordPress and some new plug-ins while I work with this. In the meantime, I will be disabling new user registrations (at least temporarily, and possibly permanently) and comments on existing posts that aren’t disabled (they should all be, but I haven’t verified). I may take Joel Spolsky’s advice on not having comments enabled, but I’m not sure. I will be deleting all currently registered users, too. There’s been a slow but steady tend of spam-users being created. They can’t do anything, but it’s still annoying.

I’m also looking for a single webhost to give me a reasonable yearly price for good performance (the hosts at OLM and 1&1 have been slowing down, and while I liked Linode, I find that I don’t actually want to manage everything that way) with shell access and compile rights (so I can put Ruby on their system if they haven’t). I might like a Linode-like system where I don’t have to manage everything. I don’t care about email; all of that needs to be directed through my Google Apps account. I’d like to consider running one of the Ruby/Rails blogging packages (and I’m not sure which one, to be honest), but I want it to be reasonably complete and easy to use with plug-ins like I can get for WordPress (I’m going to be adding twitter integration soon, via plug-ins). Any ideas? Email them to austin at zieglers dot ca and let me know.

As far as editorial responsibilities, I think that you do have a responsibility to ensure that what an author writes is both sensibly written and doesn’t contain gross errors. Editing does not mean that you change the authors’ words for them. You let them know when something isn’t clear or likley to be misunderstood. This was the policy at Artima’s Ruby Code & Style under James Britt. The words for the article that I wrote (and another that hasn’t yet seen the light of day, so I will probably publish it here after rewriting it) are all mine. Not someone else’s.

Matthew Huntbach: Somehow, I missed the reference to that other article. Aside from the fact that GameSpy made it nearly unreadable with stupid pagination, the original author’s point is as muddled as your own on this matter (of array addition). If a lot of people missed that point on array addition, I think that it’s clear your concept in writing that point was not.

You similarly missed my main point about Ada: Ada is actually a fine language and had reasonably “packaging” semantics that would act a lot like object orientation if used properly. Ada failed primarily because people could get cheap and easy compilers for C++, but Ada was completely proprietary. That’s why I pointed out Turbo C++ specifically. Pascal lasted a lot longer than it should have, because of Turbo Pascal and later Delphi. Similarly, Java would not have made the impact that it did without javac having been freely available.

You suggested that academics have had to be convinced that OOP was more than a fad. Personally, I have yet to be convinced that it’s more than a fad. I don’t mean, by that, that OO will go away. It will lose its primal position in CS at some point as people realise that it’s just another technique for organising your code, and it doesn’t automatically solve your problems. I’ve had enough arguments with people who don’t know how to do object modelling to strongly believe that most people don’t understand OO at all. I suspect that people like Booch and Rumbaugh understand it less than most, given how loudly they have trumpeted it without consideration for reality.

Let me make it clear: your impression that Ruby isn’t a silver bullet isn’t what’s embarrassing, in my opinion. Your arguments attempting to support your impression are muddled and sometimes simply incorrect.

You can say that you don’t think it’s all that radical. I’d agree with you. I’m not necessarily convinced that Ruby 1.8 is a good teaching language: there’s a few ugly, weird corner cases. Yet Chris Pine thought well enough that he wrote Learn to Program. Ruby will help people learn what it means to be intentional in their programming, and less rely on an IDE to write code for them (as is increasingly necessary with overwrought, overweight frameworks and languages like C#, Java, and to some degree C++). I have adapted my C++ code sytle to follow a very Ruby style in this—and it has really improved the readability and maintainability in the place where I work. So much so that there are people who now do the same things that I do.

For what it’s worth, I have little problems with academics, and I have significant problems with the asinine responses you received from members of the Ruby community. I do think that academics often forget that there is a real programming world out there while they’re teaching (having interviewed dozens of would-be programmers straight out of university with a CS degree but no knowledge)—but pushing for something bigger and better and smarter isn’t a bad thing, and academic researchers are necessary to push that forward. Non-researchers, though, seem to change very slowly and push ideas that, in practice, haven’t worked out very well in implementation—or, perhaps, were too early and need to be rediscovered again.

Static typing hasn’t proven to be all that useful—it catches a very small class of errors at compile-time while hiding much more serious errors (buffer overflows, for example) a run-time. (Ada wasn’t vulnerable to this, of course, because it put static typing into both compile- and run-time both.) Dynamic typing is much more likely to be useful in the long run, or some combination of it—either in Erlang’s “pattern matching” or in Haskell’s type inferencing.

It is obvious that Matthew Huntbach hasn’t done serious programming with Ruby or serious analysis of programs that are already in existence. Most of the vaporous commentary written (probably 80% of the article) would have been burned away by that simple exercise. Even if one were to accept the argument that [1, 2, 3] + [10, 20, 30] should be [11, 22, 33] instead of concatenation—an argument smashed as soon as one attempts to add heterogeneous arrays together, or arrays that don’t contain items that respond to #+—it would be trivial to add such a special case (and then abandon it because it does present such a special case to normal Array#+).

Mr Huntbach’s history is even questionable, with respect to Ada. Ada wasn’t accepted because it was something of a bastard language designed by committee (very similar to Pascal, but with amazingly strong type restrictions), not just because it was facing C++. Ada failed because Turbo C++ was cheap and, well, there was no Turbo Ada. It failed to capture the next generation of programmers because it wasn’t readily available the way that C and C++ were and introduced a number of significant development-time restrictions that made interacting with the rest of the operating system difficult. (It may not sound like it, but I actually like Ada, and I have ever since I took a class on it in early 1993. I still don’t program in it, although I have programmed in a descendant, PL/SQL.) If Java had not been available as a free download (as the JDK), no amount of computer science department gushing about it would have saved Java from the cheap and free C/C++ compilers available at the time. Ada was doomed because the whole world was changing (and not just to OO-style development) and it was stuck in a restrictive model.

The comparison of Ruby to Ada is a false one in any case: Ada was designed to solve a particular subset of programming problems (namely, those requiring exact specifications, e.g., military applications) that required a very strict specification and completely predictable behaviour. If a particular sensor can only return values between -128 and 127, then you can make a subtype of Integer restricted to those values—and you can be absolutely guaranteed that the values in variables of those types will never be outside of those ranges. So Ada wasn’t designed as a silver bullet; it was designed as a language where specifications, interfaces, and code very closely matched. Ruby, on the other hand, was designed to solve problems that Matz had. When he introduced it, other folks realized that it helped them solve problems that they had. And so it has grown from 1993 to 2007, where now literally thousands of developers realize that they are running into the limits of current programming languages. They’re writing tens of thousands of lines of code that don’t solve their problems, but provide scaffolding for their programming languages. Once those are written, they can usually write code to solve their problems. I believe that Matz has even said that if something came along that was better than Ruby and helped him solve his problems better, he’d use it. (At the very least, he’d take ideas from it to make Ruby even better, which he has shown himself willing to do many times over.)

A lot of Bitwise’s articles seem to lack editing. Mr Huntbach’s article was, at best, stream of consciousness, and had the feel of a blog posting, not an article in a magazine. If you’re going to call yourself a magazine, invest in editing resources. Ruby Code & Style, the Artima Ruby magazine, is going through some transitions and will be experimenting with a CodeProject-style system, where articles can go live without editing, but they will still be edited by a team of volunteers, and will be marked unedited until they are well-considered. Editing would raise questions about why easily refuted examples are chosen, fix errors with historical inaccuracies, and prevent articles as embarrassing as this one from being published under the magazine’s banner in the first place.

Yet, that’s not the only problems. As a long-time Rubyist, I’m actively embarrassed to read many of the comments on Mr Huntbach’s article. There are a few well-informed comments on the article, but the vast majority of them are painful to read and unnecessarily insulting. Mr Huntbach needs to be better informed, and his article needs editing, but there is no to be as insulting as some of the comments were.

Edit: I was alerted to this article through a post on RedHanded. I find Why the Lucky Stiff’s response to be useful, but there’s a lot of the same attitude that I didn’t like seeing in the Bitwise comments on RedHanded’s comments. Remember MINASWAN, folks. Regardless of what you think of Mr Huntbach’s credentials or where he teaches, it gains nothing to slag them; help him understand where he’s wrong and we’ll be that much better for it.

Transaction::Simple provides a generic way to add active transaction support to objects. The transaction methods added by this module will work with most objects, excluding those that cannot be Marshal-ed (bindings, procedure objects, IO instances, or singleton objects).

I’m going to need to look at hoe a little closer to provide a patch that will support the way that I need to use it (hoe’s default behaviour is pretty good, if not great for greenfield development; I have legacy behaviours I need to support). If Ryan doesn’t accept it (or at least the functionality I need, even if it’s in a different form), I may need to keep a private version around, because I had to do entirely too much manually even after I reorganized the project to match a lot of Ryan’s expectations.

I will get back to work on the Ruby PDF Tools soon, as I will also be updating this blog a bit more often. The nature of my posts will also be expanding to include more non-Ruby topics, but Ruby-related content will remain the majority of what I post here.

(I’m deep in the middle of wedding planning, so more on the Transaction::Simple stuff later.)

I gave a quick introduction to the Google Summer of Code process and what it did for Ruby. The introduction was an adaptation of the talk that I gave at LRUG in July. The numbers: 17 volunteers; 96 applications; 84 eligible; ~25 desired; 10 accepted; 7 or 8 completed (I know we had 8 at the beginning of July; I think one more dropped out in the interim).

Greg Brown presented about his experience of doing Ruport for the Summer of Code. He was mentored by David Pollak.

Jeff Hughes talked about porting Ruby to Symbian phones. He was mentored by Dibya Prikash.

Jason Morrison spoke on Ruby Type Inference & Code Completion for RDT. The approach he used was naïve, but based on DDP by Lex Spoon (S. Alexander Spoon), which is Demand-Driven Analysis with Goal Pruning. Type flow analysis; it unions types over contours. He was mentored by Chris Williams.

That ended RubyConf 2006. I’m looking forward to next year, and I’m campaigning very hard to have it in Toronto.

While waiting for Sasada Koichi to set up, I suggested to Matz that perhaps something like RubyInline be included with the Ruby 1.9/2.0 release so that Transaction::Simple can dump singleton objects and the marshal format would be compatible between the various interpreters.

Sasada Koichi presented on the current state of YARV. He has recently gotten a job in Akihabara Sanctuary. The advances of YARV look impressive. He was running a Rails application on YARV with no problems. Nice.

John Lam reported on his Ruby/CLR bridge, which was started because he writes programs for his son’s birthday. Fascinating work, and amazing enough that Microsoft finally twisted his arms well enough for him to work on them to improve the CLR so that it better supports dynamic language. As someone—possibly James Gray—suggested at lunch: we’ve heard from three major platform vendors that they are taking Ruby very seriously. That rocks.

• Individuals and interactions over processes and tools: the language design should focus on users.
• Working software over comprehensive documentation: the language should encourage readability.
• Customer collaboration over contract negotiation: the language should be expressive, and helps communication between developers.
• Responding to change over following a plan: the language should embrace changes, and be dynamic.

Looking at this, matz concludes that Ruby is an Agile Language.

Matz noted that Ruby has Good things (it’s a sweet language, Rails, the community—as Martin Fowler says, Ruby people are nice); Ugly things (eval.c, parse.y); and Bad things (Ruby 2 being vapourware for such a long time: it’s close to being the longest vapourware in open source—Rite the concept is older than Parrot and Perl 6).

The Bikeshed represents an easy problem. People tend to argue about little things that they know enough about to do so, such as what colour a bikeshed should be. The amount of argument caused by a change is inversely proportional to the size of the change. Ruby has several bikesheds: the discussions on String and Symbol; the possible removal of private and protected; whether Ruby needs (optional) static typing.

On the other hand, nuclear plants are complex and important, so we tend to leave discussing them to the experts. So we spend most of our time discussing relatively unimportant things, leaving important things yet to be discussed.

Some consider Ruby a fragile language, but Ruby 1.8 is generally good enough. This means that although Matz would like to get Ruby 2.0 out, we’re not in a hurry and each idea has its own value which must be discussed. Instead of stopping bikeshed arguments, matz says we should accelerate them: Extreme Arguing. If arguing is good, we should make things easy enough to be argued by anyone.

The RCR process hasn’t worked out as well as matz wanted it to. Some people didn’t take RCRs very seriously; some took RCRs far too seriously. Thus, matz is introducing what he calls the Design Game. The purpose of the Design Game is to open language design to everyone in an accessible manner. It will:

• Gather Wild & Weird Ideas
• Try to make Ruby the Best Language Ever
• Shed light to undefined corners of Ruby
• Finally (if possible), document Ruby specification.

There are some fundamental rules of the Design Game:

1. Ruby will stay Ruby. We’re not creating a new language or a new Smalltalk or a new Lisp. About 80% – 90% compatibility will be preserved, if not more. Each proposal must follow the same philosophy we love about Ruby.
2. Design Game proposals must not be vague ideas. It’s hard to impossible to start useful discussions with vague ideas.
3. Design Game proposals must have rationale and analysis. Entries that don’t have a rationale and analysis section will be rejected out of hand. Matz will probably use a format similar to the Python Enhancement Proposal (PEP).
4. Discussion will happen on one or more mailing lists, possibly a single mailing list per proposal. RCRchive will probably be a starting point, but there may be a new system to control traffic: possibly qwik or trac.
5. Proposals with a prototype implementation will be favourably smiled upon as concrete code helps a fruitful discussion.
6. matz is still the Benevolent Dictator…but promises to be as open as possible.

Matz is doing this because he wants to share the fun of language design among the community and is tired of the slow evolution of Ruby (despite him being the bottleneck). Most of us are using technology from three years ago and if we (Ruby) don’t accelerate, others will catch up. This is also to help educate developers in the community: language design shares much in common with other software design. Additionally, matz wants the process to be continuable if he were to be hit by a truck (heaven forfend).

Matz may or may not set a deadline for the Design Game and has tentatively considered 2007-04-30 as such a date. After that, we (the community) will classify proposals as either Good, Bad, or Ugly and as targeted for 1.9 or 2.0. The good proposals will be implemented, and if they are ready, they will be merged. If the game doesn’t work, it’s not a problem: we’ll try something else, we’ve lost nothing but time.

Matz is still planning on releasing a stable version of Ruby 1.9 (1.9.1) for Christmas 2007 with YARV and other changes to come.

matznote slides

Important notes:

• The Game will apply to changes to core and standard libraries, but core is preferred.
• They will be keeping up with Windows changes as Vista came.
• Not necessarily for enterprisey stuff.
• Tests should probably included with proposals, too.
• The parser could theoretically be replaced in a non-backwards-compatible form.

Laurent Sansonetti’s demonstration on OS X integration with Ruby was very well done, with excellent demonstrations of controlling iTunes with Ruby from irb and with a Ruby/Cocoa GUI. It’s nice to see Apple committing at least Laurent’s time to Ruby support. It’d be nice to see Microsoft committing resources, too.

My headache was too bad for me to attend Glenn Vandenburg’s talk about Rinda in the real world, which is too bad, because the parts of it that I caught seemed really interesting. Lunch, and then I came back half-way through the lightning talks. Very interesting stuff.

Rich Kilmer presented because Jim Weirich couldn’t be here this year, and he presented about the Indi service that he and Tom have been working on. Very interesting, and I think it could be an interesting service. I’ll probably play with it soon. (Or at least as he makes it available.)

Tim Bray presented on I18N and M17N as they relate to Ruby. The most extensive effort so far for identifying characters in human writing forms is Unicode; the Unicode Standard 5.0 is soon to be released. Unicode 5.0 and ISO 10646 are identical. Unicode characters are represented in 17 planes; the first plane is called the Basic Multilingual Plane. There’s only 9% usage in the total plane (1,114,112 available). Tim’s presentation was mostly a survey over what various languages do—and what Ruby should do. I had a chat with him after his presentation to clarify a few things, and I think we’re mostly in agreement. One interesting point: when Tim asked the audience if they understood Unicode, only about FIVE of us raised our hands (myself included). Tim recommends reading The World’s Writing Systems by Peter T. Daniels, the W3C’s Character Model for the WWW 1.0: Fundamentals and the upcoming Unicode Standard, Version 5.0.

Michael Granger presented the Linguistics package. Really neat. When it was done, I offered him the Text::Hyphen package for multilingual hyphenation. I need someone to pick it up and maintain it as I no longer have time to maintain most of the projects that I work on. He seemed interested, but I’ll catch him tomorrow and talk with him in more detail—or I’ll try to catch him after the conference by email.

Dinner was acceptable (barely; the vegetarian dish was the same thing that they had served for lunch on Day 1), but I met Kirill Sheynkman who may end up joining the PDF::Writer project in the near future to work on things that I don’t have time to work on, freeing me up from having to worry about certain maintenance issues so I can work on the next generation that includes reading. After that, the matznote.

The matz Roundtable was pretty short; not too many questions were asked this year, and the discussion didn’t continue for an hour as it did the year before. I was shot down when asking for “become” behaviour (related to the Transaction::Simple bug). After the Roundtable, I managed to snag matz to talk about the problem which led me to request this. I showed him the test case:

#!/usr/local/bin/ruby
require 'rubygems'
require 'transaction/simple'

class Child
  attr_accessor :parent
end

class Parent
  include Transaction::Simple

  attr_reader :children
  def initialize
    @children = []
  end

  def < <(child)
    child.parent = self
    @children << child
  end
end

parent = Parent.new
puts "parent.object_id: #{parent.object_id}"
parent << Child.new
puts "parent.children[0].parent.object_id: #{parent.children[0].parent.object_id}"
puts "starting transaction"
parent.start_transaction
parent << Child.new
puts "parent.children[1].parent.object_id: #{parent.children[1].parent.object_id}"
puts "aborting transaction"
parent.abort_transaction
puts "aborted transaction"
puts "parent.object_id: #{parent.object_id}"
puts "parent.children[0].parent.object_id: #{parent.children[0].parent.object_id}"
parent << Child.new
puts "parent.children[1].parent.object_id: #{parent.children[1].parent.object_id}"

producing the output:

parent.object_id: 3265800
parent.children[0].parent.object_id: 3265800
starting transaction
parent.children[1].parent.object_id: 3265800
aborting transaction
aborted transaction
parent.object_id: 3265800
parent.children[0].parent.object_id: 3265500
parent.children[1].parent.object_id: 3265800

This bug affects PDF::Writer’s table generation and contributes significantly to the high memory usage. What’s happening is that when you call Parent#start_transaction, Transaction::Simple creates a transaction checkpoint with Marshal::dump. When you call Parent#rewind_transaction or or Parent#abort_transaction, the transaction checkpoint is reverted. This reversion is extremely robust except for this one item. What we really need is something like:

self = Marshal::restore(checkpoint)

Obviously, that won’t work and this leads to the problem that is illustrated above. After long discussion with Tim Pease, Patrick Hurley, and Matz, we came up with a workaround that can work for the example bug and for PDF::Writer. It’s not super-efficient, though. Essentially, I will modify Transaction::Simple to have callback methods for post-processing after a transactional operation. Something like this:

class Parent
  def post_restore_hook
    @children.map! { |child|
      child.parent = self unless self.object_id == child.parent.object_id
      child
    }
  end
end

parent = Parent.new
puts "parent.object_id: #{parent.object_id}"
parent < < Child.new
puts "parent.children[0].parent.object_id: #{parent.children[0].parent.object_id}"
puts "starting transaction"
parent.start_transaction
parent << Child.new
puts "parent.children[1].parent.object_id: #{parent.children[1].parent.object_id}"
puts "aborting transaction"
parent.abort_transaction
parent.post_restore_hook # would be called automatically in the real case
puts "aborted transaction"
puts "parent.object_id: #{parent.object_id}"
puts "parent.children[0].parent.object_id: #{parent.children[0].parent.object_id}"
parent << Child.new
puts "parent.children[1].parent.object_id: #{parent.children[1].parent.object_id}"

Which produces the output:

parent = Parent.new
puts "parent.object_id: #{parent.object_id}"
parent < < Child.new
puts "parent.children[0].parent.object_id: #{parent.children[0].parent.object_id}"
puts "starting transaction"
parent.start_transaction
parent << Child.new
puts "parent.children[1].parent.object_id: #{parent.children[1].parent.object_id}"
puts "aborting transaction"
parent.abort_transaction
parent.post_restore_hook
puts "aborted transaction"
puts "parent.object_id: #{parent.object_id}"
puts "parent.children[0].parent.object_id: #{parent.children[0].parent.object_id}"
parent << Child.new
puts "parent.children[1].parent.object_id: #{parent.children[1].parent.object_id}"

This isn't great: it doesn't feel very Ruby to me, but it does get the job done. It's also not very efficient. After thinking about this for the better part of an hour, matz has suggested that there might be a very ugly hack that’s possible that he’ll look at for me, which may be able to implement everything in Transaction::Simple. ]]> http://www.halostatue.ca/2006/10/22/ruby-conference-2006-day-1-evening-friday-20-october-2006/feed/ 7 http://www.halostatue.ca/2006/10/20/36/ http://www.halostatue.ca/2006/10/20/36/#comments Sat, 21 Oct 2006 01:44:52 +0000 austin http://www.halostatue.ca/2006/10/20/36/ The Embassy Suites is a relatively nice hotel, although the air is far too dry for me to comfortably to sleep in. The breakfast area this morning was very nice; they make omelettes to your taste.

We were told that Bil Kleb and NASA have provided power cords throughout the room and we shouldn’t be daisy-chaining if we want to continue having power in the room. Josh Susser has the flu and so will not be presenting tomorrow (“More than enough rope to hang yourself”) and his talk will be replaced with nine five-minute lightning talks. I made an announcement that I’m in discussions with the VS developers at Microsoft and am trying to get real examples of problems with Win32 extensions. I’ll post a summary of the discussion so far to my blog a little later. When the discussion has been held, I’ll post a summary of what we got from it. Ryan Davis re-announced the RejectConf scheduled for after matz’s keynote. Wireless access has been spotty all day; the Embassy Suites isn’t much ready for a room of 300 people, all with computers that they want to get on wireless.

The first talk of the morning is Masayoshi Takahashi about the history of Ruby. As should be expected from the inventor of the Takahashi presentation method, he’s an excellent presenter. His talk was highly entertaining and informative. Most people know that there was no Ruby conference in Japan prior to 2006, although there were conferences that included Ruby (the LL day/weekend conferences, culminating in this year’s LL Ring; also the YAPRC (Yet Another Perl and Ruby Conference) events for a few years. The reason for this? There wasn’t enough passion in the Japanese Ruby community to spur the planning of such a conference. There were so many books published for Ruby in Japan that there was a bubble that burst in 2003.

The second talk was Evan Phoenix’s (née Webb) discussion of Sydney and Rubinius. This is a fascinating project that may provide some direction in the future, but is something Evan will be working on over the next while and will be worth watching as it possibly becomes another Ruby interpreter.

After lunch, Geoffrey Grosenbach gave his talk about the various dynamic graphic libraries in Ruby with demonstrations of why one would use graphical representation of data with example Ruby code on how to generate many of them. There’s definitely a lot of things to consider for future projects. I may be pulling some of this into PDF::Writer (both SVG and PNG generation will be directly useful) when I finally get back to working on that.

Kevin Clark’s presentation about mkrf (a Rake-based replacement for mkmf) was fascinating, and I think that there’s a possibility that it could be a very useful thing in the future, especially if dist-utils style capabilities are added in the near future, increasing the ability to use alternative compilers. I think it’ll be a little while before mkrf is really production-ready. Chad asked Kevin to write some code for RubyGems to help look for external capabilities (such as the presence of the MySQL library); I’d have some concern about this working well on Windows because mkrf doesn’t yet really have good Windows support, but I think it’s very important to add.

Zed Shaw gave an interesting presentation about security testing with fuzzing and and some statistical analysis. If you’re doing anything with anything that has to do security checking, you really want to read his slides when he’s posted them—the concepts he presents are good for that. By the way: if you’re writing software that people use, you’re writing something that needs security checking.

Finally, John Long talked about Radiant, the Rails-based CMS which now runs ruby-lang.org. It seems like it could turn out to be a very interesting CMS in the future, but it has a ways to go now.

Shortly after arriving, Jim Freeze, Bil Kleb, Hal Fulton, Chris Lehman, and Alan Whitaker and I went to Casa Bonita: a Mexican restaurant with third-rate food (especially the vegetarian food; Velveeta is not Mexican) and second-rate entertainment, including a sketch artist, a too-loud Mariachi band, and a gal who did high dives into a pool not too far from our table. I’m not sure I’d go back to it, but it was well worth the visit, especially with the folks who were there. We had a really good time and had some interesting discussions. RubyConf is off to a great start.

As I said back in June, the primary concern that most Ruby developers have is the difficulty in compiling extensions on Windows, especially those developers who have extensive background with UNIX-like environments, where you can easily do:

./configure && make && make test && sudo make install

or

ruby extconf.rb && make && make install

or

gem install <name-of-extension-gem>

Visual Studio 2005 (VS8) does not make this easier because of the choice to require manifests for the unmanaged assemblies with SxS installation. Much of this can be automated, but some common tools are completely missing from the VS8 tool chain that make building harder than it needs to be. To be fair, there are tools missing from the Ruby environment that make building extensions harder than it needs to be.

Ruby needs tools similar to the Python distutils, but this is something that the Ruby community must provide, not Microsoft. So far, though, no one has stepped forward to create such a beast, although tools like mkrf from this year’s Google Summer of Code could be used to improve this situation considerably.

This does not mean that Microsoft has nothing that it can do to help here. If Microsoft wishes to win the Ruby community toward using VS8 instead of continuing to use VC6 or stepping sideways toward the MinGW/MSYS combination, it needs to offer positive steps toward increased compatibility even between Microsoft’s tools and offer some way of making it easier to build tools that are primarily developed on UNIX-like platforms for Windows.

Let me be clear: I consider using MinGW/MSYS a step backwards (not sideways) for the Ruby community to take, because it goes to a known-buggy runtime (VC6’s MSVCRT.DLL) and will have a harder time interfacing with Microsoft’s modern technologies moving forward. Not everyone considers this a problem; they need Windows versions of tools that they have been using on UNIX-like environments more than they need the advanced capabilities offered by the later runtimes. This is a compelling argument, but I do not find it wholly convincing.

There is a vocal contingent of Ruby developers that would like to be able to have a version of the Ruby One-Click Installer that includes a compiler as part of its distribution, possibly as a separate download. This is something that would matter significantly for people who use Ryan Davis’s RubyInline or ZenOptimize, and other development environments. Again, I don’t find this compelling (and would prefer not having such a thing as a single distribution package because it would be unnecessarily large). It is a significant concept, though, because the one thing that the Microsoft tool chain has over MinGW/MSYS is that it is easier to install and get running, even though it’s significantly harder to build extensions with.

What Do We Need From Microsoft?

This question is the source of considerable debate and anger. There is a level of mistrust of Microsoft in the Ruby community (especially the Japanese community) based on missteps by the Microsoft developer community with regards to backwards compatibility. For this, I’m going to quote Usa Nakamura, who builds the Ruby binaries that Curt uses for the Ruby One-Click Installer for 1.8.4 and 1.8.5 with some clarification by myself and URABE Shyouhei. The original messages were provided in response to a post that I made on the 26^th of June, 2006 if they are needed.

The problem of errno is not serious, I think. We will be able to avoid the problem with some simple code (for example, replace it with a function call by macro.) The real problems [arise from separate resource management per runtime DLL version, keeping independent file descriptor tables, memory management, etc., that are internally maintained and not open through any API].

The decision [by Microsoft to break] binary compatibility between versions of runtime […] is foolish. [Did they think] that passing file descriptors between DLLs [would not happen]? As time goes by, there will be some [need] to introduce such incompatibility, I know. […] VC7, VC7.1, and VC8 were shipped [in rapid succession], and they are mutually incompatible[…]. It’s crazy, to say the least.

I decided to [stay] with VC6 for the above-mentioned reasons. If MS keeps shipping incompatible versions at each upgrade, I will throw away VC and shift to MinGW. If MinGW also follows [MS’s binary incompatibility], I’ll shift to Cygwin or throw away Windows as development environment.

[I] hope [that] Microsoft [could provide a] wrapper DLL [for] MSVCRT80.DLL named MSVCRT.DLL. If MS prepares such a mechanism, [there will be] binary compatibility[…]. If so, [it wouldn’t matter whethera program links against MSVCRT80.DLL or MSVCRT.DLL].

It’s bad enough where we have the source but prefer using pre-compiled binaries either because of time, trustworthiness, or compilation complexity. Ara Howard notes that compiling the GNU Scientific Library (GSL) on Windows is so complex without Cygwin or MinGW that there is a company that charges $600 for a compiled version of the source. Others have noted similar issues with other code.

It’s worse when we have third-party binary DLLs compiled with an earlier or later version of Microsoft tools. Ruby is receiving a lot of attention now because of Rails, and I suspect that Microsoft would love to have people using SQL Server as their database underneath Rails, but if there are problems compiling the binary extensions to link against the SQL Server interface drivers, this is less likely to happen or be common. Oracle drivers present a similar problem. We don’t have the source to recompile with the tool that we prefer, and the binary DLL we must use may have been compiled with an earlier incompatible version of the compiler.

(Let me digress for a moment and make it clear that this is an overall problem on all platforms, not just Windows; I have experienced it most often on Linux in my day job where I do C/C++ development. It is far more apparent on Windows, however, because it is unusual for there to be more than one C/C++ runtime on a UNIX-like system. Not impossible, but unusual.)

Thus, the first thing that we feel that would be beneficial from Microsoft would be some sort of runtime shim or wrapper that would allow us to use programs built with VC6, VC7, VC7.1, VC8 or even MinGW compatibly.

The second thing that we feel that would be beneficial from Microsoft would be a better command-line tool chain that is preferably compatible with many of the UNIX-style build commands. A minimal start for this would be command-wrappers that allow you to use a gcc/g++ front-end that actually calls VS8’s cl.exe with the appropriate command-line parameters. Microsoft’s move away from command-line support and toward devenv as the primary build environment helps most Windows developers, but hurts those who do their primary development on platforms other than Windows.

The third thing reaches into core functionality problems that Nobu Nakada posted about in late July:

• more unified I/O
  • nonblocking flag can be set on only sockets.
  • WaitForMultipleObjects doesn’t work with sockets.
  • APIs to tell how many bytes can be read immediately differfor ordinary file, console, socket and pipe.
• pseudo tty: pty feature is not supported at all now.

Charlie Savage had some interesting things to say in a lengthy discussion in July while I was in Europe with limited Internet access:

From my experience using both tool chains on Windows (for the ruby-prof extension and SWIG-based extensions for GEOS and GDAL).

• You can build Ruby extensions using MingW that run against Ruby built with VC++. I’ve done this with Ruby 1.8.2/1.8.4, various MingW releases and VC++ 2003 and VC++ 2005. This used to require changing a small bug in ruby.h for Ruby 1.8.2, but that bug has been fixed with 1.8.4.[…]

• However, you cannot do this with MingW using VC++ built Ruby.

  ruby extconf.rb
  make
  make install

  The problem is that extconf is quite limited – it will assume you are building your extension with the same compiler that built Ruby (VC++). Python avoids this issue because disutils will recognize the compiler being used (MingW, VC++) for the extension and provide the correct command line parameters.

• If mkrf can work like Python distutils, then it will become simple to use MingW to build extensions that work with VC++
• When compiling with MingW do not link against the ruby *.lib files. Instead, just link directly against the DLL (msvcrt-ruby18.dll). It’s faster (links much faster) and works better.
• So you need to manually compile your extension or create a makefile to do it. This actually turns out be the way GEOS and GDAL work – they have autoconf based build systems so extconf.rb wouldn’t fit in anyway.
• The advantage of MingW is that it avoids the unmanaged assemblies that VC++8 uses, so its simpler to deal with[…]
• VC++ has several large advantages on Windows.
  • First, it lets you debug your extensions while GDB does not support this on Windows (or if it does, its never worked for me).
  • Second, it compiles much faster
  • Third, there is a lot more help available.
  • [Fourth], its quite easy to build Ruby extensions.
• Using MingW on Windows is a huge barrier to entry. Gettting MingW setup, along with msys, is a time consuming process that only experienced *Nix developers will understand and be able to do.
• MingW on Windows is not very easy to use. It’s nice to think that you can download an open source project, type ./configure, make, make install and it will work. Alas, it doesn’t really work that way. There are myriad of issues you run into. First you’ll need msys. Then many projects have prequisites that you’ll have to download and compile. In addition, you often times have to change the CFLAGS and LDFLAGS to get successful compiles. Linking is a pain and requires hand-holding, and sometime just doesn’t work. Libtool is really flakey on Windows. For some projects, you’ll have to need to download/build/install the latest version of it. You also need to get autoconf/automake installed. Many projects require bison – something I’ve never been able to successfully compile on Windows. All in all – it literally took me weeks to figure out how to get everything to work together. The MingW/msys tool chain is quite complex on Windows, and most people won’t have the time or desire to put forth the effort to get it to work.

My recommendation:

• Use VC++ 2005 and get Microsoft to tell us how to properly use unmanaged assemblies so that we can avoid dll hell
• Make sure that mkrf supports building Ruby extensions “out-of-the-box” on Windows using MingW if you have it installed. I think this would be the best of both worlds – you support both tool chains. VC++ is the default one, but MingW should work fine for building extensions.

Hope this helps – I’d be glad to share more of my experiences if its helpful.

So, what can we do to move Ruby toward a highly usable environment that is based on modern Microsoft compiler technology yet remains backwards compatibility with the tools that for many reasons we cannot give up?

After releasing Ducktator, Ola Bini was surprised at the responses that were received toward his use of the term “duck typing.” The discussion went on for a while, and Ola eventually decided to make a post to his blog about it. I think that the library is cutely named (as others on the ruby-talk mailing list, I like puns) and several people have suggested that it’s going to be very useful for them. However, I don’t think that Ducktator is about duck typing, and its name is unfortunate in that sense, because it has the chance of confusing the discussion surrounding duck typing even further.

What is Duck Typing?

The simplistic answer to this, of course is, “if it walks like a duck, and
it quacks like a duck, then it must be a duck.” This is certainly true, but the
real question is more “how do I know if I’m duck typing?” Many people, myself
included, consider duck typing a matter of trusting one’s callers and
documenting your API well enough to ensure that the callers can trust that you
won’t do something they’re not expecting. The canonical example of this is a
logger.

class SimpleLogger
  def initialize(recipient = nil)
	@out = recipient || $stderr
  end

  def log(message)
	@out << "#{message}\n"
  end

  attr_reader :out
end

l = SimpleLogger.new([])
l.log "Hello"

SimpleLogger simply trusts its users to give it recipient classes that respond to #<<. Now, this is a simple case, but what about something a little more complex? Well, Text::Format can accept a hyphenator object, which must implement a particular method that has a particular arity. So when you assign a hyphenator object, Text::Format does object signature validation for both the presence of the #hyphenate method and its arity (two, I believe). I consider Text::Format’s approach less duck-typed (possibly even not duck-typed, although it is more dynamic than class-based validation) than I consider the SimpleLogger class above. Both are useful techniques, but to Text::Format is less flexible than SimpleLogger.

That’s OK, though. The increased complexity of the API for hyphenation suggests that validation is not a bad thing. When you get into the realms of problems that Ola wrote Ducktator for, you need even more complex — and arguably less flexible — validation. As I said yesterday:

As Eric Mahurin noted in a spin-off post from the main thread about Ducktator, there’s a number of possible definitions for duck typing, so it’s understandable that people find themselves confused. I’m not Dave Thomas (he and Andy applied the term first to Ruby, as far as I can tell), but as far as I’m concerned, object signature validation is not duck typing. It never has been, and it never will be. It’s object signature validation.

I don’t care whether you do object signature validation by the class of the object (in which case you’d usually be unnecessarily restricting yourself in Ruby) or by the actual method or methods you need, you’re still not doing duck typing.

I still believe this to be true. By all means, use Ducktator if it’s going to help you. It isn’t what I consider to be duck-typing, though.

We hardly ever max out [CPUs]. … The messaging middleware app handles millions of messages per day and rarely takes up 10% of a single CPU on the servers it runs on.

Most of the CPU time used by our Ruby apps is spend waiting for IO. … about 80% of the time [for the middleware] is spent in the kernel in read() or write() syscalls.…

Ruby doesn’t have to be the absolute fastest at everything. Until you’ve tried it or not, though, you won’t even know if it’s fast enough for your problem. It might be—or it might not be. I’ve got some code that I’ve been hacking at work for our build process. I should have just made the programs (a collection of shell scripts) involved with Ruby. It would have taken me less time, left me something more manageable overall, and be more capable in the future. The downside? I would have had to make sure that Ruby is installed on all of my machines, which isn’t necessarily easy (I have some AIX boxes in the mix). Unfortunately, by time I really reached that conclusion, I had already spent enough time to make it non-economical to switch for at least another year (the programs don’t need to be maintained that often). I guarantee you, though, that the Ruby code would have performed just fine and it would have been easier for people to add features to than the shell scripts that I have.

I for one am scared of Ruby because (1) it displays a stunning antipathy towards Unicode and (2) it’s known to be slow, so if you become The Next MySpace, you’ll be buying 5 times as many boxes as the .NET guy down the hall.

They’re wrong enough that the creator of the Ruby, Matz, has said something about it on the ruby-talk mailing list:

[W]e disagree in the middle.[…]

(1) Although we took different path to handle m17n issue from other Unicode centric languages, we don’t have any “stunning antipathy”.

(2) Although Ruby runs slower than other languages in those micro-benchmarks, real bottleneck in the most application lies in either network connection or databases. So we don’t have to buy 5 times as many boxes.

What can be said about Ruby and Unicode right now is that it shows apathy toward Unicode. It neither helps you nor hinders you; it leaves you in about the same place as if you were programming C or C++ without using ICU or something similar. The future is something different. I’ve done enough Unicode and m17n work to know that while Unicode is a great choice for new data, it doesn’t help you much with your legacy data—and all the languages which have chosen to be Unicode to the core have trouble with legacy-encoded data to some degree. Ruby’s m17n support is being planned to be legacy-encoding aware. That way, whatever encoding you have to deal with, it should properly be handled in Strings.

Further, Ruby may be “Known To Be Slow”, but what people Know is bunk. There’s long discussions about this on ruby-talk. There’s clear examples where changing one’s algorithm results in a dramatic performance improvement—to the point where people don’t care that the program is written in Ruby. Can Ruby’s performance improve? You bet it can. But anyone who tries to tell you that Ruby is Known To Be Slow is simply repeating outdated and inaccurate information and is not speaking from actual experience. Whether Ruby is too slow depends on what you are trying to do.

That said, I don’t use Ruby exclusively. I’m not paid to. Even if I were able to use it as an option, I know that Ruby is too slow for the specific domain problems that I have to solve at work. But for other problems (including some picture sorting that I’ve had to do at home and have written an article about), Ruby is as fast or faster than other tools.

Alex was great. The program was great. The result he’s produced is great—and I’m not the only one that thinks so. Justin Baily said:

Very impressive library! I remember when you posted about this at the beginning of the summer. I took the library and pointed it at the USCCBs online version of the Bible and got some very impressive results! It was able to identify book, chapter and verse with only[…]3 sample pages[…] [ruby-talk:210002]

My biggest regret is that I have been so insanely busy that I don’t think that I’ve been able to give Alex the amount of time he deserved as a mentor. I only hope that the bits I was able to give him truly helped and didn’t just come across as koans because I was too busy to do much more than I did.

But Alex created something useful, and something that I believe he will continue to work on—ultimately I hope that he will be able to attract others to help him build on ARIEL. It’s just a very cool technology that I think will help people make sense of the static web in the transition toward a Semantic Web.

David Vallner:

David Vallner: In dynamically typed languages, the type of an object isn’t its “class” or any other predictable concept, it’s just the protocol it adheres to during its lifetime. Most such languages also make no presumption that this protocol, or the behaviour of an object remains the same during its lifetime – which makes compile-time checks rather pointless.

Of course, you rarely use even this aspect of dynamic typing – in fact, I can’t come up with a single noncontrived example for it, these things just don’t occur in daily coding. But dynamic languages sure a heck faster to type, and the fact the compiler rarely bitches at all is very, very appealing to people that know what they’re doing most of the time.

Francis Cianfrocca:

That was a perceptive comment. The fact that there are so few true cases of objects that metamorphose in flight also accounts for the observed “magical” behavior that “duck-typing” seems to “just work” almost all of the time. (By magical, I mean unexpected for people like me with too many years of experience programming in statically-typed languages.) The intention of a well thought-out code path is generally easy to express without typing all the types along the way. It doesn’t break often, and when it does break, it doesn’t make a big mess.

This matters because when I write an API, I write it with an interface (protocol, to use David’s term) in mind. That interface might have the name of a class, say File. But if someone gives me something that acts like the portions of File that I use,(and I use sensible alternatives when possible, such as #<< instead of #write, then someone can give me something entirely different and I won’t care. But I don’t restrict the consumer of my library to only using File, and I handle it as best as I can because I expect that things will work the way that they should.

Well, on the afternoon of the 27th of July for my only day in Amsteram, I checked my email and I had an email from the indomitable Erik Veenstra of RubyScript2Exe fame. The scheduling was a bit tight, but we ate at what I consider the best pannekoeken (a Dutch pancake that is the size of a small pizza and often topped similarly) restaurant in all of Amsterdam, the Pancake House. Like the other European Rubyists we met, Erik is a pleasant person with a great sense of humour and well-informed. My fiancée was charmed by all of them that she met (Hagen, Urban, and Erik) and we had a lot of fun. Anne-Marie and I were supposed to meet her sister at 9:30, but it was closer to 9:45 when we managed it because we were enjoying our discussion with Erik so much. Thanks for emailing, Erik. It was a lovely evening. Sorry it’s taken so long to write about it.

And a general thank-you to the European Rubyists that I met. I really enjoyed meeting you guys and hearing how you use Ruby. For those whom I didn’t meet, I plan on going back to Europe in a couple of years (I want to do a North American trip for holidays next year) and I hope to meet you then. Otherwise, I hope to see you at this October’s RubyConf.

Hagen has been using Ruby in his degree program and his practicum study for the transformation of business model descriptions to web sites, etc. He lives in Potsdam, just outside of Berlin, and we met him on the evening of Thursday the 13th after we picked up our rental car (an Opel Zafira). He showed us around the Sans Souci palace park built by the Austro-Hungarian emperor Frederick II. Absolutely beautiful and a worthy trip. He was very knowledgeable of his town and obviously likes it a lot. We made it back to the car just before the rain came down on us (after what had been a very hot—35 °C—and humid day in the Berlin area) and stepped into a nice Italian restaurant just as the deluge began. After a wonderful dinner, we drove Hagen back to his apartment and went on our way to Dresden and ultimately family in Lohr am Main.

We went to München on Monday, the 17th, and met up with Urban on the evening of the 18th. Urban saved my iPod from being useless on this trip by bringing his charger. (It had fully discharged and would only charge through my laptop; the USB-based charger that I had brought did not provide enough power for the iPod.) We met Urban near the fish fountain on Marienplatz where he was wearing his Munich.rb t-shirt and went to a nice Bayernisch restaurant east of Marienplatz. During dinner we learned that Urban used Ruby to process his raw data for his bio-informatics degree (equivalent to a master’s), although he was not asked that in his thesis defence.

My traveling companions are just computer users (not programmers at all!) and have pleasantly been pleasantly surprised by how nice, friendly, and well-rounded the Rubyists they have met have been.

There’s more than one way to do it. If you like baroque languages like Perl, maybe this is a feature. Ruby has 3 ways of constructing regexps, innumerable ways of producing strings, if and unless and… well. You get the idea. Reading Ruby is an exciting adventure in learning new ways to do the same old thing! Don’t like the old method name? Alias it! After all, languages should enable free expression in whatever manner best suits the developer’s personal style. “`Twas brillig, and the slithy toves…” and all that. Moderation is a virtue.

This is a conflation of several issues. Let’s tackle them independently.

• Multiple constructors for regular expressions: I am presuming here that Adam is referring to /re/, %r{re}, and Regexp.new("re"). Each one of these makes sense, but you only see two of them commonly present in most Ruby code. Most often, you’ll see the /re/ format, like /[ab].[d-f]/. This works well until you need to match on a slash character, like /\d{4}\/\d{2}\/\d{2}/. The extended format is much more readable in this case: %r{\d{4}/\d{2}/\d{2}}. Note that Ruby doesn’t have problems with the braces because they match here. Ruby is smart that way. The third form exists because regular expressions — like everything else — are objects. One could construct a regular expression from a string you get externally in this case, like str = "[ab].[d-f]"; Regexp.new(str).
• Multiple constructors for strings: Like regular expressions, there are good reasons for this, mostly related to quote escaping. With Ruby’s string interpolation ("foo#{bar}") it becomes useful to have a form of the string that does not interpolate: 'foo#{bar}'. (It is a minor quibble that there is no clean way to later take that string and interpolate it without calling Kernel#eval.) This means that it is useful to have string forms that make it easy to have quotes: %Q{"foo#{bar}"} and %q{'foo#{bar}'}. Finally, there is the HEREDOC format, which allows multiline strings without having to have line continuations or C++’s autojoin functionality (which wouldn’t work in Ruby because each line is an expression to be evaluated).
• Multiple method names for the same method: The biggest example of this is Enumerable#map and Enumerable#collect. These two methods do the same thing, but they are named such because Lisp-oriented developers understand #map implicitly; Smalltalk-oriented developers understand #collect. When I started with Ruby, I started with #collect; now I use #map. Other examples include Array#indexes and Array#indices. Grammatically speaking, #indices is correct, but #indexes is common, too. (The same applies to if/unless, although I only use unless as a modifier with a single test, most of the time.)

Perl-style two-character variables. Because dollar-sign-squiggle is such a mnemonic and meaningful name, and distinguishing $` from $’ is just a snap.

These are not used often; it is more common to see $LOAD_PATH instead of $: these days.

Other esoterica. The flip-flop operator [...]

This has been deprecated and I’ve never used it in any of my code.

Whitespace is significant. Didn’t this go out with JCL and punch cards? I guess Python has made it fashionable again. Anyhow, I can’t tell you how thrilled I am that
1 + 2 +<br /> 3
evaluates one way and
1 + 2<br /> + 3
evaluates another way.

The problem here is not Ruby. Ruby evaluates each line as an expression. This doesn’t mean that whitespace is significant in Ruby (it isn’t), but that the two examples given are two different expressions. What is a bug, and Matz has said will be fixed is a subtly different problem, is:

(1 + 2
 + 3)

The parentheses should be sufficient indicator of an expression.

Constants aren’t. You can assign to a constant multiple times. At least it will generate a warning.
Variable/Method Ambiguity. That’s what the pickaxe book calls it, anyway… Ruby guesses whether a name is a variable or a function based on whether it has seen an assignment to that symbol before the point of call. I guess I’m naive; I thought that was so amazingly lame that I was surprised it wasn’t classified as a bug.

Both of these are fully understandable and well-explained elsewhere. The latter is a consequence of what is often called “poetry mode” where parentheses aren’t required for method calls. Ruby does resolve this very well.

The method to_i doesn’t return errors.

Use Kernel#Integer if you need this. Most of the time, I don’t.

Handling namespaces and mixins with modules. Since these concepts have no particular overlap, this just feels very kludgey to me. Apparently the Ruby community is OK with it, though.

Actually, it’s not just modules. It’s classes. I can do either:

class A; end
class A::B; end
module C; end
class C::D; end

This, to me, is a lot more sane than yet another syntactical setup with no structure (package) or the way that C++ uses namespace.

Ultimately, I think that Ruby is what Lisp would be if it were a procedurally-oriented language. Nothing else certainly comes close.

I will be landing in London mid-morning on July 8^th and will be leaving for Berlin on July 11^th. I will be attending the The Who concert on the night of the 12^th, and picking up a car on the morning of the 13^th. I may be able to meet with people in Berlin on the evening of the 11^th or early afternoon on the 12^th depending on my schedule. We’re probably headed to Lohr am Main (near Frankfurt) immediately, and will definitely be travelling to Munich, but will be in a lot of different places. We will be heading toward the northern Netherlands on the 24^th or so and be in Amsterdam on the 27^th.

I will probably have time to meet with London Rubyists on either the 9^th or the 10^th. My German and Dutch plans are much more flexible (of necessity), but can probably carve out some time either way. If you’re interested in meeting with me, please email me (austin at halostatue dot ca) and we’ll see what we can get worked out.

Everyone enjoyed themselves, and the project was interesting, but we didn’t get nearly as much done as I wanted to. The original goal was to fully reimplement libarchive in pure Ruby: we did not achieve that. In no way, though, would I call the hackahon a failure except on that measure. Most everyone who attended learned more about Ruby and about how tar files are constructed. There was high interested in continuing the work from a number of attendees, some of whom have continued to contribute to the SVN depot.
I know a bit better what needs to be done to run a hackathon, and will be ready for the next event which has been suggested for August. I’m looking forward to it.

The Toronto Ruby User Group’s first hackathon is this Saturday, 6 May 2006. If you are attending, please add your name to the TRUGhat wiki page.

When and Where?

TRUGhat 2006:1 will be held on 6 May 2006, from 10:00 A.M. to 7:00 P.M. Tucows has generously provided the space and network availability for this event at its offices in Toronto, at 96 Mowat Avenue, one block westeast of Dufferin, south of King. Transit accessibility is good, with both the King West streetcar and the Dufferin bus passing near the office. Parking is plentiful for those who will be driving.

What Do I Need to Bring?

• Yourself.
• It would be good if you brought your own computer, although we will probably be practising pair programming.
• Anything you want to drink during the course of the day.

We will be providing wireless access, and the Tucows office has wireline access.

Pizza lunch will be provided from a nearby Pizza Pizza, courtesy of yours truly.

What else?

• Make sure you have Ruby installed. We will generally be working with Ruby 1.8.4.
• Have Subversion command-line or graphical tools installed. For people with Windows, I recommend TortoiseSVN.
• Create an account on RubyForge. You will be added as a committer to the project we will be using to manage this release.
• If you have any Ruby documentation, such as either the first or second-edition Pickaxe books, this will be very useful.

See you there!

If you are worried that you might get lost, please email me at halostatue(at)gmail.com and I will provide you with my cellphone number to contact me on the day of the hackathon.

So, a month ago, I put a call out for a developer for the Net::LDAP project. I had a couple of bites, but Francis Cianfrocca had the experience and drive that I was looking for as someone to ultimately hand the project to for development. Francis has exceeded my wildest dreams. He is a firm believer in the idea that API matters, and that a pure-Ruby implementation is of prime importance here.

He’s going as far as to suggest a pure-Ruby server implementation like WEBrick for LDAP details. More than that, he’s got a C++/Ruby library that he’s intent on making possible to use if it’s available on the user’s system, for performance reasons, but will still work well without it. If you use LDAP, or you think you might want to use LDAP, pay attention to this project. Francis is looking at the idea that people don’t get LDAP and trying to abstract the nastiness away. And that’s a good thing.

I wholeheartedly endorse the work that he’s doing now, and while there’s a long way to go, this is a project to watch.

TRUGhat is TRUG’s hackathon or codefest. In a hackathon, a group of people get together and will either work toward a common goal, or simply work in a common area on code that interests them. For our first TRUGhat, we have agreed upon a common goal.

During a codefest or hackathon, there’s always lots of snacks available, and it’s common for the perfect developer’s food—pizza, of course—to be delivered.

The goal is for TRUG to have two TRUGhat events a year. Ruby developers of all skill levels will be welcomed.

TRUGhat 2006:1

When?

TRUGhat 2006:1 will be held on 6 May 2006, from 10:00 A.M. to 7:00 P.M.

Since we’re all learning something on this first TRUGhat, we probably won’t get started programming until 11:00 A.M.

Where?

Tucows has generously provided the space and network availability for this event at its offices in Toronto, at 96 Mowat Avenue, one block west of Dufferin, south of King. Transit accessibility is good, with both the King West streetcar and the Dufferin bus passing near the office. Parking is plentiful for those who will be driving. Map

Who?

The Tucows space can handle a total of fifteen people, with thirteen places available. If you’re interested in joining us, please put your name below and join the mailing list. UPDATE: This list is wrong. Visit the TRUG wiki and use the sewm list to join.

What?

The current plan is to implement the functionality of libarchive and possibly libtar for Ruby. The primary goal is pure Ruby support with a secondary goal of Ruby/C bindings so that those who have the requisite libraries can take advantage of the C code for performance reasons.

More?

Some of the details are still being worked out, but Austin Ziegler has already committed to buying pizza for whomever does show up. Check out the mailing list or the TRUGhat wiki page for more information.

I make this recipe every couple of weeks it’s so good. Again, I made it for today’s pot-luck at work, and have had several requests for the recipe, so here it is.

1. Heat olive oil and sauté chopped onion for about 5 minutes on low heat.
2. Sauté sliced and quartered eggplant and zucchini on low-medium heat for 15 minutes, until both are soft.
3. Add water to vegetables and cover the pot. Cook, stirring regularly, for 15 minutes.
4. Stir raisins and cook for another three minutes.
5. Add curry paste (Patak’s is great) and cook five minutes further.
6. Serve hot by itself, with rice or naan. Optionally top with sour cream.

Thanks to Cooking for Engineers for highly functional the recipe format.

1. Preheat the oven to 350° F.
2. Melt the butter and mix it with the crushed ginger snaps.
3. Press the ginger snap crust into a 9” springform pan and bake for ten minutes at 350° F.
4. Soften the cream cheese and beat with the sugar, vanilla, and spices until smooth.
5. Beat pumpkin and eggs into cream-cheese mixture until smooth.
6. Pour cheesecake mixture into springform pan and bake for 55 minutes at 350° F, or until a toothpick inserted into the centre comes out smooth.

You can reduce the cracking of a cheesecake by putting a pan of water in the oven at the same time. This may be topped with caramelized sugar and pecans, as desired. ½ cup pecan pieces can also be mixed into the ginger snap crust.

Thanks to Cooking for Engineers for highly functional the recipe format.

Basically, all of the work that I mentioned in Ruwiki Deployment needed to be done, plus work to support authentication on RubyForge, while trying to establish a solid API that won’t need much fixing before the 1.0 release of Ruwiki. This also meant that some of the work planned for 0.9.0, including robot exclusion based on user ID (including the exclusion of certain robots entirely because they are email harvesters).

I hope to have Ruwiki 0.9.0 out by RubyConf, but I have my doubts as I haven’t even finished testing the deployment code in Ruwiki 0.8.1. It won’t be out much longer after that, though. Maybe even during RubyConf.

It wasn’t as if I weren’t going to release Minitar as a separate package; I said that I would, last week. I just realised that doing it earlier was more important than doing it later and getting Ruwiki out quickly. Doing this also helped me find an answer to the next phase of Ruwiki development, too (following the deployment release).

While looking around at options for how to present the UI for the Ruwiki deployment utility and the minitar utility, I looked at the RubyGems code that emulates the cvs command-line in many ways. This made a lot
of sense, to me, so I created CommandPattern:

class CommandPattern
  class AbstractCommandError < Exception; end
  class UnknownCommandError < RuntimeError; end
  class CommandAlreadyExists < RuntimeError; end

  class << self
    def add(command); ...; end
    def <<(command); ...; end
    attr_accessor :default
    def command?(command); ...; end
    def command(command); ...; end
    def [](cmd); ...; end
    def default_ioe; ...; end
  end

  def [](args, opts = {}, ioe = {}); ...; end
  def name; ...; end
  def call(args, opts = {}, ioe = {}); ...; end
  def help; ...; end
end

It seems to me that this is exactly the same pattern as I need to deal with for Actions in Ruwiki. Okay, so there are differences -- but not as many as I had initially thought. The amount of code to be reused, to be quite honest, is low, but the concept reuse is high. It's very cool, to say the least.

Follow-up

With the release of minitar, the examples originally given for deployment could become:

% minitar extract ruwiki-0.8.0.tar.gz
% cd ruwiki-0.8.0
% ruwiki_servlet

Ruwiki works extremely well under the “unpack and run” model that we’ve delivered it as so far. Unfortunately, when considering how to deliver Ruwiki as a RubyGem (as Mauricio and the RPA team will deliver Ruwiki as an RPA package), it was brought to my attention that the installation is in a central location, and this is not desirable: this means that you could only have one Wiki per machine¹ and that the data would be in a location that is less than ideal (under RubyGems, it would be under ${prefix}/rubygems/gems/ruwiki-<em>version</em>/data), making it hard to have a private Wiki on a public server. Originally, I thought this would be a trivial thing to fix, but it isn’t, especially when the goal is to keep Ruwiki as simple as possible for the end user. Solving this installation problem, however, seems to solve other possible deployment scenarios.

The first step is to make it so that one doesn’t even need to have the main Ruwiki programs in the same directory as the Wiki. Some of this has been enabled for a long time, but I’ve never really tested this configuration. I determined that it would be beneficial to have the ability to keep a configuration file separate from the Ruby code.

The Flatfiles tagged format (section!item:value) was ideal for this², and I generalised it to handle any two-level hash (e.g., { 'ruwiki' => { 'version' => 0.8.1 } }). This became Ruwiki::Exportable; I verified that my implementation was correct and complete by modifying Ruwiki::Backend::Flatfiles and Ruwiki::Page to use Ruwiki::Exportable, and then implemented it in Ruwiki::Config. This required one incompatible change: previously, storage options had been stored like @opts[:<em>type</em>][:<em>name</em>]. While the storage type is still referred to as a Symbol, the options for each storage type are now stored as strings. That is, what used to be @storage_options[:flatfiles][:data_path] is now @storage_options[:flatfiles]['data-path']. Note that the name has changed slightly, too.

The second step is initial deployment. This led me into a week-long diversion into Mauricio’s RPA::Package code, leading to the creation of my Tar module (based very heavily on Mauricio’s code, but also incorporating a number of changes); this will be released as a separate package with a limited tar/untar program at a later date. I now have the ability to pull Ruwiki data, templates, and executables from a central location. Because the feature was so easy to add, I have made it possible for users to package their wiki (into a .pkg file) and unpackage it elsewhere. I am completing the deployment command-line now. It used to be that one would do:

% tar xvfz ruwiki-0.8.0.tar.gz
% cd ruwiki-0.8.0
% ruwiki_servlet

Now, one will do:

% gem install ruwiki
% ruwiki install --to ruwiki # installs the data/ and templates/ to ruwiki/
% cd ruwiki
% ruwiki_servlet # pulls from the stub installed by RubyGems

The third and final step for this is packaging. My packaging requirements have become far more complex, so I have to automate the packaging process. When I release Ruwiki, now, I will need to check out the codebase, create a default configuration file (ruwiki_servlet --save-config) for the current release, create the default deployment package for the current release, build the gem with those files, and then create the .tar.gz distribution. I also need to PGP/GPG sign the released files.

Those who have looked at my releases will have noted that I touch all of my released files to the date and time of the release; this will actually be easier to deal with using an automated packaging process, but this is part of the process.

¹ Yes, you could copy the executable files (ruwiki.cgi or ruwiki_servlet) and modify those before running, but this is less than ideal for many things.

² I didn’t use YAML because of various problems with YAML across versions of Ruby, including parts which still do not work with Ruby 1.8.2 preview2 on Windows–and I am not in a position to test post-preview2 code at the moment. I didn’t use XML because that would add another dependency to Ruwiki. Finally, I didn’t use Marshal because I wanted the files to be human editable.

I chose the TC1000 over other versions available in Canada for several reasons. The Compaq is significantly (CDN $900) cheaper than either the Toshiba or Acer versions for similar feature-sets. With a 15% discount available at the time that I purchased it, this was an even better deal. Additionally, it was the Tablet PC form-factor I desired: a “convertible.”

All versions of the TC1000 are equipped with a Transmeta Crusoe 1GHz processor; 10.4” 1024×768 XGA display; 256Mb or 512Mb RAM; a 30Gb, 40Gb, or 60Gb HDD; and a one or three year warranty. All versions can be expanded to 768Mb RAM. As with all laptops, the more RAM present, the more effect on the battery life, but note that the Crusoe requires 24Mb of RAM for its Code Morphing, so more RAM will result in better performance in Windows. All versions except the cheapest come with a mini-PCI 802.11b card. It also has standard VGA, ethernet (RJ45), modem (RJ11), and audio (headphones, microphone, and headset) connectors. For expansion, it has two USB 2.0 ports, a CompactFlash slot, and a PC Card slot. Just for reference, I have the TC1000/470045-149 (30Gb, 256Mb, 802.11b).

The Physical Tablet

Like all Tablet PCs, the TC1000 screen can be oriented in any of four directions (two each portrait [768x1024] and landscape [1024x768]). When in primary portrait mode, the COMPAQ label is at the bottom of the PC. There is a microphone below the A and Q with stereo speakers on the bottom of the PC. On the lower left-hand corner are three LEDs for AC power, battery charging, and WLAN status (solid when a WLAN connection has been made; blinking when seeking). On the top left face of the tablet are three pen hotspots that default to Windows Journal, Entry Panel, and Rotate Screen Orientation. These are backed with LEDs which light up when the hotspot is selected with the pen.

Above these hotspots is a plastic cover covering the USB ports, ethernet and phone connectors, and video connector. The AC connection is to the left of the cover; the PC Card and CF slots are to the right of the cover. The pen is stored in a spring-loaded slot to the right of the card slots. On the top right corner of the tablet is the power button, a sunken button, three raised buttons, and a rocker tab. I’ll deal with these buttons later as I talk about using the system. The power button is backed by an LED which blinks when the tablet is suspended; the button is pushed up to turn the tablet on or suspend; it is held up to turn the device off.

As I noted above, the TC1000 is a “convertible” Tablet PC, but it differs from other convertible tablets significantly. The TC1000 has a detachable keyboard, allowing use as a dedicated slate, much like the Viewsonic and other pure slates. The keyboard itself can be connected to an included pseudoleather zipcover. The connections between the three pieces are solid; they don’t come apart unless you have pulled the unlock mechanism on the back of the keyboard or tablet. The tablet can rotate on the keyboard and lay flat with the keyboard locked against the tablet. It adds about a centimetre in depth and a bit more than half a pound of weight to the tablet PC.

The keyboard is about 90%–95% normal size; only a few things are annoying about the keyboard (the location and size of the function keys, plus the fact that F12 is only accessible with the Fn key). The keyboard also has a joystick pointer in the normal “notebook” position. What is most unusual about using the keyboard with the TC1000 is the same thing that has been noted in various reviews—the tablet screen sits flush with the keyboard, making some of the function keys hard to hit reliably. Annoying, but necessary, as this is the only thing which gives the TC1000 stability in “notebook” mode.

The Tablet Pen

The pen is stored in a spring-loaded silo near the power button. When the pen is inserted into the silo, it sticks out about two or three centimetres. The pen can then be pushed almost fully into the tablet with some resistance as the spring mechanism locks so that only the rounded head of the pen sticks out of the tablet, perhaps a couple of millimetres. Pushing on the pen again pops the pen about halfway out (it’s a strong spring).

The pen itself is relatively thick, which is strange for those of us who are used to thinner styli from PDAs (unlike most people, I have actually become fond of the thin Sony stylus that came with the T615C and the NR70V). I think that I would have preferred the thinner stylus from the Acer or the Toshiba; those styli are only slightly larger than the AAAA battery that powers the stylus, as this stylus is thicker than the typical sort of pen that I would purchase in any case.

The tip of the stylus is equivalent to the left mouse button; there is a button on the barrel of the stylus that acts as the right mouse button. Unlike some styli, there is no “inverse” operation of the stylus (the Acer stylus works both normally and inverted, where the inverted operation acts as an eraser). The tablet PC is based on an active digitizer, meaning that when the stylus is floated over the screen, the mouse pointer will track with the stylus. The tip of the stylus must be in contact with the screen to “write”, although I have found a bit of stickiness to the tip of the stylus, resulting in a bit of “overwriting” from time to time. I can’t reproduce it reliably, so I haven’t reported it to HP.

Windows XP Tablet Edition

While I agree with most folks about the odiousness of WPA, I also think that Windows XP is an amazingly good operating system, such that it is the only Windows-based operating system that I run anymore. Windows XP Tablet Edition (XPTE) takes XP Pro and improves upon the existing alternative input mechanisms. The pen-based input is reliable and reasonably good at interpreting most of my handwriting. I’ve also enabled and tried the “block recogniser”, but don’t care for it (it always seemed a poor imitation of Graffiti). The voice input mechanism is inconvenient for me, so I haven’t really used it.

I use the tablet PC in “notebook” mode most often because I’m a programmer. However, in my consulting, I have used the Windows Journal to sketch designs and take notes; for this, the tablet PC is perfect. I’ve also used the tablet PC to mark up a book that I have reviewed for a publisher, which was good stuff. I printed to the journal virtual printer and marked up the document just as if I had printed the document out. I can’t emphasize how nice this was.

There are a variety of reasons (aside from programming) I don’t use the TC1000 in tablet mode more often. First, and foremost, the recognition still isn’t quite good enough. I’ve had to rewrite some sentences too often, and the spacing is too aggressive (and not configurable), meaning that I have to correct a number of smaller things. Second, while many Microsoft programs are acceptably modified for XPTE, too many other programs aren’t. Third, the text input area takes up too much space (yes, I know it’s resizable) and the “whole area” isn’t really good enough (and interferes with normal operation). Fourth, there simply aren’t enough buttons available.

That last point, to me, is the killer. There are several customizable buttons and hotspots available on the TC1000. There’s a “mail” button (recessed, immediately below the power buttton); I use it often. There’s a Q button (raised, below the mail button) which by default launches the “Q menu”. This menu is configurable, but by default allows for quick muting, brightness controls, hibernation, and even the wireless controller. Next to the Q button are buttons labeled ESC and Tab (and are by default assigned to those keystrokes). Between these buttons is a hole that the stylus tip will fit through; this brings up the Task Manager (it’s the equivalent of Ctrl-Alt-Del). Below these buttons is the jog rocker (defaulting to up arrow, down arrow, and enter). Additionally, there are the hotspots on the top of the portrait screen noted earlier.

The problem is that the settings for these buttons is system-wide. So while it may be useful in some programs to have an ESC key defined, it is not going to be the case in all programs. In web browsing, I would prefer the jog rocker act as page up and down; in word processing, I might prefer it be left and right (so that I can move by character). It gets worse when you realise that you cannot tap the pen and select the second button at the same time, losing the ability to emulate a third mouse button. My preferred web browser, Mozilla, allows me to affect behaviours by selecting Ctrl, Alt, and Shift versions of both left and right mouse clicks.

Imperfect? Absolutely. Usable? You bet. The TC1000 has become my primary computer, primarily because of its size (just over three pounds with the keyboard attached).

I dropped the TC1000 off the couch and the top layer was scratched. When I called HP support about this, the initial quote was a minimum of CDN $1,200 (the cost of replacing the entire LCD assembly). I paid about $2,800 for the device, and HP wanted 40% of the cost to repair a scratch. Right. PalmOS devices have typically been about CDN $150 to repair the screen. I can’t find the details on Sony NR-70V screen, but I wouldn’t be surprised if it’s similar. I bought the NR-70V last May for about CDN $1,100. To me, the proper cost would be about 2 times the cost of repairing a PalmOS device (call it CDN $375)—the same cost ratio. It’s costing me about that much to have the TC1000 repaired anyway, but the initial quote was just insane.

What bothers me most about the cost of support repairs is that in many cases, the manufacturer can then repair parts with lesser damage (like the scratch on my tablet PC) and then use the repaired part in future repairs. My LCD wasn’t broken; this was the top layer of plastic or glass on the TC1000. In other words, for a CDN $170 piece of equipment, I may have been charged CDN $1,200. I’m not saying that HP would have done this, and they’ve worked wonderfully with me on this to reduce the cost significantly, but I had to work harder on this than I should have. This also would have been avoided with the Compaq “CarePaq” (their name for the insurance/warranty available which allows for one repair per year for three years). Unfortunately, I don’t think that it was available when I bought the device. I will get it, but it still annoys me how much I had to work to get this repaired. (It’s also not as good as the Dell plan that I have, which is three years of on-site service.)

1. The Shape of the Problem
   1. Example 1: The C Array
2. The Limitations of the Language
3. The Shape of the Solution
   1. Example 2: The C Linked-List
4. The Solution
   1. Example 3: The PL/SQL Linked-Array
5. Further Options
6. Lessons Learned
7. Further Reading
8. Source Listings
   1. Listing 1: Data Split Package (data_split_pkg.sql)
   2. Listing 2: Data Split Package Body (data_split_pkg_body.sql)
   3. Listing 3: Schema (data_split_tables.sql)
   4. Listing 4: Data (test_data_split.sql)

In an ideal software development project, the development environment (that is, the languages, tools, and libraries) will be chosen that best suits the nature of the problem domain and the skills of the professionals involved. In the real world, the development environment is dictated by any number of constraints. The most common is that the project is a modification to or enhancement of an existing project. Just because one cannot – for whatever reason – use the language most suited to the solution does not mean that one can’t take lessons from those languages.

At one of my previous jobs, I worked on a project where the code was written in PL/SQL called by a C driver. A lot of developers, in my experience, don’t think much of PL/SQL because they believe that it performs slowly, or because its functionality is limited. (Most of the time, however, this prejudice is present because the developers involved do not know the language itself.) For this project, PL/SQL performed the tasks (mostly data manipulation) as fast or faster than an equivalent C implementation could have done because there were fewer context switches and no network communication involved. (If you aren’t familiar with PL/SQL, see the accompanying article.)

As useful as it is, PL/SQL is not without its limitations, mostly surrounding complex data structures. These limitations presented what seemed to be an insurmountable roadblock to the project, but the time-frame of this subproject did not allow for the language to be changed to one where the data structures required could be represented more ‘naturally.’

The subsystem in question required that the time-series data be viewed in ways which are not supported by Oracle SQL queries on the original data; it was necessary to restructure the data. This turned out to be more of a problem than I had originally anticipated, and the eventual solution to the problem came from an interesting combination of techniques. In the end, I adapted data structures that are more naturally represented in languages like C, C++, and Java in PL/SQL.

The Shape of the Problem

The data was stored in the database as multiple records representing the data class, the starting and ending dates, and the value. Records could have any value (positive or negative) that spans any amount of time. Multiple entries of the same data class for overlapping time periods could be present. Table 1 shows how records for a five-day period might be present in the database. Figure 1 shows how these records could be considered on a timeline, which is how these values must be considered by the program.

Figure 1: Sample Data

This particular program needed to divide and combine the records such that there was one record for each data class per day, containing the portions of values which occurred on that day. If a record crossed multiple days, it was the program needed to see it within the daily summary for each of the days in which it existed and only for the proportion of the record that existed in that day. That is, because record GHI spans two days, the program has to treat it as if it were two separate GHI records, each of value 5. Similarly, there will be a total of three ABC records; the first record will have value 2 because that is the total of the portions of ABC records on that date. Data 2 shows how the data needs to be reorganized on a daily basis. Table 2 shows how the data needs to be reorganized on a daily basis. Figure 2 shows the same information against a timeline. Compared against figure 1, the way that the records have been massaged becomes clear.

The easiest way to handle this problem would be to make it so that as each record is added to the original table, a trigger is fired that does the record division as noted in table 2 and adds the resulting records to a daily summary table. This solution has three major problems with it:

1. The source table from which the original records were pulled was a very high volume table (and significantly larger than the sample data shown). Performing the split on a per-insert basis is computationally expensive, as each data class was generally expected to have at least one record and more likely several records each day (note that the examples given in this article assume only a single data source).
2. The total volume of the data in the source table was very high, and the daily summary table would be at least as large, and most likely several times larger than the source table. Since these values are computed and are used in a single subsystem (this one) only, it didn’t make sense to use that much storage for what was essentially a temporary table.
3. Each of the records in the data source table had a limited lifespan; once the analysis had been run on the record successfully, the analyzing program would never see it again. This would have increased the volume of the summary table unacceptably, because the summaries would have to be differentiated by visibility.

Figure 2: Sample Data Converted into Daily Records

Without a temporary summary table, there was no way to do solve this problem with SQL queries (after all, the temporary summary table solved the problem of the data structure), so I needed to approach this programmatically. The data structure immediately suggested by figure 2 is an array of daily records, each of which has a dynamically-resized array of item records. If I were using a C-like language, I might use data structures like example 1 to solve this, which would produce an array like the one shown in figure 3.

typedef struct item_rec
{
  char    grouping[4];
  double  value;
} item_rec;

typedef struct day_rec
{
  int       date; /* an integer in yyymmdd format */
  item_rec* item_list;
} day_rec;

day_rec* day_list;

Figure 3: An array containing dynamically resized arrays

The Limitations of the Language

This realization was very good, except for the fact that I couldn’t rewrite the program in C or C++ for this project¹. As stated previously, there was no time to explore that option. The answer had to be reached, quickly, with the tools available. The simple solution was not an option because the PL/SQL limitations around complex data structures. PL/SQL simply does not support nested collections. The array of structures containing arrays would not work in PL/SQL.

I looked through my library of PL/SQL books for a hint on how to solve this problem, but no one had publicly tackled this problem. There was, however, a hint toward a possible solution in Programming PL/SQL (2^nd Edition) [Feuerstein 1998]. Steve Feuerstein has noted and lamented this limitation of PL/SQL, and he presented a section on how fixed-size multi-dimensional arrays could be simulated. He presented this as a package allowing the developer to predefine (almost “allocate”) an array of size r × c complete with accessor functions. Because the implementation is hidden in the body of the package, it looks and works much like a two-dimensional array in other languages.

The technique presented was to use a PL/SQL INDEX BY BINARY_INTEGER memory table and map the two-dimensional array to this single table. The position of an element at [m, n] in an array of r × c elements are found in the index-by table with the forumula:

i := ((m × (c - 1)) + n)

That is, in a 10 × 10 array, position [7, 3] is at position 66 in a single-dimensional array of size 100. Similar calculations could be used for more than two dimensions, but it only works when you have fixed-size dimensions. There are other limitations best explained by Mr Feuerstein in his book.

As noted earlier, the problem to be solved had an indeterminate number of columns (dates) and rows (data class entries per day), so this technique could not offer a solution. It did, however, offer some ideas on how more complex data structures might be emulated in PL/SQL.

The Shape of the Solution

Part of the problem that I faced when originally implementing this was that I was letting my knowledge of PL/SQL and its limitations dictate where I was willing to look for solutions. I knew that because I couldn’t do nested arrays, I didn’t look as closely as I should have at first. However, PL/SQL is a general-purpose language where one can define new data types, so it’s simply a matter of needing to adapt to the limitations. To consider how I might do this, I considered alternative options in C-like languages, knowing that I could able to adapt the solution to PL/SQL.

Figure 3 represents the data as it might be implemented in C. It’s a single array (day_list) where each entry in that array is linked to another array (item_list). Because it needed to be a dynamically resizable array, I would probably implement it as item_rec *item_list, a pointer to an item. If I allocated memory in increments of sizeof(item_rec), then I could able to take advantage of C pointer arithmetic and treat the pointer as if it were an array – making it an implicitly linked list.

Most people don’t consier an array to be a linked list because it doesn’t act like one. While most arrays or vectors are implemented as contiguous memory blocks, this is not necessarily the case. Any given vector class could be implemented as a contiguous memory block or a linked list and the programmer using that vector class should never know the difference. The linking relationship in an array is based on proximity, not on an explicit link. If I modified the item_rec definition to include an explicit link to the next item in the array, as in example 2, then the linked-list relationship becomes explicit, allowing me to consider the implementation of the solution from a different angle. Figure 4 shows the modified structure graphically.

Figure 4: An array pointing to linked lists

typedef struct item_rec
{
  char              grouping[4];
  double            value;
  struct item_rec*  next;
} item_rec;

typedef struct day_rec
{
  int       date; /* an integer in yyymmdd format */
  item_rec* item_list;
} day_rec;

day_rec* day_list;

Further analysis of the problem revealed that this particular problem did not need the biggest strength of arrays: random access to the records stored in an array. I only needed to access the records in date order and then in the order in which they appeared in the database, so sequential access was sufficient, making it very clear that a linked-list was most likely to lead to the “correct” implementation. It does not, however, reach the solution, becuase PL/SQL only has automatic memory management and references are only allowed to SQL cursor variables (in Oracle8i). Since there is no way to allocate memory programmatically like one would with malloc or new, I needed a modified approach for PL/SQL.

As it happens, PL/SQL has three different collection types, and only one of them is an array in the traditional sense (the varying array, VARRAY). Varying arrays and object tables are the newest additions to the PL/SQL language and are present to support the Oracle8i object-relational extensions. Both data types are very useful, but they are Oracle8i-specific features and are not available in Oracle7; they also require more effort and planning to use than the normal PL/SQL index-by tables. Both varying arrays and object tables are close in interface to arrays in other languages (the developer is responsible for extending the array to add values to the end of the array, and the indexes are always created in order).

The third type of collection in PL/SQL, the index-by table, was my only option in Oracle7, but index-by tables are still probably the optimal choice for this sort of problem. Index-by tables are sparse tables indexed by a binary integer value. A sparse table is rather like a set or a Perl hash (using integer keys): only one value may exist at any given index. Unlike varying arrays and object tables, index-by tables automatically allocate the space required for the variables in the collection, more like a vector class than a C array. Index values may be non-contiguous (e.g., 1, 2, 5, 7). If the keys are created contiguously, they look and act in many ways like a C array, but otherwise they allow for smart indexing just as a set or hash allows.

Because index-by tables can be non-contiguous, PL/SQL provides alternate means for navigating them. There are four methods associated with index-by tables: FIRST, NEXT, PREV, and LAST. Using FIRST or LAST will provide the index of the first or last entry in the table, respectively. NEXT and PREV are called with a parameter (e.g., day_list.NEXT(day_idx)) of the index for which you wish to find the next (or previous) index value. If there is nothing in the table, FIRST and LAST will return NULL; if there is no record that exists after (before) the requested index, NEXT (PREV) will return NULL. One other interesting behaviour in a non-contiguous sparse table is that the index value provided as a parameter for NEXT or PREV does not need to represent a value that actually exists. In this way, I can have an index-by table that has values at [5, 10, 15, 20] and NEXT(7) will return 10. The possibilities for this are extensive and exciting.

Using NEXT and PREV with an index-by table makes it appear to work much like a doubly linked list, so if I consider the binary integer type as if it were a pointer, I can interweave several linked lists within a single contiguous vector.

By mapping the linked list from figure 4 to a vector structure, I get something that looks like figure 5; example 3 gives the data structures (in PL/SQL) to implement it. The first index-by table (day_list) contains an index reference to the head of its item list, which is an entry in the second index-by table (item_list). The item records have index references to the next item in its chain, with a NULL value representing the end of the chain for that particular entry. In this way, I’m using the random-access nature of array indexing to my advantage while sequentially winding my way through my linked-list imposed on top of the index-by table.

Figure 5: The PL/SQL data structure – an array pointing to a linked list created from another sparse array.

The lists in question must be treated as an inseparable pair. The “real” lists must be declared in one place and one place only. In Oracle7 or Oracle8, then it is in your best interest to declare these as private package global (not public package global, unless you want the users of your package messing with these values and possibly messing up your linked list). You can declare these in a function and pass them around as a pair to other functions that use them, but you must remember that if one of them is IN OUT the other must be IN OUT as well. It’s not recommended that you do this for large data sets that are transformed in this way unless you’re using Oracle8i and can use the NOCOPY keyword to pass these tables by reference between functions and procedures.

The Solution

TYPE day_rec IS RECORD
    (day_date   DATE,
     item_head  BINARY_INTEGER);

TYPE day_tab IS TABLE OF day_rec
    INDEX BY BINARY_INTEGER;

TYPE item_rec IS RECORD
    (type_code  VARCHAR(3),
     amount     NUMBER,
     next_item  BINARY_INTEGER,
     prev_item  BINARY_INTEGER);

TYPE item_tab IS TABLE OF item_rec
    INDEX BY BINARY_INTEGER;

Source Listings

Listing 1: Data Split Package (data_split_pkg.sql)

-- Licensed under the Mozilla Public Licence 1.1

CREATE OR REPLACE PACKAGE data_split AS
  info_data_split_pkg CONSTANT VARCHAR2(120) := 'Version 1.0';

  PROCEDURE process_data
    (start_date IN  DATE DEFAULT SYSDATE,
     end_date   IN  DATE DEFAULT SYSDATE,
     debug      IN  BOOLEAN DEFAULT FALSE);
END data_split;

Listing 2: Data Split Package Body (data_split_pkg_body.sql)

-- Licensed under the Mozilla Public Licence 1.1

CREATE OR REPLACE PACKAGE BODY data_split AS
  info_data_split_body CONSTANT VARCHAR2(120) := 'Version 1.0';

  TYPE data_value_tab IS TABLE OF data_values%ROWTYPE
    INDEX BY BINARY_INTEGER;

  TYPE day_rec IS RECORD
    (day_date   DATE,
     item_head  BINARY_INTEGER);

  TYPE day_tab IS TABLE OF day_rec
    INDEX BY BINARY_INTEGER;

  TYPE item_rec IS RECORD
    (type_code  VARCHAR(3),
     amount     NUMBER,
     next_item  BINARY_INTEGER,
     prev_item  BINARY_INTEGER);

  TYPE item_tab IS TABLE OF item_rec
    INDEX BY BINARY_INTEGER;

  debug_on    BOOLEAN := FALSE;

  PROCEDURE pl(lo IN  VARCHAR) IS
  BEGIN
    IF (debug_on) THEN
      DBMS_OUTPUT.PUT_LINE(lo);
    END IF;
  END pl;

  -- This loads the records that started, ended, were contained in, or
  -- contained the specified start and end date. Split records will pull
  -- only the pieces we need.
  PROCEDURE load_records
    (date_start IN      DATE,
     date_end   IN      DATE,
     value_list IN OUT  data_value_tab) IS

    CURSOR get_values_cur
      (start_date_in  DATE,
       end_date_in    DATE) IS
      SELECT dv.type_code, TRUNC(dv.start_date) start_date,
             TRUNC(dv.end_date) end_date, dv.amount
        FROM data_values dv
       WHERE TRUNC(dv.start_date) BETWEEN start_date_in
                                      AND end_date_in
          OR start_date_in BETWEEN TRUNC(dv.start_date)
                               AND (dv.end_date)
          OR TRUNC(dv.end_date) BETWEEN start_date_in
                                    AND end_date_in
          OR end_date_in BETWEEN TRUNC(dv.start_date)
                             AND TRUNC(dv.end_date);

    value_idx   BINARY_INTEGER := 0;
  BEGIN
    -- Initialise the value list.
    value_list.DELETE;

    FOR gvlc IN get_values_cur(date_start, date_end)
    LOOP
      value_idx := value_idx + 1;
      value_list(value_idx) := gvlc;
      pl('#         : ' || value_idx);
      pl('type_code : ' || gvlc.type_code);
      pl('dates     : ' || gvlc.start_date ||
            ' − ' || gvlc.end_date);
      pl('amount    : ' || gvlc.amount);
    END LOOP;
  END load_records;

  PROCEDURE split_records
    (start_date IN     DATE,
     end_date   IN     DATE,
     value_list IN     data_value_tab,
     day_list   IN OUT day_tab,
     item_list  IN OUT item_tab) IS

    value_idx       BINARY_INTEGER;
    item_idx        BINARY_INTEGER := 0;

    prev_idx        BINARY_INTEGER;
    link_idx        BINARY_INTEGER;

    rec_days        PLS_INTEGER;
    rec_days_amt    NUMBER;
    first_jul_day   BINARY_INTEGER;
    last_jul_day    BINARY_INTEGER;
    test_date       DATE;

  BEGIN
    -- Initialise values
    day_list.DELETE;
    item_list.DELETE;

    pl('# of values: ' || value_list.COUNT);
    value_idx := value_list.FIRST;

    WHILE (value_idx IS NOT NULL)
    LOOP
      pl('Processing Value Record: ' || value_idx);

      -- Get the first day and last day in Julian terms.
      first_jul_day := TO_NUMBER(TO_CHAR(value_list(value_idx).start_date,
        'J'));
      last_jul_day := TO_NUMBER(TO_CHAR(value_list(value_idx).end_date,
        'J'));

      -- Figure out the amount per day.
      rec_days_amt := value_list(value_idx).amount /
        (last_jul_day − first_jul_day + 1);

      pl('Value Record ' || value_list(value_idx).type_code ||
         ' [' || first_jul_day || ' − ' || last_jul_day ||
         ': ' || rec_days_amt || ' per day]');

      FOR jul_idx IN first_jul_day .. last_jul_day
      LOOP
        pl('Julian: ' || jul_idx);
        test_date := TO_DATE(TO_CHAR(jul_idx), 'J');

        -- Make sure that we're within the boundaries.
        IF (    (test_date >= start_date)
            AND (test_date <= end_date)) THEN
          pl('In Date Boundaries');

          -- If we are, start applying the amount per day.
          IF (day_list.EXISTS(jul_idx)) THEN
            -- We've found a list already extant at the
            -- current date. Reuse it.
            pl('Day exists.');
            item_idx := day_list(jul_idx).item_head;
            prev_idx := NULL;
            link_idx := NULL;

            -- Look for our type_code.
            WHILE (item_idx IS NOT NULL)
            LOOP
              IF (item_list(item_idx).type_code =
                  value_list(value_idx).type_code) THEN
                -- Found a matching record.
                link_idx := item_idx;
                item_idx := NULL;
                pl('Matching record found at ' || item_idx);
              ELSE
                prev_idx := item_idx;
                item_idx := item_list(prev_idx).next_item;
              END IF;
            END LOOP;

            -- No such type_code, add a new record.
            IF (link_idx IS NULL) THEN
              item_idx := NVL(item_list. LAST, 0) + 1;
              item_list(item_idx).type_code :=
                value_list(value_idx).type_code;
              item_list(item_idx).amount := rec_days_amt;
              item_list(item_idx).prev_item := prev_idx;
              item_list(item_idx).next_item := NULL;
              item_list(prev_idx).next_item := item_idx;
              pl('New record added.');
            ELSE -- We have a match: increase it by the daily amount.
              item_list(link_idx).amount :=
                item_list(link_idx).amount + rec_days_amt;
              pl('Existing record increased.');
            END IF;
          ELSE -- No charges yet at this date.
            item_idx := NVL(item_list. LAST, 0) + 1;
            day_list(jul_idx).day_date := test_date;
            day_list(jul_idx).item_head := item_idx;
            item_list(item_idx).type_code :=
              value_list(value_idx).type_code;
            item_list(item_idx).amount := rec_days_amt;
            item_list(item_idx).prev_item := NULL;
            item_list(item_idx).next_item := NULL;
            pl('New date and record added.');
          END IF;
        END IF;
      END LOOP;

      value_idx := value_list.NEXT(value_idx);
    END LOOP;
    pl('# of days : ' || day_list.COUNT);
    pl('# of items: ' || item_list.COUNT);
  END split_records;

  PROCEDURE save_records
    (day_list   IN day_tab,
     item_list  IN item_tab) IS

    day_idx     BINARY_INTEGER;
    item_idx    BINARY_INTEGER;

  BEGIN
    pl('# of days : ' || day_list.COUNT);
    pl('# of items: ' || item_list.COUNT);
    day_idx := day_list.FIRST;

    WHILE (day_idx IS NOT NULL)
    LOOP
      pl('Processing day [' || day_idx || ', ' ||
         TO_CHAR(TO_DATE(day_idx, 'J'), 'YYYY-MM-DD') ||
         '].');
      item_idx := day_list(day_idx).item_head;
      WHILE (item_idx IS NOT NULL)
      LOOP
        pl('Item [' || item_idx || ']: [' ||
           item_list(item_idx).type_code || ': ' ||
           item_list(item_idx).amount || ']');

        INSERT
          INTO data_output(data_date, type_code, amount)
        VALUES (day_list(day_idx).day_date,
                item_list(item_idx).type_code,
                item_list(item_idx).amount);

        item_idx := item_list(item_idx).next_item;
      END LOOP;

      day_idx := day_list.NEXT(day_idx);
    END LOOP;
  END save_records;

  PROCEDURE process_data
    (start_date IN  DATE DEFAULT SYSDATE,
     end_date   IN  DATE DEFAULT SYSDATE,
     debug      IN  BOOLEAN DEFAULT FALSE) IS

    v_list  data_value_tab;
    d_list  day_tab;
    i_list  item_tab;

  BEGIN
    debug_on := debug;
    pl('Start : ' || start_date);
    pl('Stop  : ' || end_date);
    load_records(start_date, end_date, v_list);
    split_records(start_date, end_date, v_list, d_list, i_list);
    save_records(d_list, i_list);
  END process_data;
END data_split;

Listing 3: Schema (data_split_tables.sql)

-- Licensed under the Mozilla Public Licence 1.1
CREATE TABLE data_values
  (type_code  CHAR(3) CONSTRAINT nn_data_values_tc NOT NULL,
   start_date DATE    DEFAULT SYSDATE CONSTRAINT nn_data_values_sd NOT NULL,
   end_date   DATE    DEFAULT SYSDATE CONSTRAINT nn_data_values_ed NOT NULL,
   amount     NUMBER  DEFAULT 0 CONSTRAINT nn_data_values_am NOT NULL,
   PRIMARY KEY pk_data_values (type_code, start_date, end_date));

CREATE TABLE data_output
  (data_date  DATE    DEFAULT SYSDATE CONSTRAINT nn_data_output_dd NOT NULL,
   type_code  CHAR(3) CONSTRAINT nn_data_output_do NOT NULL,
   amount     NUMBER  DEFAULT 0 CONSTRAINT nn_data_output_am NOT NULL,
   PRIMARY KEY pk_data_output (data_date, type_code));

ALTER SESSION SET NLS_DATE_FORMAT = 'yyyymmdd';

INSERT
  INTO data_values(type_code, start_date, end_date, amount)
VALUES ('ABC', '20010601', '20010601', 1);

INSERT
  INTO data_values(type_code, start_date, end_date, amount)
VALUES ('DEF', '20010601', '20010605', 15);

INSERT
  INTO data_values(type_code, start_date, end_date, amount)
VALUES ('ABC', '20010601', '20010603', 3);

INSERT
  INTO data_values(type_code, start_date, end_date, amount)
VALUES ('GHI', '20010602', '20010603', 10);

COMMIT;

Listing 4: Data (test_data_split.sql)

-- Licensed under the Mozilla Public Licence 1.1
BEGIN
  DELETE
    FROM data_output;

  data_split.process_data(TO_DATE('20010601', 'yyyymmdd'), TO_DATE('20010605', 'yyyymmdd'), TRUE);
  data_split.process_data(TO_DATE('20010601', 'yyyymmdd'), TO_DATE('20010603', 'yyyymmdd'));

  COMMIT;
EXCEPTION
  WHEN OTHERS THEN
    DBMS_OUTPUT.PUT_LINE('CODE: ' || SQLCODE);
    DBMS_OUTPUT.PUT_LINE('MSG : ' || SQLERRM);
END;

SELECT *
  FROM data_output
 ORDER BY type_code;

PL/SQL supports exception handling in named and anonymous blocks, and allows the programmer to define exceptions and associate these with error numbers. Several of the common Oracle errors are given names, and exceptions can be defined within packages (if they are defined in the public portion of the package, then they can be referenced just as a package procedure would be referenced).

PL/SQL supports all of the Oracle built-in types, plus a few for PL/SQL only (BOOLEAN, PLS_INTEGER, BINARY_INTEGER, others). Complex data types can be created as records or, in PL/SQL 8, database object types. Records can be created from references to tables or as an explicit declaration. Collections can be created from native data types, database object types, or records, but may not contain (directly or indirectly) other collections. Two of the collection types (varying arrays and nested tables) are based on Oracle8 object facilities and provide the best flexibility when defined as objects in the schema, as they can provide additional capabilities with CAST … AS statements. The remaining collection type, index-by tables, offers a sparse collection, where the entries do not need to be contiguous and allow for smart indexes. (Entries that have not been set in index-by tables do not exist and have no value, not even NULL, meaning that “random” index selection should test that the value EXISTS before attempting to get a value.)

The performance of PL/SQL is very good on certain sets of tasks, sometimes rivalling compiled C and C++. PL/SQL performs very well on tasks that involve data manipulation, and does not perform as well as tasks that involve significant calculations. Even with these limitations, PL/SQL’s flexibility, ease of use, and portability can make it an ideal choice for wide ranges of business logic and other functionality within Oracle products.