2001-01-10

5:21:20 PM

• Memory slag. A couple of years ago, Rebecca Eisenberg offered some free advice to PR folks on how not to waste the time of online journalists (cited in TBTF for 1999-09-11). Central to her advice was this suggestion: don't send me MS Word attachments, send me URLs.

  But did they listen? Not hardly. Today I received a press release, complete with attachments, that significantly upped the ante on this poor PR practice.

  James Fallows's column in the Industry Standard this week, No Thanks for the Memories, talks about what he calls memory slag -- leftover bits of data from a hard disk, or even from DRAM, that can show up in documents. Fallows claims that such slag was very common in Microsoft environments up to the middle of the last decade, but that Microsoft made a big push starting after the release of Windows 95 to make sure its code zeros out memory that is supposed to be blank. (Here's an explanatory page on the site of the data-recovery expert who opened Fallows's eyes.)

  Fallows's example concerns an obscure output format from MS Outlook, which he guesses didn't draw the attention of the code cleaners. But from the MS Word file I received today (apparently created in Word 98 on a Macintosh), it's clear that more mainstream formats are also vulnerable to memory slag.

  The attachment was a two-page press release from a company I shall not name. They had thoughtfully attached both .DOC and .PDF versions of the release. Having just read the Fallows piece, I was curious to note that the .DOC file was more than 10 times larger than the .PDF. Instead of opening it using Microsoft Word, I dropped it onto BBEdit, a Macintosh text editor that cheerfully showed the file's binary content in all its glory.

  It quickly became clear that the extra 200K in the .DOC file contained more than just Word's apparatus. I located several lengthy lists of names -- apparently distribution lists -- and several versions of what appeared to be a completely different press release. When I mailed these back to the sender in clear text, she was rightly alarmed.

  Fallows quotes an employee of a computer-security company:

  When we get a resume, in Word, from job applicants, we put it in the hex editor and go right to the end to see what else they've been writing.

  You have been warned.

  Updated 2001-01-11, 10:42 am: John Waterson, ITS, EC, SE writes:

  It is quite alarming to think that non-zeroed blocks of malloc'ed memory might find their way into documents, but I figured it was worth mentioning that there is another -- more mundane and widely documented, although no less dangerous -- explanation for the garbage you found in the press release.

  Word includes a (mis)feature called Fast Saves, whereby text deleted from a document isn't actually removed from the file on disk when the user hits the Save button. Instead, Word just appends any new text to the end of the file, and some flags are set which result in the "deleted" text being skipped by the document parser. This saves Word from having to rewrite the whole file, which appears to be a fairly disk-intensive operation.

  However, this becomes a very widespread problem when you consider how most non-technical people use a word processor. Not having much of a grasp of stylesheets, templates and the like, an average user will almost never create a new document from scratch. Instead, they'll pick a document from their personal archive that is broadly similar in format to the one they want to create, copy it, and start deleting the stuff that they want to change. I figure that this behaviour -- coupled with the Fast Saves feature -- is the most likely explanation for the detritus in your press release.

  Anyway, the other thing that is worth mentioning about Fast Saves is that -- mercifully, and unlike the non-zeroed memory problem -- they are fairly easy to switch off. Have a look under the Save tab of the Tools, Options dialog, and just disable the relevant setting.

  Also, for reference: Microsoft have released many a technote about the Fast Saves option in Word. Some samples: [1], [2], [3]. These all reference Word for Windows, but the Mac version definitely seems to incorporate the Fast Saves feature too, viz: [4].

2001-01-09

7:04:31 PM

• Much ado about peering. The NANOG list has recently carried some close-to-the-ground messages on the murky and little-understood world of ISP peering. Someone notified Dave Farber (the Paul Revere of the Internet), and readers of his IP list enjoyed this piece of apparent news. Now unless the tech reporters read NANOG, the media will probably pick up the story, reporting that UUNet has at last published its requirements for peering and the world will be a better place.

  For the record, there's no news here.

  Background: in the earliest days of the commercial Net, at the point when the phrase "Internet backbone" ceased to have a well-defined meaning, the largest ISPs met at public exchange points and swapped traffic for free. Soon the largest of the large were exchanging for free only among themselves; they began charging to take the traffic of smaller carriers. In 1997 TBTF marked the point at which the next smaller fish, tier-2 carriers, began to charge exchange fees to the still smaller downstream regionals and ISPs.

  In recent years, as Net traffic has continued its wild growth, peering arrangements have gotten increasingly complicated. It used not to be possible for a mid-sized carrier to discover anything about the peering policies of the big guys without signing a non-disclosure agreements and engaging in exhaustive price negotiations.

  The piece that caught Farber's eye was news of UUNet's publication of its peering standards. This event was neither the first of its kind -- Genuity had published their standards last fall, as a Farber follow-up mentioned -- nor was it particularly meaningful for mid-tier ISPs. As one of NANOG's stalwarts posted, knowing what a tier-1's peering policies are is not the same as obtaining a peering agreement with one of them. UUNet's publicly disclosed policy will simply make it easier for smaller carriers to rule out the possibility of ever peering with the giant.