Webtrends irregularity

Webtrends crashed while running reports. Without some significant research I don't think I'll be able to determine the cause, so for now I'll just note the cause. The following two lines seem to be the source of the problem.

2005-09-07 19:42:49 202.147.177.70 - W3SVC12 PROJECT2061 198.151.218.130 80 GET /cgi-bin/404.asp 404;http://www.project2061.org/tools/benchol/ch2/ch2.htm 301 0 737 655 16 HTTP/1.0 www.project2061.org Mozilla/4.0+(compatible;+MSIE+5.0;+Windows+98;+{8EDEB5C2-2006-EB20-6F16-6A2EFD0A5822}) - http://www.google.com/search?q=uses+and+application+of+maths+in+every+day+life&hl=en&lr=&ie=UTF-8&start=30&sa=N

2005-09-07 20:14:52 202.147.177.70 - W3SVC12 PROJECT2061 198.151.218.130 80 GET /cgi-bin/404.asp 404;http://www.project2061.org/tools/benchol/ch2/ch2.htm 301 0 595 624 15 HTTP/1.0 www.project2061.org Mozilla/4.0+(compatible;+MSIE+5.0;+Windows+98;+{8EDEB5C2-2006-EB20-6F16-6A2EFD0A5822}) WEBTRENDS_ID=202.147.177.70-1418417440.29733860::8840CACE7014C0BFF54DA08DCA0E1F1B -


Update 2005-10-13:
Found some more lines causing a Webrends crash. More of the same, I suspect the cause may be the user agent string having the curly brackets {}. I didn't see it before because I was running on just the web content, so not all those lines were being loaded. I'm not going to bother noting the lines below; they're on the same IP and thus easy enough to find in the log.

IERI: Firefox update breaks movie playback

The new version of Firefox (currently 1.5 beta 2) has new security restrictions which prevent plugins from accessing files on the local system if the originating web page is on the Internet. This will be a big problem with IERI because the current setup requires that the movie file be local. I was able to confirm the problem by trying to view and control a movie through the utility then saving the page with the embedded movie locally and trying again.

Note: this is an existing problem with Safari. Considering the current trend in browser security I expect the other browsers will likely follow suite in the near future.

There are a couple of solutions we could consider:

• Code signing
• Creating a custom application for the IERI utility
  
• Creating an IERI "application" that consists of local html files for the shell
• hosting the movie files on our servers and streaming them

Code signing
This is probably the easiest solution to implement, but requires one of two things: a lot of money or a lot of time. Money if we were to purchase a certificate from one of the recognized authorities (such as VeriSign). A code signing certificate is expensive, even at the cheapest of providers, and has to renewed regularly. It is possible to obtain a free certificate from a provider such as CAcert, but it requires a more intense verification process whereby you actually meet with people to confirm your identity.

I know for certain that the problem can be addressed in Firefox by using signed code to modify a user setting when the movie plugin page is accessed:

try {
netscape.security.PrivilegeManager.enablePrivilege("UniversalPreferencesWrite");
navigator.preference('security.checkloaduri', false);
navigator.preference('security.checkloaduri', true);
} catch (err) {
document.write("Sorry, you can not enjoy this site because of " +err+ ".");
}

However, I'm not sure if the various browsers even support a common method of code signing ... though I'm pretty sure they do not. If this method is going to be useful it would be nice to be able to use it in more than just Firefox. Since Safari has a similar problem already any method should also be able to address the problem in this browser as well.

References confirming access restrictions for local files:

• [esqsoft.com] JavaScript Help: WINDOW Object
  
• [extensionsmirror.nl] pagePlaylist v0.7.1 for FX 1.5 Beta 1, Extension is broken.
• [jclement.ca] Firefox Weirdness

References for code signing:

• [mozilla.org] Signed Scripts in Mozilla
• [mozilla.org] JavaScript Security in Mozilla
• [docs.sun.com] Netscape Certificate Management System Administrator's Guide: Appendix F Netscape Signing Tool
• [groups.google.com] script signing 2
• [devguru.com] JavaScript METHOD: navigator::preference

Hosting & streaming movie files
This method, while probably not any less expensive than code signing, seems to be the best of the options. The main reason I say this is because it will require minimal rewriting of the utility and a minor investement in hardware infrustructure (for movie storage ... namely some large hard drives). If we really want to put some work into it we could also build a media management system on top of the storage system (something like the DAMS system at UMich).

EK seems to like this idea, so perhaps we'll investigate it a bit more.


The other two options presented are really viable. While it would provide a significant amount of flexibility in interface development, modifying mozilla to suite the needs of the utility seems a daunting prospect. And creating some kind of downloadable HTML core would likely cause more problems than is solves (mainly due to cross-site security restrictions).

No matter the solution we choose, this problem also present complications for the rewrite of the utility. The whole timecoding mechanism will need to be reworked so that this problem can be mitigated (I doubt we can avoid it completely). The rewrite may be the best time to implement one of the above options.

JS Library: Active Menu & Multi-Column List

I've completed two more scripts for our code library.

Active Menu

• Documentation
• Script: code formatting, no code formatting
• Stylesheet

Multi-Column List

• Documentation
• Script: code formatting, no code formatting
• Stylesheet

Bug: Internet Explorer standards mode and element scrollbars

I've run into a problem with my ActiveMenu script. When IE6 is running in standards mode (vs quirks mode) the width calculation of a box gets a bit screwed up because IE6 doesn't take into account scrollbars while calculating the width of an object. This means that objects with a defined height and overflow: auto or overflow: scroll will end up wider than either a calculated or assigned width. The width discrepency is a problem because the presence of scrollbars modifies the working width of a box, which affects the layout of the box.

While I was able to find a few mentions of this problem, no real information seems to be available. Certainly no indication of a fix through either styles or scripting. See:

• TheScripts.com discussion
• WebmasterWorld.com discussion

Unfortunately none of the big guys out there seem to have documented this problem. I may post a bug report on QuirksMode.

How this afffects Active Menu
If no width is specified for the story box then the area next to the floated headline list is calculated and used. Regardless of whether the width is specified or calculated, if scrollbars are added the story box is widened and no longer fits in the area next to the float. As a result the story box gets pushed below the float.

I worked around the problem by modifying the width of the story box when the document.compatMode property indicated that the page was being rendered in standards mode (via PPK) in IE6. One drawback of this approach is that the width is no longer flexible in IE6, though I could maybe add a function to correct it onresize of the parent div. I should maybe also consider disabling this check if the user has set the width of the story box in via CSS, under the assumption that they have taken the scrollbar problem into consideration.

I haven't tested in other versions of IE.

I'm not sure if this is also a problem with height calculations, though that would likely cause few problems with the document layout.

Mass e-mail list script updates

I modified the database/scripts so that actions performed by end users (subscribing, unsubscribing, updating e-mail address) are recorded in a separate table. The reason behind this update is so that I can keep the e-mail subscription database and the ichaos database synchronized.

The update is a simple INSERT that runs after each list update. The new table (MassEmailManage) has columns for old email address, new email address, action, and list id. I'll manually check it once a week or so to see if any ichaos updates need to be made.

Also, it'll be interesting to see how much activity these pages see.

Home page redesign

I've pretty much completed the redesign of the home page. While the initial design phase was pretty easy the follow-up of creating a structurally clean, progressively enhanced page took a bit longer than I expected.

The page consists of four sections at present:

• header
• "What's New" headlines
• quick links
• brief introduction

The header and introduction were fairly straightforward. A float on the logo was the most advanced styling used. The other two sections took much longer since I wanted the page to work structurally for browsers that couldn't placed the headlines in the dt and the related story in the dd. If an image was also used, it would be placed in another dd above the story.

The next step was to style the list for browsers that aren't DOM-compliant (since I use the DOM extensively in the creation of active portions of the menu). What I wanted was fairly simple, the headline having a background color spanning the width of the content area with the entire area clickable. Below the headline and indented much like a normal definition list is the associated story. If an image is present it would be floated to the left of the story. I was able to achieve the design I wanted in modern browsers and most of the based CSS is used in the interactive version as well. In Nav4 the menu looks pretty much like a normal definition list, but I decided to hide the images because Nav4's styling capability made it difficult to get the desired effect.

The script runs once the page is loaded. Since this is a first version, the structure expected is pretty strict (I expect I'll be able to clean it up when I work on future versions). The script applies a class to the menu so that the styling used for the interactive elements is added. The script creates a styled unordered listed (ul) for the headlines. The headlines are pulled dynamically from the dts and so no extra work beyond filling out the definition list is required when implementing. I used a lot of DOM coding to generate the list such as createElement, appendChild, and insertBefore.

I have yet work on the printable view, but I'll probably hammer it out while I'm documenting the code.

Multi-Column List
The design for the quick links was fairly advanced when considering the capabilities of current web browsers ... a two-column list. While I could have easily used a table or two lists and floating or some other mechanism, the desire to maintain a semantically meaningful structure made these options undesireable.

With this in mind I tried to find a method of spreading a list across two columns that would not require breaking the list into parts. Plus I wanted to keep the amount of work required to edit the list at a minimum. After a bit of searching I found an article about multi-column lists on builder.au that gave me a good start (later ALA published their own article on this topic as well). I ran into a few problems during development ... mostly with IE (see below). The multi-column list feature I ended up with isn't nearly as robust as I had hoped to make it, but it performs the intended function pretty well.

The list script relies on a standard list (ul or ol). The user doesn't need to do any prep-work beyond applying a class to the list, attaching the JavaScript, and attaching the base stylesheet. Of course extra styling can be done by the user, and I had to do this for the home page so that I could make the multi-column list function flexible enough for library code.

Once the page loads the script looks for any list with the required class. Once a list is found a second class is added that provides the base styling needed for multiple columns (though I'm thinking of moving some of this functionality to the script ... we'll see once I start working on the documentation). The script determines the height of each list item and then uses that information to decide where to divide the list (right now the script can only handle a two-column list). The first element of the second column is given a top and left margin so that it sits next to the first column. The following list items are then given a left margin. Finally, the last element of the list is given a height value so that elements following the list don't overlap it.

There are a couple of issues and enhancements I'd definitely like to make. Currently I have the list item markers disabled due to various browser problems. The list doesn't handle width resizing at all (can I use percentage for the positioning?). I had to use a recursive function to watch the height of the list to handle height resizing (there may not be any other way around this, but maybe percentages or ems can help here as well).

Again, I have yet to work on the printable view, but I'll probably work on it while I'm documenting the code.

References

• Definition lists - misused or misunderstood?
  

IE quirks
Unrelated to this work I came across an article that provided some information that helped during development. On the MSDN site there's an article about the hasLayout property. The property seems fairly innocuous, but reading up on it is important for understanding some of the styling weirdness that can be encountered in IE. Even better, there's a link to another article that gives a lot of good info, including how styling a list with layout can get ugly.


Now that I'm done with the home page I'll be spending a few days documenting the scripts for the code library.

JS Library: Show/Hide Content

I've finally, finally, completed some code documentation. This is a little JS I wrote to hide content and then show it when a link is clicked. It's nothing special, but I've done my best to make it as easy to use as possible.

Show/Hide Content Files

• Documentation
• Script: code formatting, no code formatting
• Stylesheet