Friday, October 27, 2006

IN Condition Tricks

Not your type, huh? Change for the better.
I needed to pull some data from our contact database (ichaos) based on ZIP code. The list of codes to use was fairly lengthy and I was concerned about the length of a possible WHERE clause since the data is stored as text to accommodate both the 5-digit and 9-digit US formats (not to mention postal codes from other countries).

Even though the column is stored as text I decided to use the IN() condition to provide the condition. This worked much better than I expected. Not only were the specific matches returned (we used the 5-digit variation in the conditional), but MySQL also returned any variations on the provided condition.

The following SQL statement:
SELECT … WHERE post_code IN (12345);
returns not only the records matching 12345 but also any that match on the first five characters (such as 12345-6789).

The reason this search appears to work appears to do with type conversion (see Type Conversion in Expression Evaluation). Since we are attempting to compare a string to a number the two arguments are converted to floating point numbers by default. To have better control over the comparison we could use the LEFT() function to truncate the zip code to 5 digits, then the CAST() function to control the conversion of the column used in the statement.


Your custom order
While performing my search I noticed another useful tip for the IN() condition in the comments of the SELECT syntax documentation page. If you use an IN() condition in the ORDER BY clause you can specify a custom ordering scheme for a column. So the following statement:
SELECT … ORDER BY %column_name% IN (%value1%, %value2%), %column_name%;
will order the column first by the values specified in the IN() condition followed by the default ordering scheme for the type of data stored in that column.


Update 2006/12/05:
After some further testing I've found that the custom ordering trick doesn't work as I thought. What it does is group the records matching the condition and sort those as a group. It does not sort the values according to the order specified in the parenthesis. If you want specific control of the ordering you'll have to do a separate IN() condition for each value, thus grouping all the records with that value.

What this means is that my previous example will place all records with %value1% and %value2% at the top, but the sorting of those values will be whatever the default is (the order entered in the database unless other sorting options have been specified).

To get the results I had been seeking you have to use something more like the following:
SELECT … ORDER BY %column_name% IN (%value1%), %column_name% IN (%value2%), %column_name%;

Thursday, October 05, 2006

Windows 2003 web server returns 503 Service Unavailable

Yes, lots of problems today ... but at least they're all easy to fix.

After setting up my new web sites I was getting a "Service Unavailable" response from the server. Turns out that once you create the sites you have to start the application pool associated with those sites. It's a fairly minor additional step, but it'd be nice if starting a site would start the associated application pool (or at least remind you to do so).

Oh, and don't forget to create/turn on the appropriate entries in the Web Service Extensions folder as well (such as for the asp processor).

Domain literals not supported by IIS in Windows

A little misleading of a title, but it gets the point across nicely. The MMC GUI tools don't support the creation of a domain in the SMTP console. The solution, though, is fairly easy. Shut down IIS and edit the metabase directly. If you're lucky enough to have Windows 2003 it's super-easy since the metabase in in XML format.

The procedure I used was to creat a domain of the literal sans brackets first. This makes it easy to find the appropriate entry in the metabase. Then I edited the metabase to add the brackets back in. Easy.

CDONTS not included in Windows 2003

I was setting up a new server today when I realized that Windows Server 2003 does not include cdonts.dll. We have a number of older scripts that use this function to send out mail. Eventually we'll be moving all our stuff to open source, but in the meantime I'd rather not rewrite these scripts to use cdosys.dll (mainly because it requires more work and I'm lazy). Luckily the solution is easy enough ... install the cdonts.dll file from a Windows 2000 Server installation disk.

Microsoft was kind enough to provide the info I needed:
I have actually already run into this, so I'm not sure why I hadn't noted it before.

Monday, September 18, 2006

Mass Mailing: Alerts

E-mail alert regarding early-bird workshop registration deadline sent out to 2642 recipients at 1:30 PM on 18 Sep 2006.

Wednesday, September 06, 2006

Scripting: Opacity

I'm working on a minor project for the main AAAS home page. They want to better highlight some work that was recently done to promote the organization without drastically changing the current home page design/content. The method they decided to go with was to have a section of the page transition between two sets of content. Of course, there are a couple of ways of doing that ... some easy, some not so easy. I decided to use a combination of CSS and JavaScript to make it easily editable while still providing some nice functionality.

The layout has the two content blocks displayed one after the other. I use CSS to hide the secondary content and JavaScript to switch between the two. The trick came when working out the transition. We can't have it be too abrupt or the distraction level would be high. Luckily CSS provides us with a nice feature to fade the content blocks in/out ... opacity. True this property is CSS3, but some of the browsers already support it. Plus, on top of that most newer browsers at least provide their own proprietary alternative.

I found the core of my transition script on Richard Rutter's blog (Onload image fades without Flash). The setup was a snap.

Of course, not all was bliss. I ran into a few problem with (drum roll please) IE.

First, the transition didn't work. For whatever reason, applying the opacity setting via JS had no affect. Turns out that IE seems to ignore the opacity setting when applied to an object that has no defined width and height [brianerror.net: Cross-browser BlendTrans Filter JavaScript]. Applying height: 100%; width: 100%; seems to solve the problem sufficiently without causing any collateral damage to the page design.

Second, in IE7 any bolded text was becoming horribly distorted as the opacity was applied. It looks like IE7 is unable to accurately render the text if no background color is defined on an object to which opacity is applied, if that object contains bolded text, and if font-smoothing is turned on [jcxp.net forums: IE7 Beta 3 Download, Download!][microsoft connect: Textual Distortion with Microsoft Filters]. Applying a background color seems to solve this problem and has no detrimental affects, assuming there is no variable-colored background behind the object in question.

Friday, August 18, 2006

Mass Mailing: 2061 Connections - July/August 2006

The July/August 2006 issue of 2061 Connections went out at 2:15 on Aug 18, 2006, to 3916 recipients.

Monday, August 14, 2006

Webtrends irregularity

More of the same. No obvious problem, but some quick testing points to the user-agent string. Doesn't really make sense, though, because there are other lines with the same string. Oh well. I took out a whole swath rather than test each line ... but random testing seems to indicate they'll all cause problems.

2006-07-23 05:31:15 W3SVC48143651 WIZZLE 198.151.218.130 GET /includes/styles/aaas.css - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 9256 368 2703

2006-07-23 05:31:15 W3SVC48143651 WIZZLE 198.151.218.130 GET /publications/bsl/Default.htm - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) - - www.project2061.org 200 0 0 41053 251 5374

2006-07-23 05:31:15 W3SVC48143651 WIZZLE 198.151.218.130 GET /includes/styles/2061.css - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 3884 368 937

2006-07-23 05:31:17 W3SVC48143651 WIZZLE 198.151.218.130 GET /includes/scripts/detectstyle.js - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 700 375 937

2006-07-23 05:31:18 W3SVC48143651 WIZZLE 198.151.218.130 GET /includes/scripts/foldoutMenu.js - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 9379 375 1140

2006-07-23 05:31:18 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/plus.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 298 359 812

2006-07-23 05:31:19 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/minus.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 295 360 984

2006-07-23 05:31:19 W3SVC48143651 WIZZLE 198.151.218.130 GET /includes/scripts/jslibrary.js - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 7430 373 1046

2006-07-23 05:31:21 W3SVC48143651 WIZZLE 198.151.218.130 GET /includes/scripts/menuarrays.js - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 3156 374 1031

2006-07-23 05:31:21 W3SVC48143651 WIZZLE 198.151.218.130 GET /includes/scripts/spawnwin.js - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 3314 372 999

2006-07-23 05:31:22 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/pixel.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 290 360 890

2006-07-23 05:31:22 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/main_logo.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 2645 364 968

2006-07-23 05:31:22 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/topmenu_left_off.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 361 371 828

2006-07-23 05:31:22 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/topmenu_sep1.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 399 367 828

2006-07-23 05:31:24 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/topmenu_sep2.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 464 367 874

2006-07-23 05:31:24 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/topmenu_sep3.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 465 367 812

2006-07-23 05:31:24 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/topmenu_right_off.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 368 372 828

2006-07-23 05:31:24 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/leftcol_frame_top.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 405 372 828

2006-07-23 05:31:26 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/01_mainheader_top_programs.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 2086 381 874

2006-07-23 05:31:26 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/header_search.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 689 368 937

2006-07-23 05:31:26 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/btn_search.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 570 365 921

2006-07-23 05:31:26 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/btn_go.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 735 361 843

2006-07-23 05:31:28 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/01_mainheader_sub_programs.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 24512 381 1593

2006-07-23 05:31:28 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/subheader_education_pr.jpg - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 14522 377 2046

2006-07-23 05:31:28 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/tools/bsl/BSLlogo.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 1015 372 828

2006-07-23 05:31:30 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/tools/bsl/BODcov.JPG - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 13784 371 1343

2006-07-23 05:31:30 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/greywhole3.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 2707 365 1124

2006-07-23 05:31:30 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/submenu_bg.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 557 365 828

2006-07-23 05:31:30 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/submenu_blackarrow.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 345 373 812

2006-07-23 05:31:32 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/submenu_arrow.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 387 368 812

2006-07-23 05:31:32 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/submenu3_bg.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 321 366 1015

2006-07-23 05:31:32 W3SVC48143651 WIZZLE 198.151.218.130 GET /images/submenu_bottom.gif - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL;+pagesurvey=3 http://www.project2061.org/publications/bsl/ www.project2061.org 200 0 0 514 383 968

2006-07-23 05:31:34 W3SVC48143651 WIZZLE 198.151.218.130 GET /favicon.ico - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL;+pagesurvey=3 - www.project2061.org 200 0 0 2491 290 874

2006-07-23 05:33:16 W3SVC48143651 WIZZLE 198.151.218.130 GET /includes/styles/2061.css - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL - www.project2061.org 304 0 0 210 398 796

2006-07-23 05:33:16 W3SVC48143651 WIZZLE 198.151.218.130 GET /includes/styles/aaas.css - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL - www.project2061.org 304 0 0 211 399 812

2006-07-23 05:33:16 W3SVC48143651 WIZZLE 198.151.218.130 GET /includes/scripts/detectstyle.js - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL - www.project2061.org 304 0 64 0 408 781

2006-07-23 05:33:16 W3SVC48143651 WIZZLE 198.151.218.130 GET /includes/scripts/foldoutMenu.js - 80 - 202.133.9.242 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+.NET+CLR+1.1.4322) ASPSESSIONIDAQDTQQTS=NIPAACGBPKHHPPHPIBELLLLL - www.project2061.org 304 0 64 0 408 749

Mass Mailing: 2061 Today - 2006 #1

Sent out on Monday 14 Aug 2006 at 1:05 PM to 1553 recipients.

The send-out was inadvertent. I was doing some final testing of the mailer script and had selected the 2061 Today list to pull the automated footer text. I updated the message (thank goodness) then sent out the message. What I had intended to do was send it to the ad-hoc list (with just my address) to test things and make sure they worked ok.

It's unfortunate, because there were two problems that would have been caught otherwise.
  1. There was a bad link in the e-mail address to project.org
  2. I hadn't yet uploaded the newsletter to the site. I didn't realize this until a few hours later.

Wednesday, August 09, 2006

MySQL: Using with ASP (after a long stint with MS SQL)

I've started moving tables from MS SQL to MySQL. I began the conversion process with the search from Benchmarks On-Line which went over without a hitch. That was to be expected, though, because that db access is read only ... no inserts/updates necessary. The only area I had to pay special attention to was the cursor type/location and the lock type. I updated these as necessary to best work with MySQL, especially considering the method the data was being used (client forward-only cursor, read-only lock).

Today I worked on moving over the survey from Blueprints On-Line. If I were going to run into any problems they would be pretty evident on this page. The table involved uses text (TEXT, VARCHAR), number (TINYINT, BIGINT), and date (DATETIME) fields. There is both a read and an insert on the table. The survey is unused, so I can test it without worrying about affecting the user experience too much.

I immediatly ran into a problem relating to permissions. For some reason MySQL kept telling me I didn't have access to the table even after I set up permissions. After talking to DP, though, I had a thought and checked out the DSN. I was using the same DSN that I used for the Benchmarks conversion (and plan to do the same for all scripts on the site). But the Benchmarks table is in a completely different database from the Blueprint table. The error message, while correct, was a bit misleading ... the table in question didn't exist in the database being accessed. The permission message was correct, however, in that I hadn't given the user permission to access the table specified. Since I didn't have permission to access the table in question MySQL didn't even bother to check if it existed. Sneaky. I've updated the DSN to have no default table and now specify my connection string as DSN=dnsname;Database=databasename;.

With this problem fixed I tried out the data read portion of the script. I immediately ran into a problem with some code that uses the data to calculate averages. It appears that in order to use the recordset value directly (i.e. not assigning it to a variable first) you have to first convert it to the appropriate type in VBScript. When using MS SQL I didn't run into this problem ... perhaps because MS SQL reports the data type assigned by the database. MySQL, however, just reports the type as field. I added a CInt() whenever the database values were used in a calculation.

The read was working correctly so I decided to try out the data insert. I ran into yet another problem, this one more serious than the first. The method I use to do the insert in the survey script is to create a recordset object, use the AddNew() function to create a new record, update the columns for the record, then use the Update() function to update the database. While updating the fields I was running into this error: "Multiple-step operation generated errors. Check each status value." I'm not sure what the reason for the error is, but I did find a solution on the MySQL forums in the thread multiple-step operation generated errors. That would be to turn on the "safe" option in MyODBC. Seems to work, but since I don't know the source of the error (or what the safe option does) it's conceivable that this solution doesn't actually solve the problem and that I'll run into further issues later on.

With these problems solved things seem to be running smoothly. I'll have to do some more research on the multi-step error and the safe option of MyODBC. In the meantime I'll start working on the other scripts.

I've already thought about one issue I might need to address at some point ... I currently have no development database. Not too happy about that.


Update 2006-08-10:
I've completed conversion of scripts from MS SQL to MySQL. In the process I found some more incompatibilities that need to be dealt with when converting.

First, MySQL does not allow the assignment of non-static values as column defaults (minus a few exceptions). The value must either be a constant or a literal value. Because of this requirement I was running into problems with the DATETIME fields I had set up because the database could not calculate the date when a new field was inserted. This issue led me to one of the exceptions ... TIMESTAMP fields can be calculated using the CURRENT_TIMESTAMP function. It looks like from now on I'll be using the TIMESTAMP data type rather than the DATETIME data type. For more interesting aspects of this data type check out TIMESTAMP Properties as of MySQL 4.1.

Second, some scripts were taking the result from a SELECT COUNT() statement and assigning it to a variable. MS SQL ODBC plays very nicely with ASP/VBScript and the variable type was automatically recognized as numeric. Since VBScript has some basic requirements with regards to using variables this made things very easy.

MyODBC does not play as nicely, however. The type returned using the same SQL statement is not recognized by VBScript. Applying a CInt() to these values corrects any type mismatch problems.

Yet another best practice that's not followed regularly: explicitly define a variable's type.

Third, I'm running into a problem where a date field with a null value causes the the recordset object to error out when trying to open the conneciton. The error message is "Data provider or other service returned an E_FAIL status." Checking for a null value in the SQL statement using the IFNULL() function seems provides somewhat of a solution to the problem, except for two issues: 1) the SQL statements involved have to be rewritten, and 2) at least one row is always returned, even if there are no matching rows in the table. The first issue isn't too much of a big deal, but the second one is a show-stopper. I'll have to do some more research.


Update to the the second issue mentioned above ... I'm running into more widespread problems with numeric types returned by MySQL. Even a regular INT data type seems to be having problems. If I have time I'll do some more research on this one as it'll no doubt be a repeating problem.

MS SQL to MySQL Conversion

Just a few notes I have regarding some considerations to keep in mind when moving from MS SQL to MySQL. Most of these are related to programming in ASP.

Check data types for appropriate conversion. Of special concern:
  • VARCHAR only allows 255 character in MySQL as opposed to 4000 in MS SQL (depending on the version). Use a TEXT data type instead.
  • DATETIME does not allow for dynamic defaults in MySQL. Use TIMESTAMP instead, which can be set to update to the current time on insert and optionally on update.
  • Take note of DATETIME columns from MS SQL. A precision time may be exported that doesn't import into MySQL. It's easiest to use a regex to trim the time after the seconds value.
  • BLOB data types (including TEXT) won't import into MySQL using MyODBC and DTS. Easiest way to get around this is to export to a CSV/TAB file and import into MySQL.
In ASP note the following concerns:
  • Check for any table name quoting. MySQL uses backticks (`).
  • Check the cursor type, cursor location, and lock type on any recordset objects. Some options have to be enabled in the DSN to allow certain cursor types (see CursorTypes, LockTypes, and CursorLocations). The following guidelines apply:
    • location: client; type: static - recommended for most connections. Static is the only supported cursor type on a client cursor.
    • location: server; type: forward only - fast read.
    • location: server; type: dynamic - while this cursor type can be specified it is not supported by MySQL natively. The ODBC driver performs extra communication functionality to simulate this type, resulting in significant overhead.
    • The keyset cursor type is not supported at all.
    • When not performing updates use a read only lock, otherwise use an optimistic lock.

Thursday, July 27, 2006

App Dev: Benchmarks On-line

I modified Benchmarks On-line so that the benchmark search now uses the MySQL server, referencing a table in the items database. The switch was fairly painless, though I did have to make some changes.
  • The items datbase table was based on the original one used for the web site, but some of the field names had been modified. Updating the script to reference the correct field names was fairly painless.
  • The items database table contains goals from multiple goals documents. I modified the various SQL statements to only pull from Benchmarks.
  • The original table contained a field with the URL of the chapter+section+grade range. The new table did not. It would have been fairly trivial to add this information to the table, but EK seemed reluctant and upon further review I decided it would not require too much effort to modify the script to dynamically generate the URL.
  • I updated the free text search to use MySQL's built-in full text search option rather than the rather crude text search I had developed so many years ago.
Further information regarding the third bullet above:
Once I started working on modifying the code I realized it would require more work than anticipated. The original page anchors were based on the section titles, which would have been fine but often were shortened or had different capitalization. I decided a better approach than trying to figure out each section anchor I would modify the anchors to use letter instead of the title. It took a while to make the change, but probably less time overall because it simplified the programming requirement.

The major drawback to this approach is that it will break any outside links to Benchmarks that rely on the page anchors. I have a solution in mind: use JavaScript to check for an anchor and, if one is found, see if it matches one of the old anchors used. The browser can then be redirected to the appropriate anchor. Should be easy enough to code, though I haven't decided if I'm going to go ahead do so.

Scripting: Debuggin and IE

I hate developing in IE because it provides the most absolutely completely useless error message for JavaScript. I just had to get that out there.

Ok, the error was really ocurring in the Adobe SVG plug-in inside IE, but I think it uses the same or a similar JS engine. At any rate the errors reported are the same as the kind you'd get from IE.

I was able to discover the location of the error, but not the cause. At least, not from the vague error message I received. The specific error message I would encountering was "Expected identifier, string or number" ... with line/column information that was pretty much useless. Luckily I found a blog post that mirrored my situation exactly.

The error was in the JSON returned in response to an XMLHTTPRequest. Apprently the trouble was caused by an extra comma I had forgotten to take out. Seems like something that should cause problems in any JS engine due to bad syntax, but then again I do need to do more research on object notation before I can say something like that with certainty.

References:

Tuesday, July 25, 2006

Coding: Javascript Arrays

I've been working a bit with XMLHTTPRequest lately (specifically in relation to our SVG work). I wanted to make a quick and dirty sample of some functionality discussed with FM, so I thought JSON would be an easy way to get started. I'm not too familiar with object notation so from the start I knew I'd need to do some additional research on it. Plus I've been meaning to look more into the topic.

I wanted to use an associative array to represent the data I'm working with (essentially node/value pairs). While researching the question I came across the article JavaScript “Associative Arrays“ Considered Harmful. I was taken by surprise to find out that JavaScript only supports indexed arrays natively. Those associative arrays I thought I would be working with are actually property extensions to an object. Quite a shocker for me.

This won't drastically affect the progress of the work I'm doing right now because it's very preliminary. The eventual product will use a much more complex data set that requires a different representation within the code.

Based on this discovery, however, I feel I need to do some more research on JavaScript data structures in addition to object notation. This knowledge is certainly important from a programming standpoint ... I could easily work myself into a corner by not understanding the implementation of the objects and data structures in JavaScript.

Monday, July 24, 2006

IERI: Updates

The IERI utility has been in heavy use the past few weeks, so a few minor but common issues have come up. A few of these I decided to go ahead and fix, namely:

Movie availability
In order to do an enacted analysis the timecoding of the activities has to be completed by an administrator and the movie released. Once a movie was released it could not be used for further activity-level timecoding once analysis had begun. A lot of people were releasing the movie after timecoding only a single activity, or a group of related activities, even if the movie contained other activities.

It looks like the purpose of enacting movie release restrictions in the first place are not really an issue with the utility as it's currently being used. I went ahead and made it possible to undo a movie release so that further timecoding can be done.

Movie filename
There is a filename check to ensure that the movie loaded for playback matches the one selected in the utility. This was causing some problems because there are instances of the movie name being mistyped. I updated the utility to allow for the movie information to be updated from the enacted activity timecoding page.

CD/Label information
When an activity is timecoded using the JS functionality the CD label and movie filename are updated automatically. This was not happening if the information was being entered manually. I updated the page so that manual entry would result in the movie information being updated (based on the currently playing movie if available). If not current movie information is available it will maintain the previous movie information.

In addition, I made it so that any timecoding would result in the text "Save to update information." would be placed in the label/filename table cell.

Friday, July 07, 2006

Web Site Updates: 2061 Connections Index

I updated the index of 2061 Connections issues. It now lists the issues as well as the articles from those issues. The current page is static, but it may be more benefitial to create a database-driven page since MK wants to possibly provide more than one view (such as an alphabetical list of articles).

Mass Mailing: Workshops Notification, Summer 2006

A Workshop Schedule Notification was sent out on 07 Jul 2006 4:24 to 2628 E-mail Alerts users and 13 users of the one-time notification feature.

Usage Analysis Options

Webtrends can accept custom definitions of search engines. I had previously set up the program to recognize the AAAS search. However, I remembered today that when AAAS updated the search engine I forgot to update the Webtrends search engine list. This is mostly a reference note for documentation. The settings are found in the program options under the "Search Engines" area.

Search Engine Name: AAAS Vivisimo
Identification Strings: vivisimo.aaas.org vivo.aaas.org
Keyword Indicators: &query ?query

Wednesday, June 21, 2006

Web Site: Indicating indexible content

I had been thinking that it would be nice if the navigation on our pages did not affect the overall ranking in search engines. Particularly the AAAS search engine, which doesn't do so well when the search terms include words from the navigation. Some kind of container tag or attribute that could be used to say "don't look at this" to the search engine spiders.

Supposedly a method was floated a few years ago, but nothing came of it. In all likelihood the major search engines have already come up with methods of decreasing the weight of content that is reapeated often on an individual domain.

To put a final nail in the coffin of this idea, AAAS updated their search engine this year and it appears to do a much better job than it used to.

Tuesday, June 20, 2006

IERI R3: On PEAR and Code Separation

PEAR Components

I was trying to decide if I should utilize some of the components available via the PEAR library. Some of these components are under constant development and so would provide a significant source of well-written and debugged code. Specifically, there's a authentication/permission module that might integrate nicely with the utility. Upon further thought I decided that for now I'm going to try and do a lot of the coding myself (except for the occasional module). This is my first major PHP application and I feel the best way to familiarize myself with language is to do the programming as much on my own as possible. By taking my time to develop the code myself I should be better able to evaluate third-party code in the future.

If future revisions of the utility are created it may be benefitial to look at incorporating PEAR components where possible.

Code Separation

On another note, I was thinking about the separation of processing code and user display. The more separate the two aspects of an application the easier it is to update either one. IERI is one of the more complex applications I've developed and it's a bit difficult to determine how to best achieve the separation. I've not gotten very far in the current development (authentication completed) but I can already see how even something so simple as authentication could be utilized as a separate component from the user interface for user login (allowing authentication via JavaScript, for example).

I do plan on attempting to separate the code from the display as much as possible in anticipation of AJAX-oriented development down the road. I guess one of the questions I have right now is how much extra time will be required to rewrite an integrated page into a separated one. I'm hoping that any pages I develop singularly will be fairly easy to upgrade if I utilize function calls and other methods of extracting the logic from the normal processing flow.

Web Site: Spanish Content Update

I made a relatively minor update to the Spanish content on the site ... added a lang attribute to identify the content as being in Spanish. I don't expect it will make much of a difference to users of the site, but it's good web etiquette. My one hope is that search engines will find it useful.

I should spend a little time making sure any English text in the Spanish documents is appropriately identified.

Wednesday, June 14, 2006

Webtrends irregularity

Webtrends died again on some log entries ... see below.

2006-05-24 17:07:19 W3SVC48143651 WIZZLE 198.151.218.130 GET /publications/articles/psl/pdf/fgs_sacramento.pdf - 80 - 206.15.252.158 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+yie6;+.NET+CLR+1.1.4322) - http://search.yahoo.com/search?p=Teens-Sacramento%2C+CA-Science+programs&sm=Yahoo%21+Search&fr=FP-tab-web-t&toggle=1&cop=&ei=UTF-8&vm=r www.project2061.org 200 0 64 262401 587 6734

2006-05-24 17:07:24 W3SVC48143651 WIZZLE 198.151.218.130 GET /publications/articles/psl/pdf/fgs_sacramento.pdf - 80 - 206.15.252.158 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+yie6;+.NET+CLR+1.1.4322) - http://search.yahoo.com/search?p=Teens-Sacramento%2C+CA-Science+programs&sm=Yahoo%21+Search&fr=FP-tab-web-t&toggle=1&cop=&ei=UTF-8&vm=r www.project2061.org 200 0 64 327937 587 4750

2006-05-24 17:07:27 W3SVC48143651 WIZZLE 198.151.218.130 GET /publications/articles/psl/pdf/fgs_sacramento.pdf - 80 - 206.15.252.158 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+yie6;+.NET+CLR+1.1.4322) - http://search.yahoo.com/search?p=Teens-Sacramento%2C+CA-Science+programs&sm=Yahoo%21+Search&fr=FP-tab-web-t&toggle=1&cop=&ei=UTF-8&vm=r www.project2061.org 206 0 0 161566 695 3390

Friday, June 09, 2006

Mass Mailing: 2061 Connections - May/June 2006

The May/June 2006 issue of 2061 Connections went out on 9 June 2006 at 4:20 PM to 3910 recipients.

Friday, May 19, 2006

Scripting: File Queue

I created a sister script to my file queue script that allows anyone editing a file on the development site to queue that file for upload to the live site. This is should ease the development process a little as it allows the communications staff to create/edit a page and queue it for upload without any involvement from me or the rest of the technology staff.

In order to make the script as easy as possible to use I made some modifications.
  1. The script checks the referrer page to determine what file to upload. No user input necessary.
  2. I created a loop to ensure that the destination directory(ies) exist so no copy errors are encountered.
  3. I added a RegEx that filters the body for image tags, grabs the src attribute, and uploads those images as well.
I copied the folder creation updates back to my original script so that I don't have to worry about creating those prior to uploading a file.

I may make a further modification to the original script that uses a RegEx to grab all externally referenced files. These would then be presented back in a selection list to the user. This will make it easier to queue a page because I won't have to queue each individual element of the page separately.

AAAS: Taking Over

With KG (AAAS webmaster) leaving I've been asked to take over some duties for her temporarily, mostly just posting a news item here or there. The process isn't too tedious and shouldn't take up too much of my time once I've done it a few time.

One interesting aspect of the job will be to update the RSS feed. I've not played with RSS much beyond using them through My Yahoo!. There are a number of feed editors out there but I found two that are a) free and b) easy to use. Of course, the output they generate differs in one significant way, the content is entity-encoded in RSS-W but surrounded by a CDATA delimeter in RSS-B. Both appear to be valid methods of creating an RSS 2.0 document. I like RSS-B better but it does not use the same formatting used up until now. After talking to KG, though, I think I'll use it as she seemed to think using CDATA was more SGML-y, meaning it was likely more XML-y.

References:

Wednesday, May 10, 2006

Scripting: "Print this page"

The "Print this page" functionality I developed for the Web site recently broke with the introduction of a new template. The main reason was my reliance on a regular expression to parse the document for the sections that would need to be output on a printer-friendly version of the page. The new template was missing text I had been using delineate these sections.

Unfotutnately, to accommodate the new templates (and any future additions) would have required writing a different regular expression for each template because there was no other text delimeters unique enough to work for all templates. I like to keep my programming as generic as possible so I decided to use the HTML::Parser module to pull the various sections from the document. This particular object will work fine for all pages using the AAAS template since I can rely on tags common to that template rather than text specific to any sub-templates being used.

It took a bit of trial and error to get things to work but I think I'm finally starting to get the hang of how the module works. The basic procedure is to find a tag of interest such as a div tag with the attribute id="contentBox" (which contains the document content) and then add handlers to the parser to grab all text contained within that element. You can add and remove handlers at will as the parser runs, making it easy to catch blocks of text in one pass.

The only thing I'm not sure of at present is if it's possible to capture sub-blocks easily. Thinking about it briefly, though, leads me to think that by assigning the desired output to a variable and then parsing that variable for the sub-blocks would be a feasible method.

Tuesday, May 02, 2006

Scripting: Sending e-mail

I was searching around the Web trying to find some good information on e-mail line length in relation to IERI R3. I much knew that a line length of 78 was the max, but I wanted to find out if there were better guidelines as to the formatting of e-mail. Dan's Mail Format Site has a good article on line length. Something new I learned is that quoted-printable encoding isn't necessarily the best method for avoiding line-length display problems in e-mail clients. quoted-printable is more important from an e-mail transmission point of view by keeping line lengths within the appropriate limits. However, if the e-mail client doesn't reflow the message after removing any soft breaks created by the QP encoding then long lines will still be a problem. the format=flowed Content-Type parameter (which I had been using for the mass mail templates ... though without a full understanding) is more important from a display point of view.

I'll need to do a little more investigation and testing to determine how best to format messages, particularly with regards to client display. While I was looking at this in terms of IERI R3, the information I found will require some modification to the mass mail scripts as well.

Monday, May 01, 2006

Scripting: AAAS Site Template & IE7

I've been testing out IE7 ever since Microsoft released the public beta. There are a few things that need to be fixed with the JS/CSS code I've developed. I hope to address these issues in the next few weeks.

On top of these problems I noticed there were issues with the main AAAS template. The left-side expandable menu navigation was failing horribly and completely unusable. Luckily the fix came to me easy enough as I noticed one of the library functions, the publicly available lib_bwcheck (which hasn't been updated since who knows when), was not detecting IE7 correctly. A quick fix and everything was working like normal.

I've passed the information on to KG.

Tuesday, April 25, 2006

Web-based document copy/print/save protection

We recently needed to publish some documents which required some fairly strong restrictions on usage (viewing only ... in other words no printing, no copying, and no saving). Even though I find these restrictions to be a little draconian for something destined for the Web, I always try my best to accommodate any requests.

Initially I had posted the documents in PDF format since it's easy and has built-in security against printing and copying. I had also thought you could prevent saving, but that turned out not to be the case, and something I didn't realize at first. Much fuss was made regarding this "problem" because we had signed a contract outlining the copy/print/save requirement in return for our usage of the documents.

Upon further investigation I found another problem in that the security features of PDF are voluntary to the reader. Naturally Adobe's products follow the restrictions, but it's possible that other products out there may ignore the restrictions.

Then there's always the fact that the user can just save the original file from the link to it (right click + "Save link as ..." from most Windows-basec browsers).

Many issues to consider with regard to this "no saving" clause. I think the tech group should have been consulted regarding the final contract. I don't think we would have ever signed off on that clause because it's the nature of the Internet ... in order to view something stored remotely it has to be copied locally. And even if you could prevent disc storage the document would have to be placed in memory on the local machine. So "no saving" is really a difficult problem to tackle.

But tackle it I did ... by moving to Flash. With Contribute 2 Macromedia provided a nifty little program called FlashPaper. FlashPaper is like a Flash version of PDF (or more accurately PDF lite). It's missing some featuers that make PDF useful but it pretty much serves the needs for this particular problem. I was able to make the document unsavable by using a shell Flash presentation to load the FlashPaper document. Now if someone attempts to save using the web browser's save functionality all they'll get is the shell Flash presentation.

One annoyance with this method is that the FlashPaper navigation bar no longer floats on top of the document. Viewing a FlashPaper document directly the toolbar maintains it's size and adjusts the number of on-screen controls available based on the width of the display window. If the presentation is resized with an embedded FlashPaper the toolbar resizes like any ordinary Flash object. Worst of all the toolbar buttons do not scale up very well.

That's just an annoyance, though. One important feature is lost by moving to Flash and that is the accessibility enhancements provided by PDF. When I scanned in the pages (as images) the PDF did some OCR magic so that the text of the documents was actually accessible to screen readers (awesome!). This is an incredibly useful feature that requires further investigation. At any rate, converting to Flash killed the OCRed text.

References:

Update 2006-05-09:
Note to self, pay more attention to what's going on around you. Apparently I did get a copy of the "contract" at some point and may have even signed off on it (though it's hard to say because it came via hard copy and has no date stamp). I'll keep this in my files as a sample of a bad contract.

Friday, April 21, 2006

Mass Mailing: 2061 Connections - March/April 2006

The March/April 2006 issue of 2061 Connections went out on 21 April 2006 at 2:13:20 PM to 3888 recipients.

Thursday, April 20, 2006

Scripting: Browser Security Updates

Browser makers have been focusing a lot on security with their latest updates in an attempt to decrease the exposure of their users. Mostly this is a good thing, but some of the changes are starting to interfere with development of the IERI utility. Specifically, all of the major browsers 1 now no longer allow web pages to access local files except through approved user controls such as the file upload form field.

This is a major problem for the utility because we have been relying until now on locally viewed video as a means of decreasing cost and management time. The quickest way of resolving the problem is to switch to streaming for video playback, but this assumes the user has access to a high-speed Internet connection. Plus it would require a significant investment in infrasturcture (storage, connections, and streaming server software) and administration of a movie database.

Another solution is to use signed JavaScript, which I have previously investigated and which has problems of it's own.

A final solution I have in mind would be to create a locally installed HTML application, which would cost less in terms of software but require a significant time investment to develop. This method would rely on a remote database for storage but would run the shell of the utility on the local machine. This approach might also run into security issues, however, so it would require a bit of investigation to determine its viability. Plus updates would be more difficult to implement.

At present I don't know of any other resolution, but I'm keeping an eye on relevant discussions to see if anyone else finds a solution.


(1) Starting with the following versions (not authoritative): Firefox 1.5, Safari on OS X 10.3, Internet Explorer 7

Wednesday, April 12, 2006

IERI R3: Sending mail via PHP

The mail functionality included in PHP by default is pretty light. There's not much functionality for making it easy to produce e-mail messages that conform to standards. Just one example is the lack of wrapping long lines in the body. I found a function online that will format text as quoted-printable, though it had to be modified to take into account line length requirements. I think it can probably be simplified a bit, but my concern at present is making the system functionally complete ... code efficiency, scalability, and performance will need to wait for another day.

Note: There's also a function for quoted-printable encoding in the comments for the mail() function, but it also does not take into account line-length requirements.

The built-in mail() function appears to be susceptible to manipulation by user-submitted data for the purposes of SPAM or other causes (see the comments for the mail() function). While this isn't a big concern with IERI it's something that needs to be kept in mind if I plan on using this function in the future. I found some information that will provde useful in weeding out SPAM hacks such as detecting newline characters and e-mail header text.

Unfortunately the e-mail functionality I've developed based on mail() and other functions is pretty limited in scope. The PEAR project appears to have a fairly full-featured mail package. I don't think it will be necessary to install for use with IERI, however, as e-mail is not the core purpose of the utility. Unless additional functionality needs are required down the line I'll probably stick with what I've developed so far.

Wednesday, March 29, 2006

SVG: Authoring Guidelines

The authoring guide posted at jwatt.org indicates that using the style attribute on a tag should not be done. No reason is given beyond the fact that SVG-Tiny doesn't support styling. I'll need to do some more research to determine if this is truly valid guidance. I don't think we'll be moving away from CSS since there is much common styling across the elements of the Atlas maps ... but it may mean we should investigate using classes or IDs.

SVG: Firefox text rendering

I was working on getting some consistency between IE and Firefox when viewing one of the Atlas SVG maps being developed and ran across an odd problem in Firefox 1.5.0.1. There appears to be some kind of hard limit on the minimum size that can be assigned to a font via CSS. I haven't done full testing so it's hard to describe the exact nature of the problem. However, as I was working on the map Firefox would not go smaller than 10px. To get around the problem I had to increase the size of the map (in Illustrator source) so that I could size the text appropriately.

Tuesday, March 28, 2006

IERI: JavaScript escape() function

To work around some problems I modified the utility at one point to ask for the location of a movie each time it's loaded. The main reason being the difficulty in writing any kind of JavaScript that works cross-browser/cross-platform that can determine the location of the movie automatically. To do this I used a standard HTML form with a file input. Upon submission I grab the value of the file input element and redirect to a page. In order to avoid URL problems I was using the escape() method to URL encode the filename.

Unfortunately, the escape() method does not encode the plus (+) character which, though rarely used, has been encountered in a filename. A little research led me to some new methods available as of JavaScript v1.5. I've modified the script to use the encodeURIComponent() method if available and the escape() method as a fallback.

References:

Thursday, March 16, 2006

IERI: Revision 3 Development

I've begun development of the next revision of the IERI utility. Currently I'm working on the basic shell of the utility and the login system. The process is going a bit slow as I'm spending quite a bit of time trying to think ahead about how the choices I make now regarding programming and file structure will affect things as I get further along in development. Plus I want to be sure to spend time documenting the code as I go as much as possible.

I'll try and keep notes of interesting and useful information on the blog while I work. Here's two to start.

1) I'm curious as to what the most appropriate HTTP status code would be for redirecting to a log in page. The closest status code would be the 401 but that only allows for HTTP authentication using the WWW-Authenticate header field. There doesn't appear to be a return status that would reflect the desire to authenticate via HTML forms, so for now I guess I'll stick with the standard 301.

2) I'm pretty new to PHP, but the language is easy enough to pick up considering the extensive work I've done in VBScript, JavaScript, and even perl. One interesting feature I've come across which should prove handy is that error messages normally output to the browser can be suppressed by appending '@' to the beginning of a line. The main use I have in mind is for including via PHP regular HTML that the page can live without (such as header and footer content). If I'm planning on using this, though, I should probably also investigate methods of checking for errors in case I run into an instance where a page doesn't work as expected but no errors are returned because they are suppressed (and I've forgotten that I had suppressed them). Hopefully I can create a global setting for the application in the common.php file.

References:

Tuesday, February 28, 2006

SVG: Compatibility between Adobe and Firefox

I did some compatibility testing for the SMS project. Follows are notes related to that testing.


svgDocument does not appear to be part of the W3C spec (from what I can tell). It is the equivalent of the document object, which I would recommend using from now on. Existing code can be made compatible with Firefox by adding the following line of code:
if (typeof(svgDocument) != "object") { svgDocument = document; }
Trying to declare var svgDocument will return an error in AdobeSVG because svgDocument appears to be a defined constant in AdobeSVG. Luckily JavaScript is flexible enough to still recognize this as a global variable.

While Firefox supports the getBBox() method, using it onload does not work as expected. The release notes for v1.5 indicate this is a problem with the current implementation of the load event. I was able to work around the problem by declaring the global variables that rely on getBBox() but not setting their values. The values are set in a function that's called through a setTimeout().

This setup requires that the init() function be removed from the SVG document's load event and called after the global variables have all been populated. As a fix it's not particularly elegant ... but it seems to work. I'm not entirely sure it's worth incorporating this one; I would this issue will be fixed in a later release.

See:

I believe that Firefox rewrites documents internally when rendering. Apparently the transform attribute is rewritten such that translate(0,0) is rewritten as translate(0). A way to work around this problem would be to write the function using regular expressions to parse the transform attribute. To get around this problem temporarily I rewrote the translate as (1,1).

I think the contextMenu object may be proprietary to AdobeSVG. At the very least Firefox does not recognize it. Errors can be avoided by using the following to determine if contextMenu exists prior to performing options on it:
if (typeof(contextMenu) != "undefined") { ... }

Firefox 1.5 does not support the selectSubString() method.

Despite many examples that have only two parameters in the setProperty() function it requires three. The third parameter can be null or an empty string ... but it has to be there. To fix just add a third parameter ("") to each instance of setProperty().

See:

Firefox diverges from the SVG spec with regards to the font-size style declaration. The spec indicates that a unit identifier (px, pt, etc.) is not necessary. However, Firefox chokes on a font-size declaration that does not have a unit identifier. The fix is easy enough ... provide an identifier for font sizes.

Thursday, February 23, 2006

Mass Mailing: 2061 Connections - January/February 2006

The January/February 2006 issue of 2061 Connections went out on 23 Feb 2006 at 4:29:54 PM to 3818 recipients.

Wednesday, February 15, 2006

Mass Mailing: Workshops

The workshop notification went out on 13 Feb 2006 at 3:51 PM to 2457 recipients.

Friday, January 27, 2006

Scripting: ActiveMenu script update

I've made a few modifications to the ActiveMenu script, mostly to the internal processing and order. A quick overview of the changes:
  • Added two new global variables that affect how the ActiveMenu script operates. These variables required minor modification to the activeMenu_init() function: the event assigned to the headlines in the headline list now uses the activeMenu_strEvent variable for the event type; the active item is now set on page load using the index in the activeMenu_intActiveItem variable.

  • The activeMenu_init() function was modified to enable a more accurate calculation of the story box height. This functionality was moved to the bottom of the script after the rest of the menu intialization has been completed. The calculation is also only done if the activeMenu_bolResize2Fit variable is set to true.

  • The activeMenu_init() function was changed so that the initally viewable story is selected and made visible after the menu is fully initialized. This is done by calling the activeMenu_change() function and based on the value of the activeMenu_intActiveItem variable. This was in part necessitated by the change to the height calculation functionality.

  • Modified the activeMenu_change() function so that it only requires a single parameter, the zero-based index of the story to highlight. Previously the function had to be passed the class associated with the story and the id associated with the headline, but since those values are set in script it was not really necessary as the index is sufficient to identify both.

Notes:
1) IE seems to be producing a height calculation that may not be entirely accurate on the individual nodes of each story. This only seems to affect it, though, when a floated image isn't used on the first DD. This is something that will need additional investigation.

2) One piece of weirdness I encountered was IE calculating the wrong height unless an alert() was inserted in the height calculation code. I suspect that IE may have been too slow to update the offsetHeight value after a DT/DD was set to display='block'. The alert may have been enough of a stall to allow IE to make the appropriate calculation.

I decided to change the method used to do the calculation after discovering this flaw. Now this section of code uses the previously developed activeMenu_change() function to manage the display of each story. The list nodes are then parsed for the storySelected class.

Tuesday, January 17, 2006

Scripting: VBScript and function calls

I noticed some weirdness with some of the data being placed in SQL Server for the mass mailing list. If data entered through a web form contained an apostrophe (such as O'Brien) then the records in the database would duplicate the number of apostrophes on each insert after the initial one (O''Brien, O''''Brien, etc.).

Immediately I supsected that this had to do with the SQLEncode() function provided by Microsoft in the iasutil.asp file. I was correct, and the problem was beginning to look supsiciously familiar (I'm pretty sure I've seen it before, though I'm not sure I actually took the time to resolve the problem). I took a quick glance at the function to see if I could determine the source of the problem.

Basically the problem boils down to the way that VBScript handles the passing of variables to functions. Some languages allow passing of variables by value (byval) or by reference (byref). VBScript has this functionality, but by default passes variable byref. In effect this means that any modification of the function variable will also modify the original one passed to the function.

For example, take the following code:
Dim intGlobalVar
intGlobalVar = 1
Function addOneToNum (intFuncArg)
intFuncArg = intFuncArg + 1
addOneToNum = intFuncArg
End Function
Response.Write(intGlobalVar)
Response.Write(addOneToNum(intGlobalVar))
Response.Write(intGlobalVar)

The first Response.Write() prints out "1". The second Response.Write() prints out "2". While you might expect the third Response.Write() to print out "1" it in fact prints out "2" because intGlobalVar was passed byref to the function and then modified.

There are two methods of solving this particular problem
  1. Specify byval in the function argument list
  2. Reassign function arguments to variables local to the function
I updated the SQLEncode() function so that the arguments are passed byval. I also updated the variables so that they have more uniqueness (to avoid possible confusion).

(Note: In JavaScript simple types are passed by value while complex types are passed by reference. There is no way to modify this behavior ... though I have seen indications that a future version will have this capability.)

References:

Thursday, January 12, 2006

Design: Site Updates

The web oversight group recently had a discussion about making modifications to the AAAS template header so that the end user would have an obvious indication that they were viewing content related to Project 2061. This was sparked in part by the significant modifications made by the annual conference people. That and our dislike of the default header which we seem powerless to influence (but at least the image has been slightly improved).

In the end we decided upon a rather low-key update to our template that provides the end use with the kind of information we think helps in orientation ... a breadcrumb. It's basically a hanging box off the header that provides some basic orientation. I'm not sure exaclty how it will evolve, but for now it contains the text "AAAS » Project 2061" which should be sufficient at least initially. I believe that as we further refine the site structure we'll work on adding additional levels.

Friday, January 06, 2006

Mass Mailing: 2061 Today - Fall 2005

The notice for the Fall 2005 issue of 2061 Today went out on 6 Jan 2005 at 4:23 PM to 1328 recipients.

Scripting: Mass Mailer Updates

When I updated the mass mailing script to fix the sender address template replacement I only updated the code for the HTML template. Luckily I noticed this prior to sending a text-based mail out today and updated the code for the TEXT template.

Webtrends irregularity

Webtrends crashed once again. I had previously thought it was a specific set of characters that cuases the problem. Namely "{}" in the user-agent field. But after this crash that doesn't appear to be the case. A few more problems like this and I may be able to pinpoint the problem. For now, though, I'll just keep winnowing it down until I find the culprit.

The offender this time around ...

2005-12-07 07:32:30 W3SVC48143651 WIZZLE 198.151.218.130 GET /publications/sfaa/online/Chap13.htm - 80 - 202.133.13.234 HTTP/1.0 Mozilla/4.0+(compatible;+MSIE+5.01;+Windows+NT+5.0) - http://www.google.com/search?hl=en&ie=ISO-8859-1&q=effective+learning+process+ www.project2061.org 200 0 0 32641 571 27719

Thursday, January 05, 2006

HTML Coding: CITE Revisited

In an earlier post regarding the <cite> tag I noted that it seemed to me that I was using the tag incorrectly when using it to delimit book titles.

For some reason this snuck back in my head and I spent some time yesterday trolling the newsgroups and web discussions. After a lot of reading I've come to the conclusion that there does not seem to be any kind of general concensus on how the <cite> tag is used and how a book title that is mentioned but not referenced should be marked up. Most people that even think about stuff like this seem to be of the opinion that any of the methods is fine as long as your intended audience gets the meaning.

No doubt communications would like to keep using AP style formatting. I think I'll stick with <cite> and use class names to further categorize if necessary.


References:

Blog Archive