Mass Mailing

Sent out a mailing regarding next year's workshops.

Total sent out was 2375. 2348 to web site list, 21 to workshop schedule update one-time notification, 3 more to 2061 staff not on the other lists.

The bounces weren't nearly as bad this time as last (thanks to my constant efforts to keep the list clean). There were quite a few "Out of Office" and "Mailbox Full" bounces, but that's to be expected this time of year. So far only two have been bounced because of SPAM filters, which is much better than last time. Of course, it'll take a few days to get all the bounces, but so far a good start.

HS Bio Textbook Report

Complete. Pretty Much. I mean, no doubt a few edits will need to be made. And there are a few PDFs that need to be added. But otherwise we're pretty much done.

Some of the templates ended up being a little more complex than I had initially planned (extra editable regions, options regions, etc) but overall I think it's worked out fairly well. Of course, the templates aren't flexible enough to use for the remainder of the reports (to be released who knows when) unless I add optional regions, etc ... but I think I'll just go with new templates. They won't be drastically different from the current ones so the additions should be pretty easy to work up.

I have some reservations about our "help text." There's no definitive indication of what happens when the help icon is clicked beyond the text just appearing. It's possible that a less alert user could miss this. Also, though it's not too intrusive, the quesion/circle icon does interrupt the document flow somewhat (though I have seen similar schemes used on other sites).

I was thinking of trying another method such as linking the text itself but modifying the link to be the same color as the normal text with maybe a light underline and have the cursor change on mouseover. There are two problems with this method, however. 1) It would interfere in cases where normal links were needed. 2) It might be a little too "under the radar" for the average user to notice.

I had another thought about a floating "mini-reference" that could be used to provide the additional information. Unfortunately it would require a significant amount of time to program. Time that I'm not willing to put in at this late in the game (unless specifically requested to do so).

I added a new function to the reports, a "toolbar" that's added to the top of a page with a link to print the document. The script is fairly intelligent and requires no coding to use. All that's needed is for the script to be included on the page. The main drawback is that the script only works on modern browsers. Here's a quick run-down:

• Using the addEventListener method an onLoad function is added to the document that adds the toolbar
• Not all browsers support the window.print function, so a test is done for it's availability before any other steps
• The function uses DOM scripting to add the toolbar, so a second test is done for the getElementById function
• The toolbar is added using addNode function to create a div that is absolutely positioned at the top of the page using styling

As with everything else on this project, Nav 4 is not supported for this functionality. Though it supports the window.print function it does not support DOM scripting. IE for the Mac is also not supported since it does not support the window.print function.

HS Bio Textbook Report

Finished some final edits requested by CT. I just need to finish the indicator show/hide for the instructional analysis reports, which should be easily completed tomorrow.

There might be a few last-minute edits/fixes, but we're pretty much ready for to publish the reports.

Checking the locks

Nothing spectacular today. One exploit scan that was from a smaller range. I reported the scan, but don't expect much of a result since the offending owner is in China.

HS Bio Textbook Report

Staff want the indicators to be available on the instructional analysis page via show/hide functionality. I've taken functionlity that I had written previously and modified it to be a little more generic. I'm still not sure if the method I've chosen to provide the functionality requested is the best one, but I think it'll do for now. I still have to document the code, which I guess I'll get to tomorrow.

I'm running into two problem I'm not sure how I'm going to address just yet.

One, the styling for the text around the show/hide area diverts from the requested styling. I know why it's doing this (margin stylings and those cursed rules for collapsing margins), but I'm not sure how I'm going to address it just yet. It's a fairly inocuous problem ... a little extra space below the item, but it's one that seems to be a sticking point for the comm staff. There are a number of ways to work around this problem, I just need to find the one that requires the least amount of fudging.

Two, Mozilla doesn't want to play nice with the script. It's not stopping propagation and thus following the back-up link specified for compatibilty with non-js browsers. I'm sure I'll figure out what's going on eventually.

Update 2004-12-23: Fixed the problems I referenced above.
One - since the styling was being affected by adjacent margins I ensured that the margins were not "touching" by adding a 1px padding to the bottom of the container div. IE had no problems to begin with since it does not implement collapsing margins correctly. Problem solved.

Two - the DOM event model doesn't allow for a return value from an event added using the addEventListener method. IE had no problems because 1) it ignores this rule and 2) you can always use the event's returnValue property to provide a return value. The DOM does, however, include a method to prevent the default action from occuring, preventDefault. Problem solved.

Update 2004-12-23: No sooner did I think I'm done than another problem is encountered. Looks like Safari doesn't support the preventDefault method. I'll have to do some research next week.

Update 2004-12-27: After a bit more research I found plenty of references indicating that Safari does in fact support preventDefault. Safari itself indicates that it supports the method, but I have been unable to get it to not follow a link by using the method. I'll probably do a little bit more research tomorrow, but for now I've coded up a work-around that is acceptable (if not desirable). When Safari is detected the link is modified to be "javascript: return false". It may be ugly, but it works.

Site Design

There's a lot of work to do to prepare for the upcoming site re-organization, not to mention all the work required to complete the site re-org. I spent a little bit of today modifying the site style so that two-line sub-items on the left hand navigation menu are possible. The previous style was ok for single-line entries in the navigation, but we have a few that are likely to wrap.

The previous style caused problems because the same amount of space was between soft returns (wrapped lines) as was between hard returns (br tags). Everything kind of ran together. I modified the style so that the links in the menu are treated as block items, have a smaller line-height than before, and a little space above and below to set each individual link off from it's neighbor. I also used display:none for the br tags so no extra space was present (I kept those in for browsers that don't handle display:block ... thus hiding them using styles otherwise).

SVG Plug-in OS-Browser combos

FM had me test the various OS-browser combos to see where the Adobe SVG plug-in works. Of course, we don't have a fully featured test lab so I had to make some estimations based on what I observed and what the documentation for the plug-in states.

Overall I'm not sure how accurate the testing was. I think it better to know what platforms are being targetted/supported and test for those. Still, there's only a limited amount of testing we can do and many variables that can affect the functionality of a plug-in (OS version and what updates are applied, browser major/minor version, etc).

HS Bio Textbook Report

Just about done with the styling of the reports. Today I modified the CSS to use @media declarations so I could include both a print and screen style for each document. Shared styles are currently located in an @media print, screen ... but this causes Nav 4.x to ignore the styling. Not a horrible result by any means, and Nav 4.x is a fairly small percentage of our regular traffic these days. Plus it gives me a good idea of what the reports look like without any styling at all.

I still need to check on the Mac to make sure no major problems are encountered.

One last element to add to the instructional analysis template, the indicator text. I have some ideas I want to toy around with. I also thought of a decent compromise on inclusion considering the variability of whether the indicators are addressed from one report to another. I'm waiting to hear back on the final decision in the meantime.

Update 2004-12-21: Finished testing on the Mac. Looks good on Firefox, Safari, and Mozilla-based browsers (Navigator 6+). IE works about as well as Nav 4.x. I have no intention of addressing the issue, however, since IE is a dead browser on that platform (unless we get complaints about it).

Checking the locks

Checking logs, etc. Nothing worth noting.

Site reorganization

I'm in the process of moving files around on the site to better enable automated mapping tools. It's a bear of a process and I think a virtual re-org needs to take place as well. Our content has always been a bit difficult to weed through to find the gems. The physical re-org will help us get a better feel for the documents that are on the site. But in order for our site visitors to be able to find things I think we need to do a better job of grouping and highlighting.

I'll need to spend a few days thinking about this. Maybe I'll be able to better elucidate the thoughts in my head after I've begun working with the documents.

2005 Workshops

New workshop information is ready to go up. Nothing special here, just a few HTML and PDF pages. Still, it's a lot of work to ensure that all the information is correct.

We'll be sending out a note to our e-mail alerts list announcing the new dates soon.

Checking the locks

A few exploit scans on the system, most of it fairly typical. No IP blocks small enough to make it worthwhile to report.

The reporting from last week on the unique worm that was parsing our Web site never received a response. Either the owner of the source IP block was complicit in the activity or doesn't really care. I suspect the latter and that they don't even check the address used for their whois information.

Firefox bug?

While working on the HS BIO reports I've run across what appears to be a bug in Firefox. Actually, it looks to be a bug in the gecko rendering engine as the bug is present in both Firefox and Mozilla 1.8A5.

The gist of the bug is that the rendering of an block element (div, blockquote, etc) that has no padding is done incorrectly when that element contains another block element that has a margin (h1, p, or a block element that has a styled margin). The top/bottom margin of the inner element appears to be rendered as the top/bottom margin of the outer element.

I'm working on some test cases so I can submit the bug to bugzilla. I'll need to search through the current bugs first, though, to ensure I'm not duplicating efforts. Perhaps I'll find the time on Monday to complete my introduction to the open source community.

Update 2004-12-13a: Posted a bug report ... an experience I'm a bit anxious about. I've not done it before so I hope I get a good reception. I also hope I haven't reported a bogus/duplicate bug. That would suck.

Update 2004-12-13b: Nevermind. Got the smack-down big time. I need to do more research next time. :(

Update 2004-12-13c: Upon further review I realize how I misjudged the gecko rendering of my code. It all goes back to margin collapsing. Probably one of the more difficult parts of CSS to get a handle on.

Nothing like being humbled by the masters to make you realize how much you still have to learn. I spend too much time assimilating just enough information to get the job done these days (the curse of the JOAT I guess). Or perhaps I just attempt to assimilate too much information. I should spend a little more time delving into the more arcane aspects of some of the technologies I use. Time to look into classes or certification.

I don't think I'll be submitting a bug report anytime soon. There are plenty of people more capable than I working on that kind of stuff. I'll need to keep reminding myself of the old saying about the dangers of having just a little knowledge.

HS Bio Textbook Report

I think I'm finally making some headway on the templates. I'm beginning to feel like there will be too much variability, thus decreasing the value of templating some of the elements of each page. Unfortunately there's only so much I can do with the various requests. If I spent some time on it I'm sure I could break things up a little more so that there are fewer templates overall. It's too bad I'm spending too much time on this as it is, otherwise I might give it a shot.

After spending days trying to decide on a navigation structure for the actual reports I finally decided to KISS. A more robust navigation system would be nice, but a bit out of place at the current stage of the reports. If necessary I should be able to go back and rework the navigation, but it's looking to me like the system CT and I are discussing may be good enough for the long haul.

Webtrends irregularity

Something in one of the log files was causing Webtrends to crash during logfile import. I couldn't find anything too out of the normal, but I was able to identify the section causing the problem. I probably could have narrowed it down all the way, but I didn't feel anymore time was warranted.

Checking the locks

Had a worm scan from an elementary school in Korea (at least, that's what the whois appeared to indicate). It's always nice to be able to contact a small group like that. The large ISPs are pretty much worthless when it comes to investigating abuse reports.

Update 2004-12-08: I found an attack vector I hadn't seen before. I got lucky, though, because the attack showed up in the usage reports. Since I don't parse the server logs I wouldn't have caught it otherwise. Luckily it appeared to come from a relatively small block of addresses. Maybe I'll see some action on it.

The attack appeared to be automated, otherwise I would have expected a more concerted effort. The attack appears to crawl sites looking for ASP pages that use querystrings. I guess the assumption being that one of these sites will use the querystring as input for a SQL statement intended for a MS SQL Server backend. Not a wholly unlikely scenario. Once it finds one of these ASP pages the script requests the page again with a nasty little addition that creates a VBS file using a MSSQL built-in extended stored procedure. The additional querystring data uses pretty traditional SQL injection code that is URL-encoded. The encoded is only done once, though, and so is not caught by URLScan. A follow-up request attempts to execute the VBS, which downloads an executable (http.exe) from the same machine running the exploit scan to windows\system32. Finally, a third request deletes the VBS file. I didn't see any follow-up requests that attempted to execute the downloaded file. Most likely this is only done if the file is actually downloaded (the download request include the domain of the attacked machine).

The attack didn't work in my case for a number of reasons. One, access to the SQL server is established using an account that is fairly limited (and definitely does not have access to the master database where the built-in stored procedures can be found). Two, I'm as likely to use MS Access as I am SQL server. Three, a number of the scripts utilize the recordset object for data access, which I do not believe susceptible to SQL injection. Four, a number of scripts use an INSERT statement that, while not invulnerable to attack, requires more intelligent attack code that is likely to be found in an automated script.

One thing the attack has shown me is that I need to perform better sanitation of the incoming data. Though I doubt any automated script will ever cause any problems, a concerted effort on the part of a hacker would lead to a number of pages that could be used for SQL injection. This won't cause much of a problem with regard to stored procedure access for the reason mentioned above. However, stored procedure access is not the only means by which SQL injection can cause problems.

I'm tempted to give the script-kiddie the appearance that my personal server has been compromised so I can see what happens. I'm just not sure I really want to spend the time on it. Perhaps it'll be more fun to give the appearance that the FBI has been hacked.

I am not spam

Spent most of the afternoon sending off messages trying to convice people that we do not send spam. Some of the filters were way too strict (filtering on the words "contact us"). Most of them also filtered during the SMTP session, which is annoying because then you can't forward the caught message to the postmaster for review. Actually, that's probably not true. It's probably more likely that those people have not omitted their postmaster from filtering. I'll keep my comments to myself.

Of course, all this means I didn't even get to two of the things I wanted to work on today. Alas ... the trials of a webmaster (in the classic sense of the word).

IERI Utility

Modified the movie loading page of the utility. First, on the selection page the expected movie is now displayed. Second, when the user attempts to load a movie it is compared with what is stored in the database. If the two don't match the user is asked to try again. This should prevent people from choosing the wrong movie, but the fix is a kludge. I still hope to be able to do drive detection so that the user does not have to locate the movie each time.

FM wants to discuss the porting of the utility from ASP to PHP at some point. I still think it's a premature conversation, however, as I would like to complete another revision before we attempt the conversion. As much time as I have to spend on the utility these days it'll be a few months at the least before I've finished the next revision.

Mass Mailing

Didn't accomplish much today. Got a bit sidetracked with an ISP search and then a security discussion on my mail server list for home. I'd detail the discussion more in my home blog if I had one, but I haven't gotten around to setting that up yet.

Sent out the latest issue of 2061 Connections. Went to 3680 addresses. I think the list has grown a little. Got a good number back, though. And was caught by a few more spam filters than normal as well. I think some may be the same filters that have caught us before. I'll need to check and if we go through this on the next mailing I'll remove them. In the end it's not even a 1% return because of filters so I'm not particularly concerned. Still, it'll be nice to implement the automated system ... whenever we get around to building it.

There were a few returns as well, but we (or rather I) seem to have done be weeding out the bad addresses based on past mailings. Naturally there will always be a few returns each mailing ... but we seem to get more quota violations than bad addresses these days.

HS Bio Textbook Report

Having trouble coming up with a decent navigation scheme. It's unfortunate, but the content is probably too deep for the simplified left-side navigation. I'm sure if I spent some time on it I could come up with something better. Perhaps when the textbook reviews are unified under a standard architecture I can work on something like that. Ideally I should be able to put these in XML and use on-the-fly rendering. I think that would probably allow for some flexible navigation and (one of MK's long-standing requests) the ability to generate a single document for printing (in PDF format perhaps?).

Made some minor updates to the stylesheet so that I could use html header tags to indicate the document structure. Not many programs make effective use of html hierarchy, but that's no reason to ignore it.

Think I've got an idea for navigating the reports once you've chosen a book. A little show-hide action and simulated buttons. But I'll need a good "you are here" scheme.

Need to talk to CT about the templates. There are requests to have easy access to the indicators. One option is to further divide the template so that I can provide indicator information inline with the document (perhaps using CSS+JS to enhance the usability). Another option, use JS to profide on-the-fly information on the current indicator. I dunno, not all indicators are specifically addressed, so I'll need to find out what kind of consistency there is across reports (may need to implement optional regions to attain the desired flexibility).

Web Site Updates

New Atlas workshop information has been posted to the development site. Not sure about pricing yet, so I haven't finalized the registration forms. Also not sure about scholarships. Need to start working on PDF files for the workshop since they'll be requested.

2061 Connections is pretty much ready to go. Need to have communications review the articles one more time and send a sample newsletter for review.

Checking the locks

Looking through the log files nothing seems to be out of the ordinary at work.

On my home server I got the usual number of exploits. One little gem, though, was that one of those happened to be from a business with their own range. After a little digging I was able to come up with their domain. No web site to speak of so I'm not sure if the e-mail I sent to postmaster will get to anyone. Probably some new company that just got on the net. Seeing as they're already exploit scanning they'll probably need some help securing their network.

I was bad and did a little snooping. Not enough to gleam any information, and certainly not enough to cause any trouble. But really, I shouldn't be doing any amount of poking without permission ... especially from the office.

CRM

Finally talked with someone from AAAS tech. services about how they could help us implement a CRM system. Really I don't think they'll be of much help. I suspect the systems they would recommend to satisfy all our needs would be beyond our resources … they mainly seem to think in terms of commercial software.

I looked around for some open source options. While there are dozens of so-called CRM systems out there, most of them aren't worth the time to investigate further (underdeveloped, unsupported, too specific, and little development activity). I did manage to find a few that are worth investigating more, and I'm sure one of those can probably meet enough of our needs to be useful. Unfortunately, there's no real definition of what software like this should be and so the features available vary greatly from package to package. The benefit of open source is that I should be able to add the functionality that missing, I just hope that one of the projects on my list has a modular kind of setup. In the end I still think I could do better by creating the system from scratch … not to mention I think I'd enjoy the challenge.

I've also been waffling on the feature set requirements. I've thought about it some more and feel that a completely integrated system may be beyond our reach and more than we need. My current thinking is that we would do just as well two different systems providing contact management and list management services. It would be nice if they integrated, but I honestly don't think we'd get much use out of such a system in the long run.

404 Script

I added a "jsRedirect" querystring to the javascript redirect so that I could see what percentage of bad URLs were being automatically redirected. Initially I forgot to remove this if the 404 page is encountered again, so I've modified the script to do so. I also had to add some additional conditionals to avoid a few problems I encountered with the simplified structure. Still, I've not seen too many problems as of yet.

Newsletter Signup

Finally had a chance to add the print newsletter to the newsletter signup page on the site. It's now possible to subscribe to any of our newsletters (print or electronic) from a single page. I did a little script trickery to get things working the way I wanted. Basically, the script asks for your name and which newsletters to which you would like to subscribe. Then, once that's done it asks for your e-mail address, postal address, or both depending on which selections were made. The trickery comes in when moving to the "second" form. My validation script wasn't really designed to handle multi-page forms and so I had to come up with a way to satisfy the requirements of form validation and data processing. It was easy enough to set up but I certainly did not use the cleanest method to do so. I think maybe it's time to do some rethinking on the form validation/processing script I'm using. I should be better able to segment things so that it can handle a situation like this.

In the end, though, it seems to work just fine.

Mass Mailing

Made further enhancements to the mass mailer script. Now both text and html newsletters are converted to QP and sent using a file drop. This means CDO is no longer used. The biggest benefit is that now I don't have to worry about using a different method depending on the format, simplifying things a bit. I could probaby even simplify a little more. I think in the end this will give me a little more flexibility in customizing the newsletters. I've already done one customization where the user's e-mail address is now included as part of the body of the message. Not only will this help the subscriber identify what e-mail address they used to subscribe, it will also help me in cases where remote MTAs modify the recipient information (which is rare but still an occasional problem).

HS Bio Textbook Report

I'm pretty much done creating the base templates for the reports. No doubt there will need to be modifications, but my assigned tasks have been completed.

We wanted to use a floating menu similar to the previous reports, but I was never too happy with the script we used. Sure it worked fairly well across platforms, but without JavaScript it was useless. If nothing else I'd at least like the site to be usable sans JavaScript and CSS. I managed to create a pretty good replacement using a combination of HTML, CSS, and JavaScript. The basic menu is now just an unordered list. For Mozilla and other CSS2-capable browsers I used the "fixed" positioning style to maintain the menu location. For IE I've set it up to scroll using a much simpler script that is called by the "onScroll" event. So far works like a charm (I even set up the JS so that the menu starts out fast and slows as it approaches the target).

The only problem is highlighting the menu section. I could have based it on the current page or directory by comparing it with the URLs in the menu or in an array. In the end, though, I decided to use a Dreamweaver template variable to indicate the section to be highlighted. It works well enough and doesn't require much extra effort.

404 and other Scripts

I cleaned up and simplified this page a bit. Not much of the actual functionality has changed, but now it's a bit easier to follow. I'll need to spend a few weeks watching 404 results to make sure no possible loops have been introduced (as have been in the past). Also, it's probably time to add some documentation to the 404Redirect.txt file. I have to keep reminding myself about the format every time I work on it. I also modified the case-insensitivity to use the built-in functionality of the InStr and Replace functions rather than adding a UCase function. Still have a few more items to address, which I should be able to get done easily.

Changed the form validation script to that it now uses DOM functionality to add an event to the page on which it's initialized.

Modified all sign-up scripts on the Web site to use the correct sender address (postmaster@list.project2061mail.org) and added a notice for people who use spam filtering software to add us to their whitelist.

Modified the print.pl script to use our current logo, modified it to use the current stylesheets (rather than try to reformat everything), and improved the error reporting slightly.

Modified the refer.asp script so that it provides error messages inside the site template.

Communications Database

We had a meeting about what we want to do about managing our contacts, lists, and other random data. I did a bit of research and initially thought that maybe a simplification would be in order (it certainly doesn't seem like we need to many features). However, after going over my list of questions (see below) it seems more and more like we're looking for a CRM-type system. My thoughts of having information on subscriptions and contact source and purchasing history actually rang a cord with Mary. Whether or not these ideals will be born out in practice is a whole different situation. I'm willing to give them the benefit of the doubt and it's hard to argue that the current system is worse than inadequate for our needs.

In some ways a system such as this is going to be more than we need. In others it may be just right. Finding something that meets our specific needs is going to be a challenge. I'm trying to find out what AAAS has available that we would be able to use, but no response as of yet. Right now it looks more and more like we'll be investing in a php-based open source system. There are a few out there so I think we'll be able to find something to meet most of our needs. Unfortunately, the nature of the various types of information we're using at present is going to make it a challenge.

Questions asked:

• What have we done with our data in the past?
• What would we like to do with our data in the future?
• Do we want to better integrate our data and systems?
• Do we need more advanced capabilities?
• Does simpler but more focused software better suit our needs?

Mass Mailing

Changed the mass mailer script so that the full text of the outgoing message is stored in the database instead of just the input text. This is a better option as it provides an accurate record of what went out to our lists.

Modified the one-time sign-up pages so that they now use the SQL server as their data source. I copied the data from the old access database, but should double-check next week to ensure that it's not being used by any pages.

Instructional Components Prototype

A user has been complaining of it not functioning on WinXP through AOL 9 with just the Adobe PDF viewer installed. I've been unable to duplicate the problems so far. I'm still attempting to get some useful information from the user but I really have no idea where the problem could lie.

The SMS folks have said that, while the prototype works at present, they're only getting the PDF version of the maps. They have the SVG viewer installed and are using IE. I'm not sure where the problem lies with them, either.

No doubt the problems that are cropping up have to do with the javascript. But until I can isolate the source of the problem I won't be able to provide any kind of solution. I've downloaded some stand-alone version of IE versions 3 through 5.5 so that I can test on the various platfoms. Unfortunately, the issue does not seem to be isolated to any of the supported versions as the prototype appears to function correctly as I've been testing on the older versions.

Communications Database

The bane of my existence right now, the combination of contact, list, and subscription management that is taking place in the communications database and my own database. Now communications wants to move away from using access. Jonah's organizing a meeting next week for us to consider our options. I've been assigned to investigate our alternatives. I have a few ideas, and will probably even be open to a non-homegrown application as long as it's flexible enough to be customized to fit our needs. I'll need to investigate some other options as well, though. AAAS should have something we could latch onto if we wanted. I'm sure there are some open-source products out there (Ed, as always, is pushing this option). Sean suggested at the low end we could use ACT! or something similar. And then there's always the option of me creating our own system, which I had already started in order to alleviate a number of problems that we're having (data mainenance, duplication) as well as address some needs (user-control, web site customization). I have one week to get all the info together.

Mass Mailing

I've modified the one-time only notification feature of the mass mailer so that it now uses the SQL database instead of an access database. I still have to modify the input side of this function (namely the notify.asp page), but I'll put that off until next week. I don't expect we'll need to be using it anytime soon.

Sent out the most recent issue of 2061 Connections. Not much to note here, the mailing seems to have gone off without a hitch. Still, there are some additional changes I'd like to make to the mailing script. Since the quoted-printable encoding of text seems to be working fairly well I think I'm going to implement this on the HTML portion as well. This will have a number of benefits:

• No more worrying about line length
• I can further consolidate and simplify the portion of the script that send out e-mails
• I'll have a little more flexibility in the the e-mail format, etc. as I will no longer need to rely on the CDO engine. I'll still be using a file drop in the IIS SMTP pickup directory, but it seems to be working well enough even at the current distribution of 3800.

On a related note, since communications asked me to add all e-mail addresses from their database for the current mailing of 2061 Connections I was flooded with bad addresses (over 400). I was not too happy to clean out the databases (theirs and mine) since they never seem to do any kind of maintenance on their data. Hopefully once I get a new system in place I'll be able to use VERP or something similar to handle things like this automatically. I've already got an idea of a way I can do this without varying the from address, which is good for a number of reasons, but mostly having to do with spam blocking. If I use the an errors-to header or whatever current "standard" is support by the various mail server I should be able to craft a script to check the drop directory on the server for bounces and such. Then I can implement a form of VERP by using a custom address in this header and keeping track of the returns over time.

HS Bio Textbook Report

What a pain. Converting these reports from Word took a lot more work than I expected. However, in the end it's probably taking a lot less work than the previous reports thanks to our purchase of UpCast. And the fact that Communications will be working through Contribute will only make things that much better.

I had to go through a four-stage process in order to get the reports into an editable format. First I converted the reports to HTML using UpCast. The output was a little weird (unicode format, long lines) so I used Mozilla to save in a more workable format (ISO, numeric entities, formatted code). Once that was done I used HTML-kit to remove inline styles. Finally, I hand-edited the documents to remove all unnecessary styles and weird CSS formatting.

Next week I start on the templates.

AAAS Redesign

Complete. I was able to perform this redesign in record time, especially considering how long the previous redesign took. Still, it too a bit of effort and I made a number of changes to the template provided to us by AAAS. In the end, though, I think the switch was well worth it.

Site structure
I moved a lot of things around. The new design uses a lot of javascript to generate the navigation elements. One of the scripts determines what navigational elements to activate based on the href of the current page. I had to make a number of modifications to get this working on our site since we're not under the AAAS hierarchy. I think this is one area where better content organization will benefit us greatly. I've already moved a couple of documents and sections around so that I can take advantage of this menuing system. There's still a number of documents that should be relocated, though, and I expect we'll be spending a few months figuring that out.

I've also changed the staging server hierarchy to better mimic the development/production server. All html directories have been placed back inside the top-level directory (HTML). On the plus side this better integrates the content on the site. On the down side it will require more work to ensure accurate links, etc., if any of this content is going to be repurposed. This isn't too much of a problem at present, however, since the sections in question (namely the online tools) haven't been updated in a long time.

Other random modifications
I made some improvements to the usability and scripting. Two of the biggest things conceptually (seeing as they're fairly minor in reality) being the usage of +/- indicators on the left-side nav to indicate expandable main menu items as opposed to links and highlighted elements within the expanded menus. I think these changes greatly enhance the menuing system. I've managed to keep our site tools on the left-side menu so that they are still easily accessed and probably easier to see overall than the previous design.

I made a large number of changes to the overall code structure and I think it's now much more readable than it was originally. I've unified the stylesheets into a single stylesheet that handles all the major browsers. I'm hoping to remove our "Print this page" link by using stylesheets to generate a printer-friendly version of the documents on the client side.

Best of all I've done all this without affecting the look of the site. I'm having one minor problem with the width of the content area, but it's hardly noticable … especially after looking at the changes some of the other directorates have made.

We're sending a list of corrections and suggestions for improvement to Kelly Gayden next week. The list ended up being longer than I expected, but the majority of what's in there are minor requests.

When I have time I'll prepare the work I've done so that I can send it back to Kelly. Hopefully the upgrades will make the site easier to manage. After that I want to see if I can get rid of some of the table-based layout code in favor of stylesheet-based code. I think it will significantly improve the managability of the site's code.

Mac weirdness
This one is a bit annoying. IE for the mac (am I still working around this dead program?) was randomly setting the width of some pages to about triple the size of the browser window. It turns out that there's a bug when a div element is floated and contains two block elements inside it. In this instance IEMac stretches the width of the parent element in an ungodly manner. This was a problem because of the right-side content I had to repurpose when preparing for the redesign. I ended up working around the problem by having IE just display these elements inline rather than floated.

CCMS Web Site

There was a complaint that the CCMS site was doing some weird stuff with the design (randomly choosing the text version over the graphical version). I was unable to reproduce the problem even under the same platform/browser combination. After pouring over the code I think the problem was the manner in which I was attaching events to the document when it loads. Initially there was a both an html attribute (for the init function) and javascript statement (for the stylesheet switcher). I think the two were having problems cohabitating in some browsers because the javascript statement used "onload=" rather than a less destructive "addeventlistener" function. I've modified the code so that the function call that was originally in the html attribute is now part of the javascript statement.

Contribute

Microsoft vs. Macromedia
Dave's been working to improve the setup of our network and his latest step was to not give users admin rights on their local machines. Unfortunately, after doing this these users were unable to start Contribute. Dave worked on it for a while but eventually gave up … not to mention that he's got a ton of other things to do rather than figure out why Contribute won't run. I gave it a shot and managed to figure out what was going on. Turns out that the Macromdia Licensing program runs as a server. In WinXP regular users are unable to start a service. See the problem? I've worked around the issue by having the licensing service start at boot time.

Navigation - Show/Hide

I needed to create a show/hide menu navigation for a document being posted on the site. The old script was a bit clunky and required quite a bit of work on my part to implement. I went ahead and wrote a newer script that is much more modular and requires little extra work when used on a page. The new JS file includes documentation and sample. A quick overview:

• Additional code required for use: divs around menu and each document section (id for the section divs should match the anchor links in the menu)
• attach JS file to document
• attach stylesheet to the document with a few required entities

The script is "non-destructive" … it shouldn't affect any other scripts on a page. I made the function, variable, and CSS names pretty unique and the onload statement is done using the "addeventlistener" method (which requires a DOM-capable browser … but that's less of an issue than it used to be).

Intranet

I took some time to make some minor modifications to the intranet, mainly for my own sanity.

• Since I'm not using our mail server to retreive mail anymore I had to add a special username/password combo for my own login
• I added a link to the old e-mail interface
• I added a link to the mass mail database update page
• I added a link to the file upload page on the development server
• I modified the mass mail database management pages to include the intranet header

404 Script

I finally found some time to make the 404 custom error page case insensitive. This was a problem with the 404redirect.txt lookup portion. the URLs "/2061Connections" and "/2061connections" (for example) did not both resolve … which is desireable with the variability in links that are out there. Now a UCASE is done prior to comparison and replacement.

CCMS Web Site

Done. Done. Done! I had pretty much finished all of the stuff I wanted to do with the design weeks ago. I've mostly been waiting on the content from Communications. Well, chock up another item to a last minute deadline. CCMS was being highlighted at a conference and they wanted to site redesign published prior to that date. A few items had to be knocked off the menus, but otherwise the site is finally complete.

There were a few pages I was working on myself regarding the functionality of the site. Those were fairly minor, though, and I was able to hammer them out when the site was going live. I still need to work on the site search (now that I think about it). Though I don't think it's a high priority it's still something we need to do (especially if we'll be moving more of our sites to open-source platforms). I've been distracted by the AAAS redesign, so hopefully I can get on that particular item next week.

After getting everything worked out and spending some time editing the site rather than designing it I think I'd like to make some changes to some of the navigation. The image-based navigation is too much trouble to maintain since we're not using any round-trip method of editing such as Dreamweaver/Fireworks or GoLive/ImageReady. When I get a chance I think I'll investigate using only text for the navigation again. I gave up that idea after having to deal with the designer, but now that we've completed the initial development I should be able to make that kind of change to the site and support that kind of change with consideration of the time required to make navigational modificiation.

Plus now that the designer is done with the project I won't have to bend to the wishes of someone who has no idea about the limitations of HTML.

Atlas Workshops

Almost forgot about this one. Spent the better part of two weeks getting the site ready for the addition of information and applications regarding the new scholarships being offered. Created a new form mailer so that I don't have to spend all my time creating scripts to handle any forms that we use on our site. I'm sure I'll still be spending a good amount of time on these, but at least now I can quickly create forms that require nothing more than that the information be passed along to a staff member.

The form can be fairly simple and only requires a few fields for the mailer to operate on it. However, the mailer is also capable of more complex tasks such as data validation. A number of hidden fields must be included for each form object that needs the validation, so a little more effort is required to create the form in this respect. But the payoff in processing time should be great. I hope I'll get an opportunity to use it often. A brief overview of the required and optional fields and what they do.

In order to use this script the submitting form must contain the following elements:
Hidden Field: ctrl_bolValidated (0)
Hidden Field: ctrl_bolProcess (1)
Hidden Field: ctrl_strRecipient (email address)
Hidden Field: ctrl_strSubject (text)

The form can be designed without any modifications to the structure of the form. However, user form fields also require the following information hidden form fields to enable processing:
ctrl_fieldname_userfield (1)
ctrl_fieldname_type (1=string,2=number)
ctrl_fieldname_required (1=yes,0=no)
ctrl_fieldname_format (only for string values, currently available: NUL=no format,EML=email address)
ctrl_fieldname_friendlyname (text, displayed if something is wrong with the field)

Optional fields to aid in processing the data and to provide a success message and confirmation email:
Hidden Field: ctrl_strSuccessMsg (text)
Hidden Field: ctrl_strSuccessLink (URL)
Hidden Field: ctrl_strOrdered (list of field names, these will be listed in order specified in the e-mail, all others will be listed in order provided by the Request object)
Hidden Field: ctrl_bolConfirm (1, Send an e-mail confirmation to the user)
Hidden Field: ctrl_strConfirmSubject (text, subject of the email confirmation)
Hidden Field: ctrl_strConfirmBody (text, contents of the email confirmation)

For any other field on the form, the "ctrl_" prefix will prevent those fields from showing up in the results e-mail.

AAAS Redesign

This one's a doozy. I like the look of the new design well enough, but I absolutely hate the actual coding. I just don't understand why someone would design a web site using nested tables these days (up to 6 or 7 levels deep in some places). I could have looked past a single table setting up the columns and such … maybe even two-level tables. But this thing is just horribly designed. And as if the structure weren't bad enough, the code itself is ugly and, in more than one case, invalid. I've done my best to go through and clean up the code as best as I could. Some code couldn't be touched without f'ing up the layout, but I think I've done a decent job of getting things prettied up and well-formed (considering the limitations).

I hate the methods being used by AAAS to formulate a template as well. The method they use to make it so people can't destroy the layout are ugly at best. The choice of what to place in include files and how to group things just seems completely random. I've created a pretty decent include structure that I'm going to try and pass along to Kelly to see if she'll replace the currently crappy template files. I think it would go a long way towards making the site easier to maintain.

If I have time I'm going to try and make a more CSS-oriented design. I've already shown what I can do with the CCMS site. If I can create a cleaner template using mostly CSS I think I may be able to convince Kelly the benefits of such a system. Of course, I don't know if she or anybody has time to reimplement the template in such a manner. I believe the whole reason things ended up this way is because deadlines seem to be more important than quality work … which I guess is true just about anywhere.

On the plus side the CSS file pretty much uses only classes and identifiers. No more p styles or h1 styles making my life that much harder. There was a single td style that was giving me problems, but I was able to eliminate that with hardly any problems. I'll have a lot more flexibility in setting up the styles for our site, which is great because they have once again specified an incredibly tiny font size for the content areas. The downside to their usage of CSS, though, is pretty big. They're using javascript to write out the CSS link because they wanted to use different stylesheets depending on the browser. This despite the fact that the stylesheets were amazingly similar. I was able to overcome the problem by using the !imporant declaration in a few places (it appears that the second stylesheet was targetted towards Nav4 users … ick, luckily Nav4 ignores the !important declaration). Also, in my opinion the stylesheets were not well written, declaring the same style for the same element in different states. It made for some difficult reading and I spent some time cleaning it up so that it's now easier to tell what's modified in the various states.

I've been trying to find out what constraints we'll be working under with the new design but I guess nobody's really talked it over. Kelly told me she's been getting conflicting instructions and is trying to clarify things. I've gone ahead and made some variations on the template for review by our staff. Hopefully Kelly will get back to me soon. I'd like to know what kind of constraints I'll be working under. No matter the response I'm definitely going to want to make some modifications. It obviously hadn't been considered that the templates would be applied across hundreds of documents. I can't believe a site design would be created without consideration of variations in display that would accommodate a wide range of content such as that available on the various AAAS Web sites.

At any rate, I've already taken the first step towards application of the template. I've applied the template that has no right or left side editable regions. I had to make changes to some of the documents since there were a few that did actually use these areas. No doubt I'll be making a number of adjustments over the next few months to accommodate this design. I would like to apply the design and make it live by the end of next week. I don't really see any major need to go through as extensive of a preparation period as was done for the previous design. I believe our current structure will hold up well enough under the new design. That's not to say that no changes will occur, I just think we'll be able to gradually refine our site over the next few months as we work to better integrate in the new design. It's what I had hoped to do after the previous redesign … but time constraints always seem to hamper our efforts.

404 Script

I've been running into a number of problems with the text-only portion of newsletters. Some clients just don't understand the method of URL demarcation. I've managed to modify the 404 page so that it now catches most of these problems. Unfortunately, I think I'll need to consider another method. Maybe square brackets or something similar.

I've had to make a number of modifications to the 404 page. Fix something and another problem crops up, fix that problem and break something else, yadda, yadda. It's a continual process, naturally. Luckily most of the major problems have been showing up in the URLScan log so I can address them fairly quickly. I can't really remember a number of the changes, though they're easy enough to determine just by looking at the script. I'll definitely need to spend some time going through the script so I can document what's going on in there.

Newsletters - 2061 Connections

The new newsletter went out without a hitch. Modifying the "FROM" address to use postmaster@list.project2061mail.org seems to have solved a number of our problems. We were caught by a few SpamAssassin filters … some of which I've tried to address (see below). Some of the triggers didn't seem to have any relevance to the message we sent out, so I'll just have to wait and see how the next one goes. Still getting the usual number of bounces, etc. I sometimes wonder if some of the messages are getting lost in the ether as some of the bounces seem like they should have been addressed months or years ago. I'll need to investigate this at some point, perhaps I can send a few hundred test messages to my mail server and check the results.

SA Filters Triggered and resolution (if any):
BODY: Words dis|guised with a pipe sym|bol
(Don't know how this one was triggered … unless within the base64 text)
BODY: Claims you can be removed from the list
BODY: I wonder how many emails they sent in error
BODY: {5}Letter - punctuation - {4}Letter
(Probably from URLs in the body … should be more intelligent)
BODY: To be removed from list
(Not many good ways to word this, and most are probably filtered)
BODY: HTML has unbalanced "body" tags
(Was unable to verify this as a problem)
BODY: HTML included in message
BODY: HTML comment has 3 consecutive 8-bit chars
(Was unable to verify this as a problem … I don't think we even had HTML comments)
BODY: HTML has excess "a" close tags
(Was unable to verify this as a problem)
BODY: Message only has text/html MIME parts
(Not even close to being accurate)
URI: URL of page called "remove"
(Changed URL to http://www.project2061.org/listpref)
URI: URL of CGI script has unsubscribe or remove
(See previous comment)
Spam tool pattern in MIME boundary
(Modified the MIME boundary)

One of the sysadmins also suggested we add a preface to the subject of our newsletters to better indicate their source, something like "[Project 2061 Newsletter]" or something like that. Me likee, but perhaps we can come up with something shorter. I used "[Project 2061 Update]" on our last mass mailing but it still feels a bit long.

I have a feeling we'll need to consider using SPF, Domain Keys, Caller ID for E-mail, or some combination of these soon. Drafts have been written and are getting a lot of press lately, which means more people will be paying attention. Sadly, most people prefer to require compliance rather than use header-addition. Bad sysadmin!

Instructional Components Prototype

I love creating modular code. Francis didn't want to abandon people without SVG capability and so created PDF versions of the maps being used for the demo. He requested that I come up with a way to have the IE users go to the SVG version while everyone else goes to the PDF version. The knee-jerk reaction would have been to set up a function that could be applied to each link's onClick event handler to take care of this. But that seemed like a waste of time and effort. I came up with the brilliant (if I do say so myself) method of linking to the PDF version and using the DOM to add an event listener to the document object that captures the click and redirects if necessary. On page load a script runs (obtained from a third-party source, see internal reference) that sets a variable indicating whether or not the browser has SVG support. Then, once the page has loaded completely, an event listener is added to the document that captures onClick events. This event checks to see if the event has an href property (meaning it's a link), prevents the default action (visiting the link), and then directs the browser to open the desired document based on whether or not it supports SVG (and the user is trying to access a map).

Unfortunately IE's event model (bottom-to-top) is different from the W3 recommended (top-to-bottom-to-top). So I had to add modify the script for IE using a loop that adds the event capture to each and every link tag. A little annoying, but the code is pretty much just as concise. Also had to take into account the IE event object which is slightly different from the standard.

Still, the functionality appears to work perfectly across platform. The only glitch now is that Mac OS 10.3 has no browser plug-in from Adobe, meaning the PDF version are pretty to look at but little else (as they are loaded externally).

2061 Logo

AAAS has released a new Identity Manual that includes font specification and usage, logo specification and usage, and a number of other similar things. I'm still not sure about the implementation of the directives from the manual, which appears to be non-optional. But indications from others at AAAS are that things are still up to the discretion of the people producing the media (print, web, etc). A site license wasn't even obtained for the organizational font and I'm not sure if we'll be ponying up for it. We did manage to get the new 2061/AAAS co-branded logos and I plan on using that on all new print/web productions.

Mass Mailing

I had wanted to completely rewrite the mass mailing system to allow users to control all the information we have about them. Kind of a profile system where people can update their personal information, subscriptions, contact preferences, and other information. This will tie in quite well with purchasing and workshops once it's completed. Minimal information will be required and collected (though no doubt somebody will want to try and collect more and more once they see how the system works). The best part will be the tie-in with the communications database … unifying two of our biggest contact management systems (big for us anyway).

Sadly I ran out of time to complete this project and had to resort to massive modifications to the current system. It's still disociated from the communications database, but now a lot of the electronic items are tied together better. A user can subscribe/unsubscribe from the various newsletters much easier now (though I still have one more page to adjust). It probably wouldn't take much of an effort to add the ability to subscribe/unsubscribe from the print newsletter … which would be a great stop-gap solution until I have time to complete the larger overhaul.

The majority of the work, however, went into the new mailer. I had to spend a significant amount of time figuring out the format of plain-text and HTML-based multipart/mime mail. Though more "manual labor" is required on the part of the person doing the mailing (that would be me), the new system is a lot more flexible. Plain text mailing basically utilize the same system as before with the CDO NewMail object. I've moved the standard footer portion to the client side so that I could de-couple the mail creation from the actual sending of the messages. This had the benefit of allowing me to set up a preview option where an EML file is created and downloaded (thanks to the content-disposition HTTP header). All this, however, was gravy after spending around two weeks attempting to set up a system that can do multipart mailing as well. A number of considerations had to be made:

• line length … I wanted to stick to around 69 for URL modification purposes
• what encoding format to use … I wanted to stay away from quoted-printable to make my life easier; I ended up having to set up QP for the text portion of the message anyway; perhaps I should rethink the current encoding on the HTML portion of the message
• mime encoding of images and internal reference … I had originally wanted to pre-encode the images and just insert them where necessary, but decided that could cause other problems so I went with dynamic inclusion

The system I ended up with is very nice. There are now two textarea fields, one for the text content, one for the HTML content. Upon making modifications to the text field it automatically generates QP text which is inserted into a hidden field. A standard footers is available for the text content field (inserted at the end of any current content) and is generated based on what list has been chosen. The HTML field accepts the code of a standard HTML document (preferably in HTML 4 format using CSS for styling). Up to 5 images can be attached to the message. Click on the appropriate link and submit the image. The images are base64 encoded and inserted into a hidden form field along with the name. Three options are available for each image: clear it out, check the name, and insert a reference into the HTML. The last option requires the cursor to be at the insertion point and inserts a cid reference to the mime-encoded image. The image ID is generated on form loading. HTML-based e-mails use a template for generation and a file drop into the pickup folder for sending. The mail template is fairly simple and there is an additional template that can be inserted for the images. Text replacement is used to fill out the headers and content of the message.

I'm not sure if the mime boundaries are RFC acceptable, but they seem to work well enough and I'm not overly concerned about running into duplicate boundary identifyers. All tests on the system have gone smoothly. I've already sent out a text-based message and it went off without a hitch. As a matter of fact, it went better than the previous few months because I've finally implemented the new from address with the new system. This had been giving us no end of headaches previously thanks to filters and such. There were no messages stuck in the queue at all … which is a pleasant change.

Newsletters - 2061 Connections

The new newsletter is just about ready to go. After weeks of testing HTML-based e-mail on various systems (Outlook, Outlook Express, Yahoo!, Mozilla, AOL, AOL Webmail, Hotmail, Squirrelmail) I think we've got the basic formula down. The HTML is fairly simple and uses inline styles for presentation. I wanted to try and use current standards as much as possible. I think it just makes things easier. Display is either perfect or good enough on all tested platforms except Squirrelmail which, for some reason, displays the text version of the message. I think this is probably more due to administrative settings rather than support, but I haven't done extensive testing to figure it out. Plus this is the whole reason I went with multipart/alternative. At least I know that it works the way it's meant to if the HTML version is not displayed.

Everyone seems to be happy with the newsletter and the first issue will go out on Monday. I've run the newsletter through couple of different free SpamAssassin testers and it never came back with a rating high enough to cause concern, so I think we'll be ok on that part. I've also done what I can to adhere to the CAN-SPAM laws on this (and all other) mass mailings so we shouldn't run into problems there. I'm still not happy with the staff's penchant for mailing any address we can get our hands on. At least on the bright side we are not sending commercial mail and provide a true opt-out option.

AAAS Redesign

Speaking of the boss-man. Met with Kelly Gayden the other day to review the new AAAS design and discuss what it would take to get things done over here. It actually looks like we'll be able to make the transition to the new look without much trouble. That doesn't mean, however, that it'll be a no-brainer.

The new design still uses a familiar top/left-side shell structure found on so many web sites. Replacement of the top area will be easy enough. The left-side navigation, however, should be modified to take advantage of the new functionality. Also, it looks like we'll need to forego our handy drop-down/search modifications. But I think most of these items can be easily enough integrated into the new navigational scheme.

The right-side … um … sidebar is no longer present so we'll need to rethink some of the pages on the site. I think with a number of these items we may just be able to integrate them into the actual content. For the rest I'm currently leaning towards using floated divs to allow for a similar affect. Some content I think I'd like to make more

The content areas should be a simple drop-in.

I haven't had a chance to review the stylesheets so it's hard to say if I'll need to make another end-run around their CSS code in order to have a site that's more user-friendly. Sadly, it appears they're still using tables for layout … I can't wait to boast about the work that was put in to the CCMS site. No doubt in the end I'll want to make a number of modifications. At the very least I'll need to do what I can to fit the code into our Dreamweaver templates. Much as I like what they've done to separate out things as much as possible, I'd rather see more time spent on standards-compliance.

Web Site Updates

Created a template that has no sidebar on the right. The main purpose of doing this was to accommodate content that doesn't use the sidebar and would benefit from a wider content area. I'll probably go ahead and apply this site-wide, however, since I'll need to adopt a similar design once I apply the new AAAS template.

CCMS Web Site

Done. Well, nearly done. Done with the design/coding aspect of things. Though nobody's really been looking at it much since I started work in earnest over the last month so I'm sure a few stylistic changes will need to be made. However, I've managed to create a site that is XHTML compliant, CSS2 compliant, and addresses (most of) the accessibility guidelines. Plus, in order to get away from the, um, questionable design (of white on light blue) I've been able to create a cross-browser, cross-platform stylesheet switcher (thank you ALA). Essentially, the default style displays a text-only site (black on white) with a somewhat stylized left-side navigation (current location is indicated using color). If the user has a modern browser (IE5+, Mozilla, et al) then the style automatically switches to the "CCMS design." However, the user is presented an option at the bottom of each page to choose how the site appears. It works brilliantly and I hope it is actually used to good affect. I have no doubt that people will want to rid themselves of the difficult-to-read design, I just hope they have no trouble figuring out how to do so. Plus, the site has a CSS-defined print style and degrades nicely to older browsers (even non-CSS ones). All this is well-documented in the site code.

I'm thinking we should add a breadcrumb trail to better indicate the users location; browsing through the site feels a tad abstract at present (it's sometimes difficult to get a feel for where you are). I's not difficult to add, but deciding on the location will be something of a challenge.

Still need to add a few things: content (waiting for Communications to finish), a search engine, and a few site notes regarding the stylesheet switcher and standards compatibility. However, I suspect we'll be completely done in the next few weeks.

At least I can move on to some other major project now.

IERI Utility

Finally after many, many, many months I am getting a chance again to work on the IERI utility. No doubt it needs a lot of work. I've got some great ideas about using CSS and Javascript to vastly improve the interface, javascript objects to enable a unified data entry system, and XML for client/server transfer. Perhaps I can even make this version more standards-compliant. This is going to require a lot of changes to the scripts and database. I've got some pretty good ideas of where to start. But this is going to easily be a six-month project. This is all still at the planning stage and things could change drastically.

Kathleen has a number of modification she would like to see added to the utility before anything else is done, so until I've made all those changes I'm going to have to stay in the planning phase. Most of the requested changes are bug fixes, but there are a few feature additions as well. Here's the list of changes I've made so far:

• Added teacher analyst user type with specified options. You can test this with username/password combo of teacher/p2061
• Made the idea boxes bigger in all areas, but the biggest change is on the activity detail screen
• I added a "Save Analysis Data" button to the left-side menu that is shown when an analyst is analyzing an activity.
• Fixed the bug so that you can now add an analyst after a movie has been checked in.
• Indicators that are "not observable" no longer have the rating/rating comment items attached to them. In the database these items are automatically marked as "not applicable" (I'm not sure if this is the preferred method or to just leave them as null.)
• When you duplicate an activity the relevant sighting information is also duplicated for that activity (along with it's connection to the literal information). I did have a question on this item (see below).
• Activities on the timecoding screen are now ordered first by their time-coding then below that by their activity number. So timecoded items will appear first in the order in which they appear on the video. The same is true of the sighting list on the enacted activity analysis screen.

Perhaps I should start maintaining a change log. It'd probably help when I get around to doing the code documentation. Based on how long it took me to do the documentation for the CCMS redesign I expect it will take me a good while to do the same for IERI. Especially when you consider that the majority of the system is code compared to CCMS. Plus I should make an effort to work on a user manual or online help system this time as well. It'll be nice to create a full-featured system compared to the work I normally have to do around here. Though it was a lot of effort, I found the extra work I did for CCMS to be gratifying.

CRM

I have a lot of work to do on the database. I'm still in the process of setting up the tables. Trying to determine all the information that will be required to run this thing is certainly a challenge. I think I'm actually pretty close to being done with the table setup (I'll attach a diagram next time). But once that's done I'll still need to set up all the stored procedure, views, and relationships. One thing of note, I decided to use the GUID feature for the external ID so that it's not as predictable as it would be otherwise. Internal linking will still be done via the autoincrement ID. I doubt the ID problem will be too much of a problem anyway because I'd like to do something similar to what I've done for the intranet authentication scripts (cookie does not contain identifying information but points to a file containing the user ID).

I still haven't quite decided on some of the issues that will need to be addressed. Such as:

• how much information we will have access to from a user-created entry
• how do we handle staff updates of a user-created or user-owned entry
• how will the e-mail list system be set up (especially for lists with an HTML option)
• if we use external list management software will we be able to integrate it with the user database

And I have about a week to get all this done :(

Newsletters - 2061 Connections

Worked a bit on the header image. Really there isn't a whole lot to it, but making sure the image is large enough but not too large is a challenge. I wanted to use a CSS background image so that it didn't really matter how large the user's client window was set (since the right part of the image is just for looks). Unfortunately there's not a lot of support for that out there.

Speaking of CSS, I decided to avoid using a stylesheet of any kind (external and internal). There's just too much variability in what supports out-of-line styles. Still, HTML with inline CSS looks like it'll work out nicely. Plus I can set up a section on the web site for Jonah to work on the newsletter then just copy the HTML for the newsletter.

Mass Mailing

Sent out a few more mass mailings over the past few weeks. We've been seeing a lot more returns (around 10%) than in the past. I suspect we're butting heads against SPAM filters. It looks like our biggest problem is the address from which we're sending the email (postmaster@project2061.aaas.org). I believe a number of sites are now checking the domain on that address against MX records in the DNS. Since that domain doesn't have an MX record it's being rejected. I'm going to work with Dave on setting up a new server name on project2061mail.org (lists.project2061mail.org sounds good to me) from which we can send mail. Hopefully that will prevent a lot of the problems we've been seeing.

We do have a few emails being caught by content filters, but I don't know that there's a lot we can do there. Unfortunately, a lot of SPAM uses the same language we do because it is indicative of a legitimate message. On the other hand, a lot of spammers have resorted to random text in their messages, so maybe over time we'll be less likely to be cause. I don't know how the new HTML-based newsletter we're developing will fare. I suspect a lot worse, but I should have time to test against spamassassin before we send it. Plus we should have taken care of the domain issue by then as well.

Ed's talking about using the list server built into OS X. I'm not so sure about that, my experience with a lot of list software is that they use their own proprietary format. It's be nice if it could be modified to hook into a database like mysql or mssql, but I doubt it. I'll probably stick with my current system for the time being.

Speaking of systems, I still haven't had a chance to work on the user profile/contact system that I'd like to set up to replace the communications database. Though I've talked to Serita and Barb about what I'd like to do neither of them seemed to be concerned. True, a lot of what currently done manually will be taken care of automatically with the new system, but I suspect they haven't really thought about how this will affect the random work they do with the communications database.

Web Site Updates

I finished the Atlas Survey page. I've used a lot of the newer techniques on this page, but I think I can further refine the way I do things. Some of the new stuff I've used:
- HTML Form elements: label to give some elements a label, fieldset to group related questions together
- Updated ASP/javascript: there are problems with IE, radio buttons, and my JS validator. I've modified the JS to count the number of items to be validated and increment another counter upon actual validation. If the two counts do not match then the form is marked as not validated and the ASP validator takes over. I've also set it up to use the custom error-handler (A number of scripts had to be modified to account for some of the changes I've made).

Some ideas for further enhancement: With a form this complex a lot of manual work has to go into setting up variable capturing and database updates. I think it's time to look into modifying things so that more automation is present. I'll have to investigate the options, but a I've considered variable naming schemes and a database variable table along with the question/answer table and the user response table.

There have also been various other updates: Atlas workshops, What's New, etc.

Instructional Components Prototype

Haven't really done much with this lately. FM decided to modify the way the demo works even though I told him I don't think the problem he's trying to fix is critical. Helped them debug by pointing out an error in the stylesheet.

CCMS Web Site

Been working full steam on the redesign this week. Encountering many bugs between the various browsers but I think I'm finally getting there. Hopefully I'll have this done next week. A quick overview of some of the bugs I've found:

• Safari: cookies generated by javascript from a local file do not appear to be saved. Luckily this isn't a problem for CCMS.
• IE Mac: (a reminder) @media print css selector not supported; does not support onbeforeprint javascript event ala IE Win. This one was bypassed by using an inline style that imports a print-only stylesheet using @import … print
• IE Win: despite the existence of multiple style declarations (link, "style" and "alternate style") IE attempts to use the default style. If this style is disabled (such as when it is by the style-sheet switcher) then IE ignores all other stylesheets and uses only inline styles. I was able to get around this problem by using an onbefore print and onafterprint event handler to switch the style to the default then back to the user's preferred.
• Getting closer on being able to finalize the design.

Scripting

While working on the Atlas survey form I noticed that Internet Explorer has problems with the validation of radio elements. This is probably true of checkboxes, but I haven't taken the time to investigate the extent of the problem. I've made a note to correct this.

In the meantime I've worked out a kludge.

In the ASP or HTML file where the form is located:
The number of fields to be validated is counted when the form is set up and placed in the intFormFieldCount variable. Basically the count goes like this: one for each text, textarea, file, password, select-one, and select-multiple elements; the length of the array for radio and checkbox elements.

In the validate.js file:
The script was already set up to count the number of elements that have been validated and place that number in the intValidated variable. The count will show up the same as the above because I parse through the form using the elements array instead of each validation element name. I've added some code that allows IE to skip the validation of radio form elements. After all the elements have been tested the file compares the intValidated variablee with the intFormFieldCount variable and if the former is less then the hidden form element intValidated is not set to true.

Basically what this boils down to is that if any radio form elements are to be validated then the count for IE will be less than the total and the form will be submitted as not validated, allowing the ASP companion script will take over validation chores.

CCMS Web Site

Spent most of last week working on the CCMS site. I think people who don't know the limitations of the Web should not be placed in charge of design decisions. The designer seems to think that the same level of control that is available in print is available on via HTML. The other suggestion being to just use images. Somewhat frustrating but I've given up on trying to help her understand. My goal now is to just implement the site as designed. If I had been involved earlier in the process maybe things would have worked out better. Once I get everything functional I plan to work on an alternate stylesheet that's pretty much just plain vanilla.

Web Oversight

Our statistics tell us nothing. At least that's my general feeling about the current usage analaysis of our web site. I've decided to make one of our discussion points about the information we're trying to find beyond the total traffic.

Just realized my method of finding out the number of hits for the category pages has been woefully inadequate. I've modified so that I'm only getting actual page hits and not also lines with referrer information. See sample below.

find /C /I "/parents.htm - 200" ex0312??.log

I think I'll change the total traffic chart to 16 months so that we can get a better feel for year-over-year performance. Current chart really only shows the site performance for the past year.

Instructional Components Prototype

Spent a good amount of time this week working to get the IC demo working on both Mac and windows. There were various problems with both the treeview and the slidemenu scripts. I had to make heavy modifications to the demo to get things working smoothly.

• Update the scripts to the most recent versions
• Modified the treeview script to add a highlighted menu item (perhaps this is a built-in feature … I didn't really have time to investigate)
• Modified the treeview script to allow for the use of javascript for the link
• Modified the slidemenu script to add a ID and CLASS to the menu item container TD
• Modified the slidemenu script so that menu item mouseovers call a function that modifies the container class instead of the BGCOLOR attribute.
• Modified the slidement script by adding three functions, two to handle the above calls and one to handle the switching of highlighted menu item based on the selected section (this also modifies the container class)