Friday, July 23, 2004

CRM

Finally talked with someone from AAAS tech. services about how they could help us implement a CRM system. Really I don't think they'll be of much help. I suspect the systems they would recommend to satisfy all our needs would be beyond our resources … they mainly seem to think in terms of commercial software.

I looked around for some open source options. While there are dozens of so-called CRM systems out there, most of them aren't worth the time to investigate further (underdeveloped, unsupported, too specific, and little development activity). I did manage to find a few that are worth investigating more, and I'm sure one of those can probably meet enough of our needs to be useful. Unfortunately, there's no real definition of what software like this should be and so the features available vary greatly from package to package. The benefit of open source is that I should be able to add the functionality that missing, I just hope that one of the projects on my list has a modular kind of setup. In the end I still think I could do better by creating the system from scratch … not to mention I think I'd enjoy the challenge.

I've also been waffling on the feature set requirements. I've thought about it some more and feel that a completely integrated system may be beyond our reach and more than we need. My current thinking is that we would do just as well two different systems providing contact management and list management services. It would be nice if they integrated, but I honestly don't think we'd get much use out of such a system in the long run.

404 Script

I added a "jsRedirect" querystring to the javascript redirect so that I could see what percentage of bad URLs were being automatically redirected. Initially I forgot to remove this if the 404 page is encountered again, so I've modified the script to do so. I also had to add some additional conditionals to avoid a few problems I encountered with the simplified structure. Still, I've not seen too many problems as of yet.

Newsletter Signup

Finally had a chance to add the print newsletter to the newsletter signup page on the site. It's now possible to subscribe to any of our newsletters (print or electronic) from a single page. I did a little script trickery to get things working the way I wanted. Basically, the script asks for your name and which newsletters to which you would like to subscribe. Then, once that's done it asks for your e-mail address, postal address, or both depending on which selections were made. The trickery comes in when moving to the "second" form. My validation script wasn't really designed to handle multi-page forms and so I had to come up with a way to satisfy the requirements of form validation and data processing. It was easy enough to set up but I certainly did not use the cleanest method to do so. I think maybe it's time to do some rethinking on the form validation/processing script I'm using. I should be better able to segment things so that it can handle a situation like this.

In the end, though, it seems to work just fine.

Mass Mailing

Made further enhancements to the mass mailer script. Now both text and html newsletters are converted to QP and sent using a file drop. This means CDO is no longer used. The biggest benefit is that now I don't have to worry about using a different method depending on the format, simplifying things a bit. I could probaby even simplify a little more. I think in the end this will give me a little more flexibility in customizing the newsletters. I've already done one customization where the user's e-mail address is now included as part of the body of the message. Not only will this help the subscriber identify what e-mail address they used to subscribe, it will also help me in cases where remote MTAs modify the recipient information (which is rare but still an occasional problem).

HS Bio Textbook Report

I'm pretty much done creating the base templates for the reports. No doubt there will need to be modifications, but my assigned tasks have been completed.

We wanted to use a floating menu similar to the previous reports, but I was never too happy with the script we used. Sure it worked fairly well across platforms, but without JavaScript it was useless. If nothing else I'd at least like the site to be usable sans JavaScript and CSS. I managed to create a pretty good replacement using a combination of HTML, CSS, and JavaScript. The basic menu is now just an unordered list. For Mozilla and other CSS2-capable browsers I used the "fixed" positioning style to maintain the menu location. For IE I've set it up to scroll using a much simpler script that is called by the "onScroll" event. So far works like a charm (I even set up the JS so that the menu starts out fast and slows as it approaches the target).

The only problem is highlighting the menu section. I could have based it on the current page or directory by comparing it with the URLs in the menu or in an array. In the end, though, I decided to use a Dreamweaver template variable to indicate the section to be highlighted. It works well enough and doesn't require much extra effort.

Friday, July 16, 2004

404 and other Scripts

I cleaned up and simplified this page a bit. Not much of the actual functionality has changed, but now it's a bit easier to follow. I'll need to spend a few weeks watching 404 results to make sure no possible loops have been introduced (as have been in the past). Also, it's probably time to add some documentation to the 404Redirect.txt file. I have to keep reminding myself about the format every time I work on it. I also modified the case-insensitivity to use the built-in functionality of the InStr and Replace functions rather than adding a UCase function. Still have a few more items to address, which I should be able to get done easily.

Changed the form validation script to that it now uses DOM functionality to add an event to the page on which it's initialized.

Modified all sign-up scripts on the Web site to use the correct sender address (postmaster@list.project2061mail.org) and added a notice for people who use spam filtering software to add us to their whitelist.

Modified the print.pl script to use our current logo, modified it to use the current stylesheets (rather than try to reformat everything), and improved the error reporting slightly.

Modified the refer.asp script so that it provides error messages inside the site template.

Communications Database

We had a meeting about what we want to do about managing our contacts, lists, and other random data. I did a bit of research and initially thought that maybe a simplification would be in order (it certainly doesn't seem like we need to many features). However, after going over my list of questions (see below) it seems more and more like we're looking for a CRM-type system. My thoughts of having information on subscriptions and contact source and purchasing history actually rang a cord with Mary. Whether or not these ideals will be born out in practice is a whole different situation. I'm willing to give them the benefit of the doubt and it's hard to argue that the current system is worse than inadequate for our needs.

In some ways a system such as this is going to be more than we need. In others it may be just right. Finding something that meets our specific needs is going to be a challenge. I'm trying to find out what AAAS has available that we would be able to use, but no response as of yet. Right now it looks more and more like we'll be investing in a php-based open source system. There are a few out there so I think we'll be able to find something to meet most of our needs. Unfortunately, the nature of the various types of information we're using at present is going to make it a challenge.

Questions asked:
  • What have we done with our data in the past?
  • What would we like to do with our data in the future?
  • Do we want to better integrate our data and systems?
  • Do we need more advanced capabilities?
  • Does simpler but more focused software better suit our needs?

Mass Mailing

Changed the mass mailer script so that the full text of the outgoing message is stored in the database instead of just the input text. This is a better option as it provides an accurate record of what went out to our lists.

Modified the one-time sign-up pages so that they now use the SQL server as their data source. I copied the data from the old access database, but should double-check next week to ensure that it's not being used by any pages.

Friday, July 09, 2004

Instructional Components Prototype

A user has been complaining of it not functioning on WinXP through AOL 9 with just the Adobe PDF viewer installed. I've been unable to duplicate the problems so far. I'm still attempting to get some useful information from the user but I really have no idea where the problem could lie.

The SMS folks have said that, while the prototype works at present, they're only getting the PDF version of the maps. They have the SVG viewer installed and are using IE. I'm not sure where the problem lies with them, either.

No doubt the problems that are cropping up have to do with the javascript. But until I can isolate the source of the problem I won't be able to provide any kind of solution. I've downloaded some stand-alone version of IE versions 3 through 5.5 so that I can test on the various platfoms. Unfortunately, the issue does not seem to be isolated to any of the supported versions as the prototype appears to function correctly as I've been testing on the older versions.

Communications Database

The bane of my existence right now, the combination of contact, list, and subscription management that is taking place in the communications database and my own database. Now communications wants to move away from using access. Jonah's organizing a meeting next week for us to consider our options. I've been assigned to investigate our alternatives. I have a few ideas, and will probably even be open to a non-homegrown application as long as it's flexible enough to be customized to fit our needs. I'll need to investigate some other options as well, though. AAAS should have something we could latch onto if we wanted. I'm sure there are some open-source products out there (Ed, as always, is pushing this option). Sean suggested at the low end we could use ACT! or something similar. And then there's always the option of me creating our own system, which I had already started in order to alleviate a number of problems that we're having (data mainenance, duplication) as well as address some needs (user-control, web site customization). I have one week to get all the info together.

Mass Mailing

I've modified the one-time only notification feature of the mass mailer so that it now uses the SQL database instead of an access database. I still have to modify the input side of this function (namely the notify.asp page), but I'll put that off until next week. I don't expect we'll need to be using it anytime soon.

Sent out the most recent issue of 2061 Connections. Not much to note here, the mailing seems to have gone off without a hitch. Still, there are some additional changes I'd like to make to the mailing script. Since the quoted-printable encoding of text seems to be working fairly well I think I'm going to implement this on the HTML portion as well. This will have a number of benefits:
  • No more worrying about line length
  • I can further consolidate and simplify the portion of the script that send out e-mails
  • I'll have a little more flexibility in the the e-mail format, etc. as I will no longer need to rely on the CDO engine. I'll still be using a file drop in the IIS SMTP pickup directory, but it seems to be working well enough even at the current distribution of 3800.
On a related note, since communications asked me to add all e-mail addresses from their database for the current mailing of 2061 Connections I was flooded with bad addresses (over 400). I was not too happy to clean out the databases (theirs and mine) since they never seem to do any kind of maintenance on their data. Hopefully once I get a new system in place I'll be able to use VERP or something similar to handle things like this automatically. I've already got an idea of a way I can do this without varying the from address, which is good for a number of reasons, but mostly having to do with spam blocking. If I use the an errors-to header or whatever current "standard" is support by the various mail server I should be able to craft a script to check the drop directory on the server for bounces and such. Then I can implement a form of VERP by using a custom address in this header and keeping track of the returns over time.

Friday, July 02, 2004

HS Bio Textbook Report

What a pain. Converting these reports from Word took a lot more work than I expected. However, in the end it's probably taking a lot less work than the previous reports thanks to our purchase of UpCast. And the fact that Communications will be working through Contribute will only make things that much better.

I had to go through a four-stage process in order to get the reports into an editable format. First I converted the reports to HTML using UpCast. The output was a little weird (unicode format, long lines) so I used Mozilla to save in a more workable format (ISO, numeric entities, formatted code). Once that was done I used HTML-kit to remove inline styles. Finally, I hand-edited the documents to remove all unnecessary styles and weird CSS formatting.

Next week I start on the templates.

AAAS Redesign

Complete. I was able to perform this redesign in record time, especially considering how long the previous redesign took. Still, it too a bit of effort and I made a number of changes to the template provided to us by AAAS. In the end, though, I think the switch was well worth it.

Site structure
I moved a lot of things around. The new design uses a lot of javascript to generate the navigation elements. One of the scripts determines what navigational elements to activate based on the href of the current page. I had to make a number of modifications to get this working on our site since we're not under the AAAS hierarchy. I think this is one area where better content organization will benefit us greatly. I've already moved a couple of documents and sections around so that I can take advantage of this menuing system. There's still a number of documents that should be relocated, though, and I expect we'll be spending a few months figuring that out.

I've also changed the staging server hierarchy to better mimic the development/production server. All html directories have been placed back inside the top-level directory (HTML). On the plus side this better integrates the content on the site. On the down side it will require more work to ensure accurate links, etc., if any of this content is going to be repurposed. This isn't too much of a problem at present, however, since the sections in question (namely the online tools) haven't been updated in a long time.

Other random modifications
I made some improvements to the usability and scripting. Two of the biggest things conceptually (seeing as they're fairly minor in reality) being the usage of +/- indicators on the left-side nav to indicate expandable main menu items as opposed to links and highlighted elements within the expanded menus. I think these changes greatly enhance the menuing system. I've managed to keep our site tools on the left-side menu so that they are still easily accessed and probably easier to see overall than the previous design.

I made a large number of changes to the overall code structure and I think it's now much more readable than it was originally. I've unified the stylesheets into a single stylesheet that handles all the major browsers. I'm hoping to remove our "Print this page" link by using stylesheets to generate a printer-friendly version of the documents on the client side.

Best of all I've done all this without affecting the look of the site. I'm having one minor problem with the width of the content area, but it's hardly noticable … especially after looking at the changes some of the other directorates have made.

We're sending a list of corrections and suggestions for improvement to Kelly Gayden next week. The list ended up being longer than I expected, but the majority of what's in there are minor requests.

When I have time I'll prepare the work I've done so that I can send it back to Kelly. Hopefully the upgrades will make the site easier to manage. After that I want to see if I can get rid of some of the table-based layout code in favor of stylesheet-based code. I think it will significantly improve the managability of the site's code.

Mac weirdness
This one is a bit annoying. IE for the mac (am I still working around this dead program?) was randomly setting the width of some pages to about triple the size of the browser window. It turns out that there's a bug when a div element is floated and contains two block elements inside it. In this instance IEMac stretches the width of the parent element in an ungodly manner. This was a problem because of the right-side content I had to repurpose when preparing for the redesign. I ended up working around the problem by having IE just display these elements inline rather than floated.

CCMS Web Site

There was a complaint that the CCMS site was doing some weird stuff with the design (randomly choosing the text version over the graphical version). I was unable to reproduce the problem even under the same platform/browser combination. After pouring over the code I think the problem was the manner in which I was attaching events to the document when it loads. Initially there was a both an html attribute (for the init function) and javascript statement (for the stylesheet switcher). I think the two were having problems cohabitating in some browsers because the javascript statement used "onload=" rather than a less destructive "addeventlistener" function. I've modified the code so that the function call that was originally in the html attribute is now part of the javascript statement.

Contribute

Microsoft vs. Macromedia
Dave's been working to improve the setup of our network and his latest step was to not give users admin rights on their local machines. Unfortunately, after doing this these users were unable to start Contribute. Dave worked on it for a while but eventually gave up … not to mention that he's got a ton of other things to do rather than figure out why Contribute won't run. I gave it a shot and managed to figure out what was going on. Turns out that the Macromdia Licensing program runs as a server. In WinXP regular users are unable to start a service. See the problem? I've worked around the issue by having the licensing service start at boot time.

Navigation - Show/Hide

I needed to create a show/hide menu navigation for a document being posted on the site. The old script was a bit clunky and required quite a bit of work on my part to implement. I went ahead and wrote a newer script that is much more modular and requires little extra work when used on a page. The new JS file includes documentation and sample. A quick overview:
  • Additional code required for use: divs around menu and each document section (id for the section divs should match the anchor links in the menu)
  • attach JS file to document
  • attach stylesheet to the document with a few required entities
The script is "non-destructive" … it shouldn't affect any other scripts on a page. I made the function, variable, and CSS names pretty unique and the onload statement is done using the "addeventlistener" method (which requires a DOM-capable browser … but that's less of an issue than it used to be).

Intranet

I took some time to make some minor modifications to the intranet, mainly for my own sanity.
  • Since I'm not using our mail server to retreive mail anymore I had to add a special username/password combo for my own login
  • I added a link to the old e-mail interface
  • I added a link to the mass mail database update page
  • I added a link to the file upload page on the development server
  • I modified the mass mail database management pages to include the intranet header

404 Script

I finally found some time to make the 404 custom error page case insensitive. This was a problem with the 404redirect.txt lookup portion. the URLs "/2061Connections" and "/2061connections" (for example) did not both resolve … which is desireable with the variability in links that are out there. Now a UCASE is done prior to comparison and replacement.