A little misleading of a title, but it gets the point across nicely. The MMC GUI tools don't support the creation of a domain in the SMTP console. The solution, though, is fairly easy. Shut down IIS and edit the metabase directly. If you're lucky enough to have Windows 2003 it's super-easy since the metabase in in XML format.
The procedure I used was to creat a domain of the literal sans brackets first. This makes it easy to find the appropriate entry in the metabase. Then I edited the metabase to add the brackets back in. Easy.