Inbound email filtering beyond Rules

[Starbuck]

It looks like microsoft.public.outlook is largely inactive so I'm reposting this here. I used to post in the SlipStick forum but this is my first posting here. I apologize if this isn't the right place for this inquiry.

I'd like to understand how Outlook 2010 Rules have improved over v2003.

Several years ago I found limitations in Outlook 2003 Rules. Outlook would behave weirdly and crash if there were too many rules. It also doesn't deal well with rules for non-standard headers. My solution was to create a series of VBA scripts that processed inbound emails against an external file of rules. Each rule in the file identifies a header, a condition, a value to match the condition, and the target folder when a match is made. Examples:

SenderEmailAddress|includes|linkedin.com|\\Personal Folders\Inbox\Registered Sites\LinkedIn

List-Id|is|<foo.googlegroups.com>|\\Personal Folders\Inbox\Lists\Active\Foo

X-Server-From|includes|zzz.org|\\Personal Folders\Inbox\Lists\Active\ZZZ List

From|includes|zzzz.zzz|\\Personal Folders\Inbox\Company1\Clients\ZZZZ

To|includes|sales@company2.com|\\Personal Folders\Inbox\Company2\C2 Sales

Reply-To|includes|@yyy.com|\\Personal Folders\Inbox\Lists\Inactive\YYY

A form is available in my toolbar which allows easy creation of these filters with a mail item and folder selected.

Prior to that processing, if an email matches an existing Contact, I have a path to the folder where emails from that contact should go. Example:

=MOVETO:\\Personal Folders\Inbox\Company2\Clients\ZZZZ=

Further, once the disposition of an email is determined, all non-essential headers are removed to help keep the size of my PST down.

So all of this sophisticated handling (yes, and much much more) is in Outlook 2003 VBA because Rules just didn't cut it. I have bypassed v2007 and am installing a new system with Outlook 2010.

My questions are:

1) Do I need to continue using my scripts to get this level of

filtering in Outlook 2010?

2) Is there a commercial "uber filter" out there?

3) Should I convert my filters into addins and sell them?

4) Is there a better forum/venue for these questions?

Thanks!!!

 

[Diane Poremsky]

Microsoft shut down their nntp servers about a year ago. This is the slipstick forum- it just has its own domain now.




1. Yes i think you will need to keep using your scripts. While rules have improved, its mostly to add more actions or conditions.


2. There are very few commercial filters available and i don't think any are as detailed as your filter - i use Auto-Mate because it can apply rules to mail after a period of time. None of the commerical options can remove headers, that i know of.


3. YES!!!! If you do, let us know and we'll mention it in our newsletter and on our site. http://www.slipstick.com/new-or-updated-product-submission/


4. There are other forums but none are better - some are worse. Microsoft has a forum at answers.microsoft.com but its staffed by contractors who don't know outlook.

 

[Starbuck]

Wow, thanks for the prompt response, Diane. Your enthusiastic response to #3 is particularly interesting. Searching this forum I see people asking about filters and thinking "hey, my macros do that". Auto-Mate seems to be one of just a very few options available, and while a competent offering it does have its limitations. With email filtering being so important I'm really surprised that this isn't a saturated market niche after so many years.

Indeed, as time permits I'll migrate this functionality into an addin which can be loaded by others. I've written a lot of addins for myself and clients but haven't undertaken anything for general market consumption, and nothing for v2010 yet. It's about time.

Based on what you see in my original post, please feel free to post here or PM with feature suggestions. Since I don't use Exchange Server yet my initial target audience would only be PST users. If the Exchange user base is huge enough to warrant a re-focus, please let me know. Other implementation suggestions are welcome.

 

[xbliss]

What are the best or top extensions/ add-ins for Outlook that have BETTER filtering/ sorting rules than Outlook itself.

I'd hope for something that supports:

- Variety of Boolean combinations

- Regular Expressions

Or are there some user scripts / VBA ones created by someone in this direction?

 

[Starbuck]

What are the best or top extensions/ add-ins for Outlook that have BETTER filtering/ sorting rules than Outlook itself.

I'd hope for something that supports:

- Variety of Boolean combinations

- Regular Expressions

Or are there some user scripts / VBA ones created by someone in this direction?

Based on this thread I did create an Outlook 2010 addin which does excellent filtering. I haven't published it because there seem to be issues when run over WinXP and we just haven't had the time to really beat it up for a production release.

For any mail item you can right click a folder and select a new context item "Advanced Rule Builder". That opens a form which displays significant headers and the current email's values. Select a header/value, and then select how a new rule/filter will select on this: Equals value, IsNot value, Starts With, Ends With, and Includes. The value can also be tweaked, for example, to change "FromAddress EndsWith @anydomain.tld", so a single filter can be set for an entire company.

From there the filter is added to a list, of rules that gets processed for each mail item. Sorting and moving the rules around is easy, so that you can quickly position the new rule to get executed higher up than others. For example you might want to put the above filter above a check for spam to ensure "spammy-looking" emails from a trusted partner still get through.

Of course this all works on the new item trigger, so new items get filtered, and grouping Unread Mail by folder shows all mail neatly organized.

All of that works much like the default Outlook rule builder but it's much more robust and user-friendly. You can right click any mail item and select Test Filter to see where it would go using the current set of rules, and then Filter to actually send it where it belongs. This is helpful if you revise filters and want to do a mass migration of items to their new home.

It does not yet include regular expressions but that can easily be added. I wanted to go forward with a v1.0 and then add regexps to a v1.1+.

In addition to a couple minor anomalies which prevent me from taking this public, I was concerned that Outlook 2013 was going to be a game changer for addins, so I stopped development. I've only recently realized that wasn't true, that O2013 wasn't bound hand-in-hand with Office365 and that my code "should" port without issues. Considering most people don't upgrade to a new release for some number of years I should have realized that doing a v2010-only cut would have a long life-cycle. Considering this should work with v2003 all the way through v2013 now, I know I should spend more time on this but I just haven't done it. As I think about this, I'd bet that just one corporate sale of a couple hundred low-cost licenses would justify the effort. But that also means I need to build an elegant licensing/activation model into this. That's just another tier of grief.

So ... All of this is within reach of any company that's interested, I just need the right "motivation" to complete the effort.

I hope that properly responds to your post ... and then some.

 

[Diane Poremsky]

What are the best or top extensions/ add-ins for Outlook that have BETTER filtering/ sorting rules than Outlook itself.

I'd hope for something that supports:

- Variety of Boolean combinations

- Regular Expressions

Or are there some user scripts / VBA ones created by someone in this direction?

There are very few. I use Auto-Mate because it supports moving mail after it arrived. VBA works ok (either run a script or itemadd), but it's really best for a limited number of rules.

processing-incoming-e-mails-with-macros

 

[xbliss]

Based on this thread I did create an Outlook 2010 addin which does excellent filtering. I haven't published it because there seem to be issues when run over WinXP and we just haven't had the time to really beat it up for a production release.




For any mail item you can right click a folder and select a new context item "Advanced Rule Builder". That opens a form which displays significant headers and the current email's values. Select a header/value, and then select how a new rule/filter will select on this: Equals value, IsNot value, Starts With, Ends With, and Includes. The value can also be tweaked, for example, to change "FromAddress EndsWith @anydomain.tld", so a single filter can be set for an entire company.




From there the filter is added to a list, of rules that gets processed for each mail item. Sorting and moving the rules around is easy, so that you can quickly position the new rule to get executed higher up than others. For example you might want to put the above filter above a check for spam to ensure "spammy-looking" emails from a trusted partner still get through.




Of course this all works on the new item trigger, so new items get filtered, and grouping Unread Mail by folder shows all mail neatly organized.




All of that works much like the default Outlook rule builder but it's much more robust and user-friendly. You can right click any mail item and select Test Filter to see where it would go using the current set of rules, and then Filter to actually send it where it belongs. This is helpful if you revise filters and want to do a mass migration of items to their new home.




It does not yet include regular expressions but that can easily be added. I wanted to go forward with a v1.0 and then add regexps to a v1.1+.




In addition to a couple minor anomalies which prevent me from taking this public, I was concerned that Outlook 2013 was going to be a game changer for addins, so I stopped development. I've only recently realized that wasn't true, that O2013 wasn't bound hand-in-hand with Office365 and that my code "should" port without issues. Considering most people don't upgrade to a new release for some number of years I should have realized that doing a v2010-only cut would have a long life-cycle. Considering this should work with v2003 all the way through v2013 now, I know I should spend more time on this but I just haven't done it. As I think about this, I'd bet that just one corporate sale of a couple hundred low-cost licenses would justify the effort. But that also means I need to build an elegant licensing/activation model into this. That's just another tier of grief.




So ... All of this is within reach of any company that's interested, I just need the right "motivation" to complete the effort.




I hope that properly responds to your post ... and then some.

I would love to beta test and help you with this and maybe come up with more variations / variety on it. Let me know.

There are very few. I use Auto-Mate because it supports moving mail after it arrived. VBA works ok (either run a script or itemadd), but it's really best for a limited number of rules.




processing-incoming-e-mails-with-macros

Thank you for the insights. I just wish we had better ability to filter and sort emails based on rules and regular expressions.

 

[xbliss]

There are very few. I use Auto-Mate because it supports moving mail after it arrived. VBA works ok (either run a script or itemadd), but it's really best for a limited number of rules.




processing-incoming-e-mails-with-macros

Could you please list/ name some of the top ones? Maybe with some experiences of pros and cons?