Introduction

Equinox is an email client library targeting .NET 4.0 and Mono 2.8. For now, the plan is to support IMAP and SMTP; POP3 is not planned yet. It is written in C#, consisting of 100% managed code, and can be found on CodePlex. With this article, I want to peek your interest, give you an introduction into the features of this library, and provide a little explanation of why I found it necessary to implement yet another email client.

Background and Motivation

The reason for my creating this email client library is simple. For another Open Source project, I required an Open Source library to add email support, leveraging Mono and the .NET Framework.

You might ask, why not just take one of the bazillion Open Source libraries floating around the web? Well, it is a valid question and I was in fact planning to do so. Unfortunately, reality struck hard. I tried several libs and most of them performed fine for about 70% to 80% of all mails, but in the end, all of them failed in certain aspects.

Some had problems with encodings or didn't even encode at all, some performed poorly in parsing more complex MIME constructs, many only supported a small subset of the protocols capabilities, some just seemed wrong ... and the feature that was neglected most of the time was an adequate implementation of the fetch command.

The fetch command is the most important and complex part of the IMAP protocol because it is the only command which produces dynamic responses depending on the query and the server implementation, which is probably the reason why most implementations only offered a simple fetch-headers or fetch-all solution.

On first glance, this seems to be enough, but if you take a closer look, you will see that there's more going on. It is, for instance, useful to know if a mail has been read or not; for this, we need the message flags. Sometimes it is nice to know the size and layout of the message before actually having to download the message, or to know whether it has attachments or not. Some might say it would be great to be able to read the three lines of plain text without having to download the 20MB attachment, or check for a specific header without having to fetch all 34. Combined with a proper search command, the fetch command is capable of getting exactly what you want with almost surgical accuracy and low overhead, but only very few clients utilize these features, mostly due to the lack of necessity.

On a desktop machine, we are usually connected through a broadband internet connection where it hardly matters if we download 10K, 100K, or even 1MB of data, but there are situations were it does matter. Imagine your mobile phone downloading complete messages through your UMTS, HSDPA connection, it would take quite some time until you'd be able to enjoy the 10 MB holiday photos sent to you.

Of course, this library is not intended to run on mobile phones, but to make matters more clear, the application I was talking earlier about is intended to run on tablets and pads with Linux and Windows. On those so called mobile computers, we don't always have the luxury of high speed broadband connections and more or less "unlimited memory" to store tons of data we will never use that just came along for the ride. Needless to say, I believe it is better to be safe than sorry.

Of course, every library I inspected did give the user control to sent custom so called "raw commands" to the server. Essentially, you can create this surgical efficient query by hand and sent it to the server. Unfortunately, the response you'll get will also be very "raw", it will mostly be a MIME encoded part of the message and this is where the trouble begins.

You don't need an IMAP/SMTP or POP3 library to make these "one line commands" like, for example:

Send("LIST #news.comp.mail.misc \"\");

You need it for the complex stuff like parsing MIME structures, identifying body parts, encoding and decoding for transfer, but what's the point of having a library when it comes with a "do it yourself" policy on the most important part? Although some of the libraries I tested where really well designed and easy to use, I always ended up inside a forest full of Regular Expressions and string comparisons, doing it myself when it came to fetching something else than everything.

All in all, I just wasn't satisfied with the solutions out there, therefore I took it upon myself to develop yet another email client lib trying to address some of those issues.

There are already libraries that have solved this issue very elegantly, but unfortunately, they come with a price, all in all not the first choice when it comes to Open Source projects.

Structure of the libary

I decided to split the library into four main assemblies.

  • Crystalbyte.Equinox.Core
  • Crystalbyte.Equinox.Mime
  • Crystalbyte.Equinox.Imap
  • Crystalbyte.Equinox.Smtp

This way the user can choose which libraries he needs and omit the rest. This isn't about size since all assemblies together take up like 190 KB; by dropping the IMAP assembly if you only want to send mails, it comes down to about 90 KB. No, separation forces you to properly design your application. For instance, the MIME assembly has no dependencies to the others and can therefore be used as a standalone MIME parser. The Core assembly holds shared classes that are used by both the SMTP and the IMAP assemblies to reduce redundancy. Both SMTP and IMAP, for instance, use exactly the same client class to communicate to the server. The commands are different, but everything else as in communications, SSL Encryption, and SASL authentication algorithms are identical. In addition, the SMTP lib does not depend on the IMAP lib, or vice versa.

Using the SMTP client

Compared to the IMPA client, the SMTP client is a trivial class. There is only a single method of interest which is Send(...), obviously. The Core assembly contains all model implementations including the Message class. Once created, we pass an instance into the Send(...) method and we're done, it's really not that exciting but works adequately.

using Crystalbyte.Equinox.Security;
using CRystalbyte.Equinox.Smtp;

var message = new Message();
// fill the message object

using(var client = new SmtpClient())
{
    // Connect
    // Login
    client.Send(message);
}

I know it's not much, but the SMTP protocol really isn't that spectacular, which is why we're going to continue with the IMAP client.

Using the IMAP client

Since the CodePlex documentation is not yet up to date, I will introduce the basic usage of the IMAP client in this article. I tried to keep the architecture and structure as simple as possible, so I'm just going to jump in. Let's create an instance and log us in.

using Crystalbyte.Equinox.Security;
using Crystalbyte.Equinox.Imap;

using(var client = new Client())
{
    var host = "foo.bar.com";
    var port = Ports.Default; // 143 (993 for SSL)
    client.Connect(host, port);
    client.Login("name","pass");
} // Dispose() => client.Logout(); client.Disconnect();

Like with most clients, the strongest supported SASL mechanic will be used to authenticate the client to the server, so there is no need to fiddle with those. For now, the following SASL mechanics will be supported:

  • PLAIN
  • LOGIN
  • CRAMMD5
  • OAUTH (partially automated)

If the client is unable to find a mutually supported SASL mechanic, the client raises the ManualSaslAuthenticationRequired event providing the user with the server capabilities and the means to authenticate manually. This for example is necessary when connecting to GMail, since Google nowadays only supports the XOAUTH SASL authentication.

Encryption is off by default, but can be enabled by changing the client.Security property before connecting.

client.Security = SecurityPolicies.Implicit; // STARTTLS
client.Security = SecurityPolicies.Explicit; // SSL

When leaving the scope of the using block, the client will automatically log us out and close the connection gracefully. If you don't have the luxury of a using block, you can of course call the appropriate methods manually.

Once we are logged in, we can call other methods on the client. I'm not going to present the basic IMAP methods here like Rename, Store, Copy, Delete, etc., since they are all trivial and can be found all over the web.

In fact, there is only a single aspect to this class that differs from other clients, which I will talk about next.

Fetching using LINQ to IMAP

Although the client has a regular Search() and Fetch() method, I would not recommend those for anything but the most simple requests. I previously talked about addressing some of the issues I criticized earlier, one of these were inflexible or incomplete implementations of the fetch command.

To address this, I implemented a LINQ provider that enables us to fetch messages or parts of messages directly from the server, which has two advantages. First, no matter how complex the query is or how many items we request, it will all be done in a single stroke using one fetch command; although this saves some net traffic, it is not the smoking gun we were hoping for.

The important part is the fact that we don't have to parse or map any responses manually anymore since this will be taken care of by the LINQ provider.

Let's take a look at a simple usage. In the following example, I will fetch the following items from all unread messages for the last week:

  • Envelope
  • Uid
  • Flags
  • Size
var query = client.Messages.Where(x => x.Date > DateTime.Today.AddDays(-7) 
    && !x.Flags.HasFlag(MessageFlags.Read)).Select(x => new MyContainer
{
    Envelope = x.Envelope,
    Uid = x.Uid,
    Flags = x.Flags,
    Size = x.Size
});

If we need a different fetch scenario, we just change the query. We can fetch less, like only the Envelope.

var query = client.Messages.Where( ... ).Select(x => x.Envelope);

Or we can fetch more:

var query = client.Messages.Where( ... ).Select(x => new SomeClass
{    
    Envelope = x.Envelope,
    Uid = x.Uid,    
    Flags = x.Flags,
    Size = x.Size,
    Internal = x.InternalDate,
    BodyStructure = x.BodyStructure,
    SpecificBodyPart = (Entity) x.Parts("1.2.1.TEXT")
});

We can then execute the query by iterating through the results.

foreach(var container in query)
{
    Debug.WriteLine(container.Envelope.Subject);
}

The point is that the only thing to change is the query when altering our fetch scenario. As it is with LINQ to SQL, we don't have to worry about parsing the data that comes out of SQL Server anymore, we just map the responses into our object. Without LINQ involved, we would need to either create a different parser for each of those scenarios or create a single parser that would be able to handle different but still only a finite amount of responses, and once we change the query, we would also be forced to change the parser.

As with many LINQ providers, there are limitations and restrictions since we have to work within the boundaries of the IMAP protocol. Multiple or nested Where/Select statements are not permitted; to be more precise, we need exactly one Where and one Select clause. With a few exceptions, none of the other extension methods like Any(), Single(), or SelectMany() are supported. A more detailed list of supported and unsupported methods as well as instructions for common fetch scenarios will be posted on CodePlex soon, but getting into the handling of custom providers is out of the scope for this article.

Fetching using convenience methods

Although everything can not be, everything should be fetched using the LINQ engine. When you have decided to download the complete message without any omissions, the client has two methods that will accomplish the task:

var message = client.FetchMessageByUid(187);
var message = client.FetchMessageBySequenceNumber(10);

Earlier I talked about downloading separate parts of a message, remember? Well ... for this to work, we need the body structure since it contains a compact version of the message's entity object model.

Within this model, there are four types of interest:

  • Views (plain/text, HTML, etc.)
  • Attachments
  • Nested Messages
  • Related Objects (embedded stuff and such)

Let's fetch the body structure for the first 20 mails.

var query = from m in client.Messages
            where m.SequenceNumber >= 1 && m.SequenceNumber <= 20
            select m.BodyStructure;

Let's now assume the fifth message has a nested message which contains a single attachment, still with me? Let's get it!

var structures = query.ToList(); // all our body structures
var handle = structures[4].Children[0].Attachments[0]; 

var attachment = client.FetchAttachment(handle);

Similar methods are available for the rest of our types of interest. If you are wondering why we don't fetch these things using LINQ, I must tell you that we actually do. If we take a look into the FetchAttachment( ... ) method, we will see the following:

public Attachment FetchAttachment(AttachmentInfo attachment) 
{
    var token = attachment.Token;
    var query = Messages.Where(x => x.SequenceNumber == 
                 attachment.SequenceNumber).Select(x => x.Parts(token));
    return ((Entity) query.ToList().First()).ToAttachment();
}

All fetchable items inside the body structure hold a token like "1.TEXT" or "1.2" representing the position inside the MIME encoded message according to MIME specs, created during parsing. All we are actually doing is provide the LINQ engine with this token and parse it to the expected type. Since this requires some inside knowledge of the client's structure when casting to the correct type, I decided to wrap these queries into methods.

At this point, I feel obligated to tell you that this is the first release of the software and considered Alpha. There are still bugs, but the demo app runs fine and works for more than 95% of all approx. 2000 tested mails. I'll encourage you to give it a try.

In addition, the body struct parser still has some issues; if you have ever tried to parse the body structure by hand, you'll know, it's a pain in the .... Since I wanted a real parser and not just some half baked excuse for a Regular Expression, I needed to create a fully recursive parser similar to the one used for parsing MIME. Unfortunately, parsing the body structure is much harder than parsing a MIME part, which is probably the reason only a single library I tested even bothered to try. The parser is implemented using Coco\R which, by the way, is a great piece of software, but creating a parser based on an EBNF grammar is not trivial and requires some time.

Threading

You might have noticed that there is not a single ...Async method or Begin ... End ... pattern implementation. I decided against implementing asynchronous behavior since I found it a trivial detail only obstructing the code. In essence, all we need to do is surround our code with:

new Thread(
    () => {// Do synchronous stuff }
).Start();

Perhaps, some now will say: "Wait! ... there is a lot more to consider when using asynchronous calls." I agree, but those concerns won't disappear just by implementing asynchronous methods into the client.

The end

That's pretty much it for now. Since this is my first article, I hope you can spend some time and leave a comment.

There is obviously more to this library but going into every detail would make this a really long and boring article, more like a manual if you catch my drift. I promise to update the documentation on Codeplex to reflect capabilities and limitations soon.

As mentioned earlier, there is a demo app I implemented to demonstrate the basic features and capabilities of the IMAP client, which is included into the binary release package on CodePlex.

It is important to note that the app does not run on Mac OSX although the library does. The demo app uses the WebBrowser control to display HTML views, which is not (yet) supported on Mac OSX by Mono.

关键字 yet another email client,linq to imap
推荐.NET配套的通用数据层ORM框架：CYQ.Data 通用数据层框架
新浪微博粉丝精灵，刷粉丝、刷评论、刷转发、企业商家微博营销必备工具"