POP3 E-Mail: W5100 + AVR

The point of this page is to give a concrete example of how to use the W5100. Scroll to the bottom if you just want the code. Here follows some why/how...

If there is one killer app that made the internet what it is has to be e-mail. E-mail is ubiquitous. You can send e-mail from all sorts of devices today, even the most basic cell phone can send e-mails via SMS. This makes e-mail a great way of exchanging information for nearly any project in a fixed location that has internet access. For a mobile location a cell phone module would be more practical.

E-mails can be fetched by a computer and sent to a project using some interface. However, I have found leaving my computer on all the time just for this purpose wasteful, and with power outages and reboots for updates it is also unreliable.

An embedded system would be better, but e-mail has gotten to be complex, when POP3 was invented there wasn't a lot of need for security on the internet. Now there is, and most services (IE gmail, etc.) are only accessible via SSL. Using SSL on a 8-bit microcontroller would require quite a lot of work. Thankfully there are still some services that don't require encryption for simple POP3. At the time of writing one free service I have found is lavabit.

Now, to put an AVR on the internet (to get that email) you need a TCP/IP stack. There are two kinds software stacks and hardware stacks. A good example of a software stack is uIP when combined with a enc28j60 some magnetics and other bits you can get your AVR on the internet (or you can use PIC and Microchip will give you a lot of code). This is actually a lot of work and a lot of overhead for the uC. The good news for uIP and AVR is that much of the work has already been done (IE there are ports of it such as this one). But it still is a lot of code.

The alternative is a hardware TCP/IP stack. This is more expensive but only by a few dollars (per unit). The W5100 from Wiznet is what I selected. Particularly the WIZ811MJ module. This chip basically gives you 4 sockets and 8K of buffer memory, far more than my ATMega168 has. Frankly, I don't think much of the datasheet. However, if you read carefully there is (as always) a lot of information contained in there.

The W5100 can be accessed in 3 ways: direct, indirect and SPI (Mode 0). The first and the second are basically parallel interfaces and the third is obviously serial. Depending on your micro of choice you might not have enough pins for either of the first two. SPI only requires 4 pins.

If you look on the datasheet on page 62 you may notice something interesting about SPI. It doesn't say it anywhere but the figure, but the SPI replies are 0, 1, 2, 3 for writing, and 0, 1, 2, data for reading! Useful to know if you mess up your SPI link for some reason. But this is typical for this datasheet, it is there but it isn't there very clearly.

Page 33 illustrates another important item, under Sn_TX_WR it reads:

"When reading this register, user should read upper byte (0x0424, 0x0524, 0x0624, 0x0724) first and lower byte (0x0425, 0x0525, 0x0625, 0x0725) later to get the correct value."

What it does not say is that this also basically applies to writing. So even if you know the upper byte will remain the same you must write both. This no doubt is because it really is a 16-bit register underneath.

The example code: email.c

So here I provide you with a sample bit of code to run and check email via POP3 using a single socket. It prints the emails out over the UART. Below I will talk a bit more about the code, and what is important in it.

Be sure have flashing the EEPROM enabled in the makefile and change the values of MAC, and IPs to work for your POP3 server/LAN.

Here is how I wired it up:

There are actually quite a few drivers and things that use the W5100. There is an Arduino module for it, and library. But of course it costs a fortune and doesn't teach you anything about the W5100, SPI or anything really. There is also a driver for BOSCOM, same story really. The thing is even if there is a driver out there, if there is a bug the level of abstraction used sometimes means it is a bit hard to follow. I don't like seeing some abstract name having to find that name which points to another name only to tell me the value in a header file is 0x53 which is in the memory map on page 5 of the datasheet.

In the end interfacing with the W5100 is basically stuffing data into registers and memory and tell it to do some operation.

Whenever I write something from scratch like this I start at the specific and go to the general. People might take issue with my code style in that it is very specific. In fact, as written it will only work with 2K for each socket. But that means to change it you will need to understand the hardware (a good thing). If there is a bug (which there maybe) the memory address is right there inline.

Once I see the code works you can generalize it and know that nothing else but the last thing I touched is wrong. I find this works much better on embedded systems than a very abstract approach because nature of the bugs these system have. It is best to get something working and iterate.

The first functions I wrote and most important ones therefore are:

uint8_t writeAddress(uint8_t high, uint8_t low, uint8_t value)  
uint8_t readAddress(uint8_t high, uint8_t low)

These write and read a byte on the W5100 over SPI. They are the functions everything is built on. SPI has to be setup correctly but these are the functions that I wrote first. writeAddress returns 1 for success and 0 for failure. readAddress returns the value of the memory location. Since the memory location can be any 8 bit value it does not do anything with the validity bytes r1-r3.

The next most important functions are getData and sendData. These are written differently than the datasheet suggests, because again I like the specific before the general. However, they are for reading (get) and writing (send) data from socket0 in the 2K/2K/2K/2K configuration.

getData reads only up to the size of the RxBuffer on the AVR. This is important.

One thing to note about TCP/IP that I had to learn. What happens if the server wants to send you more data than you can hold? Well by using a packet sniffer I found that the W5100 closes the window size for the TCP connection. The server will then send you 1 byte at a time until the W5100 replies that it has room in its buffer again. This probably means you need to read the buffer of the W5100 and process the data in a timely fashion before your connection is dropped by some timeout on the server.

When receiving data that is long one needs to sit and read data out of the buffer and move the pointers (getData), until you are done getting the data (an example of this is in pop3getmessage). In POP3 there are several ways to know when you have all the data. Long messages always end with a period on a new line, for instance. However, I choose to use the LIST command to know how much data I needed to expect.

POP3 basically runs like this:

You open a TCP/IP connection, and socket to port 110 on the server. The client normally has a high port number. (1024+110). openTCP(...) does this and returns 1 if successful.

Once the connection is established you do all the POP3 things (getMail() does this). First the server will reply with some welcome message. You need to get this out of the buffer. (pre Case 0).

Then you have to log in with USER and PASS command (Case 0 and 1). Next you look for how much mail there is (STAT, case 2). You parse the reply to find out the number. In the current code if there are more than 9 you just get the first 9, though with a few more lines you could get 255. Next you get the message (Case 5) and delete the message if that option is turned on (Case 6). If there is still e-mail to handle you go back to step (Case 4) and repeat. If not you quit the session (Case 7). The popCommand function returns 1 if successful letting you repeat unsuccessful steps. You may have also noticed a timeout on the whole process. This is done so you don't get trapped. The time is reset after each successful message retrieval, as if the email is long this could take some time.

pop3command and pop3getmessage are just abstractions of sendData + getData. There is a known limitation to pop3command. If you look at it because it only reads data once, if the data is longer than RxBuffer you will have an issue of data from commands mixing. The only command capable of doing this is LIST without an argument for a mailbox full of messages. This never needs to be issued which is why there is no function to handle it. RxBuffer must be long enough to handle at least a simple command reply (+OK 5 3792 or -ERR invalid command for instance). The larger the better more or less.

This code is just a snapshot. I may update it. But as I continue to add it will become less of a clear example of how to use the W5100. So while it isn't finished or polished here it is for you to learn from and use. Enjoy!