Ask Hackade: Is big (e-mail) good?

Although pundits regularly predict the end of e-mail, we still get a ton of it and we bet you will too. The e-mail has been around for a long time and during the day, being able to shoot a note asking where they want to go for lunch was quite hi-tech. What was on our computer back then was very different. Consider that the first e-mail on ARPANET was in 1971. Then some people had a hardcopy terminal. The graphics were unusual, and your main storage was probably a fraction of the smallest flash drive on your desk. No one was sending photographs, videos or huge PDF files.

Today, things are different. Our computers have gigabytes of RAM and terabytes of storage. We create and use lots of formatted documents, high resolution photographs and even videos. Naturally, we want to share those files with others, but the e-mail is too small. Of course, some systems will offer to hide your large file in the cloud and send a link, but e-mailing a multi-megabyte video to your friend across the city is more likely to simply fail. Why?

The life of e-mail

As you might expect, it has some historical features. The system is not designed to handle your latest vacation videos. More importantly, e-mail means widely interoperable across different types of systems. Of course, these days, no e-mail is likely to pass over a curved pair connecting two remote outposts, but it Can.

To understand the origins of boundaries, you need to know how an e-mail lives its life. This is much more complicated than you expected. Today, mail travels using a protocol called SMTP – Simple Mail Transfer Protocol. Let’s take the general facts first.

Let’s assume we have a e-mail address and we run our own e-mail servers (we don’t, but go with me for a minute). A user connects to the SMTP server and sends an e-mail to another address at It’s easy. The server acknowledges that this is the ultimate destination for e-mail, so it accepts and distributes it. What does delivery mean? Depending on it, however, it usually means keeping some files that the user will check with a mail program or other type of server such as POP3 or IMAP. But for our purposes, this means that the mail is kept in a place where users can find it.

A little more difficult is when someone from wants to send an e-mail to someone on Hackade (again, assuming the domain is managing its own e-mail which is not as common as before). In this case, the user connects to their own SMTP server. It realizes that @ is an alien e-mail address (maybe in multiple ways) and searches for domains in the DNS directory. One of the DNS records (MX records) is the domain’s mail server address.’s SMTP server will then send mail to which will actually deliver.

And then that?

It’s all there, isn’t it? That’s not the case. Nowadays, those DNS entries probably point to some services like a Gmail server or some other hosting company that will actually receive e-mail. The sender is probably connecting to some vendor’s server. So maybe the sender connects to the Johor SMTP server which then connects to the Gmail server to deliver the mail.

It’s more complicated when you go to a personal network. If the wd5gnr network is behind a firewall, it may not be possible to communicate directly with the hacked server wherever it is. What then? An SMTP server can receive mail on one interface and then relay it to another server. This may continue for some time. For example, the wd5gnr server can relay everything in Gmail which then finds the main Hackaday server which sends it to the internal mail server for specific users in that part of the world.

It may sound remote, but it happens all the time. For example, you could be on a ship at sea with a non-stop connection to a satellite link. A server can collect e-mails and wait for a connection to the satellite. It then sends the mail to a server via a satellite link which will take care of sending it to the original network.

You can see why e-mail has become so ripe for exploitation. Initial servers will allow anyone to connect and may even try to relay mail to anyone. Spam has certainly changed everything, and now an open relay is a rarity. Some SMTP servers reject incoming mail through known open servers. This is controversial in some circles as a form of censorship, but most servers will now require authentication and will receive e-mails intended for specific domains without authentication. The number of servers checking an SPF record to make sure a server is known to handle mail for a specific domain has also increased. DKIM goes one step further and uses a cryptographic signature.

The weight of history

So the fact that mail rarely, these days, leaves the network or flows through an open relay, to the point that it can. Why that matter? Let’s go back to the ship with the problem of the sea. The ship’s server may have to encode our e-mails in some awkward format or compress it. The Satellite Ground Station can only send e-mails via UUCP which needs further processing. So big e-mails are not just a network bandwidth problem. The server may need to process and convert files. SMTP, therefore, may require 7-bit ASCII which is great for text. But for anything, the message (with attachments) is encoded using something like the base 64 encoding specified by MIME (Multipurpose Internet Mail Extension). This means that your attachments are likely to get bigger after encoding.

Even a fully connected host may want to process e-mail messages. For example, spam or virus payload testing is common. So even with infinite network bandwidth, there is still a processing overhead associated with large connections.

Granted, processing power is scarce these days, but some mail servers may not be sophisticated, others may be hit by millions of e-mails, many of which are spam.

Ask Hackade

How do you send large files? Do e-mails need to take large files? What do you think of e-mail? Replaced by what? How easy can it be? There are services like WeTransfer and SendAnywhere, but to be honest, we can upload a file and send a link to one of our servers. Then there is always cloud storage like Dropbox, Drive or iCloud.

We have come a long way from career pigeons. Then again, maybe your mail is going over that network. If you want to dig deeper into SMTP, watch the video from Iowa State University below.

Leave a Reply

Your email address will not be published.