Can digital identity curb the spam assault on your inbox?
04 January, 2008
category: Corporate, Digital ID, Library
Daniel Butler, Contributing Editor
Whether you like it or not, spam may be here for good. The world is at war with it, and spam may be winning. You may recognize some of the terms of warfare: spam, ham, junk, not junk, UCE, UBE, Make Money Fast, Viagra, Cialis, stock scams, viruses, trojans, worms, exploits, phishing, malware, 419 fraud, mail headers, Bayesian filters, filter poisoning, spam scores, white lists, black lists, block lists, port 25 blocking, list washing, spam houses, bounce counts, address harvesting, dictionary attacks, CAN-SPAM, honeypots, botnets, zombies, opt-in, opt-out, false negatives and false positives.
The situation is critical … not just for you but for everyone who uses computers, web sites, search engines, phones, or other forms of communications. Modern society’s function is hopelessly intertwined with our ability to communicate quickly and cheaply via email, so you may ask yourself: Why do we put up with spam? Why isn’t email more reliable? What are the smartest minds trying to do about it? And, is there a silver bullet fix to the problem?
In this three-part article, we’ll discuss the origins of this assault on your inbox, investigate a current attempts to quell the flood of junk email, and propose that the concepts of digital identity may be the key to a brighter future, a cleaner inbox and secure messaging that you can trust.
Just how bad is it?
I fought with spam myself for years, having one of those email addresses that has been “out there” since around 1995. I hosted my own email server, and despite futile efforts to stem the tide with special mailer configurations, blacklists, and spam filtering, I was getting thousands of spam messages a day, nearly more than my aging server could handle. In 2005, I gave up and moved my email service to a one of the biggest providers with some of the best filtering technology that is available, and my inbox was finally liberated. My spam folder, however, was filled at its peak with over 22,000 messages per month. In the past year, the total has diminished, now hovering in the 7-8,000 per month range. Life is better now, but still, the filters are far from perfect, and many false positives and false negatives mean I must think about and deal with a spam on a daily, if not hourly basis. But I’m not the only one dealing with the thorny issue of too much junk.
Kevin Werbach writes in Death By Spam: The E-Mail You Know and Love is About to Vanish (Slate, Nov. 18, 2002): “Neither legislation nor litigation against spammers has stemmed the tide, and they’re not going to have much of an effect in the future, either. It’s time to give up: Despite the best efforts of legislators, lawyers, and computer programmers, spam has won. Spam is killing e-mail.”
Mr. Werbach sums up the essential source of the problem: “Because e-mail inboxes are open to anyone, longtime Internet users now receive hundreds of spams per day, making e-mail virtually unusable without countermeasures.”
Why the history of spam is the history of the Internet
Email has become universal because of its simplicity: anyone can use it to send you a note or a small file, and you can use it to contact virtually anyone. Usually. But why is it that way? Why doesn’t sending a message require you to identity yourself, or seek prior permission from the recipient?
Electronic mail was modeled after physical postal mail in that anyone can send anything to anyone else (assuming you have the recipient’s mailing address) and you pay the postage fee. But wait, what fee? E-mail is free to send, or seems free enough, and as long as the sender doesn’t pay anything per message, e-mail will remain distinctly different from physical mail. Imagine the tons of junk in you mailbox if direct marketers could contact you for free and as often as they like.
The Internet’s message exchange protocols were originally created in a closed system, where the hosts connected to the network were known and could be trusted. Users on those hosts could be clearly identified and also trusted to not misbehave. In the early history of spam, the anti-social, anti-network behavior of sending unsolicited messages to others on the network was taken very seriously, and offenders were dealt with fairly quickly and effectively. As the Internet opened up to commercial interests, as dial-up users started logging in, and as millions of people started exchanging messages, those safeguards of host and user identity and reputation fell by the wayside. Email’s very open nature is what may have doomed it to failure.
Paul Judge, writing in Redesigning the net to save it from spam (CNN, May 17, 2003), observes: “Simple Mail Transfer Protocol (SMTP) was developed some 20 years ago for a totally different type of internet, one that was very open and trusting. Today, the Internet is not those two things.”
As long as there is an economic incentive for spammers to shill their often-obfuscated message to people, we’ll always have spam. We are reminded of the tragedy of the commons. The idea, described in a parable in 1833 by William Forster Lloyd on population growth and popularized in 1968 by Garrett Harden, “demonstrates how unrestricted access to a resource such as a pasture ultimately dooms the resource because of over-exploitation. This occurs because the benefits of exploitation accrue to individuals, while the costs of exploitation are distributed between all those exploiting the resource.” Even the ancient Greeks had something to say to tragic nature of the email commons: “That which is common to the greatest number has the least care bestowed upon it.”
The whole game of spam is about a few bad actors exploiting the commons, and will continue to be until the nature of the commons has been changed.
Important milestones in the history of spam
How did it get this way, in this state of exploitation? Let’s follow along with the development of spam.
On September 13, 1904, the first telegram spam is sent, according to Internet sources, even Wikipedia, but no substantive evidence of this can be found. In other word, this very fact-or-not has been spammed throughout the Internet, with no regard to whether the fact is trustworthy or not.
On July 5, 1937, Hormel Foods Corporation changes the name of Hormel Spiced Ham to SPAM, possibly meaning “Shoulder of Pork and Ham”, later becoming backformed by others as “Something Posing as Meat” or “Special Purpose Army Meat”. In any case, the name ultimately becomes synonymous with something not especially appealing to consume.
On December 15, 1970, the infamous Spam Sketch premiered as the final sketch of the 25th show of Monty Python’s Flying Circus, introducing the world to a menu full of culinary delights such as “spam, egg, spam, spam, bacon and spam” and behorned Vikings loudly singing the praises of “Spam, lovely spam, wonderful spam.” And thus the meme of being overwhelmed with too much of a noxious thing in the wrong venue is born, but it will take another 23 years for the term to be generally applied to unsolicited commercial email messages sent in noxious quantities.
In November 1975, in the Internet standards process document Request for Comments (RFC) 706, On the Junk Mail Problem, internet pioneer Jon Postel notes, “In the ARPA Network, … there is no mechanism for the Host to selectively refuse messages. This means that a Host (that) desires to receive some particular messages must read all messages addressed to it. Such a Host may be sent many messages by a malfunctioning host. This would constitute a denial of service. Both the local users and the network communication could suffer. The services denied are the processor time consumed in examining the undesired messages and rejecting them, and the loss of network throughput or increased delay due to the unnecessary busyness of the network. It would be useful for a Host to be able to decline messages from sources it believes are misbehaving or are simply annoying.”
On May 3, 1978, Gary Thuerk, a marketer at DEC, using a printed directory of everyone on the Arpanet, sends a notice about an open house to show off new models of the DEC-20 computer. The message generated much discussion, as it supported neither research nor education and was thus against the acceptable use policy of the ARPAnet. A young Richard Stallman, champion of software freedom, chimes in on the debate and gains the dubious honor of being perhaps the first spam apologist.
Others vehemently oppose the unsolicited mailing. Mark Crispin observes, “I don’t see any place for advertising on the ARPAnet, however; certainly not the bulk advertising of that DEC message. From the address list, it seems clear to me that the people it was sent to were the Californians listed in the last ARPAnet directory. This was a clear and flagrant abuse of the directory!”
In September 1981, Jon Postel desired to fix this built-in weakness of email exchange via a network, but his desire for open protocols led him to publish his later Robustness Principle known as Postel’s Law, in RFC 793 in 1981: “TCP implementations will follow a general principle of robustness: be conservative in what you do, be liberal in what you accept from others.”
In August 1982, the Simple Mail Transfer Protocol was proposed in RFC 821 by Jon Postel as a replacement for a prior mail transport standard that relied on FTP to exchange data. The SMTP protocol solved many problems related to moving messages from server to server, but provided no facility to authenticate senders.
On March 31, 1993, Richard Depew’s software, called ARMM, was supposed to help automate the moderation process of USENET groups but went haywire, flooded a newsgroup, and many people were angered. In the resultant discussion, Joel Furr is credited with the first use of the term spam: “Transformed by programming ineptitude into a monster of Frankenstein proportions, it broke loose on the night of March 31, 1993 and proceeded to spam news.admin.policy with something on the order of 200 messages in which it attempted, and failed, to cancel its own messages. … This produced a flood of messages in which each header took up several screens and each message id got longer and longer and longer and each subject line started wrapping around five or six times. ARMM was accused of crashing at least one mail system and inspired widespread resentment among those who pay for each message they have downloaded.”
April 12, 1994, two lawyers from Phoenix hired a mercenary programmer and posted a message to thousands of USENET newsgroups advertising their fairly useless services in an upcoming US green card lottery, and thus sent the first deliberate mass posting to be commonly called “spam”, and the term has stuck in popular usage since, referring to the unsolicited and massive flood of the same message to multiple people or venues.
In 1995, the commercial Internet begins to open up and take off. Watch out, inboxes!
By 2001, the EU estimated that spam costs Internet users 10 billion euros per year worldwide.
In 2004, Microsoft chairman Bill Gates, in a speech to the World Economic Forum, boldly predicted that spam would be eliminated in two years. He was wrong. This same year many high profile spammers began to be convicted under new US and other nation’s anti-spam legislation.
In 2005, Russian spammer Vardan Kushnir, after having obsessively spammed nearly the entire population of the Russian-language Internet, was brutally murdered. Like so many aspects of the dark side of spam, whether his murder was vigilante justice has yet to be determined. Later that year, Britian’s most prolific spammer is sentenced to six years in prison.
By 2006 spam is said to account for 40% of all email, with anywhere from 12 to 55 billion spam messages sent daily. An IronPort study estimates that 80% of spam originates from remotely-controlled zombie computers, part of larger networks of sinister “bot-nets,” under the command of covert spam lords.
In 2007, the California legislature found that spam costs US organizations alone more than $13 billion in that year alone – including lost productivity, additional equipment, software, and manpower needed to combat the ever increasing problem.
Enter trust, reputation, and identity
Yes, there’s something that can be said about the casual nature of email. You can send messages (even semi-anonymously) to people who you would never call on the phone, talk to in person or send a letter to. But the price of that freedom is paid by everyone, billions of times a day. We would like to send and receive messages from people and organizations that we trust, from those who have built up a good reputation for treating the commons with respect, and from those whose identity can be verified. In practice, that doesn’t always work, because the noise-to-signal ratio in a world of ubiquitous spam and phishing attacks means you can’t always trust your messages to arrive, to get important messages yourself, or even for those messages to be from whom they claim they’re from.
Identity of networks, hosts, and users is important in the war on spam, because with identity, you can monitor and track reputation. Those entities with a reputation for spamming can be disconnected from the network, blacklisted and even prosecuted in the real world. Identity is required before trust relationships can be established between any two entities exchanging messages, but because anyone can join the network, there is no central trusted authority on identity. So while identity, trust, and reputation are important now, the current state of limited implementation means it cannot yet be used as the solution to the spam problem.
In the next article, we’ll discuss current efforts trying to control the flow of spam, with attempts to use the identity and reputation of users, hosts and networks as well as the content of the messages themselves to determine whether to deliver a message to your inbox. In the final installment, we will discuss the future of message exchange as email and trusted identity begin to converge.
“And, now for something completely different”
A final tribute to the Pythons’ enduring legacy, with names changed to protect the innocent.
Email User: Have you got anything without spam in it?
The Internet: Spam, egg, sausage, and spam. It’s not got much spam in it.
Email User: I don’t want ANY spam!
Tech Support: Why can’t she have egg, bacon, spam and sausage?
Email User: THAT’S got spam in it!
Tech Support: Hasn’t got as much spam in it as spam, egg, sausage and spam, has it?
Vikings: Spam, spam, spam, spam …