Golang : Get SPF and DMARC from email headers to fight spam
Problem:
You are building an email server with spam filter from scratch with Golang and you want to query a domain's SPF(Sender Policy Framework) and DMARC(Domain-based Message Authentication, Reporting and Conformance) record. How to get a domain's SPF and DMARC records?
NOTES:
An SPF record is to prevent spammers from sending messages with forged From
addresses at your domain. Recipients or spam filter program can refer to the SPF record to determine whether a message purporting to be from your domain comes from an authorized mail server.
DMARC is an email validation system designed to detect and prevent email spoofing. It is built on top of Sender Policy Framework (SPF) and DomainKeys Identified Mail (DKIM)
Solution:
A proper email has header information that you can use to extract the SPF and DMARC records of the domain. In the email headers, look for the From:
and extract the domain name. For example, for spammer@hello.com
, we only want hello.com
.
Query the domain name with the code example below.
An SPF record is a type of DNS record that shows which mail servers are allowed to send email on behalf of a domain. For example, customers of sengrid.com usually have:
include:sendgrid.net
in their SPF records. It simply means that sendgrid.net is authorized to send email on behalf of the customer. For example, email from testingmom.com will have sendgrid.net
in the From:
part in the headers.
Not all domains have DMARC records in their DNS TXT. However, it is good to query for the domain's DMARC as well. For example, Foursquare.com is a customer of sendgrid.com and it has DMARC configured in their DNS.
v=DMARC1; p=none; rua=mailto:dmarc@foursquare.com
The code that follows will show you how to query a domain for their DNS records with net.Lookup()
function, extract the SPF and DMARC records and convert the DMARC record into a map(hash table) for fast lookup.
With the SPF and DMARC information, you can configure the spam filter to allow an email through, reject it or mark it as spam and route to a spam box.
Here you go!
emailheader.go
package main
import (
"fmt"
"net"
"os"
"strings"
)
func ExtractDMARC(records []string) string {
for _, value := range records {
if strings.Contains(value, "v=DMARC1") {
return value
}
}
// default
return "no DMARC found"
}
func ExtractSPF(records []string) string {
for _, value := range records {
if strings.Contains(value, "v=spf1") {
return value
}
}
// default
return "no SPF found"
}
func main() {
if len(os.Args) != 2 {
fmt.Printf("Usage : %s <domain to query> \n", os.Args[0])
os.Exit(0)
}
domain := os.Args[1]
fmt.Printf("Getting %s DNS records...\n", domain)
// extract DNS records for the target domain
dnsTXT, err := net.LookupTXT(domain)
if err != nil {
fmt.Println("unable to query domain. Error : " + err.Error())
os.Exit(-1)
}
// sanity check
for index, record := range dnsTXT {
fmt.Printf("[%d] : %s\n", index, record)
}
spf := ExtractSPF(dnsTXT)
fmt.Printf("\n\nSPF record : %s\n", spf)
// extract DMARC records by adding "_dmarc" in the LookupTXT query
// DMARC records are published in DNS with a subdomain label _dmarc
dmarcTXT, err := net.LookupTXT("_dmarc." + domain)
if err != nil {
fmt.Println("unable to query DMARC. Error : " + err.Error())
os.Exit(-1)
}
dmarc := ExtractDMARC(dmarcTXT)
fmt.Printf("\n\nDMARC record : %s\n", dmarc)
// break down the DMARC name value tags to slice first and append to a map(hash table)
dmarcSlice := strings.Fields(dmarc)
//fmt.Println(dmarcSlice)
// from https://en.wikipedia.org/wiki/DMARC
// v is the version, p is the policy, sp the subdomain policy,
// pct is the percent of "bad" emails on which to apply the policy,
// and rua is the URI to send aggregate reports to.
dmarcMap := make(map[string]interface{})
for _, value := range dmarcSlice {
// break value by = sign
tmp := strings.Split(value, "=")
dmarcMap[tmp[0]] = tmp[1]
}
// it is up to you on what to do with these data
// store them into database
// use the information to configure spam filter
fmt.Println("DMARC version : ", dmarcMap["v"])
fmt.Println("DMARC policy : ", dmarcMap["p"])
fmt.Println("DMARC subdomain policy : ", dmarcMap["sp"])
fmt.Println("DMARC bad percentage : ", dmarcMap["pct"])
fmt.Println("DMARC send reports to : ", dmarcMap["rua"])
}
Build and run the code.
Sample output for ./emailheader sendgrid.com
:
Getting sendgrid.com DNS records...
[0] : loaderio=dc99536e3eb902ffc82885541e224e7c
[1] : MS=ms77173225
[2] : v=spf1 ip4:167.89.32.5 ip4:167.89.32.50 ip4:50.31.36.199 ip4:50.31.36.205 ip4:50.31.36.208 ip4:50.31.36.213 ip4:50.31.36.197 ip4:167.89.25.84 include:_spf.google.com include:partners.sendgrid.com include:_labs.sendgrid.com -all
[3] : google-site-verification=ou-V2e37_e-apeuJQ4IZh8VPHGkwMHCeehxZiVJUQ_s
SPF record : v=spf1 ip4:167.89.32.5 ip4:167.89.32.50 ip4:50.31.36.199 ip4:50.31.36.205 ip4:50.31.36.208 ip4:50.31.36.213 ip4:50.31.36.197 ip4:167.89.25.84 include:_spf.google.com include:partners.sendgrid.com include:_labs.sendgrid.com -all
DMARC record : v=DMARC1; p=quarantine; sp=none; adkim=r; aspf=r; fo=1; pct=100; rf=afrf; ri=86400; rua=mailto:sendgrid@rua.agari.com,mailto:dmarc@sendgrid.com; ruf=mailto:sendgrid@ruf.agari.com,mailto:dmarc@sendgrid.com
[v=DMARC1; p=quarantine; sp=none; adkim=r; aspf=r; fo=1; pct=100; rf=afrf; ri=86400; rua=mailto:sendgrid@rua.agari.com,mailto:dmarc@sendgrid.com; ruf=mailto:sendgrid@ruf.agari.com,mailto:dmarc@sendgrid.com]
DMARC version : DMARC1;
DMARC policy : quarantine;
DMARC subdomain policy : none;
DMARC bad percentage : 100;
DMARC send reports to : mailto:sendgrid@rua.agari.com,mailto:dmarc@sendgrid.com;
and sample output for ./emailheader digitalocean.com
:
Getting digitalocean.com DNS records...
[0] : v=spf1 ip4:167.89.16.30/32 ip4:167.89.16.245/32 ip4:167.89.16.183/32 include:servers.mcsv.net include:mailgun.org include:_spf.google.com -all
[1] : mailru-verification: 93fb13cddf97475f
[2] : google-site-verification=l_eZFhD-V_qPB1OYMoA4-Qg1QJT0oQm6j1Z1Dl_f65k
[3] : globalsign-domain-verification=QCdmvx2FmvCpeqYXQsTAMJMq2YjyK0VzLosSdN6aMP
SPF record : v=spf1 ip4:167.89.16.30/32 ip4:167.89.16.245/32 ip4:167.89.16.183/32 include:servers.mcsv.net include:mailgun.org include:_spf.google.com -all
DMARC record : v=DMARC1; p=reject; pct=100; rua=mailto:fdpfb1lo@ag.dmarcian.com; ruf=mailto:fdpfb1lo@fr.dmarcian.com
[v=DMARC1; p=reject; pct=100; rua=mailto:fdpfb1lo@ag.dmarcian.com; ruf=mailto:fdpfb1lo@fr.dmarcian.com]
DMARC version : DMARC1;
DMARC policy : reject;
DMARC subdomain policy : <nil>
DMARC bad percentage : 100;
DMARC send reports to : mailto:fdpfb1lo@ag.dmarcian.com;
Hope this helps and happy coding!
References:
https://www.socketloop.com/references/golang-net-lookuptxt-function-example
https://en.wikipedia.org/wiki/DMARC
https://sendgrid.com/resources/cases/
https://support.google.com/a/answer/33786?hl=en
https://www.codegists.com/snippet/go/emailheadergoamlwwalkergo
See also : Golang : Auto-generate reply email with text/template package
By Adam Ng
IF you gain some knowledge or the information here solved your programming problem. Please consider donating to the less fortunate or some charities that you like. Apart from donation, planting trees, volunteering or reducing your carbon footprint will be great too.
Advertisement
Tutorials
+20.3k Golang : Secure(TLS) connection between server and client
+5.2k Golang : Intercept, inject and replay HTTP traffics from web server
+7.9k Golang : Tell color name with OpenCV example
+5.7k Golang : Function as an argument type example
+46k Golang : Marshal and unmarshal json.RawMessage struct example
+45.7k Golang : Read tab delimited file with encoding/csv package
+7.5k Golang : Lock executable to a specific machine with unique hash of the machine
+6.8k Javascript : How to get JSON data from another website with JQuery or Ajax ?
+13.3k Golang : Get user input until a command or receive a word to stop
+14.2k Golang : On enumeration
+8.2k Golang : How to check variable or object type during runtime?
+10k Golang : cannot assign type int to value (type uint8) in range error