Golang : Get Alexa ranking data example
Problem :
You want to get a website Alexa ranking and process the data in Golang.
Maybe for tracking a competitor ranking or tracking a publisher ranking where you put your advertising money. How to do that with a software instead of visiting Alexa.com all the time?
Solution : For this tutorial example, we will use the old Alexa API version(before they got acquired by Amazon). The API can be accessed at :
http://data.alexa.com/data?cli=10&url=domainname
For instance, a query of socketloop.com popularity with
http://data.alexa.com/data?cli=10&url=socketloop.com
will yield the following result :
<ALEXA VER="0.9" URL="socketloop.com/" HOME="0" AID="=" IDN="socketloop.com/">
<SD>
<POPULARITY URL="socketloop.com/" TEXT="291466" SOURCE="panel"/>
<REACH RANK="237320"/>
<RANK DELTA="+31735"/>
<COUNTRY CODE="US" NAME="United States" RANK="191742"/>
</SD>
</ALEXA>
NOTE : If you wish to extract more data from Alexa.com, you will have to use the latest API. See http://docs.aws.amazon.com/AlexaWebInfoService/latest/
Here you go! The Golang code to extract popularity ranking :
package main
import (
"encoding/xml"
"fmt"
"io/ioutil"
"net/http"
"os"
"strings"
)
func main() {
domainName := "socketloop.com"
resp, err := http.Get("http://data.alexa.com/data?cli=10&url=" + domainName)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
defer resp.Body.Close()
alexaData, err := ioutil.ReadAll(resp.Body)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
// convert []byte to type io.Reader with strings.NewReader()
decoder := xml.NewDecoder(strings.NewReader(string(alexaData)))
// loop till we get REACH token
for {
token, _ := decoder.Token()
if token == nil {
break
}
switch startElement := token.(type) {
case xml.StartElement:
if startElement.Name.Local == "REACH" {
//fmt.Println(startElement.Name)
//fmt.Println(startElement.Attr[0])
fmt.Printf("%s has popularity rank of %s \n", domainName, startElement.Attr[0].Value)
}
}
}
}
Output :
socketloop.com has popularity rank of 237750
NOTE : I've tried to unmarshal the XML but was unsuccessful. Could it be UPPERCASE or too much attributes ? I don't know... seems like a bug to me or is there something wrong with my code? Let me know if you can help to fix the code at http://play.golang.org/p/uOUjX5immw
References :
http://stackoverflow.com/questions/3676376/fetching-alexa-data
https://www.socketloop.com/tutorials/golang-http-get-example
By Adam Ng
IF you gain some knowledge or the information here solved your programming problem. Please consider donating to the less fortunate or some charities that you like. Apart from donation, planting trees, volunteering or reducing your carbon footprint will be great too.
Advertisement
Tutorials
+12.2k Golang : Sort and reverse sort a slice of bytes
+13.8k Golang : Convert IP version 6 address to integer or decimal number
+10.4k Golang : Get UDP client IP address and differentiate clients by port number
+4.9k Golang : How to deal with configuration data?
+9.2k Golang : How to generate Code 39 barcode?
+15.5k Golang : Generate universally unique identifier(UUID) example
+6.7k CloudFlare : Another way to get visitor's real IP address
+13.7k Golang : Chunk split or divide a string into smaller chunk example
+12.6k Golang : Handle or parse date string with Z suffix(RFC3339) example
+30.8k Golang : Get local IP and MAC address
+11.6k Golang : flag provided but not defined error
+7k Golang : Check to see if *File is a file or directory