Golang : Command line file upload program to server example




Writing this tutorial for a friend who is using Golang to develop her IoT application. She needs a client program that will submit or upload a file to a data collection server. In my previous tutorial on how to upload file, the solution is for web browser, but she needs a command line program to submit/upload the file.

We will learn how to use Golang's mime/multipart package to create a web browser's "virtual form" in the command line application and upload the file.

First, we will create a server side application that prepares to receive the file.

receiveSubmission.go


 package main

 import (
 "fmt"
 "io"
 "net/http"
 "os"
 )

 func receiveHandler(w http.ResponseWriter, r *http.Request) {

 // the FormFile function takes in the POST input id file
 file, header, err := r.FormFile("fileUploadName")

 if err != nil {
 fmt.Fprintln(w, err)
 return
 }

 defer file.Close()

 out, err := os.Create("/tmp/uploadedfile")
 if err != nil {
 fmt.Fprintf(w, "Unable to create the file for writing. Check your write access privilege")
 return
 }

 defer out.Close()

 // write the content from POST to the file
 _, err = io.Copy(out, file)
 if err != nil {
 fmt.Fprintln(w, err)
 }

 fmt.Fprintf(w, "File uploaded successfully : ")
 fmt.Fprintf(w, header.Filename)
 }

 func page(w http.ResponseWriter, r *http.Request) {
 html := `  <html>
 <title>Upload your submission here.</title>
 <body>

 <h1>This the web way of submitting zip file</h1>
 <br>
 <form action="http://localhost:8888/submit" method="post" enctype="multipart/form-data">
 <label for="file">Filename:</label>
 <input type="file" name="fileUploadName" id="fileUploadName">
 <input type="submit" name="submit" value="Submit">
 </form>

 </body>
 </html>`

 w.Write([]byte(fmt.Sprintf(html)))
 }

 func main() {
 http.HandleFunc("/submit", receiveHandler)
 http.HandleFunc("/", page)
 http.ListenAndServe(":8888", nil)
 }

compile and run this application on the background or a terminal.

>./receiveSubmission &

!! See also on how to daemonize your program. https://www.socketloop.com/tutorials/golang-daemonizing-a-simple-web-server-process-example !!

Next, we will create the command line program that will upload the file.

submitfile.go


 package main

 import (
 "bytes"
 "fmt"
 "io"
 "io/ioutil"
 "mime/multipart"
 "net/http"
 "os"
 "time"
 )

 func main() {
 if len(os.Args) != 3 {
 fmt.Printf("Usage : %s <URL to upload file> <filename> \n", os.Args[0])
 os.Exit(0)
 }

 uploadURL := os.Args[1]
 fileToUpload := os.Args[2]

 //sanity check
 fmt.Println(uploadURL)
 fmt.Println(fileToUpload)

 file, err := os.Open(fileToUpload)
 if err != nil {
 fmt.Println("File open error : ", err)
 os.Exit(-1)
 }

 defer file.Close()

 // since we are not going to upload our file with a Web browser or curl -F
 // we need to prepare a "virtual form" -- similar to what you can visually see
 // on localhost:8888 generated by receiveSubmission.go

 fileInfo, _ := file.Stat()

 var fileBody bytes.Buffer
 writer := multipart.NewWriter(&fileBody)

 // https://golang.org/pkg/mime/multipart/#Writer.CreateFormFile
 // must match what is expected by the receiving program
 // so, set field to "fileUploadName" for easier life...

 filePart, err := writer.CreateFormFile("fileUploadName", fileInfo.Name())
 if err != nil {
 fmt.Println("CreateFormFile error : ", err)
 os.Exit(-1)
 }

 // remember we are using mime - multipart
 _, err = io.Copy(filePart, file)

 if err != nil {
 fmt.Println("io.Copy error : ", err)
 os.Exit(-1)
 }

 // populate our header with simple data
 _ = writer.WriteField("title", "Sample data collected on 13th Oct 2016")

 // remember to close writer
 err = writer.Close()
 if err != nil {
 fmt.Println("Writer close error : ", err)
 os.Exit(-1)
 }

 // ok, our "virtual form" is ready, time to submit our fileBody to
 // http://localhost:8888/submit with POST

 request, err := http.NewRequest("POST", uploadURL, &fileBody)
 if err != nil {
 fmt.Println("POST ERROR : ", err)
 os.Exit(-1)
 }

 // set the header with the proper content type for the fileBody's boundary
 // see https://golang.org/pkg/mime/multipart/#Writer.FormDataContentType

 request.Header.Set("Content-Type", writer.FormDataContentType())

 // upload/post/submit the file
 // with 10 seconds timeout
 client := &http.Client{Timeout: time.Second * 10}

 response, err := client.Do(request)
 if err != nil {
 fmt.Println("Client POST error : ", err)
 os.Exit(-1)
 }

 defer response.Body.Close()

 // Read response body
 body, err := ioutil.ReadAll(response.Body)
 if err != nil {
 fmt.Println("Error reading body of response.", err)
 os.Exit(-1)
 }

 fmt.Println("Output : ", string(body))

 }

Finally, test out the program. You will need to supply your own file.

>./submitfile http://localhost:8888/submit /Users/sweetlogic/img.gif

If all goes well, you should see these messages:

http://localhost:8888/submit

/Users/sweetlogic/img.gif

Output : File uploaded successfully : img.gif


NOTES:

This example here is also similar to submitting malware for further analysis after capturing one with honeypot system.

References:

https://golang.org/pkg/mime/multipart/#Writer.CreateFormFile

https://www.socketloop.com/references/golang-mime-multipart-createformfile-createformfield-and-formdatacontenttype-functions-example





By Adam Ng

IF you gain some knowledge or the information here solved your programming problem. Please consider donating to the less fortunate or some charities that you like. Apart from donation, planting trees, volunteering or reducing your carbon footprint will be great too.


Advertisement