Golang: Prevent over writing file with md5 hash
This is an additional note for previous tutorial on how to upload/submit file from command line to a server. In the receiving server program, we overwrite the file upon receiving. There are times when we need to check first if the file that we receive from the client is the same file. If the file is same, then we do not need to save the file on server.
Now, do not confuse checking if a file exist is similar to checking if a new file is the same as the old file. Exist != Same.
To check if the new file that our server program just received is the same as the old file, we need to compute the MD5(can be sha1, sha512 or sha256 too) hash and check if any file in the directory has the same MD5 hash. If no, then proceed to overwrite. If yes, ignore the uploaded file or you can save the new file but with incremental filenames. See https://www.socketloop.com/tutorials/golang-increment-string-example
The code example that follows is a modification of a file receiving program from the command line upload file tutorial.
checkSameFile.go
package main
import (
"crypto/md5"
"fmt"
"io"
"io/ioutil"
"net/http"
"os"
)
func receiveHandler(w http.ResponseWriter, r *http.Request) {
// the FormFile function takes in the POST input id file
file, header, err := r.FormFile("fileUploadName")
if err != nil {
fmt.Fprintln(w, err)
return
}
// sanity check on the MIME FileHeader
fmt.Println(header.Header)
defer file.Close()
out, err := os.Create("/tmp/uploadedFile")
if err != nil {
fmt.Fprintf(w, "Unable to create the file for writing. Check your write access privilege")
return
}
defer out.Close()
// write the content from POST to the file
_, err = io.Copy(out, file)
if err != nil {
fmt.Fprintln(w, err)
}
// now we want to check if the new uploaded file is a duplicate of our target file
// at /tmp/keepThisFile. If yes, abort writing to target file
// modified from https://www.socketloop.com/tutorials/golang-find-duplicate-files-with-filepath-walk
oldFile, err := ioutil.ReadFile("/tmp/keepThisfile")
// target file not found
if err != nil {
fmt.Fprintln(w, "Old file not found, proceed to rename /tmp/uploadedFile to /tmp/keepThisFile. \n")
// rename /tmp/uploadedFile to /tmp/keepThisFile
os.Rename("/tmp/uploadedFile", "/tmp/keepThisFile")
fmt.Fprintln(w, "Renamed /tmp/uploadedFile to /tmp/keepThisFile. You can proceed to digest the file for further analysis. \n")
return
} else {
// target file found, get the MD5 checksum of target / old file
oldHexDigest := md5.Sum(oldFile)
fmt.Fprintf(w, "Old file MD5 checksum is %x \n", oldHexDigest)
newFile, err := ioutil.ReadFile("/tmp/uploadedFile")
if err != nil {
fmt.Fprintln(w, err)
}
newHexDigest := md5.Sum(newFile)
fmt.Fprintf(w, "New file MD5 checksum is %x \n", newHexDigest)
if newHexDigest != oldHexDigest {
// rename /tmp/uploadedFile to /tmp/keepThisFile
// don't worry - /tmp/uploadedFile will be deleted in the process
os.Rename("/tmp/uploadedFile", "/tmp/keepThisFile")
fmt.Fprintln(w, "Renamed /tmp/uploadedFile to /tmp/keepThisFile. You can proceed to digest the file for further analysis. \n")
fmt.Fprintf(w, "File [%s] uploaded.", header.Filename)
return
} else {
fmt.Fprintf(w, "%s is a duplicate of previously uploaded file.", header.Filename)
return
}
}
}
func page(w http.ResponseWriter, r *http.Request) {
html := ` <html>
<title>Upload your submission here.</title>
<body>
<h1>This the web way of submitting/uploading file</h1>
<br>
<form action="http://localhost:8888/submit" method="post" enctype="multipart/form-data">
<label for="file">Filename:</label>
<input type="file" name="fileUploadName" id="fileUploadName">
<input type="submit" name="submit" value="Submit">
</form>
</body>
</html>`
w.Write([]byte(fmt.Sprintf(html)))
}
func main() {
http.HandleFunc("/submit", receiveHandler)
http.HandleFunc("/", page)
http.ListenAndServe(":8888", nil)
}
Build checkSameFile.go
and run on the background. Point your browser to http://localhost:8888/
and upload a file to see the output message for the first time.
Next, try uploading the same file, but with a different name. If all goes well, the server detects that it is a duplicate file.
Happy coding!
References:
https://www.socketloop.com/tutorials/golang-find-duplicate-files-with-filepath-walk
https://www.socketloop.com/tutorials/golang-increment-string-example
https://www.socketloop.com/tutorials/golang-generate-md5-checksum-of-a-file
https://socketloop.com/tutorials/golang-check-if-a-file-exist-or-not
See also : Golang : Command line file upload program to server example
By Adam Ng
IF you gain some knowledge or the information here solved your programming problem. Please consider donating to the less fortunate or some charities that you like. Apart from donation, planting trees, volunteering or reducing your carbon footprint will be great too.
Advertisement
Tutorials
+4.5k Golang : How to solve "too many .rsrc sections" error?
+6.8k Useful methods to access blocked websites
+9.8k Golang : Convert decimal number(integer) to IPv4 address
+10k Golang : Perform sanity checks on filename example
+10.1k Golang : md5 hash of a string
+5.2k CloudFlare : Another way to get visitor's real IP address
+6.8k Golang : How to capture return values from goroutines?
+14.8k Golang : Capture stdout of a child process and act according to the result
+6.8k Golang : HTTP Server Example
+27.2k Golang : Get and Set User-Agent examples
+6.5k Findstr command the Grep equivalent for Windows
+6k Golang : How to convert strange string to JSON with json.MarshalIndent