Creating a Go-Based Microservice for Digital Asset Management

Table of Contents

1. Introduction
2. Prerequisites
3. Setup
4. Creating the Microservice
5. Handling File Uploads
6. Managing Metadata
7. Implementing Searching

8. Conclusion

   ---

Introduction

In this tutorial, we will create a Go-based microservice for digital asset management. We will build a service that allows users to upload files, manage metadata associated with the files, and implement a search functionality. By the end of this tutorial, you will have a basic understanding of building a microservice with Go and handling file uploads, managing metadata, and implementing searching functionalities.

Prerequisites

To follow along with this tutorial, you should have a basic understanding of Go programming language and have Go installed on your machine. You should also have a working knowledge of command-line operations.

Setup

Before we begin, let’s set up our project structure and dependencies.

1. Create a new directory for our project:

   ```bash
   mkdir asset-management-microservice
   cd asset-management-microservice
   ```
   

2. Initialize a new Go module:

   ```bash
   go mod init github.com/your-username/asset-management-microservice
   ```
   

3. Install necessary dependencies:

   ```bash
   go get github.com/gin-gonic/gin
   go get github.com/jinzhu/gorm
   go get github.com/go-sql-driver/mysql
   ```
   
   Here, we are installing [Gin](https://github.com/gin-gonic/gin) for building the HTTP API, [GORM](https://github.com/jinzhu/gorm) as the ORM library, and [MySQL driver](https://github.com/go-sql-driver/mysql) for connecting to a MySQL database. Feel free to use alternative libraries or databases if you prefer.
   

   Now that our project is set up, let’s start building our microservice.

Creating the Microservice

We will begin by creating the main Go file and setting up the basic HTTP server.

1. Create a new file named main.go:

   ```bash
   touch main.go
   ```
   

2. Open main.go in your preferred text editor and add the following code:

   ```go
   package main
   
   import (
       "github.com/gin-gonic/gin"
       "github.com/jinzhu/gorm"
       _ "github.com/go-sql-driver/mysql"
   )
   
   func main() {
       // TODO: Implement the HTTP server
   }
   ```
   
   Here, we import the necessary packages and set up the main function. We will add the HTTP server implementation in the `TODO` section.
   

3. Let’s create a folder named handlers and a file named asset_handler.go inside it:

   ```bash
   mkdir handlers
   touch handlers/asset_handler.go
   ```
   

4. Open asset_handler.go and add the following code:

   ```go
   package handlers
   
   import "github.com/gin-gonic/gin"
   
   func UploadAsset(c *gin.Context) {
       // TODO: Implement the asset upload logic
   }
   
   func GetAsset(c *gin.Context) {
       // TODO: Implement fetching an asset
   }
   
   func SearchAssets(c *gin.Context) {
       // TODO: Implement asset searching
   }
   ```
   
   Here, we define three handler functions: `UploadAsset` for handling asset uploads, `GetAsset` for fetching an asset, and `SearchAssets` for implementing asset searching. We will fill in the logic for these functions later.
   

   Now that we have the basic structure in place, let’s move on to handling file uploads.

Handling File Uploads

To handle file uploads, we will use the multipart/form-data encoding and save the uploaded files locally.

1. Open asset_handler.go and update the UploadAsset function:

   ```go
   package handlers
   
   import (
       "github.com/gin-gonic/gin"
   )
   
   func UploadAsset(c *gin.Context) {
       file, err := c.FormFile("file")
       if err != nil {
           c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
           return
       }
   
       // TODO: Save the uploaded file locally
   
       c.JSON(http.StatusOK, gin.H{"message": "File uploaded successfully"})
   }
   ```
   

2. Let’s add the logic to save the uploaded file locally:

   ```go
   package handlers
   
   import (
       "github.com/gin-gonic/gin"
       "io"
       "os"
       "path/filepath"
   )
   
   func UploadAsset(c *gin.Context) {
       file, err := c.FormFile("file")
       if err != nil {
           c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
           return
       }
   
       // Create a path to save the uploaded file
       fileName := filepath.Base(file.Filename)
       filePath := filepath.Join("uploads", fileName)
   
       // Create the uploads directory if it doesn't exist
       os.MkdirAll("uploads", os.ModePerm)
   
       // Open the destination file for writing
       dst, err := os.Create(filePath)
       if err != nil {
           c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
           return
       }
       defer dst.Close()
   
       // Open the uploaded file
       src, err := file.Open()
       if err != nil {
           c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
           return
       }
       defer src.Close()
   
       // Copy the uploaded file to the destination file
       if _, err := io.Copy(dst, src); err != nil {
           c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
           return
       }
   
       c.JSON(http.StatusOK, gin.H{"message": "File uploaded successfully"})
   }
   ```
   
   Here, we retrieve the uploaded file from the request and create a path to save the file locally. We then create the necessary directories if they don't exist and open the destination file for writing. Finally, we copy the contents of the uploaded file to the destination file.
   

   Now that we can handle file uploads, let’s move on to managing metadata.

Managing Metadata

In this section, we will implement functionality to associate metadata with the uploaded files. We will use a MySQL database for storing and retrieving metadata.

1. Let’s create a folder named models and a file named asset.go inside it:

   ```bash
   mkdir models
   touch models/asset.go
   ```
   

2. Open asset.go and add the following code:

   ```go
   package models
   
   import (
       "github.com/jinzhu/gorm"
   )
   
   type Asset struct {
       gorm.Model
       FileName    string `gorm:"column:file_name"`
       Description string `gorm:"column:description"`
       Tags        string `gorm:"column:tags"`
   }
   ```
   
   Here, we define the `Asset` structure for storing metadata associated with the uploaded files. We use the `gorm.Model` embedded struct to automatically handle fields like `ID`, `CreatedAt`, `UpdatedAt`, and `DeletedAt`.
   

3. Let’s update the UploadAsset function in asset_handler.go to save the metadata in the database:

   ```go
   package handlers
   
   import (
       "github.com/gin-gonic/gin"
       "net/http"
   
       "your-username/asset-management-microservice/models"
   )
   
   func UploadAsset(c *gin.Context) {
       // ...
   
       // Save metadata in the database
       asset := models.Asset{
           FileName: fileName,
       }
   
       if err := db.Create(&asset).Error; err != nil {
           c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
           return
       }
   
       c.JSON(http.StatusOK, gin.H{"message": "File uploaded successfully"})
   }
   ```
   
   Here, we create a new instance of `Asset` with the filename and save it in the database using the `Create` method from `gorm`.
   

4. Don’t forget to import the necessary packages and initialize the database connection in main.go:

   ```go
   package main
   
   import (
       "github.com/gin-gonic/gin"
       "github.com/jinzhu/gorm"
       _ "github.com/go-sql-driver/mysql"
       "your-username/asset-management-microservice/handlers"
   )
   
   var db *gorm.DB
   
   func main() {
       // Initialize the database connection
       var err error
       db, err = gorm.Open("mysql", "user:password@/asset_management?charset=utf8&parseTime=True&loc=Local")
       if err != nil {
           panic("failed to connect to database")
       }
       defer db.Close()
   
       // Create necessary database tables
       db.AutoMigrate(&models.Asset{})
   
       // Set up the HTTP server
       router := gin.Default()
       router.POST("/upload", handlers.UploadAsset)
       router.Run(":8080")
   }
   ```
   
   Replace `user:password` with your MySQL username and password, and `asset_management` with the desired database name.
   

   With the metadata management implemented, let’s move on to implementing the search functionality.

Implementing Searching

To implement searching, we will expose an HTTP endpoint that accepts query parameters and returns the matching assets.

1. Let’s update the SearchAssets function in asset_handler.go to handle the search query:

   ```go
   package handlers
   
   import (
       "github.com/gin-gonic/gin"
       "net/http"
   
       "your-username/asset-management-microservice/models"
   )
   
   func SearchAssets(c *gin.Context) {
       query := c.Query("q")
   
       var assets []models.Asset
       if err := db.Where("file_name LIKE ?", "%"+query+"%").Find(&assets).Error; err != nil {
           c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
           return
       }
   
       c.JSON(http.StatusOK, assets)
   }
   ```
   
   Here, we retrieve the search query from the query parameters and use the `Where` method with the `LIKE` operator to search for matching assets in the database.
   

2. Update main.go to add the route for searching:

   ```go
   package main
   
   import (
       "github.com/gin-gonic/gin"
       "github.com/jinzhu/gorm"
       _ "github.com/go-sql-driver/mysql"
       "your-username/asset-management-microservice/handlers"
   )
   
   // ...
   
   func main() {
       // ...
   
       router.GET("/search", handlers.SearchAssets)
   
       router.Run(":8080")
   }
   ```
   
   Here, we add the `/search` route and map it to the `SearchAssets` handler function.
   

   Now that we have implemented the basic functionality for our microservice, let’s recap what we have learned.

Conclusion

In this tutorial, we created a Go-based microservice for digital asset management. We started by setting up the project structure and installing the necessary dependencies. Then, we implemented the functionalities for file uploads, managing metadata, and searching assets.

We learned how to handle file uploads using the multipart/form-data encoding and save the uploaded files locally. We also managed to store the metadata associated with the uploaded files in a MySQL database using the gorm package. Finally, we implemented searching by querying the database based on the provided search query.

Throughout the tutorial, we covered various concepts such as HTTP server setup, interacting with databases using an ORM, working with file uploads, and implementing search functionality. With this knowledge, you can extend the microservice further and add additional features or improvements based on your specific requirements.

Remember to explore the official documentation of the libraries and packages used for more in-depth understanding and further customization of your microservice. Happy coding!

Published: 17 November 2020

Related Articles

• Implementing a Multi-Threaded Web Spider in Go
• Building a CLI for Docker Container Management in Go
• Writing a Kubernetes Custom Scheduler in Go
• Creating a Multi-Page Web Application in Go
• Handling I/O Errors in Go
• Creating a Go Application for Automated Kubernetes Rollbacks
• Writing a Go-Based CLI Tool for IP Address Lookup
• Building a Rate-Limited HTTP Server in Go
• Working with File Metadata in Go
• How to Use the os.FileInfo Interface in Go
• Reading and Writing to a Text File in Go
• Writing a CLI for Log Parsing in Go
• Developing a Real-Time Recommendation Engine with Go
• Understanding Standard Input and Standard Output in Go
• Developing a Command-Line File Sharing Tool in Go
• Building a Real-Time Trading App with Go and WebSockets
• Go Project Structure: A Guide to Scalable Applications
• Building a CLI for File System Operations in Go
• Interacting with the OS in Go: A Comprehensive Guide
• Creating a Go-Based Data Pipeline for Real Estate Data Analysis