Developing a Go-based AWS S3 CLI

Table of Contents

  1. Introduction
  2. Prerequisites
  3. Setting up AWS Credentials
  4. Creating a Go Project
  5. Installing Required Packages
  6. Building the AWS S3 CLI
  7. Using the AWS S3 CLI
  8. Recap

Introduction

In this tutorial, we will develop a Go-based command-line interface (CLI) tool that interacts with Amazon Web Services (AWS) Simple Storage Service (S3). The tool will allow users to perform various operations on S3 buckets, such as creating buckets, uploading files, downloading files, and listing contents. By the end of this tutorial, you will have a functional CLI tool that can be used to manage AWS S3 resources.

Prerequisites

Before starting this tutorial, you should have the following:

  • Go installed and properly configured on your machine
  • An AWS account with S3 access
  • Basic knowledge of Go programming language

Setting up AWS Credentials

To access your AWS resources, you need to set up your AWS credentials. Here’s how:

  1. Visit the AWS Management Console and sign in to your account.

  2. Open the IAM service.

  3. Select Users from the sidebar and choose the appropriate user or create a new one.

  4. Under the Security credentials tab, click on Create access key.

  5. Download your access key file, which contains your Access Key ID and Secret Access Key.

  6. Configure the AWS CLI by running the following command in your terminal:

     aws configure

  7. Enter your Access Key ID, Secret Access Key, default region name, and default output format when prompted.

Creating a Go Project

Let’s start by creating a new Go project for our AWS S3 CLI. Follow these steps:

  1. Create a new directory for your project:

     mkdir aws-s3-cli
 cd aws-s3-cli

  2. Initialize the Go module:

     go mod init github.com/your-username/aws-s3-cli

  3. Create a new main Go file main.go in the project directory:

     package main
    
 import (
 	"fmt"
 	"os"
 )
    
 func main() {
 	fmt.Println("Welcome to the AWS S3 CLI!")
 	os.Exit(0)
 }

Installing Required Packages

Our CLI tool will rely on the AWS SDK for Go to interact with AWS services. Install it using the following command:

go get github.com/aws/aws-sdk-go

Building the AWS S3 CLI

In this section, we will implement various commands for working with AWS S3 using the AWS SDK for Go. Let’s begin by listing all Buckets:

package main

import (
	"fmt"
	"os"

	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/s3"
)

func main() {
	fmt.Println("Welcome to the AWS S3 CLI!")

	sess := session.Must(session.NewSessionWithOptions(session.Options{
		SharedConfigState: session.SharedConfigEnable,
	}))

	svc := s3.New(sess)

	result, err := svc.ListBuckets(nil)
	if err != nil {
		fmt.Println("Error listing buckets:", err)
		os.Exit(1)
	}

	fmt.Println("Buckets:")
	for _, bucket := range result.Buckets {
		fmt.Println(*bucket.Name)
	}

	os.Exit(0)
}

To build and run the CLI, use the following command:

go build
./aws-s3-cli

Using the AWS S3 CLI

Now that our CLI tool can list buckets, let’s implement additional commands to perform various operations.

Creating a Bucket

To create a new S3 bucket, we will use the CreateBucket method:

// ...

func main() {
	// ...
	
	// Create a new bucket
	bucketName := "my-bucket"
	_, err = svc.CreateBucket(&s3.CreateBucketInput{
		Bucket: &bucketName,
	})
	if err != nil {
		fmt.Println("Error creating bucket:", err)
		os.Exit(1)
	}

	fmt.Println("Bucket created successfully!")
}

Uploading a File

To upload a file to an S3 bucket, we can use the PutObject method:

// ...

func main() {
	// ...
	
	// Upload a file to the bucket
	file, err := os.Open("path/to/file.txt")
	if err != nil {
		fmt.Println("Error opening file:", err)
		os.Exit(1)
	}
	defer file.Close()

	_, err = svc.PutObject(&s3.PutObjectInput{
		Bucket: bucketName,
		Key:    aws.String("file.txt"),
		Body:   file,
	})
	if err != nil {
		fmt.Println("Error uploading file:", err)
		os.Exit(1)
	}

	fmt.Println("File uploaded successfully!")
}

Downloading a File

To download a file from an S3 bucket, we can use the GetObject method:

// ...

func main() {
	// ...
	
	// Download a file from the bucket
	file, err := os.Create("downloaded-file.txt")
	if err != nil {
		fmt.Println("Error creating file:", err)
		os.Exit(1)
	}
	defer file.Close()

	result, err := svc.GetObject(&s3.GetObjectInput{
		Bucket: bucketName,
		Key:    aws.String("file.txt"),
	})
	if err != nil {
		fmt.Println("Error downloading file:", err)
		os.Exit(1)
	}

	defer result.Body.Close()

	_, err = io.Copy(file, result.Body)
	if err != nil {
		fmt.Println("Error saving file:", err)
		os.Exit(1)
	}

	fmt.Println("File downloaded successfully!")
}

Listing Bucket Contents

To list the contents of an S3 bucket, we can use the ListObjectsV2 method:

// ...

func main() {
	// ...
	
	// List objects in the bucket
	result, err := svc.ListObjectsV2(&s3.ListObjectsV2Input{
		Bucket: bucketName,
	})
	if err != nil {
		fmt.Println("Error listing objects:", err)
		os.Exit(1)
	}

	fmt.Println("Bucket contents:")
	for _, item := range result.Contents {
		fmt.Println(*item.Key)
	}
}

Recap

In this tutorial, we developed a Go-based AWS S3 CLI tool. We covered the steps to set up AWS credentials, create a Go project, install required packages, and build the CLI. We also implemented commands to create buckets, upload files, download files, and list bucket contents. You should now have a better understanding of how to interact with AWS S3 using Go.

Remember to handle errors appropriately, implement additional features as needed, and consider best practices for secure and efficient application development.

Congratulations on building your AWS S3 CLI in Go!

Published:

Related Articles