Creating a Go-Based Data Pipeline for Cryptocurrency Price Tracking

Table of Contents

  1. Introduction
  2. Prerequisites
  3. Setup
  4. Creating the Data Pipeline
  5. Conclusion

Introduction

In this tutorial, we will create a Go-based data pipeline for tracking cryptocurrency prices. We will utilize Go’s networking and web programming capabilities to fetch data from an external API and store it in a data structure for further analysis and processing.

By the end of this tutorial, you will have a functional Go program that can retrieve cryptocurrency price data and perform basic operations on it.

Prerequisites

To follow along with this tutorial, you should have a basic understanding of the Go programming language. You should also have Go installed on your system.

Setup

Before we begin, we need to install a few Go packages that we will be using. Open your terminal and execute the following commands:

go get -u github.com/go-resty/resty/v2
go get -u github.com/shopspring/decimal

The first package, github.com/go-resty/resty/v2, is a simple HTTP and REST client library that we will use to make HTTP requests. The second package, github.com/shopspring/decimal, provides a decimal implementation for Go, which will be useful for handling cryptocurrency prices.

Creating the Data Pipeline

Step 1: Initializing the Go module

First, let’s create a new directory for our project and initialize it as a Go module. Open your terminal and execute the following commands:

mkdir crypto-pipeline
cd crypto-pipeline
go mod init

Step 2: Importing required packages

Next, let’s create a new Go file named main.go and import the necessary packages:

package main

import (
	"fmt"
	"github.com/go-resty/resty/v2"
	"github.com/shopspring/decimal"
)

Here, we import the fmt package for printing output, the resty package for making HTTP requests, and the decimal package for working with decimal numbers.

Step 3: Defining a data structure

Now, let’s define a data structure to hold cryptocurrency price data. In this example, we’ll use a struct with two fields: Symbol (the cryptocurrency symbol) and Price (the current price).

type CryptoPrice struct {
	Symbol string
	Price  decimal.Decimal
}

Step 4: Fetching cryptocurrency prices

Next, let’s implement a function to fetch cryptocurrency prices from an external API. We’ll use the CoinGecko API for this example. Add the following function to your main.go file:

func fetchCryptoPrices() ([]CryptoPrice, error) {
	client := resty.New()

	resp, err := client.R().
		SetHeader("Accept", "application/json").
		Get("https://api.coingecko.com/api/v3/simple/price?ids=bitcoin,ethereum,ripple&vs_currencies=usd")

	if err != nil {
		return nil, err
	}

	prices := make(map[string]map[string]decimal.Decimal)
	err = resp.Body().Unmarshal(&prices)

	if err != nil {
		return nil, err
	}

	var cryptoPrices []CryptoPrice
	for symbol, priceData := range prices {
		price := priceData["usd"]
		cryptoPrices = append(cryptoPrices, CryptoPrice{Symbol: symbol, Price: price})
	}

	return cryptoPrices, nil
}

Here, we create a new Resty client and make an HTTP GET request to the CoinGecko API. We then unmarshal the response body into a map of symbol-price pairs and convert it into an array of CryptoPrice objects.

Step 5: Printing cryptocurrency prices

Now that we can fetch cryptocurrency prices, let’s implement a function to print the prices. Add the following function to your main.go file:

func printCryptoPrices(prices []CryptoPrice) {
	for _, price := range prices {
		fmt.Printf("Symbol: %s, Price: %s\n", price.Symbol, price.Price)
	}
}

This function simply iterates over the given list of CryptoPrice objects and prints the symbol and price for each.

Step 6: Putting it all together

Finally, let’s put everything together in the main function and fetch/print the cryptocurrency prices:

func main() {
	prices, err := fetchCryptoPrices()
	if err != nil {
		fmt.Println("Error fetching prices:", err)
		return
	}

	printCryptoPrices(prices)
}

In the main function, we first call fetchCryptoPrices to fetch the cryptocurrency prices. If there’s an error, we print it and return. Otherwise, we call printCryptoPrices to print the fetched prices.

Step 7: Running the program

Save the main.go file and execute the following command in your terminal:

go run main.go

You should see the cryptocurrency symbols and their prices printed to the console.

Conclusion

In this tutorial, we created a Go-based data pipeline for tracking cryptocurrency prices. We utilized Go’s networking and web programming capabilities to fetch data from an external API and store it in a data structure. We also learned how to work with decimal numbers using the decimal package.

With this foundation, you can further extend the data pipeline to perform various operations on the fetched cryptocurrency prices, such as calculating averages, detecting trends, or storing the data in a database for further analysis.

Remember to explore the documentation of the packages used in this tutorial for a deeper understanding of their capabilities. Happy coding!

Published:

Related Articles