Implementing a Concurrent HTTP Proxy in Go

Table of Contents

  1. Introduction
  2. Prerequisites
  3. Overview
  4. Setting Up the Project
  5. Implementing the Proxy Server - Handling Client Requests - Forwarding Requests to the Target Server - Handling the Server Response
  6. Testing the Proxy Server
  7. Conclusion

Introduction

In this tutorial, we will learn how to implement a concurrent HTTP proxy server in Go. A proxy server acts as an intermediary between client applications and web servers. It allows clients to make requests to the proxy server, which then forwards those requests to the target server on behalf of the client. The proxy server also receives the server’s response and relays it back to the client.

By the end of this tutorial, you will have a functioning HTTP proxy server written in Go that can handle multiple client connections concurrently.

Prerequisites

To follow this tutorial, you should have a basic understanding of Go programming language concepts. Familiarity with networking concepts such as HTTP requests and responses will also be beneficial.

Overview

  • Setting up the project
  • Implementing the proxy server
  • Handling client requests
  • Forwarding requests to the target server
  • Handling the server response
  • Testing the proxy server
  • Conclusion

Setting Up the Project

Before we start implementing the proxy server, let’s set up the project structure and dependencies.

  1. Create a new directory for your project, for example, go-proxy.

  2. Inside the go-proxy directory, initialize a new Go module:

    ```bash
go mod init github.com/your-username/go-proxy
```

  3. Next, create a new Go file named main.go to serve as the entry point for our application.

    ```bash
touch main.go
```

  4. Open the main.go file in a text editor and import the necessary packages:

    ```go
package main

import (
    "fmt"
    "net"
    "net/http"
)
```

Here, we import the `fmt`, `net`, and `net/http` packages, which are required for our proxy server implementation.

Implementing the Proxy Server

Let’s start implementing the proxy server by setting up the basic structure and handling incoming client requests.

Handling Client Requests

In the main function, we will create a TCP listener on a specific port to accept incoming client connections. For each connection, we will spawn a new goroutine to handle the client request.

func main() {
    // Create a TCP listener on a specific port
    listener, err := net.Listen("tcp", ":8080")
    if err != nil {
        panic(err)
    }

    fmt.Println("Proxy server started on port 8080")

    // Accept and handle incoming client connections
    for {
        conn, err := listener.Accept()
        if err != nil {
            fmt.Println("Error accepting connection:", err)
            continue
        }

        // Handle the client request concurrently
        go handleRequest(conn)
    }
}

func handleRequest(conn net.Conn) {
    // Read the client's request
    request, err := http.ReadRequest(bufio.NewReader(conn))
    if err != nil {
        fmt.Println("Error reading request:", err)
        conn.Close()
        return
    }

    // Log the client's request
    fmt.Println("Received request from:", conn.RemoteAddr())

    // Continue implementing the proxy server...
}

In the main function, we create a TCP listener using net.Listen on port 8080. We then enter an infinite loop, accepting client connections using listener.Accept. For each accepted connection, we spawn a new goroutine to handle the client request by calling the handleRequest function.

In the handleRequest function, we read the client’s request using http.ReadRequest and log the client’s address. We will continue implementing the proxy server in the next section.

Forwarding Requests to the Target Server

After receiving the client’s request, our proxy server needs to forward it to the target server and relay the server’s response back to the client.

func handleRequest(conn net.Conn) {
    // Read the client's request...

    // Parse the target server URL
    targetURL := request.URL
    fmt.Println("Forwarding request to:", targetURL)

    // Dial a connection to the target server
    targetConn, err := net.Dial("tcp", targetURL.Host)
    if err != nil {
        fmt.Println("Error connecting to target server:", err)
        conn.Close()
        return
    }

    // Forward the client's request to the target server
    err = request.Write(targetConn)
    if err != nil {
        fmt.Println("Error forwarding request:", err)
        conn.Close()
        targetConn.Close()
        return
    }

    // Continue implementing the proxy server...
}

In the handleRequest function, after reading the client’s request, we parse the URL field of the request to obtain the target server’s URL. We then dial a connection to the target server using net.Dial and forward the client’s request to the target server using request.Write.

Handling the Server Response

After forwarding the client’s request to the target server, our proxy server needs to handle the server’s response and relay it back to the client.

func handleRequest(conn net.Conn) {
    // Read the client's request...

    // Forward the client's request to the target server...

    // Read the target server's response
    response, err := http.ReadResponse(bufio.NewReader(targetConn), request)
    if err != nil {
        fmt.Println("Error reading response:", err)
        conn.Close()
        targetConn.Close()
        return
    }

    // Log the target server's response
    fmt.Println("Received response from:", targetConn.RemoteAddr())

    // Relay the server's response back to the client
    err = response.Write(conn)
    if err != nil {
        fmt.Println("Error relaying response:", err)
        conn.Close()
        targetConn.Close()
        return
    }

    conn.Close()
    targetConn.Close()
}

In the handleRequest function, after forwarding the client’s request to the target server, we read the target server’s response using http.ReadResponse. We log the target server’s address and relay the server’s response back to the client using response.Write.

Testing the Proxy Server

To test the proxy server, we can use a web browser and configure it to use localhost:8080 as the proxy server.

  1. Run the proxy server by executing the following command in the project directory:

    ```bash
go run main.go
```

The proxy server will start listening on port `8080`.

  2. Open a web browser and configure it to use localhost:8080 as the proxy server.

    - In Firefox, go to `Preferences > General > Network Settings` and select `Manual proxy configuration`. Enter `localhost` as the HTTP Proxy with port `8080`.

  3. Visit any website in the web browser, and the proxy server will relay the requests and responses between the client and the target server.

Conclusion

Congratulations! You have successfully implemented a concurrent HTTP proxy server in Go. We learned how to handle client requests, forward requests to the target server, and handle the server’s response. You should now have a solid foundation for building more complex proxy servers or enhancing the existing one.

Feel free to explore additional features and optimizations for the proxy server, such as caching, logging, authentication, or HTTPS support.

Published:

Related Articles