← back
blog

godlp: A GoLang CLI Experiment

I’ve had a fascination with GoLang ever since I heard about it. Especially after my first project using it, triangular arbitrage bot, I’ve been looking for an excuse to use it for something else.

Something I find myself doing often on the comand line at home is using the amazing yt-dlp project to download music and vods from YouTube and Soundcloud. yt-dlp is a flexible tool for downloading various content types from the web. It’s amazing.. Almost too amazing…

The thing is, this project has so many flags and options that I get overwhelmed. After I figured out what long command gets me my media exactly as I want, I saved it to a doc and whenever I want to dl something else, I awkwardly copy it manually form the doc and paste it into my terminal.

Then I take my media file, say a song from soundcloud, and place it into a specific folder structure for my Plex instance to pick up correctly in PlexAmp.

”Couldnt you just make a bash script?”

Well yeah sure, but simplicity isn’t the point right now! I want to build something cooler!

Cobra

You’ve likely heard of Cobra as it is the Go CLI framework. For most projects, cobra is an obvious choice.

I’ll use the cobra-cli to bootstrap the project with viper to manage configs later:

cobra-cli init --author "Matt Creekmore [email protected]" --viper

And we can get started by adding our first command:

cobra-cli add soundcloud

Binary Dependency Management

If this project is to provide people convenience, we’re going to need a way to embed binaries into the project so that users don’t have to install them seperately as a prerequisite. With Go 1.16, the native embed directive was added that looks perfect for what we need.

Let’s create an embed package so that we can re-use our binary around our app. We’ll also need to include ffmpeg for any post-processing needs. Our module ends up looking like this:

package embed

import (
	_ "embed"
	"fmt"
	"os"
	"os/exec"
)

//go:embed yt-dlp
var ytDlpBinary []byte

//go:embed ffmpeg
var ffmpegBinary []byte

// ExecuteYtDlp executes yt-dlp from the embedded binary
func ExecuteYtDlp(args []string) {
	// Create a temporary directory
	tempDir, err := os.MkdirTemp("", "yt-dlp-embed")
	if err != nil {
		fmt.Printf("Error creating temporary directory: %v\n", err)
		return
	}
	defer os.RemoveAll(tempDir)

	// Write embedded yt-dlp binary to a temporary file
	ytDlpBinaryPath := tempDir + "/yt-dlp"
	err = os.WriteFile(ytDlpBinaryPath, ytDlpBinary, 0755)
	if err != nil {
		fmt.Printf("Error writing yt-dlp binary to temporary location: %v\n", err)
		return
	}

	// Write embedded ffmpeg binary to a temporary file
	ffmpegBinaryPath := tempDir + "/ffmpeg"
	err = os.WriteFile(ffmpegBinaryPath, ffmpegBinary, 0755)
	if err != nil {
		fmt.Printf("Error writing ffmpeg binary to temporary location: %v\n", err)
		return
	}

	// Append ffmpeg path to args
	args = append(args, "--ffmpeg-location", ffmpegBinaryPath)

	// Execute yt-dlp from the temporary location
	cmd := exec.Command(ytDlpBinaryPath, args...)
	output, err := cmd.CombinedOutput()

	if err != nil {
		fmt.Println(string(output))
		fmt.Printf("Error executing yt-dlp command: %v\n", err)
		return
	}

	fmt.Println(string(output))
}

Now we’re free to import this anywhere around the project:

embed.ExecuteYtDlp(args)

Configuration

When it comes time to reading configs like for where to download our music, it couldn’t get any easier.

Viper makes it as simple as:

musicDir := viper.GetString("music_directory")

By default reading a config file from $HOME/.godlp.yaml, and optionally can pass one with --config /path/to/config.yaml

SoundCloud commands

First up is the soundcloud command.

This ended up being fairly simple. We pass our arguments along with the provided url to our inbeded yt-dlp and tell it to download them to a /temp directory:

tempDir := cwd + "/temp"

scArgs := []string{
	"-o", "%(title)s.%(ext)s",
	"--embed-metadata",
	"--embed-thumbnail",
	"--metadata-from-title", "%(album)s",
	"--paths", tempDir,
	args[0], // Assuming soundcloudURL is the first argument
}

embed.ExecuteYtDlp(scArgs)

yt-dlp doesn’t exactly have a real way to take the Album information and embed it in the metadata. our --metadata-from-title flag copies the title into the albumn field. Not exactly what we want, so I added an --album flag and a helper function to apply it to the files with ffmpeg:

albumName, _ := cmd.Flags().GetString("album")
if albumName == "" {
	fmt.Println("No album name provided. Grabbing Artist name instead.")
	artistName, err := utils.ExtractArtistNameFromFile(tempDir)
	if err != nil {
		fmt.Printf("Error extracting artist name: %v\n", err)
		return
	}
	albumName = artistName
} else {
	fmt.Println("Album flag provided. Writing with ffmpeg...")
	utils.ChangeAlbumNameWithFFmpeg(tempDir, albumName)
}

For Plex organization, the album name is used as the final save directory.

Now that we have our completed files, its as simple as moving them to our music_directory and clearing our /temp directory!

YouTube command

Implementing a youtube command is even simpler. We only need a video_directory in our config and a url we want to download.

In our command, we’ll pass the arguments as the equivalent of this command:

yt-dlp -o '%(title)s.%(ext)s' URL

This just removes the ID from the filename to make it cleaner. Then all we have to do is move the file. Done!

Builds and Releases

I have many more features planned but at the very least, I need to be able to build and distribute my app. Thankfully goreleaser exists and does everything I’m looking for.

We can define a .goreleaser.yaml and tell it what platforms we’d like to target:

builds:
  - binary: godlp
    goos:
      - darwin
      - linux
    goarch:
      - amd64
      - arm64
    env:
      - CGO_ENABLED=0
    flags:
      - -mod=vendor

release:
  prerelease: auto

universal_binaries:
  - replace: true

checksum:
  name_template: 'checksums.txt'

For now, we’re just targeting mac and linux. The -mod=vendor flag allows us to keep a copy of all of our depencenies in our repo just in case any of them become unavailable in the future.

We’ll use GitHub Actions to actually run our builds. That can be done by defining a .github/workflows/release.yaml. Goreleaser comes with many good defaults, so we only need to specify the following:

name: goreleaser

on:
  push:
    tags:
      - '*'

permissions:
  contents: write

jobs:
  goreleaser:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - name: Set up Go
        uses: actions/setup-go@v4
        with:
          go-version: stable
      - name: Run GoReleaser
        uses: goreleaser/goreleaser-action@v5
        with:
          distribution: goreleaser
          version: ${{env.GITHUB_REF_NAME}}
          args: release --clean
        env:
          GITHUB_TOKEN: ${{ secrets.PUBLISHER_TOKEN}}

And thats it! Whenever we push to master with a tag, it will trigger our build pipeline.

Well… not quite yet. There’s something we’ve got to address.

Conditional Compilation

If we were to run our builds in this current state, we’d only have one platform that actually works. This is because we don’t just have our go code to worry about platform-specific builds. We’ve also got binaries we’re dependent on.

We’re going to need darwin and linux versions of both yt-dlp and ffmpeg binaries. Windows will come soon.

We can use the go file suffixes to make multiple versions of our godlp/embed package and go will use the GOOS env variable to determine which file to use. Our embed directory will then look like this:

.
├── embed_darwin.go
├── embed_linux.go
├── ffmpeg_darwin
├── ffmpeg_linux
├── yt-dlp_darwin
└── yt-dlp_linux

Now one of our embed files looks like this :

package embed

import (
	_ "embed"
)

//go:embed yt-dlp_darwin
var YtDlpBinary []byte

//go:embed ffmpeg_darwin
var FfmpegBinary []byte

I then moved the Execute functions into the utils package since they make more sense there.

And now we’re finally done! Platform-specific builds are that simple.

Now one last thing to really make this project seem legit:

Configure a brew tap

To add this package to the homebrew package manager, lets set up a tap (or repo) for brew to know how to access my builds:

brew tap-new mcreekmore/homebrew-mcreekmore

Then, I just set up that same repo on github and add it as my remote origin and add the following to our .goreleaser.yaml:

brews:
  - name: godlp
    folder: Formula
    homepage: https://github.com/mcreekmore/godlp
    description: 'Convenient wrapper for yt-dlp'
    repository:
      owner: mcreekmore
      name: homebrew-mcreekmore
    commit_author:
      name: mcreekmore
      email: [email protected]

Now I can push a tag to my the repo and it will build:

git tag -a v0.1.0 -m "First build"
git push origin v0.1.0

Finally, I can simply add the tap I created and download my package!

brew tap mcreekmore/mcreekmore
brew install godlp

Demo

demo of godlp in the terminal

And it’s ready to be played in PlexAmp :)

screenshot showing the album we downloaded loaded in PlexAmp

Check it out for yourself