技術メモ

技術メモ

ラフなメモ

GoDownloaderを使ってCLIツールのインストーラを作る

Goで書いたCLIツールのバイナリ配布には GoReleaser が便利だよ、というのはよく知られていることです。GoReleaser ではYAML形式の .goreleaser.yml ファイルという設定ファイルを記述し、GitHub Actionsなどと連携してリリースを自動化することができます。

GoReleaser を用いたバイナリの配布の話は

などを参考にしてください。

GoDownloader

本記事では GoReleaser で作成したバイナリのインストーラを作成する方法を紹介します。GoDownloader を使います。

github.com

GoDownloaderのインストール

ローカルへのインストール方法は本来ですと go get github.com/goreleaser/godownloader で取得できそうですが、2021/01/24現在は以下のように invalid pseudo-version によってダウンロードできません。replace ディレクティブを駆使することで対応は可能です。

$ go get github.com/goreleaser/godownloader
go get: github.com/goreleaser/godownloader@none updating to
        github.com/goreleaser/godownloader@v0.1.0 requires
        github.com/goreleaser/goreleaser@v0.118.2 requires
        code.gitea.io/gitea@v1.10.0-dev.0.20190711052757-a0820e09fbf7 requires
        github.com/go-macaron/cors@v0.0.0-20190309005821-6fd6a9bfe14e9: invalid pseudo-version: revision is longer than canonical (6fd6a9bfe14e)

以下の2つのどちらかの方法が良いでしょう。

  • 1.Releasesからv0.1.0のバイナリを取得する
$ curl -OL https://github.com/goreleaser/godownloader/releases/download/v0.1.0/godownloader_0.1.0_Linux_x86_64.tar.gz
$ sudo tar -zxvf godownloader_0.1.0_Linux_x86_64.tar.gz -C /usr/local/bin
  • 2.git clone してソースからビルドする
$ git clone --quiet --depth 1 https://github.com/goreleaser/godownloader && cd godownloader

Makefile に以下のようにタスクを書くと便利です。

repo_name := d-tsuji/qiisync
current_dir := $(shell dirname $(realpath $(lastword $(MAKEFILE_LIST))))

installer:
    sh -c '\
      tmpdir=$$(mktemp -d); \
      cd $$tmpdir; \
      git clone --quiet --depth 1 https://github.com/goreleaser/godownloader && cd godownloader; \
      go run . -f -r ${repo_name} -o ${current_dir}/install.sh; \
      rm -rf $$tmpdir'

使い方

とても簡単です。GitHub.goreleaser.yml がコミットされている状態で以下のようにコマンドを実行するだけです。

$ godownloader -f -r d-tsuji/qiisync -o install.sh
   • reading repo "d-tsuji/qiisync" on github
   • reading https://raw.githubusercontent.com/d-tsuji/qiisync/master/goreleaser.yml
   ⨯ reading https://raw.githubusercontent.com/d-tsuji/qiisync/master/goreleaser.yml returned 404 Not Found

   • reading https://raw.githubusercontent.com/d-tsuji/qiisync/master/.goreleaser.yml
   • setting defaults for loading environment variables
   • setting defaults for snapshoting
   • setting defaults for GitHub/GitLab/Gitea Releases
   • setting defaults for project name
   • setting defaults for building binaries
   • setting defaults for archives
   • setting defaults for Linux packages with nfpm
   • setting defaults for Snapcraft Packages
   • setting defaults for calculating checksums
   • setting defaults for signing artifacts
   • setting defaults for Docker images
   • setting defaults for Artifactory
   • setting defaults for S3
   • setting defaults for Blob
   • setting defaults for homebrew tap formula
   • setting defaults for scoop manifest

上記のコマンドで指定しているオプションは以下のとおりです。

オプション 説明 デフォルト
-f 出力されるファイルを上書きするか false
-r GitHubリポジトリ -
-o 出力されるファイル名 標準出力

実行すると install.sh が作成されます。

例えば以下のような .goreleaser.yml がコミットされている場合は次のような install.sh が生成されます。

.goreleaser.yml

project_name: qiisync
before:
hooks:
    - go mod tidy
builds:
-
    goos:
    - linux
    - darwin
    - windows
    goarch:
    - amd64
    - 386
    main: ./cmd/qiisync
    binary: qiisync
    ldflags:
    - -s -w
    - "-X main.version={{.Version}}"

archives:
- format: zip
    name_template: "{{ .Binary }}_{{ .Version }}_{{ .Os }}_{{ .Arch }}{{ if .Arm }}v{{ .Arm }}{{ end }}"
    files:
    - LICENSE
    - README.md

nfpms:
-
    formats:
    - deb
    - rpm
    vendor: "d-tsuji"
    homepage: "https://github.com/d-tsuji/qiisync"
    maintainer: "Tsuji Daishiro"
    description: "Qiita CLI tool, support push and pull from/to local filesystem and Qiita."
    license: "MIT"
    file_name_template: "{{.ProjectName}}_{{.Version}}_{{.Os}}-{{.Arch}}"
    replacements:
    amd64: 64-bit
    386: 32-bit
    darwin: macOS
    linux: Tux

brews:
- description: "Qiita CLI tool, support push and pull from/to local filesystem and Qiita."
    github:
    owner: d-tsuji
    name: homebrew-qiisync
    commit_author:
    name: goreleaserbot
    email: goreleaser@carlosbecker.com
    homepage: "https://github.com/d-tsuji/qiisync"
    install: |
    bin.install "qiisync"
    test: |
    system "#{bin}/qiisync"

checksum:
name_template: "{{ .ProjectName }}_{{ .Version }}_checksums.txt"

install.sh

#!/bin/sh
set -e
# Code generated by godownloader on 2021-01-23T14:08:53Z. DO NOT EDIT.
#

usage() {
  this=$1
  cat <<EOF
$this: download go binaries for d-tsuji/qiisync

Usage: $this [-b] bindir [-d] [tag]
  -b sets bindir or installation directory, Defaults to ./bin
  -d turns on debug logging
   [tag] is a tag from
   https://github.com/d-tsuji/qiisync/releases
   If tag is missing, then the latest will be used.

 Generated by godownloader
  https://github.com/goreleaser/godownloader

EOF
  exit 2
}

parse_args() {
  #BINDIR is ./bin unless set be ENV
  # over-ridden by flag below

  BINDIR=${BINDIR:-./bin}
  while getopts "b:dh?x" arg; do
    case "$arg" in
      b) BINDIR="$OPTARG" ;;
      d) log_set_priority 10 ;;
      h | \?) usage "$0" ;;
      x) set -x ;;
    esac
  done
  shift $((OPTIND - 1))
  TAG=$1
}
# this function wraps all the destructive operations
# if a curl|bash cuts off the end of the script due to
# network, either nothing will happen or will syntax error
# out preventing half-done work
execute() {
  tmpdir=$(mktemp -d)
  log_debug "downloading files into ${tmpdir}"
  http_download "${tmpdir}/${TARBALL}" "${TARBALL_URL}"
  http_download "${tmpdir}/${CHECKSUM}" "${CHECKSUM_URL}"
  hash_sha256_verify "${tmpdir}/${TARBALL}" "${tmpdir}/${CHECKSUM}"
  srcdir="${tmpdir}"
  (cd "${tmpdir}" && untar "${TARBALL}")
  test ! -d "${BINDIR}" && install -d "${BINDIR}"
  for binexe in $BINARIES; do
    if [ "$OS" = "windows" ]; then
      binexe="${binexe}.exe"
    fi
    install "${srcdir}/${binexe}" "${BINDIR}/"
    log_info "installed ${BINDIR}/${binexe}"
  done
  rm -rf "${tmpdir}"
}
get_binaries() {
  case "$PLATFORM" in
    darwin/386) BINARIES="qiisync" ;;
    darwin/amd64) BINARIES="qiisync" ;;
    linux/386) BINARIES="qiisync" ;;
    linux/amd64) BINARIES="qiisync" ;;
    windows/386) BINARIES="qiisync" ;;
    windows/amd64) BINARIES="qiisync" ;;
    *)
      log_crit "platform $PLATFORM is not supported.  Make sure this script is up-to-date and file request at https://github.com/${PREFIX}/issues/new"
      exit 1
      ;;
  esac
}
tag_to_version() {
  if [ -z "${TAG}" ]; then
    log_info "checking GitHub for latest tag"
  else
    log_info "checking GitHub for tag '${TAG}'"
  fi
  REALTAG=$(github_release "$OWNER/$REPO" "${TAG}") && true
  if test -z "$REALTAG"; then
    log_crit "unable to find '${TAG}' - use 'latest' or see https://github.com/${PREFIX}/releases for details"
    exit 1
  fi
  # if version starts with 'v', remove it
  TAG="$REALTAG"
  VERSION=${TAG#v}
}
adjust_format() {
  # change format (tar.gz or zip) based on OS
  true
}
adjust_os() {
  # adjust archive name based on OS
  true
}
adjust_arch() {
  # adjust archive name based on ARCH
  true
}

cat /dev/null <<EOF
------------------------------------------------------------------------
https://github.com/client9/shlib - portable posix shell functions
Public domain - http://unlicense.org
https://github.com/client9/shlib/blob/master/LICENSE.md
but credit (and pull requests) appreciated.
------------------------------------------------------------------------
EOF
is_command() {
  command -v "$1" >/dev/null
}
echoerr() {
  echo "$@" 1>&2
}
log_prefix() {
  echo "$0"
}
_logp=6
log_set_priority() {
  _logp="$1"
}
log_priority() {
  if test -z "$1"; then
    echo "$_logp"
    return
  fi
  [ "$1" -le "$_logp" ]
}
log_tag() {
  case $1 in
    0) echo "emerg" ;;
    1) echo "alert" ;;
    2) echo "crit" ;;
    3) echo "err" ;;
    4) echo "warning" ;;
    5) echo "notice" ;;
    6) echo "info" ;;
    7) echo "debug" ;;
    *) echo "$1" ;;
  esac
}
log_debug() {
  log_priority 7 || return 0
  echoerr "$(log_prefix)" "$(log_tag 7)" "$@"
}
log_info() {
  log_priority 6 || return 0
  echoerr "$(log_prefix)" "$(log_tag 6)" "$@"
}
log_err() {
  log_priority 3 || return 0
  echoerr "$(log_prefix)" "$(log_tag 3)" "$@"
}
log_crit() {
  log_priority 2 || return 0
  echoerr "$(log_prefix)" "$(log_tag 2)" "$@"
}
uname_os() {
  os=$(uname -s | tr '[:upper:]' '[:lower:]')
  case "$os" in
    cygwin_nt*) os="windows" ;;
    mingw*) os="windows" ;;
    msys_nt*) os="windows" ;;
  esac
  echo "$os"
}
uname_arch() {
  arch=$(uname -m)
  case $arch in
    x86_64) arch="amd64" ;;
    x86) arch="386" ;;
    i686) arch="386" ;;
    i386) arch="386" ;;
    aarch64) arch="arm64" ;;
    armv5*) arch="armv5" ;;
    armv6*) arch="armv6" ;;
    armv7*) arch="armv7" ;;
  esac
  echo ${arch}
}
uname_os_check() {
  os=$(uname_os)
  case "$os" in
    darwin) return 0 ;;
    dragonfly) return 0 ;;
    freebsd) return 0 ;;
    linux) return 0 ;;
    android) return 0 ;;
    nacl) return 0 ;;
    netbsd) return 0 ;;
    openbsd) return 0 ;;
    plan9) return 0 ;;
    solaris) return 0 ;;
    windows) return 0 ;;
  esac
  log_crit "uname_os_check '$(uname -s)' got converted to '$os' which is not a GOOS value. Please file bug at https://github.com/client9/shlib"
  return 1
}
uname_arch_check() {
  arch=$(uname_arch)
  case "$arch" in
    386) return 0 ;;
    amd64) return 0 ;;
    arm64) return 0 ;;
    armv5) return 0 ;;
    armv6) return 0 ;;
    armv7) return 0 ;;
    ppc64) return 0 ;;
    ppc64le) return 0 ;;
    mips) return 0 ;;
    mipsle) return 0 ;;
    mips64) return 0 ;;
    mips64le) return 0 ;;
    s390x) return 0 ;;
    amd64p32) return 0 ;;
  esac
  log_crit "uname_arch_check '$(uname -m)' got converted to '$arch' which is not a GOARCH value.  Please file bug report at https://github.com/client9/shlib"
  return 1
}
untar() {
  tarball=$1
  case "${tarball}" in
    *.tar.gz | *.tgz) tar --no-same-owner -xzf "${tarball}" ;;
    *.tar) tar --no-same-owner -xf "${tarball}" ;;
    *.zip) unzip "${tarball}" ;;
    *)
      log_err "untar unknown archive format for ${tarball}"
      return 1
      ;;
  esac
}
http_download_curl() {
  local_file=$1
  source_url=$2
  header=$3
  if [ -z "$header" ]; then
    code=$(curl -w '%{http_code}' -sL -o "$local_file" "$source_url")
  else
    code=$(curl -w '%{http_code}' -sL -H "$header" -o "$local_file" "$source_url")
  fi
  if [ "$code" != "200" ]; then
    log_debug "http_download_curl received HTTP status $code"
    return 1
  fi
  return 0
}
http_download_wget() {
  local_file=$1
  source_url=$2
  header=$3
  if [ -z "$header" ]; then
    wget -q -O "$local_file" "$source_url"
  else
    wget -q --header "$header" -O "$local_file" "$source_url"
  fi
}
http_download() {
  log_debug "http_download $2"
  if is_command curl; then
    http_download_curl "$@"
    return
  elif is_command wget; then
    http_download_wget "$@"
    return
  fi
  log_crit "http_download unable to find wget or curl"
  return 1
}
http_copy() {
  tmp=$(mktemp)
  http_download "${tmp}" "$1" "$2" || return 1
  body=$(cat "$tmp")
  rm -f "${tmp}"
  echo "$body"
}
github_release() {
  owner_repo=$1
  version=$2
  test -z "$version" && version="latest"
  giturl="https://github.com/${owner_repo}/releases/${version}"
  json=$(http_copy "$giturl" "Accept:application/json")
  test -z "$json" && return 1
  version=$(echo "$json" | tr -s '\n' ' ' | sed 's/.*"tag_name":"//' | sed 's/".*//')
  test -z "$version" && return 1
  echo "$version"
}
hash_sha256() {
  TARGET=${1:-/dev/stdin}
  if is_command gsha256sum; then
    hash=$(gsha256sum "$TARGET") || return 1
    echo "$hash" | cut -d ' ' -f 1
  elif is_command sha256sum; then
    hash=$(sha256sum "$TARGET") || return 1
    echo "$hash" | cut -d ' ' -f 1
  elif is_command shasum; then
    hash=$(shasum -a 256 "$TARGET" 2>/dev/null) || return 1
    echo "$hash" | cut -d ' ' -f 1
  elif is_command openssl; then
    hash=$(openssl -dst openssl dgst -sha256 "$TARGET") || return 1
    echo "$hash" | cut -d ' ' -f a
  else
    log_crit "hash_sha256 unable to find command to compute sha-256 hash"
    return 1
  fi
}
hash_sha256_verify() {
  TARGET=$1
  checksums=$2
  if [ -z "$checksums" ]; then
    log_err "hash_sha256_verify checksum file not specified in arg2"
    return 1
  fi
  BASENAME=${TARGET##*/}
  want=$(grep "${BASENAME}" "${checksums}" 2>/dev/null | tr '\t' ' ' | cut -d ' ' -f 1)
  if [ -z "$want" ]; then
    log_err "hash_sha256_verify unable to find checksum for '${TARGET}' in '${checksums}'"
    return 1
  fi
  got=$(hash_sha256 "$TARGET")
  if [ "$want" != "$got" ]; then
    log_err "hash_sha256_verify checksum for '$TARGET' did not verify ${want} vs $got"
    return 1
  fi
}
cat /dev/null <<EOF
------------------------------------------------------------------------
End of functions from https://github.com/client9/shlib
------------------------------------------------------------------------
EOF

PROJECT_NAME="qiisync"
OWNER=d-tsuji
REPO="qiisync"
BINARY=qiisync
FORMAT=zip
OS=$(uname_os)
ARCH=$(uname_arch)
PREFIX="$OWNER/$REPO"

# use in logging routines
log_prefix() {
    echo "$PREFIX"
}
PLATFORM="${OS}/${ARCH}"
GITHUB_DOWNLOAD=https://github.com/${OWNER}/${REPO}/releases/download

uname_os_check "$OS"
uname_arch_check "$ARCH"

parse_args "$@"

get_binaries

tag_to_version

adjust_format

adjust_os

adjust_arch

log_info "found version: ${VERSION} for ${TAG}/${OS}/${ARCH}"

NAME=${BINARY}_${VERSION}_${OS}_${ARCH}
TARBALL=${NAME}.${FORMAT}
TARBALL_URL=${GITHUB_DOWNLOAD}/${TAG}/${TARBALL}
CHECKSUM=${PROJECT_NAME}_${VERSION}_checksums.txt
CHECKSUM_URL=${GITHUB_DOWNLOAD}/${TAG}/${CHECKSUM}


execute

長いですがちゃんと動作します。

CLIツールを使うユーザがバイナリをインストールするときは、以下のように実行します。

$ curl -sfL https://raw.githubusercontent.com/d-tsuji/qiisync/master/install.sh | sudo sh -s -- -b /usr/local/bin
[sudo] password for tsuji:
d-tsuji/qiisync info checking GitHub for latest tag
d-tsuji/qiisync info found version: 0.0.5 for v0.0.5/linux/amd64
Archive:  qiisync_0.0.5_linux_amd64.zip
  inflating: LICENSE
  inflating: README.md
  inflating: qiisync
d-tsuji/qiisync info installed /usr/local/bin/qiisync

このコマンド一発でインストールできるので、とても便利です。一点だけ#161 にあるようにfeatureの開発は止まっている点は注意が必要です*1。ただし v0.1.0 のリリースでも十分便利に扱うことができます。

まとめ

GoReleaser を使ってGitHubにバイナリを配布している場合は GoDownloader で簡単にインストーラを作れて便利です!

*1:機能のメンテナンスは最善を尽くされるとのことです