Small Data

Cheatsheets

A combination of potential blog posts, links, stackoverflow answers and experience. Ctrl+F is your friend.

Git

Delete history

Use the following script / commands. $1 refers to the commit hash of the commit from which you would like to keep history.

#!/bin/bash
git checkout --orphan temp $1 # create a new branch without parent history
git commit -m "Truncated history" # create a first commit on this branch, note that this might result in a lot of new untracked files (can also delete unwanted items here)
git rebase --onto temp $1 master # now rebase the part of master branch that we want to keep onto this branch
git branch -D temp # delete the temp branch

# The following 2 commands are optional - they keep your git repo in good shape.
git prune --progress # delete all the objects w/o references
git gc --aggressive # aggressively collect garbage; may take a lot of time on large repos

Don't forget to git push -f to force push changes to master at the end.

Clean up a git repository

First clean up all branches and stashes that you don't require. git branch -D and git push -d

git fetch --prune # prune remote branches

git stash clear
git reflog expire --expire-unreachable=now --all 
git gc --prune=now
Remove untracked files
  • files: git clean -n
  • files: git clean -f
  • directories: git clean -fd
  • ignored files: git clean -fX
  • ignored and non-ignored files: git clean -fx
Reset local branch after a force push

Note that this may require a git clean in the local repo

git fetch
git reset --hard origin/remote-branch-name
Checkout branch from tag
git checkout tags/v1.0.0 -b bugfix-v1
Cherry pick a commit in staging mode
git cherry-pick <commit-hash> -n

Javascript

Docker

  • Build image: docker build -t [image-name] .
  • Run image in detached mode and expose docker port 25 as 3035 on localhost: docker run -d -p 3025:25 [image-name]
  • List running docker images: docker ps
  • Jump into a shell on running docker image: docker exec -it [container-id] bash, (change bash to any other command to execute a command on the running instance)
  • Stop all images: docker stop $(docker ps -a -q)
  • Stop all images: docker rm $(docker ps -a -q)

Postgresql

cli (psql)
connect to an existing db with username and password
psql <dbname> -h <hostname> -U <username> -W, make sure to use localhost when necessary
create db and user via root
sudo -u postgres psql postgres

CREATE ROLE myuser LOGIN PASSWORD 'mypass';
CREATE DATABASE mydatabase WITH OWNER = myuser;
SHOW port

psql -h localhost -d mydatabase -U myuser -p <port> 
create db and user programmatically
sudo -u postgres createuser -s $USER
createdb mydatabase
psql -d mydatabase 
show all users
\du
show tables
\dt, SELECT table_name FROM information_schema.tables WHERE table_schema = 'public';
show databases
\l, SELECT datname FROM pg_database;
show columns
\d table, SELECT column_name FROM information_schema.columns WHERE table_name ='table';
describe table
\d+ table, SELECT column_name FROM information_schema.columns WHERE table_name ='table';
quit
\q
Extract `bytea` data into a file, an image for example
Use `COPY` command with encode to hex format and then apply `xxd` shell command (with `-p` continuous hexdump style switch): \copy (SELECT encode(file, 'hex') FROM samples LIMIT 1) TO '~/image.hex' xxd -p -r ~/image.hex > ~/image.jpg

SSL

Check expiration date

echo | openssl s_client -showcerts -connect smalldata.tech:443 2>/dev/null | openssl x509 -inform pem -noout -dates

Ubuntu

Email

echo 'hello snoop' | mail -aFrom:vic@smalldata.tech -s "Test subject" snoop@gmail.com

Java

sudo update-java-alternatives

Disable touchscreen
xinput --list
xinput disable [touchscreen XID]
CLI
Count lines of code

find . -name '*.js' | xargs wc -l

PDF & Image manipulation
Conversion
convert -density 300 taxi.pdf -quality 90 taxi.jpg
convert taxi.jpg taxi.pdf
convert *.jpg wedding-party-invite.pdf
Downscaling scanned pdfs

convert -density 300x300 -quality 80 -compress jpeg fat.pdf skinny.pdf

Check dpi of images

identify -format "%w x %h %x x %y" card.png where PixelsPerInch = PixelsPerCentimeter * 2.54

making a gif with resize

convert -resize 50% -delay 20 -loop 0 *.JPG myimage.gif

resize images

for file in *.JPG; do convert $file -resize 50% lo-$file; done

Ebooks
Convert from epub to pdf using calibre
ebook-convert src.epub out.pdf --extra-css "body { font-family: Lato, Georgia; line-height: 1.5; }"  --base-font-size 10 --embed-all-fonts --margin-left 20 --margin-right 20 --margin-top 20 --margin-bottom 20 --unsmarten-punctuation