A long time ago I spent some time thinking about how I structure files on my computer. Here's what I've been using for over 7 years now and it's still going strong.
Organized Chaos
The directory structure is:
x/<broad-category>/<projects (or "package")>
In other words, there's a single-letter directory that contains a set of directories that designate broad categories. Here are some broad categories:
coding music pics temp sensitive base nixos
Under these broad categories, all I create are directories that designate some "project". A project is (in this use of the word) just a way to designate files that belong together. Typically, a project will be worked on for some time, after which the directory is just stored there. As an example, "pics/vacation-20XX", "base/.vim", or "coding/blog" (where this blog resides). Every "project" has its own internal structure but that really bears no importance on the parent or sibling directory structure. The broad category + project structure is a way to get a simple and scalable overview. There's basically no mental overhead once the broad categories are established, and they're easily changed.
Coming back to a project is easy. The projects are easily listed with a single ls
invocation. An lsa
alias lists the 20 last modified projects. This is very practical. It's also easy to search globally from the top-level directory.
Why the top-level directory "x"?
Because it makes rsync-ing easy and it keeps it from getting polluted by programs that put stuff in $HOME.What about configurations that need to be in $HOME?
I use a tool to symlink these (a custom script listed below). I've tried using GNU Stow but it's just inadequate.
Problems with GNU Stow
I have two directories, one with sensitive configurations (x/sensitive), and one with non-sensitive configurations (x/base). These overlap in the files they contain; both contain ".ssh/authorized_keys". On a system where I can allow sensitive configurations, I will first symlink from $HOME to everything in x/base, and then do the same for x/sensitive (overwriting anything that has the same name).
This is intended behavior, as I want to "expand" my capabilities when on a sensitive-allowed system by overwriting non-sensitive files with sensitive files. Because I have some files in base that are not in "x/sensitive", I need to perform this overwrite step. GNU Stow does not allow this. A workaround is to symlink from sensitive to all files in "x/base" that "x/sensitive" doesn't re-define, and just use GNU Stow to `stow sensitive`, but I don't want to do the bookkeeping for putting symlinks in "x/sensitive" to these files. That defeats the point of computing: making life easier.
At that point I'd be using the wrong tool for the job. Not only do I want to symlink a directory, but also sometimes only its contents. For instance for directories like ".config", I don't want my dotfiles polluted by programs that store into "~/.config"
GNU Stow is also not available everywhere. I just want to rsync and run the script and get my entire home setup.
Finally, this script permits me to have different files for different DPI screens. The typical flow is as follows for getting it working on a new machine:
$ rsync -auv x/ other-machine:~/x/ $ ssh other-machine $ cd x/base # Stored generally useful scripts that can be shared, configuration that can be viewed publicly $ ./install $ cd ../sensitive # Stores SSH keys, passwords, private data $ ./install $ cd ../nixos/other-machine # Stores some per-system config, (screen DPI, input setups) $ ./install
On systems without sensitive access I do not rsync the "sensitive" directory" and hence not `./install` it either.
Code
#! /usr/bin/env bash # ARG_HELP([Link to the current directory tree from a target. Will link files recursively only. Directories containing a .link file will be symbolically linked in their entirety.]) # ARG_OPTIONAL_SINGLE([target],[t],[Target directory to link from.],["$HOME"]) # ARG_OPTIONAL_BOOLEAN([dry-run],[d],[Only print intended actions.]) # ARG_OPTIONAL_SINGLE([except],[e],[Items to skip by regex, reads from .except by default.],[]) # ARGBASH_GO() # needed because of Argbash --> m4_ignore([ ### START OF CODE GENERATED BY Argbash v2.10.0 one line above ### # Argbash is a bash code generator used to get arguments parsing right. # Argbash is FREE SOFTWARE, see https://argbash.io for more info die() { local _ret="${2:-1}" test "${_PRINT_HELP:-no}" = yes && print_help >&2 echo "$1" >&2 exit "${_ret}" } begins_with_short_option() { local first_option all_short_options='htde' first_option="${1:0:1}" test "$all_short_options" = "${all_short_options/$first_option/}" && return 1 || return 0 } # THE DEFAULTS INITIALIZATION - OPTIONALS _arg_target="$HOME" _arg_dry_run="off" _arg_except= print_help() { printf '%s\n' "Link to the current directory tree from a target. Will link files recursively only. Directories containing a .link file will be symbolically linked in their entirety." printf 'Usage: %s [-h|--help] [-t|--target] [-d|--(no-)dry-run] [-e|--except ]\n' "$0" printf '\t%s\n' "-h, --help: Prints help" printf '\t%s\n' "-t, --target: Target directory to link from. (default: '"$HOME"')" printf '\t%s\n' "-d, --dry-run, --no-dry-run: Only print intended actions. (off by default)" printf '\t%s\n' "-e, --except: Items to skip by regex, reads from .except by default. (no default)" } parse_commandline() { while test $# -gt 0 do _key="$1" case "$_key" in -h|--help) print_help exit 0 ;; -h*) print_help exit 0 ;; -t|--target) test $# -lt 2 && die "Missing value for the optional argument '$_key'." 1 _arg_target="$2" shift ;; --target=*) _arg_target="${_key##--target=}" ;; -t*) _arg_target="${_key##-t}" ;; -d|--no-dry-run|--dry-run) _arg_dry_run="on" test "${1:0:5}" = "--no-" && _arg_dry_run="off" ;; -d*) _arg_dry_run="on" _next="${_key##-d}" if test -n "$_next" -a "$_next" != "$_key" then { begins_with_short_option "$_next" && shift && set -- "-d" "-${_next}" "$@"; } || die "The short option '$_key' can't be decomposed to ${_key:0:2} and -${_key:2}, because ${_key:0:2} doesn't accept value and '-${_key:2:1}' doesn't correspond to a short option." fi ;; -e|--except) test $# -lt 2 && die "Missing value for the optional argument '$_key'." 1 _arg_except="$2" shift ;; --except=*) _arg_except="${_key##--except=}" ;; -e*) _arg_except="${_key##-e}" ;; *) _PRINT_HELP=yes die "FATAL ERROR: Got an unexpected argument '$1'" 1 ;; esac shift done } parse_commandline "$@" # OTHER STUFF GENERATED BY Argbash ### END OF CODE GENERATED BY Argbash (sortof) ### ]) # [ <-- needed because of Argbash set -Eeuo pipefail shopt -s extglob if [ -z "$_arg_except" ] && [ -e .except ] && [ -r .except ]; then except=$(cat .except) elif [ -z "$_arg_except" ]; then except='^$' else except=$_arg_except fi dry=$_arg_dry_run self=$(realpath "${BASH_SOURCE[0]}") if ! [ -e "$_arg_target" ] && [ "$dry" = 'on' ]; then exit fi source "${self%install}/.bash_colors" target=$_arg_target prefix() { sed "s/^/$1: /" } is_dry() { [ "$dry" = 'on' ] } dry() { if ! is_dry; then "$@" else echo "DRY: PWD=$PWD $@" fi } link() { from=$(echo "$1/$2" | sed $'s/\'/\'\'/g') to=$(echo "$PWD/$2" | sed $'s/\'/\'\'/g') postfix=${3:-FILE} if [ -L "$from" ] && [ "$(readlink "$from")" = "$to" ]; then echo "${BCyan}ALREADY LINKED $postfix: '$from' -> '$to'${Color_Off}" return elif [ -L "$from" ] || [ -e "$from" ]; then echo "${BRed}REMOVING: $(file "$from") ($from)${Color_Off}" dry rm -rf "$from" fi echo "${Green}LINKING $postfix: '$from' -> '$to'${Color_Off}" dry ln -s "$to" "$from" } handle_directory() { from="$1/$2" to="$PWD/$2" if [ -e "$2/.link" ]; then link "$1" "$2" "DIRECTORY" link=$2/.link if [ -f "$link" ] && [ -r "$link" ] && [ -x "$link" ]; then dry "$link" "$from" |& prefix LINK fi elif [ -e "$2/.skip" ]; then echo "${Purple}SKIP: $to${Color_Off}" else if [ -L "$from" ] && [ ! -e "$from" ]; then echo "${Yellow}REMOVING BROKEN SYMLINK: $(file "$from")${Color_Off}" dry rm "$from" fi if ! [ -d "$from" ]; then dry mkdir "$from" fi if is_dry; then (cd "$2"; "$self" -t "$from" -e "$except" -d) else (cd "$2"; "$self" -t "$from" -e "$except") fi fi } for file in .* *; do if [ "$file" = '.' ] || [ "$file" = '..' ]; then continue fi if echo "$file" | grep -P "$except" > /dev/null; then echo "${Purple}SKIP: $file - part of --except: $except${Color_Off}" continue fi if [ -f "$file" ] && [ "$(realpath "$file")" != "$self" ] ; then link "$target" "$file" elif [ -d "$file" ]; then handle_directory "$target" "$file" fi done # ] <-- needed because of Argbash