Skip to main content

Linux

Linux is a Unix-based operating system. It's free and open source, and the most widely-used operating system. Linux inherits the Unix philosophy(minimalist, modular software development) - "do one thing do it well."

File and directory operation​

find​

Find files owned by user foo and change the owner to bar:group1

find /path/to/dir -user foo -exec chown -h bar:group1 {} \;

List only directories

find $HOME -maxdepth 1 -type d |tail -n +2 |cut -d '/' -f 3-4
  • find $HOME -maxdepth 1 -type d: Find directories in $HOME. Avoid recursive search, which is the default action of the find command.
  • tail -n +2: List from the 2nd line. The first line is the current directory, which is the $HOME itself
  • cut -d '/' -f 3-4: Cut the line by '', and print from 3rd to 4th

sed​

Replace foo with bar in a text file

sed -i.bak 's/foo/bar/g' file.txt

# Ensure the partial words are not matched
sed -i.bak 's/\bfoo\b/bar/g' file.txt
  • -i[SUFFIX]: Edit file in place (make backup if SUFFIX supplied)
  • \b: word boundary. This ensures the partial words are not matched.

Test sed before edit a file to ensure the outcome

echo foo bar foobar barfoo |sed 's/foo/bar/g'                                 
> bar bar barbar barbar

echo foo bar foobar barfoo |sed 's/\bfoo\b/bar/g'
> bar bar foobar barfoo

Add prefix_ and/or _suffix to every word in a text

# Prefix
echo foo bar | sed 's/\</prefix_/g'
> prefix_foo prefix_bar

# Suffix
echo foo bar | sed 's/\>/_suffix/g'
> foo_suffix bar_suffix

# Prefix and suffix
echo foo bar | sed -e 's/\</prefix_/g' -e 's/\>/_suffix/g'
> prefix_foo_suffix prefix_bar_suffix

Delete the line that matches the key word

seq -f line-'%02g' 10|sed -e '/03/d' -e '/06/d'

rsync​

Options I often use

  • --archive, -a : archive mode
  • --verbose, -v : increase verbosity
  • --stats : print a verbose set of statistics on the file transfer
  • --link-dest=/absolute/path/to/dir : unchanged files are hard linked from /complete/path/to/dir to the destination directory
  • --update, -u : skip files that are newer on the receiver
  • --dry-run, -n : perform a trial run with no changes made
  • --one-file-system, -x : don't cross filesystem boundaries
  • --delete : delete extraneous files from dest dirs
  • --exclude=PATTERN : exclude files matching PATTERN
  • --exclude-from=FILE : read exclude patterns from FILE
  • --progress : show progress during transfer
  • --log-file=FILE : override the "log file" setting
  • --backup, -b : make backups
  • --backup-dir=/absolute/path/to/dir/ : make backups into hierarchy based in /absolute/path/to/dir/

rsync examples

# Sync src to dest and store the differences in src.YYYY-MM-DD
rsync -av --backup --delete --backup-dir=$PWD/src.$(date +%Y-%m-%d)/ src/ dest

# Backup the system partition to /backup directory. (Exclude if /backup is on the system partition.)
rsync -a --stats --one-file-system --exclude=/backup / /backup

# Create a hard linked twin of src as src.YYYY-MM-DD
rsync -a --link-dest=$PWD/src src/ src.$(date +%Y-%m-%d)

# Rsync everything but use hard link if the file is the same as the link-dest. 
# This saves disk space while each of src.YYYY-MM-DD is a complete backup.
rsync -a --link-dest=$PWD/src.YYYY-MM-DD src/ src.$(date +%Y-%m-%d)

# Sync only files and ignore directories
rsync -av --exclude='/*/' src/ dest

Networking​

ip​

# Show list
ip link list

# Link up
ip link set dev <interface> up

# Link down
ip link set dev <interface> down

Other things​

script - make typescript of terminal session​

# Start the script
## newer version (util-linux 2.36.1)
script -a --log-timing log/install_ofed.tm --log-output log/install_ofed.log
## older version (util-linux 2.34)
script -a --timing=log/install_ofed.tm log/install_ofed.log

# Replay it
## newer version
scriptreply --log-timing log/install_ofed.tm --log-output log/install_ofed.log
## older version
scriptreply --timing=log/install_ofed.tm log/install_ofed.log

  • -a : Append the session if the output file exists

Remove special characters in the typescript file generated by the script command. This is helpful when you want to read the file in a text viewer - such as less.

cat log/install_ofed.log | ansi2txt | iconv | col -b > log/install_ofed.txt
less log/install_ofed.txt

Tips​

Show count down in sleep command​

# 60 seconds
for i in `seq 60 -1 1`; do echo -ne "\r$i "; sleep 1; done

Extract a specific file​

# Print the list of the files in the tar.gz and find the path of the file
tar ztf data_a.tar.gz

# Extract the file
tar zxvf data_a.tar.gz data_a/datetime/data-1/data-1-a-b.tif

# Extract the directory
tar zxvf data_a.tar.gz data_a/datetime/data-1

  • -x : instructs tar to extract files.
  • -f : specifies filename / tarball name.
  • -v : Verbose (show progress while extracting files).
  • -z : filter archive through gzip, use to decompress .gz files.
  • -t : List the contents of an archive