Category Archives: CLI

CLI – bzip2

CLI – If you have a file you want to compress, you have options as to how you will do it. The bzip2 command is one of the options that is open to you on a macOS system. Likewise, if you come across a previously compressed file with the .bz2 extension, you will want to use bzip2 to decompress it.

The description of bzip2 in the macOS man page is:

bzip2  compresses  files  using  the Burrows-Wheeler block sorting text compression algorithm, and Huffman coding. Compression is generally considerably better than   that achieved by more conventional LZ77/LZ78-based compressors, and approaches the performance of the PPM family of statistical compressors.

To learn a lot more about how to use bzip2, take a look at “Linux bzip2 Command Tutorial for Beginners (6 Examples)“. Yes, this is a Linux article, but the bzip2 command as implemented on macOS is fundamentally the same. Check the macOS man page for bzip2 for all of the available options.


See my other CLI and macOS articles


 

Advertisements

CLI – sed

Screen Shot 2018-08-06 at 7.16.16 PM.png

CLI – Sometimes you have a file that you wish to scan for every instance of a phrase and replace it with another phrase. The sed or stream editor command is just what you are looking for. In the above screen shot I first display a file with the cat command, then use se to replace all appearances of the word ‘Emma’ with the word ‘Bambi’. The sed command syntax for that is:

sed  ‘s/Emma/Bambi/g’ Review_Four\ Dominions\ copy.txt

The ‘s/Emma/Bambi/g’ says to do a global search and replace of Bambi for every instance of Emma. The name of the file being operated on is ‘Review_Four\ Dominions\ copy.txt’.

The description of sed in the macOS man page is:

The sed utility reads the specified files, or the standard input if no files are specified, modifying the input as specified by a list of commands.  The input is then written to the standard output.

A single command may be specified as the first argument to sed.  Multiple commands may be specified by using the -e or -f options.  All commands are applied to the input in the order they are specified regardless of  their origin.

This can be a very powerful and useful command when processing text files. In the SysAdmin world it can be very useful for updating configuration files.

To learn a lot more about how to use sed, take a look at “‘Sed’ Command In Linux: Useful Applications Explained“. Yes, this is a Linux article, but the sed command as implemented on macOS is fundamentally the same.


See my other CLI and macOS articles


 

CLI – zcat

Screen Shot 2018-07-24 at 11.06.57 AM

CLI – Sometimes you want to view the content of a compressed file. The zcat  Command Line  command allows you to do that. What does zcat do:

zcat uncompresses either a list of files on the command line or its standard
input and writes the uncompressed data on standard output.

The zcat command is like cat, but for compressed files.

To learn a lot more about how to use zcat, take a look at “Linux zcat Command Tutorial for Beginners (5 Examples)“. Yes, this is a Linux article, but the zcat command as implemented on macOS is fundamentally the same.


See my other CLI and macOS articles


 

Networking Cheatsheet for Linux (and macOS)

cheat_sheet_cover_sysadmin_networking

Whether you are a Unix/Linux/macOS Sys Admin or just a user, you often have to deal with various network issues. There are several ‘built-in’ utilities that can be used through the Command Line to help solve these issues. While this is not a full networking tutorial, the cheat sheet PDF file does give you some of the essential networking commands and examples of how you can put them to use.

While the cheat sheet is targeted at Linux, many of the commands listed will also work on your macOS system. This may be something you want to print out and add to your notes, or just download it and have it available for reference. I downloaded the PDF and put it in my iCoud Drive so that I can reference it from my Mac, MacBook or my iPad.

CLI – history

CLI – How often have you gone to the command line to run a command, one you just ran a few days earlier, but you don’t remember all of the options that gave you that perfect result? One way of being able to peer back at those commands you have entered at the command line is through the history command.

Just enter history at the command line and on a macOS system the last 512 commands entered will be shown in your terminal window from oldest to most recent. This is a command built into the default bash shell. With so many commands listed you will probably want to pipe the output of the history command into more or tail or grep to more easily find what you are looking for:

  • history | more
  • history | tail
  • history | grep <search term>

To learn a lot more about how to use apropos, take a look at “Linux history Command Tutorial for Beginners (8 Examples)“. Yes, this is a Linux article, but the history command as implemented on macOS is fundamentally the same.


See my other CLI and macOS articles


 

CLI – apropos

Screen Shot 2018-07-12 at 12.37.28 PM

You know what you want to do, but you are not sure what the command line command is that you should use. This is where the apropos command may help. While the man command gives you the details of a command, apropos searches the descriptions of all of the commands for the keyword used (in the example image above I used ‘apropos disk’ to obtain the list shown) and gives you a list of all commands that contain that keyword.

The macOS man page for apropos says:

apropos searches a set of database files containing short descriptions of system commands for keywords and displays the result on the standard output.

To learn a lot more about how to use apropos, take a look at “Linux apropos Command Tutorial for Beginners (5 Examples)“. Yes, this is a Linux article, but the top command as implemented on macOS is fundamentally the same.


See my other CLI and macOS articles


 

CLI – top

Screen Shot 2018-07-10 at 11.49.10 AM

If your system seems to be slow or there are running Apps (processes) that you want to know more about, you can enter top at the command line to see more. This command is part of almost every Unix, Linux, and macOS system.

This is similar to the Apple Utility “Activity Monitor”, though there are options you can invoke at the command line to give you more control over what you are seeing. This is a good tool to use if you are wanting to know why your Mac seems to bog down (which process is consuming CPU cycles and/or memory).

The macOS man page for top says:

The top program periodically displays a sorted list of system processes. The default sorting key is pid, but other keys can be used instead.  Various output options are available.

To learn a lot more about how to use top, take a look at “Linux top Command Tutorial for Beginners (8 Examples)” and “Tips for using the top command in Linux“. Yes, those are Linux articles, but the top command as implemented on macOS is fundamentally the same.


See my other CLI articles


CLI – Netstat

If you are having problems with your network or just want to learn a little more about it, the netstat command is a good way to start. The netstat command lets you print network connections, routing tables, interface statistics, masquerade connections, and multicast memberships.

The macOS man page for netstat says:

The netstat command symbolically displays the contents of various net-work-related data structures.  There are a number of output formats, depending on the options for the information presented.  The first form of the command displays a list of active sockets for each protocol.  The second form presents the contents of one of the other network data struc- tures according to the option selected. Using the third form, with a wait interval specified, netstat will continuously display the information regarding packet traffic on the configured network interfaces.  The fourth form displays statistics for the specified protocol or address family. If a wait interval is specified, the protocol information over  the last interval seconds will be displayed.  The fifth form displays per-interface statistics for the specified protocol or address family. The sixth form displays mbuf(9) statistics.  The seventh form displays routing table for the specified address family.  The eighth form displays routing statistics.

To learn a lot more about how to use netstat, take a look at “Linux netstat Command Tutorial for Beginners (8 Examples)“. Yes, this is a Linux article, but the netstat command as implemented on macOS is fundamentally the same.

See Apple’s Mac OS X Manual Pages for further command details.


See my other CLI articles


 

CLI – rsync

The rsync command is a utility common to Linux, Unix, BSD and macOS. Versions of rsync are now available on Windows systems as well.

This command is used to synchronize files and directories. This can be done between locally attached storage, or between two different network connected systems. Design of the rsync algorithm minimizes the network usage, while still maintaining file synchronization.

The rsync daemon can be run on a machine to allow other remote machines to copy file to or from it. The rsync command requires arguments indicating the source and destination locations.

There are many different command line options available to rsync. The general format is: rsync options source destination

One of the nice features of rsync is that it only copies files that have changed since the last time they were transferred. Check the reference list at the bottom of this article for suggestions of the best set for your application. Some of the simplest are:

  • rsync -avh /home/usr/dir/ /media/disk/backup/
    – this copies everything in the directory /home/usr/dir/ to /media/disk/backup/
  • rsync -avh –delete /home/user/dir/ /media/disk/backup – does the same as above except that files deleted from /home/user/dir/ will also be deleted from /media/disk/backup
  • rsync –progress -avh /home/usr/dir/ /media/disk/backup/ – does the same as the first example, but show how much of the copy is remaining

When I run rsync -h on my macOS 10.11.6 system I get the following list of options:

rsync is a file transfer program capable of efficient remote update
 via a fast differencing algorithm.

Usage: rsync [OPTION]... SRC [SRC]... DEST
 or   rsync [OPTION]... SRC [SRC]... [USER@]HOST:DEST
 or   rsync [OPTION]... SRC [SRC]... [USER@]HOST::DEST
 or   rsync [OPTION]... SRC [SRC]... rsync://[USER@]HOST[:PORT]/DEST
 or   rsync [OPTION]... [USER@]HOST:SRC [DEST]
 or   rsync [OPTION]... [USER@]HOST::SRC [DEST]
 or   rsync [OPTION]... rsync://[USER@]HOST[:PORT]/SRC [DEST]

The ':' usages connect via remote shell, while '::' & 'rsync://' usages connect to an rsync daemon, and require SRC or DEST to start with a module name.

Options

-v, --verbose               increase verbosity
 -q, --quiet                 suppress non-error messages
 --no-motd               suppress daemon-mode MOTD (see manpage caveat)
 -c, --checksum              skip based on checksum, not mod-time & size
 -a, --archive               archive mode; same as -rlptgoD (no -H)
 --no-OPTION             turn off an implied OPTION (e.g. --no-D)
 -r, --recursive             recurse into directories
 -R, --relative              use relative path names
 --no-implied-dirs       don't send implied dirs with --relative
 -b, --backup                make backups (see --suffix & --backup-dir)
 --backup-dir=DIR        make backups into hierarchy based in DIR
 --suffix=SUFFIX         set backup suffix (default ~ w/o --backup-dir)
 -u, --update                skip files that are newer on the receiver
 --inplace               update destination files in-place (SEE MAN PAGE)
 --append                append data onto shorter files
 -d, --dirs                  transfer directories without recursing
 -l, --links                 copy symlinks as symlinks
 -L, --copy-links            transform symlink into referent file/dir
 --copy-unsafe-links     only "unsafe" symlinks are transformed
 --safe-links            ignore symlinks that point outside the source tree
 -k, --copy-dirlinks         transform symlink to a dir into referent dir
 -K, --keep-dirlinks         treat symlinked dir on receiver as dir
 -H, --hard-links            preserve hard links
 -p, --perms                 preserve permissions
 --executability         preserve the file's executability
 --chmod=CHMOD           affect file and/or directory permissions
 -o, --owner                 preserve owner (super-user only)
 -g, --group                 preserve group
 --devices               preserve device files (super-user only)
 --specials              preserve special files
 -D                          same as --devices --specials
 -t, --times                 preserve times
 -O, --omit-dir-times        omit directories when preserving times
 --super                 receiver attempts super-user activities
 -S, --sparse                handle sparse files efficiently
 -n, --dry-run               show what would have been transferred
 -W, --whole-file            copy files whole (without rsync algorithm)
 -x, --one-file-system       don't cross filesystem boundaries
 -B, --block-size=SIZE       force a fixed checksum block-size
 -e, --rsh=COMMAND           specify the remote shell to use
 --rsync-path=PROGRAM    specify the rsync to run on the remote machine
 --existing              skip creating new files on receiver
 --ignore-existing       skip updating files that already exist on receiver
 --remove-source-files   sender removes synchronized files (non-dirs)
 --del                   an alias for --delete-during
 --delete                delete extraneous files from destination dirs
 --delete-before         receiver deletes before transfer (default)
 --delete-during         receiver deletes during transfer, not before
 --delete-after          receiver deletes after transfer, not before
 --delete-excluded       also delete excluded files from destination dirs
 --ignore-errors         delete even if there are I/O errors
 --force                 force deletion of directories even if not empty
 --max-delete=NUM        don't delete more than NUM files
 --max-size=SIZE         don't transfer any file larger than SIZE
 --min-size=SIZE         don't transfer any file smaller than SIZ
 --partial               keep partially transferred files
 --partial-dir=DIR       put a partially transferred file into DIR
 --delay-updates         put all updated files into place at transfer's end
 -m, --prune-empty-dirs      prune empty directory chains from the file-list
 --numeric-ids           don't map uid/gid values by user/group name
 --timeout=TIME          set I/O timeout in seconds
 -I, --ignore-times          don't skip files that match in size and mod-time
 --size-only             skip files that match in size
 --modify-window=NUM     compare mod-times with reduced accuracy
 -T, --temp-dir=DIR          create temporary files in directory DIR
 -y, --fuzzy                 find similar file for basis if no dest file
 --compare-dest=DIR      also compare destination files relative to DIR
 --copy-dest=DIR         ... and include copies of unchanged files
 --link-dest=DIR         hardlink to files in DIR when unchanged
 -z, --compress              compress file data during the transfe
 --compress-level=NUM    explicitly set compression level
 -C, --cvs-exclude           auto-ignore files the same way CVS does
 -f, --filter=RULE           add a file-filtering RULE
 -F                          same as --filter='dir-merge /.rsync-filter'
 repeated: --filter='- .rsync-filter'
 --exclude=PATTERN       exclude files matching PATTERN
 --exclude-from=FILE     read exclude patterns from FILE
 --include=PATTERN       don't exclude files matching PATTERN
 --include-from=FILE     read include patterns from FILE
 --files-from=FILE       read list of source-file names from FILE
 -0, --from0                 all *-from/filter files are delimited by 0s
 --address=ADDRESS       bind address for outgoing socket to daemon
 --port=PORT             specify double-colon alternate port number
 --sockopts=OPTIONS      specify custom TCP options
 --blocking-io           use blocking I/O for the remote shell
 --stats                 give some file-transfer stats
 -8, --8-bit-output          leave high-bit chars unescaped in output
 -h, --human-readable        output numbers in a human-readable format
 --progress              show progress during transfer
 -P                          same as --partial --progress
 -i, --itemize-changes       output a change-summary for all updates
 --out-format=FORMAT     output updates using the specified FORMAT
 --log-file=FILE         log what we're doing to the specified FILE
 --log-file-format=FMT   log updates using the specified FMT
 --password-file=FILE    read password from FILE
 --list-only             list the files instead of copying them
 --bwlimit=KBPS          limit I/O bandwidth; KBytes per second
 --write-batch=FILE      write a batched update to FILE
 --only-write-batch=FILE like --write-batch but w/o updating destination
 --read-batch=FILE       read a batched update from FILE
 --protocol=NUM          force an older protocol version to be used
 -E, --extended-attributes   copy extended attributes
 --cache                 disable fcntl(F_NOCACHE)
 -4, --ipv4                  prefer IPv4
 -6, --ipv6                  prefer IPv6
 --version               print version number
 (-h) --help                  show this help (-h works with no other options)

References:

  1. How to Backup Linux? 15 rsync Command Examples
  2. Do-It-Yourself Backup System Using Rsync
  3. Back up like an expert with rsync

See my other Command Line articles


 

CLI – cat

The cat command is used to ‘concatenate and print files’. Here, the term ‘print’ simply means to display the content of a file to the
standard output device, typically the console. This command is only useful with common text files.

An example of its use is:

Johns-Mac-mini:~
jpurvis$ ls Desktop/
BitTorrent Sync    Scott       jpurvis alias
IMG_0204.jpg    cat.txt       w.jpg
Johns-Mac-mini:~ jpurvis$ cat Desktop/cat.txt
This is a test of the ‘cat’ command

This command is useful to display the content of a file.

The Apple Man page for the ln command can be seen here.


See my other CLI articles