Originally found at http://www.ce.berkeley.edu/~kayyum/unix_tips/awktips.html (now defunct as of 2011-12-30)

UNIX and Shell Scripting Tips

This page is intended to be a quick primer on simple UNIX commands and an introduction to the UNIX scripting language.
Jump to:

Basic UNIX Commands

Command Description
List Files
ls *.ext
List Files which end in ".ext"
ls -al
List all files and attributes
ls -alt
List all files and attributes, sort by time
ls -alt | head -15 
List last 15 files sorted by time
cd [directory path or name]
Change Directory
cd ..
Move up one level in directory
cd ~/.
Move to home directory
cp [filename] [desintation]
Copy a file to another file or different destination
rm [filename]
Delete file
rm -f [filename]
Delete file without prompts
rm -R [directory]
Remove all contents of a directory
mkdir [directory]
Create a directory
find . -depth -name "*ex*" -print
Find all files with the word "ex"
cat [filename]
Send all contents of a file to the screen
pg [filename]
Send all contents of a file to the screen with page breaks
head -5 [files]
List lines 1-5 for each files
tail -5 [filename]
Displays the name of the working directory
Displays the processes running from th present shell
ps -ef
displays all processes running for the current user
Disk Usage command -reports the number of blocks contained in a file
du -ks*
Reports the number of blocks for all files within a directory in terms of 1024 byte-blocks
Display or update information about the top CPU processes. Useful to monitor CPU usage
Reports the number of total, used and available disk blocks. It also reports the physical and logical locations of disk drives.
Displays all users who are logeed on to your system
slogin [host]
Secure login to a remote host. Required when accessing programs on different platforms (SGI/Sun) and for multiprocessing

More Tips

  • Familarize yourself with a text editor (jot, nedit, vi, emacs, xemacs)

  • Use the grep command to search for a file or pattern. For example, you can use it in conjuction with the cat command to return all lines which contain a specific pattern.
    cat [filename] | grep [pattern]
    or for multiple pattern seraches, use the egrep command:
    cat [filename] | egrep -e '([pat1]|[pat2]|[pat3])'
  • Using the ">" command redirects output to a file instead of a screen. For example
     ls -1 > files.dat
    will send all the contents of the current directory to the file: files.dat.

  • Using the ">>" command appends redirected output to a file

  • For efficient file managemnt

    • always use meaningful file and directory names

    • use appropriate extensions to distinguish files:

      • ".dat" (data files),
      • ".html" (html files),
      • ".pl" (PERL files),
      • ".sh" (Shell files),
      • ".c" (C files),
      • ".cc" (compiled C files),
      • ".f" (Fortran files),

    • organize files and directories to allow for easier access and navigation

    • use shortcuts in your ".alias" file to simplify repetetive commands

    • write a "notes.txt" file for each main directory describing file contents, deveveloper, date of revisions, key strokes. Very useful for archiving and data management purposes.

    • when making revisions or updating files, keep backups by copying file and directory using the following format [filename].BUyyyymmdd (yyyy=year, mm=month, dd=day in two digit formats for mm and dd -ex Feb 1, 2003 => 20030201)

  • Use the man command to serach for online reference manuals for commands and programs

  • Practice!

File Permissions

Assisgning appropriate file permissions is critical for security potocols and accessibility. It is important to understand the variouos protocols when assigning permissions under various data groups. For example, directories stored under web directories have different file permissions and group allocations than data stored under data directories.

To view file permissions use the ls -al command. The left column contains the file type, and permisions assigned to file uses. To change file permissions, use the chmod command. The chmod command can be used to change the read, write and execute permissions for the user, group or others.

in general file users can be assigned as:

  • u (user) you, the owner of the account
  • g (group) assigned by the system administrator
  • o (other) everyone else not you, and not in your group

and permissions can assigned as:

  • r (read) allows your files to be seen and copied
  • w (write) allows the files to be changed
  • x (execute) allows an executable program to be run.

Permisisons are assigned consecutively for the user, group and others (ugo) in the read write and execute (rwx) order

Let's dissect the following example with 1 file and 1 directory:

drwx------ 2 user1 group1 512 Jul 17 15:45 tdirectory/
-rw-rw-r-- 1 user1 group1 5   Jul 18 16:03 tfile
Both the file and directory are owned by user1 and are a member of group group1. The left column of the directory, tdirectory, (drwx------) specifies that it is a directory by the "d" in the first position and it has read, write and executable (rwx) access for for the user only. The left column of the file, tfile, (-rw-rw-r--) specifies that has read and write acess for the user and group and only read access for others.

Permissions can be changed using the following syntax:

 chmod [ugo][+|-|=][rwx] file ...
For example, to add read and write privaleges for the group, type
 chmod g+rw file ...
To remove read and write privaleges for the group and others, type
 chmod o-rw file ... 
An easier way to apply permissions is by using binary codes.
  • r = 4
  • w = 2
  • x = 1
To apply read (4) and write(2) {4+2=6} privaleges to the user, use
 chmod 600 tfile 
 -rw------- 1 user1 group1 5   Jul 18 16:03 tfile
To apply read (4) and write(2) {4+2=6} privaleges to the user and group, use
 chmod 660 tfile  
 -rw-rw---- 1 user1 group1 5   Jul 18 16:03 tfile
To apply read (4) and write(2) {4+2=6} privaleges to the user and group, and read only permission to others (4) use
 chmod 664 tfile 
 -rw-rw-r-- 1 user1 group1 5   Jul 18 16:03 tfile
For more information on chmod, type man chmod at the command prompt.

In general at ERD, file permssions should not have read or write acces at the other level to maintain security, especially for web-based files and directories.

To change group allocations, use the chgrp command. For example:

 chgrp group2 tfile  
 -rw-rw---- 1 user1 group2 5   Jul 18 16:03 tfile
For more information on chgrp, type man chgrp at the command prompt.

For more information about this page, please contact K. Mansoor