Introduction

An Introduction to Shell Programming.

Shell programming (also called shell scripting), is one of the most basic forms of systems-level programming on UNIX systems today. A working knowledge of shell scripting is essential to anyone wishing to become reasonably proficient at system administration. Shell programs are used to boot and configure most UNIX systems (/etc/rc.d/*), they are used by system administrators to perform automated or repeated tasks (cron jobs), and are used by common software auto-configuration utilities(GNU’s configure scripts).As a machine boots up, it executes the shell scripts in /etc/rc.d to restore the system configuration and set up services. A detailed understanding of these startup scripts is important for analyzing the behavior of a system, and possibly modify it.

The shell programming language is by no means the most powerful or common language. Languages such as Java or Perl are much richer and more powerful. But if this is true, why is the shell programming used at all? And why should we learn it? First, thousands of shell programs already exist and are in use. We may find that we need to understand or modify these scripts, especially as a UNIX system administrator, web administrator, or software developer. Second, the shell language itself is fairly easy to learn, being relatively simple. It is arguably easier to learn than Java or Perl. Third, it has a feature that no other languages currently enjoy-it is always available in a UNIX system. When a UNIX system is in its early stages of booting, programs such as perl or tcl, etc. cannot be used for startup scripts, as the file systems on which they reside are not yet available , or worse , may have been corrupted during an unexpected, catastrophic crash. These things make the shell language an obvious (if not currently the only) choice.

The Program Development Cycle #

The program development cycle is the process of developing an application.

• First step is to create program specifications
• Second step is to create the program design
• Third step is developing the code, which is written, tested, and debugged

flowchart TD
    begin([Begin]) --> programSpecs[Program Specs] --> writeCode[Write Code]
    writeCode --> testProgram[Test Program] --> isError{Found Errors?}
    isError --> |yes| fixErrors[Fix Errors]
    fixErrors --> testProgram
    isError --> |no| endProgram[End]

Program Development Cycle.

Using High-Level Languages #

• High-level languages are computer languages that use English-like expressions
• Examples are Java, C, C++
• High-level language statements are stored in a source file, which programmers create using an editor
• High-level source files must be converted into a low-level machine language file
• A compiler is a program that converts source files into executable machine-language files
• If a source file contains syntax errors, it cannot be converted into an executable file
• A programmer must correct these errors before the program can be run

Using UNIX Shell Scripts #

• Unlike high-level language programs, shell scripts do not have to be converted into machine language
• The UNIX shell acts as an interpreter when reading script files
• Interpreters read statements in script files and immediately translate them into executable instructions and run them
• After creating shell script, the OS is instructed that the file is an executable shell script via the chmod command

Shell Script Execution #

Script files can be run in several way

• Set the path variable and type the script name at the command prompt
• Type ./filename if script is in current directory
• Type the script name preceded by the full path

When To Use Scripts #

Shell Scripting can be applied to a wide variety of system and database tasks. Though called scripting this is programming.

Repeated Tasks #

Necessity is the mother of invention. The first candidates for shell scripts will be manual tasks which are done on a regular basis.

• Backups
• Log monitoring
• Check disk space

Occasional Tasks #

Tasks which are performed rarely enough that their method, or even their need may be forgotten.

• Periodic business related reports(monthly/quarterly/yearly)
• Offsite backups
• Purging old data

Complex Manual Tasks #

Some tasks must be performed manually but may be aided by scripting.

• Checking for database locks
• Killing runaway processes

These tasks may evolve into repeated tasks.

Helper Scripts #

Don’t ignore the usefulness of “helper” scripts. Perhaps a system administrator really does need to look over the log of a system daily. But a script can help by automatically sending it on to him.

Special Tasks #

These are tasks which would not be possible without a programming language.

• Storing OS information (performance stats, disk usage, etc.) into the database.
• High frequency monitoring(several times a day or more).

There are no “one size fits all” solutions . You must evaluate any techniques or solutions based on your environments and goals. Always test any solution in a non-production environment before applying it to a production system. Make sure you completely understand new commands and techniques before applying them in your environment.

When not to use shell scripts #

• Resource-intensive tasks, especially where speed is a factor (sorting, hashing, recursion, etc)
• Procedures involving heavy-duty math operations, especially floating point arithmetic, arbitrary precision calculations, or complex numbers (use Go or Java instead)
• Complex applications, where structured programming is a necessity (type-checking of variables, function prototypes, etc.)
• Mission-critical applications upon which you betting the future of the company.
• Situations where security is important, where you need to guarantee the integrity of your system and protect against intrusion, cracking, and vandalism.
• Need native support for multi-dimensional arrays.
• Need data structures , such as linked lists or trees
• Need to generate/manipulate graphics or GUIs
• Need direct access to system hardware

Applications #

In Linux there are many ways to accomplish a given task. Given a problem to solve, we may be able get to a solution in any number of ways. Of course, some will be more efficient, be more readable, use less disk space or memory, may or may not give the user feedback on what is going on or give more accurate details and more precision to the result. Shell programming is implemented and is very useful in every technology that is implemented on Linux like platforms. It includes AIX, Linux, HP-UX, and Solaris operating systems. Many of the shell scripts are rewritten for different operating system, when it is necessary. Other shell scripts are not platform dependent. These script rewrites are sometimes needed because command syntax and output vary, sometimes in a major way, between Linux flavors. The variations are sometimes are sometimes as small as pulling the data out of a different column or using a different command switch, or they can be as major as putting several commands together to accomplish the same task to get similar output or result on different flavors of Linux.

We can use shell scripting in several ways like compiling Java programs, connecting to database from the shell and writing shell scripts to upload data from text file to database, executing SQL scripts from shell script etc. A dialog utility in scripts is used for developing small front end applications like dialog boxes within shell scripts. Some of the dialogs supported are input boxes, Menu, checklist boxes yes/no boxes, radiolist, message boxes and text boxes. Creating a dialog is very easy. We will see this designing later.

Most medium and large-scale programs are written in a compiled language, such as C, C++, or Java. The programs are translated from their original source code into object code which is then executed directly by the computer’s hardware.

Why To Use a Shell Script #

The advantage of scripting is that they often work at a higher level than compiled languages, being able to deal more easily with objects such as files and directories. The disadvantage is that they are often less efficient than compiled languages.

It can take an hour to write a simple script that would take two days to code in C or C++, and usually the script will fast enough that performance won’t be a problem. Examples of scripting languages include awk, Perl, Python, Ruby, and the shell. Because the shell is universal among Linux systems, and because the language is standardized by POSIX, shell scripts can be written carefully, used across a range of systems.

Thus, the reasons to use a shell script are -

Simplicity #

The shell is a high-level language; You can express complex operations clearly and simply using it.

Portability #

By using just POSIX-specified features, you have a good chance of being able to move your script , unchanged , to different kinds of systems.

Ease of development #

You can often write a powerful, useful script in little time.