Shell Scripting Tutorial

This tutorial will give you an overview of shell programming and provide an understanding of some standard shell programs.  This includes shells such as the Bourne Shell (sh) and the Bourne Again Shell (bash).

Shells read configuration files on multiple circumstances that differ depending on the shell. These files usually contain commands for that  particular shell and are executed when loaded; they are usually used to set important variables that are used to find executables, like $PATH, and others that control the behavior and appearance of the shell.

The Bourne Shell (sh): This was one of the first shell programs that came with Unix and is also the most widely used one. It was developed by Stephen Bourne.  The ~/.profile file is used as a configuration file for sh. This is also the standard shell used for scripting.

The C Shell (csh): The C-Shell was developed by Bill Joy, modeled on the C programming language. It was intended to improve interactivity with features such as listing the command history and editing commands. The ~/.cshrc and the ~/.login files are used as configuration files by csh.

The Bourne Again Shell (bash): The bash shell was developed for the GNU project as a replacement for sh. The basic features of bash are copied from sh, and also adds some of the interactivity features from csh.  he ~/.bashrc and the ~/.profile files are used as configuration files by bash.

Shell Scripting basically is a series of commands given to shell for its execution. From Writing any command to reading and executing it all can be done with the help of shell scripting. We can run a single executable command or a series of commands that are given as input to the shell for various output related executions anytime.

Kernel being the internal part of an OS that communicates with hardware and software uses the shell as the outer part to communicate with the user. We give inputs in the form of scripts, command, etc and execute it for the desired results.

Why Shell Scripting?

Shell scripting provides us with proper command and a platform to execute the commands with the desired input and generate the output needed. It works as a mediator to talk with the system kernel and various OS components.

We can simply think it as of a path or a way to a door that connects with the room we need for. So basically having the knowledge for shell scripting will ease the way we are willing to execute our commands further. We can start or stop any application together by just writing a simple script or we can initialize anything needed at the boot time with the help of shell scripting. It reduces the effort of a user by writing a series of commands at once.

Applications of Shell Scripting

Shell Scripts are used for many purposes some of them are like:-

• Execution of any batch job.
• Execution of any run on startup.
• Running a program.
• Linking of directories and files over.
• Routine task execution.
• Monitoring of Jobs.
• Kernel level communication.
• Root dir operations.

Example

Let us see a simple example for Shell Scripting:-

Simply saving the file with Demo.sh and running this will print on Hello this is the Demo Run on the terminal. We can implement any logic that we want to for our operations and can instruct the machine doing so.

Let us see a simple example for the Addition of two numbers.

Saving and running this will give the output as Enter the 1st no, 4

Enter the 2nd no,5

The result of addition = 9.

So in this way, we can make our own scripts and make the execution successful.

Uses of Shell Scripting in Various Fields

Shell script is a normal text file with a list of commands which will tell the operating system or system-level process to execute the command and get the desired result from that command likewise all commands will be executed line by line by checking multiple conditional statements if any, executing same command multiple times if it has used built-in functions such as loops (for, while, until), performing some logic operations like addition, subtraction or logic to find out prime numbers from a given list, changing the system-level variables, checking the processes which are running, how much disk space is used, memory usage by each process, installing prerequisites on a machine to run an application, etc. The commands in a shell script might perform any one of the operations and achieve the tasks given by the user.

1. Used by System Administrator

System administrator uses shell scripting to automate the repetitive tasks, account creation for different users, monitoring the system status, usage of memory, running process on a system, kill or terminate multiple processes, can interact directly with operating system and made changes at system level so that it will reflect after the system reboot, can develop scripts to perform during the boot operation of the system.

Creating system images with current state of the system and stored in cloud storage, connecting multiple machines, automate ssh connection for controlling multiple machines at a time, generating system-level reports of all systems in the network from one system in an automatic manner by writing shell scripts to automate all the tasks and monitor the process just by executing the shell script and get the work done.

2. Data Backup

Shell scripts can be used to automate the data backup process by creating a corn job and schedule the shell script to execute data backup commands and perform the task in an easy way along with the timestamp will be stored so it will be easy to use them later when we want to use the backup of a particular day, data archiving, restoring the system to a particular date can be done by using shell scripts, we can monitor the system status while doing backup operation, restore operation. We can do the system restore by using the backup data which we stored previously by using shell scripts we can achieve all the operations.

3. Programming

Shell scripts can be used by programmers to develop programs, scripts to perform system-level tasks, finding patterns in a file, processing the data in a file based on the user inputs, etc. can be done using shell scripts. Shell scripts can be used like other programming languages as we have built-in functions such as for loop, while loop, until loop, etc.

We can directly interact with the operating system, control the processes running in the system, terminate multiple processes and execute multiple processes at the same time, monitor the process with different metrics such as memory, CPU usage, time taken parameters can be displayed on the screen. Shell scripts can be used to perform logical operation programs such as sort an array, reverse a string, reverse a number, display a file line by line, etc. can be done using shell scripting.

4. Automation

Shell scripting can be used to automate several daily tasks, repetitive tasks, etc. If we want to execute the same command multiple times then we can use shell script built-in functions like for loop, while loop etc. If we want to find a pattern in a file we can use sed/awk and the same thing we can apply when we want to find patterns in multiple files.

Get the lists of processes running in a system and we can kill multiple processes that are consuming more CPU power, taking more time, etc. can be terminated by finding the processes using it. We can automate the process of installing the pre-requisite packages on a system before running an application or set up a fresh system with the new operating system and then install all required packages so a user can use it for his tasks.

5. Application and System Related Tasks

Shell scripts are used to develop system-level scripts that will run during system boot-up. We can develop scripts to automate the application running by automating the application startup scripts, develop scripts to automate the dependency package installation of an application.

Automate the commercial applications running and report collection, end to end automation of our own code, running same application across all servers connected in a network by developing batch commands, automate the account creation by system administrators, we can develop scripts to automate the daily backup of the application data by creating corn jobs in the system. By using scripting language we can increase the speed of the development process of an application, we can determine the performance of the application by collecting different metrics all these things can be done easily with it.

Advantages of Shell Scripting

1. 1. The syntax used in shell scripting is very simple and easy to implement. The commands are exactly the same as those used directly in the CLI.
2. 2. Often, writing and executing a shell script is faster than coding and executing in an equivalent coding language.
3. 3. The program or file selection is easy in shell scripting.
4. 4. The scripting offers for interactive debugging, logical sequencing, and decision-making linkage for the existing programs.
5. 5. The absence of compilation of code turns out to be an advantage for moderately small scripts.
6. 6. The interpretive nature makes it simpler to debug, by writing the debugging code into a script, which can be re-run multiple times to detect and fix the bugs.
7. 7. Automation of the frequently performed tasks like rolling logs, extracting data, etc., can be performed efficiently.
8. 8. A sequence of commands can be executed at once through a script.
9. 9. It is portable, that is, it can be deployed in any UNIX-like systems, without many modifications.

Disadvantages of shell scripts

• Prone to costly errors, a single mistake can change the command which might be harmful
• Slow execution speed
• Design flaws within the language syntax or implementation
• Not well suited for large and complex task
• Provide minimal data structure unlike other scripting languages. etc

If you are using any major operating system you are indirectly interacting to shell. If you are running Ubuntu, Linux Mint or any other Linux distribution, you are interacting to shell every time you use terminal. In this article I will discuss about linux shells and shell scripting so before understanding shell scripting we have to get familiar with following terminologies –

1. 1. Kernel
2. 2. Shell
3. 3. Terminal

What is Kernel?

The kernel is a computer program that is the core of a computer’s operating system, with complete control over everything in the system. It manages following resources of the Linux system –

• File management
• Process management
• I/O management
• Memory management
• Device management etc.
• It is often mistaken that Linus Torvalds has developed Linux OS, but actually he is only responsible for development of Linux kernel.
• Complete Linux system = Kernel + GNU system utilities and libraries + other management scripts + installation scripts.

What is Shell?

A shell is special user program which provide an interface to user to use operating system services. Shell accept human readable commands from user and convert them into something which kernel can understand. It is a command language interpreter that execute commands read from input devices such as keyboards or from files. The shell gets started when the user logs in or start the terminal.

                                                                                                                                       Linux Shell

Shell is broadly classified into two categories –

• Command Line Shell
• Graphical shell

Command Line Shell

Shell can be accessed by user using a command line interface. A special program called Terminal in linux/macOS or Command Prompt in Windows OS is provided to type in the human readable commands such as “cat”, “ls” etc. and then it is being execute. The result is then displayed on the terminal to the user. A terminal in Ubuntu 16.4 system looks like this –

                                                                                                                        Linux Command Line

In above screenshot “ls” command with “-l” option is executed.
It will list all the files in current working directory in long listing format.
Working with command line shell is bit difficult for the beginners because it’s hard to memorize so many commands. It is very powerful, it allows user to store commands in a file and execute them together. This way any repetitive task can be easily automated. These files are usually called batch files in Windows and Shell Scripts in Linux/macOS systems.

Graphical Shells

Graphical shells provide means for manipulating programs based on graphical user interface (GUI), by allowing for operations such as opening, closing, moving and resizing windows, as well as switching focus between windows. Window OS or Ubuntu OS can be considered as good example which provide GUI to user for interacting with program. User do not need to type in command for every actions.A typical GUI in Ubuntu system –

GUI shell :
Each shell does the same job but understand different commands and provide different built in functions.

Usually shells are interactive that mean, they accept command as input from users and execute them. However some time we want to execute a bunch of commands routinely, so we have type in all commands each time in terminal.
As shell can also take commands as input from file we can write these commands in a file and can execute them in shell to avoid this repetitive work. These files are called Shell Scripts or Shell Programs. Shell scripts are similar to the batch file in MS-DOS. Each shell script is saved with .sh file extension eg. myscript.sh
A shell script have syntax just like any other programming language. If you have any prior experience with any programming language like Python, C/C++ etc. it would be very easy to get started with it.
A shell script comprises following elements –

1. 1. Shell Keywords – if, else, break etc.
2. 2. Shell commands – cd, ls, echo, pwd, touch etc.
3. 3. Functions
4. 4. Control flow – if..then..else, case and shell loops etc.

Why do we need shell scripts?

There are many reasons to write shell scripts –

• To avoid repetitive work and automation
• System admins use shell scripting for routine backups
• System monitoring
• Adding new functionality to the shell etc.

Simple demo of shell scripting using Bash Shell

If you work on terminal, something you traverse deep down in directories. Then for coming few directories up in path we have to execute command like this as shown below to get to the “python” directory –

It is quite frustrating, so why not we can have a utility where we just have to type the name of directory and we can directly jump to that without executing “cd ../” command again and again. Save the script as “jump.sh”

1. 1. filter_none
2. 2. edit
3. 3. play_arrow
4. 4. brightness_4

For now we cannot execute our shell script because it do not have permissions. We have to make it executable by typing following command –

Now to make this available on every terminal session, we have to put this in “.bashrc” file.
“.bashrc” is a shell script that Bash shell runs whenever it is started interactively. The purpose of a .bashrc file is to provide a place where you can set up variables, functions and aliases, define our prompt and define other settings that we want to use whenever we open a new terminal window.
Now open terminal and type following command –

Now open you terminal and try out new “jump” functionality by typing following command-

just like below screenshot –

CONCLUSION

Shell scripting is very useful to perform operations in Linux and other UNIX-like operating systems. It forms one of the powerful ways of programming in Linux. The scripting language is easy and syntax is not heavy on the processor.

Many programmers and tech geeks across the world prefer to work on Linux systems to code, as it offers one of the robust means of coding like shell scripting.