Welcome to the course on the use of Unix Tools for data, software, and production engineering! As you know, increasingly every business is a software business. Being able to interact flexibly, swiftly, and efficiently with the underlying data and software systems is an indispensable skill. The knowledge of the Unix shell and its command-line tools you will acquire through this course will boost your effectiveness and productivity in your work as a software developer, an IT professional, or a data analyst, or even as a hobbyist.

The Unix tools we’ll examine were designed, written, actively used and refined by the team that defined the modern computing landscape. They allow to easily perform almost any imaginable computing task quickly and efficiently by judiciously combining key powerful concepts. The power of Unix tools for exploring, prototyping and implementing big data processing workflows, and software engineering tasks remains unmatched. Unix tools, running on hardware ranging from tiny IoT platforms to supercomputers, uniquely allow an interactive, explorative programming style, which is ideal for solving efficiently many engineering and business analytics problems that we face every day.

Through the use of Unix tools:

• as a software developer you can quickly explore and modify code, data, and tests;
• as an IT professional, a software reliability or production engineer you can scrutinize log files, network traces, performance figures, filesystems and the behavior of processes;
• as a data analyst you can extract, transform, filter, process, load, and summarize huge data sets.

The course is uniquely based on carefully selected, interactive walk-through examples that demonstrate how each command operates in practice. The examples that we use involve problems that engineers and analysts face every day. Take time to try the examples out on your computer, and, more importantly, try the commands we learn to solve problems that you face. This will allow you to get the most out of this course.

After completing the course you’ll be able to do the following.

• Enter and combine commands in the Unix command line
• Use files, data pipelines, variables, and control structures
• Select the most useful tools and commands for fetching, selecting, generating, processing, summarizing, and reporting data
• Obtain data from databases, cloud-based hosts, version control systems, object files, archives, and your desktop files
• Accomplish diverse processing tasks by putting together suitable commands and configuring their execution parameters

To follow the course, some basic prerequisite knowledge is required. This includes a basic understanding of basic programming structures, such as conditions and loops, an appreciation of variables and their use in simple expressions, the ability to create a text file using an editor, and an understanding of the concepts of files, directories, and the file system’s tree structure. If you have gaps in these areas, you may have to work a bit harder during the first module to fill them in through online reading and further experimentation.

Enjoy the course!