🗓️ Week 02
Operating Systems and the ☁️ Cloud

+ some more context into file formats

Dr Jon Cardoso-Silva

LSE Data Science Institute

25 Jan 2024

What we will cover today:

• Operating Systems
• The ☁️ Cloud
• ☕️ Time for a break
• What is in the cloud?
• Live Demo (Google Cloud)

Operating Systems

Let’s properly go into this topic 🐇

What Operating Systems (OS) do

• A computer can be divided into four parts:
  • hardware — provides the basic computing resources for the system
  • application programs — define how these resources are used
  • operating system — controls the hardware and coordinates its use among the various application programs for the various users
  • user — a person or a bot (a computer script) that requests actions from the computer.

Insight into operating systems

“An operating system is similar to a government. Like a government, it performs no useful function by itself. It simply provides an environment within which other programs can do userful work.”

– (Silberschatz, Galvin, and Gagne 2005, chap. 1)

• If this sounds a bit vague, it is because it is!
• It is actually tricky to specify which programs are part of the OS and which ones are not.
• Let’s try to define what an OS is anyways ⏭️

Definition of OS

• “The OS is the one programming running at all times on the computer.”
  • This is usually also called the kernel
• There might be other programs running alongside the OS.
  • For example, the Terminal
• 📱 Mobile computers usually have more “additional” software alongside the kernel, which we call the middleware.
  • These applications support multimedia, graphics, internal app databases, etc..

Why bother with this?

Image created with DALL·E via Bing Chat AI bot. Prompt: “a gigantic wooden question mark looms above the big ben, ultra-realistic awesome painting”

• It is improbable you will ever need to interact with the kernel directly.
• But, we often need to install custom software to perform some data analysis
  • This software might not come from Apple or Microsoft Store.
  • Those are things you have to install “manually.”

Tip

You will inevitably encounter puzzling ⚠️ error messages when programming, no matter how senior or skilled you are.

Understanding how everything is tied together will help you get to the core of the problem more quickly. You will get better at finding the right keywords to search for solutions online.

History of Operating Systems

History

• In the early days of modern computing, when computers were not accessible to everyone, software (applications) typically came with their source code open.
• Open source means you can read precisely which instructions the computer will follow when running.
• As the industry grew, most software companies released only the binaries — a type of file you can only execute, not read as if it was a text.
  • This includes Operating Systems! ⏭️

A computer from the 1950s
(Computer History Museum n.d.)

UNIX

• UNIX was the first big Operating System, developed at Bell Labs and AT&T
• It aimed to be simple* and easy to port to any hardware architecture
• But, it required a license
• In the late 1980s and early 1990s, a group of hackers and activists developed free & open source alternatives to UNIX.

How the UNIX System III looks like.

* As simple as these things get.

GNU/Linux

• This led to the birth of one of the most influential operating systems: GNU/Linux, or simply Linux.

• Android, the most popular OS for phones worldwide, is based on Linux.
  

• Two people were instrumental to the development of Linux
  • Richard Stallman
  • Linus Torvalds

GNU stands for “GNU is not Unix”. Computer nerds love a recursive joke.

macOS

• macOS is the Operating System of Apple computers
• It is a hybrid system. It has a free, open-source component called Darwin, but it also includes proprietary, closed-source components.
• iOS, Apple’s mobile operating system, is also based on Darwin
• Darwin is based on BDS UNIX, a derivative of the original UNIX system.

Windows

• Windows has its own history.
• Microsoft and IBM co-developed its predecessor, the OS/2 operating system.
• But then, Microsoft took on its own path and developed its own versions of the OS: Windows NT, Windows 95, Windows 98, Windows 2000, Windows XP, Windows 7, Windows Vista*, etc.
• Windows popularity can be traced to the success of the Office suite

Virtualization

• Virtualization is a technology that creates the illusion that you are running a separate private computer.
• You decide how much of your CPU/RAM/Hard drive to share with the virtual machine

Emulators & Virtual Machines

• You can install an emulator to run Windows inside Mac (and vice-versa)
  • Provided you own a licence to install the other OS

• You can share files to and from the virtual machine inside the emulator, but the internal machine will “think” it is a separate computer.

The remote machine you are using this week is a virtual machine!

• Example of commercial virtualization software

Windows Subsystem for Linux (WSL)

• In an attempt to entice Linux users (especially developers), Microsoft added a Linux emulator to Windows named “Windows Subsystem for Linux”

• You install your preferred Linux distribution
  • Ubuntu is one of the most popular

The Terminal

Recap 🕰️

• A terminal, or command prompt or command line is a screen or a window that lets you access the Operating System’s input and output.
• There are no graphics (images/video) in the terminal, only text.

Shell

Recap 🕰️

• Typically, the terminal runs a program (app) called the shell.
• The shell awaits, interprets, processes, executes, and responds to commands typed in by the user.
• There are many shells, each has its own features.
• Popular Linux shells:
  • sh or the Bourne shell: developed at AT&T labs in the 70s by a guy named Stephen Bourne.
  • bash or the Bourne again shell: very popular, compatible with sh shell scripts.
  • ksh or the Korn shell: provides enhancements over the sh and it is also compatible with bash.
  • csh and tcsh: shells that have a syntax similar to the programming language C.

Windows CMD vs PowerShell

Recap 🕰️

• Windows has its own thing.
• For historical reasons, there are two main terminals/shells on Windows these days:

CMD

Powershell

The ☁️ Cloud

What does all of that have to do with the cloud?

Definition of Cloud Computing

From the major cloud providers:

Amazon Web Services (AWS)

Microsoft Azure

Google Cloud Platform (GCP)

Definition of Cloud Computing

In sum:

• A network of computers stored somewhere else (not in your organisation).
• You can access them directly via a terminal app.
• Or you can use apps installed on these computers (data storage, on-demand computing, etc.).

Considerations

• An organisation does not need to build and maintain an infrastructure of computer servers
  • Maintain is truly a keyword here
  • You need a specialised workforce to mitigate problems and keep services up
  • 🤔 Think about it:
    • what if a CPU fails?
    • what if a hard drive becomes corrupted?
    • how often should you back up data? where should you keep it?
• Instead, CPU processing, data storage and access are delegated to a set of computers somewhere else
• Users, and organisations, only pay for what they use.

☕️ Time for a break

Image created with DALL·E via Bing Chat AI bot. Prompt: “robots enjoying a coffee break. Circular tables, white room, pops of color, modern, cosy, clean flat design.”

When we come back, we will talk about:

“What can we find in the Cloud
+ a live demo of a cloud provider”

What is in the cloud?

Google Cloud services

• Let’s see what is in Google Cloud
• Click on each area to see what the services offer

Let me walk you through what you can find on GCloud: GCloud Cheat Sheet ⏭️

What about AWS?

• Let’s see what is in Google Cloud
• Click on each area to see what the services offer

Let me walk you through what you can find on AWS: AWS Cloud Products ⏭️

Azure services

• You can browse by yourself

Here is the link to the relevant page of Azure Services.

Cloud computing models

We can think of these services in the following categories:

• Software as a Service (SaaS)
• Infrastructure as a Service (IaaS)
• Platform as a Service (PaaS)

Let’s go through each one of those.

Software as a Service (SaaS)

• Software that you interact with but runs elsewhere, not in your machine.
• To name a few examples:

File storage:

• Dropbox
• Google Drive
• One Drive
• Sharepoint

Web apps:

• Google Suite
• Google Docs
• Outlook Web
• Office 365

Infrastructure as a Service (IaaS)

• Direct access to virtual machines and data storage.
• Instead of buying computers for yourself/your organisation, you rent them.
  • You do not pay, or you pay very little when your cloud machines are off.
• Use cases: website hosting, Big Data analysis, run simulations.
• Complete flexibility: you install and run whatever you want on those computers.
  • Terms & Conditions apply, obviously.

For example, cryptomining is against the T&Cs. (Source)

IaaS | Examples

• AWS
  • EC2 (Amazon Elastic Compute Cloud)*
  • S3 (Amazon Simple Storage Service)
• Google Cloud
  • Compute
  • Storage
• Azure
  • Compute
  • Storage
  • Windows Server
  • Linux on Azure

• I highlighted EC2 because that is the service we will use in our labs this week

IaaS | Examples

• AWS
  • EC2 \(\rightarrow\) Virtual Machine
  • S3
• Google Cloud
  • Compute \(\rightarrow\) Virtual Machine
  • Storage
• Azure
  • Compute \(\rightarrow\) Virtual Machine
  • Storage
  • Windows Server
  • Linux on Azure

• I highlighted EC2 because that is the service we will use in our labs this week

Before I do a demonstration…

I need to tell you a tiny bit about security and cryptography, as these things are important when connecting to a remote machine.

SSH

• To ensure your connection to the cloud is secure, we often need to perform authentication.
• SSH, which stands for Secure SHell Protocol is a bit of software that comes pre-installed in most UNIX-like systems (macOS and Linux).
• SSH takes care of authenticating your connection to the server, ensuring the security and privacy of that connection.

SSH keys

• Central to SSH is the idea of public and private keys

• You might need to use SSH keys when connecting to:
  • a virtual machine in another computer (or in the cloud)
  • an API to download data
  • an online service like Github, a platform for sharing code (mostly open source)

We saved you the burden of using keys this week.

Instead, we configured our cloud’s SSH server to accept passwords.

However, SSH keys will make an important comeback next week when we start playing with Git!

Live Demo (Google Cloud)

Image created with DALL·E via ChatGPT. Prompt: “An imaginative and whimsical scene of a programmer sitting atop a fluffy, large cloud, engrossed in writing computer code on a laptop.”

• I will share my screen and show you how to set up an account and create a GCloud Compute instance ⏭️

• Create Google Cloud Compute, run and access.
• If time allows, do the same with AWS EC2

References

Computer History Museum. n.d. “1950 Timeline of Computer History.” 1950 Timeline of Computer History. Accessed September 16, 2022. https://www.computerhistory.org/timeline/1950/.

Ebrahim, Mokhtar, and Andrew Mallett. 2018. Mastering Linux Shell Scripting: A Practical Guide to Linux Command-Line, Bash Scripting, and Shell Programming, 2nd Edition. 2nd ed. Birmingham: Packt Publishing.

Silberschatz, Abraham, Peter B. Galvin, and Greg Gagne. 2005. Operating System Concepts. 7th ed. Hoboken, NJ: J. Wiley & Sons.