Cloud Computing

Concept:

Cloud Computing is the delivery of computing services (servers, storage, databases, networking, software) over the Internet ('the cloud') to offer faster innovation, flexible resources, and economies of scale.

5 Essential Characteristics (NIST Model):

• 1. On-Demand Self-Service: You can provision servers instantly without talking to a human administrator.
• 2. Broad Network Access: Services are available over the network through standard mechanisms (phones, laptops, APIs).
• 3. Resource Pooling: The provider pools computing resources to serve multiple consumers using a multi-tenant model.
• 4. Rapid Elasticity: Capabilities can be elastically provisioned and released to rapidly scale outward and inward based on demand.
• 5. Measured Service: Cloud systems automatically control and optimize resource use by leveraging a metering capability (Pay-as-you-go).

Cloud Economics:

• Traditional IT involves CapEx (Capital Expenditure): You spend a massive amount of money upfront to buy physical server racks, cooling, and real estate, regardless of if you use them fully.
• Cloud IT involves OpEx (Operational Expenditure): You pay nothing upfront. You rent resources by the hour/second and pay only for what you consume (Pay-as-you-go).

Migration Risks & Challenges:

• Vendor Lock-in: Becoming too dependent on a single provider (like AWS) making it hard to leave.
• Data Privacy/Security: Handing over sensitive user data to a third party.
• Compliance: Ensuring the cloud provider meets local laws (e.g., GDPR data sovereignty).

Concept:

Ubiquitous Cloud refers to the idea that cloud services are seamlessly available everywhere, hidden in the background, powering everyday devices.

IoT Integration:

The Internet of Things (IoT) consists of billions of edge devices (smart cars, factory sensors, smartwatches) generating massive amounts of data.

• Edge devices lack the storage and processing power to analyze this data.
• Working Mechanism: They use robust Networking Support (5G, Wi-Fi) to continuously beam telemetry data up to the Ubiquitous Cloud.
• The Cloud then uses Big Data Analytics to process it and send actionable commands back down to the devices.

The Cloud Reference Model is structured like a pyramid:

1. IaaS (Infrastructure as a Service):

• What it is: The foundational layer. You rent bare-metal servers, storage, and networking.
• Responsibility: You manage the OS, runtime, and apps.
• Example: Amazon EC2, Google Compute Engine.

2. PaaS (Platform as a Service):

• What it is: The middle layer. You rent an environment (OS + Database + Web Server) ready for coding.
• Responsibility: You only focus on writing the Application code. Provider manages the OS updates.
• Example: Google App Engine, Heroku.

3. SaaS (Software as a Service):

• What it is: The top layer. A complete, finished software application accessed via a web browser.
• Responsibility: Provider manages everything. You just use it.
• Example: Gmail, Salesforce, Office 365.

1. Public Cloud:

• Infrastructure is owned by a third-party (AWS, Azure) and shared across millions of general users over the public internet.

2. Private Cloud:

• Infrastructure is operated exclusively for a single organization. It can be physically located in the company's own on-site data center (highly secure).

3. Hybrid Cloud:

• A combination of Public and Private clouds bound together by technology that allows data and applications to move between them (e.g., keeping sensitive DBs Private, but bursting web traffic to Public).

4. Community Cloud:

• Infrastructure shared by several specific organizations that have common concerns (e.g., a shared cloud for all local hospitals to meet HIPAA compliance).

Concept:

MapReduce is a programming model designed for processing massive datasets (Big Data) in parallel across thousands of worker servers (a Hadoop cluster).

How it Works (The Flow):

• 1. Input/Split: A massive dataset is chopped up into smaller chunks (splits).
• 2. MAP Phase: Worker nodes take a chunk and apply a 'Map' function to it, transforming raw data into intermediate Key-Value pairs (K, V).
• 3. SHUFFLE & SORT Phase: The system groups all intermediate data that have the exact same Key together.
• 4. REDUCE Phase: Different worker nodes take the grouped data and apply a 'Reduce' function to aggregate/summarize the results into a Final Output.

Concept:

Virtualization is the creation of a virtual (rather than actual) version of something, such as an operating system, server, or network. It allows one physical server to run multiple isolated Virtual Machines (VMs) simultaneously.

The Hypervisor (VMM):

The software that creates and runs VMs.

1. Type 1 (Bare-Metal) Hypervisor:

• Installed directly on the physical hardware, replacing a traditional OS.
• Highly efficient, used in enterprise Data Centers.
• Examples: VMware ESXi, Xen, Microsoft Hyper-V.

2. Type 2 (Hosted) Hypervisor:

• Installed on top of an existing 'Host OS' (like Windows or macOS).
• Has overhead since it goes through the Host OS. Used for personal desktop testing.
• Examples: Oracle VirtualBox, VMware Workstation.

1. Hardware Virtualization (Full Virtualization):

• The hypervisor creates entirely fake hardware environments.
• Each Virtual Machine runs its own complete, heavy 'Guest OS' (e.g., Windows running inside a Linux host).
• Benefit: Complete isolation.

2. OS-Level Virtualization (Containerization):

• There is no Hypervisor and no Guest OS.
• 'Containers' share the exact same kernel as the underlying 'Host OS'. They only virtualize the user-space (libraries/binaries).
• Benefit: Extremely lightweight, boot in milliseconds (e.g., Docker).

Concept:

Unlike a physical cluster (a group of real computers wired together), a Virtual Cluster is a logical group of Virtual Machines.

Working Mechanism:

• These VMs don't have to reside on the same physical server. They can be distributed across multiple physical servers in a data center.
• To the user, it looks like a single unified cluster, but underneath, a Resource Manager (like Kubernetes or vCenter) is dynamically allocating CPU, RAM, and Networking resources to the VMs.
• Benefit: If one physical server fails, the Resource Manager can instantly migrate the VMs to another physical server via Live Migration.

Concept:

Securing the cloud is different from securing on-premise IT because you don't physically control the servers. This is governed by the Shared Responsibility Model.

1. Provider Responsibility (Security OF the Cloud):

• The cloud vendor (AWS/Azure) is responsible for securing the physical servers, data centers, hypervisors, and the core network infrastructure.

2. Customer Responsibility (Security IN the Cloud):

• You are entirely responsible for what you put inside the cloud. This includes securing your App Code, Customer Data, IAM Passwords, and configuring firewall rules correctly.

Concept:

Disaster Recovery is the process of regaining access to IT infrastructure after a catastrophe (flood, fire, ransomware). Business Continuity is keeping the business running during that disaster.

Working Mechanism in the Cloud:

• Traditional DR: Required paying millions for an empty, physical backup data center in another city.
• Cloud DR: You set up a 'Pilot Light' or 'Warm Standby' environment in a different Cloud Region (e.g., Ohio vs California). Data is continuously replicated.
• Failover: If the Primary Data Center goes down, traffic is immediately redirected via DNS to the Cloud DR Site, and backup servers spin up instantly to handle the load.

Service Level Agreements (SLAs):

• An SLA is a strict, legally binding contract between the Cloud Provider and the Customer.
• It defines the minimum acceptable metrics for the service, most notably Uptime Availability (e.g., 99.99% uptime guarantee).
• Penalty: If the provider breaches the SLA (e.g., the server is down for 3 hours), they must legally issue financial credits to the customer.

Trust Management:

• Because you are handing data to a third party, establishing trust is paramount.
• It involves audits, compliance certifications (SOC2, HIPAA), transparent reporting, and enforcing strict data privacy policies to assure users their data is safe.

1. Amazon Web Services (AWS):

• The pioneer and current market leader in Public Cloud.
• Began strictly as an IaaS provider (EC2, S3) but now offers hundreds of services across all models.

2. Google AppEngine (GCP):

• Google's primary entry into the cloud space was a pure PaaS (Platform as a Service) offering.
• Designed strictly for developers to write code (Python/Java) and deploy it without ever worrying about the underlying servers.

3. Microsoft Azure:

• Microsoft's massively popular cloud platform.
• Known for its seamless integration with existing Enterprise networks (Active Directory, Windows Server), making it the prime choice for Hybrid Cloud architectures.

Concept:

Aneka is a PaaS (Platform as a Service) software framework developed for building, deploying, and managing distributed applications.

Working Mechanism & Hybrid Integration:

• Instead of being tied to a specific vendor, Aneka acts as an abstraction layer (a Container).
• You install Aneka on your physical desktop grids, enterprise clusters, and public clouds (like AWS).
• It seamlessly integrates these environments into one massive compute grid.
• Role: Developers write applications using Aneka APIs (Thread, Task, MapReduce models), and Aneka intelligently deploys those tasks across the Hybrid Cloud network (using Private resources first, then bursting to Public clouds when needed).

Why use the cloud for these?

These specific applications share a common trait: they require massive, bursty computational power (HPC - High Performance Computing) that would melt a normal laptop.

1. Protein Structure Prediction (Bioinformatics):

• Requires simulating billions of molecular folds to find cures for diseases. The Cloud allows scientists to spin up 10,000 servers for 2 hours, run the simulation, and shut them down.

2. Satellite Image Processing:

• Satellites beam down terabytes of raw, high-resolution imagery daily. Cloud pipelines ingest this massive stream, use AI to analyze topography/weather, and deliver compressed results to geologists.

3. CRM (Customer Relationship Management):

• e.g., Salesforce. The Cloud allows global sales teams to securely access real-time client data simultaneously from anywhere without installing heavy software.