Algorithm — The Brain Behind Every Program
An algorithm is a finite sequence of well-defined instructions used to solve a problem or produce the desired output from a given set of inputs. Every step in an algorithm should be clear, unambiguous, and simple enough to be executed correctly. In computer science, algorithms form the foundation of programming, problem-solving, and software development.
Whenever we write a program, we are essentially implementing one or more algorithms. Whether it is searching for an item, sorting data, finding the shortest route, or optimizing resources, algorithms are involved everywhere behind the scenes.
Algorithms are important because they help solve problems efficiently. Two different algorithms may produce the same result, but one might take significantly less time or memory than the other. This is why understanding algorithms is one of the most important skills in computer science and software engineering.
Characteristics of an Algorithm
A good algorithm generally follows these characteristics:
Input
An algorithm should accept zero or more input values.
Output
It should produce at least one output or result.
Definiteness
Every instruction should be precise and clearly defined.
Finiteness
The algorithm must terminate after a finite number of steps.
Effectiveness
Each instruction should be simple and feasible to execute.
Why Every Instruction Must Be Unambiguous
This part trips up beginners more than anything else. When you write an algorithm on paper, things like "find the biggest number" feel obvious. But to a computer — and more importantly, to the formal analysis of that algorithm — every step has to be broken down until there's nothing left to interpret.
Consider binary search. The idea sounds simple: look at the middle, go left or right, repeat. But the moment you write it out precisely, you realize there are edge cases hiding everywhere. What if the array is empty? What if there are duplicate values? What if the target is larger than every element? A well-written algorithm answers all of these — not with prose, but with exact conditional logic. If you're curious about how that looks in a rigorous academic setting, the Asymptotic Notation notes on NoteHub are a solid place to start understanding how algorithmic behavior gets formalized.
Examples of Algorithms
Sorting Algorithms
Sorting is probably the first class of algorithms most students seriously study, and for good reason. The problem is dead simple to state — arrange elements in order — but the solutions vary wildly in elegance, efficiency, and use-case.
Bubble sort is what you learn first. It compares adjacent elements and swaps them if they're in the wrong order, passes through the array again and again until nothing moves. It works. It's also famously slow on large inputs. Merge sort, on the other hand, uses a divide-and-conquer strategy — split the array in half, sort each half recursively, merge the results. It's reliably O(n log n) in all cases. Quick sort is usually faster in practice, even though its worst-case is O(n²), because of how it interacts with memory caches.
There's a full breakdown of how merge sort and quick sort actually work — including recurrence relations — over in the DAA collection on NoteHub. If you want to see the divide-and-conquer pattern in its cleanest form before reaching sorting, the MiniMax using Divide and Conquer note walks through that paradigm really well.
Searching Algorithms
Once data is sorted (or even if it isn't), you need to find things in it. Linear search just walks through every element — works on anything, but slow. Binary search cuts the remaining search space in half each step, which is why it can find an element in a million-item sorted array in at most 20 comparisons. That jump from O(n) to O(log n) is genuinely dramatic in real systems.
The recurrence relation for binary search — T(n) = T(n/2) + O(1) — is one of the cleaner examples of how divide-and-conquer problems get analyzed. There's a dedicated note on that exact recurrence over at Binary Search (Recurrence Relation), which ties together the algorithmic idea with its formal complexity analysis.
Shortest Path Algorithms
If sorting and searching feel abstract, shortest path algorithms will ground you immediately. These are algorithms you're using every time you open Google Maps, every time a packet hops across the internet, every time a game NPC navigates around an obstacle.
Dijkstra's algorithm is the classic here. It maintains a set of visited nodes and greedily picks the unvisited node with the smallest known distance at each step. The key insight is that once a node is visited, you already know its shortest path — the greedy choice is provably optimal under non-negative edge weights. The Dijkstra's Algorithm note on NoteHub covers both the logic and worked examples.
When you need shortest paths between all pairs of nodes (not just from one source), Floyd's algorithm is the go-to. It's elegant in a way Dijkstra's isn't — three nested loops, a simple relaxation step, and you're done. Check out the All Pairs Shortest Path (Floyd's) note for a detailed walkthrough.
Optimization Algorithms
Optimization algorithms answer the question: among all possible solutions, which one is best? The "best" depends on the problem — shortest, cheapest, maximum value, minimum cost.
The knapsack problem is the canonical optimization example. You have items with weights and values, and a bag with a weight limit. Which items do you take? The greedy approach works for the fractional version — take the highest value-per-weight items first. But for the 0/1 version (you either take an item or you don't, no fractions), greedy fails and you need dynamic programming. Both approaches are covered in the Fraction Knapsack and Dynamic Programming & 0/1 Knapsack notes respectively.
Greedy algorithms in general have an interesting character — they make locally optimal choices at each step and hope (or prove) that these add up to a globally optimal solution. The Greedy Approach for Algorithm Design note goes into when and why this works, which is honestly one of the more nuanced questions in algorithm design.
For graph optimization, Minimum Spanning Tree algorithms — Kruskal's and Prim's — find the subset of edges that connects all nodes with minimum total weight. Both are greedy in spirit but differ in how they build the tree. The MST (Kruskal's & Prim's) note covers both with examples.
Performance Measurement of an Algorithm
In general, the performance of an algorithm is measured by two parameters: time complexity and space complexity.
Time Complexity
The time complexity of an algorithm represents the time required by the algorithm to run to its completion. Time complexity is a function of input size n, denoted by T(n), where n is the input size.
Here's something worth sitting with: time complexity doesn't measure seconds. It measures how the number of operations scales with input size. An algorithm that takes 10 seconds on 1000 elements might take 40 seconds on 2000 elements (quadratic growth) or just 11 seconds (logarithmic growth). The actual seconds depend on hardware, language, and a hundred other things. The complexity class is the thing that's intrinsic to the algorithm itself.
This is why Big-O notation matters so much. O(1) is constant time — doesn't matter how big the input gets. O(log n) is what binary search gives you. O(n) is linear — one pass through the data. O(n log n) is what merge sort achieves. O(n²) is what bubble sort degenerates to. O(2ⁿ) is exponential — fine for small n, catastrophic for large n.
Understanding how to read and compare these is foundational to everything else. The Asymptotic Notation note on NoteHub does a thorough job of explaining Big-O, Big-Omega, and Big-Theta — the three tools you'll use to formally describe algorithm behavior. And if you want to see how two different functions actually compare in growth rate, the Comparison of Two Functions note is exactly that.
There's also the question of which case you're measuring. Most algorithms have three scenarios:
Best case — the most favorable input (e.g., binary search finds the target on the first guess)
Average case — expected behavior over a typical or random input
Worst case — the input that makes the algorithm work hardest (this is what O notation usually captures)
Worst case is what matters most in practice, especially for systems where you have to make guarantees. If you're writing software for a medical device or a financial transaction system, "usually fast" isn't good enough.
Space Complexity
The space complexity of an algorithm represents the space requirement of the algorithm to run to its completion, denoted by S(n), where n is the input size.
Space includes everything the algorithm uses during execution — the input itself, any auxiliary data structures you create, the call stack if recursion is involved. An in-place sorting algorithm like quicksort uses O(log n) extra space (for the recursion stack). Merge sort, despite being the same time complexity, needs O(n) extra space to hold the merged subarrays.
This trade-off between time and space comes up constantly. Sometimes you can make an algorithm faster by precomputing and storing results. That's the core idea behind dynamic programming — trade space for time by storing subproblem solutions instead of recomputing them. It's the same intuition behind hash tables: pay O(n) space to get O(1) average lookup time instead of O(n) linear search.
In environments with limited memory — embedded systems, mobile devices, or processing very large datasets — space complexity can matter just as much as time complexity, sometimes more.
The Bigger Picture
Algorithms don't exist in isolation. Every algorithm operates on some data structure — an array, a tree, a graph, a hash table. The choice of data structure and the choice of algorithm are deeply intertwined. The DSA collection on NoteHub covers the data structure side of this, with notes on Fenwick trees, segment trees, and MO's algorithm — tools that show how clever data structure choices can take a brute-force O(n²) solution to O(n log n).
If you want to go further with graph algorithms specifically, the Graph note in the DAA collection covers representations and traversals — the foundation you need before Dijkstra's or Floyd's make full sense.
For external reading, the academic foundation of all this lives in resources like CLRS (Introduction to Algorithms) — dense, but the definitive reference. For a more approachable online resource, Khan Academy's algorithms course walks through searching, sorting, and graph algorithms with visualizations that make the mechanics clearer than most textbook descriptions.
A Note on Building Intuition
One thing that doesn't get said enough: reading about algorithms is only half the job. The other half is implementing them — getting your hands dirty, writing the code, debugging the off-by-one errors, and actually watching what happens to performance as input size grows.
The HWI Practice collection on NoteHub is a good place to start if you want problems to work through. The DAA Assignment 1 note is also worth looking at for a sense of how these concepts get tested in an academic setting.
Algorithms are, at their core, a way of thinking systematically. Once that muscle develops, you start seeing algorithmic structure everywhere — not just in code, but in how you break down any complex problem into clear, definite, unambiguous steps.
And that's the thing about that definition at the top of this post. "Clear, definite, unambiguous, and simple" — those aren't just requirements for a computer program. They're qualities worth developing in your own thinking.