ICA

Independent Component Analysis (ICA)

ICA is a technique used to separate mixed signals into independent non-Gaussian components.

Used heavily in:

• Audio processing

• Image processing

• Biomedical signals (EEG, ECG)

• Blind source separation

1. What is ICA?

ICA finds a linear transformation that makes the resulting components statistically independent.

Statistical independence:

2. Assumptions of ICA

1. Source signals are statistically independent

2. Sources are non-Gaussian

   • ICA cannot separate Gaussian components

3. Mixing is linear

   • Non-linear mixtures break ICA

3. Mathematical Representation

Observed mixed signals:

$Hidden independent components:

$Linear mixing model:

$Goal of ICA:

$Where:

• $A$ = unknown mixing matrix

• $W$ = unmixing matrix (to be learned)

ICA tries to find $W$ such that components of $s$ are as independent as possible.

Independence is measured using a function:

ICA finds $W$ that minimizes dependence.

4. Real-World Example (Party Problem)

A room has N speakers talking simultaneously and N microphones placed at different positions.

Each microphone records:

• A mixture of all speakers

• With different intensities

Goal:
Use ICA to recover each speaker’s original voice:

Where:

• $X_i$ = mixed signals

• $Y_i$​ = independent components

5. Advantages of ICA

1. Separates mixed signals

   • Excellent for blind source separation

2. Unsupervised technique

   • No labeled data needed

3. Useful for feature extraction

   • Finds important independent features

6. Disadvantages of ICA

1. Assumes non-Gaussian sources

   • Fails if sources are Gaussian

2. Assumes linear mixing

   • Ineffective for nonlinear mixtures

3. Computationally expensive

   • Hard to scale to large datasets