Introduction to Data Science

Introduction to Data Science
Quick reference guide – concise, organized, and easy to skim.

1. What Is Data Science?

2. The Data Science Workflow

1. Problem Formulation

   • Define business or research question.
   • Identify success metrics (accuracy, ROI, etc.).

2. Data Acquisition

   • Sources: databases, APIs, web scraping, sensors, public datasets.
   • Consider data licensing & privacy regulations (GDPR, CCPA).

3. Data Exploration & Cleaning

   • Inspect data (summary stats, visual checks).
   • Handle missing values, outliers, duplicate records.
   • Perform type conversion, normalization, encoding.

4. Feature Engineering

   • Create informative variables (aggregations, ratios, time‑based features).
   • Dimensionality reduction (PCA, t‑SNE) when needed.

5. Modeling

   • Choose algorithm family (regression, classification, clustering, deep learning).
   • Split data (train/validation/test) → avoid leakage.
   • Hyper‑parameter tuning (grid search, Bayesian optimization).

6. Evaluation

   • Metrics: accuracy, precision/recall, F1, ROC‑AUC, RMSE, MAE, R², etc.
   • Cross‑validation, confusion matrix, residual analysis.

7. Deployment & Monitoring

   • Package model (Docker, REST API, cloud service).
   • Set up logging, drift detection, performance alerts.

8. Communication & Visualization

   • Storytelling with charts, dashboards, and narrative.
   • Tailor visualizations to audience (executives vs. technical peers).

3. Essential Tools & Languages

4. Fundamental Concepts

4.1 Statistics & Probability

• Descriptive stats: mean, median, variance, percentiles.
• Probability distributions: Normal, Binomial, Poisson, Exponential.
• Inference: hypothesis testing, confidence intervals, p‑values.
• Bayesian thinking: priors, posteriors, MAP estimation.

4.