# Roadmap

A map of where I am and where I'm heading, not a schedule. No dates, no time estimates. It changes when my preferences or life change; see **Changes** at the bottom.

## 📍 You are here

**Taster 1 — Data analyst, 5 of 10 lessons.** Lessons 1–4 done and mastered (Lesson 4 mastered 2026-11-04). Lesson 5 (charts) done 2026-11-05; its practice skills are not mastered yet. Next: Lesson 6 (get your data from S3 into a notebook, a lab), then Lesson 7 (check and clean a CSV); both written 2026-11-06 and ready.

## The shape

Try four roles on my own review data → decide which one I want → go deep in it → show it.

| # | Milestone | Done when… |
|---|---|---|
| 1 | **Data-analyst taster** | I can describe the review data correctly (rating trends, product comparisons) and say whether a difference is real or noise |
| 2 | **Data-scientist taster** | I can show which complaint themes are **rising over time**, and what **drives low ratings**, using ML I can explain |
| ↳ | *Optional: Kaggle side quest* | One beginner tabular competition, start to submission |
| 3 | **AI-engineer taster** | My review pipeline has an **eval set** (~50 hand-checked reviews), a score, and a way to compare prompt changes |
| 4 | **ML-engineer taster** | One model runs as a reliable service in my site, with logging and monitoring |
| ✓ | **Decision checkpoint** | My taster notes point to one role |
| 5 | **Depth phase** | Defined at the checkpoint |
| 6 | **Showcase** | One public project in that role, and I can explain my work clearly (evals, data quality, model trade-offs) |

## Milestones in detail

### 1. Data-analyst taster
- **Covers:** pandas (select, filter, group, join, time), SQL, basic stats (distributions, sampling, confidence intervals, "is it noise?").
- **Environment:** start in the browser lessons; move to a Jupyter notebook during this taster, because the test runs on real data.
- **Test (proof project):** a notebook on the review data — trends and product comparisons, with at least one "real or noise?" call justified.
- **Lesson plan** (the data is CSV on S3, as one table and as separate tables, so SQL is out):
  1. ✓ Select, filter, aggregate
  2. ✓ Sort and group
  3. ✓ Summarise groups (value_counts, mean + count, share of low reviews)
  4. ✓ Dates: parse, group by month, rolling averages
  5. ✓ Charts: line and bar
  6. Get your data (lab): Jupyter notebook, AWS profiles and login, boto3 (list a prefix, get_object → read_csv) with nrows/usecols, a local copy (written)
  7. Check and clean a CSV: dtypes, missing values (incl. text like "No topic"), cleaning (written)
  8. Joins (`merge`): rebuild the one-table version from the separate tables and check they match
  9. Real or noise (1): spread, sample size, confidence intervals — including the ± on a **gap in averages** (stars) and a **gap in shares** (% incentivized), reading "does the range include 0?", what the ± does *not* cover (reviews clustered by product, product mix, who reviews), and how to say it on a slide
  10. Real or noise (2): compare two groups with a permutation test, then p-values and the t-test as the formula version of the same idea
  11. → Proof project (2–3 sessions, see Rules)

### 2. Data-scientist taster
- **Covers:** time-series basics (resampling, rolling averages, trend vs. seasonality); regression and feature importance; train/test splits and why models lie.
- **Test:** (a) which complaint themes are rising, (b) what drives low ratings — each explainable to someone who has never seen the code.

### ↳ Kaggle side quest (optional)
- Beginner tabular competition, EDA → baseline → one improvement → submission. Nothing else waits on it.

### 3. AI-engineer taster
- **Starts from:** the review pipeline I already build.
- **Covers:** evaluation — hand-labelled sets, accuracy/agreement metrics, comparing prompt or model changes, failure analysis.
- **Test:** the eval set exists, the pipeline has a score, and I can tell whether a change made it better or worse.

### 4. ML-engineer taster
- **Covers:** wrapping a model in an API, packaging (Docker), logging inputs/outputs, spotting drift or errors.
- **Test:** one model from Taster 2 runs as a service my site calls, with logs I can read.

### ✓ Decision checkpoint
Re-read the taster notes below; pick the role; write milestone 5 into this file.

### 5. Depth phase — *TBD at checkpoint*

### 6. Showcase
- One public project in the chosen role, on a public dataset.
- I can walk through it clearly: evals, data quality, trade-offs.

## Rules
- **Every milestone ends with one proof project** on real work data. It proves I can use the skills at work, which "mastered" in Practice can't. Kept short (2–3 sessions) and built only from concepts the milestone taught, so it's doable if I learned them. Written from the real data I hand over, nothing new to learn. Done when the notebook runs on my pandas version and I can explain each result without the app.
- **Every lesson has a required Read-the-docs step** with a graded question only the official docs answer (not an optional link at the end).
- Lessons use varied datasets; the review data is the **test** at each milestone, not every lesson's subject.
- Milestone work uses work data only. The one public project comes in Showcase.
- Never make a lesson a gate for daily practice. If something stalls, skip around it and note it below.

## Taster notes
*After each taster: did I enjoy it? What surprised me? Would I do this all day?*

- **Data analyst:** —
- **Data scientist:** —
- **AI engineer:** —
- **ML engineer:** —

## Changes
- **2026-11-09:** Lessons 9–10 now name the exact skills I need to tell a presentation audience how reliable a review comparison is (a Midea vs. competitors table of incentivized-share and star gaps with ±): CIs for gaps in averages and shares, limits of the ±, p-values and t-tests.
- **2026-11-06 (later):** Split Lesson 6 in two at my request: all my CSVs live in S3, so getting them into a notebook comes first. New Lesson 6 is a lab (Jupyter, AWS profiles and `aws login`, boto3 — what my projects use — into `read_csv`, `nrows`/`usecols`, a local copy; `s3fs`/`s3://` paths only as "you'll also see"); the CSV checking and cleaning moved to Lesson 7. Taster 1 is now 10 lessons.
- **2026-11-06:** Lesson 6 written: "Load and clean a CSV". The browser can't reach S3 or run Jupyter, so the lesson loads a made-up file with the work export's quirks (whole numbers as 5.0, an all-empty column, True/False with gaps, mixed capitals, "No topic" text), and its at-work step moves to a notebook and S3.
- **2026-10-26:** every milestone now ends with a short proof project on real work data (Taster 1's test is its proof project); every lesson gets a required, graded Read-the-docs step. Real data for the Taster 1 proof project: to be provided.
- **2026-10-23:** Lesson 5 (charts) written. The in-browser runner now loads matplotlib when a lesson draws charts, so charts work in lessons, practice and review cards.
- **2026-10-20:** Added a practice step between lessons and review cards: each skill is drilled until mastered (3 clean solves); cards unlock after mastery. Lessons 1–3 mastered the same day.
- **2026-10-20:** Roadmap created. Replaces the old three-phase plan (pandas/SQL → 5 Kaggle competitions → Full Stack Deep Learning + 5 projects) with role tasters on my own review data; Kaggle became an optional side quest.
