Lesson 04 · Data Science → AI/ML Engineering
Dates and trends
"Are ratings getting worse?" is a question about time. This lesson turns a date column into monthly numbers you can actually trend.
Ten reviews across four months. The date column arrives as text, which is how it comes out of most CSV files and APIs. pd and df are already loaded.
pd is the pandas library, loaded with import pandas as pd. The as pd part is just a short nickname that almost everyone uses. df is not part of pandas. It's a variable in your session that holds one table (a DataFrame). pandas built that table, but df is only a label, so reviews = df would give you the same table under another name. Rule of thumb: if you already have a table and want to act on it, call a method on df (like df.groupby(...)). If you're creating something or converting data you pass in, use a general tool from pd (like pd.read_csv(...) or pd.to_datetime(...)).
| product | date | stars | |
|---|---|---|---|
| 0 | AC | 2026-01-08 | 5 |
| 1 | Fan | 2026-01-20 | 4 |
| 2 | AC | 2026-02-03 | 3 |
| 3 | AC | 2026-02-17 | 2 |
| 4 | Fan | 2026-02-25 | 5 |
| 5 | AC | 2026-03-04 | 2 |
| 6 | Fan | 2026-03-15 | 4 |
| 7 | AC | 2026-03-28 | 1 |
| 8 | AC | 2026-04-09 | 3 |
| 9 | Fan | 2026-04-22 | 4 |
Part 1 · Learn
The syntax, and what each piece returns
ATurn text into real dates
| Syntax | Returns |
|---|---|
df["date"].dtype | str (text). It looks like a date, but pandas can't do date maths on it. |
pd.to_datetime(df["date"]) | A new Series of real dates (dtype datetime64). This is a general pandas tool, so it lives on pd and you pass the column in. It is a function, not a method: a method is called on the column itself, like df["date"].sort_values(). On its own this line leaves df unchanged. |
df["date"] = pd.to_datetime(df["date"]) | Replaces the text column with the real dates, so every later line can use them. Afterwards df["date"].dtype shows datetime64[us] (dates to the microsecond). On pandas 2 you'd see object before and datetime64[ns] after: same idea, older names. |
Do this first, every time you load data with dates. Almost every date bug starts with a column that's still text. Check with .dtype, convert, and remember to assign the result back; converting without saving it changes nothing.
BPull parts out, and filter by date
| Syntax | Returns |
|---|---|
df["date"].dt.month | A Series of month numbers (1–12). .dt unlocks date tools on a date column. Also .dt.year, .dt.day, .dt.day_name(). Compare with A: pd.to_datetime(...) makes dates from text, while .dt works on a column that is already dates. |
df[df["date"] >= "2026-03-01"] | Rows on or after 1 March. Once the column is real dates, you can compare it with a date written as text. |
Only works after step A. On a text column, .dt raises AttributeError: Can only use .dt accessor with datetimelike values.
CGroup by month
| Syntax | Returns |
|---|---|
df["date"].dt.to_period("M") | Each date turned into its month, like 2026-03. A "period" is a whole month, not one day. |
df.groupby(df["date"].dt.to_period("M"))["stars"].mean() | A Series of the average rating per month, indexed by month, in time order. This is your trend. |
….idxmin() / .idxmax() | The worst or best month (the label), as in Lesson 02. |
s.sort_index() | Sorts a Series by its labels. value_counts() sorts by count, so add this to get months back in time order. |
Before, you grouped by a column name. Here you group by a Series you computed. Same split → apply → combine, with months as the groups.
DSmooth a noisy trend
| Syntax | Returns |
|---|---|
monthly.rolling(2).mean() | Each value becomes the average of itself and the one before it (a rolling average). One bad month moves the line less. The 2 is the window size: how many months are averaged in total, counting the current one. So rolling(3) means this month plus the two before it. |
| first value(s) | NaN: a 2-month window has nothing before the first month. A 3-month window leaves 2 NaN. |
Tiny example: monthly ratings of 4, 4, 1, 4 smooth to NaN, 4.0, 2.5, 2.5. The raw line crashes from 4 to 1; the smoothed line only dips to 2.5, because the bad month is averaged with a good one (2.5 = (4 + 1) / 2).
Use it when monthly numbers jump around because each month has only a few reviews. That's the same sample-size problem as Lesson 03, solved across time instead of across products. More generally, it helps whenever a line is noisy (small counts, random swings) and you want the direction underneath. The cost: the smoothed line lags behind real changes, since each point is partly the past. If the data is already stable, or a sudden drop is a real event you need to spot at once, skip it.
Part 2 · Practice
Now write it yourself
Each run starts from the original table, where date is text. From exercise 2 on, the conversion line is already written for you.
2 · Reviews since March
Set recent to the reviews dated on or after 1 March 2026.
Hint
A mask, as in Lesson 01, comparing with "2026-03-01". See B.
3 · Average rating per month
Set monthly to the average stars per month.
Hint
Group by df["date"].dt.to_period("M") instead of a column name. See C.
4 · The worst month
Set worst to the month with the lowest average rating.
Hint
monthly is indexed by month. Which method from Lesson 02 returns the label of the smallest value?
Part 3 · Remember
Spaced review
Next, do the practice set until this lesson's skills are mastered (3 clean solves each). Then its 5 cards join your Daily review from the next day.
Use it at work tomorrow
This is the first real piece of the Taster 1 test. On your review pipeline's output:
- Convert the date column with
pd.to_datetimeand check the dtype changed. - Compute the average rating per month for one product.
- Find its worst month with
idxmin(), then filter to that month's reviews and read a few. Is there a reason, like a firmware update or a new batch?
If each month has only a handful of reviews, add .rolling(3).mean() before you trust the line.
Python for Data Analysis ch. 11: Time Series, by Wes McKinney. §11.1–11.2 cover to_datetime and indexing by date, §11.5 periods, and §11.7 moving windows (rolling).