Topic 8.8: Categorical Encoding

A theatre assigns seats on two dimensions: the row (A, B, C, D — with A being closest to the stage) and the section (Left, Centre, Right). For its ticketing database, both dimensions must be stored as numbers. For the row letters, a simple number assignment works well — A=1, B=2, C=3, D=4 — because there is a natural order: row A is genuinely closer to the stage than row D. The ordering is meaningful.

For the section (Left, Centre, Right), a simple number assignment is problematic. If Left=1, Centre=2, and Right=3, the database implicitly claims that Centre is the mathematical average of Left and Right, and that Right is three times Left. Neither claim is true — the sections are distinct locations with no numeric relationship. The correct approach is to create three separate binary columns: is_left, is_centre, is_right — one column per option.

The LabelEncoder from sklearn.preprocessing is the standard tool for this. It assigns each unique category a distinct integer. It is the correct choice when the categorical variable is ordinal — when the categories have a meaningful, rankable order. The W8 level column is a clear example: beginner < intermediate < advanced represents a genuine learning progression.

import pandas as pd
from sklearn.preprocessing import LabelEncoder

df = pd.read_csv('../Datasets/8_4_competition_results.csv')

# Inspect the categorical column
print(df['level'].unique())
print(df['city'].unique())

With the categories identified, LabelEncoder can be fit directly on the level column to assign each unique value an integer:

# Label Encoding — assign an integer to each unique category
le = LabelEncoder()
df['level_encoded'] = le.fit_transform(df['level'])

print(df[['level', 'level_encoded']].head(10))

The exact integer assigned to each category is visible via le.classes_, which lists the categories in the order they were mapped:

# See the exact mapping: class → integer
print("Classes:", le.classes_)
# Output: ['advanced' 'beginner' 'intermediate']
# advanced=0, beginner=1, intermediate=2

LabelEncoder assigns integers alphabetically — not in any meaningful learning-progression order. Here that produces advanced=0, beginner=1, intermediate=2, which is the reverse of the logical order (beginner should be the lowest level, advanced the highest). A model trained on this encoding would learn that 'advanced' (0) ranks below 'intermediate' (2), which contradicts the real-world meaning.

One-Hot Encoding converts a categorical column with N unique values into N binary (0/1) columns — one per category. Each new column represents a yes/no question: 'Is this row in this category?' For any given row, exactly one column is 1 and all others are 0 — hence 'one-hot'.

In Pandas, One-Hot Encoding is performed with pd.get_dummies(). It is the correct choice for nominal categories — categories with no meaningful ordering. The W8 city column is nominal: there is no sense in which Cairo > Alexandria or Giza < Cairo. Each city is an independent category.

# One-Hot Encoding — create a binary column for each category
df_encoded = pd.get_dummies(df, columns=['city'])

print("New columns:", [c for c in df_encoded.columns if 'city' in c])
df_encoded.head()

Notice that get_dummies() creates a separate column for every distinct string it sees — including formatting variants. Because the city column here still contains its original 7 spellings of 3 real cities (e.g. 'Cairo', 'cairo', 'CAIRO', 'cairo ' with a trailing space), one-hot encoding produces 12 columns instead of 3. pd.get_dummies() also returns boolean (True/False) columns by default, not 0/1 integers. This is exactly why format standardisation (Topic 8.7) must happen before encoding — otherwise the model receives a separate, meaningless column for every spelling mistake.

Choosing between the two encoding methods depends entirely on whether the category is ordinal or nominal. Applying the wrong method introduces either a false arithmetic relationship (Label Encoding on nominal data) or unnecessary dimensionality (One-Hot Encoding on ordinal data with many levels).

The W8 dataset applies both methods: LabelEncoder for level (ordinal — with its alphabetical-order quirk checked via le.classes_) and pd.get_dummies() for city (nominal). Using Label Encoding for city would imply an arbitrary alphabetical ranking between cities — a completely false ordering that would cause any linear or distance-based model to learn an incorrect geographic relationship.

When One-Hot Encoding a column with N categories, the N resulting columns are mathematically redundant: knowing the values of N−1 columns always determines the Nth. If the city column were already standardised to 3 clean categories, then knowing city_Cairo=0 and city_Alexandria=0 would be enough to know city_Giza=1 — three columns would carry only two independent pieces of information.

This is the dummy variable trap — including all N columns introduces perfect multicollinearity in linear models, making coefficient estimation mathematically unstable. The solution is to drop one dummy column using drop_first=True.

# Avoid dummy variable trap — drop the first category column
df_encoded = pd.get_dummies(df, columns=['city'], drop_first=True)

print("Columns after drop_first:", [c for c in df_encoded.columns if 'city' in c])

With drop_first=True, the alphabetically-first category column (city_ALEX) is dropped, leaving 11 of the original 12 columns. The same logic applies regardless of how many distinct spellings exist: for a row where every remaining dummy column is 0, the model infers it belongs to the dropped reference category. This example also shows concretely why standardising city in Topic 8.7 first matters — applying drop_first to 12 unstandardised spelling variants still leaves 11 redundant columns, instead of the 2 you'd get from 3 clean categories.

In a real project, encoding is applied after all cleaning and standardisation steps are complete. The W8 dataset has two columns requiring encoding: level (ordinal — manual map) and city (nominal — get_dummies). The full pipeline below applies both in sequence and shows the final encoded DataFrame.

import pandas as pd
from sklearn.preprocessing import LabelEncoder

df = pd.read_csv('../Datasets/8_4_competition_results.csv')

print(f"Shape before encoding: {df.shape}")
print(f"Columns: {df.columns.tolist()}")

# Step 1: Label Encoding for level (ordinal)
le = LabelEncoder()
df['level_encoded'] = le.fit_transform(df['level'])

# Step 2: One-Hot Encoding for city (nominal), with drop_first for linear models
df_encoded = pd.get_dummies(df, columns=['city'], drop_first=True)

print(f"\nShape after encoding: {df_encoded.shape}")
print(f"Columns: {df_encoded.columns.tolist()}")

This combines both Scenes from the notebook in sequence: LabelEncoder turns level into level_encoded (verify the order with le.classes_ before trusting it), and get_dummies(..., drop_first=True) expands city into one boolean column per spelling variant, minus one reference column. Because the city column here is still the original, unstandardised text, this produces many more dummy columns than the 2 you'd get after Topic 8.7's cleanup — a concrete illustration of why standardisation should happen before encoding in a real pipeline.

# Verify the encoding is correct
print("Level classes (alphabetical order):", le.classes_)
print(df_encoded[['level', 'level_encoded']].drop_duplicates().sort_values('level_encoded'))

print("\nCity dummy columns created:")
print([c for c in df_encoded.columns if 'city' in c])