Topic 6.6: Reading and Writing CSV Files

You've been creating DataFrames manually by typing data directly into code. In practice, data comes from external sources: files on your computer, databases, APIs, or shared drives. CSV files (Comma-Separated Values) are the standard format you'll encounter most often in data science.

A CSV file is a plain text file where each row represents a record, and values within each row are separated by commas. The first row typically contains column names. Pandas provides the read_csv() method that converts CSV files into DataFrames.

import pandas as pd

# Read the student scores CSV file
df = pd.read_csv('6_6_student_scores.csv')

# Display the first few rows
print(df.head())

The read_csv() method takes a file path as input. If the file is in the same directory as your notebook, you only need the filename. Otherwise, provide the complete path. The method detects column names from the first row, determines data types, and returns a DataFrame.

Once data is loaded, you need to understand its structure before analysis. Pandas provides several essential methods for this purpose.

# Check the DataFrame structure
print("DataFrame information:")
print(df.info())

# Get statistical summary of numeric columns
print("\nStatistical summary:")
print(df.describe())

The .info() method shows column names, counts of non-null values, and data types. This reveals missing data and type mismatches immediately. The .describe() method generates statistical summaries (mean, standard deviation, minimum, maximum, quartiles) for all numeric columns, showing how your data is distributed.

After modifying a DataFrame—adding columns, filtering data, or cleaning missing values—you often need to save the results. The to_csv() method writes the entire DataFrame to a CSV file, creating it if it doesn't exist or overwriting it if it does.

# Add calculated columns
df['Total'] = df['Math'] + df['Science'] + df['English'] + df['History']
df['Average'] = df['Total'] / 4

# Save to a new CSV file
df.to_csv('6_6_student_scores_updated.csv', index=False)
print("Data saved successfully")

By default, to_csv() saves the DataFrame's index as the first column in the CSV file. If your index consists of default numeric row labels (0, 1, 2, 3...), you don't need to save it—it clutters the file without adding value. Prevent this by setting index=False when calling the method.