← Python for Data Analysis

PDA·01 · Chapter 1

Preliminaries

Chapter 1 — what the book is for, why Python, and getting set up

Five short sessions to set the scene: what kind of data the book deals with, why Python won, the seven libraries everything else builds on, how to install them today, and the conventions every later chapter assumes.

1.1

What Is This Book About?

Concept

  • The book is about the nuts and bolts of manipulating, processing, cleaning and crunching data in Python — not about statistics or machine learning theory.
  • By "data" it means structured data: tables where each column may be a different type (a spreadsheet, a CSV, a SQL table), multidimensional arrays, multiple tables linked by key columns, and time series.
  • Most of the real work in analysis is getting data into that shape — McKinney calls it data wrangling or munging, and it is what the tools in this book are built for.
  • Almost any dataset can be turned into a structured form: a folder of news articles becomes a table of word frequencies, a web log becomes a table of timestamped events.

Key Terms

Structured data
data in rows and columns (or arrays) with a consistent shape
Tabular data
rows of records, each column a named field with its own type
Data wrangling
cleaning, reshaping and combining raw data until it can be analysed
Time series
observations recorded at points in time, evenly or unevenly spaced

Worked example

import pandas as pd

# one table: each row a record, each column a typed field
scores = pd.DataFrame({
    "student": ["Ploy", "Ton", "Mint"],
    "grade": [8, 8, 9],
    "score": [72.5, 88.0, 91.5],
})
print(scores)
  student  grade  score
0    Ploy      8   72.5
1     Ton      8   88.0
2    Mint      9   91.5

Exercise

Activity 1

List three datasets you use outside of programming (a class register, a phone bill, a sports table). For each, name its rows and three of its columns.

Activity 2

Which of the four kinds of structured data is each one: tabular, multidimensional array, multiple linked tables, or time series? Some fit more than one.