One loading idea
pandas loads tabular data into a DataFrame. A DataFrame is the table you will inspect, clean, join, group, chart, and export later.
Use read_* functions to load files, such as read_csv() and read_json(). Use to_* methods later when you write cleaned output.
Pick raw files
For your final HackIndia repo, keep raw files unchanged. Put them in a data/raw folder so anyone can rerun your notebook or script.
Good practice is to load at least two files. Example: orders.csv and customers.json for a small UPI payments, canteen, fest, or placement dataset.
Load a CSV
read_csv() reads a CSV path and returns a DataFrame. pandas detects common missing-value markers by default, so blanks often become missing values automatically.
Do not clean inside the raw file. Load first, then clean in code in later lessons.
import pandas as pd
orders = pd.read_csv("data/raw/orders.csv")
print(orders.head())
print(orders.shape)
print(orders.columns)Load normal JSON
JSON can be shaped in different ways. A common beginner-friendly shape is a list of records, where each record becomes one row.
If AI generates sample JSON for you, check that it is valid JSON, uses double quotes, and has the same keys in most records.
[
{"customer_id": 1, "name": "Asha", "city": "Pune"},
{"customer_id": 2, "name": "Rahul", "city": "Jaipur"}
]
import pandas as pd
customers = pd.read_json("data/raw/customers.json")
print(customers.head())
print(customers.shape)
print(customers.columns)Load JSON lines
Some APIs and data exports store one JSON object per line. This is called line-delimited JSON or JSON Lines.
For this format, pass lines=True. If you also use chunksize, pandas returns an iterator instead of loading the whole file at once.
{"order_id": 101, "amount": 249, "status": "paid"}
{"order_id": 102, "amount": 99, "status": "failed"}
import pandas as pd
orders_log = pd.read_json("data/raw/orders_log.jsonl", lines=True)
for chunk in pd.read_json("data/raw/orders_log.jsonl", lines=True, chunksize=1000):
print(chunk.shape)Common loading mistakes
Do not assume every JSON file loads the same way. If a file has one object per line, use lines=True. If it is one list of records, do not use lines=True.
Older tutorials may show string columns as object dtype. In pandas 3.0, string data is inferred as a dedicated str dtype by default, so do not panic if your dtype output differs.
Keep loading boring
Create one small loading script before you start cleaning. It should load each file and print head, shape, and columns.
This gives reviewers confidence that your project runs. It also helps you catch wrong paths before you waste time debugging cleaning code.
import pandas as pd
orders = pd.read_csv("data/raw/orders.csv")
customers = pd.read_json("data/raw/customers.json")
print("orders", orders.shape, list(orders.columns))
print(orders.head())
print("customers", customers.shape, list(customers.columns))
print(customers.head())Do this now
Do each one yourself, then tap it to tick it off. The ticks are only a checklist for you: they are not marked or scored.
0 of 5 done