Files and JSON¶
In 1 minute
- Everything in a running program lives in memory, and memory is wiped when the program ends. To keep data, write it to a file on the disk.
with open("data/orders.log", "w", encoding="utf-8") as f:opens a file and closes it for you."w"writes (and empties the file first),"a"adds to the end, no mode reads.- JSON is a text format for data: lists, dicts, numbers, text,
true/false,null. Almost every web API speaks it. json.dump(data, f)writes Python data as JSON;json.load(f)reads it back as Python data.- Expect problems: a missing file raises
FileNotFoundError, a broken onejson.JSONDecodeError. Catch them (Lesson 6).
Previously in Bazaar: Bazaar learned to handle bad input with
tryandexceptinstead of crashing.
1. Why learn this?¶
Run any Bazaar program from this module, and it builds a catalog and carts. Then the program ends, and everything is gone. The next run starts from nothing. Asha's cart, today's orders, the stock counts: all lost.
A shop that forgets every sale overnight can't work. Bazaar needs to save its data and load it again. In Level 2 that will be a database. Today, it's the simplest kind of storage there is: files. And the format: JSON, which you'll use in every API you ever build.
2. The concept¶
Memory vs disk¶
Remember Module 1: RAM (memory) is fast but forgets everything when the program stops. The disk is slower but keeps files until you delete them.
| Memory (variables) | Disk (files) | |
|---|---|---|
| Where | inside the running program | in a folder, like data/ |
| Survives the program ending? | no | yes |
| How Python uses it | catalog = [...] |
open(...), json.dump(...) |
Opening a file: with open(...)¶
"""Write Bazaar's order log to a text file, add to it, then read it back."""
# "w" = write: create the file, or EMPTY it if it exists
with open("data/orders.log", "w", encoding="utf-8") as f:
f.write("Asha bought 2 Masala Box for ₹598\n") # \n ends the line
f.write("Ravi bought 1 Cotton Kurta for ₹649\n")
# "a" = append: add at the end, keep what's there
with open("data/orders.log", "a", encoding="utf-8") as f:
f.write("Priya bought 1 Steel Water Bottle for ₹349\n")
# no mode = "r", read
with open("data/orders.log", encoding="utf-8") as f:
for line in f: # a file gives you one line per round
print(line.rstrip("\n")) # rstrip: drop the line's own \n
Asha bought 2 Masala Box for ₹598
Ravi bought 1 Cotton Kurta for ₹649
Priya bought 1 Steel Water Bottle for ₹349
open(path, mode, encoding="utf-8")opens the file.as fgives it a name for the indented block.withcloses the file automatically at the end of the block, even if an error happens. Always use it.encoding="utf-8"tells Python how to store letters as bytes. UTF-8 handles₹, Hindi, and every other script. Always pass it.f.write(text)doesn't add a new line by itself: end each line with\n.
| Mode | Means | If the file exists |
|---|---|---|
"r" (or none) |
read | read it |
"w" |
write | empty it first |
"a" |
append | add at the end |
JSON: data as text¶
A file holds text (or bytes), not Python lists and dicts. To save a dict, you must turn it into text first, in a format that can be turned back. JSON (JavaScript Object Notation) is that format. It looks almost like Python:
"""JSON is text. json.dumps turns Python data into JSON text; json.loads turns it back."""
import json
cart = {"customer": "Ravi", "items": [("Cotton Kurta", 1)], "coupon": None, "gift": True}
text = json.dumps(cart) # dumps = dump to a String
print(text)
print(type(text).__name__)
back = json.loads(text) # loads = load from a String
print(back) # the tuple came back as a list: JSON has no tuples
{"customer": "Ravi", "items": [["Cotton Kurta", 1]], "coupon": null, "gift": true}
str
{'customer': 'Ravi', 'items': [['Cotton Kurta', 1]], 'coupon': None, 'gift': True}
The small differences:
| Python | JSON |
|---|---|
{"name": "Masala Box"} dict |
{"name": "Masala Box"} object |
[1, 2] list (and tuples) |
[1, 2] array |
True / False / None |
true / false / null |
'text' or "text" |
only "text": double quotes |
json.dumps (dump to a string) and json.loads (load from a string) work with text in memory. That's exactly what an API does:
Bazaar's backend will dumps the catalog and send the text over the network (Module 7).
Saving and loading the catalog¶
To work with files, use json.dump and json.load (no s):
"""Save the catalog as JSON, then load it back: the data survives the program ending."""
import json
catalog = [
{"name": "Masala Box", "price_paise": 29900, "seller": "Patel Spices", "in_stock": True},
{"name": "Wireless Earbuds", "price_paise": 149900, "seller": "Sharma Electronics", "in_stock": False},
]
with open("data/catalog.json", "w", encoding="utf-8") as f:
json.dump(catalog, f, indent=2, ensure_ascii=False) # indent=2: easy to read; ensure_ascii=False: keep ₹ etc. as is
with open("data/catalog.json", encoding="utf-8") as f:
loaded = json.load(f)
print(loaded[1]["name"], loaded[1]["in_stock"])
print("same data?", loaded == catalog)
And this is what's now on the disk, in data/catalog.json. Open it in VS Code:
[
{
"name": "Masala Box",
"price_paise": 29900,
"seller": "Patel Spices",
"in_stock": true
},
{
"name": "Wireless Earbuds",
"price_paise": 149900,
"seller": "Sharma Electronics",
"in_stock": false
}
]
indent=2 makes it readable for people. Without it, JSON is one long line: just as correct, but hard to read.
Loading safely¶
Two things go wrong all the time: the file isn't there yet (a new customer has no saved cart), or it's broken (a save stopped halfway).
"""Loading a cart safely: a missing file means an empty cart; a broken file gets a clear message."""
import json
def load_cart(path):
try:
with open(path, encoding="utf-8") as f:
return json.load(f)
except FileNotFoundError: # first visit: no saved cart yet
return {"items": []}
except json.JSONDecodeError as error: # the file isn't valid JSON
print(f"{path} is broken ({error}); starting with an empty cart")
return {"items": []}
with open("data/broken.json", "w", encoding="utf-8") as f:
f.write('{"items": [') # half a file, as if the disk filled up mid-save
print(load_cart("data/no_such_cart.json"))
print(load_cart("data/broken.json"))
{'items': []}
data/broken.json is broken (Expecting value: line 1 column 12 (char 11)); starting with an empty cart
{'items': []}
Exactly Lesson 6's pattern: a small try, one except per expected problem, and a sensible answer for each.
An analogy¶
Memory is the whiteboard at the shop counter: quick to write on, wiped every night. A file is the register book in the cupboard: slower to write, but it's still there tomorrow.
JSON is the standard form everyone agrees on. Any shop (any program, in any language) can read a form filled in this format.
Questions you may have¶
Where is the file? The path data/orders.log is relative: it starts from the folder you're in when you run the program (Module 1). In the lab, that's /lab.
Why not just print into a file? You can (print(..., file=f)), but json.dump is what makes data loadable again.
Is a JSON file a database? For one small program, it's enough. With many customers buying at the same moment, two saves can clash. That's one reason Level 2 brings in PostgreSQL.
3. See it in action¶
Pick a situation, then press Play.
4. Code example¶
The lab's exercise saves Asha's cart and loads it back. Loading crashes:
"""Save Asha's cart to data/asha_cart.json, then load it back and print the total.
BUG: loading crashes with a JSONDecodeError.
Your task: look at how the file is written (open data/asha_cart.json in VS Code), and write real JSON instead."""
import json
cart = {"customer": "Asha", "items": [
{"name": "Masala Box", "price_paise": 29900, "quantity": 2},
{"name": "Cotton Kurta", "price_paise": 64900, "quantity": 1},
]}
with open("data/asha_cart.json", "w", encoding="utf-8") as f:
f.write(str(cart))
with open("data/asha_cart.json", encoding="utf-8") as f:
loaded = json.load(f)
total = sum(item["price_paise"] * item["quantity"] for item in loaded["items"])
print(f"{loaded['customer']}'s cart: ₹{total // 100:,}")
The traceback is long, because the error happened deep inside Python's own json code. Here are its first and last lines:
Traceback (most recent call last):
File "/lab/exercises/save_cart.py", line 16, in <module>
loaded = json.load(f)
(… lines from Python's own json code skipped …)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
(The lines in the middle, from /usr/local/lib/python3.13/json/, are Python's own files. Skip them: look for your file's line, and the last line.)
Now open data/asha_cart.json. str(cart) wrote Python's way of showing a dict, with single quotes:
{'customer': 'Asha', 'items': [{'name': 'Masala Box', 'price_paise': 29900, 'quantity': 2}, {'name': 'Cotton Kurta', 'price_paise': 64900, 'quantity': 1}]}
JSON needs double quotes, so json.load stops at character 1, the first '. The fix is one line:
with open("data/asha_cart.json", "w", encoding="utf-8") as f:
json.dump(cart, f, indent=2) # real JSON, not Python's own text for a dict
Write with json.dump, read with json.load. Never build JSON by hand with str() or string joining.
5. Under the hood¶
Things JSON can't hold¶
Traceback (most recent call last):
File "/lab/mistakes/set_to_json.py", line 4, in <module>
print(json.dumps(sellers))
~~~~~~~~~~^^^^^^^^^
(… lines from Python's own json code skipped …)
TypeError: Object of type set is not JSON serializable
JSON has no sets (and no Decimal, no dates). Convert first: json.dumps(sorted(sellers)) saves a list. Serializable means "can be turned into text".
A file that isn't there¶
Traceback (most recent call last):
File "/lab/mistakes/missing_file.py", line 1, in <module>
with open("data/old_orders.log", encoding="utf-8") as f:
~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'data/old_orders.log'
"w" empties the file, silently¶
"""Opening with "w" EMPTIES the file first. Reading it afterwards finds nothing. No error, just lost data."""
with open("data/orders.log", "w", encoding="utf-8") as f:
pass # meant to read, opened for writing by mistake
with open("data/orders.log", encoding="utf-8") as f:
print(repr(f.read()))
No error. The order log is simply gone, the moment open(..., "w") ran. To read, open with no mode (or "r"). To add, use "a".
Going deeper: pathlib, ensure_ascii, and saving safely
"""Going deeper: pathlib, ensure_ascii, and saving safely (write a temp file, then swap it in)."""
import json
import os
from pathlib import Path
# pathlib: paths as objects; read_text/write_text open and close the file for you
log = Path("data") / "deeper.log" # / joins path parts on every OS
log.write_text("Masala Box restocked: 40\n", encoding="utf-8")
print(log.name, log.exists(), log.read_text(encoding="utf-8").strip())
# ensure_ascii: by default JSON writes ₹ as ₹ (still correct, just hard to read)
print(json.dumps("₹299"), json.dumps("₹299", ensure_ascii=False))
# Safe save: if the program dies halfway, the old file is still complete
cart = {"customer": "Asha", "items": [{"name": "Masala Box", "quantity": 2}]}
target = Path("data/asha_cart_safe.json")
temp = target.with_suffix(".tmp")
temp.write_text(json.dumps(cart, indent=2), encoding="utf-8")
os.replace(temp, target) # one step: the new file replaces the old one
print(target.name, json.loads(target.read_text(encoding="utf-8"))["customer"], temp.exists())
pathlib.Pathtreats paths as objects:Path("data") / "deeper.log"works on Linux, Mac and Windows..read_text()and.write_text()open and close for you.- By default
json.dumpswrites non-English characters as codes like₹(that's₹). Both are valid JSON;ensure_ascii=Falsekeeps them readable. - Safe save: if the program crashes halfway through writing, the file is left broken (like
broken.json). Writing to a temporary file and thenos.replace()-ing it over the real one means the real file is always either the old version or the new one, never half of each.
Going deeper: other formats you'll meet
| Format | Looks like | Used for |
|---|---|---|
| CSV | Masala Box,29900,Patel Spices |
spreadsheets, seller bulk uploads (Python's csv module) |
| JSON Lines | one JSON object per line | big logs and exports, read line by line |
| YAML / TOML | name: Masala Box |
config files (Docker Compose uses YAML) |
| Binary (pickle) | not readable text | Python-only; never load a pickle you didn't make: it can run code |
Big files: for line in f: reads one line at a time, so even a 10 GB log never has to fit in memory. f.read() loads the whole file at once.
6. Lab¶
Lab folder: labs/04-python-fundamentals/07-files-and-json/
The programs write into the lab's data/ folder, so you can open every file they make in VS Code. Write and read the order log, save the catalog
as JSON and look at the file, load a missing and a broken cart safely, meet three file mistakes, and fix Asha's cart so it saves real JSON.
When you finish, run bash test_lab.sh (from your computer). You should see ALL CHECKS PASSED.
7. Common mistakes¶
- Opening with
"w"to read. The file is emptied before you read a byte. Fix: no mode (read) or"a"(add). - Saving with
str(data). Single quotes,True,None: not JSON, andjson.loadfails. Fix:json.dump(data, f). - Forgetting
encoding="utf-8". On some computers,₹becomes garbage or an error. Fix: always pass it. - Not handling a missing or broken file. The first new customer crashes the shop. Fix: catch
FileNotFoundErrorandjson.JSONDecodeError. - Running from the wrong folder.
data/...is relative, soFileNotFoundError. Fix: run from the lab folder (/lab), or build paths withpathlib.
8. Quiz¶
9. What's next¶
Next: splitting Bazaar into files that import each other, and building Bazaar v0, a command-line catalog: Modules, and Bazaar v0.
In one line: memory forgets and files remember; open files with with open(path, mode, encoding="utf-8"); save data as JSON with json.dump,
read it with json.load, and handle FileNotFoundError and JSONDecodeError.
Cheat sheet¶
| Task | Code |
|---|---|
| Write text (empties the file!) | with open("data/x.log", "w", encoding="utf-8") as f: f.write("…\n") |
| Add to the end | open(path, "a", encoding="utf-8") |
| Read line by line | with open(path, encoding="utf-8") as f: for line in f: … |
| Save data as JSON | json.dump(data, f, indent=2, ensure_ascii=False) |
| Load JSON | data = json.load(f) |
| Data ↔ JSON text (no file) | json.dumps(data) / json.loads(text) |
| No file yet | except FileNotFoundError: |
| Broken JSON | except json.JSONDecodeError: |
Next in Bazaar: Bazaar's code is spread across many little files, each copying
rupees()and the catalog. Next lesson: modules, so one file canimportanother, and then all of Module 4 comes together in Bazaar v0.