Data & APIs Lab

JSON (JavaScript Object Notation) is a lightweight data format for exchanging structured data. It's used by virtually every REST API.

{
  "name": "Alice",
  "age": 30,
  "active": true,
  "skills": ["Python", "Bash", "SQL"],
  "address": {
    "city": "Denver",
    "state": "CO"
  }
}

Data types: string ("text"), number (42, 3.14), boolean (true/false), null, array ([]), object ({}).

Rules: Keys must be double-quoted strings. No trailing commas. No comments.

YAML (YAML Ain't Markup Language) is a human-readable data format used for config files (Ansible, Docker Compose, Kubernetes).

# YAML equivalent of the JSON above
name: Alice
age: 30
active: true
skills:
  - Python
  - Bash
  - SQL
address:
  city: Denver
  state: CO

Key differences from JSON: Uses indentation (spaces, not tabs). Supports comments with #. No braces or brackets needed. Strings usually don't need quotes.

REST (Representational State Transfer) is an architectural style for APIs using standard HTTP methods.

# HTTP Methods
GET    /api/users        # Read all users
GET    /api/users/42     # Read one user
POST   /api/users        # Create a user
PUT    /api/users/42     # Replace a user
PATCH  /api/users/42     # Partial update
DELETE /api/users/42     # Delete a user

Status codes:

200 OK            # Success
201 Created       # Resource created (POST)
204 No Content    # Success, no body (DELETE)
400 Bad Request   # Invalid input
401 Unauthorized  # Auth required
403 Forbidden     # Auth OK but no permission
404 Not Found     # Resource doesn't exist
500 Internal Error # Server failure

curl is the universal CLI tool for making HTTP requests.

# GET request
curl https://api.example.com/users

# GET with headers
curl -H "Authorization: Bearer TOKEN" https://api.example.com/me

# POST with JSON body
curl -X POST https://api.example.com/users \
  -H "Content-Type: application/json" \
  -d '{"name": "Alice", "age": 30}'

# PUT (update)
curl -X PUT https://api.example.com/users/42 \
  -H "Content-Type: application/json" \
  -d '{"name": "Alice", "age": 31}'

# DELETE
curl -X DELETE https://api.example.com/users/42

# Useful flags
curl -s    # Silent (no progress bar)
curl -v    # Verbose (show headers)
curl -o file.json  # Save output to file
curl -I    # HEAD request (headers only)

jq is the standard CLI tool for parsing and transforming JSON.

# Pretty-print
echo '{"a":1}' | jq .

# Extract a field
echo '{"name":"Alice","age":30}' | jq '.name'    # "Alice"
echo '{"name":"Alice","age":30}' | jq -r '.name'  # Alice (raw)

# Array operations
echo '[1,2,3,4,5]' | jq '.[0]'          # 1 (first element)
echo '[1,2,3,4,5]' | jq '.[-1]'         # 5 (last element)
echo '[1,2,3,4,5]' | jq '.[1:3]'        # [2,3] (slice)
echo '[1,2,3,4,5]' | jq 'length'        # 5

# Filter objects
cat data.json | jq '.[] | select(.age > 25)'
cat data.json | jq '[.[] | select(.language == "Sindhi")]'

# Transform
cat data.json | jq '.[] | {name, id}'   # pick fields
cat data.json | jq 'map(.name)'         # extract names
cat data.json | jq 'sort_by(.version)'  # sort
cat data.json | jq 'group_by(.language)' # group

# GET request
$response = Invoke-RestMethod -Uri "https://api.example.com/users"

# POST with body
$body = @{ name = "Alice"; age = 30 } | ConvertTo-Json
Invoke-RestMethod -Uri "https://api.example.com/users" `
  -Method POST -Body $body -ContentType "application/json"

# JSON parsing
$json = '{"name":"Alice","age":30}' | ConvertFrom-Json
$json.name    # Alice

# Convert back to JSON
$json | ConvertTo-Json -Depth 10

# Read JSON file
$data = Get-Content "data.json" -Raw | ConvertFrom-Json
$data | Where-Object { $_.language -eq "Sindhi" }
$data | Sort-Object version | Select-Object name, version

import requests
import json

# GET request
r = requests.get("https://api.example.com/users")
data = r.json()           # parse JSON response
print(r.status_code)      # 200

# POST with JSON
payload = {"name": "Alice", "age": 30}
r = requests.post("https://api.example.com/users", json=payload)

# Headers
headers = {"Authorization": "Bearer TOKEN"}
r = requests.get("https://api.example.com/me", headers=headers)

# JSON module
import json
text = '{"name": "Alice", "age": 30}'
obj = json.loads(text)     # parse string -> dict
back = json.dumps(obj, indent=2)  # dict -> pretty string

# Read/write JSON files
with open("data.json") as f:
    data = json.load(f)
with open("output.json", "w") as f:
    json.dump(data, f, indent=2)

Regular expressions match patterns in text. Used in grep, sed, Python, PowerShell, and API validation.

# Metacharacters
.     # Any character (except newline)
\d    # Digit [0-9]
\w    # Word character [a-zA-Z0-9_]
\s    # Whitespace
^     # Start of string
$     # End of string
\b    # Word boundary

# Quantifiers
*     # 0 or more
+     # 1 or more
?     # 0 or 1
{3}   # Exactly 3
{2,5} # Between 2 and 5

# Character classes
[abc]     # a, b, or c
[a-z]     # lowercase letter
[^0-9]    # NOT a digit

# Groups and alternation
(abc)     # Group "abc"
a|b       # a OR b

# Common patterns
\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}   # IPv4 (basic)
^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$  # Email
^https?://                                           # URL start

Usage:

# grep
grep -E '\d{3}-\d{4}' contacts.txt

# Python
import re
re.findall(r'\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b', text)

# PowerShell
"test@example.com" -match '^[^@]+@[^@]+\.[^@]+$'  # True
[regex]::Matches($text, '\d+')

# sed
sed -E 's/[0-9]{3}-[0-9]{4}/XXX-XXXX/g' file.txt