What you'll build

A small CLI, ask.py, that reads prompts from a text file, sends each one to a chat completions endpoint using raw HTTP, and writes answers plus timing data to JSON lines. It reads secrets only from the environment, handles authentication errors, rate limits and timeouts the way a production script should, logs to a file, and comes with offline tests and a shell one-liner report. You will also profile a run and explain where the time actually goes.

Requirements

1. Secrets. Configuration comes from environment variables LLM_BASE_URL, LLM_MODEL and LLM_API_KEY, optionally loaded from a .env file that is gitignored. Commit a .env.example with placeholder values only. The key is never printed or written to logs (log only whether it is set).
2. Raw HTTP. Calls go to {LLM_BASE_URL}/chat/completions using urllib.request or requests, with an explicit timeout. No vendor SDK is required.
3. Batch CLI. python ask.py --prompts prompts.txt --out results/answers.jsonl processes at least 10 prompts you wrote yourself about a topic you care about. Each output line contains: prompt id, status (ok or error), HTTP status code, attempts, latency in ms, the answer text, and token counts when the response includes a usage field.
4. Error handling. 401 or 403 prints a clear "check your API key" message and exits with code 2 without retrying. 429, 5xx and timeouts are retried with exponential backoff (for example 1s, 2s, 4s) up to 3 attempts. Any other failure is logged and the batch continues.
5. Logging. Use the logging module with timestamps and levels, writing to both the console and logs/run.log.
6. Offline tests. tests/test_client.py replaces the HTTP call with a fake function and checks three cases without any network: 429 then 200 succeeds on attempt 2, 401 stops immediately, and a timeout on every attempt ends in an error record after 3 attempts. Runnable with python -m pytest or plain python tests/test_client.py.
7. Shell report. scripts/summarize.sh uses grep, awk, sort or jq over results/answers.jsonl to print total calls, error count, mean latency and max latency.
8. Profiling. Run the batch once under python -m cProfile -s cumtime and save the top 15 lines to results/profile.txt. Use a Timer context manager to report network time vs JSON parsing and file writing time for the run.
9. Lint. ruff check . passes with no errors.

Constraints

• Any OpenAI-compatible endpoint is fine: a local model served by Ollama or LM Studio (free, runs on a laptop CPU with a small model), the KatalockAi API, or any provider's free tier. No paid key is required; say in the README which one you used.
• CPU only, ordinary laptop, Python 3.11+ in a project virtual environment.
• Keep the batch small (10 to 30 prompts) and short answers (set max_tokens around 200) so a local CPU model finishes in a few minutes.

Stretch goals

• Run the batch in a tmux session or with nohup ... &, then watch logs/run.log with tail -f from another pane.
• Add a --concurrency flag with a thread pool and compare total wall time.
• Use tracemalloc to report peak memory of a 100-prompt batch.

How to submit

Public GitHub repo with README.md, ask.py (or src/), .env.example, prompts.txt, tests/test_client.py, scripts/summarize.sh, results/answers.jsonl, results/profile.txt, and a pyproject.toml or requirements.txt. The README must give setup and run commands, name the endpoint and model you used, show your summary numbers, explain what the profile shows about where time goes, and describe one bug you hit while building it and how you found it. Never commit a real key.