Call Me Maybe

Call Me Maybe is a milestone M3 Python project about function calling with a small language model. Given a list of function definitions and a list of natural-language prompts, your program must decide, for each prompt, which function to call and with which typed arguments, and write the result as JSON, without executing anything. The model is Qwen3-0.6B, too small to produce valid JSON reliably on its own, so the heart of the project is constrained decoding: you steer generation token by token so the output is always valid. Details below come from student repositories (Reported by students); check them against your subject.

What you will learn

• How a causal language model generates text: tokens, logits, greedy decoding
• Tokenisation with byte pair encoding and why token boundaries matter
• Constrained decoding: masking logits so only tokens that keep the output valid can be chosen
• JSON Schema thinking: names, types, required fields
• Validating inputs and outputs with Pydantic
• Prompt design with few-shot examples
• Measuring accuracy and speed on a CPU

Before you start

• Previous nodes: Python Modules (Pydantic from module 09 is used directly) and A-Maze-ing.
• Tools: Python 3.10+, uv (reported as the expected runner), numpy, pydantic, flake8, mypy.
• The subject provides an llm_sdk folder that loads Qwen3-0.6B and exposes encoding, decoding, the vocabulary file and logits for a sequence of token ids (Reported by students). There is no key-value cache, so every token costs a full forward pass.
• Reported input files: data/input/functions_definition.json and data/input/function_calling_tests.json; output defaults to data/output/function_calls.json.

Plan of attack

1. Day 1: read the SDK, load the model, encode and decode a few strings, and print the top tokens for a prompt. Build a table from token id to decoded string from the vocabulary file.
2. Day 1: parse the input files with Pydantic models; fail clearly on malformed definitions.
3. Day 2: write the output skeleton yourself: the braces, keys and quotes are forced, so the model only fills the function name and argument values.
4. Day 2: constrain the function name: at each step, keep only tokens that extend a prefix of one of the known names.
5. Day 3: constrain arguments by type: digits and sign for integers, a decimal point for floats, a closing quote rule for strings, true or false for booleans.
6. Day 3: build a prompt with the function list and two or three examples. Students report that few-shot context makes a large difference on function selection.
7. Day 4: run the public prompts, compare with expected calls, and time the full run.
8. Day 5: Makefile (install, run, debug, clean, lint, lint-strict), CLI flags for the input and output paths, README with the AI usage section.

import numpy as np

def pick_token(logits: np.ndarray, allowed: list[int]) -> int:
    mask = np.full_like(logits, -np.inf)
    mask[allowed] = 0.0
    return int(np.argmax(logits + mask))

Common pitfalls

• Tokens that contain several characters, including a closing quote plus a comma: check the whole token against your grammar, not only its first character.
• Escaped characters inside strings breaking your parser state; re-derive the state from the decoded text at each step.
• Producing 5.0 when the schema says integer, or "5" when it says number.
• Forgetting to disable the model's thinking mode, which wastes tokens.
• A run so slow it exceeds the time limit (students report a few minutes for the full set).
• mypy strict errors around numpy arrays: annotate with numpy.typing.

How it is evaluated

• Peer evaluation, plus an automatic check of the output file (Reported by students): function name, argument types and exact values for each prompt, on a public set and a hidden set.
• The evaluator runs make lint and make lint-strict, then your program on the public prompts.
• You must explain constrained decoding, how you build the allowed token set, and why prompting alone is not enough.
• No bonus is clearly identified in student repositories.

Resources

• Qwen3-0.6B model card
• Hugging Face: summary of the tokenizers
• Pydantic documentation
• JSON Schema: getting started
• uv documentation
• Student repo: barraRRR call-me-maybe