This book is built from data pulled directly from the Neon CRM API. Every figure is reproducible: data is extracted, cleaned, geocoded, and summarized by the hh package, then read from saved parquet tables in data/20_processed/. All numbers in the prose are computed from the data at render time, so they stay correct as the data is refreshed. Every API pull is recorded in a manifest (endpoint, filters, timestamps, row counts, file hashes, code commit), and raw extracts are immutable — see the project README for the full provenance story.
All published charts and tables are aggregates (counts and dollar totals by category, distance band, and day of week) — no individual is identified.
A.2 About the data
The raw data is pulled from the Neon CRM REST API (v2) and saved exactly as Neon returns it. There are four entities, and every record retrieved streams straight to disk — analysis never depends on the live API.
Accounts — every constituent: donors, ticket-buyers, and class-takers, as individuals and households. One row per Neon account. Carries identity (account id, account type, first / last / full / company name), household membership (household id and name), mailing address (street, city, state, ZIP, country), contact info (email, phone), and the deceased and do-not-contact flags used to filter people out.
Donations — every gift. One row per donation, tied to an account by account id. Carries amount, date, type (DONATION or PLEDGEPAYMENT), status (e.g. SUCCEEDED), and fund / campaign / purpose / source. It carries no household id; that link is recovered during cleaning.
Events — every scheduled event: a single performance, class, or community happening. One row per Neon event. Carries the event name, Neon’s event category, start and end dates, capacity, and Neon’s attendee count. There is no field linking the nights of a multi-night run; runs are reconstructed later.
Registrations — every ticket / registration purchase, linking an account to an event. One row per registration, with nested ticket and attendee records (used to count true headcount). Carries the registrant’s account id, amount, date-time, and source / origin.
Code
import pandas as pdfrom hh.io import load_raw_raw = pd.DataFrame( {"entity": ["accounts", "donations", "events", "registrations"],"what it is": ["Constituents — donors, patrons, class-takers (individuals & households)","Gifts, one per donation","Scheduled events — one performance, class, or community event","Ticket/registration purchases linking an account to an event", ], })_raw["records"] = [len(load_raw(e)) for e in _raw["entity"]]_raw[["entity", "records", "what it is"]]
entity
records
what it is
0
accounts
5166
Constituents — donors, patrons, class-takers (...
1
donations
9100
Gifts, one per donation
2
events
2605
Scheduled events — one performance, class, or ...
3
registrations
28057
Ticket/registration purchases linking an accou...
Record counts are computed from the latest saved pull at render time, so they stay correct as the data is refreshed. Format & storage. Each pull is written as newline-delimited JSON (.jsonl, one record per line) into an immutable, timestamped folder data/00_raw/neon/pull-<UTC>/, alongside a manifest.yaml that records the endpoint, filters, output fields, Neon-reported counts, timestamps, file SHA-256 hashes, and the code commit. Accounts, donations, and events are pulled in one pass (scripts/pull.py); registrations are gathered by a per-event sweep (scripts/sweep_registrations.py) because Neon serves them event-by-event. Downstream code always reads these saved files (hh.io.load_raw), never the live API.
A.3 Data cleaning
Cleaning turns the raw API records into standardized analytic tables (the hh.clean package). It is purely deterministic and re-runs from the saved raw files on every scripts/build.py, so the processed tables in data/20_processed/ can always be rebuilt.
Column standardization. Neon’s output-field names are mapped to short snake_case names. Each entity has an explicit field map (for example, Account ID → account_id, Donation Amount → donation_amount); any column not in the map is converted by vmap, a faithful port of the R project’s renaming function (hh.clean.standardize).
Type parsing. Dollar amounts are coerced to numeric and dates to datetimes, with malformed values turned into missing rather than crashing.
Flag conversion. The deceased and do-not-contact text fields ("Yes") become real booleans.
Household rollup. Every analytic record gets a canonical household id, name, and group (hh.clean.households.add_rollup): id is the household id when Neon assigns one, otherwise the account id; group is household, account, or other; name is the first non-empty of household name, full name, or company name.
Recovering links the API omits. Donations carry an account id but no household id, so household membership is recovered by joining the cleaned accounts on account id. Registrations likewise join accounts (for the household) and events (for the event’s category), and fall back to a swept event id when the registration’s own event id is missing.
Event categorization. Each event is classified into a major category (performance / class / community / other) and, for performances, a minor category (theater / music / dance / opera / other). The major scheme is Hubbard Hall’s own (2026-08), informed by the earlier R rules but rewritten against the full event inventory, case-insensitive, and tolerant of Neon’s category-name quirks (irregular spacing). Its principles, in order: Neon’s own event category is the primary signal; youth-program showcases and recitals count as classes even when filed under a performance category (they are the camp’s product, not mainstage programming); ticketed public programming — including film screenings — is a performance; community gatherings (breakfasts, dinners, forums, exhibits and lectures) are community; and galas, fundraisers, auditions, and uncategorized miscellany are other, except fundraisers that are essentially concerts (Brews & Blues, the cabaret nights), which are performances. An event that matches no rule is flagged ERROR on purpose, so anything uncategorized surfaces instead of hiding; a real-data test fails if the latest pull contains any. The minor rules remain ported from R for now. See Definitions for the categories themselves.
Event indicators. Beyond the exclusive major category, events carry boolean cross-cutting flags — is_fundraiser, is_youth_program, is_film — because one bucket cannot say “a performance that is also a fundraiser” (Brews & Blues) or “a class that is also a youth program” (the camps). The flags are independent name/category evidence and never change the major category: attendance rows stay summable by major category, and cross-cutting questions filter on the flags.
Manual overrides. Curated one-off corrections live in config/overrides.yaml, kept separate from code.
Geographic cleaning. Addresses are resolved to a point with a documented precision — a real street address first, then a household member’s street address for PO-box accounts, then a ZIP centroid as a last resort, and unmapped if nothing works (see Geography). PO-box-only households are flagged so the tight distance counts stay honest.
A.4 Definitions
How we select and define the people and events in this book. Every definition rolls a constituent up to their household first (see Data cleaning); person-level counts are at the household level.
Household. The analytic unit: the household id when Neon assigns one, otherwise the single account id. A person in a Neon household is counted once with their housemates; a standalone account is its own household.
Donor. A household whose total succeeded individual gifts are greater than zero. A gift counts when its status is SUCCEEDED, its type is DONATION or PLEDGEPAYMENT, and the account is an Individual; we then drop gifts from accounts marked deceased or do-not-contact, and a known junk account (36805). Variables used: donation_amount, donation_status, donation_type, account_type, deceased, do_not_contact.
Patron. A household with one or more performance registrations (is_patron = n_performance > 0).
Class-taker. A household with one or more class registrations (is_class_taker = n_class > 0).
Major category (performance / class / community / other). Assigned from Neon’s event category and the event name, in order, first match wins. For example, an event in a Performances, Theater Performances, Music Performances, or Opera Performances category is a performance (with a few named exceptions that are really classes); events in class or community categories fall to those buckets; anything unmatched is other. An event matching no rule at all is flagged ERROR as a tripwire.
Minor category (theater / music / dance / opera / other — performances only). Assigned after the major category, again first match wins; anything unmatched is other. Theater and opera lean on Neon’s category plus title patterns; music and dance use category plus title patterns.
Theater — category Shakespeare or Theater Performances, or a title matching a known theater production.
Opera — category Opera Performances, or the title contains “Opera”.
Music — a Music category, or the title matches music patterns (Concert, Cabaret, named artists, and the like).
Dance — a Dance category or title.
Attendance (headcount). A registration is a purchase, not a person; it averages about two attendees. Headcount counts ticket attendees whose registration status is SUCCEEDED (see Attendance headcount).
Production and run. A production is a title matched from the event name to a curated list; a run is that title’s performances split when dates gap by more than 45 days (see Reconstructing multi-day production runs).
A.5 Donor size and persistence
Annual giving tiers. A household’s giving in a single year is placed in one of four bands: <$100, $100–999, $1,000–4,999, or $5,000+. These drive the over-time and persistence chapters. A household is counted once per year it gives (so a multi-year donor appears in several years).
Lifetime tiers. Cumulative giving uses the same bands; households giving $1,000+ lifetime are “large donors” and $5,000+ are “major donors.” These drive the preferences chapter, where lifetime giving measures the depth of the relationship.
Donor basis. Size and persistence start from succeeded_individual_gifts (status SUCCEEDED, type DONATION/PLEDGEPAYMENT, account type Individual) rolled up to the household — the same basis as households_summary.total_donations, so the figures reconcile with the headline donor counts.
Retention. Of households that gave in year Y, the share that also gave in Y+1. The retention table drops the final year (it has no “next year”), and the trend drops the row whose next year is the partial current year. Retention is an upper bound on voluntary churn: a donor who has died also appears as “not retained.”
Partial years. The current calendar year is year-to-date and is left out of every over-time trend; where its figure is shown, it is labeled year-to-date and will rise as the year fills out.
A.6 FY26 annual appeal
The appeal chapter compares households that were mailed the fall 2025 annual appeal with those that were not.
Who was appealed comes from the development office’s appeal workbook (three sheets: segmented top prospects, general list, theater artists), which carries Neon account ids; recipients are mapped to households through the current Neon accounts. Subtotal rows, duplicate rows, and placeholder rows are dropped on load; a handful of recipients with no id at all are excluded from household-level rates.
Responded means at least one SUCCEEDED DONATION/PLEDGEPAYMENT gift to the campaign Annual Fund Drive - 2025-2026 dated 2025-10-01 through 2026-01-31, on any account in the household — the gift often arrives on a spouse’s account rather than the one that was mailed.
Prior giving is lifetime giving before October 1, 2025 (same donor basis and tiers as above; FY25 = July 2024 – June 2025).
Engagement before the appeal counts registrations for events that started by September 30, 2025 (event start date, not registration date), profiled as arts (performances), class, both, community-only, or none.
The chapter reports aggregates only. Household-level detail, staff notes, and Don’s stewardship notes stay in local-only outputs (a separate non-published analysis site and an Excel workbook under data/).
A.7 Geography
Distances are straight-line (haversine) from Hubbard Hall (25 E Main St, Cambridge NY) to each household, bucketed into half-open mile bands. Households whose only address is a PO box are placed at their ZIP’s centroid and flagged “approximate”; they are excluded from the tight 1-mile count so it stays honest.
A.8 Attendance headcount
Registration counts understate attendance: one registration is a purchase, and averages about two attendees. Where the text says headcount, it counts ticket attendees whose registration status is SUCCEEDED — this also automatically drops canceled and refunded attendees.
A.9 Reconstructing multi-day production runs
Each performance night is a separate Neon event (e.g. “Fun Home - Friday, February 3 at 7:30 pm”) — there is no field linking the nights of a run. Runs are therefore reconstructed by a manually curated mapping from event names to productions (hh.analytics.productions):
Mapping. Every theater event name matches exactly one production title pattern (coverage is exhaustive and tested). For music and opera, only titles with multiple performances are mapped; one-off concerts cannot form a run and are dropped.
Runs. A production’s performances are split into separate runs when consecutive dates are more than 45 days apart — this separates recurring titles (Winter Carnival of New Work, Whispering Bones, Bread & Puppet) into their yearly runs.
Exclusions. Events that were not regular public performances are dropped before mapping: canceled or moved dates, test events, open/dress/pay-what-you-will rehearsals, online productions, day-long special events, and weekday 10 am school shows.
Full-weekend runs. The within-run day-of-week analysis keeps only runs that played all three of Friday, Saturday, and Sunday, so day comparisons hold the production constant.
Time slots. A performance is classified a matinee if its name contains “2 pm” or “matinee”; otherwise evening.