Specialist teams across the post-training stack.
We field small, senior teams that plug directly into your workflows and tooling. We scope every project up front and manage it end to end.
Model & agent evaluations
Human evals and agent task suites run by calibrated raters.
- Multi-dimension rubrics: accuracy, sourcing, uncertainty
- Blind side-by-sides with error taxonomies that feed training
- Agent task suites: model builds, reconciliations, tie-outs
- Programmatic end-state checks for verifiable results
RLHF & preference data
Preference rankings from experts trained in comparative judgment.
- Rankings tuned to what a finance professional would sign off on
- Reward-model datasets with documented rater rationale
- Agreement tracking and drift checks throughout production
- Fast escalation paths when tasks fall outside rubric coverage
SFT & instruction data
Worked demonstrations from credentialed professionals.
- Expert-written completions with step-by-step reasoning traces
- Domain corpora built to spec: accounting treatments, modeling tasks, market analysis
- Two-layer review with senior sign-off before anything reaches your pipeline
- Format-flexible delivery matched to your ingestion tooling
Red-teaming & adversarial testing
Structured probing for failure modes that automated testing misses.
- Finance-specific threat models: hallucinated figures, unsupported claims, misapplied standards
- Compliance-sensitive scenario testing designed with practitioners
- Jailbreak and edge-case discovery with reproducible prompts
- Fixed-scope pre-launch packages with prioritized failure-mode reports
Recent engagements in finance and accounting.
A sample of projects, with details anonymized to protect client confidentiality.
Rubric architecture for financial reasoning evals
A scoring system that turned subjective grading into repeatable measurement.
Side-by-side capability evals across finance domains
Blind comparative grading across valuation, accounting, and markets.
Stress-testing an AI finance assistant
Probing for hallucinated figures and misapplied standards in production.
Gold-standard accounting datasets
Expert-written walkthroughs of the treatments models most often get wrong.
Preference data for analyst-grade output
Rankings tuned to the standard a professional would put their name on.
Task suites for spreadsheet agents
Realistic modeling and reconciliation tasks with verifiable end states.
Low risk to start. Built to scale.
Scope call
We define the tasks, rubric needs, volume, and tooling. You talk to the people who will run the work.
Calibration pilot
A short, paid, bounded task set. You judge the quality before committing to volume.
Production
A trained team ships against the rubric, with QA layers and throughput reporting as standard.
Scale or wind down
Expand the team, extend the rubric, or close the project cleanly. No lock-in.
Quality control you can audit.
Every engagement runs on the same QA spine, and its artifacts ship with the handoff: rubrics, gold sets, error taxonomies, and QA logs are yours.
Gold sets before production
Raters work a gold set built with your team before touching live tasks, and only those who clear it move on. Calibration is per-project, against your rubric. There's no generic badge.
Blind QA through every batch
Senior reviewers audit blind duplicates through live batches. Disagreements go to adjudication, and the outcomes feed back into rater guidance the same week.
Rubric change control
When live tasks fall outside rubric coverage, we log the ambiguity, a single owner amends the rubric, and the change propagates with re-anchoring, so drift is caught as it happens.
Frontier models are built on expert judgment.
S46 is built for the part of data work that determines whether your metrics move: sustained calibration, high agreement, and consistency deep into production.
Credentialed, then trained
Every specialist completes hands-on training in annotation tooling, rubric calibration, model comparison, and quality review before touching a client project. Domain expertise is the entry requirement, and data work is the discipline layered on top of it.
Selective by design
We know every specialist we place, and the network stays deliberately small. That selectivity is what keeps agreement rates high and staffing accountable, on the hundredth task as much as the first.
Managed end to end
S46 handles staffing, calibration, QA, and throughput reporting. You get a single point of contact and data you can trust, with no dashboard of freelancers to manage.
Ready when your roadmap moves
Specialists train before they are staffed, so we can scope a project and field a team in days, from a single senior reviewer to a full evaluation team, then flex as priorities shift.
Deep expertise across finance.
For us, finance and accounting is the entire practice rather than one vertical among twenty. Our specialists come from public accounting, banking, buy-side research, and corporate finance teams: selected for judgment, then trained for data work.
Common questions.
How do engagements start?
With a scope call and, usually, a short, paid calibration pilot: a bounded task set that shows you the quality bar before anything scales.
How do you handle confidentiality?
NDAs as standard, with flow-down agreements for every expert staffed. Work runs inside your tooling wherever possible, we don't retain client data beyond the engagement, and we conflict-screen every project.
How is pricing structured?
Scoped up front, per task or per hour depending on the work. No platform fees and no volume minimums. Rate card available on request.
How fast can a team start?
Experts train before they're staffed, so a team can typically start within days of the scope call, pilot included. Pre-launch red-team packages run against your ship date.
Why not hire a CPA contractor directly?
Domain knowledge alone doesn't produce training-grade data. That takes rubric design, calibration, and agreement measurement. We field finance professionals trained for exactly that.
Who owns what you build?
You do. Rubrics, gold sets, task suites, error taxonomies, and the data itself transfer to you at handoff. Your team can keep running them after we're gone.
For experts: work on frontier AI.
Paid project work on frontier AI systems for finance and accounting professionals. We provide the training, match you to projects in your domain, and handle the logistics. You bring the judgment.
- Remote and project-based, designed to fit around your practice
- For credentialed professionals with hands-on deal or month-end close experience
- Apply with your CV and a note on your domain
Tell us what you're building.
Whether you need a standing eval team or a one-off data sprint, we'll scope it with you and field the right people.
Include the model or product, the task domain, and rough volume. You'll hear back with a proposed scope within one business day.
Get in touchhello@s46holdings.com