ProEval Studio
Multi-Language AI Code & RLHF Evaluation Simulator for Outlier AI, Code4Ai, and High-Stakes Technical Assessments
The Problem
Leading AI data evaluation and RLHF platforms (such as Outlier AI, Code4Ai, and Alignerr) require engineers to audit, debug, and rate LLM-generated code under strict time gates (typically 8–15 minutes).
Candidates with strong algorithmic backgrounds routinely fail these screenings—not because they can't code, but because:
- They overlook subtle negative prompt constraints (e.g., forbidden packages, specific time complexities, strict schema invariants).
- They miss silent runtime edge cases (such as
Noneinputs, empty collections, float precision drift, or mutation of input arguments). - They write generic or hand-wavy rationales instead of structured, reviewer-grade justifications citing exact contracts, defects, and surgical fixes.
The Solution
I engineered ProEval Studio: a high-fidelity sparring simulator and training workbench that transforms theoretical evaluation rubrics into an active, timed practice arena.
⚡ 8-Min Bug-Hunt Sprints
Generates realistic problem briefs with subtly flawed AI code. Includes an in-browser editor, live Pyodide Python WASM execution, and a mandatory 4-point Pre-Submit Verification Gate.
⚖️ Model A vs. B Arena
Pairwise evaluation with interactive 0–3 rubric scoring pills across Instruction Adherence, Edge-Case Safety, Idiomatic Quality, and Justification Precision.
🤖 Adversarial AI Reviewer
Acts as an unforgiving Senior QA Reviewer powered by Gemini structured JSON schemas, catching missed boundary inputs and diffing rationales against an exemplary benchmark.
🎓 Socratic Mode & Mock Exam
Features 4-step progressive line-traced hints for practice, and a locked-down 25-minute 3-task gauntlet that issues shareable Readiness Scorecards.
Key Engineering & Architectural Highlights
- Deterministic QA via Gemini Structured Outputs (responseSchema) Leveraged the Google Gemini API with strict JSON schema definitions to ensure deterministic drill generation, automated rubric category grading (0–3 scale), and machine-parseable diff rationales without hallucinations.
- Zero-Install In-Browser Execution Sandbox (Pyodide WASM) Integrated Pyodide 0.27.2 to execute and test Python 3.11+ code locally in the browser memory space with sub-second execution times and zero server infrastructure cost. Paired with a local WebSocket bridge daemon for multi-language execution (JS/TS, SQL, C++, Go, Rust).
- Tamper-Proof Firebase Architecture & Quota Guardrails Designed Firestore security rules enforcing verified email authentication, immutable graded drill records (anti-tamper scoring), and strict 24-hour rate-limiting (15 drills/day), supplemented with a client-side BYOK (Bring Your Own Key) toggle for unlimited practice.
- Offline-First Progressive Web App (PWA) Configured Workbox caching strategies for instant asset delivery, offline indicator states, and seamless client-side storage persistence across browser reloads.
Results & Competencies Demonstrated
Full-Stack Product Engineering
From product discovery to end-to-end implementation across React 19, TypeScript, Express backend, and Cloud infrastructure.
Advanced AI & LLM Engineering
Mastery of structured output constraints, prompt engineering for evaluation benchmarks, and Socratic tutoring control flows.
Cloud Security & Data Integrity
Production-grade Firestore security rules, rate-limiting, and defense-in-depth data validation.