I treat data provenance, evaluation, deployment boundaries, and recovery as part of the system design. Each case study shows the release condition, the evidence I generated, and the limits of the result. The reported metrics come from controlled or synthetic runs, not customer production traffic.
Architecture at a glance
I compare the same operating decisions across three systems.
I start with source and lineage, define a gate that can reject a candidate, then specify serving, monitoring, and recovery. The table makes those choices comparable.
4 automated tests; deployment artifacts are definitions
6 automated tests; 7-case synthetic evaluation
460 automated tests; crypto research foundations tested on synthetic fixtures; PostgreSQL and AWS resources are not deployed
01 · End-to-end delivery
FraudShield
I built FraudShield to make a fraud model reproducible from input data to API image. DVC records the pipeline stages, MLflow captures training metadata, and a regression gate blocks weaker candidates. I package the approved model behind a typed FastAPI interface and define probes, drift signals, and alerts for operation.
I use GitHub Actions to install pinned requirements, run four tests, retrain a controlled fixture, and execute the promotion policy. The pipeline builds a Docker image tied to the commit only after those checks.
OPERATIONS
Checks before the API runs
I check the feature schema and model regression before packaging the API. I include Prometheus alerts, probes, autoscaling manifests, and PSI drift signals as an operating design; the cloud deployment is not running.
02 · Evaluation-first LLMOps
RAGOps
I built RAGOps around an answer quality contract: retrieve relevant material, cite it accurately, and decline when the evidence is insufficient. I test grounding, citation coverage, unsafe requests, and latency before a release can pass.
I run six automated tests and seven synthetic evaluation cases in GitHub Actions. A retrieval or safety regression blocks the release gate and the commit-addressed Docker build.
GUARDRAILS
Cite the source or decline to answer
I version the knowledge inputs and test citations, response time, prompt injection, and unsafe medical advice. When the source does not support an answer, I expect the service to abstain.
AI control proof · offline evaluation
A policy gate can cite, abstain, or refuse.
Replay a recorded case from the RAGOps evaluation report. The application applies a retrieval threshold, checks the request against safety rules, and returns only a supported source answer or a refusal.
Choose a case and run it to inspect the recorded result.
Scope: these are recorded results from a small synthetic evaluation. RAGOps currently uses deterministic retrieval and application-level filters; it does not call a hosted LLM in this demo. This proves the tested service policy, not control over a model’s internal behavior or performance on arbitrary real-world inputs.
03 · Risk-controlled operations
SignalOps
I built SignalOps as a paper-only research and operations platform. It traces authorized alerts through deterministic risk policy, simulated fills, durable evidence, and recovery controls. The Crypto Discovery Engine now has offline-tested social and DEX feature foundations, Solana mint and pool identity checks, a pinned Raydium CPMM layout parser, and an opt-in public read-only collector. These components establish data contracts, not market coverage: fixtures are synthetic, live collection has not been performed, and no crypto ranking, backtest, return, or live execution is claimed.
I run 460 behavioral and integration tests in CI, generate a deterministic CycloneDX SBOM, scan dependencies, validate infrastructure, and build a commit-addressed container. I also verify signed release provenance against trusted and revoked keys. Altered signatures, malformed reports, or missing inputs block verification.
ROLLOUT & RECOVERY
Observe a candidate before promoting it
Incident readiness: I compare candidate and current models in shadow windows using calibration, PSI drift, expectancy, drawdown, and market regimes. Signed promotion gates, kill switches, reconciliation, incident records, and exact-model rollback constrain paper operation.
DATA READINESS
Check whether the history is useful first
I reject underpowered history before fitting by checking span, observation and timestamp counts, label availability, and declared bull, bear, and sideways coverage without reading outcomes. I sign the report hash into candidate lineage, so altered or missing evidence blocks verification.
DEPLOYMENT STATUS
What runs in tests and what is still a design
Adversarial promotion gate: I require stress scenarios to pass before promotion. I have tested the Python and SQL controls. PostgreSQL, AWS/Fargate, Multi-AZ recovery, and immutable evidence storage remain infrastructure definitions, not deployed services.
CRYPTO DATA LIMITS
Structure is tested; market validation is pending
Crypto fixtures contain seven synthetic social messages, two DEX pairs, and six synthetic Solana RPC exchanges. There are zero live social messages, live RPC captures, historical token outcomes, or verified real-world token returns. The next step is to bind normalized DEX observations to verified mint and pool identities, then add separately pinned PumpSwap and Meteora parsers.