On-Prem Wafer Inspection AI: From Folder to Annotated Result in Minutes

A High-Volume Silicon Fabrication Operation

Our client is a multi-site silicon wafer manufacturer in Japan, producing substrates for the semiconductor industry at scale. Their fabrication process demands precision at every stage — and surface inspection is among the most labour-intensive of them. Carrier scratches, vertical scratches, cracks, chips, and particulate contamination must all be identified before affected wafers progress downstream, where the cost of a defect multiplies with every subsequent step.

The inspection operation had historically relied on trained operators reviewing image folders from automated scanning equipment. At the volumes this facility runs, that model was beginning to buckle under its own weight.

Why the Problem Could No Longer Wait

Semiconductor manufacturers face a structural tension: production throughput targets rise every cycle, but the inspection step — by its nature image-intensive and judgment-heavy — scales poorly with human reviewers alone. The client had GPU infrastructure on the network, a well-defined set of defect classes, and production image folders with clear, consistent naming conventions. Everything needed to run inference at scale was already present. What was missing was a deployable system.

Prior attempts at automating the pipeline had failed at the inference layer: batch calls to the GPU server resulted in out-of-memory errors that forced fallback to slow, single-shot processing. The instability made automation unreliable enough that operators continued to treat AI tooling as secondary — something to test, not to trust. The business needed a system that worked first time, every time, and that gave supervisors full visibility without adding process overhead.

What Inspection Actually Looked Like on the Floor

Before this engagement, the inspection workflow looked like this:

• 01Operators received image folders from scanning equipment organised by slot → wafer → image.
• 02Images were reviewed using ad-hoc scripts — no standard UI, no validation, no progress tracking.
• 03Inference calls were single-shot: one image, one request, one result. GPU batching had been tried and abandoned after repeated OOM failures at batch size 32.
• 04There was no operator accountability record: no log of who reviewed what, on which terminal, at what time.
• 05A lot of 108 images (3 wafers) took in excess of seven minutes — and that was on a clean run without retry loops from GPU failures.
• 06Six defect classes needed to be caught in a single pass. Manual review meant inconsistency across shifts and operators.

The underlying GPU infrastructure — an NVIDIA server running on the fab LAN — was underutilised due to misconfiguration and the absence of a stable inference client. The CUDA memory pool had not been tuned; instance counts were wrong for the workload. The hardware was capable. The software stack around it was not.

What Aarya Global Built — and Why

We built a production-grade, end-to-end inspection automation system with no cloud dependency, no MES modification, and no developer tooling required on the shop floor. The architecture has two clear roles: a Windows client for operators, and a Linux GPU host for inference — connected by gRPC over the fab LAN.

Parallel Preprocessing Pipeline

Every wafer image is large enough that sending it directly to the model is both slow and imprecise. We implemented a 10-part tiling system using OpenCvSharp: each image is split into overlapping tiles, padded, and normalised into the CHW format the model expects. This runs in parallel across all available CPU cores using Parallel.For — typically achieving 50–70% faster preprocessing compared to sequential approaches on standard multi-core workstations.

After inference, tile-level detections are remapped to full-image coordinates and merged back, so the annotated output reflects the original wafer image as the operator would recognise it.

Native C++ Inference Layer (TritonClientWrapper)

The most consequential architectural decision in this engagement was how the Windows client talks to the Triton server. Most implementations use HTTP REST — it is easy to integrate from managed code but carries significant per-request overhead and is not Triton's native production path. We chose gRPC via Triton's own C++ client libraries.

We wrote TritonClientWrapper: a native DLL built against the Triton SDK (grpc_client, gRPC++, protobuf, absl). It handles batch inference requests, parses YOLOv8 ONNX output tensors, and marshals results back to the C# application layer as JSON — all in compiled native code. The UI's C# layer calls into it via P/Invoke. The hot inference path never enters managed code.

This was the reason previous batch attempts had failed. The problem was not the model or the GPU — it was CUDA memory fragmentation caused by untuned memory pools and multiple Triton instances competing for GPU memory. Once we set a 2 GB CUDA memory pool, configured a single instance per model, and switched ONNX to BFCArena allocation, batch-32 ran cleanly on the first attempt.

NVIDIA Triton Inference Server

Triton runs in Docker on the client's on-prem Linux host (RHEL / Ubuntu). The YOLOv8 model is served in ONNX format with dynamic batching enabled and max_batch_size set to 32–64. The same client-server stack runs unchanged on any CUDA-capable GPU — RTX 3080 class through H100/A100. Upgrading the model means pointing the client at a new version in the Triton model repository; no application rewrite is needed.

Operator-First WinForms Application

We deliberately kept the UI surface minimal. Operators on a production floor do not need configuration panels or advanced options — they need to start a job and trust that it finishes. The WinForms application gives them exactly that:

• 01Browse input and output folders (familiar Windows file picker)
• 02Test Connection — verifies the Triton server is reachable before committing to a long batch
• 03Start / Stop with safe mid-run cancellation
• 04Live progress: current wafer, current image, elapsed time — visible to supervisors at a glance
• 05Configurable batch size (1–64) for tuning speed vs. GPU load without engineering involvement
• 06View Output Folder — annotated images ready for QC, one click
• 07Task ID assigned per run — every batch is isolated, auditable, and reproducible

The application was built on .NET 8 with fully async orchestration and pre-flight validation. It runs on standard Windows fab desktop PCs with no additional runtime dependencies beyond the DLL.

Defect Coverage

Six defect classes are detected in a single inference pass:

Confidence threshold defaults to 0.7 and is configurable per deployment. Bounding boxes are padding-aware, so overlays align correctly with the full-resolution wafer image regardless of tile placement.

Deployment Architecture

The system deploys entirely within the fab network. There is no cloud endpoint, no license server, no external SLA dependency:

Before and After — Measured on the Same Workload

Performance was measured on the same 108-image lot (3 wafers) before and after server tuning and the production deployment:

What Changed Operationally

• 01Operators run the same model with the same rules on every shift — not ad-hoc scripts that vary by person or terminal.
• 02Batch-32 inference is now the standard path, not a failed experiment. GPU utilisation is no longer a problem to manage around.
• 03Every batch produces: annotated JPEGs, a detections.csv with per-image class and confidence data, a plain-text summary, and a gRPC execution log. QC teams have structured outputs without manual extraction.
• 04Supervisors see live job status — wafer number, image count, elapsed time — without needing to interrupt operators or open a separate tool.
• 05An unexpected benefit: the task ID system created a natural audit trail that the team subsequently used in a compliance review. It was not originally specified as a requirement.

“

“The system runs the same model with the same rules on every shift. Our supervisors can see the status in real time and every batch is auditable — without anyone having to chase it down.”

“

“Batch-32 now works first time. We spent months thinking it was a hardware limitation. It turned out to be a configuration problem, and once it was fixed properly, the throughput numbers changed immediately.”

A Platform, Not a One-Off

The architecture is designed to scale beyond a single inspection line. The same pattern — Windows client, Linux Triton host, C++ gRPC bridge, large-image tiling — applies to any folder-based inspection workflow at any resolution. Deploying to a second inspection line means pointing a new client at the same Triton host and configuring a folder path. Deploying to a new fab site means running the same Docker container and distributing the client binary.

The client is in active discussions with Aarya Global about extending the system to a second inspection category at an adjacent production line. The model update path — dropping a new ONNX file into the Triton repository — means no application changes are required when defect class definitions evolve.

The Decisions That Made the Difference

Three choices defined the outcome of this engagement:

• 01Native over managed. We used native C++ for the inference path, not managed HTTP. This was the single decision that unlocked stable batch-32 inference — everything else in the performance improvement followed from it.
• 02Non-invasive integration. We did not touch the MES or change any existing production workflow. The system operates entirely on folder handover — the same mechanism the fab already used to move images. Go-live beside existing systems, not after replacing them.
• 03Production configuration, not demo defaults. We tuned the GPU stack for production rather than accepting demo-level configuration. Understanding CUDA memory pool behaviour, BFCArena allocation, and single-instance Triton config for this specific workload was what separated a working system from a failed experiment.

If You're Navigating a Similar Challenge

High-resolution image inspection with GPU inference, an on-prem data sovereignty requirement, and an operator population that cannot use developer tooling is a specific and underserved problem. We have built the production patterns for it.

If your organisation is dealing with inspection bottlenecks, idle GPU infrastructure, or failed automation attempts in a similar environment, we would be glad to share how we approached this problem and whether the same model applies to your context.