From Manual Counting to Zero Shipment Errors: AI-Powered Pallet Verification for the Assembly Line

Takenaka is a high-volume Japanese manufacturing operation running structured assembly and palletisation lines across multiple production floors. Products are packed in precise tier-by-tier pallet configurations before shipping — a process that requires accurate counts at every layer to ensure the correct quantities reach the warehouse and, ultimately, the customer.

The workforce is multilingual and spread across several production lines. Shift handovers happen frequently, and operators work in gloves under factory-floor conditions that make traditional keyboard and password-based systems impractical.

The challenge was not new, but it had become impossible to ignore. As volume grew and shift patterns diversified, the cost of manual counting errors — wrong quantities shipped, downstream inventory mismatches, and quality investigations after the fact — started accumulating in ways that affected both customer confidence and internal productivity.

Takenaka needed a system that would work in the actual environment of a production floor: fast logins for gloved workers, robust across multiple languages, capable of handing off a partially-counted pallet between shifts without losing any progress, and — critically — one that would prevent a wrong shipment from ever leaving the floor rather than detecting it afterward.

Manual tier-by-tier counting sounds straightforward. In practice it is not. Products are visually similar, arranged in tight grids across multiple layers, and counted under variable lighting by operators who are also managing other tasks on the line. Fatigue, distraction, and shift pressure all introduce errors — and none of those errors are visible until something is already wrong downstream.

When an operator finished a pallet, they recorded the count manually. There was no independent verification, no image evidence, and no way to confirm which specific person had completed which count at what time. If a dispute arose — a customer receiving fewer items than invoiced, or a warehouse showing a discrepancy — the floor had nothing to show.

The problem was also structural: pallets sometimes had to be left mid-count when an operator was reassigned. With no persistent session, the count would restart with the next available operator, introducing double-counting risks or gaps.

We designed and built a two-app iOS system purpose-built for the production floor. The architecture was shaped by one core constraint: the system had to work in the way operators actually work — not the way a traditional enterprise software workflow assumes they do.

Two-Tablet Architecture with Live Streaming

One tablet is mounted at the pallet station and acts as a dedicated camera device. It streams a live WebRTC video feed over the factory Wi-Fi. The operator's handheld tablet connects to this feed and receives the live view in real time. This means operators are not hovering their own device over each tier — they watch a stable, well-positioned camera feed while the system handles the counting.

On Apple devices, the live feed persists in a picture-in-picture window even when the operator navigates elsewhere — to check packaging guidelines, scan a barcode, or respond to a supervisor message. The count context is never lost.

AI Counting Engine (YOLOv8)

Each product has its own trained YOLOv8 model, built from real production-floor images captured in the actual environment — variable lighting, metal containers, partial occlusion, natural placement variation. When an operator photographs a tier, the image is sent to the AI service, which returns a count and a visualised result showing bounding boxes around every detected item.

Two counting modes are supported. In manual mode, the operator takes a photo and the AI counts it. In automated video mode, the system samples the live camera feed continuously and advances to the next tier automatically once the expected count is confirmed. Both modes produce the same audit output.

Mandatory Release Gate

No pallet can be released until the operator scans the pallet barcode a second time and it matches the barcode scanned at the start of the session. If it does not match, the release is blocked. This one mechanism — deceptively simple — addresses the most common cause of wrong-pallet-wrong-location errors that the floor had experienced repeatedly.

Session Persistence and Shift Handover

If an operator is pulled away mid-count, the session is saved automatically. Any operator — on any shift — can resume it from exactly where it was left, with all tier images and counts intact. This eliminated the most common source of restart-and-double-count errors in multi-shift environments.

Packaging Guides, Multi-Language, and Audit Trail

Operators can access product-specific packaging guides — photos, PDFs, and step-by-step image sequences — directly from the counting screen. No separate binder, no walking to a supervisor. The app supports seven languages (English, Japanese, Hindi, Nepali, Portuguese, Tagalog, Vietnamese) and switches automatically based on the operator's login profile.

Every count, every tier image, every barcode scan, every login and pallet status change is recorded with timestamp, device serial number, operator identity, and production line. Quality teams have a complete, immutable audit trail for every pallet that has ever moved through the floor.

The engagement was structured in two phases. The first phase focused on building and calibrating the AI counting models — collecting real production-floor footage, extracting and deduplicating frames, annotating training data, and training individual YOLOv8 models per product. Takenaka's floor team was involved directly in data collection and validation, ensuring models reflected actual operating conditions rather than idealised samples.

The second phase built the full application layer: the iOS two-tablet system, the Django REST backend, WebRTC streaming infrastructure, barcode login, the release gate, and the supervisor dashboard. Takenaka's operators participated in structured pilot testing before full rollout, and feedback from the floor shaped several UX decisions — including the persistent background count notification and the packaging guide integration.

One outcome that was not explicitly planned at the outset: device accountability. By tying every session to a specific tablet's serial number — not just the operator account — the system gave management and IT the ability to trace exactly which physical device was involved in any count. This became relevant during a compliance check shortly after rollout and prevented what would otherwise have been a lengthy manual investigation.

“

“We used to find out about counting mistakes after the pallet had already left the floor. Now we catch them before they happen, every single time.”

“

“The app switching to my language automatically made a real difference. I don't have to guess at the instructions anymore — everything is clear before I start.”

The core system is in active use across Takenaka's production lines. Several capabilities built during the engagement — including warehouse mismatch alerting and a formal pallet rejection workflow with email notification — are implemented and ready to activate as the team expands the system's scope. Model retraining for new product types is a planned next step as the product range grows.

If your organisation is navigating a similar challenge — manual counting on a production floor, no traceability, errors that surface too late — we are happy to share how we approached it and whether a similar model could apply to your environment. There is no obligation in that first conversation.