← Back to Demo

Receipt Processing API Documentation

Complete API reference for receipt image processing

Endpoint

POST /api/receipt/process

Upload a receipt image and receive structured JSON data with merchant information, line items, totals, taxes, and payment details. The API automatically validates that the uploaded image contains a valid receipt.

AI-Powered Receipt Validation

⭐ Enhanced with Gemini 3.0 Flash AI

When GEMINI_API_KEY is configured, the API uses advanced AI reasoning to determine if an image is actually a receipt, providing significantly better accuracy than pattern matching alone. The AI understands context and is lenient with OCR errors.

How Validation Works

With Gemini AI (Recommended)

  • ✅ AI analyzes text semantically
  • ✅ Recognizes ALL receipt types
  • ✅ Lenient with OCR quality
  • ✅ Provides detailed reasoning
  • ✅ Adapts to context

Without API Key (Fallback)

  • • Pattern-based validation
  • • Fixed scoring rules
  • • Less flexible
  • • May reject some valid receipts

Receipt Types Supported

The AI validation recognizes receipts from:

Retail StoresRestaurantsGrocery StoresMedical ServicesVeterinaryProfessional ServicesGas StationsPharmaciesAnd more...

💡 Tip: For best results, always configure GEMINI_API_KEY. The AI validation is dramatically more accurate and handles edge cases much better than pattern matching. See RECEIPT_OCR_SETUP.md for setup instructions.

Request Format

Content-Type

multipart/form-data

Required Field

FieldTypeDescription
imageFileReceipt image file

Supported Formats

  • JPEG / JPG
  • PNG
  • WebP
  • HEIC

File Size Limit

Maximum: 10 MB

Example Request

// JavaScript/TypeScript Example
const formData = new FormData();
formData.append('image', fileInput.files[0]);

const response = await fetch('/api/receipt/process', {
  method: 'POST',
  body: formData,
});

const receipt = await response.json();
# cURL Example
curl -X POST https://your-domain.com/api/receipt/process \
  -F "image=@receipt.jpg"

Response Format

Success Response (200)

Returns a ParsedReceipt object with the following structure:

{
  // Merchant Information
  "merchantName": "Fresh Market Grocery",
  "merchantAddress": "123 Main Street, Springfield, IL 62701",
  "merchantPhone": "(555) 123-4567",
  "merchantTaxId": "12-3456789",
  
  // Transaction Details
  "transactionId": "TXN-1234567890-ABCDEF123",
  "date": "2024-01-15",
  "timestamp": "2024-01-15T14:30:00.000Z",
  
  // Line Items
  "items": [
    {
      "name": "Organic Bananas",
      "quantity": 1.5,
      "unitPrice": 0.69,
      "totalPrice": 1.04,
      "category": "Produce",
      "sku": "PROD-001",
      "discounted": false
    },
    {
      "name": "Whole Milk (1 gal)",
      "quantity": 1,
      "unitPrice": 4.99,
      "totalPrice": 4.99,
      "category": "Dairy"
    }
  ],
  
  // Financial Summary
  "subtotal": 23.45,
  "discount": 2.35,
  "taxes": [
    {
      "name": "Sales Tax",
      "rate": 8.25,
      "amount": 1.74
    }
  ],
  "tip": 0,
  "total": 22.84,
  
  // Payment Information
  "payments": [
    {
      "method": "Credit Card",
      "cardLast4": "4532",
      "amount": 22.84
    }
  ],
  "amountTendered": null,
  "changeGiven": null,
  
  // Additional Details
  "currency": "USD",
  "cashierName": "John D.",
  "registerNumber": "REG-1",
  
  // Processing Metadata
  "metadata": {
    "processedAt": "2024-01-15T14:35:00.000Z",
    "confidence": 0.92,
    "imageFormat": "JPEG",
    "imageSize": 2457600,
    "isMockData": false
  }
}

Error Responses

400 Bad Request - Invalid File

Invalid request (missing file, wrong format, file too large)

{
  "error": "Invalid file type",
  "status": 400,
  "details": "Supported formats: JPEG, PNG, WebP, HEIC. Received: image/gif"
}

400 Bad Request - Not a Receipt

Image validation failed - not recognized as a valid receipt

{
  "error": "Invalid receipt image",
  "status": 400,
  "details": "Not a valid receipt: insufficient receipt indicators found"
}

500 Internal Server Error

Server error during processing

{
  "error": "Failed to process receipt image",
  "status": 500,
  "details": "OCR service unavailable"
}

TypeScript Types

Import the types from the API module for full type safety:

import type {
  ParsedReceipt, 
  ReceiptItem,
  ReceiptTax,
  ReceiptPayment,
  ReceiptProcessError 
} from '@/types/receipt/types'

// Use in your code
const processReceipt = async (file: File): Promise<ParsedReceipt> => {
  const formData = new FormData()
  formData.append('image', file)
  
  const response = await fetch('/api/receipt/process', {
    method: 'POST',
    body: formData,
  })
  
  if (!response.ok) {
    const error: ReceiptProcessError = await response.json()
    throw new Error(error.error)
  }
  
  return await response.json()
}

Best Practices

  • Image Quality: Use well-lit, clear images with the entire receipt visible for best results.
  • File Size: Compress images before upload to reduce processing time and bandwidth usage.
  • Error Handling: Always check the response status and handle errors gracefully.
  • Validation: Validate the confidence score in metadata before trusting the extracted data.
  • Privacy: Handle receipt images securely as they may contain sensitive personal or financial information.

Production Considerations

Rate Limiting

In production, implement rate limiting to prevent abuse (e.g., 100 requests per hour per IP).

Authentication

Add API key authentication or JWT tokens to secure the endpoint for production use.

OCR Technology

Currently using Tesseract.js for real OCR processing. For even better accuracy:

  • Google Cloud Vision API - Excellent for general OCR with very high accuracy
  • AWS Textract - Specialized for forms and receipts with structured data extraction
  • Azure Form Recognizer - Pre-trained receipt models with high confidence

LLM Enhancement (Recommended)

For production-grade accuracy, consider adding LLM post-processing:

  • Error Correction: Use GPT-4/Claude to fix OCR misreads (e.g., "1" vs "l", "0" vs "O")
  • Gap Filling: LLMs can infer missing or garbled text from context
  • Business Validation: Cross-reference merchants against Google Places API or business databases
  • Smart Structuring: LLMs better understand receipt format and can parse complex layouts

Example: Send OCR output to LLM with prompt: "Fix errors in this receipt text and structure as JSON. Validate merchant name against known businesses."

Data Storage

Consider storing processed receipts in a database for auditing and user history.

Async Processing

For large volumes, use a queue system (e.g., Bull, AWS SQS) to process receipts asynchronously.