Image comparison & applications

You can send multiple images in a single request. The AI can compare, contrast, and find differences between them. This is powerful for product comparison, quality control, and before/after analysis.

Multi-image analysis

How the vision API processes and compares multiple images

There is no hard limit on the number of images, but each image adds significant tokens. Sending 10 high-resolution images could easily exceed the context window. In practice, 2-5 images per request works well. For larger batches, process them in groups.

02-multimodal.ipynb
python
# Compare two restaurant interiors
image1 = Image.open("restaurant_modern.png")
image2 = Image.open("restaurant_rustic.png")

comparison = analyze_image(
    "Compare these two restaurant interiors. "
    "List 3-5 specific differences in style and atmosphere.",
    [image1, image2]  # Pass multiple images!
)
print(comparison)
# Output: "1. Lighting: First uses sleek pendant lights,
# second has warm Edison bulbs. 2. Materials: First is
# glass and steel, second is reclaimed wood..."

Pass a list of images to analyze_image(). The AI understands they are separate images and can compare them.

Now let us turn image comparison into a real application. By combining vision AI with a system instruction, we can automatically generate professional menu descriptions from food photos.

02-multimodal.ipynb
python
# Real-world app: Food photo → menu description
menu_system = """You are a professional food writer.
Write appetizing menu descriptions (2-3 sentences)
that highlight ingredients and cooking techniques."""

description = analyze_image(
    "Write a menu description for this dish.",
    [food_image],
    system_message=menu_system
)
# Output: "Pan-seared king oyster mushroom steaks,
# nestled on a bed of saffron risotto with roasted
# cherry tomatoes and a balsamic reduction drizzle."

Combine vision AI with system instructions for domain-specific applications like menu writing.

Another practical application: accessibility. Vision AI can automatically generate alt text descriptions for images, making web content usable for visually impaired users.

02-multimodal.ipynb
python
# Accessibility: Generate alt text for images
alt_text = analyze_image(
    "Write a concise alt text for this image. "
    "Describe what a visually impaired person needs to know.",
    [food_image],
    temperature=0.3  # Consistent, factual descriptions
)
# Output: "A colorful Buddha bowl with quinoa, roasted
# vegetables, avocado slices, and chickpeas in a
# white ceramic bowl on a wooden table."

Vision AI can improve accessibility by automatically generating alt text descriptions for images.

Limitations & cost awareness

Limitations to be aware of: - Cost: Images consume many tokens (a single image can be 1000+ tokens) - Accuracy: Small text and precise numbers can be misread - Privacy: Don't send sensitive images to external APIs - Resolution: Higher resolution = more tokens = better accuracy - Speed: Vision calls are slower than text-only calls

Ordering exercise: Order by token cost (lowest to highest)

Loading practice…

Checkpoint: Multimodal AI concepts

Loading practice…

Validation checklist: Multimodal AI checklist

Loading practice…