logo
Development
OverviewDevSpaceWorkSpaceAPI ReferenceBest PracticeChangelog
Search
English
WorkSpace / Work Area / Work / Multimodal IO
Multimodal Input & Output and File Preview
Last updated：30 days ago

Multimodal Input & Output and File Preview

Overview

Work supports rich multimodal interaction: users can send various types of content such as images, documents, and audio to the Agent; the Agent's replies can include multiple output formats such as code, documents, and charts, all supporting inline preview and full-screen viewing.

Multimodal Input

Supported Input Types

Content Block Type Description Example
text Plain text User messages, code snippets
image Image PNG, JPEG, GIF, WebP (base64 encoded)
document Document PDF documents (base64 encoded)
audio Audio Audio files
file_ref File reference Local file path reference

Upload Methods

  1. Drag and Drop — Drag files directly into the input area
  2. Click to Upload — Click the attachment button in the input area to select files
  3. Paste — Paste images from the clipboard

Multimodal Input-CN

Model Format Adaptation

The system automatically converts the input format based on the currently used LLM provider:

Provider Image Document Audio
Anthropic Native base64 image block Native base64 document block Not supported
OpenAI image_url (data URI) Downgraded to text description Native audio block
DeepSeek Not supported Not supported Not supported

The system uses the resolveCapabilities function to automatically detect the multimodal capabilities of the current model (vision / document / audio). Unsupported types will prompt the user or be handled with a downgrade.

Multimodal Output and Preview

9 Preview Types

Files output by the Agent are automatically identified by type and rendered as the corresponding preview component:

File Type Preview Component Function Buttons
PDF PdfPreviewWindow Zoom (zoom in / zoom out / fit width), page turning, page number display
PPT PptPreviewWindow Slide navigation (previous / next), zoom
Excel / Word OfficePreviewWindow Zoom, page turning
Code CodePreviewWindow Syntax highlighting (multiple languages), copy to clipboard
Markdown MarkdownPreviewWindow Markdown rendering preview
HTML HtmlPreviewWindow Embedded iframe rendering
Image MediaPreviewWindow Zoom, drag to pan
Video MediaPreviewWindow Play / pause, progress bar, volume control
Audio MediaPreviewWindow Play / pause, progress bar

Inline Preview Card

Files output by the Agent first appear in the conversation flow as an inline preview card (InlinePreviewCard):

Inline Preview Card-CN

Element Description
Title Bar File name, type badge, action buttons
Thumbnail Content File content preview (collapsed state, bottom gradient + "Show more" hint)
Hover Effect Highlighted border + elevated shadow

Action Buttons (icon + tooltip):

Button Function
Expand Full-screen preview window
Copy Copy content to clipboard
File Manager Show in the system file manager (APP only)
Open Externally Open with the default application (local files) or open in browser (URL)

Full-Screen Preview Window

Click the "Expand" button to enter the full-screen preview window:

Full-Screen Preview Window-CN

Preview Toolbar (PreviewToolbar):

  • Zoom control: zoom in / zoom out / fit width
  • Page navigation: previous / next (multi-page documents)
  • Page number status: current page / total pages

Keyboard Shortcuts:

Shortcut Function
+ or = Zoom in
- Zoom out
0 Reset zoom
Esc Close preview window

Preview Layout Modes

Mode Platform Description
Overlay APP default Fixed-positioned overlay window
Docked Web Embedded in the right panel of WorkPage

Web and APP Capability Sharing

The core components of multimodal preview are implemented in the claw-shared layer, with Web and APP sharing the same codebase:

  • DocumentPreviewModal — Preview window container
  • InlinePreviewCard — Inline preview card
  • PreviewToolbar — Preview toolbar
  • previewUtils.ts — File type identification and loading utilities

Both platforms abstract the file reading method through the FileAccess interface:

  • APP side: Reads via the Tauri local file API
  • Web side: Loads via HTTP URL or data URL

Operation Guide

How to Send Images/Documents to the Agent

  1. Drag and Drop: Drag files directly from the desktop or file manager into the Work input area
  2. Click to Upload: Click the 📎 attachment button on the left of the input area and select files
  3. Paste Images: Copy an image and paste it in the input area with Ctrl+V / Cmd+V
  4. The attachment appears in the preview bar above the input area
  5. Type a text description of what you want the Agent to do (e.g., "Analyze this image") and press Enter to send

File Size Limit: The Read File tool supports files up to 5MB. For larger files, it is recommended to upload them to cloud storage first and provide the link.

How to Use File Preview

  1. When the Agent's output includes a file, an inline preview card appears in the conversation flow
  2. Hover over the card to view more information
  3. Click the Expand button to enter full-screen preview
  4. Use the toolbar operations in full-screen preview (zoom, page turning, etc.)
  5. Press Esc or click the close button to exit preview

FAQ

Why didn't the Agent analyze the image after I sent it?

  • The current model may not support vision capability. Check whether the model in Model Configuration is marked with vision support. DeepSeek series models do not support image input.
  • It is recommended to switch to Claude Sonnet 4 or GPT-4o.

Why does the PDF only return "Not supported" after sending?

  • The document capability is currently only natively supported by Anthropic series models (Claude). When using OpenAI models, PDFs are handled with a downgrade.
  • Alternative: Have the Agent use the Read File tool to read the PDF file path.

File displays abnormally in the preview window

  • Confirm the file format is correct (e.g., an .xlsx file cannot use the .csv extension)
  • Very large files may load slowly; please wait for loading to complete
  • Some Office files in special formats may need to be opened in an external application