Microsoft

Florence-2

About

Florence-2 is a 232M open-weight model from Microsoft. Vision foundation model for captioning, OCR, detection, and region tasks. It accepts text and images. Released June 15, 2024.

Parameters
232M
Published parameter count.
License
MIT
License on the weights.

Pricing

Token rates are US dollars per 1M tokens. Image rates are per 1K images. Audio rates are per 1M audio seconds.

Rate	Price
Input / 1M	$0.10
Cached input / 1M	$0.03
Output / 1M	$0.30

Compare

Florence-2 is the smallest of these detection models, at 232M.

Model	What it does	Price	Parameters	Context
Florence-2	Vision foundation model for captioning, OCR, detection, and region tasks.	$0.10 / 1M input	232M	—
Qwen3.5 0.8B	Multimodal chat model with up to 64 images or one video per request.	$0.08 / 1M input	0.8B	256K
DiffusionGemma 26B-A4B	Block-diffusion Gemma 4 26B-A4B: chat over text and up to 8 images, and System One typed decisions on POST /typesafe/v1/systemone.	$0.35 / 1M input	26B	16K
Gemini 3.5 Flash Lite	Fastest and cheapest Gemini tier. Multimodal chat; emits no reasoning tokens.	$0.30 / 1M input	—	1M

Benchmarks

Published scores for Florence-2, from the Florence-2 model card.

Evals

Reading	Example
Qualitative	Clear structure, grounded in the input
Quantitative	COCO caption CIDEr: 140.
Cost and performance	Lower listed rate, mid-pack latency

Performance

Latency is the end-to-end round trip. Throughput is completion tokens divided by that time. It is a request rate, not decode speed.

Latency
254 ms
Throughput
—
Requests
324

Examples

Dense region captions
Florence-2 names every region in a vegetables-on-counter shot.
{"regions":[{"label":"a tomato","box_xyxy":[58,140,224,396]}, ...]}
Dashboard caption
`more_detailed_caption` describes a shadcn dashboard screenshot.
{"caption":"A modern analytics dashboard with KPI cards, ..."}

Supported inputs

textIgnored
A text part beside the media is accepted and ignored. A request with no media part is a 400.
image_urlYes
One per request. A second image_url part is a 400.
video_urlNo
Not accepted.
document_urlNo
Not accepted.

Default method caption. The caption and ocr methods emit a Markdown string; the region methods emit a region container.
Alias: microsoft/Florence-2-base-ft. Weights: microsoft/Florence-2-base-ft.

Methods

Methods this model serves. Payload shapes are in the docs.

Method	Returns
caption (default)	Short image caption
detailed_caption	Longer caption
more_detailed_caption	Longest caption
ocr	All text on the image
od	Region items with `bbox_xywh` and `label`
dense_region_caption	Region items with `bbox_xywh` and `label`
region_proposal	Region items with `bbox_xywh`, no `label`
ocr_with_region	Region items with `bbox_xywh` and `text`

Each method maps to one Florence-2 task token. The task token itself (<CAPTION>, <OD>, ...) is an implementation detail and is not part of the reply: a caption is the string, and a region method is the container.

Boxes are normalized 0-1 against the image at precision decimals (default 4), and a region method with no hits returns "items": [].

Estimate cost

Input tokensOutput tokens

$0.40

Quick start

Call this model on the OpenAI-compatible gateway. The model id is already filled in.

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

response = client.chat.completions.create(
    model="microsoft/florence-2-base-ft",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.caption/car.jpg"
                    },
                },
            ],
        }
    ],
    extra_body={"method": "caption"},
)

print(response.choices[0].message.content)

Response

The caption string alone, with no wrapper:

A cat sitting on a windowsill.