Runnable protocolOn documents

Financial report analysis

With the report open at the right page, does a model do the first layer of a financial analyst's work?

Updated 8 October 2026 · Run 2026-10-07_analyse-financiere+2026-10-08_analyse-financiere · 27 models · 64 questions

Financial report analysisAccuracy × cost per test

Each square is a model, in its lab's colour and monogram. Click to open its page.

Key takeaways

7 models tie for first place at 100.0% accuracy: DeepSeek V4.1 Flash, Kimi K2.6, Qwen3.8 Max (0902), Claude Opus 5.5, Qwen3.8 Max Prime, Claude Fable 5.1 and GPT-6 Astra. This test does not separate them; the table orders them by price, then by speed.
Best accuracy for the price: Qwen3.7 Flash, 4.7 points behind the best score at $0.0003 per question — 3.5 times cheaper than DeepSeek V4.1 Flash.
25 of 27 models invent nothing: 0% hallucinations.
At the other end, Nova Pro 1.0 invents a value in 66.7% of cases where there was nothing to find.
The hardest field: “Compute a metric”, answered correctly 74.0% of the time on average across the models.

Which model to choose

If accuracy comes first

DeepSeek V4.1 Flash

100.0% accuracy

If volume matters

Qwen3.7 Flash

$0.0003 per question

If errors are costly

DeepSeek V4.1 Flash

0.0% invented items

The leaderboard

Click a header to re-sort. Accuracy orders the table; the other measures are never blended into it.

Financial report analysisFinance

Task

Rank	Model (27)					Input / output price
1	DeepSeek V4.1 Flash	100.0%±0.0	100.0%	0.0%	$0.0010	$0.3 / $1.2	3.5 s
2	Kimi K2.6	100.0%±0.0	100.0%	0.0%	$0.0052	$0.434 / $1.828	11.3 s
3	Qwen3.8 Max (0902)	100.0%±0.0	100.0%	0.0%	$0.0076	$2 / $6	7.2 s
4	Claude Opus 5.5	100.0%±0.0	100.0%	0.0%	$0.014	$4 / $20	4.8 s
5	Qwen3.8 Max Prime	100.0%±0.0	100.0%	0.0%	$0.015	$4 / $12	6.0 s
6	Claude Fable 5.1	100.0%±0.0	100.0%	0.0%	$0.030	$10 / $50	5.6 s
7	GPT-6 Astra	100.0%±0.0	100.0%	0.0%	$0.031	$10 / $50	3.9 s
8	GLM 5.3 Flash	98.4%±3.0	98.4%	0.0%	$0.0005	$0.15 / $0.5	3.1 s
9	Gemini 3.8 Flash	98.4%±3.0	98.4%	0.0%	$0.0035	$0.75 / $3.75	4.7 s
10	Claude Sonnet 5	98.4%±3.0	98.4%	0.0%	$0.0059	$2 / $10	2.9 s
11	Muse Spark 1.3	98.4%±3.0	98.4%	0.0%	$0.0061	$1.25 / $4.25	9.7 s
12	GPT-6 Sol	98.4%±3.0	98.4%	0.0%	$0.0067	$2 / $10	3.6 s
13	GPT-6 Luna	96.9%±4.3	96.9%	0.0%	$0.0003	$0.1 / $0.5	3.8 s
14	Qwen3.8 27B	96.9%±4.3	96.9%	0.0%	$0.0017	$0.42 / $3	9.5 s
15	Grok 4.7	96.9%±4.3	96.9%	0.0%	$0.010	$2 / $6	7.7 s

What this benchmark measures

The first layer of a financial analyst's work: turning a report into metrics and a judgement. The questions, written by analysts for FinanceBench, bear on the annual and quarterly reports of US listed companies: compute an EBITDA margin or a cash conversion cycle, say whether a margin is improving, which segment held growth back, or that a metric makes no sense for a bank. The model receives the relevant page or pages as images and the original question; it returns a number, a yes / no, or the name of a line item, compared with the answer a human analyst wrote, backed by the page of evidence. Questions calling for a written or opinion-based answer are left out: only a human could grade them.

Where the documents come from

150 analyst questions published by Patronus AI on the reports of US listed companies (annual, quarterly, earnings releases). Every answer was written by a human, with the page of evidence. We prepared 67: 50 drawn by a fixed rule among those whose answer can be checked automatically, then 17 added calculations. Three were dropped afterwards for their ambiguous scale; the table below says which and why. The sorting of all 150 is published line by line.

Source: FinanceBench — Patronus AI
Licence: CC BY-NC 4.0 · usage non commercial

Sample: 64 questions
What the model receives: Documents (images or PDF)
Models tested: 27
Protocol: Runnable protocol
Status of the figures: Real measurement
Run: 2026-10-07_analyse-financiere+2026-10-08_analyse-financiere

Where models pull apart

The average across all models on each graded field, from best mastered to hardest, and the best model on each.

Read a figure97.2%DeepSeek V4.1 Flash100.0%
Name a line item92.9%DeepSeek V4.1 Flash100.0%
Settle a fact87.7%DeepSeek V4.1 Flash100.0%
Compute a metric74.0%DeepSeek V4.1 Flash100.0%

The documents that split the models

These questions come from a public dataset: no difficulty was planted in them. They are the ones the models contradicted each other on most. Here are the question, the report page, the analyst's answer and what each model answered — enough to judge the scoring without taking our word for it.

Calculer un indicateur

17 of 27 models got it wrong · field: Calculer un indicateur

We want to calculate a financial metric. Please help us compute it by basing your answers off of the statement of income and the statement of cash flows. Here's the question: what is the FY2015 unadjusted EBITDA % margin for Netflix? Calculate unadjusted EBITDA using unadjusted operating income and D&A (from cash flow statement).

Report page for question fb-04458 — pages shown to the model: 2 · fb-04458

The analyst's answer: 5.4
Nova Lite 1.0: 45.9
Nova Pro 1.0: 20.1
Claude Fable 5.1: 5.4
Claude Opus 5.5: 5.43
Claude Sonnet 5: 6.6
Command A+: 5.43
DeepSeek V4.1 Flash: 5.43
Gemini 3.5 Flash Lite: 55.33
Gemini 3.8 Flash: 5.43
Gemma 4 31B: 10.1
Llama 4 Maverick: 22.7
Muse Spark 1.3: 56.83
Ministral 3 8B 2512: 4.8
Mistral Large 3 2512: 10.2
Mistral Medium 3.5: 56.83
Mistral Small 4: 4.51
Kimi K2.6: 5.43
Kimi K3: no usable answer
GPT-6 Astra: 5.43
GPT-6 Luna: 56.83
GPT-6 Sol: 5.43
Qwen3.7 Flash: 56.83
Qwen3.8 27B: 56.83
Qwen3.8 Max (0902): 5.43
Qwen3.8 Max Prime: 5.43
Grok 4.7: 56.83
GLM 5.3 Flash: 56.83

Calculer un indicateur

10 of 27 models got it wrong · field: Calculer un indicateur

What is the FY2019 cash conversion cycle (CCC) for General Mills? CCC is defined as: DIO + DSO - DPO. DIO is defined as: 365 * (average inventory between FY2018 and FY2019) / (FY2019 COGS). DSO is defined as: 365 * (average accounts receivable between FY2018 and FY2019) / (FY2019 Revenue). DPO is defined as: 365 * (average accounts payable between FY2018 and FY2019) / (FY2019 COGS + change in inventory between FY2018 and FY2019). Round your answer to two decimal places. Address the question by using the line items and information shown within the income statement and the balance sheet.

Report page for question fb-04103 — pages shown to the model: 2 · fb-04103

The analyst's answer: -3.7
Nova Lite 1.0: 100.02
Nova Pro 1.0: 10.91
Claude Fable 5.1: -3.7
Claude Opus 5.5: -3.7
Claude Sonnet 5: -3.7
Command A+: -3.7
DeepSeek V4.1 Flash: -3.7
Gemini 3.5 Flash Lite: 42.42
Gemini 3.8 Flash: -3.7
Gemma 4 31B: 35.1
Llama 4 Maverick: 34.97
Muse Spark 1.3: -3.7
Ministral 3 8B 2512: 52.74
Mistral Large 3 2512: 47.63
Mistral Medium 3.5: 30.54
Mistral Small 4: 23.34
Kimi K2.6: -3.7
Kimi K3: -3.7
GPT-6 Astra: -3.7
GPT-6 Luna: 16.3
GPT-6 Sol: -3.7
Qwen3.7 Flash: -3.71
Qwen3.8 27B: -3.7
Qwen3.8 Max (0902): -3.7
Qwen3.8 Max Prime: -3.7
Grok 4.7: -3.7
GLM 5.3 Flash: -3.7

Calculer un indicateur

9 of 27 models got it wrong · field: Calculer un indicateur

In agreement with the information outlined in the income statement, what is the FY2015 - FY2017 3 year average net profit margin (as a %) for Best Buy? Answer in units of percents and round to one decimal place.

Report page for question fb-02608 — pages shown to the model: 1 · fb-02608

The analyst's answer: 2.8
Nova Lite 1.0: 3.5
Nova Pro 1.0: 3.2
Claude Fable 5.1: 2.8
Claude Opus 5.5: 2.8
Claude Sonnet 5: 2.8
Command A+: 2.8
DeepSeek V4.1 Flash: 2.8
Gemini 3.5 Flash Lite: 2.7
Gemini 3.8 Flash: 2.8
Gemma 4 31B: 2.6
Llama 4 Maverick: 3.1
Muse Spark 1.3: 2.8
Ministral 3 8B 2512: 3.1
Mistral Large 3 2512: 3
Mistral Medium 3.5: 2.9
Mistral Small 4: 3.1
Kimi K2.6: 2.8
Kimi K3: 2.8
GPT-6 Astra: 2.8
GPT-6 Luna: 2.8
GPT-6 Sol: 2.8
Qwen3.7 Flash: 2.8
Qwen3.8 27B: 2.8
Qwen3.8 Max (0902): 2.8
Qwen3.8 Max Prime: 2.8
Grok 4.7: 2.8
GLM 5.3 Flash: 2.8

The questions asked

The 67 questions asked, from the most-missed to the most agreed-on: the company, report and page, the answer an analyst wrote as reference, how many models found it, and a link to the report where it is published. Those that fell out of the ranking are listed at the bottom, with their reason.

The questions asked
Question	Expected answer	Models correct	Invented value	Original
Netflix · NETFLIX_2015_10K · p. 40 ; 42 · Compute a metric We want to calculate a financial metric. Please help us compute it by basing your answers off of the statement of income and the statement of cash flows. Here's the question: what is the FY2015 unadjusted EBITDA % margin for Netflix? Calculate unadjusted EBITDA using unadjusted operating income and D&A (from cash flow statement). No usable answer from Kimi K3 — Aucune réponse écrite : le modèle a réfléchi jusqu'au plafond de jetons, deux fois de suite (8 000 puis 16 000 jetons), sans jamais écrire son JSON. Les 26 autres modèles ont tous répondu ; la réponse la plus longue reçue sur cette question compte 7 931 jetons (Kimi K2.6). Counted as “missing” for it alone: the question stays ranked for everyone else.	5.4	10 / 27	—	open
General Mills · GENERALMILLS_2019_10K · p. 53 ; 55 · Compute a metric What is the FY2019 cash conversion cycle (CCC) for General Mills? CCC is defined as: DIO + DSO - DPO. DIO is defined as: 365 * (average inventory between FY2018 and FY2019) / (FY2019 COGS). DSO is defined as: 365 * (average accounts receivable between FY2018 and FY2019) / (FY2019 Revenue). DPO is defined as: 365 * (average accounts payable between FY2018 and FY2019) / (FY2019 COGS + change in inventory between FY2018 and FY2019). Round your answer to two decimal places. Address the question by using the line items and information shown within the income statement and the balance sheet.	-3.7	17 / 27	—	open
Best Buy · BESTBUY_2017_10K · p. 56 · Compute a metric In agreement with the information outlined in the income statement, what is the FY2015 - FY2017 3 year average net profit margin (as a %) for Best Buy? Answer in units of percents and round to one decimal place.	2.8	18 / 27	—	open
Corning · CORNING_2021_10K · p. 65 · Compute a metric Taking into account the information outlined in the income statement, what is the FY2019 - FY2021 3 year average unadjusted operating income % margin for Corning? Answer in units of percents and round to one decimal place. No usable answer from Mistral Large 3 2512 — Deux réponses contradictoires dans le même texte : {"answer": 13.6}, puis {"answer": 10.6}, après un calcul qui passe par 10,28 et 10,3. Aucune des deux n'est juste (référence 10,3). Rejoué quatre fois, avec chaque fois des réponses différentes et contradictoires. Counted as “missing” for it alone: the question stays ranked for everyone else.	10.3	18 / 27	—	open
Nike · NIKE_2018_10K · p. 46 · Compute a metric We need to calculate a reasonable approximation (or exact number if possible) of a financial metric. Basing your judgment by information plainly provided in the statement of income, what is Nike's three year average of cost of goods sold as a % of revenue from FY2016 to FY2018? Answer in units of percents and round to one decimal place. No usable answer from Mistral Large 3 2512 — Deux réponses contradictoires dans le même texte : {"answer": 57.3}, puis après un recalcul {"answer": 55.2}. Rejoué plusieurs fois avec le même résultat. La dernière est juste, mais nous ne choisissons pas à la place du modèle. Counted as “missing” for it alone: the question stays ranked for everyone else.	55.1	18 / 27	—	open
Adobe · ADOBE_2022_10K · p. 54 · Settle a fact Does Adobe have an improving operating margin profile as of FY2022? If operating margin is not a useful metric for a company like this, then state that and explain why.	no	19 / 27	—	open
Activision Blizzard · ACTIVISIONBLIZZARD_2019_10K · p. 69 ; 70 · Compute a metric What is the FY2019 fixed asset turnover ratio for Activision Blizzard? Fixed asset turnover ratio is defined as: FY2019 revenue / (average PP&E between FY2018 and FY2019). Round your answer to two decimal places. Base your judgments on the information provided primarily in the statement of income and the statement of financial position.	24.26	19 / 27	—	open
PepsiCo · PEPSICO_2022_10K · p. 62 ; 64 · Compute a metric What is the FY2022 unadjusted EBITDA less capex for PepsiCo? Define unadjusted EBITDA as unadjusted operating income + depreciation and amortization [from cash flow statement]. Answer in USD millions. Respond to the question by assuming the perspective of an investment analyst who can only use the details shown within the statement of cash flows and the income statement.	9068.00	19 / 27	—	open
Block · BLOCK_2020_10K · p. 86 · Compute a metric What is the FY2019 - FY2020 total revenue growth rate for Block (formerly known as Square)? Answer in units of percents and round to one decimal place. Approach the question asked by assuming the standpoint of an investment banking analyst who only has access to the statement of income.	101.5	19 / 27	—	open
MGM Resorts · MGMRESORTS_2020_10K · p. 65 ; 67 · Compute a metric What is the FY2018 - FY2020 3 year average of capex as a % of revenue for MGM Resorts? Answer in units of percents and round to one decimal place. Please utilize information provided primarily within the statement of cash flows and the statement of income.	7.9	19 / 27	—	open
PepsiCo · PEPSICO_2022_10K · p. 62 ; 64 · Compute a metric What is the FY2022 unadjusted EBITDA % margin for PepsiCo? Calculate unadjusted EBITDA using unadjusted operating income and D&A (from cash flow statement). Give a response to the question by relying on the details shown in the statement of cash flows and the P&L statement.	16.5	19 / 27	—	open
Walmart · WALMART_2019_10K · p. 48 · Compute a metric Based on the information provided primarily in the statement of income, what is the FY2018 - FY2019 change in unadjusted operating income % margin for Walmart? Answer in units of percents and round to one decimal place.	0.2	19 / 27	—	open
Walmart · WALMART_2018_10K · p. 57 ; 59 · Compute a metric What is FY2018 days payable outstanding (DPO) for Walmart? DPO is defined as: 365 * (average accounts payable between FY2017 and FY2018) / (FY2018 COGS + change in inventory between FY2017 and FY2018). Round your answer to two decimal places. Please base your judgments on the information provided primarily in the statement of financial position and the P&L statement.	42.69	19 / 27	—	open
Amazon · AMAZON_2017_10K · p. 38 ; 40 · Compute a metric What is Amazon's FY2017 days payable outstanding (DPO)? DPO is defined as: 365 * (average accounts payable between FY2016 and FY2017) / (FY2017 COGS + change in inventory between FY2016 and FY2017). Round your answer to two decimal places. Address the question by using the line items and information shown within the balance sheet and the P&L statement.	93.86	19 / 27	—	open
Walmart · WALMART_2020_10K · p. 51 ; 56 · Compute a metric What is the FY2018 - FY2020 3 year average unadjusted EBITDA % margin for Walmart? Define unadjusted EBITDA as unadjusted operating income + depreciation and amortization from the cash flow statement. Answer in units of percents and round to one decimal place. Calculate what was asked by utilizing the line items clearly shown in the P&L statement and the cash flow statement. No usable answer from Mistral Large 3 2512 — Deux réponses contradictoires dans le même texte : {"answer": 5.2}, puis après un recalcul {"answer": 6.2}. Rejoué plusieurs fois avec le même résultat. La dernière est juste, mais nous ne choisissons pas à la place du modèle. Counted as “missing” for it alone: the question stays ranked for everyone else.	6.2	19 / 27	—	open
Corning · CORNING_2020_10K · p. 70 ; 72 · Compute a metric Based on the information provided primarily in the balance sheet and the statement of income, what is FY2020 days payable outstanding (DPO) for Corning? DPO is defined as: 365 * (average accounts payable between FY2019 and FY2020) / (FY2020 COGS + change in inventory between FY2019 and FY2020). Round your answer to two decimal places.	63.86	19 / 27	—	open
Lockheed Martin · LOCKHEEDMARTIN_2022_10K · p. 63 · Compute a metric What is Lockheed Martin's 2 year total revenue CAGR from FY2020 to FY2022 (in units of percents and round to one decimal place)? Provide a response to the question by primarily using the statement of income.	0.4	20 / 27	—	open
CVS Health · CVSHEALTH_2018_10K · p. 302 ; 304 · Compute a metric What is the FY2018 fixed asset turnover ratio for CVS Health? Fixed asset turnover ratio is defined as: FY2018 revenue / (average PP&E between FY2017 and FY2018). Round your answer to two decimal places. Calculate what was asked by utilizing the line items clearly shown in the P&L statement and the balance sheet.	17.98	20 / 27	—	open
Adobe · ADOBE_2016_10K · p. 62 · Compute a metric What is Adobe's year-over-year change in unadjusted operating income from FY2015 to FY2016 (in units of percents and round to one decimal place)? Give a solution to the question by using the income statement.	65.4	20 / 27	—	open
Activision Blizzard · ACTIVISIONBLIZZARD_2019_10K · p. 70 ; 73 · Compute a metric What is the FY2017 - FY2019 3 year average of capex as a % of revenue for Activision Blizzard? Answer in units of percents and round to one decimal place. Calculate (or extract) the answer from the statement of income and the cash flow statement.	1.9	20 / 27	—	open
JPMorgan · JPMORGAN_2021Q1_10Q · p. 19 · Name a line item Which of JPM's business segments had the lowest net revenue in 2021 Q1?	Corporate	21 / 27	—	open
JPMorgan · JPMORGAN_2023Q2_10Q · p. 85 · Settle a fact Looking at VaR, did the risk that JPM faced in the second fiscal quarter of 2023 decrease compared to the same period in the prior year?	yes	21 / 27	—	open
Nike · NIKE_2021_10K · p. 59 ; 61 · Compute a metric When primarily referencing the income statement and the statement of financial position, what is the FY2021 inventory turnover ratio for Nike? Inventory turnover ratio is defined as: (FY2021 COGS) / (average inventory between FY2020 and FY2021). Round your answer to two decimal places.	3.46	21 / 27	—	open
American Water Works · AMERICANWATERWORKS_2021_10K · p. 86 ; 88 · Compute a metric Basing your judgments off of the cash flow statement and the income statement, what is American Water Works's FY2021 unadjusted operating income + depreciation and amortization from the cash flow statement (unadjusted EBITDA) in USD millions?	1832.00	21 / 27	—	open
Lockheed Martin · LOCKHEEDMARTIN_2020_10K · p. 67 ; 69 · Compute a metric We need to calculate a reasonable approximation (or exact number if possible) of a financial metric. Basing your judgment by information plainly provided in the balance sheet and the P&L statement, what is Lockheed Martin's FY2020 asset turnover ratio? Asset turnover ratio is defined as: FY2020 revenue / (average total assets between FY2019 and FY2020). Round your answer to two decimal places.	1.33	21 / 27	—	open
Block · BLOCK_2016_10K · p. 68 · Compute a metric Considering the data in the balance sheet, what is Block's (formerly known as Square) FY2016 working capital ratio? Define working capital ratio as total current assets divided by total current liabilities. Round your answer to two decimal places.	1.73	21 / 27	—	open
Amazon · AMAZON_2017_10K · p. 38 · Compute a metric What is Amazon's year-over-year change in revenue from FY2016 to FY2017 (in units of percents and round to one decimal place)? Calculate what was asked by utilizing the line items clearly shown in the statement of income.	30.8	21 / 27	—	open
Microsoft · MICROSOFT_2023_10K · p. 60 · Settle a fact Has Microsoft increased its debt on balance sheet between FY2023 and the FY2022 period?	no	22 / 27	—	open
Johnson & Johnson · JOHNSON_JOHNSON_2022Q4_EARNINGS · p. 1 · Settle a fact Is growth in JnJ's adjusted EPS expected to accelerate in FY2023?	no	22 / 27	—	open
Amcor · AMCOR_2023_10K · p. 50 · Settle a fact Does AMCOR have an improving gross margin profile as of FY2023? If gross margin is not a useful metric for a company like this, then state that and explain why.	no	22 / 27	—	open
Lockheed Martin · LOCKHEEDMARTIN_2021_10K · p. 68 · Compute a metric What is Lockheed Martin's FY2021 net working capital? Define net working capital as total current assets less total current liabilities. Answer in USD millions. Respond to the question by assuming the perspective of an investment analyst who can only use the details shown within the balance sheet. No usable answer from Ministral 3 8B 2512 — Le modèle a écrit une soustraction à la place d'un nombre : {"answer": 18150 - 13997}, ce qui n'est pas du JSON valide et ne respecte pas la consigne (« un nombre décimal »). Le résultat de la soustraction, 4 153, n'aurait pas été juste non plus (référence 5 818). Rejoué : même réponse. Counted as “missing” for it alone: the question stays ranked for everyone else.	5818.00	22 / 27	—	open
Adobe · ADOBE_2017_10K · p. 57 ; 61 · Compute a metric What is the FY2017 operating cash flow ratio for Adobe? Operating cash flow ratio is defined as: cash from operations / total current liabilities. Round your answer to two decimal places. Please utilize information provided primarily within the balance sheet and the cash flow statement.	0.83	22 / 27	—	open
Adobe · ADOBE_2015_10K · p. 59 ; 63 · Compute a metric You are an investment banker and your only resource(s) to answer the following question is (are): the statement of financial position and the cash flow statement. Here's the question: what is the FY2015 operating cash flow ratio for Adobe? Operating cash flow ratio is defined as: cash from operations / total current liabilities. Round your answer to two decimal places.	0.66	22 / 27	—	open
Coca-Cola · COCACOLA_2021_10K · p. 62 · Compute a metric What is Coca Cola's FY2021 COGS % margin? Calculate what was asked by utilizing the line items clearly shown in the income statement.	39.7	22 / 27	—	open
Verizon · VERIZON_2022_10K · p. 77 · Settle a fact Has Verizon increased its debt on balance sheet between 2022 and the 2021 fiscal period?	no	23 / 27	—	open
3M · 3M_2022_10K · p. 25 · Name a line item If we exclude the impact of M&A, which segment has dragged down 3M's overall growth in 2022?	Consumer	23 / 27	—	open
American Express · AMERICANEXPRESS_2022_10K · p. 98 · Name a line item What was the largest liability in American Express's Balance Sheet in 2022?	Customer deposits	23 / 27	—	open
General Mills · GENERALMILLS_2020_10K · p. 52 · Compute a metric According to the information provided in the statement of cash flows, what is the FY2020 free cash flow (FCF) for General Mills? FCF here is defined as: (cash from operations - capex). Answer in USD millions.	3215.00	23 / 27	—	open
Kraft Heinz · KRAFTHEINZ_2019_10K · p. 50 ; 52 · Compute a metric What is Kraft Heinz's FY2019 inventory turnover ratio? Inventory turnover ratio is defined as: (FY2019 COGS) / (average inventory between FY2018 and FY2019). Round your answer to two decimal places. Please base your judgments on the information provided primarily in the balance sheet and the P&L statement.	6.25	23 / 27	—	open
MGM Resorts · MGMRESORTS_2023Q2_10Q · p. 11 · Name a line item Which type of debt received the largest investment among the short term investments for MGM in H1 FY2023?	corporate bonds	24 / 27	—	open
AMD · AMD_2022_10K · p. 48 · Name a line item From FY21 to FY22, excluding Embedded, in which AMD reporting segment did sales proportionally increase the most?	Data Center	24 / 27	—	open
General Mills · GENERALMILLS_2020_10K · p. 50 · Compute a metric By drawing conclusions from the information stated only in the statement of financial position, what is General Mills's FY2020 working capital ratio? Define working capital ratio as total current assets divided by total current liabilities. Round your answer to two decimal places.	0.68	24 / 27	—	open
Pfizer · PFIZER_2021_10K · p. 59 · Settle a fact Did Pfizer grow its PPNE between FY20 and FY21?	yes	25 / 27	—	open
Best Buy · BESTBUY_2024Q2_10Q · p. 17 · Settle a fact Was there any change in the number of Best Buy stores between Q2 of FY2024 and FY2023?	yes	25 / 27	—	open
American Express · AMERICANEXPRESS_2022_10K · p. 1 · Name a line item Which debt securities are registered to trade on a national securities exchange under American Express' name as of 2022?	none — there is none	25 / 27	2	open
PepsiCo · PEPSICO_2022_10K · p. 26 · Settle a fact Has Pepsico reported any materially important ongoing legal battles from FY2022 and FY2021?	no	25 / 27	—	open
AMD · AMD_2015_10K · p. 56 ; 60 · Compute a metric Answer the following question as if you are an equity research analyst and have lost internet connection so you do not have access to financial metric providers. According to the details clearly outlined within the P&L statement and the statement of cash flows, what is the FY2015 depreciation and amortization (D&A from cash flow statement) % margin for AMD?	4.2	25 / 27	—	open
Amcor · AMCOR_2020_10K · p. 50 · Read a figure What is Amcor's year end FY2020 net AR (in USD millions)? Address the question by adopting the perspective of a financial analyst who can only use the details shown within the balance sheet.	1616.00	25 / 27	—	open
3M · 3M_2018_10K · p. 58 · Read a figure Assume that you are a public equities analyst. Answer the following question by primarily using information that is shown in the balance sheet: what is the year end FY2018 net PPNE for 3M? Answer in USD billions.	8.70	25 / 27	—	open
JPMorgan · JPMORGAN_2022_10K · p. 3 · Settle a fact Are JPM's gross margins historically consistent (not fluctuating more than roughly 2% each year)? If gross margins are not a relevant metric for a company like this, then please state that and explain why.	not_applicable	26 / 27	—	open
Ulta Beauty · ULTABEAUTY_2023_10K · p. 1 · Name a line item Which debt securities are registered to trade on a national securities exchange under Ulta Beauty's name as of FY2023?	none — there is none	26 / 27	1	open
3M · 3M_2018_10K · p. 60 · Read a figure What is the FY2018 capital expenditure amount (in USD millions) for 3M? Give a response to the question by relying on the details shown in the cash flow statement.	1577.00	26 / 27	—	open
Nike · NIKE_2019_10K · p. 54 · Read a figure According to the details clearly outlined within the balance sheet, how much total current assets did Nike have at the end of FY2019? Answer in USD millions.	16525.00	26 / 27	—	open
Boeing · BOEING_2022_10K · p. 62 · Settle a fact Are there any product categories / service categories that represent more than 20% of Boeing's revenue for FY2022?	yes	27 / 27	—	open
Ulta Beauty · ULTABEAUTY_2023_10K · p. 57 · Name a line item What are major acquisitions that Ulta Beauty has done in FY2023 and FY2022?	none — there is none	27 / 27	—	open
AMD · AMD_2022_10K · p. 12 · Settle a fact Did AMD report customer concentration in FY22?	yes	27 / 27	—	open
Verizon · VERIZON_2021_10K · p. 85 · Name a line item Among all of the derivative instruments that Verizon used to manage the exposure to fluctuations of foreign currencies exchange rates or interest rates, which one had the highest notional value in FY 2021?	Cross currency swaps	27 / 27	—	open
Nike · NIKE_2023_10K · p. 62 · Name a line item Among operations, investing, and financing activities, which brought in the most (or lost the least) cash flow for Nike in FY2023?	operations	27 / 27	—	open
Best Buy · BESTBUY_2023_10K · p. 42 · Name a line item Among operations, investing, and financing activities, which brought in the most (or lost the least) cash flow for Best Buy in FY2023?	operations	27 / 27	—	open
AMD · AMD_2022_10K · p. 58 · Name a line item Among operations, investing, and financing activities, which brought in the most (or lost the least) cash flow for AMD in FY22?	operations	27 / 27	—	open
AES Corporation · AES_2022_10K · p. 132 · Read a figure What is the quantity of restructuring costs directly outlined in AES Corporation's income statements for FY2022? If restructuring costs are not explicitly outlined then state 0.	0	27 / 27	—	open
Costco · COSTCO_2021_10K · p. 38 · Read a figure Using only the information within the balance sheet, how much total assets did Costco have at the end of FY2021? Answer in USD millions.	59268.00	27 / 27	—	open
Best Buy · BESTBUY_2019_10K · p. 52 · Read a figure What is the year end FY2019 total amount of inventories for Best Buy? Answer in USD millions. Base your judgments on the information provided primarily in the balance sheet.	5409.00	27 / 27	—	open
Microsoft · MICROSOFT_2016_10K · p. 52 · Read a figure What is the FY2016 COGS for Microsoft? Please state answer in USD millions. Provide a response to the question by primarily using the statement of income.	32780.00	27 / 27	—	open
Coca-Cola · COCACOLA_2017_10K · p. 74 ; 76 · Compute a metricdropped What is the FY2017 return on assets (ROA) for Coca Cola? ROA is defined as: FY2017 net income / (average total assets between FY2016 and FY2017). Round your answer to two decimal places. Give a response to the question by relying on the details shown in the balance sheet and the P&L statement. Dropped after the run — Échelle ambiguë : la référence est le ratio arrondi à deux décimales (0,01), alors que la valeur exacte est 0,01425, soit 1,425 %. 21 modèles sur 23 ont répondu à l'échelle du pourcentage (de 0,14 à 1,62, la plupart autour de 1,42), comme la consigne du prompt l'illustre (« 12,5 % s'écrit 12.5 ») ; seuls Claude Opus 5.5 et Claude Fable 5.1 ont écrit 0,01. Cette question mesure le choix de l'échelle, pas le calcul. Écartée après le run, sans remplacement.	0.01	—	—	open
General Mills · GENERALMILLS_2022_10K · p. 45 ; 49 · Compute a metricdropped We want to calculate a financial metric. Please help us compute it by basing your answers off of the cash flow statement and the income statement. Here's the question: what is the FY2022 retention ratio (using total cash dividends paid and net income attributable to shareholders) for General Mills? Round answer to two decimal places. Dropped after the run — Échelle ambiguë : la référence est le ratio arrondi à deux décimales (0,54), soit environ 54 %. Sur les 17 modèles qui ont répondu, 8 ont écrit la valeur à l'échelle du pourcentage (de 53,66 à 54,11), comme la consigne du prompt l'illustre (« 12,5 % s'écrit 12.5 »), et 3 seulement ont écrit 0,54. Même défaut que les deux questions de rendement des actifs. Le taux de distribution des dividendes, dont celle-ci est le complément, avait été écarté avant le run ; le taux de rétention nous avait échappé au tri. Écartée après le run, sans remplacement.	0.54	—	—	open
AES Corporation · AES_2022_10K · p. 130 ; 132 · Compute a metricdropped Based on the information provided primarily in the statement of financial position and the statement of income, what is AES's FY2022 return on assets (ROA)? ROA is defined as: FY2022 net income / (average total assets between FY2021 and FY2022). Round your answer to two decimal places. Dropped after the run — Échelle ambiguë : la référence est le ratio arrondi à deux décimales (−0,02), alors que la valeur exacte est −0,01531, soit −1,531 %. 22 modèles sur 26 ont répondu à l'échelle du pourcentage (de −0,31 à −1,53, la plupart autour de −1,42), comme la consigne du prompt l'illustre ; seuls Claude Opus 5.5, Claude Fable 5.1 et Grok 4.7 ont écrit −0,02. Cette question mesure le choix de l'échelle, pas le calcul. Écartée après le run, sans remplacement.	-0.02	—	—	open

How these figures are produced — the scoring grid, the exact prompt, and what this test does not measure.