Lab 2: Faithfulness

Does every claim in the answer come from the context? Faithfulness = faithful_claims / total_claims

๐Ÿ”’
Faithfulness: Is the answer grounded in the context?
We extract every claim from the answer and check each one against the retrieved context. Claims not supported by context are hallucinations โ€” even if they sound plausible.
The Formula
Faithfulness = faithful_claims / total_claims
Each sentence in the answer is extracted as a claim and verified against the context
Panel 1 โ€” Retrieved Context
SECTION 2: WELLNESS DAYS Every NovaCorp employee receives 7.5 Wellness Days per quarter (non-transferable, non-encashable). Wellness Days cannot be combined with annual leave. Wellness Days must be approved by your direct manager at least 48 hours in advance. Wellness Days expire at the end of each quarter โ€” they do not roll over. SECTION 3: PATERNITY LEAVE NovaCorp offers 14 days of fully paid paternity leave. To apply, employees must submit Form W-77B to the Culture Team at hr-culture@novacorp.internal. Form W-77B must be submitted at least 2 weeks before the expected date of birth. Paternity leave must be taken within 3 months of the child's birth. Part-time employees are eligible for pro-rated paternity leave. SECTION 5: REMOTE WORK POLICY All employees at Level 3 and above may work remotely up to 3 days per week. Level 1 and Level 2 employees may work remotely 1 day per week after completing 6 months of service. Work-from-Mars Fridays are a special perk approved only for Level 6+ engineers โ€” full remote with no check-ins required. Remote work on public holidays is not counted as a remote day. SECTION 9: HEALTH AND FITNESS BENEFITS NovaCorp reimburses up to $500 per year for gym memberships and fitness classes. Claims must be submitted via the benefits portal with receipts within 30 days of payment. Eligible expenses include: gym memberships, yoga classes, swimming lessons, and sports equipment up to $100. Mental health counseling sessions are fully covered โ€” up to 12 sessions per year with any licensed counselor.
Panel 2 โ€” Answer (with claim highlights)
Q: What are the benefits of working at NovaCorp?
NovaCorp employees receive 7.5 Wellness Days per quarter, which are non-transferable and non-encashable. These days must be approved by your direct manager at least 48 hours in advance. Wellness Days expire at the end of each quarter and cannot be carried forward. NovaCorp also reimburses up to $500 per year for gym memberships and fitness classes, and covers up to 12 mental health counseling sessions per year.
Panel 3 โ€” Claim Breakdown
Run evaluation to see claim-by-claim breakdown
Faithful โ€” supported by context
Hallucinated โ€” not in context