Medical Lie Detector

Detailed Instructions

Back to Home View Results

1. How the System Works

The Medical Lie Detector helps validate medical summaries against their original documents using advanced AI techniques:

  1. Document Upload - You provide both an AI-generated summary and the original medical document
  2. Question Generation - The system extracts key claims from the summary as true/false questions
  3. Context Search - For each question, the system finds relevant sections in the original document using both semantic and keyword searches
  4. Validation - Each claim is validated against the original document's content
  5. Result Display - Results are categorized as correctly supported or potentially inaccurate

2. Required API Keys

OpenAI API Key

Used for generating questions and validating claims. You'll need a GPT-capable API key.

  • Starts with sk-...
  • Create an account at OpenAI's platform
  • Generate a key in the API section of your account

Jina AI API Key

Used for generating embeddings to find semantically similar content between documents.

Note: Your API keys are only used for processing your current session and are not stored on our servers.

3. Uploading Documents

The system accepts the following file formats:

  • PDF documents - Both for the summary and original document
  • TXT files - Plain text format for either document
Important: For best results, ensure your original document is properly formatted and contains all the reference text.

4. Understanding Results

The validation results page shows both correctly validated statements and potential issues:

Result Interpretations:
  • Model: True - The claim is supported by the original document
  • Model: False - The claim is not supported or contradicted by the original document
Confidence Indicators:
  • Confident - High confidence that there is a meaningful discrepancy
  • I think the summary is wrong - Potential discrepancy detected, but it might be due to terminology or formatting differences
Example

Question: The patient was diagnosed with pneumonia on October 15, 2023.

Model: False Confident

Context from Original: "Patient was diagnosed with bronchitis on October 15, 2023, not pneumonia as initially suspected."

5. Troubleshooting

Common Issues:
  • API Key Errors - Ensure you're using the correct API keys and that they have not expired
  • Processing Time - Large documents may take longer to process, especially during embedding generation
  • Validation Quality - The quality of validation depends on how well the original document is structured
Tip: If a validation result seems incorrect, check the context segment and issue excerpt to understand the reasoning behind the determination.

© 2025 Medical Lie Detector. All rights reserved.