OpenAI, HealthBench, Claude and HIPAA Compliance: What Healthcare IT Needs to Know

Artificial intelligence is rapidly transforming the healthcare industry, and major technology companies are now introducing specialized AI tools designed specifically for healthcare organizations. Companies such as OpenAI, Google and Anthropic have recently launched healthcare-focused AI solutions, which are expected to influence how hospitals and health systems make decisions about purchasing and implementing AI technologies.

One of the major developments is OpenAI’s HealthBench, a benchmarking system designed to evaluate how well AI models perform in healthcare-related tasks. HealthBench measures AI performance using thousands of evaluation criteria created by physicians, focusing on areas such as medical accuracy, clarity of responses and completeness of recommendations. While HealthBench shows that AI performance in healthcare is improving, experts say benchmarks alone are not enough. Healthcare organizations must also test AI tools in real-world environments before fully adopting them.

In addition to benchmarking tools, companies are also releasing enterprise-level AI platforms for hospitals. Anthropic’s Claude for Healthcare can integrate with healthcare databases and systems and can help automate administrative tasks such as prior authorization and healthcare data exchange. Google’s Gemini platform focuses on multimodal capabilities, meaning it can process different types of healthcare data such as text, images, voice and medical scans. OpenAI for Healthcare includes tools such as ChatGPT for Healthcare, which helps healthcare staff summarize medical information, generate discharge summaries and assist with administrative tasks.

All of these AI platforms are designed to help healthcare organizations manage large amounts of medical data and reduce administrative workload. However, when healthcare organizations consider adopting these tools, one of the biggest concerns is HIPAA compliance and data security. Experts explain that no AI tool can automatically be considered HIPAA compliant. Instead, compliance depends on how the organization implements the tool, what data is shared and what security controls are in place.

Some AI companies offer Business Associate Agreements (BAAs), which are required when handling protected health information under HIPAA. OpenAI, Google and Anthropic all provide BAAs under certain enterprise agreements, which helps healthcare organizations use AI tools while maintaining compliance requirements. However, healthcare organizations must still ensure proper governance, security controls and risk management when deploying these AI solutions.

Another important development is ChatGPT Health, a consumer-focused AI tool that allows individuals to connect their health records and wellness data to get personalized health insights. While this tool is designed for patients and not directly for hospitals, it may still impact healthcare organizations. As more patients begin using AI tools to manage their health information, healthcare providers may need to adapt their digital strategies to maintain patient engagement and trust.