Harvard releases million books to train AI models

Harvard University has opened up nearly one million digitised books for use in training artificial intelligence models – a move likely to be closely watched by publishers as they negotiate with or sue AI companies over access to copyrighted content.

The dataset, unveiled through Harvard’s Institutional Data Initiative, includes around 394 million pages and an estimated 242 billion tokens, making it one of the largest public domain corpora available for AI research. The collection spans texts from the 15th century onwards, covering more than 250 languages, with a particular concentration of material from the 19th century.

It marks the first time Harvard has extracted public domain texts from Google Books specifically for AI training, a shift in purpose that comes nearly two decades after its original digitisation project with Google, which faced legal challenges at the time but was ultimately allowed to continue following a U.S. Supreme Court decision.

The new initiative is designed to address growing concerns about the quality and legality of the data used to train large language models. “A lot of the data that’s been used in AI training has not come from original sources,” said Greg Leppert, executive director of the project, suggesting that access to curated, historically rich texts could improve the accuracy and integrity of future models.

The move is also strategic. Tech companies including Microsoft and OpenAI have contributed funding, keen to access lawful alternatives to web-scraped data that may include copyrighted material. As lawsuits mount against firms accused of using protected content without permission Harvard’s public domain corpus offers a cleaner option with fewer legal risks.

For publishers, the project underlines an emerging battleground: control over high-quality source material. While many organisations are currently pursuing compensation through licensing deals or the courts, the Harvard dataset could dilute some of that leverage if AI developers increasingly rely on large-scale public archives.

Still, there are limitations. A significant portion of the material is out of date or potentially harmful. “There are real risks in exposing AI models to historic content that may contain embedded biases,” said Kristi Mukk, a coordinator at Harvard’s Library Innovation Lab. The initiative plans to provide guidance on responsible use, though the scale and scope of the collection will pose challenges for any moderation effort.

The project is also framed as a way to redistribute power in AI development. “We’re trying to move some of the power from this current AI moment back to these institutions,” said Aristana Scourtas, also from the Library Innovation Lab. That includes working with libraries and cultural institutions worldwide to ensure the benefits of the dataset flow back to the communities that preserved these works.

The linguistic diversity of the collection is striking – fewer than half of the books are in English – and its availability on platforms such as Hugging Face is intended to encourage open experimentation. The hope is that developers will use it not just to build more effective AI systems, but ones that better reflect the breadth of human knowledge and experience.

Source: Noah Wire Services

Noah Fact Check Pro

The draft above was created using the information available at the time the story first
emerged. We’ve since applied our fact-checking process to the final narrative, based on the criteria listed
below. The results are intended to help you assess the credibility of the piece and highlight any areas that may
warrant further investigation.

Freshness check

Score:
10

Notes:
The narrative is fresh, with no evidence of prior publication. The earliest known publication date of similar content is June 12, 2025, as reported by the Associated Press. ([apnews.com](https://apnews.com/article/e096a81a4fceb2951f232a33ac767f53?utm_source=openai)) The report is based on a press release from Harvard’s Library Innovation Lab, which typically warrants a high freshness score. No discrepancies in figures, dates, or quotes were found. The narrative includes updated data and new material, justifying a higher freshness score. No recycled content or republishing across low-quality sites was identified. The report is original and exclusive.

Quotes check

Score:
10

Notes:
The quotes from Greg Leppert and Aristana Scourtas are unique to this report. No identical quotes appear in earlier material. The wording matches the original sources, with no variations found. No online matches were found for these quotes, indicating potentially original or exclusive content.

Source reliability

Score:
10

Notes:
The narrative originates from the Associated Press, a reputable organisation known for its journalistic standards. The report is based on a press release from Harvard’s Library Innovation Lab, a credible source. All individuals and organisations mentioned, including Greg Leppert, Aristana Scourtas, Microsoft, and OpenAI, have verifiable public presences and legitimate websites.

Plausability check

Score:
10

Notes:
The claims about Harvard releasing nearly one million digitised books to AI researchers are plausible and supported by the report. The narrative is covered by other reputable outlets, including the Associated Press. ([apnews.com](https://apnews.com/article/e096a81a4fceb2951f232a33ac767f53?utm_source=openai)) The report includes specific factual anchors, such as the number of books, languages, and the involvement of Microsoft and OpenAI. The language and tone are consistent with the region and topic. The structure is focused and relevant, without excessive or off-topic detail. The tone is formal and appropriate for a corporate or official announcement.

Overall assessment

Verdict (FAIL, OPEN, PASS): PASS

Confidence (LOW, MEDIUM, HIGH): HIGH

Summary:
The narrative is fresh, original, and based on a credible source. All claims are plausible and supported by specific details. No signs of disinformation or recycled content were found.