Microsoft’s Copilot AI now understands your terrible handwriting

Microsoft is preparing to enable Copilot in OneNote to read and analyze handwritten notes. The feature entered beta testing late last month, and will allow OneNote users to write handwritten notes using a stylus and then summarize them, ask questions, or even generate to-do lists based on the notes.

The AI-powered Copilot feature in OneNote will even be able to convert your handwritten notes into text for easy editing and sharing. Microsoft first launched Copilot inside OneNote in November, and this upgrade will be available to existing Copilot for Microsoft 365 subscribers and Copilot Pro users once it rolls out more broadly.

I’ve been briefly testing Copilot’s ability to read my handwriting, and I’m impressed that it’s even able to decipher it. Summarization works well across small handwritten notes and even larger ones. I asked Copilot to rewrite an entire paragraph of handwritten notes which it managed to format into easy to read text that was true to the original but also a bit more jovial. That’s impressive given generative AI models have a tendency to just make stuff up half the time.

If you use OneNote for handwritten to-do lists, then this feature definitely makes it easier to convert those into text at a later stage. I created a handwritten list and Copilot was able to accurately convert this into a text list within seconds.

I’m not sure how Copilot will handle even worse handwriting, though. Samsung’s Galaxy AI has a similar feature that can automatically format handwritten lists, but it struggles with some of the worst handwriting I’ve ever seen (sorry, Allison!).

If you want to test Copilot’s handwritten note recognition you’ll need to be a Microsoft 365 Insider running the latest OneNote on Windows build (17628.20006 or later) and have a Copilot Pro or Copilot for Microsoft 365 subscription.