NotebookLM: hands

Jul 29, 2023

By David Pierce, editor-at-large and Vergecast co-host with over a decade of experience covering consumer tech. Previously, at Protocol, The Wall Street Journal, and Wired.

What if you could have a conversation with your notes? That question has consumed a corner of the internet recently, as companies like Dropbox, Box, Notion, and others have built generative AI tools that let you interact with and create new things from the data you already have in their systems.

Google’s version of this is called NotebookLM. It’s an AI-powered research tool that is meant to help you organize and interact with your own notes. (Google originally announced it earlier this year as Project Tailwind but quickly changed the name.) Right now, it’s really just a prototype, but a small team inside the company has been trying to figure out what an AI notebook might look like. Ultimately, if NotebookLM sticks around, it’ll probably be as a feature of Google Docs or a tool inside of Drive that can ingest and understand all your files. But for now, it’s its own extremely barebones app.

I’ve been using NotebookLM for the last couple of weeks, both testing out the app’s capabilities and trying to figure out where an AI research tool fits into my own workflows. I’m not sure I’ve found the right answers yet, and I’m not sure the tech is quite ready, either. But I’m increasingly convinced that a personalized AI, trained on all the stuff I care about and very little else, is going to be a seriously powerful thing.

To get started with NotebookLM, you create a new project. In my case, I’ve been doing a bunch of research on the history and culture of spreadsheets, so I called my project “Spreadsheet history.” (Clever, right?) The app then prompted me to begin adding sources — right now, it only accepts and imports Google Docs, but in its finished form, it will apparently take in many other kinds of information. Each project can have up to five sources, and each source can be up to 10,000 words long, but I only know that because someone at Google told me. If you try to import too many or too large sources, NotebookLM just sort of quietly fails. The app is new enough that the team hasn’t even customized the error messages. Again: prototype.

But after a bit of trial and error, I got my sources in: a version of Steven Levy’s seminal “A Spreadsheet Way of Knowledge” story, a series of blog posts from VisiCalc creator Dan Bricklin, a section from Laine Nooney’s book The Apple II Age, and a few thousand words of other compiled research. As I imported each source, NotebookLM generated what it calls a “Source Guide,” with a paragraph summarizing the doc and then a list of key topics and suggested questions to ask. In general, the guides were very good: for that Levy story, it surfaced “Electronic spreadsheet,” “VisiCalc,” “Lotus 1-2-3,” “Spreadsheet modeling,” and “Spreadsheets and decision-making.” Three of those are variations of the same thing, but hey, it’s a long article about spreadsheets. Most of the key topics are going to be spreadsheets.

The whole reason NotebookLM exists is to give you a new way to interact with docs

The whole reason NotebookLM exists is to give you a new way to interact with these docs. Rather than a tool for organizing or enhancing your research, it’s essentially a chatbot trained specifically on the sources you’ve provided that can either reference them one at a time or all at once. In the Levy example, one of the source guide’s suggested questions was, “What are the advantages of using spreadsheets?” When I asked this of the NotebookLM chatbot, it thought for a second and then came back with five attributes that made spreadsheets so powerful for early computer users. The answer wasn’t based on the whole internet; it was based entirely on the 5,000 or so words I’d pasted into that Google doc.

(Tiny aside: There are obviously huge privacy issues and concerns with all these personalized AI tools. But in this case, I’m not that worried about it — all this data already lives in other Google products anyway, so I’m not convinced that having a large language model parse it is a meaningfully different thing. But as with all things AI, you should always think carefully about where your data is going and how it might be used.)

You can also, of course, ask any other question you can think of. I eventually started using NotebookLM mostly to find commonalities across things. Who are the people that come up most often in all these documents? What links are referenced most often? What are the main competitors in this space? With a few questions, I can usually get a decent set of jumping-off points for more research. You can also ask things like “What’s the most surprising information in here?” and get sometimes interesting examples. NotebookLM can also generate summaries or outlines of documents to make skimming your research a bit easier.

Along with every answer, NotebookLM provides citations. They’re not sources, exactly, since the underlying model isn’t just searching for and returning text; they’re more like points on a map, the 10 bits of text that NotebookLM deemed most relevant to the question and then synthesized and used in order to provide an answer. NotebookLM told me “Speed” was a crucial advantage of spreadsheets not because Levy wrote that but because he quoted a bunch of executives talking about the things they were able to do with this radically faster tool.

In my experience so far, the connection between citation and answer is sometimes obvious and sometimes deeply confusing, but I like the approach of the model trying to show its work. And for the most part, I’ve found the citations to be vastly more useful than the answers themselves; the actual synthesis and answering that NotebookLM does is somewhat unreliable, but it does a really good job of identifying the bits of information that are relevant to my question.

Raiza Martin, the product manager in charge of NotebookLM, says my experience seems to match other NotebookLM users. “The source guide and citations are the two top features that get called out the most,” she says. “We’re also seeing behavior change, where more and more people are like, ‘Oh, I have to read something, so I put it into Notebook so I can generate the source guide.” (Everyone at Google seems to call NotebookLM just “Notebook.” Take from that what you will.)

Ultimately, Martin says, part of what she’s looking for is to see how people interact with bots differently when they’re trained on personal data and not the internet. “When we change the context sufficiently, does it change user behavior?” she asks. “And what we’re discovering is that it does.” Users are doing more targeted investigating and probing of information, it seems, rather than just asking blue-sky questions of the AI.

Speaking of the internet: one odd quirk about NotebookLM is that it actually does know things that aren’t in your documents. At one point, I asked for information about an old Excel competitor that was referenced in one document I’d uploaded, but only by name and with no other information, and NotebookLM spat back some basic information about when it was founded and what it did. My documents didn’t know this! What gives?

Steven Johnson, a longtime author and the editorial director of Google Labs working on NotebookLM, says the team has wrestled with what to do in these situations. “There’s certainly some general-knowledge things that the model knows that are actually quite accurate.” How true that is, and how to show that process to users, is the ongoing question. “We spent a lot of time fine-tuning so that the model will say, ‘I’m sorry, that information is not in your source,” Johnson says, and this kind of humility and transparency is a good thing in an LLM. But it should also try and help when it can, right? “We’re trying to figure out how much of that is sort of us blending that,” Martin says, “and making it clear to the user that, ‘Hey, it’s not in your sources, but here is some general knowledge or knowledge from the web.’”

The long-term answer might be to just plug NotebookLM into Keep or Docs

In addition to improving the model and working on interactions, Google’s other big project for NotebookLM is to make it a better app for actually taking notes. Right now, you get a super-simple scratchpad in case you want to copy and paste a chatbot answer or jot down a thing you remembered, but that’s really it. The long-term answer there might be to just plug NotebookLM into Keep or Docs, but NotebookLM might also turn into a more full-fledged notes app over time as well. (Given Google’s penchant for launching a thousand versions of the same kind of product, I’m betting on that last outcome.)

After a few weeks of using NotebookLM, it hasn’t totally upended the way I do everything. But I absolutely buy the idea that there are better ways to interact with notes than a lot of organizational busywork and keyword searches. And it seems clear that if Google can figure out how to make NotebookLM work reliably both with my stuff and the broader web and interact with everything else Google already knows about me, this could be the most powerful and personal chatbot on the internet. We’re definitely still in the prototype phase of all that, but it’s building toward something potentially huge.

/ A weekly newsletter by David Pierce designed to tell you everything you need to download, watch, read, listen to, and explore that fits in The Verge’s universe.

/ Sign up for Verge Deals to get deals on products we've tested sent to your inbox daily.