This site now has a chat page where you can ask questions about me and my research. That is not particularly remarkable in itself; chatbots are everywhere these days. What I find more interesting is that this one runs entirely in your browser. There is no server involved, no API keys, and no data leaving your machine. Your questions are answered on your own hardware, by a language model that your browser downloads and runs locally. Here are some reflections on how it is set up and what I have learned along the way.
Why local?
I have been curious about how far “local AI” has come. Most chatbots send everything you type to a data centre somewhere. That is convenient, but it also means that someone else stores and processes your questions. For a personal website, I find it much more appealing that the whole thing runs client-side. It is also a nice demonstration of how capable browsers have become. The fact that a laptop can download a multi-billion-parameter language model and run it inside a web page would have sounded like science fiction only a few years ago.
The building blocks
The chat is built on WebLLM, a project that compiles language models to run on WebGPU, the new browser standard for talking directly to the graphics card. When you open the chat page and click “load”, the browser downloads a quantised model and runs it on your GPU. The default is Meta’s Llama 3.2 3B, a small model by today’s standards, but good enough for conversational questions. The first load is a couple of gigabytes, which the browser caches, so subsequent visits are much faster. There is also a smaller 1B model for less powerful machines and Phi 3.5 Mini for those who want to experiment.
A small language model on its own does not know much about me, and what it thinks it knows, it happily makes up. The solution is called retrieval-augmented generation (RAG): before the model answers, the page searches a local knowledge base for text passages relevant to the question and passes them along as context. The model then writes an answer grounded in that material rather than in its own imagination.
The knowledge base is a single JSON file that the page downloads alongside the model. A Python script on my machine builds it from all the blog posts and pages on this site, in both English and Norwegian, as well as PDFs of my publications. The text is split into overlapping chunks of a couple of hundred words each, and every chunk is indexed with TF-IDF weights. That is an old-school information retrieval technique, no neural embeddings involved, but it is fast, transparent, and runs comfortably in JavaScript. When you ask a question, the browser scores all nine thousand or so chunks against it, picks the best few from different sources, and hands them to the model. A “Sources” list appears under each answer to show where the material came from.
Lessons learned
Getting this to work taught me a few things. The first was about WebGPU itself. On Linux, both Chrome and Firefox still ship with WebGPU disabled by default, even on machines with capable graphics cards. The browser reports that the API exists but then refuses to use the GPU adapter, which makes debugging confusing. If the chat tells you that no GPU adapter was found, the page now explains which browser flags to flip.
The second lesson was about the knowledge base. My first version tried to be clever: for simple factual questions it skipped the language model entirely and pasted the best-matching sentence straight from the source. That was efficient but produced terrible answers, raw lowercase fragments of PDF text, complete with hyphenation artefacts. Now the model always writes the answer, and the retrieved chunks are only its source material.
The third lesson was about knowing your corpus. When I first indexed my publication PDFs, the chatbot confidently told me that I study electoral quotas in India. It took me a moment to realise what had happened: the folder of PDFs had been collected by searching for my surname, so it included a political science dissertation by my sister and papers on medieval wooden churches by my father. The folder also contained dozens of duplicate PDFs of my own papers under different file names, which skewed the retrieval towards whatever happened to exist twice. Cleaning that up, and adding automatic duplicate detection to the indexing script, improved the answers considerably. This was a reminder that the quality of the data matters more than the sophistication of the algorithm.
I did much of this debugging and refinement together with Claude Code, which feels fitting: an AI in the cloud helping me build an AI that avoids the cloud.
Try it
The chat works best in a recent Chrome or Edge on a machine with a decent GPU. Ask it about musical gestures, micromotion, or standstill, and it should give you grounded answers with sources. Ask it about electoral quotas in India, and it should now, finally, admit that it does not know.