AuthonAuthon Blog
tutorial8 min read

How to Run an Offline AI Assistant With No Internet (Even on a Raspberry Pi)

Survival LLM missed 8 of 22 emergency questions on a 16 GB machine. Build your own kit with Ollama and Kiwix, with disk and RAM for each tier.

AW
Alan West
Authon Team
How to Run an Offline AI Assistant With No Internet (Even on a Raspberry Pi)
Survival LLM, the offline AI assistant Hacker News spent this week arguing about, got 14 of 22 blackout and first-aid questions right on a 16 GB machine. The other eight missed the main point of the official guidance, and these were emergency questions.

It's free, MIT-licensed, and bundles a local model with an offline emergency library so you can ask questions with no internet. I like the idea. I also think the library is the half that matters and the model is mostly a search box on top of it. Below I go through what Survival LLM ships and what hardware it wants, then how to build the same kind of offline kit on Linux with Ollama or llama.cpp plus Kiwix, including on a Raspberry Pi, which Survival LLM doesn't support.

What Survival LLM actually ships

The app is small (212 MB on Mac, 25 MB on Windows). On first run it checks how much memory you have, picks a model and downloads it. These are the published tiers from the project page:

| Your machine | Model | Emergency test score | Model download |
|---|---|---|---|
| Mac, 128 GB+ | NVIDIA Nemotron 3.5 Super | 21 / 22 | 80 GB |
| Mac, 32–96 GB | Qwen3.8 27B | 21 / 22 | 18 GB |
| Windows PC, 32 GB+ | NVIDIA Nemotron 3.5 Lightning | 19 / 22 | 25 GB |
| Mac or PC, 16–31 GB | IBM Granite 4.2 8B | 14 / 22 | 6 GB |

The score counts answers that got the main point of official guidance right on 22 blind-graded questions. The page notes the tests ran against an earlier build of the library.

The library is the interesting part. The full version is a 9 GB download (19 GB installed) of plain-text, searchable copies of:

  • about 930,000 of Wikipedia's most-read articles
  • WikiMed, MedlinePlus, UK Medicines A to Z and WikEM (emergency medicine)
  • Ready.gov (FEMA) guides, CDC travelers' health, and U.S. military medicine and survival manuals
  • wikiHow, iFixit, Appropedia
  • Stack Exchange Q&A on home repair, cooking, gardening, outdoors and ham radio

There's also a 1.3 GB "smaller library" (3 GB installed) that drops Wikipedia and iFixit. For disk space, plan on the model size plus 5 GB, and about 27 GB of temporary space while the full library installs. You can add your own PDFs, ebooks and notes, and each answer shows its sources. Version 0.3.4 (Oct 27, 2026) added a reasoning budget setting: Off, 1,000, 4,000, 16,000 tokens, or no limit.

Platform support is Apple Silicon on macOS 14+ or 64-bit Windows 10/11. There's no Linux build, which got the expected grumbling in the HN thread.

Power: about 100 W per answer, 100 to 300 answers per charge

The project publishes this, which I appreciate. On a MacBook Pro it draws about 100 W while answering, for 5 to 40 seconds per question, which works out to roughly 100 to 300 answers on a full battery.

At 20 questions a day that's 5 to 15 days per charge, and recharging is the hard part when the grid is down. One HN commenter asked whether you'd really run your generator for a local model. Plain Kiwix, by comparison, is a web server reading compressed files, and it sips power next to a GPU doing inference.

Why the LLM shouldn't be the only thing on the drive

The HN thread splits in two. One side treats the LLM as search: it finds the article when you don't know the keyword. The other asks what it adds over a plain index, which is smaller and faster and can't invent anything. I think both sides have a point. One commenter asked how to build a bow drill and got an answer that was partly wrong about how the bow works. Another, running Qwen 27B on a 32 GB M1 (about 19 to 23 GB used), noticed that answers the model pulled from its own memory came with no sources, and that it ran a real search once told to "look that up".

So my rule is to use the model to find the page, then read the page. The library has to be something you can browse without the model running. Kiwix gives you that, and it's what I'd build around.

Build your own offline AI kit on Linux: Ollama plus Kiwix

You need two pieces. For the model runtime, Ollama is the easy path, and llama.cpp is there if you want fewer layers between you and the GGUF file. For the library, use Kiwix ZIM files served by kiwix-serve. Most of what Survival LLM bundles (Wikipedia, WikiMed, iFixit, Appropedia) is available as ZIM files: compressed, searchable archives, with or without images.

Sizes from the Kiwix mirror as of this month:

| ZIM file | Size |
|---|---|
| wikipedia_en_all_maxi_2026-08.zim (full, with images) | 119 GB |
| wikipedia_en_all_nopic_2026-06.zim (full, no images) | 49 GB |
| wikipedia_en_all_mini_2026-09.zim (intros only) | 13 GB |
| wikipedia_en_top_maxi_2026-09.zim (top articles) | 6.5 GB |
| wikipedia_en_medicine_maxi_2026-10.zim | 1.7 GB |
| mdwiki_en_all_maxi_2025-11.zim (WikiMed) | 2.1 GB |
| ifixit_en_all_2025-12.zim | 3.3 GB |
| zimgit-post-disaster_en_2024-05.zim | 615 MB |
| appropedia_en_all_maxi_2026-02.zim | 555 MB |
| zimgit-water_en_2024-08.zim | 20 MB |

Disk and RAM by tier, from Raspberry Pi 5 to a 32 GB laptop

Model sizes are from the Ollama library pages for Granite 4.2 and Gemma 4. Gemma 4 12B was the model HN commenters suggested for modest hardware.

| Tier | Model (Ollama tag, size) | Library | Total disk |
|---|---|---|---|
| Raspberry Pi 5, 16 GB | granite4.2:3b, 2.2 GB | top_maxi + WikiMed + iFixit + post-disaster + Appropedia | ~15 GB |
| Old laptop, 16 GB | granite4.2:8b, 5.3 GB | full nopic Wikipedia + same extras | ~61 GB |
| Mini PC or laptop, 32 GB | gemma4:26b, 16–19 GB | full maxi Wikipedia + same extras | ~145 GB |

The 16 GB row uses the same Granite 8B that Survival LLM picks for that tier, so expect roughly the same 14/22 accuracy. That's another reason to keep the articles one click away.

On the Pi, be realistic. One Pi 5 benchmark under Ollama measured qwen2.5:3b at 5.2 tokens/s and llama3.2:3b at 4.69 tokens/s, both Q4_K_M. Those same authors set OLLAMA_LOAD_TIMEOUT=60m0s because some models load that slowly. Another write-up works out a theoretical ceiling of about 8.6 tokens/s for Llama 3.2 3B at Q4_K_M from the Pi 5's 17.1 GB/s memory bandwidth. It also notes that inference pins all four cores at 100%, so get the active cooler. Either way, on a Pi the model is the slow extra and Kiwix is the product.

Setup commands (do all of this while you still have internet)

Install Ollama and pull a model:

bash
curl -fsSL https://ollama.com/install.sh | sh
ollama pull granite4.2:8b
ollama run granite4.2:8b

Or with llama.cpp, using the commands from its current README:

bash
curl -LsSf https://llama.app/install.sh | sh
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF   # swap in the GGUF repo you want
llama serve gives you an OpenAI-compatible API and a built-in web UI. Run it once while you're online so the weights are already on disk.

Then the library. Install kiwix-tools from your distro (Debian and Ubuntu package it), download your ZIMs, and serve them:

bash
sudo apt install kiwix-tools
mkdir -p ~/zim && cd ~/zim
wget https://download.kiwix.org/zim/wikipedia/wikipedia_en_all_nopic_2026-06.zim
wget https://download.kiwix.org/zim/ifixit/ifixit_en_all_2025-12.zim
kiwix-serve -p 8080 ~/zim/*.zim

Open http://localhost:8080 and you have searchable offline Wikipedia and iFixit. Any phone on the same Wi-Fi can reach it too, which is the main thing a Pi tier gets you.

If you'd rather not wire it up yourself, Project NOMAD (Apache 2.0) bundles Kiwix, Ollama, OpenStreetMap offline maps and Kolibri in a Docker install on Ubuntu or Debian. It recommends 32 GB RAM and a 1 TB SSD, though, so it's not the "old laptop" route.

The pre-outage checklist

  • Pull the model and run it once with the network cable unplugged. If it tries to fetch something, find out now.
  • Check the ZIM files open in kiwix-serve before you go offline.
  • Put the whole thing on a second drive, and keep a cheap USB stick with just the Kiwix files.
  • Print the few pages you'd need in the first hour: CPR, water purification, your own medications. Several HN commenters made this point, and they're right: paper doesn't need a charger.
  • I'd install Survival LLM on a Mac without thinking twice. Just remember the 14/22 at the 16 GB tier, and treat the library as the source of truth.

    If the internet went down for a month, what would you want on your offline drive besides a model and Wikipedia? One HN commenter said the default library feels "more like a tool for city dwellers" because it skips hunting and field dressing. What's missing from yours?

    Sources

    How to Run an Offline AI Assistant With No Internet (Even on a Raspberry Pi) | Authon Blog