{"id":370,"date":"2026-10-28T15:27:26","date_gmt":"2026-10-28T15:27:26","guid":{"rendered":"https:\/\/felixaugenstein.com\/blog\/?p=370"},"modified":"2026-10-28T15:27:26","modified_gmt":"2026-10-28T15:27:26","slug":"i-built-a-local-ai-shopping-agent-with-ibm-bob-from-recipe-files-to-a-full-bring-list","status":"publish","type":"post","link":"https:\/\/felixaugenstein.com\/blog\/i-built-a-local-ai-shopping-agent-with-ibm-bob-from-recipe-files-to-a-full-bring-list\/","title":{"rendered":"I built a local AI shopping agent with IBM Bob: from recipe files to a full Bring! list"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><em>A Vue chat UI, a small Node backend, a 4B-parameter LLM running on my laptop, and an MCP server. Built with\u00a0<a href=\"https:\/\/bob.ibm.com\/\">IBM Bob<\/a>\u00a0in minutes.<\/em> Find the repo here: <a href=\"https:\/\/github.com\/FelixAugenstein\/bring-app-agent\" target=\"_blank\" rel=\"noopener\">https:\/\/github.com\/FelixAugenstein\/bring-app-agent<\/a><\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">The problem<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Every week it is the same routine: decide what to cook, open the shopping list app, type in 40 ingredients. I already keep my favourite recipes as text. I use&nbsp;<a href=\"https:\/\/www.getbring.com\/\">Bring!<\/a>&nbsp;for shopping. The missing piece was something that connects the two without me doing the typing.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">So I wanted a small agent that handles three situations:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>&#8220;Plan my week.&#8221;<\/strong>\u00a0It picks recipes, shows me the plan, and only adds ingredients after I confirm.<\/li>\n\n\n\n<li><strong>&#8220;Add the ingredients for Pfannkuchen.&#8221;<\/strong>\u00a0One recipe, straight to the list.<\/li>\n\n\n\n<li><strong>&#8220;2 \u00c4pfel und 3 Bananen.&#8221;<\/strong>\u00a0Loose items, no recipe involved.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">And I had three requirements: it should run&nbsp;<strong>locally<\/strong>, it should need&nbsp;<strong>no API keys<\/strong>&nbsp;(apart from my own Bring! login), and the AI should be&nbsp;<strong>in the loop for every case<\/strong>, not just a fancy front end for a fixed script.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The result<\/h2>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"295\" src=\"https:\/\/felixaugenstein.com\/blog\/wp-content\/uploads\/2026\/10\/bring-app-agent-architecture-1024x295.png\" alt=\"\" class=\"wp-image-371\" srcset=\"https:\/\/felixaugenstein.com\/blog\/wp-content\/uploads\/2026\/10\/bring-app-agent-architecture-1024x295.png 1024w, https:\/\/felixaugenstein.com\/blog\/wp-content\/uploads\/2026\/10\/bring-app-agent-architecture-300x86.png 300w, https:\/\/felixaugenstein.com\/blog\/wp-content\/uploads\/2026\/10\/bring-app-agent-architecture-765x220.png 765w, https:\/\/felixaugenstein.com\/blog\/wp-content\/uploads\/2026\/10\/bring-app-agent-architecture-1536x442.png 1536w, https:\/\/felixaugenstein.com\/blog\/wp-content\/uploads\/2026\/10\/bring-app-agent-architecture-998x287.png 998w, https:\/\/felixaugenstein.com\/blog\/wp-content\/uploads\/2026\/10\/bring-app-agent-architecture-229x66.png 229w, https:\/\/felixaugenstein.com\/blog\/wp-content\/uploads\/2026\/10\/bring-app-agent-architecture-2048x589.png 2048w, https:\/\/felixaugenstein.com\/blog\/wp-content\/uploads\/2026\/10\/bring-app-agent-architecture-480x138.png 480w, https:\/\/felixaugenstein.com\/blog\/wp-content\/uploads\/2026\/10\/bring-app-agent-architecture-348x100.png 348w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Frontend:<\/strong>\u00a0a Vue 3 chat window with a sidebar listing all recipes. Click a recipe to see its ingredients with amounts and the full instructions. Click the cart icon to add it to the list.<\/li>\n\n\n\n<li><strong>Backend:<\/strong>\u00a0a Node\/TypeScript Express server that keeps the chat history, talks to the LLM and executes the actions.<\/li>\n\n\n\n<li><strong>LLM:<\/strong>\u00a0<code>qwen3.5:4b<\/code>\u00a0running through\u00a0<a href=\"https:\/\/ollama.com\/\">Ollama<\/a>. About 3.4 GB, runs on a normal laptop.<\/li>\n\n\n\n<li><strong>Recipes:<\/strong>\u00a0plain Markdown files with a bit of YAML front-matter. Drop in a new\u00a0<code>.md<\/code>\u00a0file and it shows up in the sidebar and for the agent, no restart needed.<\/li>\n\n\n\n<li><strong>Bring! access:<\/strong>\u00a0the open-source\u00a0<a href=\"https:\/\/github.com\/florianwittkamp\/bring-mcp\"><code>bring-mcp<\/code><\/a>\u00a0MCP server, started on demand by the backend.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">One command starts everything:&nbsp;<code>.\/start.sh<\/code>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Find the repo <a href=\"https:\/\/github.com\/FelixAugenstein\/bring-app-agent\" data-type=\"link\" data-id=\"https:\/\/github.com\/FelixAugenstein\/bring-app-agent\" target=\"_blank\" rel=\"noopener\">here<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">How it works: the LLM decides, the server executes<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The most important design decision was to let the model&nbsp;<em>choose<\/em>&nbsp;what to do, but never let it&nbsp;<em>invent<\/em>&nbsp;results. The backend gives the model a system prompt that lists only the recipes that really exist, plus six tools:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><th class=\"has-text-align-left\" data-align=\"left\">Tool<\/th><th class=\"has-text-align-left\" data-align=\"left\">What it does<\/th><\/tr><\/thead><tbody><tr><td><code>propose_week_plan<\/code><\/td><td>Picks N recipes, stores them as &#8220;pending&#8221; and asks me to confirm. Nothing is added yet.<\/td><\/tr><tr><td><code>confirm_week_plan<\/code><\/td><td>After my &#8220;yes&#8221;, sends all ingredients of the pending recipes to Bring!.<\/td><\/tr><tr><td><code>swap_recipe_in_plan<\/code>&nbsp;\/&nbsp;<code>cancel_week_plan<\/code><\/td><td>Replace one recipe in the proposal, or throw the proposal away.<\/td><\/tr><tr><td><code>add_recipe_to_list<\/code><\/td><td>Adds the ingredients of a single recipe.<\/td><\/tr><tr><td><code>add_items_to_list<\/code><\/td><td>Adds free-form items like &#8220;2 \u00c4pfel und 3 Bananen&#8221;.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">When I type a message, the model answers with a&nbsp;<strong>tool call<\/strong>. The server runs it and shows me the&nbsp;<em>real<\/em>&nbsp;result. That means:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>The model cannot claim &#8220;done!&#8221; when nothing was added.<\/li>\n\n\n\n<li>Ingredients for recipes come from the recipe file, not from the model&#8217;s imagination.<\/li>\n\n\n\n<li>Adding to the list always requires an explicit confirmation step for the weekly plan.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">All of it lands on one specific Bring! list (in my case &#8220;Einkauf Supermarkt&#8221;), which the backend looks up by name and which is configurable in&nbsp;<code>.env<\/code>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What I learned along the way<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Building this was not a straight line, and the detours were the most interesting part.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>1. Small models and made-up output formats don&#8217;t mix.<\/strong>&nbsp;My first version asked the model to write a special tag like&nbsp;<code>&lt;action&gt;\u2026&lt;\/action&gt;<\/code>&nbsp;in its answer, which the server then parsed. A 4B model mostly ignored that format. The fix was to switch to Ollama&#8217;s&nbsp;<strong>native tool calling<\/strong>. The model is trained for exactly that, and the reliability difference was big.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>2. Turn off &#8220;thinking&#8221; for this kind of job.<\/strong>&nbsp;Reasoning mode made replies noticeably slower with no benefit for &#8220;pick a tool and fill in two arguments&#8221;. Setting&nbsp;<code>think: false<\/code>&nbsp;made the chat feel instant.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>3. Make the server the source of truth.<\/strong>&nbsp;Everything that has side effects, such as writing to Bring!, happens in server code, not in model text. The model&#8217;s job is to understand German or English and route the request.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>4. Hot-reload what the dev tooling doesn&#8217;t watch.<\/strong>&nbsp;<code>tsx watch<\/code>&nbsp;restarts the backend when code changes, but it does not watch the&nbsp;<code>recipes\/<\/code>&nbsp;folder. Rather than telling myself to restart the server, the backend simply re-reads the folder on every request. New recipe files just appear.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>5. Stale processes are sneaky.<\/strong>&nbsp;At one point an old backend still held port 3001 and happily served old code while I debugged the new one.&nbsp;<code>start.sh<\/code>&nbsp;now refuses to start when its ports are taken, waits until both services are really ready, and prints prefixed&nbsp;<code>[backend]<\/code>&nbsp;\/&nbsp;<code>[frontend]<\/code>&nbsp;logs and the links to open.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Where IBM Bob fit in<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">I did not hand-write this project line by line. I described what I wanted, and&nbsp;<a href=\"https:\/\/bob.ibm.com\/\">IBM Bob<\/a>&nbsp;did the work with me: it planned the sub-tasks, scaffolded the Vue frontend and the Node backend, wired up the MCP client, wrote the recipe loader and the start script, and fixed the bugs above as we hit them. It also reviewed the plan against the code, translated and extended the README (including the architecture diagram), and kept the type-checks green.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The first working version took minutes, not days. The rest of the time went into the interesting refinements: switching to tool calling, targeting the right shopping list, and the recipe detail view. Those were conversations (&#8220;make recipes clickable and show the full recipe&#8221;), not tickets.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A few things made that workflow pleasant:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>A written plan first.<\/strong>\u00a0Bob produced an implementation plan with sub-tasks and a decision log, which I could correct before any code existed.<\/li>\n\n\n\n<li><strong>Verification as part of the loop.<\/strong>\u00a0Type-checks and small endpoint tests were run before anything was reported as done.<\/li>\n\n\n\n<li><strong>It respected my constraints.<\/strong>\u00a0For example, &#8220;the LLM must stay involved in every use case&#8221; was not traded away for a quick server-only shortcut.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Try it yourself<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">You need Node.js 22+, Ollama, and a Bring! account with email and password login. Find the repo <a href=\"https:\/\/github.com\/FelixAugenstein\/bring-app-agent\" data-type=\"link\" data-id=\"https:\/\/github.com\/FelixAugenstein\/bring-app-agent\" target=\"_blank\" rel=\"noopener\">here<\/a>.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>git clone https:\/\/github.com\/FelixAugenstein\/bring-app-agent.git\ncd bring-app-agent\ncp .env.example .env        <em># add your Bring! email and password<\/em>\nnpm install\n(cd server &amp;&amp; npm install)\n(cd frontend &amp;&amp; npm install)\n.\/start.sh                  <em># pulls the model on first run, then opens everything<\/em>\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Then open&nbsp;<code>http:\/\/localhost:5173<\/code>&nbsp;and try:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>Plane meine Woche<\/code><\/li>\n\n\n\n<li><code>F\u00fcge Zutaten f\u00fcr Spaghetti Bolognese hinzu<\/code><\/li>\n\n\n\n<li><code>2 \u00c4pfel und 3 Bananen<\/code><\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Your credentials stay in your local&nbsp;<code>.env<\/code>, which is git-ignored.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What&#8217;s next<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The obvious next steps are scaling servings up or down per recipe, skipping ingredients I already have at home, and letting the agent suggest a plan based on what is already on my list. Because every capability is just a tool definition plus a small function, adding them is straightforward.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Credits<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/github.com\/florianwittkamp\/bring-mcp\"><code>bring-mcp<\/code><\/a>\u00a0by Florian Wittkamp, the MCP server that makes the Bring! integration possible.<\/li>\n\n\n\n<li><a href=\"https:\/\/ollama.com\/\">Ollama<\/a>\u00a0and the Qwen model family for local LLM inference.<\/li>\n\n\n\n<li><a href=\"https:\/\/bob.ibm.com\/\">IBM Bob<\/a>\u00a0for building the app and agent with me.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><em>This project is not affiliated with or endorsed by Bring!.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>A Vue chat UI, a small Node backend, a 4B-parameter LLM running on my laptop, and an MCP server. Built with\u00a0IBM Bob\u00a0in minutes. Find the repo here: https:\/\/github.com\/FelixAugenstein\/bring-app-agent The problem Every week it is the same routine: decide what to cook, open the shopping list app, type in 40 ingredients. [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":372,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[61,28,44,55,65,66,30,46],"tags":[],"class_list":["post-370","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-agentic-ai","category-application","category-express-js","category-generative-ai","category-ibm-bob","category-mcp","category-node-js","category-vue-js"],"_links":{"self":[{"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/posts\/370","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/comments?post=370"}],"version-history":[{"count":1,"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/posts\/370\/revisions"}],"predecessor-version":[{"id":373,"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/posts\/370\/revisions\/373"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/media\/372"}],"wp:attachment":[{"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/media?parent=370"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/categories?post=370"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/tags?post=370"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}