{"id":112,"date":"2025-07-15T14:24:01","date_gmt":"2025-07-15T14:24:01","guid":{"rendered":"https:\/\/engineeringharmony.com\/?p=112"},"modified":"2025-07-15T14:42:45","modified_gmt":"2025-07-15T14:42:45","slug":"ai-with-a-local-model-on-a-raspberry-pi","status":"publish","type":"post","link":"https:\/\/engineeringharmony.com\/?p=112","title":{"rendered":"AI with a Local Model on a Raspberry Pi"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">Acquire Hardware<\/h2>\n\n\n\n<p>Buy a Raspberry Pi 5: <a href=\"https:\/\/www.raspberrypi.com\/products\/raspberry-pi-5\/\">https:\/\/www.raspberrypi.com\/products\/raspberry-pi-5\/<\/a> and follow the standard Raspbian install process.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Local AI Infrastructure<\/h2>\n\n\n\n<p>Install Ollama with the following command. There may be better ways because running unverified shell scripts is not advised, but I just threw caution to the wind and ran it.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>curl -fsSL https:\/\/ollama.com\/install.sh | sh<\/code><\/pre>\n\n\n\n<p>In case this command is updated in the future, I found the command here: <a href=\"https:\/\/ollama.com\/download\/linux\">https:\/\/ollama.com\/download\/linux<\/a><\/p>\n\n\n\n<p>After reading about the various models, it looks like phi3 is the best (as of this post in July 2025) for a Raspberry Pi. Use the following command to install that model. <\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ollama run phi3<\/code><\/pre>\n\n\n\n<p>Note: It should take a while to pull the model the first time. Once the model is pulled, it&#8217;s much quicker, though still take a little bit of time to instantiate the model for each conversation\/session.<\/p>\n\n\n\n<p>And there you go! You have a local AI! You can now chat with your local AI model!<\/p>\n\n\n\n<p>As a side note, I tried to use the Raspberry Pi AI Hat+, which is based on the Hailo-8 hardware. Unfortunately, this does not work for LLMs. It is more focused on image\/video analysis. Hailo-10 will have LLM support, but is not released yet. (as of July 2025)<\/p>\n\n\n\n<p>So I think my next step will be to purchase an Nvidia Jetson Orin Nano Developer Kit. Stay tuned for those updates!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Acquire Hardware Buy a Raspberry Pi 5: https:\/\/www.raspberrypi.com\/products\/raspberry-pi-5\/ and follow the standard Raspbian install process. Local AI Infrastructure Install Ollama with the following command. There may be better ways because running unverified shell scripts is not advised, but I just threw caution to the wind and ran it. In case this command is updated in [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[9],"tags":[],"class_list":["post-112","post","type-post","status-publish","format-standard","hentry","category-documentation"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/engineeringharmony.com\/index.php?rest_route=\/wp\/v2\/posts\/112","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/engineeringharmony.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/engineeringharmony.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/engineeringharmony.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/engineeringharmony.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=112"}],"version-history":[{"count":6,"href":"https:\/\/engineeringharmony.com\/index.php?rest_route=\/wp\/v2\/posts\/112\/revisions"}],"predecessor-version":[{"id":120,"href":"https:\/\/engineeringharmony.com\/index.php?rest_route=\/wp\/v2\/posts\/112\/revisions\/120"}],"wp:attachment":[{"href":"https:\/\/engineeringharmony.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=112"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/engineeringharmony.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=112"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/engineeringharmony.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=112"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}