Almost Timely News: 🗞️ How to Set Up Private, Local AI (2026-01-25)

Almost Timely News: 🗞️ How to Set Up Private, Local AI (2026-01-25) :: View in Browser

The Big Plug

I’ve got a sneak peek next week coming for a new AI GEO/AIO/whatever-EO basic auditing tool that I’ve been working on. If you want advance notice, join the Analytics for Marketers Slack group (free). That’s where I’ll be announcing it and asking for beta testers.

Content Authenticity Statement

100% of this week’s newsletter content was originated by me, the human. Learn why this kind of disclosure is a good idea and might be required for anyone doing business in any capacity with the EU in the near future.

Watch This Newsletter On YouTube 📺

Click here for the video 📺 version of this newsletter on YouTube »

Click here for an MP3 audio 🎧 only version »

What’s On My Mind: How to Set Up Private, Local AI

This week, let’s talk about setting up completely private, local AI. There are many reasons to do this, from big tech and similar surveillance to working with the most confidential data that you cannot, for any reason, expose to a third party. This week’s issue is aimed at the individual, like you or me. It’s not aimed at companies and enterprises; company IT departments are responsible for assisting you with privacy protection at the organization level.

Pro Tip: If you have access to a tool like Claude Code, Claude Cowork, Google Antigravity, Qwen Code, OpenAI Codex, or other agentic frameworks, you can just copy paste this entire newsletter in and have the tool work through it with you step by step.

Part 0: Ethics and Disclaimers

Before we begin, it would be remiss if we didn’t touch on the ethics of completely private AI. AI is a crazy powerful tool, and today’s tutorial allows you to escape the watchful eye of anyone. This can enable good and bad things. Good, if you’re using private AI to protect sensitive data, or to work towards helping people make their lives better. Bad if you’re using it to make others’ lives worse, to do things that are blatantly harmful, etc. Because AI is only a tool, the onus of responsibility for its moral use lies with you and me, the users of it.

Please use it to help, not harm.

When it comes to AI models that you can run privately, some of the best models in the world come from China, particularly the Alibaba Qwen family of models. These models are largely uncensored and highly capable. There’s an important distinction between using AI hosted by companies in China and using Chinese AI models that you run on your own computer. In terms of data privacy and safety, Chinese AI companies make it very clear in their terms of service that they’re using your data. Using AI hosted by those companies with private, sensitive data is unsafe.

Using models made by Chinese AI companies on your own hardware, as long as they’re from reputable sources like Alibaba, is as safe as the rest of your infrastructure.

Finally, a disclaimer: you are solely responsible for how you use AI in general, and how you use these instructions specifically. Neither I nor my company provide support, guarantees, or assurances that any of this works, is safe, or will keep you hidden from entities you want to be hidden from. I’ve provided my best reasonable efforts, but nothing in life is guaranteed. If you use these instructions in any capacity, you are solely responsible for the consequences.

Part 1: Foundation Technologies

If you want complete privacy in the use of AI and still have it be very capable, you’ll need to use AI tools and models that have access to the Internet in some capacity. The reason for this is that ALL smaller language models, ones that you’d run on your own infrastructure, are hallucination machines. They hallucinate a crazy amount of the time, anywhere from 5% to 90% of the time, because they simply aren’t large enough to store and have factually correct information in them. The smaller a model is, the more it is likely to hallucinate.

To counter this, having search capabilities is essential. However, this raises a privacy risk - what you search for on the Internet can, without proper precautions, be traced back to you and your AI. To counteract this, you have to invest in some additional infrastructure - notably, a VPN and some privacy-safe search capabilities.

When you’re shopping for a VPN, there are two phrases you want to look for in any commercial service: zero data retention, and no logging. Zero data retention means that the VPN provider retains absolutely no data about how you use the service. No logging means that they do not keep server logs of any kind, which in turn means that an employee cannot produce logs upon request from anyone else; many providers have paid for third party audits to prove they do not log user activities, and thus have no records to produce when asked.

When choosing a VPN provider, look for one outside your jurisdiction. Here’s why: almost all technology companies of any kind state in their terms of service that they will comply with lawful requests from their respective governments to hand over user data, if they have the data (hence the no logging and zero data retention features). But that’s based on the locus of business for the provider.

It is much more challenging for a government to obtain information from a foreign provider - even with mutual legal assistance treaties, a government cannot generally subpoena a foreign provider directly, instead having to work with the appropriate authorities in that jurisdiction, a maze of red tape. Combining provider features like no log retention / zero data retention with a VPN provider outside your jurisdiction provides you the most security.

As a sidebar, having a VPN is useful ESPECIALLY if you travel. Places like airport and public wifi are incredibly dangerous for your devices, so using a VPN in those is almost always a good idea.

Besides a VPN, you’re going to need the ability to use the Tor network. This is a planet-wide routing network that further masks activities. Tor is free to access and free to use, powered by millions of users around the globe. To install Tor services for use with your AI, you’ll want to use either Chocolatey (Windows) or Homebrew (Mac) or apt/yum/dnf (Linux, built in). You’ll also want to install Python and Node.

If this sounds intimidating, don’t worry - help is one prompt away. In the AI of your choice, use a prompt like this to have it walk you through setting it up, step-by-step, selecting the options that make the most sense.

You’re a cybersecurity expert skilled at privacy software. Today you’ll be helping me set up command line Tor for use with generative AI tools like MCP services and Python libraries. I’ll be using Tor through a generative AI tool, not as a direct user. I have a {Windows/Mac/Linux} computer. I need your help giving me step by step instructions for installing {Chocolatey/Homebrew} and then installing Tor, node.js, and Python 3.12. I am a {novice/intermediate/advanced} technical user, so create your instructions accordingly, with step by step, specific instructions. Ask me questions until you have enough information to complete this task successfully, keeping in mind my level of technical skill.

Let the AI of your choice guide you through the process. If you have access to tools like Claude Cowork or similar (Claude Code, Qwen Code, Google Antigravity) you could even ask it to help you by installing those things for you.

Another piece of technology you’ll want is a network activity blocker. On the Mac, the best choice is Lulu. On Windows, it’s Simplewall. What these two free, open source packages do is monitor network traffic from your computer and allow you to block applications that are trying to call home. This is especially important if you have privacy-invading apps like those from Meta or similar big tech companies that are constantly calling home.

If you’re unsure how to configure them, again, help is a prompt away. Paste this into the AI of your choice:

You’re a cybersecurity expert skilled in firewalls and network activity detection. I just installed {Lulu/Simplewall} o