AI Tech Support: Trust, but Verify

AI has plenty of shortcomings and I don’t recommend trusting it blindly. But it’s also an incredibly useful tool when used carefully. Let me give you an example use case that almost everyone would find valuable.

LLMs Are Not Like Search Engines

Large Language Models (LLMs) or “chatbots” – like ChatGPT, Gemini, Claude, or Lumo (my pick) – are often treated like fancy search engines. Instead of giving you a list of links with titles that you have to click through to find your answer, it just gives you what it thinks is the answer. (This is much to the consternation of all these sites whose content helped train these models and are now no longer getting web traffic from search results…. one of AI’s many problems today.)

But it’s much more than that. Chatbots have context – you can have ongoing conversations about a topic, like you would with a human, where you don’t have to repeat what you’ve already discussed. That’s a huge advantage. Chatbots can even identify hardware from photos and help troubleshoot decades-old equipment… but I’ll come back to that in a minute.

Also, most search engines sort of presume you know what you’re looking for. With a chatbot, you can start with something as simple as “My laptop suddenly gets hot after ten minutes” and work backwards from there. It will ask you probing, clarifying questions – questions you probably didn’t even think to ask. And it will create hypotheses and help you test them out. As new evidence appears, it can adapt on the fly. These LLMs have been trained on enormous collections of writing, including technical documentation, support discussions, manuals, and countless examples of people troubleshooting similar problems – and they can synthesize it all for you.

Your Own Personal IT Resource

Now, if you are lucky enough to have a “techie” amongst your family and friends, they are excellent resources when you need to troubleshoot a problem on your computer or fix something around the house or help you identify odd device behavior or decipher some error message. But humans have to sleep and work, and sometimes they get sick or go on vacation. Not true of chatbots. They are generally available 24/7. They are infinitely patient and never get tired of you asking questions.

Furthermore, you can tailor your experience completely. If you already know about the subject or have some relevant background, you can tell the LLM about that up front; conversely, you can literally ask the chatbot to explain it to you like you’re a 10-year-old. You’ll want to make sure the chatbot has important relevant information like make, model and version of hardware and software. Better yet, tell the chatbot not to start troubleshooting until it has asked for any information it needs. It’ll often tell you exactly where to find it.

You can also ask chatbots to help you review your settings, or explain every app you have installed, or audit your privacy settings, or help you switch from Windows to Mac or Android to iOS… really, the possibilities are endless.

Approach troubleshooting like a good technician: gather evidence first, make small reversible changes one at a time, understand what you’re changing, and have a backup before doing anything risky.

A Picture is Worth 1000 Words

Here’s something you may not have thought of, but can be extremely useful: most LLMs will allow you to upload pictures, including screenshots. Have an error message on your computer or TV screen? Have an old baseball card, Beanie Baby, vintage car or some potentially valuable collectable you want to identify?

Here’s a real example from my life. My daughter found a really old stereo console from Magnavox that wasn’t fully working. She’s asked for my help. I opened the cabinet and took photos of the circuit boards and a few identifying markings. Even though the labels were badly faded, ChatGPT correctly identified the model, then suggested several likely causes based on the symptoms – all for a stereo that was more than 50 years old. I was blown away.

Important Caveats

First and foremost, do not give personal information to chatbots, unless maybe you’re using a privacy-focused model like Lumo or using a purely local model that doesn’t send information to the internet. I would not share medical, financial, mental health or personal documents or statements. Do not share phone numbers, email addresses, physical addresses, passwords, government IDs, account logins, names of people you’re related to, or other such info.

As Carissa Véliz puts so eloquently in her recent book Prophecy, LLMs are not omniscient or prescient. They don’t know things the way people do. They generate each response based on patterns they’ve learned, which means they can sound extremely convincing even when they’re wrong. While hallucination (or “confabulation” as Carissa likes to call it) was a common problem with early chatbots, they’ve gotten much better. However, don’t take their advice as gospel – “trust, but verify” for anything important. When possible, ask for links to the official documentation from the operating-system or software vendor, not just forum posts or blog articles. And if you feel it’s dangerous, don’t do it just because the chatbot said to.

Finally, everyone is pushing “agentic AI” now … giving AI the ability to act for you, to directly control your computers, your phones, your software, your browsers – even logging into your accounts as you. This is not yet safe. We may get there. Apple’s upcoming Siri AI will probably be safer than most, but we’ll see. We’ll figure this out eventually, but I would not currently give AI control to act on its own. Just let it give you instructions and do the work yourself.

Security, Privacy & Verification

A chatbot may confidently recommend high-impact changes like disabling services, uninstalling software, editing the registry, deleting files, changing permissions, or running commands copied from the internet. If you’re ever unsure or scared, just stop. Challenge these recommendations. Also, now that you have more knowledge, try some regular web research to verify the advice. Even asking a different chatbot for a “second opinion” can be helpful. Remember that a chatbot can interpret the symptoms, screenshots, logs, settings, and command output you give it. It cannot necessarily see what is actually happening on your machine – and it may fill gaps with assumptions. If it makes assumptions, ask it to state them explicitly.

Be careful uploading log files. If you feel this is needed, you would want to redact them first, as best you can. Replace usernames, hostnames, public IP addresses, domain names, and tokens with obvious placeholders – but say that you redacted them so the model does not misinterpret the placeholders.

Finally, before uploading a screenshot, check browser tabs, filenames, account names, email addresses, serial numbers, QR codes, notifications, and anything visible in the background. Crop or blur what the chatbot does not need.

How to Ask Good Questions

Like learning how to do powerful web searches (using techniques sometimes called Google dorks), in the age of chatbots we’re going to have to learn how to improve the prompts that we give to the LLMs. This article has some nice, general tips, but I’ll give you some more here.

Start with these general tips:

• Tell the AI your goal.
• Give enough context.
• Ask it to explain its reasoning and explain any assumptions explicitly.
• Ask for references – preferably official documentation.
• Ask it what information would increase its confidence.

Here is a sample starting prompt:

• You are helping me troubleshoot this as a cautious, experienced system administrator. Do not guess or invent commands, settings, or citations. Separate confirmed facts from hypotheses. Before recommending changes, ask for any information needed to narrow the diagnosis. Rank likely causes by confidence. Prefer safe, reversible diagnostic steps before corrective actions. Explain every command and warn me before anything that could delete data, reduce security, expose private information, or require administrator privileges. Cite authoritative documentation when practical.

And finally, this prompt has saved me many times. While chatbots can theoretically remember past conversations, in many cases this history (or “memory”) can be lost or “compressed”, and not be available if you need it again later. It can also be very helpful to provide to your “techie” helper later or even real tech support people, if you still need them.

• Summarize everything we have learned into a concise problem report. Include the symptoms, environment, timeline, tests performed, results, unresolved questions and any next steps.

In Conclusion

Used wisely, an LLM won’t replace a skilled technician – but it can often replace the panic you feel when something breaks. It can explain unfamiliar concepts, guide you through troubleshooting, and help you communicate more effectively with human support when you need it. Keep your private information private, verify anything important, and remember that you’re still the one in control. Think of it as a patient, knowledgeable assistant – not an infallible expert.