How I built a personal AI assistant that has to show its work
By Chrysti Reichert | A practical case study from My AI Evolution
The first design for my Practical AI Safety guide came back with green accents and serif headings. My brand uses orange and system fonts.
Nice design. Wrong brand.
I had asked for a guide about checking AI. The AI handed me a reason to check it.
I asked it to align the guide with my brand. The design was corrected against the written standards, then checked on the published site. I caught the mismatch. The assistant did not catch it for me.
That detail matters. I could tell this as a story about my assistant correcting its work and leave myself out of the middle. It would sound more impressive. It would also leave out the person who noticed the problem.
So, that is where this story starts. With orange. And an assistant that already had the instructions.
The agents showed up with separate files
Earlier, I wrote about giving five AI agents a job: build a quiz to help someone understand their AI skills. They produced five separate files. Each had done its own part, without combining the work into the thing I had asked for.
I had a team. I also had the job of assembling the team’s homework.
I added coordination: who worked with whom, who challenged an assumption, and where I stepped in. That experience became my post about AI agents ghosting each other. The useful part was the failure. Giving several agents the same goal had not given them a shared way to finish it.
That is one of the problems my operating system is built around. The assistant needs more than a task. It needs a way to find the current instructions, work within them, and bring back something I can check.
What I mean by a personal AI operating system
My personal AI operating system is a collection of working files, instructions, tools, and checks that gives my assistant a consistent way to handle my work. I call the assistant Deb. The operating system tells Deb where to look, what she can do, when she needs me, and what evidence belongs with a claim that something is finished.
It is an application-level setup around AI tools. It does not replace Windows, and I did not train a new language model.
The distinction matters if you want to build your own AI assistant. A personality gives the conversation a familiar voice. A working system also needs somewhere to keep current decisions, a way to retrieve them, and limits on what the assistant can do with them.
I built mine around a question: when the assistant says something is true or finished, what can I inspect?
Give each fact a home
My setup assigns a source to each kind of information. Brand rules live with brand rules. Current services have their own source. Decisions have a record. A summary can help the assistant find the right file, but the file that owns the topic is where it needs to check the claim.
Yes, part of my AI setup is filing. Very futuristic. The filing matters because the person opening the drawer can write a persuasive answer from the wrong document.
An old draft can contain an old decision. A remembered instruction can have been replaced. A search result can point to the right subject without containing the current answer. My instructions require the assistant to follow those clues back to the source before treating them as current facts.
The branding mistake shows the limit. Having a source does not guarantee that the assistant consults it at the right moment or applies it correctly. The source gave the correction a clear target. It did not prevent the first miss.
Find it. Check it. Use it.
Read the diagram as text
Find the clue
A saved note or search result points to the topic.
Check the source
Open the current file that owns the fact or decision.
Keep the limit
If evidence is missing, keep the claim uncertain.
Separate what the assistant knows from what it assumes
I use explicit evidence states in the operating rules. Something recorded, read, or found has a different standing from something assumed or unknown.
An assumption can be useful while drafting. It should remain visible as an assumption. It should not become permission to take an action or a fact in public copy.
For example, imagine an assistant that cannot open a report. The honest result is that the report could not be checked. It cannot turn that missing access into a claim that there was nothing in the report. That example is hypothetical, but the distinction is part of my actual design.
This is also why I want evidence attached to the specific claim. A successful check of one page says something about that page. It does not establish that every related process works.
Give the assistant room to work and a clear stopping point
An assistant that asks permission to fix every sentence is exhausting. An assistant that treats a request for a draft as permission to publish it is a different problem.
My operating rules distinguish preparation from outward action. Research, drafts, and reversible local work can move forward within the task. Sending, publishing, spending, and other consequential actions have separate approval requirements. For public writing under my name, I review the finished piece.
There are two layers here. Written instructions tell the assistant how to behave. Tool permissions and specific checks can enforce parts of that behavior. A rule in a document is not, by itself, a technical barrier.
I need both, and I need to know which one I am relying on. If the assistant has more access than the task requires, a paragraph asking it to be careful does not remove that access.
Where I stay in the loop
Read the diagram as text
Prepare the work
Research, draft, and make reversible changes within the task.
Bring me the piece
Show the finished work and the action it is ready for.
Check permission
Publishing, sending, or spending needs the relevant approval.
Make “done” point to something
When I was publishing the Practical AI Safety guide, I asked for it to be pushed to main. Then I asked for it to be deployed until it was fully published.
Those were separate requests for a reason. A file can be saved on my computer. Code can be sitting on GitHub. Neither tells a reader where to open the guide.
I wanted the page people could actually use. That meant following the work past the repository and checking the public address.
The release checks included comparing the served files with the committed version and checking the public page in a browser. Those checks supported a specific conclusion: the approved guide was available, and the checked page interactions worked.
They did not prove that search engines had indexed it, that anyone had read it, or that it had brought me a customer.
That is the kind of completion report I want. Tell me what changed, show me where it exists, and stop the claim where the evidence stops.
You can inspect the public result in my Practical AI Safety guide and its GitHub repository. Those are public teaching materials. My private operating files are not included.
What does “done” prove?
Read the diagram as text
Saved locally
The files exist on the computer. Readers cannot use them yet.
Pushed to GitHub
The code reached the repository. The live page still needs a check.
Checked in public
Open the published page and test the intended reader path.
Keep changed instructions from leaving old copies behind
Instructions can spread. The same rule may appear in a project file, a tool configuration, and a routine that runs later. Updating the original does not necessarily update every place that depends on it.
My system includes a process for tracking those dependencies. A standing rule change gets an identifier. The registered places that depend on it must be accounted for, with evidence showing whether each was updated, already inherits the rule, or still has a gap.
A checker validates that declared structure and the required evidence. It does not read every sentence in the system and decide whether all the prose agrees. A live setting outside the repository also needs its own inspection.
That limitation belongs in the description. Otherwise, a useful check turns into a much larger promise than the code can keep.
Treat a lesson as a candidate before making it a rule
I also have a written process for learning from mistakes. An observation starts as a candidate. Promoting it into a standing behavior requires further confirmation, my confirmation, and checks that the change reached the places that depend on it.
This matters because an assistant can draw an overly broad lesson from one exchange. A preference for one piece of writing does not automatically apply to every piece I will ever write.
Remembering more is only helpful if the remembered material keeps its context. I want to be able to trace a rule back to the evidence and decision that justified it, then change or retire it when that basis no longer holds.
What this case study does and does not establish
This is a practitioner case study of a system I use and continue to revise. I have not run a controlled comparison that establishes how much this setup reduces errors, saves time, or improves business results.
The public release provides an inspectable example of work and verification. The branding correction provides an example of a miss that still required me. Neither is a measured reliability rate for the whole assistant.
There are costs, too. Instructions need maintenance. Sources can become stale. Too many approval steps can interrupt useful work. A passing automated check can invite more confidence than its limited scope deserves.
To test the design more seriously, I would compare the same defined tasks with and without the added controls. I would record missed instructions, unsupported completion claims, unnecessary interruptions, and the human effort needed to catch and repair errors. Until that comparison exists, I should not put a percentage on the benefit.
I also cannot support a claim that nobody else has built something similar. What I can show is how I connected source checking, permission boundaries, change tracking, and completion evidence around my own work, including where the connection failed.
Build a version you can inspect
If you are building your own personal AI assistant, start with a real task whose result you know how to check. Give the assistant the relevant source, make its authority explicit, and decide what would prove the task is finished.
Try it when information is missing or instructions conflict. Watch whether it shows you the gap or writes around it. That behavior will tell you more than a polished introduction about everything it can do.
I published a free personal AI assistant starter with reusable materials and fictional examples. The examples are for practice. Passing their checks is not a certificate that your assistant is safe.
If you want help building around your own work, my personal AI assistant coaching is the place to start that conversation.
I still think about those separate quiz files when I add another instruction or tool. Does this help the assistant finish the actual job, or have I given myself another thing to manage?
And I still check the orange.