Five Things I Learned About Safety, Building With AI Without a CS Degree

I was building my first real project with Operum. A few months in, my PM asked: "Have you thought about CI?"

I had no idea what they meant.

"What is that?" I asked.

My agent explained it to me. Very simply. A system that automatically checks your code every time you ship something, catches problems you missed, and remembers to verify the things you built months ago that nobody has touched since. Runs without you doing anything. Catches the quiet failures.

That explanation is what this post is. Not the acronym. Not the machinery. What CI actually is, whether you need it, what it costs, and why almost nobody has told non-technical builders this story yet.

One: What CI actually is

Let me start with what CI is not. It is not the same thing as testing.

Testing is something you (or an agent) does: you write code, you verify it works the way you intended. Testing answers one question: Did we build the right thing?

CI is something that runs continuously, automatically, without you asking. It's a system that watches your project. Every time you ship a change, CI runs checks. It verifies that your change didn't break something that already existed. CI answers a different question: Did we break something that used to work?

Testing happens once. CI happens every time, forever.

These are two different problems. And you can have one without the other.

When I ask an AI to build a feature, the agent writes code and tests it. That's testing. But three months later, when I ask for a different feature and accidentally change something unrelated, who remembers to re-check that old feature? Nobody. Except CI. CI remembers. It checks every time.

That's what CI actually is: a system that checks the past so you don't have to remember it.

Two: Only CI covers the past

This is the part I want you to understand, because almost nobody explains it this way.

There are three ways code gets checked:

CatchesWhenWho does it
Your manual smoke testDid we build the right thing? Does it feel right? Does it do what you asked for?Once, after a changeYou — Operum adds a smoke-test checklist to your To Do when a task closes
The Tester agentIs this change correct? Logic errors, edge cases, things a human wouldn't think to checkPer change, whenever you askOperum Tester
CIDid we break something that used to work? Regressions in other featuresEvery time, forever, automaticallyGitHub Actions

The punchline—the thing that took me months to understand:

Only CI covers the past.

Manual testing looks at the change in front of you. The Tester agent looks at the change in front of you. Neither one asks: does this change break something that worked three months ago? Neither one remembers to re-check the feature that quietly broke because nobody has touched it recently.

But that's exactly the thing that quietly fails in production. The old login system that nobody has looked at since February. The payment confirmation email that used to work but the provider changed something. The customer list that slowly got out of sync.

Only CI catches those. And it catches them every time, whether you remember to check or not.

Three: Not every project needs it

Not every project needs CI. But if you do need it, you need it badly.

CI earns its cost when:

  • You can't read the diff. When I ask an AI to build something, I don't understand every line it writes. That's the whole point. But it means I can't just look at the code and tell if it's right. I need automated verification to catch what my eyes can't see.
  • The project is live with real users. If people depend on it, you can't afford to ship a broken change. The cost of fixing it in production is higher than the cost of catching it before it ships.
  • A test suite already exists. Running the same tests every time makes sense only if the tests are there.
  • The change surface is risky. Payments, authentication, data handling—these are places where a small mistake costs a lot.

CI is overhead when:

  • The project is a throwaway prototype. You're exploring an idea for two days. Infrastructure is waste.
  • It's a static site with no logic. If there's nothing that can break, there's nothing to test.
  • You have no tests yet. CI with no tests is theatre. It runs. It goes green. It verifies nothing. The first job is tests. Not CI.

I have two projects. They're completely different because one was simple and the other wasn't.

My photography website is a static site. Beautiful, but no logic. No payments. No user accounts. Just information and photos. I shipped that without CI. I tested it manually. That was enough.

A CRM for my photography business—built with Operum—is different. It handles bookings, payments, client information. When I make a change, there's real risk that something breaks. That's where CI earned its cost.

The contrast is the lesson: both projects are mine, both use Operum, both matter. One needed CI. One didn't. The difference is what you're protecting—static content or live data.

Four: What it really costs

Before you commit to CI, you should know what you're actually paying for and whether it's worth it.

You're not paying for peace of mind. You're paying for the specific peace of mind that you didn't break something that used to work. That's valuable if you have users depending on you. It's expensive overhead if you don't.

Let me be specific about money, because vague numbers are useless.

If your repository is public, CI is completely free. You get unlimited CI on public repositories, no matter which plan you're on.

If your repository is private, you pay. GitHub gives you free CI minutes depending on your plan:

  • Free plan: 2,000 minutes
  • Pro plan: 3,000 minutes
  • Team plan: 3,000 minutes

Once you exceed your allocation, GitHub charges overage rates (rates and allowances as of September 2026):

  • Linux (1-core): $0.002 per minute
  • Linux (2-core): $0.006 per minute
  • Windows (2-core): $0.010 per minute
  • macOS: $0.062 per minute

For a small project, you will usually pay nothing. A short test suite run only on pull requests fits inside the free monthly minutes. If you do go past them, expect roughly $4-10 per month on Linux runners.

For a mid-size project, running 30-minute test suites once per day, expect around $50 per month.

For a large, complex project like Operum itself, the cost can be up to $600 per month depending on your test complexity and runner choices.

Here's how to spend less on it:

The golden rules of using CI wisely:

  1. Run CI only on pull requests, not every push. You don't need to verify the entire suite every time you commit to your own branch. Check it when you open a PR. That alone cuts your CI usage in half.
  1. Skip unnecessary test runs on certain file changes. Did you only change a comment or a README? Tell CI to skip the test suite when only documentation changed.
  1. Cache your dependencies. Don't reinstall Node modules or Python packages on every run. Cache them between runs. That saves minutes per run and adds up quickly.

The principle: CI is not about running everything every time. It's about running the right checks at the right moment. If you notice CI eating too many minutes, ask your PM whether it can be optimized.

Five: You do not have to learn it

Here's what you don't need to do: learn GitHub Actions YAML syntax. You don't need to understand how to write CI workflows. You don't need to become an expert in pipeline configuration.

Here's what is actually true: Ask your PM, and your agents will write the CI configuration for you. The same way they write any other file.

That's how I got CI on my projects. I asked. My agents wrote it. It worked.

Quick Start projects don't come with CI—if you're building something complex enough to need it, connect your own GitHub repository and ask your PM. They'll route the configuration to your agents. No one-click setup. No magic. Just: ask, and it gets written.

The real lesson

Here's what I learned that surprised me most:

Tests used to be the expensive part of building. Writing tests was skilled, slow work, which is why small projects skipped them entirely. Now that I build with AI agents, something changed: the same agent that writes your feature writes the tests alongside it.

The barrier fell. Tests aren't expensive anymore.

This means something almost nobody has noticed yet: the thing that made building fast also made verifying it cheap. The speed gain and the safety gain came together.

You don't need to understand the code to ship safely. You need something that does. Your agents understand it. CI verifies it. Together they catch what you can't see.

Here's the thing I want you to know: your confusion about CI was completely normal. I've talked to other non-technical founders building with AI. Almost all of them had the same confusion. They thought CI was about testing. They thought if they tested carefully, CI was optional. They didn't understand that CI catches the thing that happens in production three months later when nobody's looking—the quiet failure, the forgotten edge case, the change that silently broke something else.

Now you know the difference.

Learn while you build

Eight months into building, I still don't know everything about testing. I probably never will. But I know enough to ship safely, to catch my own mistakes before customers see them, and to understand which of my projects actually need the safety net.

That's what this learning looked like. Not mastery. Enough.

And that's how it works when you build with AI. You learn as you go. You ask questions. Your agents explain. You try things. You see what works and what doesn't. By the time you need CI, you already understand it—not because you read a tutorial, but because you built something complex enough to need it.