How I was forced into AI engineering, and what I am learning along the way

I am a self-taught¹ full-stack developer for everything my company ships. For many years I have run a micro incubator — a handful of products and services that I build, deploy and own, all by myself. I write the code, the tests, and the deployment scripts; I run the servers. And I was quite content with that job description.

Then one of those products grew into a problem I could not keep up with.

The problem

The product is called PublishersGlobal. It started as a directory of the international publishing industry and has now grown to be a B2B SaaS for mid-small publishers and suppliers. It has some 20,000 publishers and service providers across 136 countries, publishing in 118 languages and used by 112,000 registered members. And like every directory, it rots as businesses close, domains expire and get bought by squatters, publishers get acquired and their websites quietly redirect to the parent company. Suppliers pivot into different businesses. The web keeps moving, and a directory that is accurate today will not be in six months.

For years every entity was manually reviewed and edited — deciding whether the organisation is real, active, and correctly described. On average a human being would spend 10–15 minutes per profile. Multiply that by 20,000 profiles. Now add to it the fact that the review process is never done — by the time you finish the last profile, the first one is stale again. Over the years the backlog grew to a place where it was mathematically a lost cause. Hiring an ever-larger verification team is a fix not available to a bootstrapped business. The economics simply don’t work.

How I was “forced” into AI

“AI engineer” was never a career plan — it was simply the only door left. The verification problem just could not be solved with my constraints — time and budgets. It could only be solved by teaching software to make judgments that previously needed a person: is this a publisher? is this description still true? And that meant learning to build with AI models properly: at this scale you cannot just paste things into a chatbot, you have to wire up models into a live production system that runs without ongoing human supervision.

So I was forced into teaching myself AI engineering.

As an aside, for the benefit of non-technical readers there seems to be a clear distinction now between AI engineering and Machine Learning engineering. The latter are the ones who build the LLMs, while the former use existing LLMs to build solutions. In that sense it fits well with my general trajectory — I love solving problems with code.

The shape of the solution

The solution I envisioned can be imagined as an assembly line with six stations. A profile — new submission or a pre-existing member on its re-verification rotation — moves down the line:

1. Is the front door open? Software fetches the website and runs a battery of cheap, non-AI checks. Is the domain alive? Is it a “domain for sale” page? A casino (someone bought an expired publisher’s domain and reused the domain)? Many profiles are caught right here, at almost zero cost, before any AI is even involved.
2. What is this? The first AI call reads the site and classifies the organisation — a publisher, a service provider, a self-publishing platform, something irrelevant, or something dangerous. Because we are a well-known directory with a high standing with search engines a lot of SEOs try to sneak in tangentially related or sometimes even unrelated websites.
3. Describe it. The second AI call writes the profile: a brief, a full description, and hints about the organisation’s subjects, languages, media and services.
4. File it. Regular software — no AI — matches those hints against a controlled vocabulary, so that ‘book agents’, ‘book scouts’, and ’literary representatives’ all show up under ‘Literary Agents’ — different parts of the world often use different words to say the same thing.
5. Decide. Approve, flag for a human review, or mark for removal — with the reasoning recorded at every step.
6. Do it. The decision is executed: the profile goes live, gets updated, or comes down.

Building and testing it took some time but this approach worked well enough that I deployed it to the live production server. Within the first couple of months I noticed that close to a third of the directory churned — a significant number of entities dropped because their sites were dead or irrelevant to the industry, but many had also pivoted their primary work, going from publisher to supplier or the other way round.

The ‘machine’ was working well at running verification passes on hundreds of profiles every day — work that would have needed a team I did not have. And the entire AI bill for a full pass over the directory costs less than a part-time freelancer. Not counting my time to build it, of course.

Turns out that there was much more to the story than the ‘machine’ itself. I learned a lot along the way — so much that I discovered I like this AI engineering role, enough to change my LinkedIn profile and add ‘AI engineer’ ahead of ‘full-stack developer’. Going forward, I’d love to do much more AI engineering.

What I am learning along the way

The models were not the hardest part. There are many models available at different price points, and I will talk about it in greater detail in Part 2. The really hard part was everything around the model: deciding when and how to use them, deciding what counts as a “publisher” (harder than it sounds — I ended up rewriting those rules about 70 times and counting), knowing when to trust the automation and when to keep a human in the loop, noticing when the system fails silently… The pipeline running today makes fewer AI calls than the one I launched originally — in the end it got even cheaper and faster, which taught me more than the original build did. Well, to be accurate the use of the phrase “in the end” is incorrect because this process never really ends.

That learning is what the rest of this series is about:

• Part 2 — What I learned running an AI verification pipeline over 20,000 publisher profiles: the engineering. The architecture, the real cost arithmetic across model tiers, the prompt-calibration war stories, and the bug that nearly deleted legitimate publishers.
• Part 3 — A survey of what a full-stack developer learned doing AI engineering on a live production system: the rules I’d start with if I were doing a project like this again — most of them learned the hard way.

If you run a content business sitting on a pile of data that’s quietly going stale, some version of this story is probably in your future. I’d be happy to compare notes — and if you think I got something wrong, do say so. I’d genuinely like to know.