How to develop AI-powered products effectively
There are a lot of techniques to use when developing AI-powered products. Fine-tuning, training your own models, prompt engineering, RAG, etc. What should you use?
Pre-maturely building software of any kind is a waste of time and money. Here's how to know when it makes sense.
We’ve all seen such memes as the one above.
Automation is great and is a common use case for AI-powered software. But it doesn’t always make sense, of course.
Some degree of planning (and expertise with the technology) should allow you to figure it out. I’ll give you a few examples.
The Gist of It is a very small Chrome extension that uses chatgpt-4o-mini and a little bit of prompt engineering to help you summarize any content you come across in your browser (emails, long tweets, articles, etc).
It took me just a day to make it and it’s entirely free and open-source.
It’s most certainly a very simple, no moat, wrapper over ChatGPT. And you could always copy+paste text and prompt ChatGPT yourself with the OpenAI web interface.
But then you’d have to copy, navigate there, figure out what prompt to write and paste the text in. Might take you a few minutes.
With the extension, you just right click the text and click “Get The Gist of It”. In a few seconds, you have a summary.
If you’re thinking of the above meme, you’re wrong.
I have to skim / summarize long texts multiple times a day to figure out if I want to read them. My reading list was unsurmountable previous to this little extension. Now it’s entirely empty.
In between The Gist of It and ThatNeedle Gist (an extension that does longer form summaries for YouTube videos), I managed to clean up my “watch later” list too.
And all in less than a week.
It’s a huge help with my information overload and likely will save me tens of hours, maybe even a hundred over the next year.
Another example of a good automation are…
Whether internal or external, a documentation-based chatbot can be quite powerful. It allows its users to extract actionable insights from a knowledge base that may very well be huge.
Klarna is a good example of a company that did this for customer service. The AI assistant they’ve implemented was able to handle about two-thirds of Klarna’s customer service interactions, equivalent to the work of 700 full-time agents [1] [2]
That was estimated to drive a $40 million USD profit improvement to Klarna in 2024.
Now you might not be Klarna. You might not have 700 employees in total, let alone customer service agents.
I’ve seen companies with as little as 20 employees have relatively big, unmanageable knowledge bases split up over Notion, Hubspot and Linear.
I alone was probably wasting at least a few hours a week going through the overhead of finding what I need, keeping things updated, etc. I can’t imagine what people in the biz dev team or the PMs had to do.
Some of it could be automated to various degrees without AI, sure. But back then (2021-2022) we didn’t even have the option of using AI. Nobody really knew what RAG is.
So we just kind of pushed through and ate the overhead.
I imagine a similar company with a few AI-powered tools could easily shave off a collective 40-60 hours per week. Very possibly more. We can quantify that in the range of $50k-$150k per year.
That’s big enough that someone should fix this inefficiency. Maybe not big enough for them to make a custom tool, but that’s arguable.
In other words you have to quantify the cost of *not* having
If you are building an internal tool, I think it’s easier to do the napkin math. How many hours do your employees spend doing X? Get a quote from a few development shops like mine for the tool. How far into the future is the expected break-even point?
If you’re comfortable with the numbers, go for it.
If you’re a startup figuring out if a tool has value on the larger market, it will be trickier. You not only have to quantify the cost your potential users are incurring by not having this. But also how many of them are out there, if they really even care about this cost and what are their alternatives.
And you have to build this tool as a product, which is always more work than an internal tool. Interal tools can be less polished and don’t usually require billing and other similar systems.
Those are all questions you should be asking yourself before diving into this endeavour.
As you can see, there are nuances to this.
If you’re a startup that needs help ideating / strategizing on an AI-powered product… If you’re a company that needs a custom automation solution…
Book a free consultation with me and let’s figure out if we’re a fit and I can help you.
There are a lot of techniques to use when developing AI-powered products. Fine-tuning, training your own models, prompt engineering, RAG, etc. What should you use?
Learn how to efficiently build a minimum viable product (MVP) in just 30 days while keeping costs low.
Serverless tends to push the 'edge' paradigm. The app server lives close to the user to save latency. But it's problematic if you have a database because, generally, you will have multiple db queries for each page / API request. Let's talk about that