Balancing Complex and Simple ML Solutions

If you’re building state-of-the-art AI solutions, you’re probably excited by the prospect of what you can accomplish. Maybe you’ve got ideas for every last part of your product - marketing, design, sales, legal, whatever. Or maybe you’re laser-focused on a specific internal tool - one that will unlock dozens of hours of productivity per week for your team.

It’s important to recognize when to build simple, and when to build large, complex systems. Coming to this decision should be a team effort, and it’s important to come to the discussion informed.

(We speak from experience.)

Let’s go over a few strategies for balancing complex and simple ML solutions.

A Little Research Goes a Long Way

Those middle two points are especially critical.

Some companies have robust, state-of-the-art data practices with well-funded data science teams. Others send their data to each other via email. To get from email to a data pipeline can take months of hiring, building, and testing - and that’s all before even touching an ML model.

Similarly, priorities change a project. It could be the team is willing to spend big if it means accuracy is as performant as possible. Or, they might have drank too much of the AI hype kool-aid and are expecting this project to wrap up ASAP. The answer to that question changes how complex the solution is.

But we’re getting ahead of ourselves. Does the project even NEED ML? 

“We see a lot of solutions that don’t need an LLM at all. All the client needs is the right developer on the project with a SQL query, and they save months of work and hundreds of thousands of dollars of R&D.” says Jurgis Samaitis, an MLOps Engineer at NineTwoThree.

Many times, a solution like a forecasting tool can be done cheaply and effectively with a simple library like XGBoost. No fancy LLMs involved. 

Understanding the current state of the team is table stakes for solving the question of “How complex does this need to be?”

Speak Their Language - Cost

If you do all this research, you’re going to have to make the decision with your teammates.

Instead of saying “This is the right way to do it”, give them options. Like we discussed before with priorities - even if they make it crystal clear that accuracy is their priority, it’s a great idea to give them solutions that are 2% less accurate but 25% cheaper and quicker to develop. It could be that 2% doesn’t matter as much to them, and they’re not realizing cost is an issue.

That type of discussion and realization can only happen if you approach this as a conversation, and give them options.

“Don’t give them yes or no questions. Give them multiple options, where one solution is faster, the other is cheaper, one is more costly and sophisticated, and one might be a blend. It’s always important to give alternatives. Our job is to communicate this and understand what the customer is telling us.” says Vitalijus Chernei, an ML Engineer at NineTwoThree.

And when you’re presenting a complex solution, it’s helpful to break it down further. If they want to add state-of-the-art image generation, explain the added cost and timeline impact. 

Want to really go the extra mile? Explain it from a “running the business” perspective. 

Having an agency (like NineTwoThree) that understands how a business runs comes in handy here. We can calculate the costs to run the project per user, per month, and explain that the $100 cost outweighs the $10 subscription fee you’re charging users. 

That will really resonate with team members, and help them see how architecture decisions impact their bottom line.

Understand the Lifecycle of a Project

Some projects are useful for a quarter. Others are meant to form the backbone of the company’s strategy for the next decade.

As you might expect, this affects complexity. Not just in how it's built, but how it's maintained. You’ll need to figure out what maintenance looks like.

“We were building a revenue forecasting system for more than 20,000 restaurants. Using historical data, we reached 90% accuracy. Within 10%, we were able to predict revenue, months in advance. And then, the performance plummets. For seemingly no reason. The only indicator was the date - March of 2020. So the system worked great, until something no one predicted happened.” says Vitalijus.

Hopefully, we’re not due for another global pandemic to throw off everyone’s algorithms. But the meta point is clear - there are some things you can’t account for in building these solutions.

Even with issues you can account for (like model drift), there’s constant maintenance and upkeep. Who’s going to handle that? Which internal team will work on it? Do they have the expertise? Can you hire for it? All important questions that affect how complex the solution is.

Understand It’s Not All-or-Nothing

While it’s hard to break up an AI model, it’s much easier to break up an application into smaller parts. 

Some parts of the architecture might be easier to work on, with simpler solutions. Others might require more work.

Deciding on a proof-of-concept can help reel in work and costs, too. If you’re not happy with a two-week-sprint MVP that does one core thing well, it might not be worth the effort to build a large, complex system. That helps you find things out early, and avoid wasted time and money.

“Quick and dirty” looks a lot different in consumer apps vs regulated financial products. 

Where Do We Go From Here?

There are many considerations that go into building out a ML solution. It comes down to the priorities of the team, what they’re comfortable tackling, and just as importantly, what they’re comfortable supporting long-term.

Approach the conversation as a collaboration, and understand there’s not usually a perfect answer. Building effective ML solutions means letting go of the hype and treating this like any other project - with realistic goals, effective budgets, and the right team behind it.