There is a common misconception that choosing the right model is the most critical decision in developing an AI-powered application. With the variety of AI models available today, it is easy to get caught up in the race to find the “best” one. However, model selection is not the most important factor in the early stages of AI app development. In fact, it is not even among the top priorities.
When initiating the development of AI applications, the first focus should be on the quality of the data, rather than on which model to choose. The conversation with clients often revolves around model selection, but the crucial question to ask is, “What does the data look like?” The quality of data is far more important than the choice of model, particularly in the early stages of development.
To ensure an AI model works effectively, it is essential to have data that is relevant, clean, and formatted in a way that the model can understand. This involves creating extraction layers to pull data from various sources and transforming that data into a consistent format. Whether the data consists of text, images, or numbers, the performance of the AI model will only be as good as the quality of the data fed into it. The adage “garbage in, garbage out” holds true here.
After extracting and transforming the data, the next step is to ensure its cleanliness. This means addressing issues like missing values, outliers, and inconsistencies that could negatively impact the model’s performance. While this process may be painstaking, it is where the majority of effort should be invested in the initial stages.
Once the data is prepared, the next phase involves experimenting with different models. The objective at this point is not to select the perfect model but to find one that works sufficiently well to move forward. Often, this can be achieved with older, less expensive models.
There is a natural inclination to want to use the latest and most advanced model available. Today, that might be Claude, yesterday it was GPT-4, and tomorrow it could be Google’s Gemini. However, these top-tier models often come with significant costs, and during the early stages of development—when it is still unclear what will work and what won’t—such an expense might not be justified.
Instead, consider starting with a model like GPT-3.5, which is still highly capable but less expensive. The aim at this stage is to achieve about 80% completeness. This means building something functional enough to test, gather feedback, and iterate upon. Once it becomes clear which features are most important and how the application will be used, it can then be determined whether upgrading to a more advanced model is worth the investment.
As data and feedback are collected from initial iterations, it will become apparent which features are critical to the application’s success and which are less significant. This is when model selection begins to matter more.
If a less expensive model has been used and it becomes clear that it is not meeting the application’s needs, that is when it makes sense to explore more advanced options. Perhaps better natural language understanding is required, or the application demands more sophisticated decision-making capabilities. Whatever the case may be, this is the point at which the cost of upgrading to a more powerful model can be justified.
Even then, it is essential to approach this decision with careful consideration. A more advanced model does not automatically guarantee it will be the best choice for a specific application. In some cases, an older model with custom fine-tuning may perform just as well, or even better, than the latest model straight out of the box.
Spending weeks or even months trying to choose the perfect model before starting training is a costly mistake—not only in terms of money but also time and resources. The sooner a working prototype is up and running, the sooner valuable insights can be gained regarding what truly matters for the application. Often, it turns out that the “perfect” model is not what was initially expected.
When developing an AI-powered application, it is crucial not to become overly focused on model selection, at least not at the start. The primary focus should be on data quality and preparation, followed by experimentation with different models. The goal should be to achieve 80% completeness with a less expensive model and then iterate based on the insights gained. Only after sufficient feedback has been gathered and key features have been identified should upgrading to a more advanced model be considered.
In the end, building an AI app is an iterative process that requires flexibility, creativity, and a willingness to learn from mistakes. By focusing on data quality and iterative development, it is possible to build an AI application that truly meets the needs of the project, without incurring unnecessary costs.
