1
The rich text element allows you to create and format headings, paragraphs, blockquotes, images, and video all in one place instead of having to add and format them individually. Just double-click and easily create content.
A rich text element can be used with static or dynamic content. For static content, just drop it into any page and begin editing. For dynamic content, add a rich text field to any collection and then connect a rich text element to that field in the settings panel. Voila!
Headings, paragraphs, blockquotes, figures, images, and figure captions can all be styled after a class is added to the rich text element using the "When inside of" nested selector system.
NineTwoThree helped design and implement the system alongside CR’s engineering and product team.
Imagine shopping for a mattress and being able to give an AI chatbot your preferences such as: “I’m a tall person who sleeps on my back, what is a good mattress for me?” and getting a personalized recommendation.
For over 80 years, CR has earned the reputation of delivering unbiased and well researched product information to consumers. CR is well-positioned to deliver a powerful AI chatbot that can help consumers get their product questions answered with reliable, unbiased information.
CR’s innovation team was quick to move towards building an AI-powered tool that can help people doing product research get to the right answers faster. They hired NineTwoThree as their development partner because of our engineers’ proven AI/ML expertise launching similar systems to hundreds of thousands of users in production, as well as our experience working with innovation labs from both public and non-profit companies.
The goal of this 8 month project was to:
A shopper should be able to ask AskCR a question like, “I’m a tall person who sleeps on my back, what is a good mattress for me?” and get a personalized recommendation. To do this AskCR has to understand the nuance and intent of a consumer’s question and provide relevant and meaningful information to inform their purchase.
The biggest challenge for our teams was building a sophisticated RAG system that could understand user’s ambiguous queries and then elegantly find answers and relevant information from CR’s vast database of articles, ratings and reviews. The system needed to be able to respond to questions from “Is the Toyota Prius a good car?” to “I'm looking for a 36 inch Gas stovetop with a child-lock feature under $1800” and provide an answer in the voice of a CR expert.
We started by conducting data exploration to understand the structure of the data we were working with. At the same time, NineTwoThree and CR’s product managers worked with CR product experts to establish the ground truth of expected answers so we could evaluate the quality of AskCR’s responses.
The ability to respond with an informed recommendation relies on a custom approach to agentic AI that relies on a set of dynamically configured steps, going from user input to system output. Our teams went through numerous iterations and tested several techniques to balance maintainability, cost, quality and security.
CR has ratings, articles and information about thousands of products and cars in their databases and all of these data sources have different features.
Our first step was to write detailed instructions for the model on how to interpret CR’s structured ratings and reviews data. To do this at scale and quickly, we built a robust data pipeline capable of handling hundreds of product categories and dynamically generating data schemas for explainability purposes. To do this, we worked with CR’s product experts to define explicitly what features of an air purifier such as water removal and humidistat accuracy mean to a consumer making a purchasing decision so we could instruct the AI on how to interpret the data like a human would.
Next, we had to design a query refinement and routing system that would allow the model to transform the user’s question and then route to the right place in the data to pull in the context that could help answer the question.
The balance we had to find was between simplicity and maintainability vs. accuracy and quality. The initial approach was easily maintainable, but not as accurate, so we decided to replace our general-purpose agent with a customly built agent made up of multiple dynamically configurable subsystems. These take each query through a refinement, routing, source retrieval, and final synthesis step in order to return an output.
As quality of responses improved, the challenge was making sure that AskCR could still respond in a few seconds rather than minutes. To optimize for speed, we implemented parallelization and a stepwise approach which allowed multiple systems to run in parallel to summarize the information.
Next, the model needed to be able to understand if a shopper is doing general research, asking about a specific product, or looking for help navigating between choices. To do this, we instructed the model to identify intent and topic with a few examples and taught it to look for similar examples.
LLMs by their nature do not remember previous questions you asked in a conversation on their own. To make sure AskCR had awareness of past questions,we had to implement memory. Initially, it was just a buffer of the latest interactions but by the end of the project, we built a hybrid solution that could retain both the latest interactions and long-term memory.
Quality evaluation and security are always important for AI, and even more so for consumer-facing experiences like AskCR. To ensure the system didn’t get worse at answering both basic questions and illegal or dangerous questions as we continually changed prompts, routers and agents, we implemented guardrails and an evaluation suite. It was critical to have an evaluation suite to know if the system was getting better or worse with time, especially on critical security questions and questions about harmful topics like violence and illegal activity. Our teams tested extensively, including security testing, “red teaming,” and iterative evaluation of AskCR’s responses to a wide variety of questions. We were able to improve guardrails performance by over 10X and implemented a product retriever to allow the model to be able to answer more of CR’s products.
Throughout the entire schedule of the project, NineTwoThree collaborated with CR’s internal user research team to conduct numerous rounds of external user feedback on the design and concept, as well as internal testing with Consumer Reports product testers and employees who stress tested the actual system to ensure it could handle the concurrent load of beta users.
NineTwoThree designed screens for the web and mobile experience. We collaborated closely with CR’s internal design team to ensure consistency with CR’s brand.
AI chat experiences are everywhere now, but many users still don’t know how to use them. The challenge for NineTwoThree’s design team was to design a system that felt familiar and intuitive. Our designers created several iterations of a user interface that explored interaction experiences that strategically used starter prompts and animations to model to users what types of questions they could ask of AskCR and how it could help them with research.
In June 2024, CR announced the successful beta launch of AskCR, available by invitation only. NineTwoThree was excited to bring our experience in AI/ML projects to this cutting-edge use of AI and support CR’s Innovation Lab team in building the next generation of CR tools that help consumers navigate the market.
You can read more about the process of developing AskCR on CR’s Innovation blog. And if you’re interested in experiencing AskCR for yourself, you can join the waitlist here. The rollout of AskCR will be gradual, ensuring thorough testing and continuous improvement based on user feedback and performance evaluations.
