What to Look For in an AI Chatbot for Shopify

An AI chatbot on a Shopify store speaks for your brand to every visitor who opens it. When it is good, shoppers get a straight answer at 2 a.m. and your team gets fewer repeat questions. When it is bad, it invents a returns window, quotes a price you changed last week, or traps an angry customer in a loop.

The demos all look alike. The differences show up in the details below, so test each one on your own store before you commit.

1. Where its answers come from

This matters more than anything else. A general-purpose language model knows a lot about the world and nothing about your store. Ask it about your shipping times and it will produce a plausible answer, which is worse than no answer.

A useful Shopify chatbot looks things up in your store’s own information before it replies:

• Products, read live, so prices, stock and variants are current.
• Orders, so it can give a shopper the status of their own order.
• Policies and pages, such as your refund and shipping policies.
• Answers you write yourself, for the questions your policies don’t cover.

Ask each vendor how up to date each source is. Some sources can be read live at the moment of the question. Others are copied in advance and refreshed on a schedule, which means a policy you edit this morning may not reach the chatbot until the next refresh. Neither is wrong, but you need to know which is which.

2. What it does when it doesn’t know

Every assistant will meet questions your store information doesn’t answer. Watch what happens. The right behavior is to say so plainly and offer a person. The wrong behavior is a confident guess.

Test it deliberately. Ask about a product you don’t sell, a policy you don’t have, or a discount that doesn’t exist. If it makes something up, keep looking.

With WOLE, every answer that isn’t small talk has to be backed by something it looked up or by your own house rules. If it cites nothing, the reply is replaced with “I don’t have that information yet.” and an offer to talk to a person.

3. Product and order cards, not just text

Shoppers buy from pictures and buttons, not paragraphs. A chatbot that answers “do you have this in blue?” with a card showing the product, its variants, stock and an Add to cart button turns an answer into a sale. Check that Add to cart puts the item in your store’s own cart rather than sending the shopper somewhere else.

The same goes for orders. “Where is my order?” deserves the order’s status and a tracking link, not a paragraph telling the shopper to check their email.

Also check what the chatbot reads to answer an order question. It needs the status and tracking, not the shopper’s address or payment details. Narrow access is a good sign. WOLE’s order lookup, for example, reads the order name, date, fulfillment status and tracking only, never line items, addresses, email or phone.

4. Whether it knows which page the shopper is on

A shopper on a product page who types “does this come in a larger size?” means this product. A chatbot that can see the page type, the product and what is in the cart can answer that without asking “which product do you mean?”. Without page context, every conversation starts from zero.

5. How it hands a chat to a person

No chatbot should handle everything. Look for:

• A way for the shopper to ask for a human at any time, which cannot be switched off.
• Built-in triggers for the situations a bot should never handle alone: money already paid, an upset shopper, sensitive topics, and its own low confidence.
• Rules you can write, for situations specific to your store, such as wholesale inquiries or VIP customers.
• Business hours, so a shopper outside them is told when to expect a reply.
• A shared inbox where your team picks the conversation up with the full history, and the bot stops talking once a person joins.

We cover this in more depth in When Should a Chatbot Hand Off to a Human?.

6. Languages

If you sell across borders, check whether the assistant replies in the shopper’s language without you translating everything first. Then check the parts that don’t change: the chat window’s buttons and labels, your greeting, and your suggested questions may still appear in the language you wrote them in. Ask which parts are translated and which are not. Also ask about right-to-left languages such as Arabic and Hebrew if you serve those markets; support is not universal.

Our guide to multilingual customer support goes through the rest.

7. Testing before launch

You would not publish a product page without previewing it. Treat the chatbot the same way. The best setups give you a private test area that runs the real assistant on your real store information, so you can ask your top questions before any shopper does.

Even better is a test area that explains itself: which sources it used, why it handed off, and how confident it was. That turns testing from “does this look right?” into “what do I need to fix?”. When you find a gap, the fix is usually a better FAQ answer; see How to Write FAQ Answers an AI Can Use.

WOLE’s Sandbox works this way. It runs the real AI on your knowledge and settings, costs nothing to use, and shows for each turn what happened, the evidence used, the language and the confidence.

8. How it is priced

AI chatbots are priced in very different ways: per conversation, per resolved conversation, per seat, or by usage. Questions worth asking:

• What counts as a billable unit, and is testing free?
• What happens when you hit your limit? Does the bot stop, start charging more, or hand chats to your team?
• Can you cap your spend?
• Will you be warned before you reach the limit?

Predictability matters more than the headline price. A plan that quietly runs up overage in your busiest week is more expensive than it looks.

WOLE uses monthly credits, including a Free plan. Shopper replies and AI drafts for your team use credits; the Sandbox does not. Going past your allowance is off by default, so when credits run out new questions go to your team instead. The store owner gets email alerts at 80% of the allowance and at 100%. On a paid plan outside a trial, you can turn overage on and set a monthly dollar cap, and the owner is alerted at the cap too. The pricing page has the current plans.

A short test script

Before you choose, run every candidate through the same ten minutes:

1. Ask your three most common questions, word for word.
2. Ask about a product on the page you’re viewing, using “this”.
3. Ask for something you don’t sell.
4. Ask where an order is, as a guest.
5. Say you want a refund for something already paid.
6. Ask to talk to a person.
7. Ask a question in another language your shoppers use.

The chatbot that gets these right on your store, not in a demo, is the one to keep.

Where WOLE fits

We built WOLE around the points above: answers only from your products, orders, policies, pages, FAQ, files and links; product and order cards in the chat; page context; handoff rules you can see; replies in the shopper’s language; and a free Sandbox to try it first. You can see how it learns your store on the AI training page.

Whatever you choose, judge it on your own questions. That is the only test that counts.