NL2SQL Explained: What It Is, Why It Flopped, and How It’s Being Reinvented

Sales: “Which products brought in the most revenue this quarter?”

Marketing: “What’s the conversion rate from sign-up to purchase this week?”

Logistics: “Which suppliers delivered late shipments last month?”

HR: “What is the average salary by department?”

Finance: “What’s the outstanding amount in unpaid invoices?”

If you work in any one of these roles, you’ll have heard a variation of at least one of these questions; many more times than you can count, and probably more often than you’d like. 

But if you work in data, you’ll have heard them all; every variation, thousands of times, day-in-day out. 

Side effects include : hearing your colleagues in your nightmares.  

Now imagine if you could just simply ask your database directly, as if it were a friend? 

A friend you can depend upon because he’s (annoyingly) smart, never lies and always keeps your secrets safe.

The Rise (And Fall) of NL2SQL

Natural Language Processing or NLP (the science of getting computers to talk) has improved massively over the past couple of years. 

LLM technology the likes of ChatGPT, Claude, Gemini are obvious examples.

But there are also very specific use cases where NLP has been applied, drumroll please… 

… to write good old traditional SQL.

And that’s how NL2SQL was born. 

When the first tools came out, NL2SQL was the prodigal child of data analysis.

The "El Dorado" of BI with a golden promise:

Getting data-driven answers to business analytics questions, as quickly and easily as talking. No SQL required.

In theory, by translating plain language into structured SQL queries; NL2SQL suddenly gives everyone (marketers, business analysts, sales, HR, finance) direct access to interact with databases, explore data and get insights. 

Even as a data analyst or someone familiar with SQL, it : 

• Frees you from having to write SQL queries manually

• Allows you to focus on higher level strategic analysis

• Removes the friction for iterative deep-dive analysis

With NL2SQL, you could just think:

“I wonder which marketing campaigns led to the most signups that then became paying customers within 30 days?" 

And instead of having to make a complex sequence of multiple SQL requests that take days to write and feel like dragging your brain through sand, you would just get an answer.

That was the theory at least. 

NL2SQL Limitations in Real-World Data Analysis

It's easy to see why NL2SQL sounds great on paper. The problem is that the original versions of the tech weren’t able to deliver in real life situations.

Data Schemas

First of all, most businesses (even multi-billion dollar corporate giants) have messier data environments than they’d like to admit. Well structured data schemas are rare, and modern businesses can have 100s, even 1000s of tables, spread across several databases. 

So when a user asks for “Q1 enterprise deals”, how is an NL2SQL AI supposed to know which datasets are relevant or not? 

Context & Ambiguity

The second problem is that users - especially non-tech users - often ask ambiguous questions. Asking for “top customers” can mean lots of different things. 

Is it how much they spent? How many orders they placed? Their lifetime value?

Without clarity or context, the AI can generate SQL that misunderstands the question or the expected results. 

One-Shot NL2SQL

Last but not least, first generation NL2SQL tools only provided one-shot translation. 

Question in -> SQL query out -> Results

No nuance. No follow up questions. No digging deeper.

It helped save time, but wasn’t the true “chatbot-style conversation with your data” that analysts expected. Keep in mind this was all happening as we were discovering LLMs like ChatGPT, Claude and Gemini - so there was a clear disconnect. 

AI NL2SQL Agents: The Future of Business Analytics

If you were worried we’d give you a list of problems and just leave you hanging, I’ve got some good news. 

There’s a solution : An AI agent for data analysis.

A new era in NL2SQL copilots that understands nuance, reasons like a human and is adapted to the realities of business data.

With these new generation data agents, you could just think:

“I wonder which marketing campaigns led to the most signups that then became paying customers within 30 days?" 

And instead of having to make a complex sequence of multiple SQL requests that take days to write and feel like dragging your brain through sand, you would just get an answer.

In more technical terms for the data analysts out there, NL2SQL agents : 

• Truly understand business context, metadata, and semantic layers

• Go through result validation protocols before rendering to the user

• Provide output justification and fully auditable access to the AI’s workflow

  
  

How Myriade Improves on NL2SQL

At the risk of sounding cliché, Myriade is an intelligent agent for data analysis, that was truly designed by data analysts for data analysts. 

We’ve built the copilot we always wished we had. 

Because you the answers you need should be the very minimum you expect out of an AI data copilot.

Myriade iterates, refines, double-checks, and explains as an AI agent should… but power and speed shouldn’t come at the expense of security - and convenience is always a plus.

Which is why Myriade was designed specifically to include : 

• Secure-by-design architecture (read-only, zero knowledge protection, zero data retention…)

• Explainability baked into every answer

• Plug & play integration onto databases/datalakes (<5 minutes) 

• Optional open-source or cloud solutions

If you’re interested in trying it, just click here. It’s free (no credit card required), and takes minutes to set up. Your data analysis agent could be up and running before your coffee gets cold.