Ask your database in Turkish
How a text-to-SQL model turns a Turkish question into a query, why the database and not the model produces the numbers, and what an honest evaluation of Hati Glint SQL does and does not tell you.
Imagine you run Çınar Atölye, a small online shop that sells soap and tea. Every Monday you want a few numbers: who spent the most last month, what was ordered in October, who lives in Istanbul. The answers are in the shop’s database, but you don’t write SQL, the person who does is busy, and what you want is the way you’d ask a colleague: in Turkish, “Hangi müşteri toplamda en çok harcama yaptı?”, and rows back, not a lecture.
That job has a name: text-to-SQL. A model reads your question and a description of the database, and writes the query that answers it. SQL is the language databases speak: a query says which table to look in, which rows to keep and what to add up. The model writes the query. The database runs it. This post is about the seam between the two, and about what a Turkish text-to-SQL model called Hati Glint SQL does at it.
By the end you will know:
- what a text-to-SQL model reads and writes, and why the database, not the model, produces the numbers;
- how a query can run, return rows and still be wrong, and how a person catches that;
- what execution accuracy measures, what Hati Glint SQL scores on it in our evaluation, and what it is not meant for.
A chatbot that has never seen your orders can only guess the numbers, and a confident guess about your sales is worse than no answer. The database holds the real rows and does exact arithmetic on them. A text-to-SQL model gives it the one thing it can’t do alone: it turns your Turkish question into the precise instruction the database understands.
Meet Hati Glint SQL
#Hati Glint SQL is RuneLab’s first Hati model. It is a 9B model, made by fine-tuning Qwen3.5-9B with LoRA (a method that trains a small set of extra weights instead of all of them) on Turkish-to-SQL work. It reads Turkish and English questions and writes SQLite queries, it is released under Apache 2.0 (on 2026-10-10), and it can run on-premise, on hardware you control. The model card calls it “the strongest open Turkish text-to-SQL model under 10B in our evaluation”, and we will come back to what that qualifier means.
The card is just as plain about what it is not for. Do not run its SQL with write permissions or without review, do not expect it to write any dialect other than SQLite, and do not use it to decide who may see which data. Keep those three in mind: the figure below is built to show why.
It is told. Before the question, the model is given the database’s
From question to rows
#Here is Çınar Atölye’s database: three tables, customers, products and orders, with every row invented for this post. Pick a question and press the button under the question to step through it. First the model is given the
This figure needs JavaScript. It walks five Turkish questions about a made-up shop database (customers, products, orders) through three steps: the schema the model is given, the SQL it writes, and the read-only run. For “Hangi müşteri toplamda en çok harcama yaptı?” the query joins orders, customers and products and adds up quantity times price for each customer: the database returns one row, Mehmet Kaya, 329. A second path shows an invented mistake, a query that adds up prices without the quantity, which runs without an error and returns Elif Şahin, 168. The other questions are the customers in İstanbul, the orders of October 2026, an average age the schema cannot answer, and an instruction to raise prices that the read-only check refuses.
In every case the model’s part ends at a few lines of SQL. When you ask a question, the model reads the schema and writes a query; the database, not the model, produces the numbers. Press through “Top spender” again and read the green rows: Mehmet Kaya’s two orders are 5 × 45 and 2 × 52, and the database added them up to 329. No model counted anything, and you can check every number against the table.
In the figure, which part produces the number 329 for Mehmet Kaya?
- The model, from what it learned about shops.
- The schema, which holds the totals.
- The database, when it runs the SQL on its rows.
Show answer
The database. The model only writes the query; 329 is 5 × 45 + 2 × 52, worked out from two rows of orders and two of products, which are the green rows in the figure.
Not A: the model never sees the rows in this example, so it has no numbers to add up. It could only guess one.
Not B: the schema lists table names, column names and types, and no values at all.
The other questions each teach one more thing. “Customers in Istanbul” is the simplest: one table, one filter. “October orders” compares dates, which this database stores as plain text such as 2026-10-08, so the query compares text. “Average age” has no answer in the schema, so watch what a careful reply does: it says so and writes no query, and nothing is made up. And “Raise prices” is not a question at all but an instruction. Run it and see why the run is called read-only.
When it runs and is still wrong
#That is the case to take seriously, so the figure has a switch for it. Pick a question, step to the end, then change the path from “Normal path” to “What could go wrong”. Each question gets one
Look at what happened. The database was never wrong: it ran exactly what it was asked, every time. “Top spender” gave a different name (Elif Şahin, 168 instead of Mehmet Kaya, 329) with no error at all. “Customers in Istanbul” returned nothing, because the table spells the city with a dotted capital İ and the query did not, and an empty answer is easy to read as “nobody there”. “October orders” let last year’s October in. Three of the five mistakes ran without a single warning.
A query for the customers in İstanbul runs without an error and returns 0 rows. What is the best next step?
- Check the SQL against the data, starting with the spelling.
- Trust it: no error means no customers live there, so the answer is zero.
- Run it again until some rows appear.
Show answer
Compare the query with the data. In the figure the table writes İstanbul with a dotted capital İ and the query wrote Istanbul, so nothing matched. A result you did not expect is a reason to read the SQL.
Not B: “no error” only means the query was valid. It does not mean it asked the right thing, and the table has two customers in İstanbul.
Not C: the same query gives the same rows each time, so running it again changes nothing.
Two of the mistakes are different. “Average age” asks for a column the schema does not have, and the database refuses with no such column: age. That is a safe failure: no number appears. And with “Raise prices”, the only thing between the model’s UPDATE and your data is the read-only check, which is why the card lists running generated SQL with write permissions or without review as out of scope.
How good is it? The honest numbers
#So how often is a model’s query right? The usual measure is execution accuracy: a question counts as right when running the model’s query returns the same result as running a reference query written by a person. Two Turkish test sets are used for it here, TURSpider and BIRDTurk. BIRDTurk is clearly the harder one: every score on it is lower than on TURSpider. All the numbers below are from the model card, measured in RuneLab’s own evaluation on the dev sets, and they say how the model does on those questions, not how it will do on your database.
The figure compares the base model, Qwen3.5-9B, with Hati Glint SQL. Press through the three settings. Direct is one answer with thinking mode off, Thinking turns thinking mode on, and Agent adds feedback (more on that in a moment).
Execution accuracy (%), base model and Hati Glint SQL
One answer, thinking mode off.
Score, from 0 to 100
Hover, tap or focus a row to read it.
- Base model
- Hati Glint SQL
Bars run from 0 to 100. Changes the card does not state are computed from the rounded scores shown.
| Setting | Base model | Hati Glint SQL | Change |
|---|---|---|---|
| Direct | 64.12 | 73.21 | +9.09 |
| Thinking | 71.18 | 76.79 | +5.61 |
| Agent (thinking + feedback) | 66.63 | 77.85 | +11.22 |
Bars run from 0 to 100. Changes the card does not state are computed from the rounded scores shown.
| Setting | Base model | Hati Glint SQL | Change |
|---|---|---|---|
| Direct | 40.09 | 47.26 | +7.17 |
| Thinking | 48.11 | 54.37 | +6.26 |
| Agent (thinking + feedback) | 43.35 | 55.35 | +12.00 |
Bars run from 0 to 100. Changes the card does not state are computed from the rounded scores shown.
In the direct setting Hati Glint SQL scores 73.21 on TURSpider dev against 64.12 for the base model (+9.09), and 47.26 against 40.09 on BIRDTurk dev (+7.17). It is ahead in the other two settings too: 76.79 against 71.18 with thinking on TURSpider dev, and 55.35 against 43.35 with the agent on BIRDTurk dev.
More than four in ten, in that test, yes: a score of 55.35 leaves 44.65 questions in a hundred that did not match the reference result. It is a fair thing to worry about. It is also why the SQL stays on screen next to the rows, why runs are read-only and why a person reads the query before the answer is trusted for anything that matters. A higher score means fewer mistakes, not none.
Specialising also has a cost, and the card reports it. On four general tests the numbers dip slightly after training:
General benchmarks (%), base model and Hati Glint SQL
Score, from 0 to 100
Hover, tap or focus a row to read it.
- Base model
- Hati Glint SQL
Bars run from 0 to 100. Changes the card does not state are computed from the rounded scores shown.
| Task | Base model | Hati Glint SQL | Change |
|---|---|---|---|
| TurkishMMLU | 66.22 | 65.11 | −1.11 |
| Belebele-TR | 86.67 | 86.22 | −0.45 |
| XCOPA-TR | 91.80 | 91.60 | −0.20 |
| MMLU | 78.80 | 78.60 | −0.20 |
Bars run from 0 to 100. Changes the card does not state are computed from the rounded scores shown.
TurkishMMLU goes from 66.22 to 65.11, Belebele-TR from 86.67 to 86.22, XCOPA-TR from 91.80 to 91.60 and MMLU from 78.80 to 78.60. The card also reports gains on English Spider (79.40 to 84.24) and on a held-out Turkish tool-calling set (76.90 to 78.90). The qualifier in the card’s headline matters for the same reason: “the strongest open Turkish text-to-SQL model under 10B in our evaluation” is a claim about these tests, run by the people who made the model.
A question counts as right in execution accuracy when…
- the model’s query is written exactly like the reference query.
- running the model’s query gives the same result as the reference.
- the model’s query runs without an error.
Show answer
It compares results, not text: two differently written queries that return the same rows both count as right. That is why the measure is called execution accuracy, and why the score in the figure above says how often the rows matched.
Not A: many different queries return the same rows, and execution accuracy accepts all of them.
Not C: a query can run without an error and still return the wrong rows, like “Top spender” in the first figure.
Adding a loop: the agent
#The third setting in the figure, Agent, is “thinking plus feedback”. The model card ships a small wrapper in its agent/ folder, HatiSQLAgent, that runs the model’s queries in a read-only sandbox and tries again when a query fails with an error. In plain words: instead of one shot, the model gets to see that the database complained, and to try once more. In the agent setting the base model scores 66.63 on TURSpider dev and Hati Glint SQL 77.85, and on BIRDTurk dev 43.35 against 55.35: the largest gaps in the figure.
Thinking mode has a cost you can count. The model writes out reasoning before it answers, and Hati Glint SQL used fewer reasoning tokens than the base model on the same dev sets: 299 against 1,089 on TURSpider dev, and 522 against 1,582 on BIRDTurk dev. A token is a small piece of a word, so fewer reasoning tokens means less text to generate before the query appears.
That is the limit of the loop. The “Top spender” mistake in the figure would pass straight through an agent that only retries on errors, because the database had nothing to complain about. A loop makes errors cheaper to recover from. It does not replace a person reading the SQL and spot-checking the rows when the answer matters.
Trying it, and what not to do
#The model page, Hati Glint SQL, has the facts, the numbers and the code to run it, and the Hugging Face card is at runelab-ai/Hati-Glint-SQL-9B-v1. The weights in bf16 need about 16.8 GB of GPU memory, and the card lists Hugging Face transformers and vLLM as ways to run it. Thinking mode is off by default, and the card’s example turns it on with enable_thinking=True.
You can run the model on hardware you control, which is what on-premise means here. Whatever else you connect to it, a chat window, a dashboard, a script that sends the rows somewhere, still has its own rules about where data goes.
If you try it, a short list of what works and what doesn’t, from what we saw above:
- Do give it the schema of a SQLite database, and keep the run read-only.
- Do show the SQL next to the rows, and read it when the answer matters. Spot-check one number by hand.
- Do treat an empty result or a surprising number as a reason to read the query.
- Don’t give it write permissions, or run its SQL unreviewed on data you care about.
- Don’t expect queries for other SQL dialects, or let it decide who may see which data.
Further reading
#The model card is the source of every number about Hati Glint SQL in this post: Hati-Glint-SQL-9B-v1 on Hugging Face. The names TURSpider and BIRDTurk echo two English benchmarks that are the usual starting points for this task: Spider, a dataset for cross-domain text-to-SQL, in “Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task” (Yu and colleagues, 2018), and extended to larger, messier databases in “Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs” (Li and colleagues, 2023). LoRA comes from “LoRA: Low-Rank Adaptation of Large Language Models” (Hu and colleagues, 2021).
This post leaves a lot out: how the model is trained, how large real databases are described to a model, and every kind of mistake a query can contain. The shop, its rows, the model’s replies and the mistakes in the first figure are invented for teaching, and the queries shown were run in a real SQLite to check that they give the rows shown, but no model produced them live.
Conclusion
#Text-to-SQL splits a question into two jobs. The model reads your schema and writes a query; the database runs it and produces the numbers. That split is what makes the answers checkable, since every number traces back to rows in a table, and it is also where the risk sits, because a query that runs is not a query that is right. Hati Glint SQL scores higher than its base model on two Turkish test sets in RuneLab’s own evaluation, with small dips on general tests, and it is still far from perfect. Use it with a schema, read-only, with the SQL on screen and a person checking what matters.