manuelaschrittwieser/Qwen2.5-SQL-Assistant-Prod

Qwen 2.5 (1.5B) - SQL Assistant Production Adapter

📑 Model Overview

Qwen2.5-SQL-Assistant-Prod is a specialized parameter-efficient fine-tune (PEFT) of the Qwen 2.5 1.5B Instruct large language model.

This model is engineered to function as a technical assistant capable of translating natural language questions into syntactically correct SQL queries. It relies on a provided database schema (context) to ensure column and table names are hallucination-free.

Key Capabilities

• Context-Aware Generation: Adheres strictly to the CREATE TABLE schema provided in the prompt.
• Low-Latency Inference: Optimized for consumer hardware (CPU friendly) due to the lightweight 1.5B parameter count and LoRA adapter architecture.
• Agentic Integration: Designed to serve as the reasoning engine for autonomous SQL agents.

---

📊 Training Telemetry & Metrics

The training process was tracked and visualized using Weights & Biases to ensure stability and convergence.

👉 Click here to view the full W&B Training Run / Charts

• Training Loss: Monitored to ensure no overfitting on the SQL syntax patterns.
• Hardware: Trained on NVIDIA T4 GPU.
• Framework: Hugging Face trl (SFTTrainer) + peft.

---

🛠️ Technical Specifications

Architecture

• Base Model: Qwen/Qwen2.5-1.5B-Instruct
• Adaptation Method: QLoRA (Quantized Low-Rank Adaptation)
• Quantization: 4-bit NormalFloat (NF4) via bitsandbytes

Hyperparameters

Parameter	Value	Description
LoRA Rank (r)	16	Dimension of the low-rank update matrices.
LoRA Alpha	16	Scaling factor for LoRA.
Dropout	0.05	Regularization to prevent overfitting.
Target Modules	q_proj, k_proj, v_proj, o_proj	Layers targeted for adaptation.
Learning Rate	2e-4	Initial learning rate.
Batch Size	4	Per-device training batch size.
Epochs	1	Single pass over the instruct dataset.

Dataset

The model was fine-tuned on b-mc2/sql-create-context, a dataset containing pairs of:

1. Context: SQL Schema definition (CREATE TABLE...).
2. Question: Natural language inquiry.
3. Answer: Gold-standard SQL query.

---

💻 Usage Instructions

This model is an adapter. You must load the base Qwen model first and then attach these adapter weights.

Prerequisite

pip install transformers peft torch

Inference Code

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# 1. Configuration
BASE_MODEL_ID = "Qwen/Qwen2.5-1.5B-Instruct"
ADAPTER_ID = "manuelaschrittwieser/Qwen2.5-SQL-Assistant-Prod"

# 2. Load Base Model (Load in 4-bit for efficiency if using GPU, or float32 for CPU)
base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL_ID,
    device_map="auto",
    torch_dtype=torch.float16
)

# 3. Load the Adapter
model = PeftModel.from_pretrained(base_model, ADAPTER_ID)
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL_ID)

# 4. Define Context & Question
schema = "CREATE TABLE users (id INT, name VARCHAR, age INT, city VARCHAR)"
question = "How many users live in Paris?"

# 5. Format Prompt (Qwen Chat Template)
messages = [
    {"role": "system", "content": "You are a SQL expert."},
    {"role": "user", "content": f"{schema}\nQuestion: {question}"}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

# 6. Generate
inputs = tokenizer(text, return_tensors="pt").to(model.device)
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=100)

# 7. Decode Output
print(tokenizer.decode(outputs[0], skip_special_tokens=True).split("assistant")[-1].strip())

---

⚠️ Limitations

• Prompt Sensitivity: The model performs best when the database schema is explicitly provided in the system or user prompt.
• Complex Logic: While proficient at JOIN, GROUP BY, and WHERE clauses, highly complex nested sub-queries or database-specific dialect functions (like PostgreSQL JSONB operators) may not be generated correctly.
• Security: This model generates SQL code. Always sanitize and review generated queries before executing them on a production database to prevent SQL injection vulnerabilities.

---

📜 Framework Versions

• PEFT: 0.18.0
• Transformers: 4.37.0+
• TRL: 0.26.2
• PyTorch: 2.1.0+