Don't let the LLM count: fixing a job-skills digest

One of the workflows in my n8n agent sends me a weekly digest of the skills employers ask for in ML engineer postings. The idea is simple: pull fresh postings from the HH.ru API, extract the requirements, normalise them, count them, report the top fifteen.

The first version handed all of that to one prompt. Here is a real output, from a run capped at three postings:

This week, I have saved 3 fresh vacancies in my chosen field.
The most popular requirements are:
Python: 2 times
NLP: 2 times
LLM: 2 times
Backend Frameworks: 2 times
Machine Learning Engineers: 2 times
AI Agents: 1 time
Distributed Team Management: 1 time
...

It reads fine at a glance, and it is wrong in three different ways.

What went wrong

"Machine Learning Engineers" is not a skill. It's the job title. The model was asked to extract requirements and pulled a noun phrase that sounded important.

"Backend Frameworks" is a category, not a requirement. One posting probably said FastAPI and another said Django. The model "normalised" them by going one level up, which destroys exactly the information the digest exists to show.

The counts can't be checked. Did Python really appear in two of three postings, or in all three with one phrased as "Python 3.10+"? There is no way to know from the output, because extraction, normalisation and counting all happened inside one opaque step.

None of this is surprising. Language models are good at reading messy text and bad at bookkeeping. Counting across a list is bookkeeping.

The design I'd use instead

Split the job by what each part is good at:

HH.ru API → per posting: LLM extracts skills as JSON
          → Code: normalise with a dictionary
          → Code: count and rank
          → LLM (optional): one friendly sentence on top
          → Telegram

1. Extract per posting, as structured output

One call per posting, with a narrow job: return the explicit technical requirements as a JSON array of short strings. No counting, no summarising, no categories.

Return a JSON array of the technical skills and tools this posting explicitly
requires or prefers. Use the names as written (e.g. "FastAPI", not "backend
frameworks"). Do not include job titles, soft skills, or benefits.
Return [] if there are none.

Per-posting calls cost a little more than one big call, but each output is small enough to inspect, and one bad posting can't contaminate the others.

2. Normalise in code

Normalisation is where consistency matters most, so it shouldn't be left to a model that may decide differently next week. A plain dictionary in an n8n Code node does it deterministically:

const aliases = {
  'питон': 'python', 'python3': 'python', 'py': 'python',
  'pytorch': 'pytorch', 'torch': 'pytorch',
  'k8s': 'kubernetes',
  'postgres': 'postgresql', 'postgre': 'postgresql',
};
 
const normalise = (s) => {
  const key = s.toLowerCase().trim().replace(/\s+\d+(\.\d+)*\+?$/, ''); // "Python 3.10+" → "python"
  return aliases[key] ?? key;
};

The dictionary grows as you read the outputs. That's the point: every new alias is a decision you made once, on purpose, instead of one the model makes differently on every run.

3. Count in code

Count each skill once per posting, so a posting that says "Python" five times still counts once:

const counts = {};
for (const item of $input.all()) {
  const skills = new Set(item.json.skills.map(normalise));
  for (const s of skills) counts[s] = (counts[s] ?? 0) + 1;
}
 
const top = Object.entries(counts)
  .sort((a, b) => b[1] - a[1])
  .slice(0, 15);
 
return [{ json: { total: $input.all().length, top } }];

Now the numbers are exact, repeatable and easy to check against the source postings.

4. Let the model write, if you want it to

The final message can be a template. If you'd like it to read more naturally, give the model the finished counts and ask for one or two sentences on top. It is now formatting facts, not inventing them.

The general rule

When a workflow mixes language and arithmetic, give the language to the model and the arithmetic to code. Extraction, classification and summarising are model jobs. Counting, sorting, deduplicating, joining and anything you'd want to audit later are code jobs.

It is the same rule I follow in data pipelines: keep the non-deterministic step as small as possible, and make everything after it boring.