Engineering notes · Sources
Five financial supervisors, four different answers to the same question about reuse
There is a question the Spanish commercial gazette cannot answer, and it is often the one that matters.
BORME records that a company was incorporated, who its directors are, what its shareholders resolved, when it was wound up. It is a faithful record of corporate acts. It says nothing at all about whether anyone is allowed to do anything: that a firm may take deposits, place insurance, run a betting site, provide telecoms, or manage other people's pension money. Those permissions live with supervisors, and supervisors keep their own registers.
We added five of them: the securities regulator, the insurance and pension supervisor, the telecoms regulator, the gambling regulator, and the central bank. It took a day, and almost none of that day went on the thing we thought we were collecting.
The identifiers were the least of it
Every one of these registers publishes tax identifiers, and together they moved our NIF coverage by about a quarter of a percentage point. That is real and it is not the point.
What they carry that BORME structurally cannot is authorisation, and its loss. A supervisor's register does not merely list who exists; it records when an entity was entered, what it was cleared to do, and — the part nobody else publishes — the day that was taken away.
One register turned out to be entirely historical: every provider in it struck off, none active, because the regime moved elsewhere. That is not a defect in the file. It is a complete record of a category of business that existed and then stopped existing, with both dates for every firm in it, and there is nowhere else to read it.
Another carries thousands of separately dated authorisations naming the exact activity permitted and the law it rests on. Another lists the domains an operator trades under, which is the one thing a gazette could never print: a company's brand is not its registered name, and no official record connects them.
Trap one: a checksum cannot tell a company from a person
A Spanish tax identifier for a company carries a check character computed from the digits before it. We use that arithmetic constantly — it is the cheapest data-quality instrument we own, and we have written about it before.
One of these registers holds around 56,000 entries, and about 40,000 of them are private individuals. An insurance agent is usually a person, not a firm. Their registration key embeds a document number, and more than three thousand of those keys pass the company check character.
They are arithmetically perfect. They are also not companies. One of them resolves to a named individual whose registration code happens to satisfy the same formula a Sociedad colectiva's identifier would.
Had we trusted the arithmetic, we would have filed several thousand private people into a commercial dataset, under their own names, as if they were businesses. Nothing would have failed. The check character would have agreed with us the whole way.
What saved it was the register's own classification: every entry says whether the holder is a legal or a natural person. So the rule became — the register decides who is a company, and only then do we look for an identifier. Never the other way round.
The same question came up in the last register we looked at, and there the answer had to be different. It publishes names, registration numbers and dates, with no identifier and no field distinguishing a company from a person — and the names are plainly both. There was no way to keep the firms without taking the individuals too, so we left the whole register alone. Five thousand rows we could technically parse and had no honest way to filter.
Trap two: the default hides the half you came for
One supervisor's search form defaults to "currently authorised". Submitted as it stands, it returns about a thousand firms.
Cleared of that default, it returns three times as many — and two thirds of those are cancelled, with a handful more in liquidation.
Taking the default would have collected only the entities whose authorisation is in force today, and silently discarded the single most interesting thing this source has to say: that a regulator withdrew one. A register of who is allowed is useful. A register of who was allowed, and until when, is a different and better instrument.
The general shape: a public search form is designed for a person asking about one company today. Its defaults encode that assumption. When you are reading the whole register, every default is a filter someone else chose for a purpose that is not yours.
Trap three: a file that reads as empty, and does not say so
A spreadsheet can store a piece of text in one of two ways — as a reference into a shared table of strings, or written inline in the cell itself. Both are correct, both are common, and one publisher here ships both.
Its largest register is inline throughout: over thirty thousand such cells, and not one of the other kind. Our reader knew only the shared table. It opened a 2.5MB worksheet, found nothing it recognised, and returned zero rows without raising anything.
For a while we believed that register was empty. That is the failure worth dwelling on: a file that parses to nothing looks exactly like a file that contains nothing, and no exception separates them. It is the same shape as the mistake where a firewall's rejection page arrives with a 200 and gets recorded as "no data for this month".
The fix is small — handle both encodings — but the habit is the thing. When a source comes back empty, the first question is not "why does this publisher have no data", it is "am I able to see data if it is there".
Four kinds of permission, under one law
The part we expected to be boilerplate turned out to be the most varied.
All five bodies are Spanish public-sector organisations, and Spanish law sets out general conditions under which their documents may be reused, commercially included. You would expect one answer. We got four.
One publishes those conditions in full on its own site, in its own words, explicitly permitting commercial reuse. That is the publisher speaking, and we record it as such.
One publishes nothing of the kind — not on the register, not in its ministry's legal notice. The same law binds it, so the citation is derivable, but derivable is not the same as stated. We record what it is: a reasonable inference, not the publisher's word. Those two facts should never be averaged into one optimistic flag.
One states nothing we could establish at all. An earlier note in our own files claimed a specific share-alike licence; we looked and found no evidence for it anywhere — the national open-data catalogue leaves the field blank, and the register's own legal page is twenty-nine characters long. Withdrawing an unfounded claim is not the same as proving the opposite, and the entry says so.
One attaches a live condition. It authorises reuse — including selling it — on the condition that anyone who does tells the buyer the same information is free at the source. Not once in a terms page: *every time the information is put at their disposal.*
That last one changed code, not just documentation. A clause in our terms covers "before they pay" and nothing else. "Every time" means the notice has to travel in the response itself, so it does: any record whose identifier came from that supervisor carries the sentence, in the payload and in the assistant interface both, and no record that took nothing from them carries it at all.
It cost an afternoon. It is also the clearest statement of what a data business actually sells: not access to public facts, which remain free at their source, but the collection, normalisation, resolution and delivery of them.
What we would tell anyone doing this
Let the source classify its own rows. Any inference you make from an identifier's shape — legal form, entity type, whether it is a company at all — is a guess dressed as arithmetic. If the publisher labels each row, use the label. If it does not, and the distinction matters, that is a reason to leave the data alone rather than a puzzle to solve cleverly.
Read the defaults as somebody's assumption. A form built for one lookup at a time will filter for currency. You usually want history.
Treat an empty result as a claim about your reader, not about the world, until you have proved the reader can see something that is definitely there.
Read the terms per publisher, not per country. Five bodies under one statute produced four different standings to assert it, and one of them imposed an obligation that had to be built rather than written down.