Scraping Google Maps is easy; useful data is not

I wanted to know how many private schools exist in Mairiporã (São Paulo) with a phone number, website and email, and I wanted to know it without opening Maps and copying listing after listing. I asked Grok Bot, which delegated execution to agy, the command line agent from Google Antigravity, and the chosen tool was gosom/google-maps-scraper, a Go project widely used for this kind of survey. The command that worked was: docker run --rm --name gmaps-cli -e DISABLE_TELEMETRY=1 -v /workspace/gmaps-scraper:/work gosom/google-maps-scraper -input /work/query.txt -results /work/out.csv -lang pt -depth 2 -email -exit-on-inactivity 3m -c 4. In 131 seconds it returned 23 rows; 19 had a phone, 15 had a website and only 1 had an email. After cleaning the result by hand, 8 real private schools remained. The tool worked. What the number reveals is that extracting Maps is the cheap part of the problem, and that useful data lives somewhere else.

Before running anything, I read the code. Not out of paranoia, but because a scraper is by definition a program that opens a browser, visits arbitrary pages and writes to disk, and I prefer to know what else it does. The architecture is clean: the Go binary uses scrapemate, a job framework by the same author, on top of Playwright, so collection is a queue of chained jobs. A search job scrolls the results list, each result spawns a place job, and, if -email is on, each place with a website spawns an email extraction job. The same core is exposed through several runners: the file runner (CLI, one query per line in -input), the web runner (-web, with UI and REST API, persisted in SQLite inside -data-folder), the database runner (-dsn with PostgreSQL, which lets you split producers from consumers with -produce and scale horizontally) and even an AWS Lambda runner. To me this is the project’s best engineering decision: collection semantics are defined once, and the execution mode is only the edge that decides where queries come from and where results go.

I also found something that was not in the pitch. In runner/runner.go, the telemetry function sends anonymous run events to PostHog (eu.i.posthog.com) unless DISABLE_TELEMETRY=1 is set. It is not malicious code, and the README is honest about it, but it is opt-out, and opt-out in a tool you run on your machine against a third party service is the kind of default I prefer to turn off explicitly. That is why -e DISABLE_TELEMETRY=1 is in the command. The web server sets headers such as Content-Security-Policy and X-Frame-Options: DENY; even so, I bound the port only to 127.0.0.1:8080, because a UI without authentication has no business being exposed on the network.

The flags look cosmetic, but each one is a contract about cost. -depth is not link depth; it is how many times the scraper scrolls the search results list. The default is 10, and I used 2 because for a small town the list runs out early and each extra scroll is browser time for nothing. -c sets scrapemate concurrency, that is, how many jobs run in parallel. There is a curious detail here: the help text says the default is half the CPU cores, but the code is min(runtime.NumCPU()/2, 1), which in practice yields 1. Whoever trusts the help text runs serially without noticing, so I passed -c 4 explicitly. -exit-on-inactivity 3m solves a problem the CLI mode has by nature: a job queue does not know when it is done, since any job may spawn others. Without this flag (default 0, disabled) the process may wait forever; with it, three minutes without activity end the run. It is an empirical stop criterion, not a completeness guarantee, and should be treated as such.

Email is where expectation and code drift furthest apart. The flag is called -email and suggests something like “find the company’s email”, but what gmaps/emailjob.go does is much narrower: it fetches the website URL listed on the place, looks for mailto: links in the HTML and, failing that, runs a regex over the response body. The job is created with MaxRetries set to zero. There is no crawl to the contact page, no wait for JavaScript to render the footer, no second attempt when a school’s site on a cheap hosting plan is slow to respond. This explains 1 out of 23 better than any theory about Google hiding data: Maps almost never stores email, and the second collection layer depends on sites that are slow, client rendered, or that simply publish contact through a form. The failure is not a bug; it is the direct consequence of a cost decision. A naive, single attempt extraction is cheap and scales; a careful one would be a different program.

The REST mode made this boundary even more visible. I started the container with -data-folder mounted, submitted the same job via POST /api/v1/jobs with depth: 2, email: true and max_time: 300, and it stopped at 315 seconds with 18 rows and zero emails. The same 23 places were found; what was missing were the five still waiting in the email queue when time ran out. The difference between modes is therefore not extraction quality but stop criterion: the CLI stops on inactivity and writes everything that finished, the web job stops on the clock and drops what was in flight. If I needed operational predictability, I would pick the web mode; if I needed completeness, the CLI with generous inactivity.

Cleanup was the most expensive part, and it is exactly the one no flag solves. The query “escola particular em Mairiporã SP” returned sports schools, a computer school, a language school, colleges, an open university hub, private tutoring and even a local government office, all inside Mairiporã, because Maps search ranks by geographic and semantic relevance, not by legal category. Of 23 rows, 8 were actual private schools. The other fifteen rows were correct as data (address, phone and rating matched the listing) and wrong as an answer. This is the conceptual problem I think many people ignore when talking about scraping: the scraper guarantees fidelity to the listing, not relevance to the question. Relevance requires classification, and classification requires human judgment or an explicit second step.

There is also a boundary that is not technical, and the README itself acknowledges it by carrying a legal notice saying unauthorized use may violate terms of service. The data I collected belongs to businesses and is public on the listing, and the use was a one off survey; still, phone numbers and emails can identify individuals (the owner of a small school often answers on a personal phone), and then Brazil’s LGPD applies with purpose, necessity and legal basis. Such a list does not authorize mass outreach just because it was easy to build. My rule is simple to state and hard to follow: collect the minimum, keep it for the minimum, and never turn technical ease into a justification for use.

What I take from the experiment is an inversion of priority. I expected the difficulty to be getting around Maps, and gosom/google-maps-scraper solves that with a queue architecture cleanly separated from its execution modes, in a one command container. The real difficulty came after: in an email extraction that by design gives up at the first slow site, in a stop criterion that is a heuristic, and in a search that returns what is near, not what is true. Scraping is infrastructure. Knowing what you scraped is still work.