Generating Pages at Scale Without Producing Junk: Method and Limits

Scaled page generation has a reputation problem, largely deserved. The technique that produces genuinely useful location pages for a marketplace is the same technique that produces ten thousand pages saying "the best plumber in [town]" with nothing behind the bracket. The mechanics are identical; what differs is whether there is anything real inside.

This article covers doing it properly with the Semalt platform: deciding whether your dataset can support it, testing at small scale first, and auditing a set of generated pages that already exists — which is the situation most people are actually in.

The condition that decides everything

There is one test, and it is not about volume or technique. Does each page contain something specific that could not be written by swapping a word in a template?

If your dataset cannot supply the specificity, the honest answer is that this technique is not available to you, and twenty well-written pages will outperform two thousand generated ones. That is a genuinely common outcome and it is better established before the build than after.

Working parameters from the process described below.

Test with fifty before generating five thousand

1. Pick fifty representative cases

   Not the fifty best. A mix: high demand, medium, marginal — because the marginal ones tell you where the useful boundary sits, and that boundary is the whole point of the test.

2. Build them properly, including the awkward parts

   If the plan is to include unique data per page, include it now. Testing a stripped-down version tells you nothing about the real thing.

3. Wait eight weeks and read the distribution

   Not the average. How many of the fifty earned impressions at all? If ten did well and forty received nothing, generating five thousand produces four thousand pages of dead weight.

4. Find the threshold, then generate only above it

   The test usually reveals a clear cut-off — a level of demand or data richness below which pages do not perform. Generate to that line and stop.

What "unique" has to mean per page

Every generated page needs at least one element that exists only on that page and that a reader would actually want. In practice there are four sources.

The fourth item on the left is the most underused. On a marketplace or directory, the pages that succeed at scale usually do so because users contributed to them — a review, a question, a photograph. That content is unique by construction and it accumulates without editorial cost, which is exactly what scaled pages need.

Writing a template that does not read like one

Assuming the data qualifies, the remaining craft is in the template itself. Two sets built from identical data can perform very differently depending on how the page is composed, and the difference is not decoration.

Put the specific content first. Most generated templates open with a paragraph of shared boilerplate and reach the actual data halfway down. Invert it. The visitor came for the listings, the prices or the numbers, and so did the search engine's assessment of what this page is about. Shared explanatory text belongs below the specific content, not above it.

See also: Your Own Results Page.

Vary the structure, not just the words. A template that renders the same six sections in the same order regardless of what data exists produces obviously empty sections when data is missing. Conditional structure — show the reviews block only when there are reviews, show the comparison only when there is something to compare — makes each page reflect its actual substance, which is both more useful and more honest.

Write the titles and descriptions as if by hand. Templated titles that concatenate variables read as machine output in the results page, which costs clicks even when the ranking is good. A template with three or four title patterns, selected by the shape of the data, produces something far more natural at no additional cost per page.

We cover this in detail in Is It Worth Competing? Reading the Results Page Before Investing in a Topic.

Give the page one human sentence. On a location or category page, a single line of genuine knowledge — what is actually true about that place or product group — changes how the whole page reads. At a hundred pages this is an afternoon of work. At ten thousand it is not feasible, which is itself a useful signal about whether ten thousand pages was the right target.

Structure and internal linking

A generated set that nobody links to is a set nobody crawls. Three structural decisions determine whether the pages are reachable in practice.

Hub pages. Each cluster of generated pages needs a real parent that lists them, is itself worth reading, and is linked from the main navigation. A set of five hundred orphans reachable only through a sitemap will be crawled slowly and treated as peripheral, because structurally it is.

Lateral links. Related generated pages should link to each other — neighbouring areas, similar products, adjacent routes. This is both genuinely useful for the visitor and what makes the set navigable rather than a flat list.

A crawl budget check. On a large site, adding thousands of URLs redirects crawl attention away from the pages that already earn revenue. Watch what happens to the commercial pages after launch; if their crawl frequency drops noticeably, the generated set is too large for the site's current authority.

Auditing a set that already exists

Most people meet this topic in the form of an inherited problem: several thousand pages generated two years ago by someone who has left, and a suspicion that they are doing more harm than good.

The audit is straightforward and takes an afternoon.

1. Split the set by impressions

   Pages with meaningful impressions, pages with a handful, pages with none. In a badly built set this is usually a very skewed distribution, with a small minority carrying everything.

2. Read five pages from each band

   The difference between the bands is almost always the presence of real content. Confirm that hypothesis before acting on it, because occasionally the cause is technical instead.

3. Improve the top band, remove the bottom

   Pages with demand and thin content are worth investing in. Pages with no demand and no content should be removed or consolidated into a parent — redirecting where a sensible parent exists.

4. Change the generation rule

   Otherwise the set regrows. Whatever threshold the audit revealed should become the condition for creating a page at all.

Scale in a smaller language market

Two adjustments matter when generating pages in Portuguese, and both push in the same direction: smaller sets, more substance per page.

The demand distribution is thinner. A combination that has adequate volume in English frequently has none here, which means the threshold from your test batch will sit higher and the resulting set will be much smaller than the equivalent English project. That is the correct outcome, not a disappointment — a hundred pages that each serve a real query beat three thousand that do not.

On the other hand, competition is thinner too. In many Portuguese-language verticals, nobody has built the structured pages at all, so a modest, genuinely useful set faces very little resistance. The opportunity here is rarely to out-scale someone; it is to be the first to do it properly at any scale.

The honest position

Scaled generation is a legitimate technique with a narrow condition: you need a dataset that makes each page genuinely different, and demand that justifies each page existing. Where both hold, it is one of the most efficient things in search. Where either is missing, it produces a liability that someone will have to clean up later.

The discipline is entirely in the testing and the threshold. Fifty pages, eight weeks, read the distribution, generate only above the line, and write down the rule for removing what fails.

If you have inherited a large generated set and do not know whether it is helping, the split by impressions is where to start. Open the dashboard, crawl the set, and see what share of it has ever been seen by anyone.