Split your feeds into multiple pages

If you’re building your own feed via a landing page, it can get really big. Really big. Some feed subscribers therefore prefer to download it in smaller portions rather than as one huge file, and this is called pagination: you split the feed into multiple pages, which are downloaded one at a time.

This guide explains how pagination works in custom feeds, how to set it up in your feed template, and what you need to keep in mind when sharing the URLs with services like DataFeedWatch, Partner Ads, or Google Merchant Center.

How a custom feed works

A custom feed consists of two parts:

• A landing page that determines which products are included. You’ll find it under Content, Landing Pages.
• An XML template in your theme that determines how the feed looks. Enter the file name in the XML File Name field under the Custom XML heading when you edit the landing page.

The feed is then located at the landing page’s URL with .xml appended, e.g., https://dinshop.dk/mit-feed.xml.

Since your template generates the content, it must also be capable of paginating the feed. There is no built-in pagination that activates automatically.

How to paginate your feed

All parameters in the URL are available in the template via the $get variable. For example, if you retrieve ?page=2, the template can read the value as $get.page. You choose the name yourself: “page,” “side,” “offset,” or something else entirely—any of these work just as well.

Below is a minimal example with 500 products per page, where the page count starts at 0:

<{$per_page = 500}>
<{$page = $get.page|default:0}>
<{$offset = $page * $per_page}>

<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:g="http://base.google.com/ns/1.0">
<channel>
<{foreach $landing_page->getProducts($per_page, $offset) as $product}>
  <item>
    <g:id><{$product->getProductId()}></g:id>
    <title><{$product->getName()|escape}></title>
  </item>
<{/foreach}>
</channel>
</rss>

The two numbers you pass to ` getProducts()` are the number of products and the number to skip. Page 0 skips 0, page 1 skips 500, page 2 skips 1,000, and so on. When a page no longer returns any products, you’ve reached the end.

How many products should there be per page?

You determine that yourself in the template. The “Number of products on the landing page” field controls how the landing page is displayed as a regular HTML page, but it is not automatically used in your own XML feed.

A reasonable starting point is 300 to 1,000 products per page. If you have many variants, lengthy descriptions, or many images per product, choose the lower number. Keep in mind that a page in the feed may well contain more lines than products if your template outputs one line per variant.

Retrieve all pages in a script

If you need to retrieve the feed from another server yourself, you can loop until a page no longer contains any products:

$page = 0;
while (true) {
    $url = "https://dinshop.dk/mit-feed.xml?page=".$page;
    $xml = file_get_contents($url);

    if (empty(trim($xml)) || stristr($xml, "<item>") === false) {
        break;
    }

    // save or process the content here
    $page++;
}

Use the tag that identifies a product in your specific feed. In the example, it’s <item>, but it could also be <product> or something else entirely.

Be aware of the offset between page 0 and page 1

The most common mistake in practice is that the feed and the recipient don’t agree on where the page count starts. If your template starts at 0, while the feed recipient fetches pages 1, 2, 3, and so on, the first products in the feed will be completely missing without anyone receiving an error message.

Therefore, be aware of two things:

• The URL without a parameter corresponds to the first page, because the template typically resets to 0. Some feed recipients retrieve this as the first page and then start at 1.
• If you’d rather count from 1, subtract 1 from the template: <{$offset = ($page - 1) * $per_page}>. This ensures compatibility with readers that always start on page 1.

Always check after setup to make sure the first product on the first page is actually the first product on the landing page.

Speed: The feed is automatically generated in the background

If you retrieve the feed URL without parameters, Shoporama serves a pre-built file. It is built automatically in the background when your products are updated, and it is also cleared when you edit the landing page itself. Therefore, the parameter-free retrieval is both fast and gentle on your server, regardless of the feed’s size.

If you include parameters, the response cannot come from that file, because you’re requesting something other than the entire feed. Such retrievals are therefore generated on the spot. This is perfectly fine when each page contains only a portion of the catalog, but it’s also why it doesn’t make sense to split the feed if the recipient can easily handle it in one piece.

Feeds That Cannot Be Paginated

Pagination applies only to feeds that you build yourself using an XML template. Two other feed types in Shoporama cannot be split, nor is there a need to do so:

• The Google Shopping feed on a landing page (the address ending in .rss) always contains all products on the landing page and is automatically generated in the background.
• Product feeds under Products are also generated in the background and are always delivered as a single file. Read more in the article “Build Your Own Feeds.”

Pagination on landing pages and the blog

Regular landing pages and your online store’s blog, on the other hand, have a fixed parameter that works without you having to do anything: ?p=, where the count starts at 0.

https://dinshop.dk/min-landingsside        (same as ?p=0)
https://dinshop.dk/min-landingsside?p=1    (second page)
https://dinshop.dk/blog?p=1                (second page in the blog)

On landing pages, the number of items per page is controlled by the “Number of products” field, up to a maximum of 500 at a time. The blog displays 10 posts per page. Only these HTML pages use ?p=. In your own XML feed, this parameter has no effect unless your template reads it itself.

Frequently Asked Questions

Which parameter should I use for my feed?

The one your feed template reads. If you had the theme created by a developer, ask them or check the XML file mentioned in the “XML File Name” field on the landing page. The most common ones are ?page= and the combination of ?limit= and ?offset=.

I’m getting the same content on every page. What’s wrong?

That means your template isn’t reading the parameter. Load two pages and compare the first products on each: if they’re the same, the template is ignoring the pagination and displaying the entire catalog every time. This needs to be fixed in the template itself, as shown in the example above. Alternatively, you can simply retrieve the feed without parameters as a single file.

Does the page count start at 0 or 1?

That’s determined by your template. The default example above starts at 0, where the parameter-less URL is the first page. Make sure the feed receiver starts at the same point; otherwise, the first batch of products will be skipped.

How do I know I’ve retrieved all pages?

When a page no longer contains products, there are no more. Stop the loop there. You can also have the template output the total number using $landing_page->getProductCount(), so the recipient can calculate the number of pages in advance.

Will I get duplicates if a product changes pages along the way?

This can happen if products are added or removed while you’re fetching the pages. The likelihood is low, but if you want to be absolutely sure, fetch all pages in quick succession, or remove duplicates by product ID after you’ve collected them. Note that some feed tools do not remove duplicates on their own, and that a repeated product ID can cause Meta, for example, to reject the entire feed.

Can I send a single, consolidated URL to Google Merchant Center?

Yes. Submit the feed URL without any parameters, and you’ll receive the entire feed as a single file. Splitting the feed into pages is only necessary if the recipient explicitly requests it.

Do I even need to split my feed today?

Rarely. Because the parameter-free retrieval is served as a pre-built file, even very large feeds are quick to retrieve. Only split the feed if your recipient requires it, or if you’re importing the feed into a system that can’t handle a single large file.

Can I use the same method for CSV or JSON?

The field on the landing page is called “XML filename,” but the template can output any format you want, including CSV or JSON. If you need a true CSV or JSON feed without writing a template yourself, Product Feeds is usually the easier option.

What should I do if my ad partner can’t handle multiple URLs?

In that case, give them the parameter-free URL that contains the entire feed. If you need to consolidate the pages on your end, remember to remove the XML header from page two onward so that the file remains valid XML.

Need help? Contact us at support@shoporama.dk.