Building a live updating search page

For this implementation we will make use of a combination of Cotonic routing topics together with a small Zotonic widget to programmatically initialize the form from the search parameters in the URL.

A simple search page

We start with a simple page, where I will assume that the base template is working correctly. We will make our page on the URL /formtest for this we place the following template in priv/templates/static/formtest.tpl:

{% extends "base.tpl" %}

{% block content %}
    <div id="live"></div>

    {% live topic="model/location/event/qlist"
            template="static/_formtest.tpl"
            target="live"
            method="patch"
            is_new_query
    %}
{% endblock %}

In the Javascript include of the base template, make sure that the forminit widget, the live tag and the Cotonic loadmore model are added, we will need it later:

{% lib "js/modules/z.forminit.js"
       "js/modules/z.live.js"
       "js/models/loadmore.js"
%}

How does the live tag work? On page load the Cotonic location model parses the current URL. The query arguments are posted as a list to the topic model/location/event/qlist.

For example if our URL is:

 https://example.com/formtest?qs=hallo&qcat=text

then the following payload will be published to the qlist topic:

[
   [ "qtext", "hallo" ],
   [ "qcat", "text" ]
]

Because our live tag is subscribed to this topic, it will be triggered and render the template static/_formtest.tpl in the target div, with id #live.

The template is loaded using the patch method. Instead of just replacing the complete inner HTML of the div this will update the inner HTML to reflect the newly rendered template. This patching ensures that any input elements keep their focus and input values.

The is_new_query is passed as true to the rendered template, we will need it later when we discuss endless scrolling.

The search form

This the template we are loading:

<h1>Form test</h1>
<form class="do_forminit"
      data-onsubmit-topic="model/location/post/qlist/submit"
      data-oninput-topic="model/location/post/qlist/submit"
      method="GET">
    <input type="text" name="qtext" value="">
    <input type="checkbox" name="qis_published" value="1">
    <select name="qcat">
        <option value="">Any category</option>
        <option value="text">Text</option>
        <option value="media">Media</option>
    </select>
</form>

{% include "static/_formtest_page.tpl" %}

As you can see, this is a simple GET form. Let's explain the moving parts.

First the class do_forminit, as this class is prefixed with do_, it will call the widget forminit after the form element is created. The forminit widget will take the search arguments from the current document location to initialize the input elements of the form. In this way we do not need to check any checkboxes or add the correct values to text inputs. This makes building the form much simpler.

There are two Cotonic topics added to the form. One for the submit if the enter key is pressed in the text input (note that we don't have a submit button). The other one is for any changes of input values.

Both topics publish the contents of the form to the topic model/location/post/qlist/submit.

This topic of the location model updates the current search arguments of the document location with the posted values and then published the new query arguments to model/location/event/qlist, which will trigger our live tag to re-render and update the content of the #live div.

Searching

The search form template loads a page with the search results, place this in priv/templates/_formtest_page.tpl:

{% with m.search.query::%{ qargs: true, page: q.page } as result %}
    <h2>Page #{{ result.page }}</h2>

    <ul>
        {% for id in result %}
            <li><a href="{{ id.page_url }}">{{ id.title }}</a></li>
        {% endfor %}
    </ul>

    {% pager dispatch="none"
             result=result
             topic="model/location/post/push"
             qargs
    %}
{% endwith %}

This does the query, lists the results and shows a pager. For the query we pass the current page number, from q.page, and the query arguments by adding the option qargs. The system knows which search arguments are query arguments by taking all search arguments starting with a q. That is why in the form we used names like qtext and qcat.

For the list of found pages we simply loop over the returned result and render links to the pages. For this example this is kept simple.

Pagination

The pager uses qargs to render the correct URLs with all search arguments. The current page is fetched from the search result. A special dispatch rule none is used, which generates links without a path and hostname, for example:

?qtext=hello&qcat=text&page=2

When clicked, the browser will add the current hostname and path to these links.

The pager has a topic argument, having this topic prevents the browser to update the current location. Instead the URL is published to the topic. The location model (subscribed to the topic) then updates the current location using history.pushState(), ensuring that the URL reflects the correct page number and that the back button will indeed go back to the previous page.

After updationg the browser history, the new location is published to the location event topics. Which triggers our live tag to re-render the form with the page results.

A load more button

Instead of the pager we could also add a load more button to load the next page with results:

{% if result.next %}
        <div id="more-results-{{ result.next }}"
             data-onclick-topic="model/loadmore/post/replace"
             data-template="static/_formtest_page.tpl">
           <a class="btn btn-primary" href="{% url none page=result.next qargs %}">
              {_ Show more _}
           </a>
       </div>
{% endif %}

If there is a next page after the current result page then we show a button with Show more.  If the button is clicked, the href-attribute is published to the loadmore model. This will replace the div with the contents of the rendered templates.  All search arguments of the href URL are passed as query arguments to the template renderer.

This renders the next page, add it below the current results, and add a new load more button. Repeatingly clicking the button will load more and more pages until there is no next page.

The loadmore model updates the current document location with the new URL, reflecting the currently shown page. It uses the history.replaceState() function, so no pages are added to the browser history.

Note that if anything in the search form is changed the subsequent submit will replace the URL again, this time without a page number.

Endless scrolling

Instead of manually clicking a load more button we can also automatically load the next page if we scroll far enough.

For this we render an element, and if it becomes visible it will be replaced with the next page. Just like the load more button, but now without clicking:

{% if result.next %}
    <div id="more-results-{{ result.next }}"
         data-onvisible-topic="model/loadmore/post/replace"
         data-template="static/_formtest_page.tpl"
         data-url="{% url none page=result.next qargs %}">
        Loading...
    </div>
{% endif %}

The onvisible topic uses an intersection observer to observe changes to the element's visibility. It will trigger a publish to the topic whenever the element becomes visible.

We publish to the loadmore model topic to replace the div with the newly rendered page. We now use the onvisible topic and pass the new URL in the data-url argument (as there is no anchor tag with a href).

The current browser location is updated to reflect the new location. The loadmore model uses history.replaceState()to prevent adding pages to the browser history. Just like the load more button.

Reloading a page with load more or endless scrolling

If a page with a load more button or endless scrolling is reloaded, then it will only display the current page from the location's search arguments. This because our query searches for that page:

{% with m.search.query::%{ qargs: true, page: q.page } as result %}

But on a reload we might want to see all pages up to an including the given result page number, as if the user repeatingly clicked on the load more themselves until they reached the correct page number.

This is easily done by using the is_new_query argument we added to the live tag. It makes enables the pager template to see if the whole form and page was loaded because of fresh page load (or new query) or as part of pagination. With the pagination we do not pass this variable, so its presence signals that the template is rendered as part of a complete refresh.

Remember that if we perform a new search that the page search argument is not passed, it is only passed with pagination.

{% with q.page|default:1|to_integer as page %}
    {% if is_new_query %}
        {% for p in 1|range:(page-1) %}
            {% with m.search.query::%{ qargs: true, page: p } as result %}

                <h2>Page #{{ result.page }}</h2>

                <ul>
                    {% for id in result %}
                        <li><a href="{{ id.page_url }}">{{ id.title }}</a></li>
                    {% endfor %}
                </ul>
            {% endwith %}
        {% endfor %}
    {% endif %}

    {% with m.search.query::%{ qargs: true, page: page } as result %}
        <h2>Page #{{ result.page }}</h2>

        <ul>
            {% for id in result %}
                <li><a href="{{ id.page_url }}">{{ id.title }}</a></li>
            {% endfor %}
        </ul>

        {% if result.next %}
            <div id="more-results-{{ result.next }}"
                 data-onvisible-topic="model/loadmore/post/replace"
                 data-template="static/_formtest_page.tpl"
                 data-url="{% url none page=result.next qargs %}"
            >
                Loading...
            </div>
        {% endif %}

    {% endwith %}
{% endwith %}

If the is_new_query argument is set we loop through all pages from 1 to the passed page. Rebuilding the complete search result page as if there was a manual scroll to the given page number.

An alternative is to add a load before button, which we leave as an exercise to the reader.

Conclusion

We have seen that with simple templates and a combination of Cotonic models it is easy to make live search pages. On these pages you can use a variety of strategies for the pagination.

As the pages are patched by updating the DOM it is possible to make a dynamic form that adapts to the search results. Which is very useful when using faceted search queries.