Ningle Tutorial 15: Pagination, Part 2
by NMunro
Contents
- Part 1 (Hello World)
- Part 2 (Basic Templates)
- Part 3 (Introduction to middleware and Static File management)
- Part 4 (Forms)
- Part 5 (Environmental Variables)
- Part 6 (Database Connections)
- Part 7 (Envy Configuation Switching)
- Part 8 (Mounting Middleware)
- Part 9 (Authentication System)
- Part 10 (Email)
- Part 11 (Posting Tweets & Advanced Database Queries)
- Part 12 (Clean Up & Bug Fix)
- Part 13 (Adding Comments)
- Part 14 (Pagination, Part 1)
- Part 15 (Pagination, Part 2)
Introduction
Welcome back! We will be revisiting the pagination from last time, however we are going to try and make this easier on ourselves, I built a package for pagination mito-pager, the idea is that much of what we looked at in the last lesson was very boiler plate and repetitive so we should look at removing this.
I will say, my mito-pager can do a little more than just what I show here, it has two modes, you can use paginate-dao (named this way so that it is familiar to mito) to paginate over simple models, however, if you need to perform complex queries there is a macro with-pager that you can use to paginate. It is this second form we will use in this tutorial.
There is one thing to bear in mind, when using mito-pager, you must implement your data retrieval functions in such a way to return a values object, as mito-pager relies on this to work.
I encourge you to try the library out in other use-cases and, of course, if you have ideas, please let me know.
Changes
Most of our changes are quite limited in scope, really it’s just our controllers and models that need most of the edits.
ningle-tutorial-project.asd
We need to add the mito-pager package to our project asd file.
- :ningle-auth)
+ :ningle-auth
+ :mito-pager)
src/controllers.lisp
Here is the real payoff! I almost dreaded writing the sheer volume of the change but then realised it’s so simple, we only need to change our index function, and it may be better to delete it all and write our new simplified version.
(defun index (params)
(let* ((user (gethash :user ningle:*session*))
(req-page (or (parse-integer (or (ingle:get-param "page" params) "1") :junk-allowed t) 1))
(req-limit (or (parse-integer (or (ingle:get-param "limit" params) "50") :junk-allowed t) 50)))
(flet ((get-posts (limit offset) (ningle-tutorial-project/models:posts user :offset offset :limit limit)))
(mito-pager:with-pager ((posts pager #'get-posts :page req-page :limit req-limit))
(djula:render-template* "main/index.html" nil :title "Home" :user user :posts posts :pager pager)))))
This is much nicer, and in my opinion, the controller should be this simple.
src/main.lisp
We need to ensure we include the templates from mito-pager, this is a simple one line change.
(defun start (&key (server :woo) (address "127.0.0.1") (port 8000))
(djula:add-template-directory (asdf:system-relative-pathname :ningle-tutorial-project "src/templates/"))
+ (djula:add-template-directory (asdf:system-relative-pathname :mito-pager "src/templates/"))
src/models.lisp
As mentioned at the top of this tutorial, we have to implement our data retrieval functions in a certain way. While there are some changes here, we ultimately end up with less code.
We can start by removing the count parameter, we wont be needing it in this implementation, and since we don’t need the count parameter anymore, the :around method can go too!
- (defgeneric posts (user &key offset limit count)
+ (defgeneric posts (user &key offset limit)
-
- (defmethod posts :around (user &key (offset 0) (limit 50) &allow-other-keys)
- (let ((count (mito:count-dao 'post))
- (offset (max 0 offset))
- (limit (max 1 limit)))
- (if (and (> count 0) (>= offset count))
- (let* ((page-count (max 1 (ceiling count limit)))
- (corrected-offset (* (1- page-count) limit)))
- (posts user :offset corrected-offset :limit limit))
- (call-next-method user :offset offset :limit limit :count count))))
There’s two methods to look at, the first is when the type of user is user:
-
- (defmethod posts ((user user) &key offset limit count)
+ (defmethod posts ((user user) &key offset limit)
...
(values
- (mito:retrieve-by-sql sql :binds params)
- count
- offset)))
+ (mito:retrieve-by-sql sql :binds params)
+ (mito:count-dao 'post))))
The second is when the type of user is null:
-
- (defmethod posts ((user null) &key offset limit count)
+ (defmethod posts ((user null) &key offset limit)
...
(values
- (mito:retrieve-by-sql sql)
- count
- offset)))
+ (mito:retrieve-by-sql sql)
+ (mito:count-dao 'post))))
As you can see, all we are really doing is relying on mito to do the lions share of the work, right down to the count.
src/templates/main/index.html
The change here is quite simple, all we need to do is to change the path to the partial, we need to simply point to the partial provided by mito-pager.
- {% include "partials/pager.html" with url="/" title="Posts" %}
+ {% include "mito-pager/partials/pager.html" with url="/" title="Posts" %}
src/templates/partials/pagination.html
This one is easy, we can delete it! mito-pager provides its own template, and while you can override it (if you so wish), in this tutorial we do not need it anymore.
Conclusion
I hope you will agree that this time, using a prebuilt package takes a lot of the pain out of pagination. I don’t like to dictate what developers should, or shouldn’t use, so that’s why last time you were given the same information I had, so if you wish to build your own library, you can, or if you want to focus on getting things done, you are more than welcome to use mine, and of course, if you find issues please do let me know!
Learning Outcomes
| Level | Learning Outcome |
|---|---|
| Understand | Understand how third-party pagination libraries like mito-pager abstract boilerplate pagination logic, and how with-pager expects a fetch function returning (values items count) to handle page clamping, offset calculation, and boundary correction automatically. |
| Apply | Apply flet to define a local adapter function that bridges the project’s posts generic function with mito-pager’s expected (lambda (limit offset) ...) interface, and use with-pager to reduce controller complexity to its essential logic. |
| Analyse | Analyse what responsibilities were transferred from the manual pagination implementation to mito-pager — count caching, boundary checking, offset calculation, page correction, and range generation — contrasting the complexity of both approaches. |
| Create | Refactor a manual pagination implementation to use mito-pager by simplifying model methods to return (values items count), replacing complex multi-step controller calculations with with-pager, and delegating the pagination template partial to the library. |
Github
- The link for the custom pagination part of the tutorials code is available here.
Common Lisp HyperSpec
| Symbol | Type | Why it appears in this lesson | CLHS |
|---|---|---|---|
defpackage |
Macro | Define project packages like ningle-tutorial-project/models, /forms, /controllers. |
http://www.lispworks.com/documentation/HyperSpec/Body/m_defpac.htm |
in-package |
Macro | Enter each package before defining models, controllers, and functions. | http://www.lispworks.com/documentation/HyperSpec/Body/m_in_pkg.htm |
defgeneric |
Macro | Define the simplified generic posts function signature with keyword parameters offset and limit (the count parameter is removed). |
http://www.lispworks.com/documentation/HyperSpec/Body/m_defgen.htm |
defmethod |
Macro | Implement the simplified posts methods for user and null types (the :around validation method is removed). |
http://www.lispworks.com/documentation/HyperSpec/Body/m_defmet.htm |
flet |
Special Operator | Define the local get-posts adapter function that wraps posts to match mito-pager’s expected (lambda (limit offset) ...) interface. |
http://www.lispworks.com/documentation/HyperSpec/Body/s_flet_.htm |
let* |
Special Operator | Sequentially bind user, req-page, and req-limit in the controller where each value is used in subsequent bindings. |
http://www.lispworks.com/documentation/HyperSpec/Body/s_let_l.htm |
or |
Macro | Provide fallback values when parsing page and limit parameters, defaulting to 1 and 50 respectively. |
http://www.lispworks.com/documentation/HyperSpec/Body/m_or.htm |
multiple-value-bind |
Macro | Capture the SQL string and bind parameters returned by sxql:yield in the model methods. |
http://www.lispworks.com/documentation/HyperSpec/Body/m_multip.htm |
values |
Function | Return two values from posts methods — the list of results and the total count — as required by mito-pager:with-pager. |
http://www.lispworks.com/documentation/HyperSpec/Body/a_values.htm |
parse-integer |
Function | Convert string query parameters ("1", "50") to integers, with :junk-allowed t for safe parsing. |
http://www.lispworks.com/documentation/HyperSpec/Body/f_parse_.htm |