Ningle Tutorial 12: Clean Up & Bug Fix

Contents

• Part 1 (Hello World)
• Part 2 (Basic Templates)
• Part 3 (Introduction to middleware and Static File management)
• Part 4 (Forms)
• Part 5 (Environmental Variables)
• Part 6 (Database Connections)
• Part 7 (Envy Configuation Switching)
• Part 8 (Mounting Middleware)
• Part 9 (Authentication System)
• Part 10 (Email)
• Part 11 (Posting Tweets & Advanced Database Queries)
• Part 12 (Clean Up & Bug Fix)

Introduction

Hello, and welcome back! We have done some pretty hefy work lately, so as we are drawing towards the end of the year we will be taking it a bit easier, we will be looking, at better organising and structuring our project. There is also a small bug we shall fix, which is in fact where we will start!

Fixing a bug

An oversight on my part last month was that a change stopped the username from appearing on posts. The solution is quite simple, little more than another join on our query.

In our logged-in-posts and not-logged-in-posts controllers, we need to make a small change, they’re basically the same two line change in both.

I will be testing out the ability to simulate the output of git diff here, so if you have feedback on this change, let me know!

logged-in-posts

(defmethod logged-in-posts ((user user))
  (let ((uid (slot-value user 'mito.dao.mixin::id)))
    (mito:retrieve-by-sql
        (sxql:yield
            (sxql:select
                (:post.*
+                 (:as :user.username :username)                        ;; Add this line
                  (:as (:count :likes.id) :like_count)
                  (:as (:count :user_likes.id) :liked_by_user))
                (sxql:from :post)
+               (sxql:left-join :user :on (:= :post.user_id :user.id))  ;; Add this line
                (sxql:left-join :likes :on (:= :post.id :likes.post_id))
                (sxql:left-join (:as :likes :user_likes)
                                :on (:and (:= :post.id :user_likes.post_id)
                                          (:= :user_likes.user_id :?)))
                (sxql:group-by :post.id)
                (sxql:order-by (:desc :post.created_at))
                (sxql:limit 50)))
            :binds (list uid))))

not-logged-in-posts

(defun not-logged-in-posts ()
    (mito:retrieve-by-sql
        (sxql:yield
        (sxql:select
            (:post.*
+             (:as :user.username :username)                        ;; Add this line
              (:as (:count :likes.id) :like_count))
            (sxql:from :post)
+           (sxql:left-join :user :on (:= :post.user_id :user.id))  ;; Add this line
            (sxql:left-join :likes :on (:= :post.id :likes.post_id))
            (sxql:group-by :post.id)
            (sxql:order-by (:desc :post.created_at))
            (sxql:limit 50)))))

This should now allow the usernames to come through. The reason for this is that although the “user” column would come back, it only contains a number, since it is a foreign key, so to get the rest of the actual information we must perform an sql join, so we can “join” information from different tables together.

As a result of this change though, we do need to change two template.

src/templates/main/index.html

- <p class="card-subtitle text-muted mb-0">@{{ post.user.username }}</p>
+ <p class="card-subtitle text-muted mb-0">@{{ post.username }}</p>

src/templates/main/post.html

- <h2>{{ post.user.username }}
+ <h2>{{ post.username }}

That should be everything we need, so onto cleaning up our project!

Cleaning up project

The clean up process is rather simple, but I find it helps. Our main.lisp file has gotten quite large and busy and it contains conceptually two things, our routing, and our controllers and while it’s certainly possible to have both in the same file, it can perhaps make the routing difficult to see, so we will be creating a new controllers.lisp file and putting our functions in there, and simply attaching the function name to the route.

src/controllers.lisp

We will be taking each of the functions from our main.lisp and declaring them as real functions here, of course remembering to export them from this package so that they can be accessed externally.

(defpackage ningle-tutorial-project/controllers
  (:use :cl :sxql :ningle-tutorial-project/forms)
  (:export #:logged-in-index
           #:index
           #:post-likes
           #:single-post
           #:post-content
           #:logged-in-profile
           #:unauthorized-profile
           #:people
           #:person))

(in-package ningle-tutorial-project/controllers)


(defun logged-in-index (params)
    (let* ((user (gethash :user ningle:*session*))
           (form (cl-forms:find-form 'post))
           (posts (ningle-tutorial-project/models:logged-in-posts user)))
        (djula:render-template* "main/index.html" nil :title "Home" :user user :posts posts :form form)))


(defun index (params)
    (let ((posts (ningle-tutorial-project/models:not-logged-in-posts)))
        (djula:render-template* "main/index.html" nil :title "Home" :user (gethash :user ningle:*session*) :posts posts)))


(defun post-likes (params)
    (let* ((user (gethash :user ningle:*session*))
           (post (mito:find-dao 'ningle-tutorial-project/models:post :id (parse-integer (ingle:get-param :id params))))
           (res (make-hash-table :test 'equal)))
        (setf (gethash :post res) (ingle:get-param :id params))
        (setf (gethash :likes res) (ningle-tutorial-project/models:likes post))
        (setf (gethash :liked res) (ningle-tutorial-project/models:toggle-like user post))
        (com.inuoe.jzon:stringify res)))


(defun single-post (params)
    (handler-case
        (let ((post (mito:find-dao 'ningle-tutorial-project/models:post :id (parse-integer (ingle:get-param :id params)))))
            (djula:render-template* "main/post.html" nil :title "Post" :post post))

        (parse-error (err)
            (setf (lack.response:response-status ningle:*response*) 404)
            (djula:render-template* "error.html" nil :title "Error" :error err))))


(defun post-content (params)
    (let ((user (gethash :user ningle:*session*))
          (form (cl-forms:find-form 'post)))
        (handler-case
            (progn
                (cl-forms:handle-request form) ; Can throw an error if CSRF fails

                (multiple-value-bind (valid errors)
                    (cl-forms:validate-form form)

                    (when errors
                        (format t "Errors: ~A~%" errors))

                    (when valid
                        (cl-forms:with-form-field-values (content) form
                            (mito:create-dao 'ningle-tutorial-project/models:post :content content :user user)
                            (ingle:redirect "/")))))

            (simple-error (err)
                (setf (lack.response:response-status ningle:*response*) 403)
                (djula:render-template* "error.html" nil :title "Error" :error err)))))


(defun logged-in-profile (params)
    (let ((user (gethash :user ningle:*session*)))
        (djula:render-template* "main/profile.html" nil :title "Profile" :user user)))


(defun unauthorized-profile (params)
    (setf (lack.response:response-status ningle:*response*) 403)
    (djula:render-template* "error.html" nil :title "Error" :error "Unauthorized"))


(defun people (params)
    (let ((users (mito:retrieve-dao 'ningle-auth/models:user)))
        (djula:render-template* "main/people.html" nil :title "People" :users users :user (cu-sith:logged-in-p))))


(defun person (params)
    (let* ((username-or-email (ingle:get-param :person params))
           (person (first (mito:select-dao
                            'ningle-auth/models:user
                            (where (:or (:= :username username-or-email)
                                        (:= :email username-or-email)))))))
        (djula:render-template* "main/person.html" nil :title "Person" :person person :user (cu-sith:logged-in-p))))

With the exception of the defpackage and in-package, the only thing that changes here is that we are giving these functions a name, the params is unchanged from when there were in main.lisp.

src/main.lisp

This allows main.lisp to be flattened down.

(defpackage ningle-tutorial-project
  (:use :cl :ningle-tutorial-project/controllers)
  (:export #:start
           #:stop))

(in-package ningle-tutorial-project)

(defvar *app* (make-instance 'ningle:app))

;; requirements
(setf (ningle:requirement *app* :logged-in-p)
      (lambda (value)
        (and (cu-sith:logged-in-p) value)))

;; routes
(setf (ningle:route *app* "/" :logged-in-p t) #'logged-in-index)
(setf (ningle:route *app* "/") #'index)
(setf (ningle:route *app* "/post/:id/likes" :method :POST :logged-in-p t) #'post-likes)
(setf (ningle:route *app* "/post/:id") #'single-post)
(setf (ningle:route *app* "/post" :method :POST :logged-in-p t) #'post-content)
(setf (ningle:route *app* "/profile" :logged-in-p t) #'logged-in-profile)
(setf (ningle:route *app* "/profile") #'unauthorized-profile)
(setf (ningle:route *app* "/people") #'people)
(setf (ningle:route *app* "/people/:person") #'person)

(defmethod ningle:not-found ((app ningle:<app>))
    (declare (ignore app))
    (setf (lack.response:response-status ningle:*response*) 404)
    (djula:render-template* "error.html" nil :title "Error" :error "Not Found"))

(defun start (&key (server :woo) (address "127.0.0.1") (port 8000))
    (djula:add-template-directory (asdf:system-relative-pathname :ningle-tutorial-project "src/templates/"))
    (djula:set-static-url "/public/")
    (clack:clackup
     (lack.builder:builder (envy-ningle:build-middleware :ningle-tutorial-project/config *app*))
     :server server
     :address address
     :port port))

(defun stop (instance)
    (clack:stop instance))

I hope you agree that seeing main.lisp like this helps us focus principally on the routing without worrying about the exact implementation.

ningle-tutorial-project.asd

As always, since we have added a new file to our project we must ensure it gets included and compiled into our project.asd file.

:components ((:module "src"
              :components
              ((:file "contrib")
               (:file "middleware")
               (:file "config")
               (:file "models")
               (:file "forms")
               (:file "migrations")
+              (:file "controllers")
               (:file "main"))))

Conclusion

I appreciate that this is a very short lesson this time, but after the last few lessons (and next times lesson) I think we might both appreciate a small break. It is also important to look at refactoring projects and structuring them correctly before they get too unwieldily. There isn’t a lot of information out there about style guides or best practice so it was best to introduce some in our own project while we had a chance.

Next time we will be looking at adding comments to our system, I had thought perhaps the application was good enough as an example, but there’s still some areas we might want to look at, such as self referential models, which is where comments come in, cos a comment is technically a post after all!

As always, I hope you found this helpful, and thanks for reading.

Learning Outcomes

Level	Learning Outcome
Understand	Explain how separating routing and controller logic improves readability and maintainability. Describe how defpackage and symbol exports control what functions are visible across modules. Summarize why refactoring helps prevent future complexity in growing projects.
Apply	Move controller functions from main.lisp into a new package file, update main.lisp to call them via route bindings, and modify the .asd file to include the new component. Implement a small bug fix involving SQL joins and template references.
Analyse	Compare a monolithic main.lisp file with a modular project layout in terms of structure and debugging clarity. Identify how exported symbols, package imports, and route bindings interact across files. Evaluate the trade-offs of consolidating or splitting functions by purpose.
Evaluate	Assess the maintainability and clarity of the refactored code. Recommend naming or packaging conventions that could further streamline the project.

Github

• The link for this tutorials code is available here.

Resources

Common Lisp HyperSpec

Symbol	Type	Why it appears in this lesson	CLHS
defpackage	Macro	Define ningle-tutorial-project/controllers and ningle-tutorial-project packages with :export.	http://www.lispworks.com/documentation/HyperSpec/Body/m_defpac.htm
in-package	Macro	Enter the package before definitions.	http://www.lispworks.com/documentation/HyperSpec/Body/m_in_pkg.htm
defvar	Special Operator	Define *app* as a global.	http://www.lispworks.com/documentation/HyperSpec/Body/s_defvar.htm
defun	Macro	Define controller functions like index, post-content, etc.	http://www.lispworks.com/documentation/HyperSpec/Body/m_defun.htm
defmethod	Macro	Specialize ningle:not-found and logged-in-posts.	http://www.lispworks.com/documentation/HyperSpec/Body/m_defmet.htm
make-instance	Generic Function	Create the Ningle app object: (make-instance 'ningle:app).	http://www.lispworks.com/documentation/HyperSpec/Body/f_mk_ins.htm
let / let*	Special Operator	Local bindings for user, form, posts, etc.	http://www.lispworks.com/documentation/HyperSpec/Body/s_let_l.htm
lambda	Special Operator	Inline route requirement: (lambda (value) …).	http://www.lispworks.com/documentation/HyperSpec/Body/s_fn_lam.htm
setf	Macro	Assign route table entries and response status; generalized places.	http://www.lispworks.com/documentation/HyperSpec/Body/m_setf.htm
gethash	Function	Pull :user from ningle:*session*.	http://www.lispworks.com/documentation/HyperSpec/Body/f_gethas.htm
make-hash-table	Function	Build JSON-ish response map in post-likes.	http://www.lispworks.com/documentation/HyperSpec/Body/f_mk_has.htm
equal	Function	Hash table :test 'equal.	http://www.lispworks.com/documentation/HyperSpec/Body/f_equal.htm
list	Function	Build :binds list for SQL and other lists.	http://www.lispworks.com/documentation/HyperSpec/Body/f_list.htm
first	Accessor	Take first result from select-dao.	http://www.lispworks.com/documentation/HyperSpec/Body/f_firstc.htm
slot-value	Function	Access user id ((slot-value user '…:id) in the bug-fix snippet).	http://www.lispworks.com/documentation/HyperSpec/Body/f_slot__.htm
parse-integer	Function	Convert :id param to integer.	http://www.lispworks.com/documentation/HyperSpec/Body/f_parse_.htm
format	Function	Debug-print validation errors.	http://www.lispworks.com/documentation/HyperSpec/Body/f_format.htm
handler-case	Macro	Trap parse-error/simple-error for 404/403 pages.	http://www.lispworks.com/documentation/HyperSpec/Body/m_hand_1.htm
parse-error	Condition Type	Caught when parsing route params fails.	http://www.lispworks.com/documentation/HyperSpec/Body/e_parse_.htm
simple-error	Condition Type	Used for CSRF or general failures.	http://www.lispworks.com/documentation/HyperSpec/Body/e_smp_er.htm
multiple-value-bind	Macro	Unpack (valid errors) from validate-form.	http://www.lispworks.com/documentation/HyperSpec/Body/m_mpv_bn.htm
progn	Special Operator	Group side effects before error handling.	http://www.lispworks.com/documentation/HyperSpec/Body/s_progn.htm
when	Macro	Conditional steps after validation (when errors / when valid).	http://www.lispworks.com/documentation/HyperSpec/Body/m_when_.htm
declare	Special Operator	(declare (ignore app)) inside not-found.	http://www.lispworks.com/documentation/HyperSpec/Body/s_declar.htm
and / or	Macro	Logical composition in route requirements and user lookup.	http://www.lispworks.com/documentation/HyperSpec/Body/a_and.htm