jecs:~/posts

Thoughts on Common Lisp

Wed, 19 Jun 2024

I'm on a "programming language world tour" to try to find an elegant, expressive, language to replace my usage of C#. Why? For fun. 3 weeks into my Common Lisp (CL) excursion, here is what I think its greatest strength and weakness is.

Unmatched Interactivity

The experience of programming with the REPL in CL is truly like no other. I haven't found another language with a development cycle quite like this.

While writing Python I often find myself writing several hundred lines of code then testing it all at the same time. Encounter a few errors off the bat, and have to find smaller logical errors as I test the code. Even worse when developing in systems with slow compilation steps, I find myself actively trying to avoid testing my code because it would take too long. Surely if I'm just careful and write it correctly the first time it wouldn't be a problem…

With aid from the CL REPL this is never a problem. I find that I'm always able to build up to my solution.

Let's say I'm writing a function to compute the sample variance of a list of numbers. While this is already solved with libraries, let's try ourselves.

The formula for variance is $S^{2}=\frac{\sum_{i=1}^{n}\left(x_{i}-\overline{x}\right)^{2}}{n - 1}$

Here I'll try to capture the process, building up to the final solution:

;; Task: compute sample variance of a list

;; Ok, I'll let this guy be my test list
(defparameter *my-list* '(9.98 9.66 10.41 11.1 9.63))

;; Ok, I need a mean function.
(/ (reduce #'+ *my-list*)
   (length *my-list*)) ;; => 10.156

;; *I realize that it iterates over the list twice*

;; rewrite so it only iterates once.
;; potentially making a few mistakes along the way
(labels ((aux (lst accum len)
             (if (null lst)
                 (/ accum len)
                 (aux (cdr lst) (+ accum (car lst)) (1+ len)))))
    (aux *my-list* 0 0)) ;; 10.156, seems right

;; wrap this into a function
(defun mean (lst)
  (labels ((aux (lst accum len)
             (if (null lst)
                 (/ accum len)
                 (aux (cdr lst) (+ accum (car lst)) (1+ len)))))
    (aux lst 0 0)))

;; write sample variance function
(defun sample-variance (lst)
  (let ((mean (mean lst)))
    (labels ((aux (lst accum len)
              (if (null lst)
                  (/ accum len)
                  (aux (cdr lst) (+ accum (expt (- (car lst) mean) 2)) (1+ len)))))
     (aux lst 0 -1))))

;; use this function to perform some other task

While writing this function I used the REPL dozens of times, to check my work, as I do it. Had my use case/example involved mutable state, I would have been checking my work in the live environment that my application would actually end up running in. Interactivity like this would be invaluable in a large system involving mutable state, you can debug your applications with access to the entire lisp image. It's hard to convey just how powerful that is, you have to try it!

Lack of standardized documentation

My biggest gripe with CL is its documentation. I've used SLIME and SLY, arguably the defacto standard IDE's for CL, and I have problems with language documentation. I can't imagine what it's like if you didn't use Emacs.

Common Lisp is not a single language, but rather a specification from 1996. The specification has many implementations, the most popular of which is SBCL.

Some implementations may do certain optimizations or little things that others might not, for example SBCL will do a thorough type check given type annotations whereas another implementation might not. This, among other reasons, has created a fragmentation in the CL ecosystem.

It makes sense to me that there isn't a good LSP or culture of standardized documentation given the fragmentation. It would be subject to the scrutiny of developers when an issue could simply be an idiosyncrasy of the CL implementation.

Here is the extent to which I've searched for documentation:

Documentation directly in my editor (great!)
Find documentation online (most frequently HyperSpec or the project's Github)
Go to definition in editor (occurs more frequently than I'd like)

From the perspective of an API user, I found CL's powerful macro system to be more of a hindrance. More often than I'd like, I find myself learning a brand new domain specific language to perform some action, when it could have been a simple function call. Adding fuel to the fire, a DSL is way more difficult to document than a simple function call.

I understand the argument "just learn the language", but my issue isn't with the language core - rather, I have problems with libraries for which this argument doesn't apply.

Conclusion

While I enjoy the highly interactive workflows of Common Lisp, I find the lack of standardized documentation in the ecosystem makes CL difficult to work with.

I'm sure theres another language with interactivity that comes relatively close to CL. I plan to continue my "programming language world tour" with OCaml.

Path Tracer

Wed, 20 Sep 2023

When I was learning Rust, I made a path tracer as a fun project. This is a brute force path tracer, not meant to produce images in real time, but rather realistic images.

Description

In university I took a computer graphics course and I really enjoyed it! I wanted to take it to the next level and re-wrote the project in Rust and added some features from the classic Ray Tracing in One Weekend Series.

Github Repository for the path tracer

Here are some images that I rendered:

Pepe's Bubbles

Sun, 01 May 2022

Pepe's Bubbles, is a group project during I worked on during schooling. I was the sole backend developer of the project, and I learned much about backend design by working on it.

Pepes Bubbles is a online storefront for a fictional Bubble Tea store called "Pepe's Bubbles" that we made up. The name/theme of the storefront was arbitrary, as the project was mainly to exercise and demonstrate our web-development skills.

During the development of this project I worked as a the sole back-end developer for our application. I implemented the REST API and GraphQL API with Node.js, and also worked on deployment of our web-app using Dockerized containers running various technologies such as Nginx, MongoDB, Memcached, and Graylog.

The REST API was mainly for easy third party integrations with things like Google SSO, Paypal, Google Maps, and the GraphQL API was used for the project's business logic.

See an extensive description of this project in the project repository.

Github Repository for Pepe's Bubbles

Learnera

Sat, 04 Sep 2021

Project Learnera, originally named Project BFS, is a group project during I worked on during schooling.

Project Learnera is a collaboration project with The Bridge from UTSC. The goal of the project is to tackle the African Impact Challenge by creating a web platform that combines a community and an E-Learning platform. The goal of the African Impact Challenge is to build the Africa we want to see, by investing in the continent's aspiring entrepreneurs-to-be. The goal is to enable them to build market-creating innovations, which tackle their country's biggest challenges with technology.

We worked on the project from May 2021 to August 2021. The name of our development team was Brute Force Solutions, but we later changed the product name to Learnera.

Features requested at start of development by the product owner:

Individual profile creations which use the data from the applications/registrations in an automated way, to guide each person’s user journey and recommended actions.
Company profile creations with consent-provided access to uploaded information by the founding teams on pitch decks, financials, MCs, founding team, etc.
Partner profile creations (investors, service providers, corporations) with relevant information about them.
Enabling organic and guided interaction within this community.
Virtual pre-recorded content (video/audio and visual) that can be consumed in a self-paced manner.
Deliverable completion and submission.
Calendar and scheduling features for participants in the incubation program.
Video conferencing integration for live workshops/sessions.

Github Repository for Learnera

Caseware

Sat, 09 Jan 2021

A journal and some artifacts from my time working at Caseware.

What is this?

This page is a short journaling and contains some artifacts from my time working at Caseware.

I worked at Caseware, a software solutions company for accounting, auditing, financial, risk and governance professionals from January 2020 to December 2020.

I worked as a application developer, working on CaseWare's Audit Template product for their Working Papers application for about 3 months, then I was transferred to CaseWare's cloud platform team to work on their new Cloud Platform 2.0 product.

While on CaseWare's cloud platform team I worked as a full stack developer, working on new stories, and features. We used Java with the Spring Boot framework as our back-end, and Angular as our front-end framework.

Below are some artifacts from my time working at Caseware,

Artifact 1

I am nearing the end of my 12 month co-op placement at CaseWare International, and I've been working hard on delivering the minimum viable product (MVP) for the new cloud product that CaseWare is investing in. As a fullstack developer there are a lot of technologies that I should have some level of understanding and competence in, and having that breadth I found was difficult. Advice that I acted on to try to overcome this difficulty is to try to become the subject matter expert on one area of our architecture. I decided to dive deep into the Angular framework and our front end technologies, and I was able to deliver several features into the product.

Aside from the front end architecture that I have decided to specialized in, I have also been assigned architecture design tasks. I have presented such designs in meetings with the members of the wider CaseWare team. An example architecture that I have designed is the architecture shown in the first picture. It is a high level design and leaves out some implementation details.

Now that I am mainly focusing on working on architecture design tasks, and so working on removing tech debt from our product. I am thinking back to the past 4 months of development to see if there's any instances I wrote code that could be considered tech debt. If I were to participate in the past 4 months of development again, I would try to be more prudent to ensure that the code I am writing would be easy to understand and not have to be changed to remove future tech debt.

Last updated 12/22/2020

Artifact 2

I have been on my co-op placement at CaseWare International for 8 months now and a lot has changed in the industry due to Covid-19. Within the company I changed teams as a result of this change and my responsibilities have changed to that of a fullstack developer. The future seems to be cloud computing for CaseWare and I am now working as a part of the Cloud Platform team. CaseWare currently has a working cloud platform accounting solution but it has scalability issues with larger clients, due to this we are developing a new solution and I am a part development for the minimum viable product. Since the product is not yet released there are rules of confidentiality in place and I cannot share much about the product at this time. To solve the scalability issues, we are using a new software architecture described in the first picture. I am working on developing microservices to keep data and perform actions on the abstractions: users and entities. As a fullstack developer I also have to work with the client side, and I am working on UIs like that of the second picture. I am working developing UIs to interact with the server and microservices I am involved with. As a result of my involvements in development in the past several sprints, the team has picked up velocity and we are projected to finish the MVP ahead of schedule. Since my role has changed from that of a software developer to a fullstack developer, the technical stack that I work with has increased in difficulty and required level of understanding drastically. My skills of research and accelerated learning has increased as well. If I could participate in development made in these past 4 months again, I would try to learn one technology to level of competence that I am confident in, instead of trying to learn several technologies at the same time, but at a slower pace. As an onboarding fullstack developer it is common to be overwhelmed trying to learn all the different technologies, and with that change I would have been more confident in myself and that might have resulted in faster learning.

Last updated 8/31/2020

Artifact 3

After joining CaseWare International in the final phases of development for a new version of the Audit product, I fixed numerous bugs across several dialog boxes. The team had recently made a huge overhaul to the dialogs and as a result several bugs had been discovered. The screenshots below showcase a few of the dialogs I worked on. I made several UI changes to these dialogs and fixed several logical errors in the code. In my efforts to fix bugs in these dialogs I took advantage of my research skills and was able to identify key information of internal or external documentation which allowed me to complete these assignments in my 2 week time constraints. The end result of my efforts was a product that worked properly after a huge code refactoring and overhaul. After fixing these several issues in time for the set release date of the new Audit product I developed my research and time management skills which will enable me to work more effectively in the future. I also learned new technologies and programming languages while working on these types of bugs. If I could go back in time and fix these same bugs now with the knowledge I now posses, I would try to make my code changes to a minimal level thus easier to read for code reviewers.

Last updated 4/25/2020