Laravel Package Development Part 1: projecting a user session recording package

8 min readFeb 18, 2021

How many times did it happen for you to think about a feature you want to use in your project but none of the present packages implementing it satisfies you?

For me, fortunately, not a lot, but there were some fancy cases where I wanted to implement a very specific feature but I had no time so I made compromises and pick something similar. I had the idea for this package while writing the article about Data-Driven Strategies. It was simple to set up the whole project, record user sessions and query data, but what about making it available for everyone and add cool features like heat maps? So this time I decided to start building from scratch my Laravel package to record user sessions.

Laravel has a good guide to build packages, we can refer to that!

The starting point: package boilerplate

The core of Laravel is to help you focus on your project logic instead of losing time on legacy stuff. For package development, you can use the Laravel Package Boilerplate Generator that will automatically generate the package structure including a basic structure, composer.json with illuminate dependency, some tools like Travis, StyleCI, Scrutinizer built-in support and basic markdown files.

Package Boilerplate Generator Welcome Page

After you get in the boilerplate generator website it will be prompted if you want to build a Laravel or a PHP package.

I obviously picked Laravel, now you have to fill in your package details:

Vendor name: in most cases, this is your nickname, if you’re building it for an organization, this will be your organization nickname. If you want to build an open-source package for Github or some other public version control systems, this will be your nickname.
Package name: the name you want to give to your package.
Author name: who are you?
Author email: the email people can eventually use to contact you.
Package Description: what does your package do? Be short here, you will have the entire README.MD to explain it!
License: the choice is yours, guiding you through all license type it’s kinda hard, but they help you through the “Choose an open-source license” tool. For this, I picked the MIT License: simple and permissive!

And that’s all for the boilerplate, you just click next and download your ZIP file.

Now you will have your package structure you can extract to any folder and edit using your favourite editor. My package name, in particular, is Laravel Spyhole!

Your package structure will be something like this:

$ tree
.
├── .editorconfig
├── .gitattributes
├── .gitignore
├── .scrutinizer.yml
├── .styleci.yml
├── .travis.yml
├── CHANGELOG.md
├── CONTRIBUTING.md
├── LICENSE.md
├── README.md
├── composer.json
├── config
│   └── config.php
├── phpunit.xml.dist
├── src
│   ├── LaravelSpyhole.php
│   ├── LaravelSpyholeFacade.php
│   └── LaravelSpyholeServiceProvider.php
└── tests
    └── ExampleTest.php3 directories, 17 files

What’s inside?

Markdown files: CHANGELOG.MD to keep your package version updates (to stay coherent I suggest using Semantic Versioning); CONTRIBUTING.MD to explicit how you want other people to contribute to your package; LICENSE.MD where the license code you picked will be put, if you didn’t pick the license you wanted to, you can edit this; README.MD what people will see from entering your repository.
composer.json: basic composer specification file with the data you wrote through the tutorial and the dependencies.
phpunit.xml.dist: the PHPUnit Configuration to run tests.
.travis.yml: configuration file if you want to use the Travis CI tool (note that Travis CI is free for open-source projects).
.scrutinizer.yml: configuration file if you want to use the Scrutinizer CI tool (even Scrutinizer CI is free for open-source projects).
.styleci.yml: configuration file if you want to use the StyleCI tool (StyleCI is free for open-source projects).
Basic Git configuration files (.gitattributes and .gitignore).
Config folder containing a file for laravel package configuration files.
The source folder will contain the package code.
The test folder containing the package tests.

The first things I did was to tweak composer.json file:

adding PHP 8.0 usage: "php": "^7.3|^8.0"
adding support for multiple support package version: "illuminate/support": "^7.0|^8.0"
adding the Orchestral Testbench package in the dev dependencies to test the package as in a laravel application (as suggested in the laravel guide) composer require --dev orchestra/testbench.

Now just install everything

$ composer install

And we’re ready to code!

The idea

Well, before starting coding, it’s better to have a clear idea of what you want to build.

The Laravel Spyhole purpose is to have a nice and simple way to record and rewatch user sessions.

That’s the easy way to tell it, now explode it.

Laravel Spyhole purpose is to provide a way to embed the recorder to a view, the recorder will post data to a Spyhole route where data will be stored. Spyhole will record user events and mouse movements. During each recording session, the DOM may change: these changes must be tracked to make recordings coherent.

Recorded sessions must be available to be watched. I used many laravel packages with some kind of dashboard integration, each one was different and integrating their behaviour inside a custom system was kinda painful due to the various interfaces, configurations and permissions to tweak. Taking inspiration from my previous experiences, I want to give user full control over their integration providing:

an embeddable view to embed just the player disabling every other interface. For everyone who already has a dashboard and just wants to embed the player.
a minimal dashboard to check all recordings, rewatch and delete them (basically a CRUD), with a Gate for permissions checking. For everyone who wants to use the package out-of-the-box.

Architecting a system is complex, a package that you want to be used by other people is more complex, so I’ll embrace the KISS Philosophy (Keep It Simple, Stupid).

The database

The structure to be implemented is really simple, it practically consists of a single table: session_recordings.

The table should store:

ID: an auto-increment integer.
path: this will help to filter and rewatch how a page UX works over time and to calculate further statistics that can be helpful (like checking how long does a user pass on that page for example).
session ID: this can be the real Laravel session ID or a UUID (if you don’t want to track the real ID), this will help to track the full user navigation matching session ID, paths and timestamps.
user ID: the user identifier for user tracking. This is not an integer but a string to stick with the getAuthIdentifier of Authenticatable interface. In many cases, the identifier is an ID but some developer may want to use a UUID or some different kind of identifier.
recordings: the recording payload as a JSON coming from the frontend.
Created and updated at timestamps: for time tracking.

Once the database study is completed, implementing the migration (in the folder database/migrations as in a Laravel application) it’s fairly easy.

To let Spyhole load the migration, the Service Provider boot method should be uncommented the migration line:

$this->loadMigrationsFrom(__DIR__.'/../database/migrations');

The assets

While playing around with cimice (that I covered in the other article), I discovered a more maintained library: RRWeb (Record and Replay the Web) available here.

rrweb is an open source web session replay library, which provides easy-to-use APIs to record user’s interactions and replay it remotely.

While the last cimice commit on Github was 5 years ago, the last RRWeb commit was less than one month ago (while I’m writing this 📓). Another good stuff about RRWeb is that it records DOM changes automatically while recording the session!

RRWeb is made of different files for recording and replaying, so the first thing to do is to download these assets and to make them available in the package. Laravel way to do so is to let you publish your assets to the final application public folder.

I decided to use the resources/assets folder to store the assets to be published. So I downloaded the rrweb.min.js and the rrweb-replay.min.js from the JSDeliver page, this way you the library is always available bound with the package.

Now just advertise them in the service provider. For publishing, you also must specify the public folder where these assets will be copied, in general, packages follow the convention to put assets into a folder named as the package in the vendor public folder. The boilerplate code embeds the publishes function code into the check for the application running environment, in particular, this kind of publishing must just be executed when running in console (during the artisan vendor:publish invocation).

$this->publishes([
    __DIR__.'/../resources/assets' => public_path('vendor/laravel-spyhole'),
], 'assets');

The config

The configuration of the package is ready to go. The config folder already contains a file named config.php where config must be written. Laravel publishing will push this file in the config/laravel-spyhole.php so I promptly renamed it to make autocomplete work. The config file contains just an array with config key and their associated values so for not it can be omitted, it will be populated while in development.

If for your package you don’t need a configuration file, just delete the config folder and remove from the Service provider the publishes call to the config publishing.

$this->publishes([
    __DIR__ . '/../config/laravel-spyhole.php' => config_path('laravel-spyhole.php'),
], 'config');

Setup completed

“Change begins at the end of your comfort zone” ~ Roy T. Bennett

I never built a package before, that’s my “going outta my comfort zone”.

Config is built-in, assets registered, migration placed, now the game begins. I decided to split the package development into multiple articles due to the development and testing time. This article covered the setup and the study around package development.

In the next part, I’m gonna build the recorder controller, add validation and test it!

Stay tuned for the complete series and if you want, take a moment to leave a comment about how you would do the projecting or if you would have changed something in this step! ☕️