How to Digitize Paper Personnel Records with Laravel and Move a Team from FTP to Git

A provincial government office kept its personnel records in paper files and Excel sheets, built its oversight reports by hand every cycle, and deployed code by uploading files over FTP. I was on the in-house IT team that replaced all three.

Paper records are hard to audit, spreadsheets drift, and FTP deploys let one bug fix overwrite a colleague's change. This post shows how to turn a long paper form into a clean data model, keep every change as history instead of overwriting it, fix the database problems that make reporting slow, and move a team onto Git and code review while the work keeps going.

Background

Where the office started

HR managed civil servants the way many local governments do, paper personnel files in cabinets, plus Excel sheets tracking deductions, leaves, and assignments. Reports for oversight agencies were handcrafted every cycle. Visitor-arrival data came in as forms typed up by field officers and was summarized by counting rows.

The brief: get the records and the reports off paper, and make visitor numbers something planners could look at without a meeting.

Constraints

• On-premise hosting. Government data, no cloud. Everything ran on the organization's own servers.
• Mixed database reality. Existing legacy applications used SQL Server. New work used MySQL. Some queries had to span both worlds.
• Mixed audience. HR staff who had never used a CRUD app. Field officers with patchy connectivity. Planners who wanted clean charts.
• A team learning as we built. Most of my colleagues had shipped PHP, but not with Laravel, Git, or a code review workflow. Production deployments were FTP uploads.

Three ideas this post relies on

• Append-only history. Instead of overwriting a value, each change adds a new entry and the previous one stays readable. That is what turns a digital form into an auditable record.
• N+1 queries. Code that loads a list with one query, then runs one more query per row to fetch something related. Invisible with ten rows, painful with the whole office hitting it.
• Indexes. Without an index on the columns a query filters or joins on, the database reads the whole table every time.

Building the records platform

Digitizing the personnel file

The first deliverable was a digital personal data sheet. The paper version is a long, multi-page form with a couple of hundred fields, many of them repeating sections like family, work history and training. I modeled it carefully, not as one giant table, but as a core personal record with separate related records for each repeating section of the form.

Critically, every change to a record was version-tracked. A civil servant who updated their employment history did not erase the previous entry, they appended. The audit trail is what made the data trustworthy enough for HR to retire the cabinet.

Visitor analytics dashboard

Field officers collected visitor arrivals at entry points. The old workflow ended in a spreadsheet attached to an email. The new workflow ended in a Vue.js dashboard pulling aggregates straight from the database, sliced by area, month, origin, and purpose of visit.

The dashboard was deliberately small. Three views, no filters beyond date range and area, and a CSV export. Planners wanted a thing they could glance at, not a BI tool to learn.

Internal tooling suite

The records platform was the headline deliverable. The unglamorous deliverable, which moved the office more, was the smaller tools.

• Insurance deduction file generator, produced the monthly deduction file a government insurance agency consumes. The old workflow took half a day. The new one is a button.
• Tax reporting export, produced the report files the national tax authority requires, in their exact expected format. The fragile part was always the format, not the math.
• SMS notification service, small Laravel service that sent payroll, leave, and announcement messages. SMS was the channel field officers actually read.
• Android employee app for personnel files, Java app letting employees view their own record, request edits, and see leave balances. Built specifically so employees did not have to walk to the HR office for everything.

Fixing slow reporting endpoints

The reporting views on the legacy applications had a familiar smell: response times measured in tens of seconds for endpoints hit by the whole office during cutoff weeks. I profiled, fixed the obvious N+1s, added the missing indexes on the high-traffic queries, and consolidated a few report-generation loops into batched queries.

High-traffic reporting endpoints came down by around 60% in response time. The wins were not exotic. They were the standard set of database hygiene tasks that legacy applications never get because nobody is paid to revisit them.

Moving the team from FTP to Git

The most durable contribution from this stretch was not a system. It was the change to how the team shipped software.

When I joined, the production deployment story was: copy files over FTP, hope nothing on the server changed. There was no version control of substance. A bug fix could overwrite a colleague's in-flight change.

I paired with colleagues through real changes, not tutorial code, until they were comfortable opening pull requests and rolling back failed deployments. By the time I left for full-time freelance work, the team was operating their own workflow without me in the loop.

Design decisions

Three audiences, three different needs: HR staff new to web apps, field officers on patchy connections, and planners who wanted clean charts.

• The form became a record, not a table

  The long paper form was split into a core personal record with a related record for each repeating section, instead of one giant table.

• Changes append, never overwrite

  Every edit is version-tracked. That visible history is what made HR trust the system enough to retire the cabinet.

• A dashboard to glance at

  Three views, filters only for date range and area, and a CSV export. Planners wanted a glance, not a BI tool to learn.

• Meet people on their channel

  SMS for payroll and leave notices because field officers read SMS, and an Android app so employees could check their own personnel file without walking to HR.

Practical examples

Simplified example. The snippets below are generic Laravel and shell code showing the techniques from this post, with neutral names. They are not the production code or schema.

1. Split a long form into a core record and related sections

use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\Relations\HasMany;

class Employee extends Model
{
    // One core record, one related table per repeating section of the form.
    public function workHistory(): HasMany
    {
        return $this->hasMany(WorkHistoryEntry::class);
    }

    public function trainings(): HasMany
    {
        return $this->hasMany(Training::class);
    }

    public function versions(): HasMany
    {
        return $this->hasMany(EmployeeVersion::class);
    }
}

2. Keep every change as history

class Employee extends Model
{
    protected static function booted(): void
    {
        // Before any update, store a snapshot of the values being replaced.
        static::updating(function (Employee $employee) {
            $employee->versions()->create([
                'data' => $employee->getRawOriginal(),
                'changed_by' => auth()->id(),
            ]);
        });
    }
}

class EmployeeVersion extends Model
{
    protected $fillable = ['data', 'changed_by'];

    protected $casts = ['data' => 'array'];
}

// Usage
$employee->update(['last_name' => 'Smith']);

$employee->versions()->latest()->first()->data['last_name'];

Expected output: "Doe", the value before the edit. The current record says Smith, and the history still says Doe.

3. Find and fix N+1 queries

// N+1: one query for the employees, then one more per employee.
foreach (Employee::all() as $employee) {
    echo $employee->department->name;
}

// Eager loading: two queries, however many employees there are.
foreach (Employee::with('department')->get() as $employee) {
    echo $employee->department->name;
}

// AppServiceProvider::boot(): throw on lazy loading outside production.
Model::preventLazyLoading(! app()->isProduction());

// Migration: index the columns a hot report filters on.
Schema::table('time_entries', function (Blueprint $table) {
    $table->index(['employee_id', 'recorded_on']);
});

Expected result: with 200 employees, the first loop runs 201 queries and the second runs 2. (Illustrative numbers, not a measurement from this project.)

4. A minimal Git-based deploy script

#!/usr/bin/env bash
set -euo pipefail   # stop at the first failing command

cd /var/www/app
git pull --ff-only origin main
composer install --no-dev --optimize-autoloader
php artisan migrate --force
php artisan config:cache

A rollback is the same steps against the previous commit or tag, plus a migration rollback if the failed release changed the schema.

Common mistakes

• Modeling a long form as one giant table, with repeating sections stored as numbered columns.
• Updating records in place, so the previous value is gone for good.
• Mass updates through the query builder, such as Employee::where(...)->update(...). They skip model events, so they also skip the history snapshot.
• Lazy loading relations inside report loops.
• Adding indexes by guesswork instead of from the slow queries you actually profiled.
• Editing files directly on the production server, or deploying over FTP where one upload can overwrite another.

Best practices

• Profile first, then fix N+1s, add missing indexes on high-traffic queries, and batch report loops.
• Keep dashboards small: a few views, a few filters, a CSV export.
• Send every change through a pull request with review, and deploy only from main.
• Teach the workflow by pairing on real changes, not tutorial code.
• Keep seed data and a few HTTP integration tests on the core CRUD paths.
• Plan to land on one database engine instead of querying across two.

Conclusion

Digitizing paper records is mostly a modeling and trust problem. Split the form into a core record plus its repeating sections, and keep every change as history, so people trust the system enough to retire the cabinet. Slow legacy reporting is usually fixed with ordinary database hygiene, not exotic tuning.

The takeaway: the workflow change can outlast the software. If your team still deploys by copying files, start with Git, pull requests and a one-command deploy script before anything else.

What it does today

The records platform, the dashboard, and the internal tools are still in production use across the organization's offices. The workflow changes, Git, pull requests, code review, outlived my time on the team. That second outcome matters more in the long run than any one feature I shipped.

What I would do differently

• Standardize on one database earlier. The cross-engine queries between MySQL and SQL Server were the worst-feeling code in the project. A migration plan to land on one engine should have been the first roadmap item, not the last.
• Invest in seed data and integration tests. A government HR system is a forms project. Forms projects rot when nobody owns them. A seed dataset plus a handful of HTTP integration tests on the boring CRUD paths would have made the platform survivable past my tenure with less ceremony.
• Build the field officer Android app as a PWA. Native Java was overkill for what was, in the end, a thin client over the same API. A PWA would have been faster to ship and faster to update.

References and further reading

• Laravel documentation: Eloquent relationships and eager loading
• Laravel documentation: Eloquent model events
• Laravel documentation: Migrations and indexes
• Laravel documentation: Deployment
• Vue.js guide
• MySQL documentation
• SQL Server documentation
• nginx documentation
• Git documentation
• GitHub Docs: Pull requests
• Related on this site: How to enforce attorney-client privilege in a Laravel case management app
• Related on this site: How to transcribe confidential audio offline with Whisper and CTranslate2