Lessons Learned Building a Production System - Part 3

17 Dec 2025

Lessons Learned Building a Production System - Part 3

"Real challenges, debugging stories, and interview presentation strategies from building a production document management system"

In Part 1 and Part 2, I built and deployed a document management system. This final part covers the real challenges, debugging war stories, and how to present this in interviews.

Spoiler: I learned more from things breaking than from things working.


The Bugs That Taught Me the Most

Bug #1: The Mysterious HTTP 400

The Symptom:

# From host machine - works perfectly
curl -X PUT http://localhost:8080/exist/rest/db/docs/test.xml \
  -u admin: -H "Content-Type: application/xml" \
  -d '<test>hello</test>'
# HTTP/1.1 201 Created ✅

# From PHP container - fails
curl -X PUT http://exist:8080/exist/rest/db/docs/test.xml \
  -u admin: -H "Content-Type: application/xml" \
  -d '<test>hello</test>'
# HTTP/1.1 400 Bad Request ❌

Same request, different results. Why?

The Investigation:

I spent 4 hours debugging this. Here's my process:

  1. Checked container networking

    docker exec php-fpm ping exist  # ✅ Works
    docker exec php-fpm curl http://exist:8080  # ✅ Works
    
  2. Compared curl versions

    # Host (Ubuntu)
    curl --version  # curl 7.81.0, OpenSSL 3.0.2
    
    # Container (Alpine)
    docker exec php-fpm curl --version  # curl 8.5.0, LibreSSL 3.8.2
    
  3. Checked HTTP headers with verbose mode

    docker exec php-fpm curl -v -X PUT ...
    

    Found it! Alpine's curl sends minimal headers:

    PUT /exist/rest/db/docs/test.xml HTTP/1.1
    Host: exist:8080
    User-Agent: curl/8.5.0
    Content-Type: application/xml
    Content-Length: 18
    

    Host curl sends more headers:

    PUT /exist/rest/db/docs/test.xml HTTP/1.1
    Host: localhost:8080
    User-Agent: curl/7.81.0
    Accept: */*
    Content-Type: application/xml
    Content-Length: 18
    
  4. Checked eXist-db version

    Discovered: eXist-db 6.x has stricter HTTP parsing via Jetty. Small requests from Alpine curl trigger a validation bug.

The Solution:

Downgrade to eXist-db 5.3.0:

exist:
  image: existdb/existdb:5.3.0  # ✅ Works with Alpine curl

Lessons Learned:


Bug #2: Permission Denied (The Volume Mount Trap)

The Symptom:

Warning: move_uploaded_file(): Unable to move file
Warning: mkdir(): Permission denied

First Attempt (Wrong):

# Inside container
chmod 777 /var/www/html/uploads
# Seems to work... until container restarts ❌

The Real Issue:

I had mounted a host directory that didn't exist:

volumes:
  - ./php-app:/var/www/html

The directory "./php-app/uploads" didn't exist on the host, so Docker created it with root:root ownership. PHP-FPM runs as "www-data", which can't write to root-owned directories.

The Correct Solution:

# On HOST machine (before docker compose up)
mkdir -p php-app/uploads
chmod 755 php-app/uploads
sudo chown 82:82 php-app/uploads  # 82 = www-data UID in Alpine

Better Solution (for production):

Don't mount "uploads/" from host. Use a named volume:

volumes:
  - uploads-data:/var/www/html/uploads

Lessons Learned:


Bug #3: The eXist-db Dashboard Disaster

The Symptom:

After updating "docker-compose.yml" to add persistence:

volumes:
  - ./exist-data:/exist/data
  - ./exist-config:/exist/config

eXist-db dashboard showed:

HTTP 404: Document /db/apps/dashboard/index.html not found

What Happened:

When you mount empty host directories over container directories, you overwrite the container's files. I accidentally deleted eXist-db's built-in applications!

The Fix:

Use named volumes instead of bind mounts:

volumes:
  - exist-data:/exist-data  # ✅ Docker manages this

If you need to backup:

docker run --rm -v exist-data:/data -v $(pwd):/backup \
  alpine tar czf /backup/exist-backup.tar.gz /data

Lessons Learned:


Bug #4: Nginx Rejecting Everything (The Cloudflare Mystery)

The Symptom:

After setting up Cloudflare Tunnel:

curl https://existdbapp.lioprojects.xyz
# 404 Not Found

But locally:

curl http://localhost:8081
# 200 OK ✅

The Investigation:

# Check nginx logs
docker logs nginx -f
# Shows: "host not found in upstream"

The Problem:

My nginx config:

server_name existdbapp.lioprojects.xyz;

But Cloudflare Tunnel sends requests with Host: localhost:8081 or other internal headers.

The Fix:

server_name _;  # Accepts ANY hostname

Lessons Learned:


What I'd Do Differently

1. Start with Logging Earlier

I added proper logging after debugging for hours. Should have started with:

nginx:
  volumes:
    - ./logs/nginx:/var/log/nginx
// In PHP
error_reporting(E_ALL);
ini_set('display_errors', 1);
file_put_contents('/tmp/debug.log', print_r($_FILES, true), FILE_APPEND);

2. Write Tests Before Deployment

I manually tested everything. Should have written:

#!/bin/bash
# test-deployment.sh

echo "Testing upload..."
curl -F "title=Test" -F "author=Test" -F "file=@test.pdf" \
  http://localhost:8081/upload.php | grep -q "success" && echo "✅" || echo "❌"

echo "Testing eXist-db..."
curl -u admin: http://localhost:8080/exist/rest/db/docs/ | grep -q "test.xml" && echo "✅" || echo "❌"

3. Use Environment Variables

Hardcoded credentials are bad:

curl_setopt($ch, CURLOPT_USERPWD, "admin:");  // ❌

Should be:

$user = getenv('EXISTDB_USER') ?: 'admin';
$pass = getenv('EXISTDB_PASS') ?: '';
curl_setopt($ch, CURLOPT_USERPWD, "$user:$pass");
# docker-compose.yml
php:
  environment:
    - EXISTDB_USER=admin
    - EXISTDB_PASS=${EXISTDB_PASSWORD}

4. Implement Health Checks from Day 1

services:
  nginx:
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost"]
      interval: 30s
      retries: 3
      start_period: 10s

This would have caught the nginx config issue immediately.


Resources That Helped Me

Documentation:

Communities:

Tools:


Next Steps for This Project

If I were to continue developing:

  1. Add search functionality - Full-text search with eXist-db FT extensions
  2. User authentication - JWT-based auth with role management
  3. Document versioning - Track changes over time
  4. Monitoring stack - Prometheus + Grafana
  5. CI/CD pipeline - GitHub Actions for automated deployment
  6. API documentation - OpenAPI spec for REST endpoints
  7. Performance testing - Load testing with k6 or Locust

Conclusion

This project took me from "I know Docker basics" to "I can deploy production systems." The bugs taught me more than tutorials ever could.

Key takeaway: Build things that break. Debug them. Document everything. That's how you grow.


Questions? Drop a comment or reach out on LinkedIn.

Want to see the code? GitHub

Thanks for following this series. Now go build something that breaks!