Data Exports
Data Exports
The exports page provides a flexible interface for exporting data from any database table with optional PDF attachments.
Overview
Located at /exports, this page allows users to:
- Select any table from the database
- Apply custom filter conditions
- Choose specific columns to export
- Include PDFs for supported resources (invoices, quotes, etc.)
- Export as JSON or CSV with optional ZIP packaging
UI Components
Header Section
- Title: "Exports"
- Settings Button (⚙️): Toggles custom conditions panel
- Fetch Data Button: Loads data from selected table
Export Options
The main interface is split into two columns:
Left Column
- Table Selector: Dropdown to choose the database table
- Limit: Number of rows to fetch (default: 1000)
- Export Buttons: JSON and CSV export options
- Stats: Row count and selected column count
Right Column (PDF Resources)
Only shown for tables with PDF support (e.g., invoices):
- Include PDFs: Toggle to add PDFs to ZIP export
- Generate Missing: Auto-generate PDFs before download
- Concurrency: Parallel processing limit (1-100)
- PDF Status: Shows ready/missing counts
- Action Buttons: Generate missing or regenerate all PDFs
- Progress Bar: Real-time generation progress with ETA
Custom Conditions Panel
Toggled by the settings button (⚙️):
┌─────────────────────────────────────────────────────┐
│ Custom conditions (+ Add) │
├─────────────────────────────────────────────────────┤
│ Column: [invoice_id ▼] Value: [________] [×] │
│ Column: [status ▼] Value: [paid_____] [×] │
└─────────────────────────────────────────────────────┘
```typescript
Each condition creates an equality filter (`column = value`).
### Column Settings Panel
Shows all available columns in a grid layout with checkboxes:
- **Select All**: Toggle all columns
- **Grid Layout**: 2-4 columns depending on screen size
- **Scrollable**: Max height with overflow for many columns
## Usage
### Basic Export Flow
1. Select a table from the dropdown
2. Set the row limit
3. Click "Fetch data"
4. (Optional) Add custom conditions via ⚙️ button
5. (Optional) Toggle columns in column settings
6. Click "Export JSON" or "Export CSV"
### Export with PDFs
For invoice exports:
1. Select "invoices" table
2. Fetch data
3. Enable "Include PDFs in ZIP"
4. (Optional) Enable "Generate missing PDFs"
5. Set concurrency (higher = faster, but more resource intensive)
6. Click export button
7. Server-side ZIP is created with data file + PDFs
## API Integration
### Fetch Data
```typescript
// POST /fetch/data
{
table_name: "invoices",
conditions: [
{ eq_column: "status", eq_value: "paid" }
],
limit: 1000,
offset: 0
}
// Headers
{
"X-Company-Id": "uuid",
"X-User-Id": "uuid",
"Content-Type": "application/json",
"Cache-Control": "no-cache" // Optional
}
```typescript
### Bulk PDF Generation
```typescript
// POST /api/invoice-gen/bulk
{
invoice_ids: ["uuid-1", "uuid-2"],
number_format: "nl-NL",
locale: "en",
object_type: "invoice",
concurrency: 15,
force: false
}
```typescript
### Server-Side Export with PDFs
```typescript
// POST /api/invoice-gen/bulk-export
{
invoice_ids: ["uuid-1", "uuid-2"],
number_format: "nl-NL",
locale: "en",
object_type: "invoice",
concurrency: 10,
format: "csv", // or "json"
include_data: true
}
```typescript
Response includes download URLs for ZIP file(s).
## PDF Resource Configuration
Supported tables and their PDF configurations:
```typescript
const PDF_RESOURCE_CONFIG = {
invoices: {
idField: "invoice_id",
objectType: "invoice",
pdfField: "pdf_url",
bulkEndpoint: "/api/invoice-gen/bulk",
},
// Future support for quotes, receipts, credit_notes
};
```typescript
## Progress Tracking
During PDF generation, the UI shows:
- **Percentage**: Animated counter with decimals
- **Progress Bar**: Visual fill indicator
- **Counts**: Processed / Total
- **Timing**: Elapsed time and ETA
- **Status Breakdown**:
- ✓ Done (green)
- ✗ Failed (red)
- ○ Pending (gray)
- **Errors**: Up to 3 unique error messages displayed
## State Management
Key state variables:
```typescript
// Data
const [rows, setRows] = useState<Row[]>([]);
const [selectedColumns, setSelectedColumns] = useState<Record<string, boolean>>({});
// PDF Processing
const [pdfProgress, setPdfProgress] = useState<number | null>(null);
const [pdfSuccessCount, setPdfSuccessCount] = useState<number>(0);
const [pdfFailedCount, setPdfFailedCount] = useState<number>(0);
const [pdfErrors, setPdfErrors] = useState<string[]>([]);
// UI
const [showCustomConditions, setShowCustomConditions] = useState<boolean>(false);
const [showColumnSettings, setShowColumnSettings] = useState<boolean>(false);
```typescript
## Error Handling
### Fetch Errors
- Automatic retry with reduced chunk size
- Minimum chunk size: 10 rows
- User notification on failure
### PDF Generation Errors
- Individual failures logged with invoice ID
- Failed PDFs have `pdf_url` set to null
- Error messages collected and displayed (max 3 shown)
- Auto-abort after 3 consecutive batch failures
## Performance Tips
1. **Start with smaller limits** for initial testing
2. **Use custom conditions** to filter data before fetch
3. **Adjust concurrency** based on server load (10-25 recommended)
4. **Enable caching** for repeated fetches of same data
5. **Use server-side export** for large PDF exports (handles chunking automatically)
## File Structure
```files
apps/
└── dashboard/
├── app/
│ ├── (dashboard)/
│ │ └── exports/
│ │ └── page.tsx
│ └── api/
│ └── invoice-gen/
│ ├── bulk/
│ │ └── route.ts
│ └── bulk-export/
│ └── route.ts
Security
- Company ID filtering applied automatically
- User permissions checked via headers
- Only allowed tables exposed in dropdown
- No direct SQL injection possible (parameterized queries)