XYLEX Group

Data Exports

Data Exports

The exports page provides a flexible interface for exporting data from any database table with optional PDF attachments.

Overview

Located at /exports, this page allows users to:

  • Select any table from the database
  • Apply custom filter conditions
  • Choose specific columns to export
  • Include PDFs for supported resources (invoices, quotes, etc.)
  • Export as JSON or CSV with optional ZIP packaging

UI Components

Header Section

  • Title: "Exports"
  • Settings Button (⚙️): Toggles custom conditions panel
  • Fetch Data Button: Loads data from selected table

Export Options

The main interface is split into two columns:

Left Column

  • Table Selector: Dropdown to choose the database table
  • Limit: Number of rows to fetch (default: 1000)
  • Export Buttons: JSON and CSV export options
  • Stats: Row count and selected column count

Right Column (PDF Resources)

Only shown for tables with PDF support (e.g., invoices):

  • Include PDFs: Toggle to add PDFs to ZIP export
  • Generate Missing: Auto-generate PDFs before download
  • Concurrency: Parallel processing limit (1-100)
  • PDF Status: Shows ready/missing counts
  • Action Buttons: Generate missing or regenerate all PDFs
  • Progress Bar: Real-time generation progress with ETA

Custom Conditions Panel

Toggled by the settings button (⚙️):

┌─────────────────────────────────────────────────────┐
│ Custom conditions                        (+ Add)   │
├─────────────────────────────────────────────────────┤
│ Column: [invoice_id ▼]  Value: [________]   [×]    │
│ Column: [status ▼]      Value: [paid_____]  [×]    │
└─────────────────────────────────────────────────────┘
```typescript

Each condition creates an equality filter (`column = value`).

### Column Settings Panel

Shows all available columns in a grid layout with checkboxes:

- **Select All**: Toggle all columns
- **Grid Layout**: 2-4 columns depending on screen size
- **Scrollable**: Max height with overflow for many columns

## Usage

### Basic Export Flow

1. Select a table from the dropdown
2. Set the row limit
3. Click "Fetch data"
4. (Optional) Add custom conditions via ⚙️ button
5. (Optional) Toggle columns in column settings
6. Click "Export JSON" or "Export CSV"

### Export with PDFs

For invoice exports:

1. Select "invoices" table
2. Fetch data
3. Enable "Include PDFs in ZIP"
4. (Optional) Enable "Generate missing PDFs"
5. Set concurrency (higher = faster, but more resource intensive)
6. Click export button
7. Server-side ZIP is created with data file + PDFs

## API Integration

### Fetch Data

```typescript
// POST /fetch/data
{
  table_name: "invoices",
  conditions: [
    { eq_column: "status", eq_value: "paid" }
  ],
  limit: 1000,
  offset: 0
}

// Headers
{
  "X-Company-Id": "uuid",
  "X-User-Id": "uuid",
  "Content-Type": "application/json",
  "Cache-Control": "no-cache"  // Optional
}
```typescript

### Bulk PDF Generation

```typescript
// POST /api/invoice-gen/bulk
{
  invoice_ids: ["uuid-1", "uuid-2"],
  number_format: "nl-NL",
  locale: "en",
  object_type: "invoice",
  concurrency: 15,
  force: false
}
```typescript

### Server-Side Export with PDFs

```typescript
// POST /api/invoice-gen/bulk-export
{
  invoice_ids: ["uuid-1", "uuid-2"],
  number_format: "nl-NL",
  locale: "en",
  object_type: "invoice",
  concurrency: 10,
  format: "csv",        // or "json"
  include_data: true
}
```typescript

Response includes download URLs for ZIP file(s).

## PDF Resource Configuration

Supported tables and their PDF configurations:

```typescript
const PDF_RESOURCE_CONFIG = {
  invoices: {
    idField: "invoice_id",
    objectType: "invoice",
    pdfField: "pdf_url",
    bulkEndpoint: "/api/invoice-gen/bulk",
  },
  // Future support for quotes, receipts, credit_notes
};
```typescript

## Progress Tracking

During PDF generation, the UI shows:

- **Percentage**: Animated counter with decimals
- **Progress Bar**: Visual fill indicator
- **Counts**: Processed / Total
- **Timing**: Elapsed time and ETA
- **Status Breakdown**:
  - ✓ Done (green)
  - ✗ Failed (red)  
  - ○ Pending (gray)
- **Errors**: Up to 3 unique error messages displayed

## State Management

Key state variables:

```typescript
// Data
const [rows, setRows] = useState<Row[]>([]);
const [selectedColumns, setSelectedColumns] = useState<Record<string, boolean>>({});

// PDF Processing
const [pdfProgress, setPdfProgress] = useState<number | null>(null);
const [pdfSuccessCount, setPdfSuccessCount] = useState<number>(0);
const [pdfFailedCount, setPdfFailedCount] = useState<number>(0);
const [pdfErrors, setPdfErrors] = useState<string[]>([]);

// UI
const [showCustomConditions, setShowCustomConditions] = useState<boolean>(false);
const [showColumnSettings, setShowColumnSettings] = useState<boolean>(false);
```typescript

## Error Handling

### Fetch Errors
- Automatic retry with reduced chunk size
- Minimum chunk size: 10 rows
- User notification on failure

### PDF Generation Errors
- Individual failures logged with invoice ID
- Failed PDFs have `pdf_url` set to null
- Error messages collected and displayed (max 3 shown)
- Auto-abort after 3 consecutive batch failures

## Performance Tips

1. **Start with smaller limits** for initial testing
2. **Use custom conditions** to filter data before fetch
3. **Adjust concurrency** based on server load (10-25 recommended)
4. **Enable caching** for repeated fetches of same data
5. **Use server-side export** for large PDF exports (handles chunking automatically)

## File Structure

```files
apps/
└── dashboard/
    ├── app/
    │   ├── (dashboard)/
    │   │   └── exports/
    │   │       └── page.tsx
    │   └── api/
    │       └── invoice-gen/
    │           ├── bulk/
    │           │   └── route.ts
    │           └── bulk-export/
    │               └── route.ts

Security

  • Company ID filtering applied automatically
  • User permissions checked via headers
  • Only allowed tables exposed in dropdown
  • No direct SQL injection possible (parameterized queries)